[Zope3-dev] Re: zope.error is a 3.5 egg, but is needed by 3.4.x releases

[Martijn Faassen]

Hey Jim,

To start off, my original post was colored too strongly by a lot of 
frustration I was venting. It was in response to the following line:

"For many of us, we don't even have to think about a Zope 3 version any
more, and we like it that way."

Jim Fulton wrote:
> 
> On Oct 5, 2007, at 1:55 PM, Martijn Faassen wrote:
> ...
>> Grok will pick up the balls Zope 3 dropped here.
> 
> Hm. I didn't think Zope 3 was animate.  Who are you referring to?

That I was pretty annoyed by what I read to be a pretty cavalier 
attitude towards the pain people have been going through with eggs, 
after all the frustration and work that many people have gone through 
recently.

I think Zope 3.4 is currently not usable unless you already know exactly 
what you're doing with egg dependencies. What you're supposed to be 
doing isn't exactly documented anywhere.

Zope 3 the code isn't animate, but the open source community better be.

>> We actually care about a Grok version as it's the main way to get 
>> people to actually use Zope 3 stuff.
>>
>> We noticed this while we were going through egg dependencies by the 
>> way, not "using a Zope 3 version".
> 
> I don't know what you are trying to say.

Fred expressed he (and "many of us") are happy he doesn't have to think 
about a Zope 3 version anymore. The result is, as far as I can see, that 
*everybody* has to think about versions. If there'd been some release of 
3.4, we'd be able to use the versions that release is using, but 
instead, everybody is making their own collections of eggs and hoping 
for the best.

I understand that eggs can evolve at different rates, but having to 
determine the base from which to start ourselves wasn't exactly a 
pleasant exercise. Since everybody will have to come up with their own 
list I imagine others aren't exactly thrilled either.

>> Anyway, I personally don't care much that Zope 3.4 is unusuable 
>> without a massive investment in time sorting through eggs,
> 
> It's a shame that you are taking this view.

I and several others had Zope 3 based code become uninstallable 
repeatedly over the last months, because it's so unstable.  This is 
unworkable and pretty frustrating. I personally had two different 
customer installations break repeatedly because of this.

I realize that's because I didn't know what I was doing early on, and 
because the community was learning. I realize it's because I was on the 
forefront of development and that we didn't anticipate all the problems.

But if nobody in the Zope 3 community steps up and starts to clearly 
point out what people *should* be doing, the situation will continue to 
be unworkable. New users will not be able to adopt Zope 3 at all anymore.

I do care about getting new users to adopt Zope 3 technologies. I don't 
see the Zope 3 community think much about new user adoption.

The only way the eggified Zope can be used reliably currently is with a 
lot of upfront knowledge most people don't have, and a lot of work to 
sort through the eggs. There are scripts floating around to extract a 
list of versions from a buildout run, which helps some, but initially 
the answer I got on #zope3-dev when asking about such things was "write 
your own script, it's easy". I thought open source communities were 
about sharing solutions for problems, not just sharing the problems 
themselves.

I think in order to get new users to adopt the system, besides clearly 
documenting quite involved instructions on what they should be doing, 
there should be a way to use Zope 3 reliably *without* having to have 
all this knowledge and doing all this work.

Anyway, I know you personally are concerned with this issue, and I 
realize that buildout has been attempting to grow features that help 
here. I appreciate the help.

The Grok project is trying to solve these issues. We published a 
document that tells users what to do. We now publish lists of eggs on a 
HTTP server (one per grok release). We have adjusted grokproject so that 
a versions list will be used automatically when you create a new grok 
project. We attempt to reach a situation where users can install and use 
Grok and build their own applications without having to spend too much 
thought on dependencies. We aim for a situation where the community 
manages the list of dependencies, instead of everyone for themselves.

Why did this have to happen within the Grok project? I have the 
impression that too much, people in the Zope 3 community think that as 
soon as they themselves and the other expert developers know what to do 
(assimilated on the mailing list and online), this is the end of the 
problem. We don't have a single Zope 3 release anymore, but we do want 
people to use our code and report bugs on it. As far as I can see this 
*requires* a list of versions somewhere that is shared between people 
and that can be communicated about easily. It requires a *release 
procedure* around this list of versions. As I tried to point out before 
long ago, the Zope 3 project is not somehow so special it doesn't need 
releases.

>> as we have fixed the problem with Grok. If non-Grok users are 
>> interested in our fixes, please let us know though. We've just made 
>> this massive investment. I'd suggest people to switch to Grok anyway, 
>> as we actually think about this stuff and care about having our house 
>> in order.
> 
> Are you suggesting that other people aren't thinking about this and 
> don't care?

Fred doesn't appear to care to think about this much, certainly, and is 
liking it that way.

> I hope that the proposal for setting up a known-good-set index will be 
> helpful.  I'm sure Stephan would like to know the versions you chose. I 
> imagine he'll be able to get those by looking at Grok.

After 3 days of hard work by 2 people, we're still not done with the 
Grok eggs. We get weird effects like final releases of packages 
*suddenly* needing an entirely new package, between the beta and the 
final. I'm quite bothered by this. We still have quite a few dev-r515135 
style eggs in the mix as well.

A single known-good index, by the way, doesn't solve all problems. It 
will evolve forward as people release newer known-good versions of eggs. 
This means that nobody can really rely on such an index, as suddenly 
they might be starting to use 3.6 eggs where they were using 3.5 eggs. 
Even bugfix releases are currently hardly safe: using 3.4.1 for some 
Zope 3 package might mean that suddenly it requires an entirely new 
package altogether, introducing new deprecation warnings and so on (as 
in this thread).

With our Grok versions list, we publish a list *per* version of Grok. I 
hope this is the intent for the known-good index as well. If someone 
says they use Grok 0.11, we want to know *exactly* which versions they 
are using without requiring them to send us a long list too. One version 
number should be enough. Our installation tools support this, and our 
documentation documents this. I think this is the only maintainable way 
to actually have a community developing and using a common code base.

If we ignore the last monolithic release, Zope 3.3, we get the following 
irony: the easiest way to reliably use Zope 3 technology is to use Zope 
2.10. The next easiest way is Grok (should be almost as easy as Zope 2 
once we release our work, with a community managing dependencies for 
you). The hardest way is actually Zope 3 itself (you'll have to manage 
dependencies yourself, assuming you even managed to find out how to do 
this).

Regards,

Martijn