[Zope3-dev] Re: zope.error is a 3.5 egg, but is needed by 3.4.x releases

[Martijn Faassen]

Jim Fulton wrote:
> 
> On Oct 6, 2007, at 4:56 AM, Martijn Faassen wrote:
> 
>> I think Zope 3.4 is currently not usable unless you already know 
>> exactly what you're doing with egg dependencies. What you're supposed 
>> to be doing isn't exactly documented anywhere.
> 
> I think you are probably right.  I think part of the problem is that we 
> no longer have a clear vision for what Zope 3 is.  I don't think this is 
> entirely bad, as the vision has been evolving to match reality.  I think 
> most of us are settling on the idea of Zope 3 as "library", but we're 
> still figuring out how to articulate and implement this vision.

Yes, good point. I understand why we're going through this. We do have a 
vision for what Grok is: making Zope 3 technologies easier to use; of 
course it's debatable whether we're doing it right, but that's our goal. 
I think there is a need for projects that do this.

>> Zope 3 the code isn't animate, but the open source community better be.
> 
> Right. I consider you to be a part of this community. Several of the 
> major players in the community have been highly engaged in the 
> discussions over the past several days.  I think it's safe to say that 
> we're trying to make things better.

Yes, I apologize for the implication that nothing at all was happening. 
I've been highly engaged in this discussion as well. I think with Grok 
we hit smoe of the problems a bit earlier than most people, as it's a 
framework other people build on, and we went to eggs relatively early in 
the process. This might explain why our pain rose to intolerable 
magnitudes a bit more quickly than it did for other people.

[snip]
> Agreed.  Of course, before we can point out what people "should" be 
> doing, we have to figure out what that is.  To do that, people have to 
> remain engaged.
> 
>> I do care about getting new users to adopt Zope 3 technologies. I 
>> don't see the Zope 3 community think much about new user adoption.
> 
> Lots of people care.  Of course, most of us also have day jobs.  Many of 
> us are doing the best we can.

While having a day job is part of the story, I believe having a day job 
isn't the full story. I think it's also a cultural issue in Zope 3. Zope 
3 is a system that is very good for expert use, and not very good for 
beginners or more casual users. And while the Zope 3 community is in 
many ways quite healthy, I believe it doesn't do as good a job it could 
to grow the community of experts.

I believe there are technical reasons why Zope 3 isn't very good for 
casual reasons. That is only part of it though: there are also community 
cultural reasons. I don't actually particularly mind that this is so at 
this point, as we can evolve a related culture (Grok's) separately that 
has different emphasises and strengths (and weaknesses). Hopefully we 
can maintain a symbiotic relationship between the two, like we've had 
between Zope 2 and Zope 3 for a while now.

>> The only way the eggified Zope can be used reliably currently is with 
>> a lot of upfront knowledge most people don't have, and a lot of work 
>> to sort through the eggs. There are scripts floating around to extract 
>> a list of versions from a buildout run, which helps some, but 
>> initially the answer I got on #zope3-dev when asking about such things 
>> was "write your own script, it's easy". I thought open source 
>> communities were about sharing solutions for problems, not just 
>> sharing the problems themselves.
> 
> Again, we have to have the solutions before we can share them.  
> Sometimes, we arrive at these solutions through experimentation.

What bothered me wasn't that there was no solution at the time. What 
bothered me is that there was an attitude of "figure out for yourself 
what I already figured out" - we could've communicated better there. I 
got, perhaps entirely unwarranted, that the problem was considered 
solved, and that there was actually no further need to discuss solutions.

> As always, too, we have to understand the different problem sets we have 
> and the perspectives that gives us.  Many of us are building 
> applications with Zope 3.  Others, like you, are building platforms.  
> Solutions that work for single applications, like nailing versions in 
> meta egg or buildout configuration don't work so well for platforms, as 
> I think you've discovered.

Yes, I realize this. I've been communicating this for a while now and 
I'm glad you mentioned it just now. Thanks.

[snip]
>> As far as I can see this *requires* a list of versions somewhere that 
>> is shared between people and that can be communicated about easily. It 
>> requires a *release procedure* around this list of versions. As I 
>> tried to point out before long ago, the Zope 3 project is not somehow 
>> so special it doesn't need releases.
> 
> I think the Zope 3 project is somewhat special, which it may need some 
> kind of releases. :) After all, we have releases of individual eggs.  I 
> don't think it makes much sense to release the traditional Zope 3 
> application except perhaps as a testing platform.  I do think we need 
> something like what linux distros provide.  We need to establish a 
> better understanding of this.

All right. If this is the philosophy of Zope 3 I'm fine with it. I think 
there's a need for a platform to get people to adopt Zope 3 
technologies, but we can do this in the scope of other projects (such as 
Zope 2 and Grok). I wonder how we communicate this best. I'd like there 
to be a "mission statement" for the Zope 3 project that describes what 
we're trying to accomplish (it's integrated in some sense, but consists 
of separate releases).

[snip]
>>> Are you suggesting that other people aren't thinking about this and 
>>> don't care?
>>
>> Fred doesn't appear to care to think about this much, certainly, and 
>> is liking it that way.
> 
> So based on a snarky comment by one person, you tar the entire 
> community.

It wasn't the first time I thought to hear a message like this. As I 
said before though, my tone was too strong. I apologize for that. 
Instead in the future I will focus on codifying the focuses of the Zope 
3 project in a way we hopefully agree on. If Zope 3 is stepping back 
from being a platform, we should figure out ways for people to deal with 
the transition somehow, and try to deliniate the scope of Zope 3. I know 
you and others have been trying to do this.

> I'll note, BYW, that if you had provided more details, as 
> Philipp eventually did, I think Fred's response would have been more 
> sympathetic. (Just guessing)

I am sorry, I thought the situation was more obvious than it was, given 
the version numbers involved. I did bring more details in myself later.

[snip]
>> With our Grok versions list, we publish a list *per* version of Grok. 
>> I hope this is the intent for the known-good index as well. If someone 
>> says they use Grok 0.11,
> 
> Is Grok 0.11 a feature release?  Would you expect the version list for 
> Grok .11 to change as bug fixes are made?

Yes, 0.11 is a feature release (it hasn't been released yet, actually). 
Basically I'd like the list to be frozen. When a bugfix release is made, 
a new list is published for that bugfix release.

So, every release, feature or bugfix, would have a new list (or 
potentially even the same list except for Grok, if nothing else changes, 
but in this case it'd be a copy that we publish separately).

I think this is important for a platform with releases of its own. If 
someone reports a bug with Grok x.y.z, we want to know exactly which 
packages they were using, without having to ask them for the complete 
list of versions in buildout. Of course if an individual application 
developer overrode particular versions in their own buildout.cfg all 
bets are off, so this should be one of the first questions we ask them. :)

>> we want to know *exactly* which versions they are using without 
>> requiring them to send us a long list too. One version number should 
>> be enough. Our installation tools support this, and our documentation 
>> documents this. I think this is the only maintainable way to actually 
>> have a community developing and using a common code base.
> 
> My suggestion is that there should be stable indexes for each "feature 
> release".  Changes to these indexes should be made to fix bugs, not add 
> new features.  Of course, there could be other less stable indexes for 
> people who want them.  If you want to nail all of the versions for a 
> specific release of grok, then it seems to me that versions in a meta 
> egg or a buildout configuration should work well.

Yes, a meta egg could've worked.

A meta egg has the drawback of locking out individual application 
developers from varying versions at all in their own buildout.cfg 
(unless you add the feature we talked about where you can override egg 
selections in a meta egg). A mechanism for getting updated eggs in Grok 
itself would be application developers using newer underlying packages 
to get bugfixes or features, and then asking the Grok core developers to 
consider adopting this newer version in a future Grok release.

The nice thing about such a community mechanism is that hopefully 
there'll be an active motivation for people to communicate this 
information to core developers so this can be managed like the software 
itself. Could've worked with a meta egg as well, though.

We do use a buildout configuration, over a URL using the extension 
mechanism of buildout. A local buildout versions listing wouldn't work 
as we'd need to communicate this list directly to all application 
developers. With the current approach using the extends mechanism we 
have one URL (latest grok release) that grokproject uses to create a new 
buildout for Grok. To update an existing Grok buildout to a newer 
version of Grok requires the changing of just one URL.

I'm not convinced yet that even a well-managed evolving index with 
bugfix releases would be the right thing for a framework like Grok (or 
future Zope 2, for that matter, actually). But we'll keep an eye on it, 
and I'm happy this work is happening.

Again, I apologize for my outburst, and thanks for your kind and 
constructive response.

Regards,

Martijn