[Zope3-dev] Re: what dependency to use for "zope 3"

[Brian Sutherland]

On Fri, May 11, 2007 at 11:59:37PM +0100, Martin Aspeli wrote:
>  Brian Sutherland wrote:
> 
> > It's just going to get very difficult very quickly to manage such a meta
> > egg with over 100 or so dependencies. Though I guess there'll be
> > automated tools to manage this.
> 
>  Better we do the difficult part than each and every user does it.

I'm not saying "don't do it", but more "don't try do it by hand" or
you'll burn out your release manager.

A meta egg that only specifies a few dependencies is managable by hand,
but a meta egg that exactly specifies over 100 packages and exact
versions is not.

>  In the 
>  ideal world, there are no version conflicts. In the real world, there will 
>  be subtle version conflicts that mean that you can't just always get the 
>  latest release of each egg. 

Yep.

>  Anyone who's used something like Gentoo Linux, 
>  with frequent and unsynchronized releases of different products know what 
>  happens when two packages require different version of the same library, or 
>  some automatically updated package suddenly remove the little bit of the API 
>  you were depending on.

Yep, that's why you want some way to get a set of packages that work
together for lots of people and an easy way to add bugfix releases to
this set.

But I do think that a hand-managed super meta-package is a very
difficult thing to do.

>  Put differently, you may know that you need zope.annotation and 
>  zope.app.content, but I suspect that most people won't quite know where to 
>  start, and won't quite know what's available to them. If they start with 
>  some particular dependency and start exploring the dependency graph, they'll 
>  probably miss something that's either standalone or on a completely separate 
>  dependency subtree to the one they're on.
> 
> > I think another alternative is to separate the releases into different
> > "repositories", kinda like "All zope 3.0.0 eggs are at
> > http://download.zope.org/distribution/3.0.0/".
> 
>  Having eggs doesn't absolve us from doing release management. :) 

Add a meta egg with a few non-versioned dependencies to the above
collection of eggs and it becomes a release (or multiple releases) :)
Anyone, as long as they only download eggs from that URL, it guaranteed
a set of eggs that works well together.

Bugfixes are simple, simply drop an egg in that directory.

One very bad thing about such a URL is that it makes it very tempting
for developers to upload forked versions of things like twisted, pytz or
mechanize there.

However, buildout's approach of "saving" a known good dependency tree is
also reasonable:

http://svn.zope.org/zc.buildout/trunk/specifications/repeatable.txt?view=auto

Though I'm not sure how well it will deal with bugfix releases or
allowing others to extend a release you make (i.e. plone on top of
zope3).

Perhaps even buildout can become the method to create a directory of
eggs/tarballs that will be a "release".

/me realizes that his hands are waving vigorously!

>  We still 
>  need to take some of the pain of testing and making sure there are adequate 
>  tests and making sure things work well together. It's part of the QA process 
>  (which Zope 3 has done immensely well to date, in my opinion). If that 
>  overall responsibility eventually falls on no-one, then I fear we'll lose 
>  the great integration power we have in Zope 3 at the moment.

I'm sure both the "repository" and "saved" release types can be tested
together automatically using buildout.

>  Most people just want a "known good" that they can depend on, where they 
>  know that changes are going to be limited to important bug fixes. They don't 
>  care if they get ten extra packages, what's a few megabytes of disk space.

Yep, for this you can have some meta packages with a few un-versioned
dependencies in the included in the directory of "safe" packages.

>  It's all well and good (great, even) that people can specify their own 
>  dependency subtrees (I need the annotation and component parts of Zope, I 
>  don't want the rest), not at least because it gives entirely different 
>  platforms a better chance of re-using our code, but someone still needs to 
>  look after the whole. There are too many packages to have independent 
>  maintainers who each (a) are available to do timely releases (b) don't 
>  forget to make release after a while and (c) are able to test that the code 
>  that is supposed to work with other things, continues to work with those 
>  things.
> 
>  Martin
> 
>  _______________________________________________
>  Zope3-dev mailing list
>  Zope3-dev at zope.org
>  Unsub: http://mail.zope.org/mailman/options/zope3-dev/jinty%40web.de
> 
> 

-- 
Brian Sutherland