[Zope3-dev] Dependencies and distribution

[Phillip J. Eby]

At 12:17 PM 5/26/03 +1000, Adrian van den Dries wrote:
>On May 24, Thomas Guettler wrote:
> > > Constructive criticisms and suggestions will certainly help.
> >
> > It would be nice if the new Catalog would only depend on ZODB. Then
> > you can take ZODB and the new Catalog and write your own frontend:
> > e.g. Corba or an other remote procedure call method.
>
>Extending this, I've always wondered why Zope isn't packaged, or why
>there are no moves to package, Zope(2|3) as a simple package with
>simple dependencies on each of the bits that can be used standalone
>(like ZODB, PageTemplates, Catalog, etc.), and zope.org simply
>distributes all the bits individually in source form (to aid package
>maintainers like Debian's) and only combine them for the binary
>builds, and perhaps one "everything" source tarball.  Wouldn't this
>appeal to both the only-the-bits-you-need crowd and the
>all-in-one-place crowd, between which the Zope community seems equally
>divided (although the OSS/FS/filesystem-developers subgroup would
>always, I suspect, be biased toward the latter group).  Are there any
>insurmountable hurdles in working towards this distribution model?

Speaking as both a packager-of-Pythonic-things, and a user of such 
packages, there is significant overhead right now to produce separate 
distributions.  The distutils, for example, require a nice tidy list of 
every package and extension, and if you're doing anything that's the 
tiniest bit custom, you need to replicate it across your setups for each 
package.

In short, distutils is optimized for building and distributing *one* 
"bundle", not lots of small ones.  (This is entirely aside from the 
dependency management issues.)

In order to work around this for my own package distributions, I've begun 
creating a number of "include" files that I can use in setup.py via 
'execfile()'.  These files reside in a subdirectory of each distribution, 
and the subdirectory is CVS-symlinked so that it's shared across different 
packages.  What I don't currently have is a mechanism to manage partial 
builds/installs/distributions.

Well, I *do* have a mechanism, it's just boolean flags at the top of 
setup.py, e.g. 'install_foo', 'install_bar', etc.  I'd like to replace it 
with something that you could give --with-foo / --without-foo type options 
to, and a more metadata-driven setup model.
In order to cleanly handle separate packaging, it'd be necessary to define 
the larger distribution in terms of the individual pieces.  Right now, the 
typical setup.py consists of a big, centrally located, all-in-one glob of 
information.  I think I'd want to have directory-local 'setup_info' files, 
that contain option data that could be processed into a MANIFEST or 
MANIFEST.in, as well as data for the setup's build/install options.

Anyway...  I don't see any "insurmountable" hurdles, but I think the 
challenge for anybody who wants this is to build some better mousetraps for 
the distutils on the packaging side.