[Zope3-dev] Re: Known working sets II [was: Eggification redux]

[Martijn Faassen]

Hey,

On 9/28/07, Tres Seaver <tseaver at palladion.com> wrote:
> Martijn Faassen wrote:
>
> > On 9/28/07, Tres Seaver <tseaver at palladion.com> wrote:
> > [snip]
> >> Total effort involved in maintaining the "gated community" then becomes
> >> keeping a set of tarballs available at some web-downloadable location,
> >> and re-running the script after adding / removing them to regenerate
> >> the index.
> >
> > How many of these communities are you going to need? Why can't you
> > simply maintain a list of exact versions with version numbers to pull
> > from the cheeseshop instead?
>
> Because you can't trust that packages will not get removed, or even
> re-released under the same version number, on PyPI:  not everybody has
> the same "package hygeine" ethos.

True.

[snip]
> I've been using buildout and its precursors for *years*, and I still
> have my "repeatable" builds break on occoasion, e.g.:
>
>  - The Postgres guys decide to yank an older package version from
>    their servers because they've released a newerone.
>
>  - Somebody does "repository surgery" in a way which breaks my checkout
>    (e.g., because Subversion checks the revision number *after*
>    traversal rather than before).

This one wouldn't be relevant if one only relies on releases, right? I
mean, it's relevant for buildouts of course, but doesn't seem to apply
to the cheeseshop use scenario directly.

>  - Somebody uploads a "fixed" tarball of a relase without bumping
>    the version number.

Okay, good examples of how this can go wrong. I haven't been bitten by
these yet, but you have more experience.

> In the end, if you want predictable / repeatable deployment, you have to
> mirror the sources.  The fact that easy_install's '--index-url' feature
> makes such a mirror convenient is just a bonus.

How would this work in scenarios where you have a framework, like
Grok, which maintains its own package list, and an application that
uses Grok which also needs a few extra packages? Would the application
developer need to maintain his own package index too which includes
the Grok list? I imagine the application developer couldn't use the
particular grok index *and* the cheeseshop, as that risks slipping in
newer eggs.

What about new releases? If we release grok 0.10.1, would that involve
the creation of a new package index for it?

gated communities concern me for some reasons:

* We want to avoid the scenario where Zope packages don't make it to
the cheeseshop at all anymore, or that they make it but they're
unusable in that form.

* We don't want to make life harder for the casual developer who just
wants to try this Zope stuff.

* We don't want to make life harder for the application developer who
wants to release his application.

* I'm still hoping we can forge the way for the rest of the python
community instead of going off on our own.

How would you propose dealing with these concerns? We want our tools
to be "standard" if possible, and if we're going to invent a new way,
we want to make it very easy to use, which means easy technology but
also good beginner's documentation. I'd like to avoid creating another
barrier for adoption of our stuff.

I wonder how the Perl community has dealt with problems like this in
CPAN use (if they have at all).

Regards,

Martijn