[Zope-dev] Change in repository approach to software sharing

[Tim Peters]

[Jim Fulton]
> The Zope project includes a number of interrelated subprojects,
> such as:
>
>    - Zope 2
>
>    - Zope 3
>
>    - ZODB
>
>    - ZConfig
>
> Software from the ZODB and ZConfig projects are shared by Zope 2
> and Zope 3.

Note that ZODB also depends on ZConfig.

> We want this sharing to be very convenient for people working on Zope 2
> and Zope 3.  We don't want users of the Zope 2 and Zope 3 repository to
> have to do separate checkouts for ZODB and ZConfig.  CVS supports such
> software sharing through it's "module" system. The module system has
> some flaws, so we use symbolic links instead.
>
> In a response to:
>
>    http://dev.zope.org/Zope3/MovingSCMToSubversion
>
> John Belmonte has suggested that Zope 2 and Zope 3 should depend
> on specific versions of shared packages, rather than on the head.
> I'm inclined to agree.

Didn't we endure a lot of pain over the last year to get away from that
model?  At one point there were 7 lines of ZODB I was wrestling with every
day.  For example, there was the ZODB 3.1 branch, which "in theory"
corresponded to the Zope 2.6 branch, but in practice never matched it.  Life
got much better (for both Zope 2.6 maintenance and ZODB 3.1 maintenance)
when we abandoned the ZODB3-3_1-branch branch of ZODB3 and did all ZODB 3.1
maintenance directly on the Zope-2_6-branch branch of Zope.  Likewise for
ZODB3-3_2-branch versus Zope-2_7-branch:  the former was abandoned, and life
got better for everyone because of that.  Currently the HEAD of the ZODB
module is the same as the HEAD view of ZODB when viewed from the Zope and
Zope3 modules, which isn't a pure win, but isn't a pure loss either.

> People working on ZODB and ZConfig have to test their changes against
> both Zope 2 and Zope 3 to avoid breakage.

I don't see that this part can change, no matter how names are shuffled.

> This is very burdensome

That part either -- all the tests in all contexts still need to be run.

> and causes much pain when they get it wrong.

Ditto, although you're (re)introducing mechanism that alleviates this at the
cost of increasing the number of ZODB snapshots in active use.

> Fortunately, subversion provides a mechanism for sharing specific
> revisions.  We'll be able to have the convenience of getting
> ZODB and ZConfig (and other shared software) when we do a checkout
> *and* we'll be able to control what parts we get.
>
> To see how this will work, we'll look at ZConfig as an
> example (because it has a single package) of reusable software
> that we will include in Zope 3.
>
> In the repository, we'll have a top-level Zope3 project directory,
> with the standard svn subdirectories "trunk", "branches", and "tags".
>
> We'll also have a top-level ZConfig project directory.  The "trunk"
> of the ZConfig Python package will be in ZConfig/trunk/src/ZConfig.
> If we create a tag T1 of ZConfig, then the Python package for that tag
> will be in ZConfig/tags/T1/src/ZConfig.
>
> Now, when we set up the Zope 3 repository, we will create the ZConfig
> package in Zope 3 by copying a *tag* from the ZConfig project:
>
>    svn copy svn+ssh://svn.zope.org/repos/ZConfig/tags/T1/src/ZConfig \
>             svn+ssh://svn.zope.org/repos/Zope3/trunk/src/ZConfig
>             -m 'Bring ZConfig T1 into main branch'
>
> Then, whenever someone checks out the Z3 trunk, they'll get
> the ZConfig from T1.

I don't yet know enough about svn to guess how the next step works:  since
ZODB all by itself depends on ZConfig, presumably similar stitching of
ZConfig will take place in the top-level ZODB project.  Now when we go on to
stitch ZODB into Z3, how will ZODB's attempt to stitch in its own version of
ZConfig play with Z3's attempt to stitch in T1 above?  I'm not picturing how
this works.  Maybe it's simple!

> If we need to, we can even make Zope3-local changes to ZConfig.

That would bloat ZConfig maintenance headaches exponentially.  I'm confident
of that, because of ZODB experience with (at least) 7 concurrently active
ZODB snapshots -- it's simply intractable to keep straight what should and
shouldn't get cross-ported across this set; mistakes and oversights are as
much the norm as the exception then; the non-ZODB people doing stitching of
ZODB into their projects won't be confident about exactly which tag they
should stitch in; expediency will cause them to stitch in "whatever seems to
work" at the time; and then the latter becomes an active snapshot that needs
to be maintained too.

Don't make Zope3-local changes to ZConfig or ZODB, though, and at least that
nest of rats can be sidestepped.

> Later, we may decide to upgrade the Zope 3 head to use ZConfig tag 3.

Ya, and the Zope head may decide to use ZConfig tag 44, while the Zope 2.8
maintenance branch decides to use ZConfig tag 37 despite that ZConfig 3.3.1
(which is supposed to go out with the next Zope 2.8 release) is actually
released at tag 39.  The dark side of "flexibility" is that it creates that
many more ways to get out of synch.

> At that point, we can recopy from the tag, or we can merge changes
> made between the two tags.

Right now, the burden for making ZConfig changes "work" is entirely on Fred,
because the changes he makes show up everywhere at once.  Similarly for ZODB
versus me.  The "we" in the above is muddy:  whose responsibility will it be
to recopy/merge?  A clear answer to that may make it work better.  As a
counterexample, it was never clear whose responsibility(ies) it was to
coordinate ZODB versions with Zope(2) releases, and before we eliminated the
distinct brances for ZODB and Zope, the version of ZODB shipped with a given
Zope release never corresponded with a ZODB release.  It's only in the last
year that we managed to create ZODB releases that truly matched Zope
releases.  Then again, there was a lot of hosage in those processes <0.5
wink>.