[Zope3-dev] package branches

Mon Jul 2 14:42:42 EDT 2007

On Jun 27, 2007, at 11:36 AM, Gary Poster wrote:

>
> On Jun 27, 2007, at 2:30 AM, Bernd Dorn wrote:
>> On 26.06.2007, at 21:44, Gary Poster wrote:
>>> On Jun 26, 2007, at 3:29 PM, Bernd Dorn wrote:
>>>> i think as long the package has a dev dependency like ZODB 3.9  
>>>> it should at least have alpha or beta status
>>> Hi Bernd.
>>>
>>> Why?
>>
>> because it pulls in software that has development status like  
>> zodb  3.9 and the release of 3.9 will take at least a half a year  
>> from now on imho.
>
> My basic response--"don't use this version of  
> zope.app.keyreference"--doesn't appear to be sufficient for you (or  
> Theuni, from his "Specifying upper limits in dependencies" email  
> today).

I have asserted, and I think people agree that setting upper limits  
in setup scripts is a bad idea as it will soon lead to situations  
where the current algorithm used by setuptools to build working sets  
of compatible revisions will fail.

Of course, it isn't a problem to introduce tighter version  
requirements in "application" setup scripts or buildout configurations.

>   It is ok for me because my project is specifying versions, which  
> is also the strategy he says he is pursuing.  I don't know of a  
> better one, actually.  Both you and he seem to be looking for one  
> though, which I appreciate.
>
>>>> gary, is it possible to be compatible to 3.8 too?
>>>
>>> Not productively.  We could have "if the PersistentReference  
>>> doesn't have the 3.9 stuff then just refuse to do a  
>>> ConflictError" but then that's no different that the keyreference  
>>> 3.4 behavior.  Heh, actually, that's effectively the behavior we  
>>> probably have now for keyreference 3.5dev running against ZODB  
>>> 3.8, since errors in the conflict resolution will simply cause  
>>> the resolution to fail, and the 3.5dev changes would generate  
>>> AttributeErrors against ZODB 3.8 during conflict resolution.
>>>
>>> So...it would be a bit of a lie to claim to be compatible with  
>>> 3.8.  The changes are useless without the 3.9 changes.  But the  
>>> code *should* technically work with the same restrictions we have  
>>> now.  That said, I don't really want to support the changes  
>>> against 3.8.
>>>
>>> ...I could move the releases to our ZC download location, rather  
>>> than the zope.org one, if folks want...
>>
>> i don't think that this is a good idea, for example our company  
>> uses both of the download locations
>
> Well, I can move it any number of places, but this does sound like  
> a hack, hiding the basic problem.

Yup.

>>> What's the problem?  I'm happy to help, especially if it doesn't  
>>> take too much time, and you can wait a day or two.
>>
>> ok, i think if another new feature is implemented in keyreference  
>> and we want this feature for zodb3.8 we have to do a version  
>> inbetween, so if you call this a 3.5 release, what should that  
>> other version be?  3.6
>
> I guess this is a problem.  I certainly understand the story.
>
> We (the community/dev mailing list) had said before that you needed  
> to start a new major version when you broke backwards  
> compatibility.  I did not break backwards compatibility, I just  
> depended on a new feature of another package.  The only answer I  
> see to solve the story you just told is to also say that you have  
> to rev the major version whenever you depend on the new feature of  
> another package.

I think that in *technically* depending on a newer version introduces  
a backward incompatibility. (It's like strengthening a precondition  
of a method.)  This is especially a problem if people are specifying  
upper limits for their requirements, which is only necessary if there  
backward incompatible changes.  ZODB 3.9 won't be backward  
incompatible in any meaningful way.

ZODB 3.9, like 3.8 *is* backward incompatible because it drops  
features that we agreed, as a community, to drop. Arguably, ZODB 3.8  
should have been ZODB 4 and ZODB 3.9 should be ZODB 5, but I don't  
think we want to do that.

So, let''s say, as a practical matter, that ZODB 3.9 is backward  
compatible with 3.8.  We don't know that 3.9 is ready for use yet.   
It's possible  that there have been bugs introduced in 3.9 (or soon  
will be) that will be discovered and fixed during the beta cycle.   
Many people rightly don't want to depend on 3.9 until it has been  
released in a final form.

Bernd has suggested a policy that I think we should adopt, which is  
that a package's release status should be no higher than the status  
of it's dependencies. So, for example, if A depends on a dev version  
of B, then the version of A that has this dependency should be a dev  
version.  A should not be able to go to alpha, or beta, or final  
without a corresponding advance of its dependency on B.

> However, I am afraid that this would lead to ugly (and expensive to  
> maintain) if-else clauses in our software, or rapidly escalating  
> version numbers, and even maintaining multiple branches  
> simultaneously.
>
> Dependencies are precisely what is supposed to address this  
> problem, I thought.

Except we've found corners in the dependency model that don't work or  
that we don't understand yet.

>> we use egg based releases and if you hardcode the zodb 3.9  
>> dependency in setup.py we have to switch to zodb3.9 just because  
>> of that package if we want to use a new feature of it
>>
>> maybe i am anticipating here and it's best to make it zodb 3.8  
>> compatible when we need a newer version of keyreference for some  
>> reason. the problem with this is, that we (zope committers) can do  
>> this, but another company may not be able to change the package.

...

> - For this particular problem, I wish my ZODB changes could go into  
> ZODB 3.8.  As I mentioned in the commit message, the change  
> includes one bug fix for a potentially serious problem that could  
> cause data integrity issues;

This could go into 3.8.

> one new feature (more informative PersisntentReferences) that  
> supports an arguable zope.app.keyreference bugfix (conflict  
> resolution breaks with persistent key references;

Can you remind me or point me at something that describes in more  
detail?

Does zope.app.keyreference work as well as it does now without this?

> and one feature (conflict resolution now supports multi databases).

This is a bug fix and could go into 3.8.

>   Then I'd argue that the zope.app.keyreference change could go  
> into 3.4.

Are we really talking about 3.4?

> - Even if people agreed with me for the previous bullet, that  
> doesn't make this kind of problem go away.
>
> - Also for this particular problem, I suppose I could remove the  
> dependency on ZODB 3.9 and add a test to show that it should still  
> "work" (as well as it did before at least) with ZODB 3.8 persistent  
> references.  I'd really rather not, of course, but I can do it in  
> the next week and a half.  I guess it would still be "3.5" but it  
> wouldn't depend on 3.9 (...except to actually have any improvements  
> over 3.4!)

I think you should do this or make your version of  
zope.app.keyreference a dev version. Unfortunately, the later doesn't  
solve the problem by itself.

> My current opinion is that version numbers are the best solution of  
> a bad lot.

I think we really need a way to limit the algorithm for finding  
distributions to avoid immature (releases).

I just brainstormed this with Benji and we both think that it would  
be enough to have an option that says: "I prefer final versions"  
meaning that we always want the latest final version that satisfies  
our requirements, if there is one.  Ideally, this would be a  
setuptools option, however, this would be a new feature and I don't  
think we'll see new setuptools features any time soon.  I could add  
this as an option to buildout.

I'll try to summarize as a proposal in a separate message.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org