[Zope3-dev] Re: Known-good-sets problem

[Martijn Faassen]

Jim Fulton wrote:
> 
> On Oct 6, 2007, at 5:24 AM, Martijn Faassen wrote:
[snip]
>> What I really don't like about this proposal is that it talks about 
>> updating index pages. If I understand this right, an updated index 
>> page will force everybody that uses this index page into an update.
> 
> Only people who ask for updates.

Yes, but it's important in releases. If I release my application to 
someone else, they will be asking for an update while I might never had. 
I cannot test this as the updated versions might've been released 
*after* I make my application release.

You can argue I should maintain my own index for releases (or do some 
other packaging). This would then only be a tool to assist developers. 
Then again, developers also share code and could run into similar issues.

>> I don't think this is acceptable.
> 
> I think this depends on the use cases.  I think most people would like 
> to get bug fixes.  The intent of such a 3.4 index would be to give 
> people the latest 3.4 updates which should give no new features.  This 
> is intended to mimick what happens with linux package respositories.

Yes, bugfixes is less of a problem than feature releases, if this is 
well managed. Since packages evolve at different rates, what is done for 
feature releases of packages? Do they end up in the same index or 
separate indexes? How to determine when to make a new index?

Let me make the case for bug-for-bug compatibility:

The problems during release as I sketched out above still exist. Granted 
for bugfix releases the chances are lower that something breaks than it 
is now, but we all know people sometimes disagree about what actually 
constitutes a bugfix (or larger change), and people also sometimes have 
workarounds or code assumptions that depend on bugs. Bugfixes can break 
working code.

>> Instead I'd suggest you institute a rule that index pages should never 
>> be changed after initial publication, just like eggs shouldn't be 
>> changed after publication either. Instead, a new index page should be 
>> published for each update. People can then choose to switch whenever 
>> they like.
> 
> You could manage an index like that.  I don't think that would be very 
> useful.  IMO, if you want to really nail version numbers, then it would 
> be better to just use meta eggs or buildout version specifications.
> 
> The goal of the index is to give people a stable source of distributions 
> that are known to work together.

Okay, understood. I can see some utility for this, if only to help me 
generate the lists of pinned versions. :)

[snip]
> If the software is an application, the best approach IMO is for each 
> release to specify precisely the versions it's using.  If software is a 
> platform, then more flexibility may be needed,  This index idea seeks to 
> learn from experience with linux distributions.

I see some differences between this and a Linux distribution.

A linux distribution releases bugfix releases when:

* there's a security problem

* in some cases when there's something obviously broken in an application

I don't know whether linux distributions commonly release bugfix 
releases in *libraries*. Do Linux distributions commonly release bugfix 
versions of glibc, or Python libraries? I guess it depends on how 
severely the bug impacts important applications.

In our case, we generally release bugfixes to developers of 
applications, not users of applications. For security bugfixes, I can 
see the argument for getting them out as quickly as possible, even at 
the risk of sometimes breaking applications.

What other kinds of bugs do we have? How do they affect developers?

>> We've gone another direction with the Grok project this week, where we 
>> publish a versions.cfg on the web that gets included in a 
>> buildout.cfg. This approach was based on suggestions by yourself to 
>> me, and had been discussed by Philipp on the mailing list earlier, but 
>> apparently the choice was made to go for index pages instead.
> 
> Yup.  Of course, it only works with buildout.  At the time, I remember 
> getting push back from someone that they wanted a solution that didn't 
> require buildout.  A custom index is more open both wrt packaging 
> technology and to use by different applications.  I think Zope 3 would 
> benefit from a stable repository of eggs that can be used even if people 
> aren't using Grok or buildout.

Sure. I don't think having such an index is a bad idea. I'm watching 
this experiment with much interest and I think it helps us asking good, 
concrete questions about potential problems to refine our story here.

> Of course, the index technology isn't worth much without solid processes 
> behind it.

 >> Anyway, we'll be using this for a while at least and we'll wait and
 >> see how this new works pans out.
 >
 > I think the index complements techniques like meta eggs or
 > buildout.cfgs with nailed versions.

That's a good point. I can see how much of the process could be shared 
between the maintainers of an evolving index and the maintainers of 
version lists.

[snip]
> Also, keep in mind that this effort results from a desire to help and 
> because people care.  Be careful about accusing people of not caring and 
> then dismissing their efforts to help.

I wasn't intending to dismiss this project at all, I am trying to give 
my feedback to it and trying to understand the goals. I understand it a 
bit better now. My apologies for saying people don't care.

> I really appreciate the efforts of Philipp and Baiju to help us make 
> incremental progress on the release process for individual projects. I 
> think we need to start working (thinking & talking) about the release 
> process for the larger Zope 3 ecosystem.

I see this as part of the conversation.

To me personally this issue evolved from "serious" to "problematic" to 
"critical" over the last month or so, as I noticed our Grok release 
broke over and over again (which sucks if you try to reach out to 
beginners), and I got several customers contacting me with buildout 
problems. Several other developers also reported such problems. So I got 
pretty frustrated over this. I should've shown it less.

Regards,

Martijn