[Zope3-dev] Roadmap for Zope 3.4

Tue Sep 12 09:50:29 EDT 2006

Jim Fulton wrote:
> 
> On Sep 12, 2006, at 8:40 AM, Martijn Faassen wrote:
[snip]
>>> One problem we have is getting things to be tested.  It hardly 
>>> motivates people to test for and report bugs if their reports
>>> don't affect he release. I think we have a serious problem that
>>> needs to be addressed.  I don't think the right way to address it
>>> is to release despite known serious bugs.  Note that some
>>> judgement goes into considering whether a bug is serious enough
>>> to block a release.  We don't block a release for just any bug.
>> 
>> Before a release, bug triaging needs to take place.
> 
> Which we do.

I know, but perhaps we need to be a bit more aggressive. Christian
Theune's point about empowerment to defer a bug may be useful. We may be
a bit too fearful sometimes to defer bugs right now.

>> I recall you saying we defer bugs too often and bugs never get
>> fixed, so we should stop doing any feature work until all bugs are
>> fixed, etc.
> 
> We should. We have yet to do this.  As I mentioned in another note,
> we should prevent new features at the beginning of a release cycle
> until we've caught up on past bugs.
> 
> Trying to address old bugs was not responsible for the delays in the
> 3.3 release.

I think it contributed, though fully agreed it isn't the only source of
the delay.

>> I call that perfectionistic.
> 
> Then your idea of perfection and mine are far apart. Letting bugs 
> languish for months or even years is not acceptable.  Ignoreing bugs
>  reported during beta testing, when we get too little testing to
> begin with is unacceptable.

*Ignoring* bugs is unacceptable. Explicitly deferring a bug for months 
is acceptable, as there are always more bugs than we can fix, and we 
should fix the bugs of the highest priority. A low-priority bug may 
therefore languish. I'm not saying we are doing this correctly *now*, 
but such would be a reasonable process.

>> I think we have been blocking this release with a selection of bugs
>>  that included quite a few that weren't absolutely critical.
> 
> Name a few.

Good point. Going through the bugs fixed over the last period makes them 
seem all important to me or alternatively, lesser important bugs fixed 
by the reporter themselves. Perhaps a hard-headed release manager who 
will say "important schrimportant" and defers the issues anyway would've 
been useful.

Still:

http://www.zope.org/Collectors/Zope3-dev/553

http://www.zope.org/Collectors/Zope3-dev/576

http://www.zope.org/Collectors/Zope3-dev/540

http://www.zope.org/Collectors/Zope3-dev/530

http://www.zope.org/Collectors/Zope3-dev/537

http://www.zope.org/Collectors/Zope3-dev/583

http://www.zope.org/Collectors/Zope3-dev/534

Some issues were only discovered way after we should've done the 
release. We've been
fixing these as part of the release process. All of these issues appear
to be reported after mid-july or later, though. I  think those are good 
bugfix release candidates, and shouldn't have been blocking our 3.3 
release (not all of them are marked critical, but these all have been 
fixed):

http://www.zope.org/Collectors/Zope3-dev/677

http://www.zope.org/Collectors/Zope3-dev/675

http://www.zope.org/Collectors/Zope3-dev/676

http://www.zope.org/Collectors/Zope3-dev/679

http://www.zope.org/Collectors/Zope3-dev/674

http://www.zope.org/Collectors/Zope3-dev/683

http://www.zope.org/Collectors/Zope3-dev/681

http://www.zope.org/Collectors/Zope3-dev/683

http://www.zope.org/Collectors/Zope3-dev/682

http://www.zope.org/Collectors/Zope3-dev/680

http://www.zope.org/Collectors/Zope3-dev/673

http://www.zope.org/Collectors/Zope3-dev/690

http://www.zope.org/Collectors/Zope3-dev/696

http://www.zope.org/Collectors/Zope3-dev/691

http://www.zope.org/Collectors/Zope3-dev/697

Since these all were reported in july or later, we couldn't possibly 
have fixed them if we'd have released in june. :)

>>> I would suggest we triage bugs a lot more aggressively instead. I
>>> say this as someone who spent a few afternoons staring at Zope 3
>>> bugs trying to get them out of the way.
> 
> I've spent way more than a few afternoons.

Gratefully acknowledged.

>>> I can think of a number of ways to approach this problem: 1. Do
>>> less frequent releases.
>> 
>> I do not believe this helps a lot for bug fixes. If we have twice
>> the release period, people won't start fixing bugs twice the amount
>> of time in advance,
> 
> Right. This, alone is not a solution.
> 
>> and we won't get twice the volunteers either.
> 
> No, but it will halve the work for the small existing group of
> volunteers.

I do not believe this to be necessarily the case. The list of bugs fixed 
that were reported after our supposed june release is one reason why I 
think that. The other reason is that there'd be two times as much space 
between bugfixing rounds, which will make bugs languish and people get 
out of the habit of fixing them.

>>> 2. Feature freeze the trunk until the previous release has made
>>> it to release candidate status
>> 
>> You mean don't branch the trunk (and thus let it be the release 
>> branch) until the release has made it to release candidate status?
> 
> Yes.  I think it is a shame to have to do this, but I think this is
> now necessary.

I think this is a good idea.

> I would go further.  I would not unfreeze the trunk until until we've
>  cleaned up all open bugs, either by fixing them or rejecting them.

-1

>>> 3. Release less.  I think it's time to start thinking of some
>>> sort of "core" Zope 3 that we can manage with the very limited
>>> number of volunteers we have now.
>> 
>> +1, though I wonder how much this has been blocking the release - 
>> important bugs that could block releases don't tend to be in out of
>>  the way components, one would think.
> 
> I spent a lot of time on crap that wasn't core at all.

So why were these critical issues? What happened during triaging?

>>> 4. Get more volunteers.
>> 
>> Getting a release out is *difficult* and the amount of volunteers 
>> available, while important, is only partially related. More
>> volunteers won't speed it up tremendously much and can even slow
>> things down. Plone has many volunteers and has had much problems in
>> the past doing timely releases. Silva has had far less volunteers
>> and can do more agile releases, because there's less people to
>> manage.
> 
> And because it is much smaller. 

Yes, smaller helps.

> I think we're far from having too
> many volunteers on Zope 3 maintenance.  I'm not optimistic that we'll
> get more volunteers, which is why I'm inclined to release less.

I'm coming from the perspective of trying to work best with the existing 
volunteers as well.

>>> These are just some ideas. But something has to give and I don't
>>>  think it should be responsible bug fixing prior to release.
>> 
>> Large Zope 3 projects have been working against 3.3 for many months
>>  now. If there was any absolute showstopper in Zope 3.3, why have
>> these projects successfully transitioned?
> 
> It all depends on how large you want the audience to Zope 3 to be.  A
>  small community on a limited number of platforms can work around 
> well-known problems.

I think such a larger audience needs to grow organically and has fairly 
little to do with the bugfixing issues.

>> Who or what would have been hurt exactly had Zope 3.3 been released
>> in june?
> 
> Well, we would have been hurt badly as the first beta release, the
> one made at around that time was badly broken.

Who is we? Why would we have been hurt? Even if such a badly broken beta 
would've gone out (which I was not suggesting, obviously we do *some* 
testing), we could've done a bugfix release.

> There were some serious bugs.

Agreed.

> We would have demonstrated pretty clearly that we don't care about
> quality.

To whom? What if we'd have followed up with bugfix releases? What about 
demonstrating we can release when we say we will?

>> I don't think it's Zope 3's reputation that would've been hurt, as
>>  Zope 3.3 in june was not *that* buggy. Bugfix releases are also a
>>  completely accepted practice.
> 
> Except that we don't do much of those and we put little effort into
> the ones we do do.

Let's do more bugfix releases. I don't think they can be avoided anyway.

>> I still think our quality standards for a release have been too
>> high. Getting people to fix more bugs is good, sure, but perhaps we
>> should separate this at least somewhat from the release itself.
> 
> Sorry, I agree very much.  I'd be willing to defer to a Zope 3
> release manager, if there was one.

I'm not sure what 'sorry, I agree very much' means.

>> If we're going to do timebased releases, we should have a button 
>> somewhere that says 'make a release', and a date on which the
>> button is pressed. If the release is botched, we can press the
>> button again for bugfix releases, and we have 6 months in which to
>> do a better job next time.
> 
> IMO, that would be a disaster.

In what way? To whom?

Regards,

Martijn