[Zope3-dev] Why do we restrict our egg testing?

[Jim Fulton]

On Sep 27, 2007, at 10:00 AM, Stephan Richter wrote:

> On Thursday 27 September 2007 09:35, Jim Fulton wrote:
>> On Sep 26, 2007, at 7:57 PM, Roger Ineichen wrote:
>>> Hi
>>>
>>> Can anybody tell me why we restrict our test setup
>>> in zope eggs and only use a subset of package for
>>> our test setup?
>>
>> It isn't practical, during development, to test all of the eggs that
>> might be affected by a change, which is, BTW a *much* larger set than
>> the old Zope 3 tree.
>
> I am not saying that testing *all* packages is necessary, but a  
> healthy set of
> them. We did not consider this impractical when developing on the  
> entire
> trunk. Back then, it was part of our development responsibility. We  
> always
> ensured that some set of Zope 3 was always stable together.

But we were developing the entire trunk as a whole.  We were  
developing a monolith. For example, it was acceptable, when changing  
a particular Python package to "fix" the tests in other packages that  
broke by changing the tests (or the code they test) to reflect the  
change in the original component.  This provided less than no  
protection to 3rd-party packages.

Recently, y'all removed some files from zope.app.security.  I bet you  
adjusted other files in the Zope 3 tree to reflect that change, but  
that didn't help the many more 3rd-party packages and applications  
that were broken.


> This is
> completely gone now. I want a solution; buildbot is okay,

Good. Now we need someone to implement it.

...

>> Depends on what "this" is.  I think running all of the tests in the
>> old Zope 3 tree whenever we change a component is a waste of time.
>
> I beg to differ. It has something to do with responsibility to the  
> community.
> I am not saying you have to test all of Zope 3 when changing zc.* for
> example. But I think if a package in a defined "core" set is changed,
> requiring to run all the tests of the "core" set should be  
> required. A really
> good example is ``zope.component``. Not testing it across several  
> packages is
> very dangerous for our stability.

That's a good example.  The last time I changed zope.component, I  
*did* test it with the tree.  It would be useful to be able to do  
that conveniently in the future.

What I really want is a buildout for a big Zope 3 application,  
similar to what we've released in the past.  Then, I will often  
choose to test a change in something as core as cope.component as a  
develop egg in that buildout.  I would definitely do that.
(I don't want a meta egg.)

I'll note that I don't work on "core" components very often so  
testing against the Zope 3 app wouldn't be something that I would do  
often, but I'll concede that on those rare occasions that I do,  
having something like this would be useful.

Jim

--
Jim Fulton
Zope Corporation