[Zope-Coders] CVS and unit testing discipline

[Tres Seaver]

I spent some time gardening CVS this evening, particularly in order
to get a stable sandbox with all unit tests passing.  I managed this
on the Zope-2_4-branch (which is good, as Matt is going to cut a
2.4.2 beta from it tomorrow morning).  I got started down this path
because I think I've found a bug in the way range searching is done
on FieldIndexes (actually, it looks like KeywordIndexes will try to
do range searching, too!) and I wanted to write a test case for it;
without a clean sandbox, I couldn't move forward on my fix.

I do have a couple of gripes here:

  - We aren't doing a good job of "walking our talk";  in particular,
    the tacit agreement to leave broken code or tests in place makes
    it very difficult for anyone else to know how to contribute.  Tests
    which can't be made to run should be commented out, with notes
    explaining the difficulty, and probably a follow-up note to this
    list explaining why.

  - Now that they are all running on the 2.4 branch, commenting them
    out is *not* properly an option;  if you break the test, you are
    supposed to fix it, not make it go away by cheating.  Running the
    *whole* suite before a checkin is a reasonable requiremenet:

      $ cd /usr/local/zope/Zope-head
      $ cvs -q up -dP
      $ python2.1 utilities/testrunner.py -q -a

  - I managed to fix all the long-standing test problems (DateTime
    floating point stuff, for instance, and the breakage in the ZCatalog
    test for creation w/o a vocabulary).  In a couple of cases, the
    ickiness of my fix (inserting a new security manager with a dummy
    user to get the catalog c'tor to work, for instance) are signs of
    ugliness (what the XP folks call "code smells".)  Such ickiness
    should probably also result in a note to this list, so that the
    folks who know about the sources of ickiness can think about ways
    to un-ick them.

  - The head still has guts all over the table:  the recipe above
    (with no modified code in the sandbox) has import errors on more
    than a dozen modules.  One suspicious error is a failure to import
    'AccessControl.cAccessControl';  wasn't the C-based security
    checking supposed to occur on a branch?

    In fact, *anything* which can cause test breakage, or delay other
    development, should be occurring on branches, and not on the trunk.

    The policy states:

      In the Zope area, no actual development work may happen on the
      "trunk". All development activity happens in unique branches,
      and are merged into the trunk only when the work is stable enough
      to be included in the next beta release. The trunk should never be
      in a state where  we would be uncomfortable making an alpha or beta
      from it on 2 days notice.")

    Getting the head back to stability should be a pretty high priority,
    methinks.

Tres.
-- 
===============================================================
Tres Seaver                                tseaver@zope.com
Zope Corporation      "Zope Dealers"       http://www.zope.com