[Zope3-dev] Re: Windows eggs

[Marius Gedminas]

On Fri, Jul 13, 2007 at 11:09:30PM +0200, Wichert Akkerman wrote:
> Previously Tres Seaver wrote:
> > Stephan Richter wrote:
> > > I think in the long term it will be most beneficial, if we convert all tests 
> > > to doctests; then a reasonable on-line documentation is not that hard to 
> > > provide.
> > 
> > - -1.  "Good tests" don't always make "good documentation";  I prefer the
> > isolation of traditional unittests for anything other than "main line"
> > use cases, for which doctests are best suited.
> 
> Amen. I find failing doctests to be much harder to debug as well. I use
> doctests as a method to make sure examples in my documentation are
> correct, which is very useful.

Doctests are hard to get right.  I've finally settled on classifying my
tests into four categories and using different styles for them:

  - API documentation: human readability is the primary concern, doctests
    are in there just to make sure the documentation stays up to date.
    These are .txt files.

  - Short usage examples: sometimes it's simpler to show an example than
    to describe the function in words:

        def parse_date(date_string):
            """Parses an ISO-8601 date.

                >>> parse_date('2007-07-14')
                datetime.date(2007, 7, 14)

            """

    This is the only case when I allow doctests in code modules.
    Putting long test suites in class/function docstrings clutters the
    code and makes it harder to read.

  - Unit tests: there are many of those, they're independent (thus a
    single .txt for a collection of tests is a Bad Idea), they're short
    (so understanding and debugging is easy) and expressive.  I put
    those into .py files full with functions that look like

        def doctest_FooClass_does_this():
            """Test that FooClass is able to ...

                >>> foo = FooClass()
                >>> results = foo.do_this()
                >>> for fruit, score, comments in results:
                ...     print fruit.ljust(10), '|', score.ljust(5), '|', comments
                Orange     | 9     | Tastes good, a bit sour
                Apple      | 8     | Varies

            """

    and have a traditional test_suite() function at the end that returns
    a DocTestSuite with the appropriate setUp/tearDown/optionflags.

    In effect this is a more or less direct translation of the
    traditional unittest tests.  I find that rewriting the assertions
    into doctest format helps me make the tests more readable.  Compare
    the above with

        class TestFooClass(unittest.TestCase):

            def test_does_this(self):
                foo = FooClass()
                results = foo.do_this()
                self.assertEquals(results,
                                  [('Orange', 9, 'Tastes good, a bit sour'),
                                   ('Apple', 8, 'Varies')])

    and especially compare the output you get when the test fails.

  - Functional tests: these are .txt files that use zope.testbrowser and
    are the hardest to debug.  There ought to be a better way to make
    assertions about HTML output than using ELLIPSIS and then pulling
    your hair out looking at huge and incomprehensible diffs.

        Digression: syntax highlighting the diffs helps immensely.
        Check out http://svn.zope.org/zope.testing/branches/colorized-output/

        Another digression: this is why I want Zope 3 to run on Python 2.5.
        I want to make XPath queries on the HTML, and I hope I'll be
        able to do that with cElementTree.

Marius Gedminas
-- 
Added mysterious, undocumented --scanflags and --fuzzy options.
        -- nmap 3.0 announcement
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mail.zope.org/pipermail/zope3-dev/attachments/20070714/74feb809/attachment.bin