[Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

This kind of goes with Lennart's frustration about trying to port the
ZTK packages, or a core subset, to Python 3.

I would like to see the ZTK packages have really excellent
documentation, as well as 100% test coverage.  I think we need to look
at refactoring how the testing story is done to get both needs
satisfied:  trying to achieve both with a single set of doctests is
pretty near impossible:

- - Testing setup and teardown code obscures the narrative flow / intent
  of the documentation.

- - Edge case testing *really* obscures the same.

- - Test coverage suffers, because it is hard to write edge-case tests
  in doctest.

- - Porting the doctests where people have tried to juggle both goals
  across Python versions is a mess.

The refactoring I would like to see happen is to move the main narrative
documentation into a separate, Sphinx-driven 'docs' directory in each
ZTK package.  As part of this move, we can start adding some of the
really nice Sphinx features (cross references, indexing, auto-generation
of API reference information, etc.).

We can still leave the "main line" of the code illustrated using
doctest-style blocks:  Sphinx has nice support for testing those blocks
*while building the docs*.  At the end of this part of the change, each
package would have a much improved documentation set, with tested
examples.  At the end of the process, we would be able to put the docs
for each package into an "intersphinx" tree underneath the main ZTK
docs, which could serve as the gateway into them.

The trickier testing bits we would re-write as super thorough, no
shortcuts-taken unit tests:  one testcase class per class (or API
function) under test, at least one method per class-under-test method,
with more preferred to get all code paths / preconditions covered.  The
goal here would be to comment out the doctests from the test_suites and
get the code to 100% coverage using pure unit tests.  Meanwhile, the
doctests would be refactored into the Sphinx documentation, losing all
the bits which obscure their intent as documentation.

I would say that the risks here are slight, given that aiming for 100%
test coverage is likely to catch any oversights made during porting to
the new style.  The amount of work for any given package is fairly-well
contained, I think:  most of the effort will be in reverse-engineering
the intent of the "edge-case" tests.

I made a sketch at what these changes would look for the zope.event
package like on a branch in Subversion:

  http://svn.zope.org/zope.event/branches/tseaver-new_style_docs

I select zope.event largely because it is small enough that the scope of
the structural changes wouldn't be lost in the diffs for the actual
tests (and also because I didn't want to do the work for a bigger
package before getting feedback).

The rendered docs from that branch are here:

  http://palladion.com/static/zope.event-docs/

I pushed the bzr branch to Launchpad, to let you see the kinds of
changes I made to get there (our viewcvs story is not nearly as easy to
review as the Launchpad changeset view):

  https://code.launchpad.net/~tseaver/zope.event/new_style_docs

I am attaching an annotated diff for those who would prefer to see the
changes in that format.

I would say that this cleanup effort should start at the "bottom" layers
of the ZTK, dependency-wise (zope.interface, zope.component, etc.) and
move "upwards" (in the depenency sense) over time.

Thoughts?



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkvJEakACgkQ+gerLs4ltQ6X/wCgueT8RD10bKrM4OInF0sIj/Tk
rs4AnAiQlBWZoHd35ti0tPlae/JsVgvC
=K8N0
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: annotated.patch
Type: text/x-patch
Size: 31858 bytes
Desc: not available
Url : http://mail.zope.org/pipermail/zope-dev/attachments/20100416/c89de8ff/attachment-0001.bin