[Zope3-dev] Doctests and coverage reports

[Marius Gedminas]

Hi,

The Zope 3 test runner has an option (-T or --trace) to produce coverage
reports.  It breaks when doctests are used.  Here are the reasons and
workarounds:

 1. trace.py in the standard library does not grok file names of the form
    "<doctest ...>" and aborts with an assertion error.

    Workaround: copy trace.py from the standard library into Zope3/src
    and apply the following one-liner patch:

--- /usr/lib/python2.3/trace.py 2004-12-03 15:12:09.000000000 +0200
+++ src/trace.py    2004-12-29 21:27:21.000000000 +0200
@@ -245,7 +245,7 @@

         for filename, count in per_file.iteritems():
             # skip some "files" we don't care about...
-            if filename == "<string>":
+            if filename.startswith("<"): # e.g., "<string>" or "<doctest ...>"
                 continue

             if filename.endswith(".pyc") or filename.endswith(".pyo"):

    (I have experienced this with Python 2.3.4, and I've noticed that
    Python 2.4 does not have that assert statement, however I have not
    tried running test.py -T with Python 2.4.)

 2. zope.testing.DocTestRunner has some code to support pdb.set_trace()
    within a doctest.  Among other things, DocTestRunner.__run calls
    self.debugger.set_continue() which, in turn, calls
    sys.settrace(None) and therefore stops tracing that is used to
    generate coverage reports.

    I have committed the fix for this problem into Zope 3 trunk as revision
    28704 (plus a brown paper bag bug fix in revision 28705).

I am not sure how to solve problem 1.  One possible solution is to add
a fixed version of trace.py into the Zope 3 source repository and use it
rather than the standard library version.  Another solution is to change
DocTestRunner and replace file names like "<doctest ...>" with either
"<string>" (that trace.py recognized and to ignores) or just use the
real file name there.

I've also noticed that if you use pdb.set_trace() in a doctest and then
type 'list' in the pdb session, you will not see the source.  Maybe it
is possible to solve both problems by using the real source file name
and perhaps adding a bunch of empty lines to get matching line numbers
as well?

Marius Gedminas
-- 
A "critic" is a man who creates nothing and thereby feels qualified to judge
the work of creative men. There is logic in this; he is unbiased -- he hates
all creative people equally.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mail.zope.org/pipermail/zope3-dev/attachments/20041229/f3f34823/attachment.bin