[Checkins]
SVN: Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/test
Add checker tests.
Uli Fouquet
uli at gnufix.de
Thu Feb 14 08:15:33 EST 2008
Log message for revision 83830:
Add checker tests.
Changed:
U Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/testrunner.txt
A Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/checkertest.chk
A Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/samplesetup_short5.py
A Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/samplesetup_short6.py
-=-
Modified: Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/testrunner.txt
===================================================================
--- Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/testrunner.txt 2008-02-14 13:14:30 UTC (rev 83829)
+++ Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/testrunner.txt 2008-02-14 13:15:32 UTC (rev 83830)
@@ -192,6 +192,93 @@
Apparently now the custom ZCML file in the ``cave`` package was used.
+Setting output checker for functional tests
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Output checkers modify the way, output of tests is recognized. This is
+important for output which can not be foreseen exactly, timestamps for
+example, or local file paths. In this case sometimes a regular
+expression would match every single expression, but how can I tell the
+testrunner, that all timestamps of the form 'N.NNN seconds' are
+acceptable? Easy: use a checker::
+
+ >>> setupfile = os.path.join(cavepath, 'samplesetup_short5.py')
+ >>> print open(setupfile).read()
+ import re
+ from zope.testing import renormalizing
+ import z3c.testsetup
+ mychecker = renormalizing.RENormalizing([
+ (re.compile('[0-9]*[.][0-9]* seconds'),
+ '<SOME NUMBER OF> seconds'),
+ (re.compile('at 0x[0-9a-f]+'), 'at <SOME ADDRESS>'),
+ ])
+ test_suite = z3c.testsetup.register_all_tests(
+ 'z3c.testsetup.tests.cave',
+ checker = mychecker,
+ extensions = ['.chk',],
+ fregexp_list = ['.*checker.*',],
+ )
+
+This setup will find exactly one testfile, the file
+``checkertest.chk`` in the ``cave`` package, that checks for output of
+the form 'N.NNN seconds', with an arbitrary number of numbers.
+
+Running the testrunner with this setup will result in::
+
+ >>> defaults = [
+ ... '--path', cavepath,
+ ... '--tests-pattern', '^samplesetup_short5$',
+ ... ]
+ >>> sys.argv = 'test -f '.split()
+ >>> testrunner.run(defaults)
+ Running z3c.testsetup.doctesting.FunctionalLayer tests:
+ Set up z3c.testsetup.doctesting.FunctionalLayer in ... seconds.
+ Ran 2 tests with 0 failures and 0 errors in ... seconds.
+ Tearing down left over layers:
+ Tear down z3c.testsetup.doctesting.FunctionalLayer ... not supported
+ False
+
+The same setup, but without a modified checker will deliver::
+
+ >>> defaults = [
+ ... '--path', cavepath,
+ ... '--tests-pattern', '^samplesetup_short6$',
+ ... ]
+ >>> sys.argv = 'test -f '.split()
+ >>> testrunner.run(defaults)
+ Running z3c.testsetup.doctesting.FunctionalLayer tests:
+ Set up z3c.testsetup.doctesting.FunctionalLayer in ... seconds.
+ <BLANKLINE>
+ <BLANKLINE>
+ Failure in test ...checkertest.chk
+ Failed doctest test for checkertest.chk
+ File "...checkertest.chk", line 0
+ <BLANKLINE>
+ ----------------------------------------------------------------------
+ File "...checkertest.chk", line 10, in checkertest.chk
+ Failed example:
+ print "%s seconds" % 0.123
+ Differences (ndiff with -expected +actual):
+ - <SOME NUMBER OF> seconds
+ + 0.123 seconds
+ ----------------------------------------------------------------------
+ File "...checkertest.chk", line 15, in checkertest.chk
+ Failed example:
+ print "A memory address at 0x1a0322ff"
+ Differences (ndiff with -expected +actual):
+ - A memory address at <SOME ADDRESS>
+ + A memory address at 0x1a0322ff
+ <BLANKLINE>
+ Ran 2 tests with 1 failures and 0 errors in ... seconds.
+ Tearing down left over layers:
+ Tear down z3c.testsetup.doctesting.FunctionalLayer ... not supported
+ True
+
+Note that checkers are currently only supported for functional
+doctests.
+
+
+
Extended setups
---------------
Added: Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/checkertest.chk
===================================================================
--- Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/checkertest.chk (rev 0)
+++ Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/checkertest.chk 2008-02-14 13:15:32 UTC (rev 83830)
@@ -0,0 +1,18 @@
+============
+Checker test
+============
+
+:Test-Layer: checker
+
+First we check, whether the <SOME NUMBER OF> term is matched by the
+modified checker::
+
+ >>> print "%s seconds" % 0.123
+ <SOME NUMBER OF> seconds
+
+Then we check the <SOME ADDRESS> term::
+
+ >>> print "A memory address at 0x1a0322ff"
+ A memory address at <SOME ADDRESS>
+
+That's it.
Added: Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/samplesetup_short5.py
===================================================================
--- Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/samplesetup_short5.py (rev 0)
+++ Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/samplesetup_short5.py 2008-02-14 13:15:32 UTC (rev 83830)
@@ -0,0 +1,14 @@
+import re
+from zope.testing import renormalizing
+import z3c.testsetup
+mychecker = renormalizing.RENormalizing([
+ (re.compile('[0-9]*[.][0-9]* seconds'),
+ '<SOME NUMBER OF> seconds'),
+ (re.compile('at 0x[0-9a-f]+'), 'at <SOME ADDRESS>'),
+ ])
+test_suite = z3c.testsetup.register_all_tests(
+ 'z3c.testsetup.tests.cave',
+ checker = mychecker,
+ extensions = ['.chk',],
+ fregexp_list = ['.*checker.*',],
+ )
Added: Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/samplesetup_short6.py
===================================================================
--- Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/samplesetup_short6.py (rev 0)
+++ Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/samplesetup_short6.py 2008-02-14 13:15:32 UTC (rev 83830)
@@ -0,0 +1,8 @@
+import re
+from zope.testing import renormalizing
+import z3c.testsetup
+test_suite = z3c.testsetup.register_all_tests(
+ 'z3c.testsetup.tests.cave',
+ extensions = ['.chk',],
+ fregexp_list = ['.*checker.*',],
+ )
More information about the Checkins
mailing list