[Checkins] SVN: Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/test Add checker tests.

[Uli Fouquet]

Log message for revision 83830:
  Add checker tests.

Changed:
  U   Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/testrunner.txt
  A   Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/checkertest.chk
  A   Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/samplesetup_short5.py
  A   Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/samplesetup_short6.py

-=-
Modified: Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/testrunner.txt
===================================================================
--- Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/testrunner.txt	2008-02-14 13:14:30 UTC (rev 83829)
+++ Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/testrunner.txt	2008-02-14 13:15:32 UTC (rev 83830)
@@ -192,6 +192,93 @@
 Apparently now the custom ZCML file in the ``cave`` package was used.
 
 
+Setting output checker for functional tests
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Output checkers modify the way, output of tests is recognized. This is
+important for output which can not be foreseen exactly, timestamps for
+example, or local file paths. In this case sometimes a regular
+expression would match every single expression, but how can I tell the
+testrunner, that all timestamps of the form 'N.NNN seconds' are
+acceptable? Easy: use a checker::
+
+    >>> setupfile = os.path.join(cavepath, 'samplesetup_short5.py')
+    >>> print open(setupfile).read()
+    import re
+    from zope.testing import renormalizing
+    import z3c.testsetup
+    mychecker = renormalizing.RENormalizing([
+        (re.compile('[0-9]*[.][0-9]* seconds'),
+         '<SOME NUMBER OF> seconds'),
+        (re.compile('at 0x[0-9a-f]+'), 'at <SOME ADDRESS>'),
+        ])
+    test_suite = z3c.testsetup.register_all_tests(
+        'z3c.testsetup.tests.cave',
+        checker = mychecker,
+        extensions = ['.chk',],
+        fregexp_list = ['.*checker.*',],
+        )
+
+This setup will find exactly one testfile, the file
+``checkertest.chk`` in the ``cave`` package, that checks for output of
+the form 'N.NNN seconds', with an arbitrary number of numbers.
+
+Running the testrunner with this setup will result in::
+
+    >>> defaults = [
+    ...     '--path', cavepath,
+    ...     '--tests-pattern', '^samplesetup_short5$',
+    ...     ]
+    >>> sys.argv = 'test -f '.split()
+    >>> testrunner.run(defaults)
+    Running z3c.testsetup.doctesting.FunctionalLayer tests:
+      Set up z3c.testsetup.doctesting.FunctionalLayer in ... seconds.
+      Ran 2 tests with 0 failures and 0 errors in ... seconds.
+    Tearing down left over layers:
+      Tear down z3c.testsetup.doctesting.FunctionalLayer ... not supported
+    False
+
+The same setup, but without a modified checker will deliver::
+
+    >>> defaults = [
+    ...     '--path', cavepath,
+    ...     '--tests-pattern', '^samplesetup_short6$',
+    ...     ]
+    >>> sys.argv = 'test -f '.split()
+    >>> testrunner.run(defaults)
+    Running z3c.testsetup.doctesting.FunctionalLayer tests:
+      Set up z3c.testsetup.doctesting.FunctionalLayer in ... seconds.
+    <BLANKLINE>
+    <BLANKLINE>
+    Failure in test ...checkertest.chk
+    Failed doctest test for checkertest.chk
+      File "...checkertest.chk", line 0
+    <BLANKLINE>
+    ----------------------------------------------------------------------
+    File "...checkertest.chk", line 10, in checkertest.chk
+    Failed example:
+        print "%s seconds" % 0.123
+    Differences (ndiff with -expected +actual):
+        - <SOME NUMBER OF> seconds
+        + 0.123 seconds
+    ----------------------------------------------------------------------
+    File "...checkertest.chk", line 15, in checkertest.chk
+    Failed example:
+        print "A memory address at 0x1a0322ff"
+    Differences (ndiff with -expected +actual):
+        - A memory address at <SOME ADDRESS>
+        + A memory address at 0x1a0322ff
+    <BLANKLINE>
+      Ran 2 tests with 1 failures and 0 errors in ... seconds.
+    Tearing down left over layers:
+      Tear down z3c.testsetup.doctesting.FunctionalLayer ... not supported
+    True
+
+Note that checkers are currently only supported for functional
+doctests.
+
+
+
 Extended setups
 ---------------
 

Added: Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/checkertest.chk
===================================================================
--- Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/checkertest.chk	                        (rev 0)
+++ Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/checkertest.chk	2008-02-14 13:15:32 UTC (rev 83830)
@@ -0,0 +1,18 @@
+============
+Checker test
+============
+
+:Test-Layer: checker
+
+First we check, whether the <SOME NUMBER OF> term is matched by the
+modified checker::
+
+   >>> print "%s seconds" % 0.123
+   <SOME NUMBER OF> seconds
+
+Then we check the <SOME ADDRESS> term::
+
+   >>> print "A memory address at 0x1a0322ff"
+   A memory address at <SOME ADDRESS>
+
+That's it.

Added: Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/samplesetup_short5.py
===================================================================
--- Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/samplesetup_short5.py	                        (rev 0)
+++ Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/samplesetup_short5.py	2008-02-14 13:15:32 UTC (rev 83830)
@@ -0,0 +1,14 @@
+import re
+from zope.testing import renormalizing
+import z3c.testsetup
+mychecker = renormalizing.RENormalizing([
+    (re.compile('[0-9]*[.][0-9]* seconds'), 
+     '<SOME NUMBER OF> seconds'),
+    (re.compile('at 0x[0-9a-f]+'), 'at <SOME ADDRESS>'),
+    ])
+test_suite = z3c.testsetup.register_all_tests(
+    'z3c.testsetup.tests.cave',
+    checker = mychecker,
+    extensions = ['.chk',],
+    fregexp_list = ['.*checker.*',],
+    )

Added: Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/samplesetup_short6.py
===================================================================
--- Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/samplesetup_short6.py	                        (rev 0)
+++ Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/tests/cave/samplesetup_short6.py	2008-02-14 13:15:32 UTC (rev 83830)
@@ -0,0 +1,8 @@
+import re
+from zope.testing import renormalizing
+import z3c.testsetup
+test_suite = z3c.testsetup.register_all_tests(
+    'z3c.testsetup.tests.cave',
+    extensions = ['.chk',],
+    fregexp_list = ['.*checker.*',],
+    )