[Checkins]
SVN: z3c.testsetup/branches/ulif-sepfunctesting/src/z3c/testsetup/functional/functionaldoctestsetup.txt
Move test to new location.
Uli Fouquet
uli at gnufix.de
Tue Jun 24 17:37:36 EDT 2008
Log message for revision 87719:
Move test to new location.
Changed:
A z3c.testsetup/branches/ulif-sepfunctesting/src/z3c/testsetup/functional/functionaldoctestsetup.txt
-=-
Copied: z3c.testsetup/branches/ulif-sepfunctesting/src/z3c/testsetup/functional/functionaldoctestsetup.txt (from rev 87718, z3c.testsetup/branches/ulif-sepfunctesting/src/z3c/testsetup/functionaldoctestsetup.txt)
===================================================================
--- z3c.testsetup/branches/ulif-sepfunctesting/src/z3c/testsetup/functional/functionaldoctestsetup.txt (rev 0)
+++ z3c.testsetup/branches/ulif-sepfunctesting/src/z3c/testsetup/functional/functionaldoctestsetup.txt 2008-06-24 21:37:35 UTC (rev 87719)
@@ -0,0 +1,485 @@
+##############################################################################
+#
+# Copyright (c) 2008 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""
+=========================
+Functional Doc Test Setup
+=========================
+
+``FunctionalDocTestSetup`` helps to find and setup functional doctests
+contained in a package. The most important method therefore might be
+``getTestSuite()``, which searches a given package for doctest files
+and returns all tests found as a suite of functional doctests.
+
+Functional doctest setups find and register only doctests. Those tests
+can also be Python modules but if you defined real `unttest.TestCase`
+classes in your tests, then you most likely have 'normal' Python unit
+tests. Please see 'pythontestsetup.txt' in this case. For simple unit
+doctests, that do not require a more or less complex framework setup
+to be done for each test, the setups described in
+'unitdoctestsetup.txt' might suit your needs better.
+
+There are also real 'oneliners' possible, that wrap around the classes
+described here. See 'README.txt' to learn more about that.
+
+The work is done mainly in two stages:
+
+1) The package is searched for appropriate docfiles, based on the
+ settings of instance attributes.
+
+2) The tests contained in the found docfiles are setup as functional
+ tests and added to a ``unittest.TestSuite`` instance.
+
+There are plenty of default values active, if you use instances of
+this class without further modifications. Therefore we will first
+discuss the default behaviour and afterwards show, how you can modify
+this behaviour to suit your special expectations on the tests.
+
+
+Setting up a simple test suite
+------------------------------
+
+We want to register the doctests contained in the local ``cave``
+package. This can be simply archieved by doing::
+
+ >>> from z3c.testsetup import FunctionalDocTestSetup
+ >>> setup = FunctionalDocTestSetup('z3c.testsetup.tests.cave')
+ >>> setup
+ <z3c.testsetup.doctesting.FunctionalDocTestSetup object at 0x...>
+
+Apparently the package to handle was passed as a string in 'dotted
+name' notation. We could also pass the package itself, if it was
+loaded before::
+
+ >>> from z3c.testsetup.tests import cave
+ >>> setup = FunctionalDocTestSetup(cave)
+ >>> setup
+ <z3c.testsetup.doctesting.FunctionalDocTestSetup object at 0x...>
+
+This setup is ready for use::
+
+ >>> suite = setup.getTestSuite()
+ >>> suite
+ <unittest.TestSuite tests=[...]>
+
+To sum it up, writing a test setup for a project now can be that
+short::
+
+ import z3c.testsetup
+ def test_suite():
+ setup = z3c.testsetup.FunctionalDocTestSetup('z3c.testsetup.tests.cave')
+ return setup.getTestSuite()
+
+This will find all .rst and .txt files in the package that provide a
+certain signature (see below), register the contained tests as
+functional tests and run them as part of a `unittest.TestSuite`.
+
+Note: in many test setups you will find a code fragment like the
+ following at the end of file::
+
+ if __name__ == '__main__':
+ unittest.main(default='test_suite')
+
+ This is not neccessary for usual testrunner setups. A testrunner
+ will look for appropriate filenames (modules) and if those
+ modules provide a callable ``test_suite`` (usually a function)
+ this callable will be called to deliver a test suite.
+
+FunctionalDocTestSetup default values
+-------------------------------------
+
+Understanding the defaults is important, because the default values
+are driving the whole process of finding and registering the test.
+
+
+Which files are found by default?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Basically, all files are accepted that
+
+1) reside inside the package passed to the constructor. This includes
+ subdirectories.
+
+2) have a filename extension `.txt` or `.rst` (uppercase, lowercase
+ etc. does not matter).
+
+3) are *not* located inside a 'hidden' directory (i.e. a directory
+ starting with a dot ('.'). Also subdirectories of 'hidden'
+ directories are skipped.
+
+4) contain a ReStructured Text meta-marker somewhere, that defines the
+ file as a functional doctest explicitly::
+
+ :Test-Layer: functional
+
+ This means: there *must* be a line like the above one in the
+ doctest file. The term might be preceeded or followed by whitspace
+ characters (spaces, tabs).
+
+Only files, that meet all four conditions are searched for functional
+doctests. You can modify this behaviour of course, which will be
+explained below in detail.
+
+
+What options are set by default?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Many options can be set, when registering functional doctests. When
+using the default set of options, the following values are set::
+
+* The setup-instance's ``setUp``-method is set as the ``setUp``
+ function.
+
+* The setup-instance's ``tearDown``-method is set as the ``tearDown``
+ function.
+
+ >>> setup.setUp
+ <bound method FunctionalDocTestSetup.setUp of
+ <z3c.testsetup.doctesting.FunctionalDocTestSetup object at 0x...>>
+
+* The setup-instance's `globs` attribute is passed as the `globs`
+ parameter. By default `globs` is a dictionary of functions, that
+ should be'globally' available during testruns and it contains::
+
+ >>> setup.globs
+ {'http': <zope.app.testing.functional.HTTPCaller object at 0x...>,
+ 'sync': <function sync at 0x...>,
+ 'getRootFolder': <function getRootFolder at 0x...>}
+
+ The functions `sync` and `getRootFolder` are provided by
+ `zope.app.testing.functional`.
+
+* The setup-instance's `optionsflags` attribute is passed. It
+ includes by default the following doctest constants:
+
+ >>> from zope.testing import doctest
+ >>> setup.optionflags == (doctest.ELLIPSIS+
+ ... doctest.NORMALIZE_WHITESPACE |
+ ... doctest.REPORT_NDIFF)
+ True
+
+* The setup-instances `encoding` attribute is passed. Setting it in
+ the constructor will expect doctest files to provide the appropriate
+ encoding. By default it is set to 'utf-8':
+
+ >>> setup.encoding
+ 'utf-8'
+
+ You can set it to a another value for differently encoded doctests.
+ If no encoding is set (`encoding` is None), 7-bit ASCII will be
+ assumed.
+
+* The `checker` attribute helps to renormalize expected
+ output. A typical output checker can be created like this::
+
+ >>> import re
+ >>> from zope.testing import renormalizing
+ >>> mychecker = renormalizing.RENormalizing([
+ ... (re.compile('[0-9]*[.][0-9]* seconds'),
+ ... '<SOME NUMBER OF> seconds'),
+ ... (re.compile('at 0x[0-9a-f]+'), 'at <SOME ADDRESS>'),
+ ... ])
+
+ By default a `FunctionalDocTest` instance has no checker::
+
+ >>> setup.checker is None
+ True
+
+Because functional tests require a ZCML layer, that defines a ZCML
+setup for the tests, we provide a layer, that is driven by the file
+`ftesting.zcml`, which comes with `z3c.testsetup`. The layer is
+accessible as the setup instance attribute `layer`::
+
+ >>> setup.layer
+ <zope.app.testing.functional.ZCMLLayer instance at 0x...>
+
+ >>> setup.layer.config_file
+ '...ftesting.zcml'
+
+You can define a custom layer. This is described below.
+
+
+No other options/parameters are set by default.
+
+
+Customizing functional test setup:
+----------------------------------
+
+You can modify the behaviour of z3c.testsetup.FunctionalTestSetup such,
+that a different set of files is registered and/or the found tests are
+registered with a different set of parameters/options. We will first
+discuss modifying the set of files to be searched.
+
+
+Customizing the doctest file search:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The searching of appropriate doctest files is basically done by the
+base class `BasicTestSetup`. Its purpose is to determine the set of
+files in a package, that contain functional tests. See the testfile
+`basicsetup.txt` to learn more about the procedure.
+
+The functional test setup, however, provides a special
+`isDocTestFile()` method, which does additional checking. Namely it
+checks for the existance of the above mentioned ReStructured Text
+meta-marker::
+
+ `:Test-Layer: functional`
+
+This is determined by a list of regular expressions, which is also
+available as an object attribute::
+
+ >>> setup.regexp_list
+ ['^\\s*:(T|t)est-(L|l)ayer:\\s*(functional)\\s*']
+
+This is the default value of functional doctest setups.
+
+There are two files in the `cave` subpackage, which include that
+marker. We can get the list of test files using
+`getDocTestFiles()``::
+
+ >>> testfile_list = setup.getDocTestFiles()
+ >>> testfile_list.sort()
+ >>> testfile_list
+ ['...file1.txt', '...subdirfile.txt']
+
+ >>> len(testfile_list)
+ 2
+
+The ``isTestFile()`` method of our setup object did the filtering
+here::
+
+ >>> setup.isTestFile(testfile_list[0])
+ True
+
+The file file1.rst does not contain a functional test marker::
+
+ >>> import os.path
+ >>> path = os.path.join(os.path.dirname(cave.__file__),
+ ... 'test1.rst')
+ >>> setup.isTestFile(path)
+ False
+
+The `regexp_list` attribute of a ``FunctionalTestSetup`` contains a
+list of regular expressions, of which each one must at least match one
+line of a searched file to be accepted. If you want to include files
+with different marker-strings, just change this attribute. The value
+will influence behaviour of the `isTestFile()``, ``getDocTestFiles()``
+and ``getTestSuite()`` methods.
+
+If you need more complex checks here, you can derive your customized
+test setup class and overwrite ``isTestFile()``.
+
+See `basicsetup.py` for further methods how to modify test file
+search, for example by choosing another set of accepted filename
+extensions.
+
+
+Customizing the functional doctest setup
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To customize the setup of your tests, you have three options:
+
+- Pass appropriate parameters to the constructor.
+
+- Set attributes of an existing `FunctionalDocTestSetup` instance.
+
+- Create your own class derived from `FunctionalDocTestSetup`.
+
+The first two ways should suit most testing environments. All the
+attributes mentioned above are settable at creation time, namely:
+
+- `setup`::
+
+ >>> def myfunc(test):
+ ... """A useless function."""
+ ... print "Hello!"
+ >>> mysetup = FunctionalDocTestSetup(cave, setup=myfunc)
+ >>> mysetup.setUp(None)
+ Hello!
+
+- `teardown`::
+
+ >>> mysetup = FunctionalDocTestSetup(cave, teardown=myfunc)
+ >>> mysetup.tearDown(None)
+ Hello!
+
+- `globs`:
+
+ `globs` is a dictionary of things (objects, functions, vars) that
+ are available for every test immediately (without import or
+ similar)::
+
+ >>> mysetup = FunctionalDocTestSetup(
+ ... cave, globs={'myfunc': myfunc})
+ >>> mysetup.globs
+ {'myfunc': <function myfunc at 0x...>}
+
+- `optionflags`
+
+ See the `zope.testing.doctest` module for all optionflags::
+
+ >>> from zope.testing import doctest
+ >>> mysetup = FunctionalDocTestSetup(
+ ... cave, optionflags=(doctest.ELLIPSIS +
+ ... doctest.REPORT_UDIFF))
+ >>> mysetup.optionflags & doctest.REPORT_NDIFF == 0
+ True
+
+ >>> mysetup.optionflags & doctest.REPORT_UDIFF == doctest.REPORT_UDIFF
+ True
+
+- `checker`
+
+ An output checker for functional doctests. `None` by default. A
+ typical output checker can be created like this::
+
+ >>> import re
+ >>> from zope.testing import renormalizing
+ >>> mychecker = renormalizing.RENormalizing([
+ ... (re.compile('[0-9]*[.][0-9]* seconds'),
+ ... '<SOME NUMBER OF> seconds'),
+ ... (re.compile('at 0x[0-9a-f]+'), 'at <SOME ADDRESS>') ])
+
+ Then, a setup with this checker can be created::
+
+ >>> mysetup = FunctionalDocTestSetup(cave, checker = mychecker)
+ >>> mysetup.checker
+ <zope.testing.renormalizing.RENormalizing instance at 0x...>
+
+ Let's see, whether we got the wanted checker, by passing an
+ example string, which should match the first of the terms defined
+ in the checker::
+
+ >>> mysetup.checker.check_output(
+ ... '''\
+ ... Test took 0.012 seconds
+ ... ''', '''\
+ ... Test took <SOME NUMBER OF> seconds
+ ... ''', 0)
+ True
+
+ See the `zope.testing.renormalizing` module for more things, you
+ can do with checkers.
+
+- `encoding`
+
+ If your doctests contain non-ASCII characters, this might lead to
+ problems. You can circumvent this by setting an appropriate
+ encoding string in the header of your doctest files. Another
+ possibility is to pass the encoding keyword. By default
+ z3c.testsetup uses 'utf-8' as default encoding::
+
+ >>> setup.encoding
+ 'utf-8'
+
+ But you can set it as you like::
+
+ >>> mysetup = FunctionalDocTestSetup(cave, encoding = 'ascii')
+ >>> mysetup.encoding
+ 'ascii'
+
+To setup layers, there are the following constructor options
+available:
+
+- `zcml_config`
+
+ The path to a ZCML file, often named `ftesting.zcml`. If a
+ package, provides a file `ftesting.zcml` in its root, then this is
+ taken as default. The ``cave_to_let`` package in the tests/
+ directory provides such an `ftesting.zcml`::
+
+ >>> from z3c.testsetup.tests import cave_to_let
+ >>> setup = FunctionalDocTestSetup(cave_to_let)
+ >>> pnorm(setup.layer.config_file)
+ '.../tests/cave_to_let/ftesting.zcml'
+
+ The fallback-solution is to take the layer from the
+ `z3c.testsetup` package::
+
+ >>> setup = FunctionalDocTestSetup(cave)
+ >>> pnorm(setup.layer.config_file)
+ '...z3c/testsetup/ftesting.zcml'
+
+ Now the fallback `ftesting.zcml` was taken, because the cave got
+ no own ftesting.zcml.
+
+- `layer_name`
+
+ A string. The default is ``FunctionalLayer``.
+
+- `layer`
+
+ A ``zope.app.testing.functional.ZCMLLayer`` object. Setting a
+ layer overrides `zcml_config` and `layer_name`.
+
+Their usage is explained in the next section.
+
+
+Cutomizing the ZCML layer
++++++++++++++++++++++++++
+
+The ZCML layer of a FunctionalDocTestSetup is searched in four steps:
+
+1) If a setup is called with `layer` parameter, take this.
+
+2) If a setup is called with `zcml_config` paramter, take this.
+
+3) If an `ftesting.zcml` can be found in the root of the package to
+ search, take this (default).
+
+4) Take the (very poor) ftesting.zcml of the z3c.testsetup package
+ (fallback).
+
+The ZCML layer registered as fallback by ``z3c.testsetup`` is very
+poor. In fact it only exists, to satisfy dependencies. In most cases,
+you would like to write your own ZCML configuration to register
+principals etc. during functional doctests.
+
+For this purpose, ``FunctionalDocTestSetup`` supports the constructor
+parameters `zcml_config` and `layer_name`. If the first is set, then
+the ZCML file denoted by the path in this variable will be used
+instead of the dummy ZCML contained in ``z3c.testsetup``.
+
+ >>> setup_w_custom_layer = FunctionalDocTestSetup(
+ ... cave,
+ ... zcml_config = 'sampleftesting.zcml')
+ >>> pnorm(setup_w_custom_layer.layer.config_file)
+ '.../tests/cave/sampleftesting.zcml'
+
+You can also pass a keyword parameter `layer`, which should provide a
+value with a ready-to-use ZCML layer. If this happens, the
+`zcml_config` and `layer_name` parameter will have no effect.
+
+To show this, we first create a custom layer::
+
+ >>> from zope.app.testing.functional import ZCMLLayer
+ >>> mylayer = ZCMLLayer(
+ ... os.path.join(os.path.dirname(__file__), 'ftesting.zcml'),
+ ... __name__,
+ ... 'MyFunctionalLayer')
+
+and create a functional doctest setup with it::
+
+ >>> setup_w_custom_layer = FunctionalDocTestSetup(
+ ... cave,
+ ... zcml_config = 'sampleftesting.zcml',
+ ... layer = mylayer)
+ >>> pnorm(setup_w_custom_layer.layer.config_file)
+ '.../testsetup/ftesting.zcml'
+
+As we can see, the `mylayer` config file is registered and the
+`zcml_config` parameter was skipped.
+
+
+"""
More information about the Checkins
mailing list