[Zope3-dev] testrunner layers

[Jim Fulton]

Florent Guillaume wrote:
> I didn't find the info I was looking for in the unit tests and docs  for 
> the testrunner:
> 
> How are layers defined?

This is definately a hole in the testrunner docs.

 > Are they just a class with a setUp and  tearDown
> classmethod?
 >
 > Is it enough to provide a layer attribute in the layer class?

A layer attribute is not part of the api for layers.

 > Is  there
 > a use of a 'base' attribute or is it just the unit tests for  the
 > testrunners that have it (they are full of "# Internal to  samples. Not
 > part of layer API" so it's hard to know).

Layers are objects with setUp and tearDown methods and
__bases__, __module__, and __name__ attributes.  Classes
with class methods are a common and convenient
implementation approach.

In addition, a layer's tearDown method is allowed to raise
NotImplementedError to indicate that the layer doesn't support
tear down. This is actually documented in testrunner-layers-ntd.txt.

It is also a requirement that layers must be available for import
using their __module__ and __name__.

The __bases__ attribute indicates that a layer builds on other layers.
In that case, the base layers will be setUp before the layer and torn
down after the layer (assuming that the layer and base layers support
tear down.)

> How does a test class reference a layer? Just by setting the layer  
> attribute to the dotted name of the class? Or to the class itself?

A test or test suite indicates it's layer through it's layer attribute,
which must be either the dotted name of a layer or the layer itself.


> Along what algorithm does the --layer directive match layers? Is it  an 
> exact match, or a substring? A prefix? How does the base come into  play?

A regular expression match is used. This is documented in the
testrunner-layers.txt file, although that documentation doesn't make
clear whether a regular expression or substring match is used.
The options help does say that this is a regular expression.

> Also, what's the intent behind no-tearDown layers?

To support layers that can't be torn down. :)

Most notably, changes made by ZCML can't be torn down.
See the layer defined in zope.app.functional for an example
of a ZCML-defined layer.

Contributions to the documentation would be welcome. In
particular:

- The command line help should provide a pointer to the
   text files. A lot of people haven't realized that there
   was extensive documentation for the test runner.
   (We should also apidoc bookify these files.)

- We need a chapter on writing layers.

- It would be good to clarify, where necessary
   in the text when regular expressions are used.

I won't have time to do this myself this year.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org