[Grok-dev] Getting started with grok testing

Uli Fouquet uli at gnufix.de
Sun Jul 4 06:36:51 EDT 2010


Hi Achim,

Achim Domma wrote:

> I want to learn more about the grok testing infrastructure.
>  http://grok.zope.org/documentation/how-to/tests-with-grok-testing is
>  marked as outdated. I'm reading
>  https://www.packtpub.com/grok-1-0-web-development/book?utm_source=grok.zope.org&utm_medium=link&utm_content=pod&utm_campaign=mdb_002592,
>  which is a great book, but I already encountered some differences
>  between 1.0 and 1.1, so the information about testing might also be
>  not up to date. So I don't know how to get started.

You ask about testing with grok. This is a wiiiide topic. Nearly as wide
as testing with Python in general. Grok itself does not provide much
infrastructure beside from the two functions in 'grok.testing' which are
supposed to be used during tests:

grok.testing.grok('<pkg_name>')

  'groks' a package during test runs. I.e. it looks up the given 
  package or module and executes all configuration actions (like 
  registering adapters, utilities, views, etc.) defined in the module.

  This is useful in tests, where you do not like to run a fully blown
  zope.app.testing.functional (see below) setup or where you want to
  grok complete fixtures defined in a testing environment.

grok.testing.grok_component(component, name_of_component)

  'groks' a component during test runs.

  Sometimes you want to test components defined in the test itself only:

    >>> import grok
    >>> class MyView(grok.View):
    ...   grok.context(IMyContext)
    ...   def render(self):
    ...     return u'Hi!'

    >>> grok.testing.grok_component(MyView, 'MyView')
    True

  Only after grokking 'MyView' explicitly, this component is registered 
  correctly with the Zope Component Architechture (i.e. here the class
  is registered globally as a multi-adapter). Afterwards multi-adapter 
  lookups like

   zope.component.getMultiAdapter((my_context, request), name='myview')

  should succeed which would not be the case without grokking the 
  component.

This is all what grok itself provides.

> I guess the answer is: Look a existing tests! I'm willing to do so, but
>  a good starting point would be very helpful. Any advice?

If you're willing to look at the tests, this could be very helpful.
It's, however, even then difficult to tell which tests in Zope
repository are the most advanced ones or best for beginners with grok.

If you don't mind, I'd tell about some differentiations we often make,
when it comes to that topic. You then have to decide yourself, what
approach would suit your needs and personal preferences best.

'Functional testing'/'unit testing'
-----------------------------------

The most Grok- or Zope-specific part in testing is certainly, what is
often called 'functional testing'. I.e. testing things, where generally
some fully blown 'Zope-Server' is simulated and in tests you can send
virtual requests to this virtual server using zope.testbrowser or
similar.

Some packages in the Zope repos do this type of testing via
zope.app.testing which provides methods and classes to setup the
'virtual server'. You can have a look into the `ftests/` subdir of grok
itself:

http://svn.zope.org/grok/trunk/src/grok/ftests/

to see how that works. The setup is done in
`ftests/test_grok_functional.py` module and all the subdirs in `ftests/`
provide doctests that partially make use of zope.testbrowser, for
instance:

http://svn.zope.org/grok/trunk/src/grok/ftests/viewlet/viewlet_security.py?view=markup

The term 'functional testing' for this kind of tests is not necessarily
completely correct, as it in Zope-context often refers to _how_ things
are tested instead of _what_ is tested. So it's more about how to setup
a test and not about what is tested.

The counterpart to this kind of 'functional testing' are more plain
'unittests', that test merely single components of a package or module.
You can have a look in the `tests/` subpackage of grok itself to see
this kind of tests in action. I also use the term 'unittests' here in
quotation marks, as the differentiation 'functional testing'/'unit
testing' in that context refers merely to how tests are setup and not to
what is tested.

These latter kind of 'unittests' (I think we need a better term for
that) normally do not use 'zope.app.testing' but sometimes
'zope.testing' components and, especially in recent times, only core
testing functionality from the Python standard lib (i.e. 'unittest'
and/or 'doctest' module).

Getting rid of zope.app.testing
-------------------------------

With recent efforts to make the whole Zope stuff more Pythonic, there
was also work done to get rid of as many 'zope.app.*' packages as
possible, including development of replacements for zope.app.testing.

Generally, zope.app.testing setups are quite heavy. They provide much
power as you have a complete virtual server running, but they also have
lots of things to do for each test setup, not necessarily needed or even
wanted in tests.

That led to development of more lightweight and faster tests based on
'zope.app.wsgi'. I think Vincent Fretin and Martijn Faassen did most of
that work yet and results are already quite usable.

I personally think, that this is the way to go in future and we're
already using this different kind of setup in:

http://svn.zope.org/grokcore.view/trunk/src/grokcore/view/ftests/

and other packages. Here you can see, that a 'functional test' setup (in
the sense described above) can be done mainly using 

  from zope.app.wsgi.testlayer import BrowserLayer, http

in test setup and then using a special testbrowser like this:

  from zope.app.wsgi.testlayer import Browser

There are also efforts to get rid of 'zope.testing' and to use only core
'unittest'/'doctest' modules which will make it easier to integrate
other testing tools/environments etc. from other parts of the Python
community.

Using zope.testrunner
---------------------

Beside setting up tests, Zope provides a special package for running
tests: `zope.testrunner`. It's a quite powerful tool normally deployed
in the buildout.cfg file of a project.

The zope testrunner normally looks for an object called `test_suite` in
certain modules of your project, collects the unittest.TestSuite
returned by it, runs this suite and returns the results in a
human-readable form.

As you can see from the description the zope.testrunner accepts any kind
of tests, i.e. it does not care wether something is a 'functional' or
whatever test. It only has to be a unittest.TestSuite in the end.

The testrunner for a project can be configured in the buildout.cfg like
this::

  [test]
  recipe = zc.recipe.testrunner
  eggs = grokcore.view
         grokcore.view[test]
  defaults = ['--tests-pattern', '^f?tests$', '-v']

(taken from
http://svn.zope.org/grokcore.view/trunk/buildout.cfg?view=auto) which
means:

- Use the 'zc.recipe.testrunner' recipe to setup a testrunner for 
  this project (i.e. create an executable script 'test' in bin/)
  
- When setting up the testrunner make available (set in sys.path):

  - The package `grokcore.view`

  - All packages required in 'test'-requirements of setup.py

- Set the following defaults for the created testrunner:

  - Look for test suites in modules called 'ftests' or 'tests'

  - Be verbose when runnung the testrunner

In a created grokproject you can run ./bin/test -h to see all the
options of a zope testrunner (there are tons of it).

You can, however, also use other testrunners from the Python world.


Oh, and BTW: I think the examples from the grok.zope.org HOWTO should
generally still work with grok 1.1.1. You should only replace::

  test_suite = grok.testing.register_all_tests('sample')

with

  import z3c.testsetup
  test_suite = z3c.testsetup.register_all_tests('sample')

With grok 1.1 it might be sufficient to add 'zope.app.testing' as
requirement in setup.py and rerun buildout to make it work). Hm, I think
I have to give this HOWTO a quick review.

I only doubt, that the approach described there (deploying
z3c.testsetup) will stay the same in future. You might prefer the
approaches chosen in the grok and grokcore.view packages as linked
above. Although they are more complicated, they are also more Pythonic
and thus better to understand.

Sorry for the long text. It most probably still lacks important pieces
but maybe it helps at least to get you starting. Please don't hesitate
to ask! I'm also currently preparing some texts in German language about
that topic. So, if you're interested in them, please tell.

Best regards,

-- 
Uli

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Dies ist ein digital signierter Nachrichtenteil
Url : http://mail.zope.org/pipermail/grok-dev/attachments/20100704/1525eeac/attachment.bin 


More information about the Grok-dev mailing list