[Zope3-dev] Performance Testing

[Benji York]

Tarek has started some very interesting work on adding performance 
testing to the Zope 3 testing infrastructure and it so happens that Jim 
and I were discussing something very similar last week, so I'd like to 
suggest some functionality we might want to have (which I should be able 
to help implement).

1) warn about regressions: the test runner will keep per-test, 
machine-independent records of how long tests take and will report 
regressions larger than a predefined percentage.  These records will be 
checked in so that if someone else makes changes (in a fresh checkout) 
that causes a particular test to slow down drastically, they will be warned.

2) testbrowser should keep up with a (machine-independent) metric of how 
long the previous request took so performance assertions can be made 
inside tests.  E.g.

     >>> browser.open('http://localhost/foo')
     >>> browser.last_request_time < 0.5

3) the functional testing framework should be extended to allow the 
collection of total time (again, machine-independent) per request and 
the test runner should have an option to display the top "n" slowest 
requests.

Comments?
-- 
Benji York
Senior Software Engineer
Zope Corporation