[Zope-Coders] summaries of automated test results

[Steve Alexander]

Hi folks,

I've written a script that I can run nightly to send a summary of zope 
tests run during the previous 24 hours. (Maybe I should set it to 24 and 
a bit hours just to make sure I don't miss anything.)

The data is scraped from the pipermail archives.


The output looks like this:

--------
Subject: OK: Test Zope 2 HEAD / Python 2.3 / Linux
From: jeremy at zope.com
Date: Wed Dec 10 05:11:21 EST 2003
URL: http://mail.zope.org/pipermail/zope-tests/2003-December/000001.html

Subject: Tests - HEAD/Py2.3 -  Not OK :-(
From: zope-tests at plope.com
Date: Wed Dec 10 05:12:22 EST 2003
URL: http://mail.zope.org/pipermail/zope-tests/2003-December/000005.html

Subject: Tests - 2_7-branch/Py2.3 -  errors:1
From: zope-tests at plope.com
Date: Wed Dec 10 05:12:22 EST 2003
URL: http://mail.zope.org/pipermail/zope-tests/2003-December/000006.html

Subject: Tests - 2_6-branch/Py21 -  failures:1
From: zope-tests at plope.com
Date: Wed Dec 10 05:12:22 EST 2003
URL: http://mail.zope.org/pipermail/zope-tests/2003-December/000004.html

Subject: Tests - 2_6-branch/Py22 -  failures:2
From: zope-tests at plope.com
Date: Wed Dec 10 05:12:22 EST 2003
URL: http://mail.zope.org/pipermail/zope-tests/2003-December/000002.html

Subject: OK: Test ZODB HEAD / Python 2.3 / Linux
From: jeremy at zope.com
Date: Wed Dec 10 06:22:31 EST 2003
URL: http://mail.zope.org/pipermail/zope-tests/2003-December/000003.html
--------

I'd also like to say "all ok!" or to summarize the nature of any errors 
in the subject line of the summary email.

It would help me to do this if the subject lines of automated test 
emails all followed the same pattern.

The subjects need to describe what is being tested, for example:

   ZODB HEAD / Python 2.3 / Linux
   2_6-branch/Py22

They should also report on success or otherwise. This is the part I want 
to be able to extract easily. The message conveyed will say one of the 
following things:

   OK

   BAD

   BAD failures:2

   BAD errors:1

   BAD failures:2 errors:1


I would like to propose that the following format be used:

   Whatever you like -- DESCRIPTION -- SUCCESS

Put whatever you like at the front, provided it doesn't contain '--'.

The DESCRIPTION shouldn't contain '--' either.

SUCCESS should take the form I described above.


Does this work for the people running the automated tests?

--
Steve Alexander