[ZCM] [ZC] 2145/ 1 Request "2.9.x doesn't find tests in dirs named ftests, but 3.2.x does"

[Collector: Zope Bugs, Features, and Patches ...]

Issue #2145 Update (Request) "2.9.x doesn't find tests in dirs named ftests, but 3.2.x does"
 Status Pending, Zope/bug+solution medium
To followup, visit:
  http://www.zope.org/Collectors/Zope/2145

==============================================================
= Request - Entry #1 by slinkp on Jul 7, 2006 5:28 pm

The zope 3 test runner by default looks only for directories named "tests".  The test.py script for zope 3 then overrides this to allow directories named "tests" or "ftests", with this line:

defaults = ['--tests-pattern', '^f?tests$', '--test-path', src]

But the test.py and bin/test.py scripts for zope 2 doesn't find any directories named ftests, because of this line:

defaults = '--tests-pattern ^tests$ -v'.split()

IMO this is a bug because:

- it's an undocumented behavior change between 2.8 and 2.9. There is nothing about this in CHANGES.txt or in the help output of test.py.

- it's inconsistent behavior between zope 2 and 3 which surprises the user given that they share testing infrastructure.

- third-party products such as Plone use the "ftests" naming convention extensively.  Under 2.9, these simply won't run and there is no explicit documentation explaining why this changed or how to work around this (use the --tests-pattern argument) or how to move forward (presumably the answer is "use layers").

The simple fix would be to change the line in zope 2's bin/test.py and test.py to read:

defaults = '--tests-pattern ^f?tests$ -v'.split()

After that, it would no longer be necessary to move our functional tests into directories named "tests".

Unfortunately that still doesn't bring full backward compatibility; the "-f" option under 2.9 can't work as expected until third-party functional tests are updated to (as I understand it) run in layers whose names don't contain "unit".  Instead, *no* tests would run with -f, and *all* tests would run without -f.
--tests-pattern can be provided as a workaround.

But at least we wouldn't have to rearrange our source trees in a way that contradicts the zope 3 books, which means losing history if we happen to use CVS ...

The docs in zope/testing are great, but all the above could really be better spelled out IMO. I wasted several hours figuring this out.
==============================================================