Hi,
I hate DeprecationWarnings at the best of times, since no one actually does anything about them until whatever they're bleating about is actually gone anyway, but zope.testing has outdone itself.
Whoever introduced that warning, if you're going to do so, please solve any problems with code in the actual package itself before releasing.
zope.testing.testrunner.debug imports doctest from zope.testing and so bleats whenever tests are run with zope.testing 3.8.6.
Why was 3.8.6 released when it still emits these warnings itself?
Chris
On Tue, Dec 29, 2009 at 04:04:34PM +0000, Chris Withers wrote:
I hate DeprecationWarnings at the best of times, since no one actually does anything about them until whatever they're bleating about is actually gone anyway, but zope.testing has outdone itself.
Some background, because you're obviously not following all the threads currently active on zope-dev (nobody can!):
* there was a zope.testing 3.8.4 release that dropped some "unused imports" from zope.testing.doctestunit
* that turned out to break many things, including zope.container and (according to some reports) zope.interface
* there were changes made to zope.testing trunk, backpedaling a bit and adding those legacy imports back, with a deprecation warning for the whole zope.testing.doctestunit, and (for good measure) a deprecation warning for zope.testing.doctest.
* I released zope.testing 3.8.5 with the deprecation warning (which turned out to be triggered by zope.testing itself, making the warning quite useless) because I thought having a spurious warning is better than having broken zope.interface
* I then had to release zope.testing 3.8.6 because the 3.8.5 egg was broken (thank you setuptools for the sudden but inevitable stab in the back)
* Fabio Tranchitella is working to make the zope.testing.doctest deprecation warning useful by doing scary things like converting zope.testing.doctest from a module into a package that has all the code in __init__.py. He asked for a review of his changes. I'm too scared to do that.
* Meanwhile there are discussions about issues switching from old zope.testing.doctest to stdlib's doctest with Windows and newlines.
* I'd rather revert back the state of things as of zope.testing 3.8.4 with the legacy zope.testing.doctestunit imports added back and a single deprecation warning for zope.testing.doctestunit, until we figure out the difficult part: what to do with zope.testing.doctest itself.
Opinions?
Whoever introduced that warning, if you're going to do so, please solve any problems with code in the actual package itself before releasing.
zope.testing.testrunner.debug imports doctest from zope.testing and so bleats whenever tests are run with zope.testing 3.8.6.
Why was 3.8.6 released when it still emits these warnings itself?
Because 3.8.5 broke running code.
Why was 3.8.5 released when it broke running code? Because there were no comments explaining that those "unused imports" were part of the API, and no buildbots to give a timely warning about unexpected breakage of other packages.
Welcome to the wonderful world of non-monolithic Zope 3. Fasten your seat-belt, it could get bumpy.
Marius Gedminas
* 2009-12-29 21:54, Marius Gedminas wrote:
Fabio Tranchitella is working to make the zope.testing.doctest deprecation warning useful by doing scary things like converting zope.testing.doctest from a module into a package that has all the code in __init__.py. He asked for a review of his changes. I'm too scared to do that.
Meanwhile there are discussions about issues switching from old zope.testing.doctest to stdlib's doctest with Windows and newlines.
Note that the current trunk of zope.testing is already using the standard doctest; zope.testing.doctest is still there for backward compatibility, and emits a single deprecation warning at import time. We were considering about switching to a deprecation warning issued at each usage of zope.testing.doctest.{DocFileSuite,DocTestSuite}, though.
I tested the whole ZTK (hey, with the zope.app.* packages too :)) and there were no regressions.
I'd love if somebody would review my changes, though, and help me to make a release.
Fabio
On Tue, Dec 29, 2009 at 10:55:37PM +0100, Fabio Tranchitella wrote:
- 2009-12-29 21:54, Marius Gedminas wrote:
Fabio Tranchitella is working to make the zope.testing.doctest deprecation warning useful by doing scary things like converting zope.testing.doctest from a module into a package that has all the code in __init__.py. He asked for a review of his changes. I'm too scared to do that.
Meanwhile there are discussions about issues switching from old zope.testing.doctest to stdlib's doctest with Windows and newlines.
Note that the current trunk of zope.testing is already using the standard doctest; zope.testing.doctest is still there for backward compatibility, and emits a single deprecation warning at import time. We were considering about switching to a deprecation warning issued at each usage of zope.testing.doctest.{DocFileSuite,DocTestSuite}, though.
I tested the whole ZTK (hey, with the zope.app.* packages too :)) and there were no regressions.
That's kinda reassuring, but ZTK is a (small) subset of packages that rely on zope.testing.
I don't know enough about the differences between stdlib's doctest.py (in its various Python 2.4/2.5/2.6 incarnations) and zope.testing.doctest, other than that I've seen diffs, they were non-trivial, with bugfixes and new features; I've heard about monkey-patching the stdlib's doctest.py (which fills me with dread; when exactly is the monkey-patching performed?), and I'd rather not touch the issue without either complete understanding or a very large test suite (all of the packages that were in the Zope 3 KGS at the very least) run on various platforms.
I'd love if somebody would review my changes, though, and help me to make a release.
Where are the changes, again? Are you talking about r107023? http://zope3.pov.lt/trac/changeset/107023/zope.testing/trunk
Ah, I understand the point of putting code into __init__.py now! Clever.
+1 for all the changes, but AFAICS if people move from zope.testing.doctest to stdlib's doctest they lose at least two things:
* custom doctest exception formatting
* support for the INTERPRET_FOOTNOTES feature
and since zope.testing.doctest still reimplements large bits of doctest, I don't know what other bugfixes might be lost too (like the universal newline thing that punishes people for daring to release packages from Windows machines).
Overall, I'm still -1 for deprecating zope.testing.doctest at this point. A PendingDeprecationWarning would be more appropriate, IMHO.
Cheers! Marius Gedminas
Marius Gedminas wrote:
+1 for all the changes, but AFAICS if people move from zope.testing.doctest to stdlib's doctest they lose at least two things:
custom doctest exception formatting
support for the INTERPRET_FOOTNOTES feature
and since zope.testing.doctest still reimplements large bits of doctest, I don't know what other bugfixes might be lost too (like the universal newline thing that punishes people for daring to release packages from Windows machines).
Not even release, develop and test...
Overall, I'm still -1 for deprecating zope.testing.doctest at this point. A PendingDeprecationWarning would be more appropriate, IMHO.
I'm -sys.maxint, lets not have the zLOG debacle over again...
Chris
On Tue, Dec 29, 2009 at 23:47, Marius Gedminas marius@gedmin.as wrote:
I don't know enough about the differences between stdlib's doctest.py (in its various Python 2.4/2.5/2.6 incarnations) and zope.testing.doctest, other than that I've seen diffs, they were non-trivial, with bugfixes and new features; I've heard about monkey-patching the stdlib's doctest.py (which fills me with dread; when exactly is the monkey-patching performed?), and I'd rather not touch the issue without either complete understanding or a very large test suite (all of the packages that were in the Zope 3 KGS at the very least) run on various platforms.
zope.testing runs a lot of doctests. Lot's and lot's. In the Python 3 branch I've moved them over to using stdlibs doctest. There was two bugs that are fixed in Zopes doctest that is not fixed in Pythons doctest. I've added monkey-patches on import for them. Obviously they should also be reported upstream, but new releases won't remove the bugs in old releases, so the monkey patches need to stay anyway.
The alternative to this is that somebody port zopes doctest.py to Python 3, so we don't deprecate it but keep it separate, for ever adding to the frustration and complication in doing testing and in particular doctesting that persists.
* custom doctest exception formatting
What does that formatting actually do?
* support for the INTERPRET_FOOTNOTES feature
Then they should use Manuel. I find it highly unlikely that this feature will make it into stdlib.
and since zope.testing.doctest still reimplements large bits of doctest, I don't know what other bugfixes might be lost too (like the universal newline thing that punishes people for daring to release packages from Windows machines).
There's no test for that one, so I didn't know about it. I think the problem with having a sperate doctest.py is evident in the fact that there has been bugs fixed for *years* but not moved upstream. The custom doctest was there to support Python 3, and nothing else. Now it's become it's own creature, and that's very bad.
Overall, I'm still -1 for deprecating zope.testing.doctest at this point. A PendingDeprecationWarning would be more appropriate, IMHO.
That will just leave less time between people getting warned and before it gets removed. I must say I find PendingDeprecationWarnings completely and utterly pointless.
On Wed, Dec 30, 2009 at 08:58:52AM +0100, Lennart Regebro wrote:
On Tue, Dec 29, 2009 at 23:47, Marius Gedminas marius@gedmin.as wrote:
I don't know enough about the differences between stdlib's doctest.py (in its various Python 2.4/2.5/2.6 incarnations) and zope.testing.doctest, other than that I've seen diffs, they were non-trivial, with bugfixes and new features; I've heard about monkey-patching the stdlib's doctest.py (which fills me with dread; when exactly is the monkey-patching performed?), and I'd rather not touch the issue without either complete understanding or a very large test suite (all of the packages that were in the Zope 3 KGS at the very least) run on various platforms.
zope.testing runs a lot of doctests. Lot's and lot's.
Do you mean the test suite of zope.testing itself? Yes, it's not tiny, but that doesn't mean it is representative of all the other packages out there that currently use zope.testing.doctest.
In the Python 3 branch I've moved them over to using stdlibs doctest. There was two bugs that are fixed in Zopes doctest that is not fixed in Pythons doctest. I've added monkey-patches on import for them.
On import of what? zope.testing's __init__.py?
I think I found it: in regebro-python3-reloaded branch I see zope.testing's __init__ doing 'import monkeys', which is
http://zope3.pov.lt/trac/browser/zope.testing/branches/regebro-python3-reloa...
Would you mind terribly adding a couple of comments explaining why DocTestCase __init__ and tearDown need to be patched? Is that to prevent the globs from keeping state between different tests? Just guessing, by looking at the most interesting thing that happens in the overridden methods.
Obviously they should also be reported upstream,
The word "should" meaning they haven't been?
What are the bugs? The globs thing, and allowing mixed unicode + UTF-8 output inside doctests?
but new releases won't remove the bugs in old releases, so the monkey patches need to stay anyway.
*nod*
The alternative to this is that somebody port zopes doctest.py to Python 3, so we don't deprecate it but keep it separate, for ever adding to the frustration and complication in doing testing and in particular doctesting that persists.
I'm +1 for dropping the forked doctest.py, and I'm very grateful for your Python-3-ification efforts of zope.stuff.
I'm -1 for leaving existing users of zope.testing.doctest hanging without a known and documented migration path.
* custom doctest exception formatting
What does that formatting actually do?
I don't remember, exactly, I was just grepping the diffs for DocTestFailureException. It's probably the bit that lets bin/test -c colorize things like diffs, and tracebacks in doctest failures. If that's the case, I'd truly hate to lose this feature, it helps me immensely.
* support for the INTERPRET_FOOTNOTES feature
Then they should use Manuel.
Good, so we have a migration path for this bit!
How mechanical can such a transformation be? Is it sufficient to replace
zope.testing.doctest.DocFileSuite('foo.txt', **kw)
with
manuel.testing.TestSuite(manuel.doctest.Manuel(), 'foo.txt', **kw)
or do we need other changes, like removing INTERPRET_FOOTNOTES from optionflags and include the manuel.footnote.Manuel() in some way?
I've never used Manuel, I'm guessing what the syntax ought to be by looking at http://packages.python.org/manuel/
I find it highly unlikely that this feature will make it into stdlib.
I agree.
and since zope.testing.doctest still reimplements large bits of doctest, I don't know what other bugfixes might be lost too (like the universal newline thing that punishes people for daring to release packages from Windows machines).
There's no test for that one, so I didn't know about it.
I still don't know anything definite about it, other than remembering that at some point people on Windows would release .zip sdists with .txt files inside that had DOS line-endings, and that would cause doctest failures in some circumstances, again, I'm not sure which. It was brought up in another thread recently, and I got the impression that zope.testing.doctest had a fix of some kind (using universal newlines?).
I suppose we could ignore the issue and wait for some hapless Windows user to bring it to our attention, at which point we'll fix it and make sure our fix has a unit test. Maybe.
*shrug*
I think the problem with having a sperate doctest.py is evident in the fact that there has been bugs fixed for *years* but not moved upstream. The custom doctest was there to support Python 3, and nothing else. Now it's become it's own creature, and that's very bad.
Yes, I absolutely agree!
Overall, I'm still -1 for deprecating zope.testing.doctest at this point. A PendingDeprecationWarning would be more appropriate, IMHO.
That will just leave less time between people getting warned and before it gets removed. I must say I find PendingDeprecationWarnings completely and utterly pointless.
Then let's have no warning at all until we have a clear and documented migration path (and one that doesn't involve loss of features) for people who use zope.testing.doctest.
tl;dr:
I'm feeling a probably misplaced and crushing sense of responsibility to make sure existing users of zope.testing and zope.testing.doctest don't suffer needlessly by the inevitable eventual deprecation and subsequent removal of zope.testing.doctest.
Marius Gedminas
On Wed, Dec 30, 2009 at 13:45, Marius Gedminas marius@gedmin.as wrote:
Obviously they should also be reported upstream,
The word "should" meaning they haven't been?
As far as I can tell, yes.
What are the bugs? The globs thing, and allowing mixed unicode + UTF-8 output inside doctests?
Yes. And apparently also the Windows lineendings.
I don't remember, exactly, I was just grepping the diffs for DocTestFailureException. It's probably the bit that lets bin/test -c colorize things like diffs, and tracebacks in doctest failures. If that's the case, I'd truly hate to lose this feature, it helps me immensely.
OK, so that colorizing support could probably also be moved into stdlib. If someone (ie you, you seem to be on top of the problems) get diffs into the Python tracker then maybe we could get it into Python 2.7 and 3.2, which would be cool.
How mechanical can such a transformation be? Is it sufficient to replace
zope.testing.doctest.DocFileSuite('foo.txt', **kw)
with
manuel.testing.TestSuite(manuel.doctest.Manuel(), 'foo.txt', **kw)
No idea, the Manuel experts need to answer that. :)
On Wed, Dec 30, 2009 at 03:08:26PM +0100, Lennart Regebro wrote:
On Wed, Dec 30, 2009 at 13:45, Marius Gedminas marius@gedmin.as wrote:
I don't remember, exactly, I was just grepping the diffs for DocTestFailureException. It's probably the bit that lets bin/test -c colorize things like diffs, and tracebacks in doctest failures. If that's the case, I'd truly hate to lose this feature, it helps me immensely.
OK, so that colorizing support could probably also be moved into stdlib. If someone (ie you, you seem to be on top of the problems) get diffs into the Python tracker then maybe we could get it into Python 2.7 and 3.2, which would be cool.
The only thing needed from doctest.py, AFAIU, is a custom exception class inheriting from AssertionError, so the test runner (which does the colorization) can distinguish doctest errors from regular assertion failures.
It should be simple to monkey-patch this too, by setting DocTestCase.failureException, so we can support Python 2.5 through 3.1
How mechanical can such a transformation be? Is it sufficient to replace
zope.testing.doctest.DocFileSuite('foo.txt', **kw)
with
manuel.testing.TestSuite(manuel.doctest.Manuel(), 'foo.txt', **kw)
No idea, the Manuel experts need to answer that. :)
I was afraid you'd say that...
Marius Gedminas
On Wed, Dec 30, 2009 at 17:29, Marius Gedminas marius@gedmin.as wrote:
The only thing needed from doctest.py, AFAIU, is a custom exception class inheriting from AssertionError, so the test runner (which does the colorization) can distinguish doctest errors from regular assertion failures.
Oh, so *that* is why it was there. I wondered. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
Marius Gedminas wrote:
It should be simple to monkey-patch this too, by setting DocTestCase.failureException, so we can support Python 2.5 through 3.1
How mechanical can such a transformation be? Is it sufficient to replace
zope.testing.doctest.DocFileSuite('foo.txt', **kw)
with
manuel.testing.TestSuite(manuel.doctest.Manuel(), 'foo.txt', **kw)
No idea, the Manuel experts need to answer that. :)
I was afraid you'd say that...
I wonder if Benji missed this thread?
Chris
On Wed, Dec 30, 2009 at 7:45 AM, Marius Gedminas marius@gedmin.as wrote:
On Wed, Dec 30, 2009 at 08:58:52AM +0100, Lennart Regebro wrote:
On Tue, Dec 29, 2009 at 23:47, Marius Gedminas marius@gedmin.as wrote:
* support for the INTERPRET_FOOTNOTES feature
Then they should use Manuel.
Good, so we have a migration path for this bit!
How mechanical can such a transformation be?
The transformation should be reasonably mechanical. Something like this (untested):
m = manuel.footnote.Manuel() m += manuel.doctest.Manuel(**kw) return manuel.testing.TestSuite(m, 'foo.txt')
Lennart Regebro wrote:
zope.testing runs a lot of doctests. Lot's and lot's. In the Python 3 branch I've moved them over to using stdlibs doctest. There was two bugs that are fixed in Zopes doctest that is not fixed in Pythons doctest.
Are there open issues for these in the python bug tracker? If not, could you please add them?
I've added monkey-patches on import for them.
:'(
There's no test for that one, so I didn't know about it. I think the problem with having a sperate doctest.py is evident in the fact that there has been bugs fixed for *years* but not moved upstream.
Yes, because people appear to prefer monkey patching or forking ;-)
That will just leave less time between people getting warned and before it gets removed. I must say I find PendingDeprecationWarnings completely and utterly pointless.
I feel the same about deprecation warnings...
Chris
On Fri, Jan 1, 2010 at 23:31, Chris Withers chris@simplistix.co.uk wrote:
Lennart Regebro wrote:
zope.testing runs a lot of doctests. Lot's and lot's. In the Python 3 branch I've moved them over to using stdlibs doctest. There was two bugs that are fixed in Zopes doctest that is not fixed in Pythons doctest.
Are there open issues for these in the python bug tracker? If not, could you please add them?
No, and Yes. Although I must say it's obvious Marius understand the issues better, so I'm hoping he will do it. :-)
On Tue, Dec 29, 2009 at 10:54:03PM +0200, Marius Gedminas wrote:
On Tue, Dec 29, 2009 at 04:04:34PM +0000, Chris Withers wrote:
I hate DeprecationWarnings at the best of times, since no one actually does anything about them until whatever they're bleating about is actually gone anyway, but zope.testing has outdone itself.
Some background, because you're obviously not following all the threads currently active on zope-dev (nobody can!):
there was a zope.testing 3.8.4 release that dropped some "unused imports" from zope.testing.doctestunit
that turned out to break many things, including zope.container and (according to some reports) zope.interface
there were changes made to zope.testing trunk, backpedaling a bit and adding those legacy imports back, with a deprecation warning for the whole zope.testing.doctestunit, and (for good measure) a deprecation warning for zope.testing.doctest.
I released zope.testing 3.8.5 with the deprecation warning (which turned out to be triggered by zope.testing itself, making the warning quite useless) because I thought having a spurious warning is better than having broken zope.interface
I then had to release zope.testing 3.8.6 because the 3.8.5 egg was broken (thank you setuptools for the sudden but inevitable stab in the back)
Fabio Tranchitella is working to make the zope.testing.doctest deprecation warning useful by doing scary things like converting zope.testing.doctest from a module into a package that has all the code in __init__.py. He asked for a review of his changes. I'm too scared to do that.
Meanwhile there are discussions about issues switching from old zope.testing.doctest to stdlib's doctest with Windows and newlines.
I'd rather revert back the state of things as of zope.testing 3.8.4 with the legacy zope.testing.doctestunit imports added back and a single deprecation warning for zope.testing.doctestunit, until we figure out the difficult part: what to do with zope.testing.doctest itself.
Opinions?
Whoever introduced that warning, if you're going to do so, please solve any problems with code in the actual package itself before releasing.
zope.testing.testrunner.debug imports doctest from zope.testing and so bleats whenever tests are run with zope.testing 3.8.6.
Why was 3.8.6 released when it still emits these warnings itself?
Because 3.8.5 broke running code.
Why was 3.8.5 released when it broke running code?
Both of the previous sentences were meant to say 3.8.4. Thanks again, setuptools.
Because there were no comments explaining that those "unused imports" were part of the API, and no buildbots to give a timely warning about unexpected breakage of other packages.
Welcome to the wonderful world of non-monolithic Zope 3. Fasten your seat-belt, it could get bumpy.
Marius Gedminas
Marius Gedminas wrote:
Meanwhile there are discussions about issues switching from old zope.testing.doctest to stdlib's doctest with Windows and newlines.
I'd rather revert back the state of things as of zope.testing 3.8.4 with the legacy zope.testing.doctestunit imports added back and a single deprecation warning for zope.testing.doctestunit, until we figure out the difficult part: what to do with zope.testing.doctest itself.
Yes please!
For me, this likely means getting the Windows newline stuff fixed and in a Python 2.x release *before* deprecating zope.testing.doctest!
Chris
-
Benji York -
Chris Withers -
Fabio Tranchitella -
Lennart Regebro -
Marius Gedminas