[Zope-dev] whitespace filtering corner case in doctest

[Reinout van Rees]

On 2009-08-17, Marius Gedminas <marius at gedmin.as> wrote:
>
> On Mon, Aug 17, 2009 at 02:46:46PM +0000, Reinout van Rees wrote:
>> In some cases, importing readline can result in the escape code
>> ^[[?1034h= to be send ("8bit on").
>
> According to the gentoo bug report (liked from your blog post), this
> happens if your termcap/terminfo define smm/rmm codes ("meta on/meta
> off").

... which I didn't do, at least not consiously.  Just a pretty normal
OSX installation.

> BTW, regarding your workaround: I'd suggest using TERM=3Ddummy instead of
> TERM=3Dlinux, as a safer choice.  Not that it should matter much in
> practice.

That'd be better, yes.  I never do anything tweaking with TERM anyway,
so I don't know the options/effects.

jladage suggested to do the workaround in buildout, btw: add the
following to the part that generates the bin/test script:

 environment-vars =
     TERM linux

>> This escape code isn't visible, which leads to hard to find test
>> errors, see
>> http://reinout.vanrees.org/weblog/2009/07/16/invisible-test-diff.html
>> Granted, it are basically corner cases.  On the other hand, it does
>> seem to cause problems now and then, according= > to my googling.
>> Would it be ok to add it to the whitespace normalization of
>> doctests?  Opinions?
>
> Well, it's not whitespace, really...  My gut feeling is that this fixup
> doesn't belong in the core doctest code.

Ok.

> If this happened in my project, I'd either
>
>   1) make sure I import readline at module level, before any tests are
>      run

This doesn't work, surprisingly.  It *is* a corner case.
It are tests where z3c.testsetup tests that it runs tests correctly.
That's a lot of "tests" in one sentence, which means that the test
output comes from separate python processes that run tests.  So if I
import readline in z3c.testsetup's tests, I still get the output from
the test runners that are being tested.

Sigh, difficult to explain, such a recusive testing thingy :-)

> or
>
>   2) add a renormalizer that removes the escape sequence for the test
>      that triggers this

And you'd do this per-project and not in core zope.testing, right?
Fine.  Uli already send me some example code, so I'll do that (and put
the code snippet on my weblog so that google can find it).

> Perhaps it would be nice if doctest's differ escaped ASCII control
> characters?  (You could do that too with a renormalizer.)

Can we safely assume that a specific set of control characters needs
to be escaped?  It sounds a bit dangerous to me.



Thanks!

Reinout


-- 
Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org
Software developer at http://www.thehealthagency.com
"Military engineers build missiles. Civil engineers build targets"