[Zope-Perl] Re: Weird output in pyperl

[Felix Schwarz]

Hi Jeff,

Jeff Bachtel wrote:
>> Therefore I think that pyperl should force the Perl interpreter to
>> flush all buffers before returning to Python. And there is another
>> bug: The order of the items is reversed.

> Have you tried running pclose? My pipe-fu isn't great, but the unix
> manpage indicates this can be used to close the stream, which might
> (or might not) force the flushing of the associated buffered output.

I can't see where I should use pclose in caller.py or callee.py. I'm
already using pipe.close in caller.py.

>> Does anybody have any experience with Un*x popen and buffering? If I
>> run callee.py separately the order of the items is okay. Only
>> executing it with popen reveals the bug.

> I don't believe this is a bug so much as an artifact of the way popen
> and its buffering work. The difference between, in this case, being
> that the behaviour is deterministic but painful to the user.

I don't think there is a bug in the underlying libraries but I would
consider this behavior of pyperl as a bug. It contradicts the
assumptions programmers have when using Perl/Python.


Reading the pyperl source code didn't brought too many insights for
me. I found an easier way to trigger the "strage behavior"(tm):
./callee.py => ok
./callee.py | cat => wrong

That could mean that the "Perl" string doesn't get printed to stdout
(maybe due to buffering) or that it is not printed on stdout, right?

Obviously the brigding function for perl.eval is eval in perlmodule.c
(line 413). eval_pv (line 435) will call the perl interpreter. eval_pv
itself is correct - I'm getting the correct behavior with test program
in C which embeds the Perl interpreter and calls eval_pv, too.

When looking at the results of callee2.py I think that Python and/or
Perl have their own independent buffers for stdout.

Regarding my buffering idea:
Forcing Perl into the unbuffered mode when the Perl interpreter is
instanciated the first time in pyperl and calling fflush on stdout
just before evaling any expression, helps for some tests.

With modifications on the test cases I'm down to three test failures
and one skipped test. I produced some patches (including Jeff's) for
my modifications and put them on my webspace at
   http://user.cs.tu-berlin.de/~scfe/pyperl-patches.tar
If you have too much time I would be glad if you could look into the
12th patch. I didn't replace all the deprecated methods in
perlmodule.c as Perl provides compatibility makros but I'll replace
them soon.

My test scripts for the buffering problem are online, too:
   http://user.cs.tu-berlin.de/~scfe/pyperl-buffering.tar

-- 
Felix