[Zope-Perl] features.

[Gisle Aas]

"Chris McDonough" <chrism@digicool.com> writes:

> Some potential issues....
> 
> 1.  Requiring folks to manually marshal nonscalar Perl data into
>     Python native datatypes in return expressions to get any
>     real use out of them is sort of.. painful.  Although it's
>     arguably more explicable in docs because there's not a
>     lot of magic going on.  Does anybody have any opinions on
>     whether this marshalling should be more transparent?  For
>     example, it'd be nice to be able to "return %hash;" at the

It would have to be "return \%hash;" otherwise the hash is expanded
into a list of key/value pairs.  When I think of it, a Script(Perl)
thingy is called in scalar context, which means that this actually
returns a fractional string like "1/8".  Just yet another strange
feature of perl.  Mainly useful as a boolean test to tell that the
%hash contains elements.

>     end of a restricted Perl method and have it marshalled
>     into a Python dictionary for you for use in DTML.  I understand
>     that doing this with mutable data is dangerous due to expectations
>     that the data struct is mutable between languages, but
>     the current behavior is to return a string rep of the hash,
>     which is generally unhelpful.  Gisle made restricted Perl methods
>     much more useful when he sent the patch to Zope.pm which exposed
>     the Python::* marshalling routines, I wonder if this is enough?

We should find out what is the "correct" fix to make "perl ref"
objects do the sensible thing inside the Zope security machinery.

> 2.  Why does this work in an unrestricted Perl Method but not in a
>     restricted Perl method?
> 
>     args: self
> 
>     my @names;
>     for ($self->{Control_Panel}->objectIds()) {
>         push (@names, $_);
>     }
>     return Python::list(@names);
> 
>     (In an unrestricted Perl method, you need to pop $self off the arg
>     stack when it comes in, but otherwise they're identical.)
> 
>     In a restricted Perl method, this fails with:
> 
>     Error Type: PerlError
>     Error Value: Not a HASH reference at testreturn line 2.

getitem() like overloading is simply not provided for
"Script(Perl,restricted)".  The only thing you can do with python
objects here are getattr.

>     I'm sure this is probably a stupid question, but I don't care. :-)
>     Pointers to docs on Perl indirection and OO notation accepted,
>     particularly any that can help me understand this phenomena.

The $self->Foo works because we provide a Python::Object::AUTOLOAD
method that catch these calls and transforms them into a
Python::getattr() call (with automatic Python::apply()) if the fetched
attribute happens to be callable).

For restricted Script(Perl) we provide an alternative
Python::Object::AUTOLOAD (the compartment_pyobject_AUTOLOAD function
you find in zoperl/lib/perl/Zope.pm) that does the same but also
invoke the Zope security machinery.

The $self->{Foo} syntax works because we provide '%{}' (hash access)
overloading in Python::Object.  For the safe compartments we don't try
to set up something similar.  I think there might even be issues with
getting it to work at all.  A workaround might be to simply export
Python::getitem() to the compartment so that the example could be
written like:

   for (Python::getitem($self, "Control_Panel")->objectIds) {
       ...
   }

and work in both places.

> 3.  Help screens for zoperl need to be added (I can do this).

Nice.  I think it makes sense for DC to take over all GUI issues with
the Perl* products.  Writing Zope products is probably something you
do better than us.

> 4.  Docs for DBI_DA need to be written if we want to include it
>     in zoperl (I can also do this).  What are the gotchas with
>     DBI_DA right now?

You get a separate database connection for each thread Zope spawns.
Database connections are opened on demand (when you send SQL to the
server) and there is actually no way to close them in the DA
interface.  The "Connect immediately" is a no-op.

The reason we can't open/close handlers on demand is that this will
need to happen separately in all threads.  It does not make sense to
only open/close in the thread that happen to run the DA interface
method.

> 5.  Restricted PMs probably want to use bindings like Python Scripts,
>     so you get get to context, container, etc, as opposed to passing
>     them on the args list.  If DC still has credit with Activestate,
>     and Gisle has time, we can maybe hook him up with Evan Simpson
>     so he can understand how to do this.

I agree that this needs to happen, but I think it can wait until after
1.0.  If I have time or not depends on priorities outside my control
here at ActiveState, so I can't really tell.

> 6.  We may want to consider a binary distribution for Windows users.
>     That said, I'm very impressed that I could use nmake to compile
>     pyperl on Windows!  I'm not sure of the relative importance of
>     this.  I imagine we will not distribute Zope-Perl with the default
>     Zope, as its functionality depends on potentially patched versions
>     of Python and particular Perl versions.

On windows no patching of perl or python should be needed.  We should
make it possible to simply install DC's binary Zope dist (with its own
Python) and then drop in ActivePerl and some bindist of 'pyperl' on
top of that.  I'm not a Windows guy, so I don't know too much about
what the issues are.  One issues appeared to be that the DC python did
not include the stuff needed to build additional stuff for it.

> 7.                                   Relatedly, do we know of a stock
>     Python/Perl combination that does not require patching of Python?
>     I'm using Zope-Perl happily with AS Perl 623 and stock Python 1.5.2
>     on Windows.  Are there particular configurations which require
>     patching?  If we adopt Python 2.X, will we still potentially
>     require that folks build their own patched Python?  It's not
>     really clear from the docs under what configurations you need
>     to patch.

The patching is needed on platforms that use dlopen() to load dynamic
libraries.  This include Linux and Solaris.  I don't know what the
requirements are for other Unix platforms.  For windows no patching
should be needed.

If the Python dist provide the "dl" module then it should actually be
possible to avoid patching even on Linux and Solaris.  The RTLD_GLOBAL
can actually be turned on after the library has been loaded by code
like this:

    import perl  # python will load it with dlopen() the standard way

    # turn on the RTLD_GLOBAL flag for the perl module so that perl
    # can load perl XS-extensions.
    try:
       import dl
       dl.open(perl.__file__, dl.RTLD_NOW | dl.RTLD_GLOBAL)
    except:
       pass

    # we should now be able to use perl extensions
    perl.eval("use Socket");

One problem is that this technique appears to require glibc-2.2 to
work and current Linux distributions have not really moved there yet.

> 8.  Understanding threading gotchas from a high level is desirable.
>     Maybe a few FAQ entries should be concocted with example failure
>     scenarios.

Good idea.  Issues I can think on off my head are:

   - can't really rely on perl data stored in Zope objects (as they
     are not very useable in threads other than the one that created it).
     Creating and using python data from perl avoids this problem.
   - ZDBI_DA open/close connection does not work
   - ZDBI_DA make a separate database connection for each thread.
   - Each thread has a separate copy of the perl op-trees (the compiled
     program).  This might require a lot of memory.
   - On a multi-CPU system all threads that are inside perl should actually
     run in true parallel (this should be a good thing :-).
   

Regards,
Gisle