[Zope-Perl] Benefits of Zope multithreading

[Gisle Aas]

--=-=-=

Monty Taylor <mtaylor@information-innovation.com> writes:

> Gisle Aas wrote:
> 
> > The consequence of this is really that PerlMethods should not be used
> > for anything that will not return fast.  This probably also make the
> > ZDBI_DA not very usable.
> >
> 
> Damn it.

Fear not.  Perl ithreads is going to save us :-)

> 
> >
> > I'm now investigating if there is a way to fix this.  If we make sure
> > references to perl data do not escape to python (that means not
> > providing the "perl ref objects") then we can probably let there be
> > separate perl interpreters for each thread.  Then we don't need any
> > perl lock at all and PerlMethods will be able to run in true parallel
> > on multiple CPUs.  When these PerlMethods call back to python in order
> > to invoke methods on the Zope objects passed as argument(s) they will
> > be sequentialised though.
> >
> 
> Would this mean we couldn't instatiate a Perl Object from within Python? What
> happens to dbi.py/ZDBI_DA in that case?

We have to drive all the DBI logic in perl then.  The attached
stripped down test works for me now.  It runs a separate DBI
connection from each thread even though we only have a single python
DB object.

For ZDBI_DA it means that if you enable one such connection you will
after a while have as many database connections as you have threads
running in Zope (and it will be hard to close them down :-( ).

Regards,
Gisle



--=-=-=
Content-Type: text/python
Content-Disposition: attachment; filename=thread-dbi.py
Content-Description: Test program

import time
import perl
import sys
import thread

class DB:
    def __init__(self, source):
        self.source = source
        self.query("select 1")

    def query(self, sql):
        if not perl.defined("ZopeDBI::query"):
            self.dbi_init()
        return perl.call("ZopeDBI::query", self, sql);

    def dbi_init(self):
        perl.require("ZopeDBI")

    def close(self):
        pass

db = DB("DBI:mysql:database=fotodb")
print db.query("select count(*) from img")

def f(wait_time):
    id = thread.get_ident()
    print "Thread", id
    for i in range(10):
        print str(id)+":", db.query("select id from img limit 3")
        time.sleep(wait_time)

for i in range(10):
    thread.start_new_thread(f, (0.1,))
f(1)

print "waiting..."
time.sleep(10*60)


--=-=-=
Content-Type: text/perl
Content-Disposition: attachment; filename=ZopeDBI.pm
Content-Description: Perl library to go with it.

package ZopeDBI;

print "Loading ZopeDBI\n";

use DBI;
use Python;

sub dbi_connect {
    my $self = shift;
    my $source = $self->source;
    print "Connecting to $source\n";
    DBI->connect($source, "gisle", "",
		 {
		  RaiseError => 1,
		  PrintError => 0,
		  AutoCommit => 1,
		 }
		);
}

my %dbh;  # cache of DBI handles

sub query {
    my($self, $sql) = @_;
    my $id = Python::id($self);
    my $dbh = $dbh{$id} || ($dbh{$id} = dbi_connect($self));
    my $sth = $dbh->prepare($sql);
    $sth->execute;
    my $rows = Python::list();
    while (my @row = $sth->fetchrow) {
	$rows->append(Python::list(@row));
    }
    $rows;
}

1;

--=-=-=--