Zope transactions and connections was: Re: CT: Stumped was RE: [Zope] Asynchronous dtml?

[Dieter Maurer]

albert boulanger writes:
 > Here is another example use for something like this. The Plumtree
 > Portal Server, to build the content of a mypage, will use multiple
 > threads to collect the content for the table cells of the mypage from
 > what they term gadget servers. It is important to collect this content
 > in parallel otherwise the time to build can be really bad especially
 > if multiple sources time out which can occur more often than one would
 > expect. If one were to build mypages for CMF for example and rely on
 > external content provision (say stock quotes, etc) for any of the
 > sources, one would need something like this.
Your use case should be okay:

    Threads become only problematic, if they access a
    database (including the ZODB) (or global data,
    but that is not Zope specific).

If they do access a database, they need to play well with
the transaction and persistence machinery of Zope:

    Zope associates a transaction with each thread.
    If you create a new thread, you will get a new transaction.
    The transaction is accessed with "get_transaction()".

    Zope registers objects with the transaction. These
    objects are examined when the transaction is commited
    or aborted in order to make potential changes persistent
    or discard them, respectively.

    ZODB objects are registered when they are changed,
    many other objects, e.g. DatabaseConnections, when
    they are accessed.

    Thus, if you change or access transaction aware objects, you may
    get a pending state, unless you call "get_transaction().commit()"
    or "get_transaction().abort()" at the end of your thread.


    Zope associates a ZODB connection with each HTTP request.
    This is done during traversal of the root object:
    ZODB.ZApplication.ZApplicationWrapper.__bobo_traverse__
    (db.open(version)).
    This connection is used to load objects on demand from
    the ZODB.
    It is propagated from an object to its persistent children
    during loading of the object.

    Thus, if you pass a Zope object to a new thread, the
    new thread uses the original thread's connection
    when it accesses (children) of the object or modifies the
    object itself.

    Apparently, the connection does not protect itself against
    concurrent use by different threads.
    This is quite understandable as such protection would
    cause a significant performance penalty.
    It is quite likely, that you get non-deterministic
    difficult to explain and analyse problems when you
    access the connection from different threads.
    I expect that even read access is dangerous.

    You can create a new connection in your thread
    and then access an object identified by an absolute
    URL "url" through

	import Zope
	root= Zope.app()
	object= root.unrestrictedTraverse(url)

    Note, however, that ZODB connections are pooled with
    a limited pool size (7 or 3 (for a version)). If
    you try to create more connections your thread blocks
    until there is an available connection.
    This may hit ZServer, too, when all available connections
    are used. Thus, your site may freeze, if you are not careful.


Dieter