[ZODB-Dev] Threads, ZODB, and how to install ZODB without distutils

[Manuel Vazquez Acosta]

Chris,

Thanks for your response.  I will try the locking first and the see if I 
can implement a transaction manager of my own.

I will read about virtualizing Python, I'm a debian user :))

Thanks and best regards,
Manuel.


Chris McDonough wrote:
> On Feb 11, 2007, at 7:29 PM, Manuel Vazquez Acosta wrote:
>
>> Hi all,
>>
>> I'm writing a small application for which I want to use ZODB's 
>> persistence machinery. I have a couple of question, though.
>>
>> I have read that each thread should have its own connection to the 
>> DB. However, in my case, each thread should be aware of what is 
>> actually in the DB at all times. So I wonder if I can shared the 
>> connection between those threads as long as I take the means to 
>> protect it (i.e RLock).
>
> I am not certain, actually.  The mechanism for doing this is in 
> there.  The interfaces.py file states:
>
>     Synchronization
>     ---------------
>
>     A Connection instance is not thread-safe.  It is designed to
>     support a thread model where each thread has its own transaction.
>     If an application has more than one thread that uses the
>     connection or the transaction the connection is registered with,
>     the application should provide locking.
>
> And it goes on to talk about the sync=False parameter vaguely.
>
> But the story doesnt really end there... there is also the capability 
> to register a different "transaction manager" at DB open time.. the 
> default one assumes one transaction per thread.
>
> I'd need to dig through all this code to reunderstand it, but it looks 
> possible.
>
>>
>> My scenario is akin a consumer-producer with shared buffer. Consumers 
>> pull items from the buffer whilst producers put items in the buffer. 
>> The buffer is an OOBTree along with an IOBTree which gives "serial" 
>> numbers to the keys of the OOBTree.
>
> The typical way to do this using one-connection-per-thread and a 
> threaded txn manager would be to call transaction.commit() when 
> mutating the shared data structures.  To detect changes to the objects 
> provided by a connection (due to other threads committing during the 
> connection's transaction), you can connection.sync().  Zope calls 
> transaction.begin() at the beginning of a request and either 
> transaction.abort() (on failure) or transaction.commit() (on success) 
> at its termination.  It never needs to do an explict connection.sync() 
> because it never wants to do a "dirty read" (each request has its own 
> connection and sees its connection data as canonical for the duration 
> of its existence).  Zope handles conflicts by retrying the request.  
> But Zope has it easy because the web has natural transaction 
> boundaries that don't always exist in GUI apps or other apps with a 
> different request/response cycle.
>
>>
>> The other question is about compiling ZODB without using the 
>> out-of-the-box distutils installation procedure. I'm also playing 
>> with Zope and Plone, so I have several instances on the same machine. 
>> I think installing with distutils may cause conflicts with the Zope 
>> instances. Am I right? If so, then how should I install ZODB 
>> side-by-side the consumer-producer application?
>
> The easiest thing to do here is to install a separate Python instance 
> for each project.  Alternately if you're on UNIX, you can use "virtual 
> python" (http://peak.telecommunity.com/dist/virtual-python.py) which 
> makes a tree of symlinks that acts like a new Python instance but has 
> its own "site packages" directory in which distutils stuff is installed.
>
> If you're on Windows, I believe Zope/Plone ship with their own copy of 
> Python on that platform, so installing to the system python shouldnt 
> create a conflict.
>
> - C
>
>