[ZODB-Dev] design issue: job queue is concurrency bottleneck

[John Belmonte]

Christian Robottom Reis wrote:
> On Thu, Mar 04, 2004 at 10:11:51AM -0500, John Belmonte wrote:
> 
>>>Have you considered trying to add the key, catching ConflictError, and
>>>trying again adding a small time offset?
>>
>>My conflict errors are not due to writing to the same keys.  From my 
>>limited understanding of BTrees, adding keys may cause buckets to be 
>>shuffled around, resulting in a write error.
> 
> Well, the same suggestion applies: catch the ConflictError, sync() your
> current connection (or abort() your current transaction), and try again?

Yes, I'm aborting and running the transaction again.  However, I'm 
trying to eliminate write conflicts because I believe, given a 
high-write application, that's the only scalable solution.  For example, 
with my very slow development machine acting as client and server (a 
nice worst-case test), and with two simultaneous requests, one client 
must retry up to three times due to the job queue conflict.  With three 
simultaneous requests, up to six times.  I haven't tried four requests 
because I don't want to find out that it's an exponential progression :-).

-John


-- 
http:// if  ile.org/