[ZODB-Dev] design issue: job queue is concurrency bottleneck

[John Belmonte]

Toby Dickenson wrote:
> On Thursday 04 March 2004 05:04, John Belmonte wrote:
> 
>>I finally got through the first round of concurrency testing on my app.
>>    Despite being a high-write application, virtually no write conflicts
>>occur-- with one major exception.  I have a job queue, which is a BTree
>>keyed by scheduled job time.  It's the only BTree in my app that doesn't
>>have random keys.  Incoming requests to my server tend to generate a few
>>jobs, and unfortunately they must be scheduled at various constant time
>>periods in the future.  As a result, simultaneous requests generate jobs
>>scheduled for approximately the same time, mapping to the same area of
>>the BTree, and causing write conflicts.
> 
> That makes sense for keys that are *exactly* the same time. For approximately 
> the same time then surely conflict resolution would take care of the write 
> conflict?
> 
> Or do you have read conflicts?

 From what I can tell, the BTree conflict resolution cannot handle 
certain cases of adding new keys.

I don't think I've had any read conflicts since I started using MVCC. 
The two cases of conflicts I get are, for example:

   database conflict error (oid 0000000000000f0d, class
     BTrees._OOBTree.OOBucket, serial was 035382a075371277, now
     035382a073d9e100)

and

   database conflict error (oid 000000000000000a, class
     BTrees._OOBTree.OOBTree, serial was 035382a07f41b900, now
     035382a0343dd3ee)


-John


-- 
http:// if  ile.org/