[ZODB-Dev] Re: ZODB Benchmarks

[Christian Theune]

Hi,

Am Freitag, den 02.11.2007, 09:56 -0400 schrieb David Binger:
> On Nov 2, 2007, at 8:39 AM, Lennart Regebro wrote:
> 
> > On 11/2/07, Matt Hamilton <matth at netsight.co.uk> wrote:
> >> That may just end up causing delays periodically in  
> >> transactions... ie delays
> >> that the user sees, as opposed to doing it via another thread or  
> >> something.  But
> >> then as only one thread would be doing this at a time it might not  
> >> be too bad.
> >
> > But wouldn't then all other threads get a conflict?
> 
> If they are trying to do insertions at the same time as the  
> consolidation, yes.
> This data structure won't stop insertion conflicts, the intent is to  
> make them
> less frequent.

Hmm.

Wouldn't a queue be a good data structure to do that? IIRC ZC already
wrote a queue that doesn't conflict:

http://svn.zope.de/zope.org/zc.queue/trunk/src/zc/queue/queue.txt

If you store key/value pairs in the queue, you can do a step-by-step
migration from the queue to the btree.

Probably this should be encapsulated into a new data structure that
looks btree-like and has an additional `consolidate` method.

Calling the `consolidate` method would have to happen from the
application that uses this data structure. Two issues I can think of
immediately:

- General: We need an efficient way to find all data structures that
need reconciliation, maybe a ZODB-wide index of all objects that require
reconciliation would be nice. 

- With Zope and ZEO: which Zope server is responsible for actually
performing the reconciliation? One of the Zope servers that is marked in
the zope.conf? Or maybe the ZEO server?

Christian

-- 
gocept gmbh & co. kg - forsterstrasse 29 - 06112 halle (saale) - germany
www.gocept.com - ct at gocept.com - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Dies ist ein digital signierter Nachrichtenteil
Url : http://mail.zope.org/pipermail/zodb-dev/attachments/20071102/987d2684/attachment.bin