[ZODB-Dev] ZEO and replication of BTree based objects

[Roché Compaan]

* Jeremy Hylton <jeremy at zope.com> [2003-06-04 22:37]:
> On Wed, 4 Jun 2003 22:16:01 +0200
>  Roché Compaan <roche at upfrontsystems.co.za> wrote:
> > Hi there
> > 
> > What exactly happens to a ZEO ClientCache when another
> > ZEO client makes
> > modifications to BTree based objects? Let's say I have an
> > object that
> > has an OOBTree data attribute. Will the whole object be
> > retrieved from
> > the server each time new keys are set or will only the
> > transactions the
> > object were involved in be replicated to the client?
> 
> ZEO uses invalidation-based cache consistency.  When one
> client modifies an object, the other clients receive
> invalidation messages.  The other clients don't load the
> object unless they attempt to read it.

Yes I am aware of this. I was wondering if you would consider to force a
load on a client as soon as it receives an invalidation message based on
a configuration variable. It doesn't make sense as a general use case
but we have a case where we have very few zeo clients, some big
infrequent transactions that touches a lot of objects and a slow
connection. We've done all we could to optimise the slow connection and
have very big cache sizes for clients. Although local reads are very
fast a client sometimes waits a long time for a bunch of objects that
was part of a big transaction.

> 
> A BTree is composed of many persistent objects -- typically
> a parent btree node and a bunch of buckets hangning.  Each
> persistent object is invalidated independently.  

Ah, I hoped it would be this economical. I was reading through the ZEO
source to try an figure out how the cache works and got worried about
BTrees when I saw that a whole object is invalidated. I suppose I should
have looked at the ZODB source to see how BTrees are persisted.

> If you add one key to a BTree, that will modify and, thus, invalidate
> just one bucket.  (Unless the bucket splits, in which case the parent
> is also modified.)

Sorry for my ignorance but when will a bucket split? Guess when it's
full ;-)


-- 
Roché Compaan
Upfront Systems                 http://www.upfrontsystems.co.za