[ZODB-Dev] Fan-out support / horizontal cache sharing for blobs

[Christian Theune]

Hi there,

Zagy and I have been discussing an issue that we have with fan-out
setups and blobs.

So, assume the following setup:

- A main ZEO server which has the blob storage locally available

- A fan-out ZEO server which runs on a different machine and has 4
  clients connected

Now, the fan-out ZEO server pulls the blobs via ZEO RPC and stores them
in a blob cache directory.

The same is true for the 4 clients which also create a cache directory
and store blobs there.

We now have 5 cache directories on the second machine. The cache of the
fan-out server will be approximately the combined set of blobs from the
other clients.

As all those 5 directories live on the same disk, we'd like them to be
shared between the various clients.

This currently doesn't work because we apply coarse grained locking to
the whole directory instead of individual files. 

We propose to change this so that locking will become fine-grained and
allow multiple ZEO clients to work on the same blob cache directory.

Comments?

Christian

-- 
Christian Theune · ct at gocept.com
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 7 · fax +49 345 1229889 1
Zope and Plone consulting and development
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url : http://mail.zope.org/pipermail/zodb-dev/attachments/20090609/ede2e670/attachment.bin