[ZODB-Dev] Notes on using wrapper storages for record transformation

[Christian Theune]

On 05/18/2010 10:08 PM, Jim Fulton wrote:
> The design of ZODB views pickles as opaque to storages. Storages
> primarily store strings.  I have always assumed that writing wrapper
> storages for compression or encryption would be easy. I was wrong. :)
>
> Storages sometimes need to know about pickles:
>
> A. When doing garbage collection, a storage (or garbage collector)
>     needs to extract object references.
>
> B. When doing conflict resolution, the conflict-resolution code needs
>     to be able to extract state from object records.
>
> C. When doing blob-aware iteration, a storage needs to be able to
>     recognize blob records.
>
> D. When raising conflict errors, the exception instances *want* to get
>     the object class name from the database record.
>
> (A) was dealt with by the existing way the referencesf function was
> passed into storage pack functions.

Wow, good to know about those! Thanks for the info!

> I dealt (B) and (C) by expanding and renaming the interface of objects
> passed to the storage registerDB call.  These objects now provide an
> up-call mechanism from a storage to anything that might wrap it,
> including databases, storage servers, and intermediate wrapping
> layers.
>
> I punted on (D). You can't always get what you want. :)
>
> I'm mostly convinced what I've done is reasonable, although I'm just a
> little uncomfortable.  It has always been the theory that storages
> can't count on the contents of database records. Now that is
> reality.
>
> I've decided I really want to provide a database compression option. I
> could have built something directly into ZODB or used a wrapper
> storage. I chose the later because it was more flexible.  I think I
> would have run into the same issues without a wrapper storage.
>
> Soon I expect to release a wrapper storage implementation that
> provides record compression.

Cool! :)

-- 
Christian Theune · ct at gocept.com
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1
Zope and Plone consulting and development