[Zope-dev] BerkeleyStorage (was Alternative Storages)

[Phillip J. Eby]

At 11:43 PM 5/3/00 -0700, Michel Pelletier wrote:
>
>I'm suprised that more people don't use BerkeleyStorage today.  Is this
>because it doesn't undo?

It's probably also ecause the docs are sparse to nonexistent.  :)


>I don't immagine it would be too dificult to
>extent it to support undo.  Ty, what do you think?

Difficulty isn't the question as much as time budget.  I've been keeping Ty
busy with LoginManager and ZPatterns stuff.  BerkeleyStorage was more a
proof-of-concept for us, a kind of insurance that we really could use a
ZODB for our future apps.  We will probably revisit it, and the undo issue,
when we get closer to building an app that needs undo as well as
high-volume writes.  The key complication is that having a Berkeley ZODB
doesn't solve the various issues associated with ZCatalog performance in
that environment, which generates a high volume of Berkeley logfiles.  The
logfiles are far easier to clean up than a FileStorage is to pack, but they
actually generate *more* filesystem write activity than a FileStorage.

Our long-term solution to this is to not put Z BTrees inside of Berkeley
b-rees, but instead to use Berkeley directly for indexing.  This should
result in a dramatic lowering of logging activity, because Berkeley will
then only be logging that an index key or value changed, not logging the
entire page on which it was changed.  But to implement this long term
solution, we need to integrate an indexing plug-in mechanism to either
ZCatalog (or more likely to the Specialist/Rack model), so that it's
straightforward to, as an application developer, specify the right indexes
for your application's needs.