[ZODB-Dev] Newbe ZODB questions

[Greg Ward]

On 30 May 2001, Patrick Down said:
> I am looking at ZODB as a storage solution for a python application I am
> writing (not web based).  Currently I use a home grown system that
> serializes objects to and from an XML text file.  While this was great for
> initial development it has become slow for larger files sizes.  I am looking
> at ZODB as a replacement solution for my document storage.  However I have a
> couple of questions:

If you decide against ZODB in the end, there are definitely persistence
mechanisms that are easier and much more efficient than that!  I'm
currently hacking on a bug-tracking database that uses a Berkeley DB
hash file full of Python pickles.  Now that there's *finally* decent
Python support for DB 3.x (see pybsddb.sourceforge.net), this works like
a charm.  It's a tad more burdensome than ZODB (need to explicitly
save/load and cache/uncache objects), but it's far less mysterious and
depends on far less code.  It's also fast enough and small enough -- I
think; this system hasn't been put into production use yet.  ;-)

That said, I still think ZODB is the bee's knees for large systems where
you don't want to be forever screwing around with saving/loading
objects, and where you need transactions.

> 2. I see in the news groups and mail archives some discussion about ZODB
> being much better for more "read" and less "write" intensive applications.
> My application is fairly read/write balanced.  However only small
> subsections of the document are changed at any one time.  A large document
> might contain as many as 10K to 20K objects but only 100 might be modified
> and committed at one time.  Should ZODB perform well in a situation like
> this?  I also have seen discussion that the FileStorage grows with every
> write and needs to be periodically compressed.  Is this true?

Dunno about most of your questions, but it is true that FileStorage is
append-only.  I gather the benefit of this is that it makes it really
hard to trash the storage -- if your system crashes in the middle of a
write, you just lose that write.  Or something like that.  But the
obvious drawback is that you have to pack the storage periodically.  (We
do it nightly on our production system.)

        Greg
-- 
Greg Ward - software developer                gward@mems-exchange.org
MEMS Exchange                            http://www.mems-exchange.org