[ZODB-Dev] Newbie question - Filestorage size ?

[Paul Winkler]

On Sun, Nov 21, 2004 at 08:14:23PM -0600, Luis Machado wrote:
> Hi,
> 
> I would like to know how big can a filestorage grow ? 

I've had filestorage up to 3.5 GB at times
and I'm sure others can top that, I've heard of some much larger 
like 20 GB or so.  I don't remember for sure.

> I'm working on an
> application where I'm only adding data to a ZODB database. I'm not
> modifying existing data. In this application, there is one task writing
> to the database and many clients reading from it.
> 
> Looks like ZODB is appropriate for this application, as I need a Tree
> database due to the nature of the data.
> 
> If I'm only adding new data and no modifying nor deleting data from the
> database, do I need to regularly pack the database ?

Not if you really never delete nor modify.
If all you ever do is add new persistent instances, packing will 
not accomplish anything.
 
> What are the chances that the file gets corrupted ? 

FileStorage is pretty robust in general. I don't know how to quantify it.  
I've run had a few problems from time to time... e.g. the packing
bug in zope 2.6.2 and thereabouts that would leave POSKeyErrors
lying around.  
And some earlier problems that may have been
caused by bugs in Versions, not sure about that, it was
a long time ago.

For a year or so I have been using DirectoryStorage, which
is less efficient both in terms of space and CPU time,
but I have had zero problems with it.
AFAIK there has been exactly one reported case of a potentially 
data-loss-causing bug in the wild, and that was in a pretty 
weird (and wrong) situation and the user was able to recover
the data pretty easily. And it's fixed now 
(see the dirstorage-users mailing list, nov. 9th of this year.)

> Is this related to
> the size of the file ?

That should be irrelevant except if your OS and/or your installation
of python doesn't support large files. In that case, transactions
will fail once you hit 2 GB.
 
> Is there a way to read data from a compressed backuped file ? Like MySQL
> that after compressing the table I can still read from it.

Not taht I know of.
 
-- 

Paul Winkler
http://www.slinkp.com