[ZODB-Dev] FileStorage database unrecoverable?

[Toby Dickenson]

On Friday 28 March 2003 12:58 am, Greg Czajkowski wrote:
> This brings up a question of mine, does packing time
> increase with the size of FileStorage? If so, is it
> linear?

Packing time starts out linear. memory usage is linear too, so packing time 
will drop below linear for reasonably large data sets.

> Now how does PartitionedFileStorage compare?

There should be no substantial difference.

> The reason I am asking is my database will be growing
> to over 10GB within the next six months, it stores one
> OOBtree which consists of
> strings,lists,tuples,longs,ints, and dicts.

> I have been trying to find the best Storage solution
> in terms of first size, then speed of analyzing every
> single BTree item. (I dont need undo support, just
> read/write/del)

For a 10G FileStorage you might need up to 1G of ram to hold FileStorage's 
index. If thats OK, then FileStorage will be hard to beat.

> Berkeley was out of the question because database grew
> way too large. Not enough memory for MappingStorage.
> Haven't tried DirectoryStorage. 

> Read CompressedStorage
> has bugs, 

Do you have evidence that CompressedStorage would help you much? I considered 
adding native compression to DirectoryStorage, but my data sets showed little 
to be gained.

> so I stuck with FileStorage.


-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson