[ZODB-Dev] Storage Benchmarks

[sean.upton@uniontrib.com]

On a related note, anybody have any feelings on the stability of the fsIndex
branch?  I'm really tempted to use this given I have a process that consumes
several hundred MB on a big transaction adding and indexing ~330,000 objects
to a BTreeFolder, though I'm not sure whether or not the current 22bytes per
object in the index is that big of a deal (my ZODB has 1,072,082 objects
between this folder and the Catalog I am using)...  If this buys me
something (significantly less RAM usage, better performance, better Zope/ZSS
restart times), I'll check it out of CVS (I'm using Zope 2.4.2 at the
moment). Thoughts?

Sean

-----Original Message-----
From: Toby Dickenson [mailto:tdickenson@geminidataloggers.com]
Sent: Monday, December 17, 2001 6:23 AM
To: zodb-dev@zope.org
Subject: [ZODB-Dev] Storage Benchmarks


Ive recently been performing some benchmarks and systems tests comparing 
FileStorage (the usual version and Jim's fsIndex branch) against Berkeley 
Full and Packless.

Full details are at:
http://www.zope.org/Members/htrd/benchmarks/storages

A summary of the highlights:

* Storage choice had little impact on overall system throughput  
Specifically, I didnt see the performance problems that Chris W reported 
(although my databases are somewhat smaller).

* Jim's fsIndex branch to FileStorage was the suprise high-performer. It has

the lowest memory usage of all storages tested.

* All three BerkeleyStorage implementations have very high memory 
requirements for packing: higher than FileStorage. Im not convinced that 
these current implementations deserve their 'high scalability' reputation.


Im most familiar with BerkeleyStorage Full, where the memory usage issues 
come from using a python dictionary with one element per object in the 
database.... obviously not a scalable strategy.

I am suprised that I had not heard about this limitation before..... Does 
anyone with a 'large' BerkeleyStorage know how much memory it uses when 
packing?

_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://lists.zope.org/mailman/listinfo/zodb-dev