[ZODB-Dev] Storage Benchmarks

[Chris Withers]

Toby Dickenson wrote:
> 
> * Storage choice had little impact on overall system throughput
> Specifically, I didnt see the performance problems that Chris W reported
> (although my databases are somewhat smaller).

Yeah, I think what was happening here is that BerkleyStorage was chewign up RAM,
causing the sytem to thrash, hence the poor performance...

> * Jim's fsIndex branch to FileStorage was the suprise high-performer. It has
> the lowest memory usage of all storages tested.

I can back that up, it's still not fast enough for my needs, but the memory
usage is a lot less :-)

> * All three BerkeleyStorage implementations have very high memory
> requirements for packing: higher than FileStorage. Im not convinced that
> these current implementations deserve their 'high scalability' reputation.

I'd agree with that :-S

> Im most familiar with BerkeleyStorage Full, where the memory usage issues
> come from using a python dictionary with one element per object in the
> database.... obviously not a scalable strategy.

Indeed ;-(

> I am suprised that I had not heard about this limitation before..... Does
> anyone with a 'large' BerkeleyStorage know how much memory it uses when
> packing?

I've never managed to successfully pack a large BerkleyStorage ;-)

cheers,

Chris