[ZODB-Dev] Making ZODB / ZEO faster

[Shane Hathaway]

Jim Fulton wrote:
> I find this a bit confusing.  For the warm numbers, It looks like ZEO didn't
> utilize a persistent cache, which explains why the ZEO numbers are the
> same for hot and cold. Is that right?

Yes.  It is currently difficult to set up ZEO caches, which I consider 
an issue with this early version of zodbshootout.  zodbshootout does 
include a sample test configuration that turns on a ZEO cache, but it's 
not possible to run that configuration with a concurrency level greater 
than 1.

> What poll interval are you using for relstorage in the tests?
> 
> Assuming an application gets reasonable cache hit rates, I don't see any
> meaningful difference between ZEO and relstorage in these numbers.

You are entitled to your opinion. :-)  Personally, I have observed a 
huge improvement for many operations.

>>> Second, does the test still write and then read roughly the same
>>> amount of data as before?
>> That is a command line option.  The chart on the web page shows reading and
>> writing 1000 small persistent objects per transaction,
> 
> Which is why I consider this benchmark largely moot. The database is
> small enough
> to fit in the servers disk cache.  Even the slowest access times are
> on the order of .5
> milliseconds. Disk accesses are typically measured in 10s of
> milliseconds.  With magnetic
> disks, for databases substantially larger than the server's ram, the
> network component
> of loading objects will be noise compared to the disk access.

That's why I think solid state disk is already a major win, 
economically, for large ZODB setups.  The FusionIO cards in particular 
are likely to be at least as reliable as any disk.  It's time to change 
the way we think about seek time.

Shane