[ZODB-Dev] Speeding up ZODB (was "redis cache for RelStorage")

[Shane Hathaway]

On 05/06/2011 06:22 AM, Pedro Ferreira wrote:
> But isn't RelStorage supposed be slower than FileStorage/ZEO?

No, every measurement I've tried suggests RelStorage (with PostgreSQL or 
MySQL) is faster than ZEO on the same hardware.  ZEO has certainly 
gotten faster lately, but RelStorage still seems to have the advantage 
AFAICT.  OTOH, the speed difference is not dramatic.  For many apps it's 
not even noticeable.

>> But remember that throwing more caches at the problem isn't a
>> solution. It's likely the way you store or query the data from the
>> database that's not optimal.
>
> I agree, many things could be improved regarding the data structures we
> use. However, it is also true that we have a large number of objects
> that are rarely changed, and that there is no need to fetch from the DB
> if we can keep them in memory.

It sounds like you primarily need a bigger and faster cache.  If you 
want to make minimal changes to your setup, try increasing the size of 
your ZEO cache and store the ZEO cache on either a RAM disk (try mount 
-t tmpfs none /some/path) or a solid state disk.  Remember that seek 
time is 5-10 ms with spinning drives, so putting a ZEO cache on a 
spinning drive can actually kill performance.

Shane