[ZODB-Dev] Speeding up ZODB (was "redis cache for RelStorage")

[Jim Fulton]

On Fri, May 6, 2011 at 4:07 PM, Shane Hathaway <shane at hathawaymix.org> wrote:
> On 05/06/2011 06:22 AM, Pedro Ferreira wrote:
>> But isn't RelStorage supposed be slower than FileStorage/ZEO?
>
> No, every measurement I've tried suggests RelStorage (with PostgreSQL or
> MySQL) is faster than ZEO on the same hardware.  ZEO has certainly
> gotten faster lately, but RelStorage still seems to have the advantage
> AFAICT.  OTOH, the speed difference is not dramatic.  For many apps it's
> not even noticeable.
>
>>> But remember that throwing more caches at the problem isn't a
>>> solution. It's likely the way you store or query the data from the
>>> database that's not optimal.
>>
>> I agree, many things could be improved regarding the data structures we
>> use. However, it is also true that we have a large number of objects
>> that are rarely changed, and that there is no need to fetch from the DB
>> if we can keep them in memory.
>
> It sounds like you primarily need a bigger and faster cache.  If you
> want to make minimal changes to your setup, try increasing the size of
> your ZEO cache and store the ZEO cache on either a RAM disk (try mount
> -t tmpfs none /some/path) or a solid state disk.  Remember that seek
> time is 5-10 ms with spinning drives, so putting a ZEO cache on a
> spinning drive can actually kill performance.

If this on Linux and you have enough RAM, the data should be in the
disk cache anyway, so I don't see any benefit to a RAM disk.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton