[ZODB-Dev] Understanding the ZODB cache-size option

[Jim Fulton]

On Mon, Mar 22, 2010 at 2:05 PM, Jeff Shell <jeff at bottlerocket.net> wrote:
> Are there any metrics about how to set the ZODB 'cache-size' (or cache-size-bytes) option? We've been using '5000' (arbitrarily chosen) for our Zope 3.4-ish app servers. We have access to zc.z3monitor which can output the "number of objects in the object caches (combined)" and "number of non-ghost objects in the object caches (combined)".
>
> But I don't know understand how to interpret those numbers and use them to make better settings.

The more interesting numbers are the number of objects loaded per unit
time and the number of records added to the cache per unit time, which
you can also get from zc.z3monitor.

Basically, object loads are somewhat expensive and loads from the ZEO
server (and into the cache) are quite a bit more expensive.  You want
to reduce these numbers by:

- Analyzing your application to make sure it isn't loading more objects than
  it  needs to.

- Adjusting ZODB and ZEO cache sizes, subject to resource limitations.

Some notes:

- The Python profiler (cProfile) can help analyze object loading,
especially if you have good profiler analysis tools. Unfortunately,
I'm not aware of one that's included with Python.  My nutty
pstats2html script,

  http://svn.zope.org/Sandbox/J1m/pstats2html.py?view=auto

which I wrote before I learned javascript :),  can show you who's
calling Connection.setstate, and thus, who's doing lots of loads.

- Making ZEO persistent cache sizes too large can cause excessive IO
  on process restart.

Jim

-- 
Jim Fulton