[Zope-dev] zope2/zodb cache size question

[Jim Fulton]

On Thu, Apr 22, 2010 at 9:00 AM, Jürgen Herrmann
<Juergen.Herrmann at xlhost.de> wrote:
>
> hi there!
>
> today i ran a check script which iterates over approx 150000 objects
> and does some sanity check calculations on these. during this loop
> i saw the zope process use up to about 4.5gb memory. the database
> has ~3.5million objects in it.
>
> i set the zodb cache size for the mount point in question to 10000
> objects. obviously this limit is not honoured during one transaction:

Right.

> so two questions here:
> - would the byte-limited zodb cache setting help here?

No. The limits aren't really limits. They are more like
suggestions. :)  In particular, they are only enforced at
transaction (or subtransaction) boundaries, when a connection is
opened or closed, or when applications call certain APIs explicitly.

> - if no - how can i iterate over a big set of objects without
>  forcing them to stay in the cache for the whole transaction?
>  after all i just need each object once during the iteration.

As you're iterating through the objects, if you know you aren't going
to need an object again or otherwise think it would be OK to free it
from memory, you can call _p_deactivate on it. If the object hasn't
been modified, then it's state will be released and it will become a
ghost.

Jim

-- 
Jim Fulton