No memory leak! (was: Re: [Zope] Memory Leak in Session Data Container)

Andy Yates andy at nnu.com
Mon Apr 17 17:35:33 EDT 2006


Leak, bug, improper configuration, what ever.  ;-)

With the onDelete in place for 1 week now I checked the temporary Cache
Parameters as you requested.  Since I put the onDelete on this server
memory use has still grown, but at a much slower rate.  I think it will
go about 10 days like this instead of 1 before using up all system
memory.

For whatever reason we have 7 caches on this server.  I do not see where
this is configured.
 
/Control_Panel/Database/temporary/manage_cacheParameters
Total number of objects in the database   363761
Total number of objects in memory from all caches    34952
Target number of objects in memory per cache    5000

I flushed the cache and about 1 hour after flushing, memory use has not
gone down at all.  During that hour objects in all caches went from 0 to
15341.  Objects in the transient object container are still holding
steady at 2000-3000.  Before the onDelete they grew without bound.

/Control_Panel/Database/temporary/manage_cacheParameters
Total number of objects in the database    369885
Total number of objects in memory from all caches    15341
Target number of objects in memory per cache    5000

It seems odd to me that the total objects in the db are growing so
large.

If I understand you correctly a workaround would be to change the number
of objects per cache from 5000 to something more reasonable.  Where is
this configured and what is reasonable?  5, 50, 500.  I do not see an
item for this in the config file.  

Should there be any cache at all on an in-memory temp db?


Andy Yates


> -----Original Message-----
> From: Dieter Maurer [mailto:dieter at handshake.de]
> Sent: Friday, April 14, 2006 7:54 AM
> To: Andy Yates; zope at zope.org
> Subject: No memory leak! (was: Re: [Zope] Memory Leak in Session Data
Container)
> 
> Dieter Maurer wrote at 2006-4-11 22:14 +0200:
> >Andy Yates wrote at 2006-4-10 12:43 -0500:
> >> ...
> >>Fortunately, I have an easy way to reproduce the problem and even
better
> >>I've found a work around.  My hope is that the work around will lead
> >>somebody more familiar with the innards of Zope to where the problem
is
> >>located.
> >
> >Congratulations!
> >
> >I will look into this within the next days (in Zope 2.8.1, however).
> 
> I checked this:
> 
>    There is no memory leak related to session handling in Zope 2.8.1!
> 
> > ...
> >The ZODB cache may keep your session objects (and thereby its
> >content) alive.
> 
> Almost surely, this is what happened in Andy's test.
> 
> 
> Here is a description how I checked for the existence of a memory
> leak:
> 
>   I instrumented the pickles in
"tempstorage.TemporaryStorage.TemporaryStorage"
>   such that I can find out the number of pickles stored
>   (similar to the reference counts in "Control_Panel/DebugInfo").
> 
>   I performed a mass test, creating thousands of sessions
>   (similar to Andy's test).
> 
>   The pickle number stored by "TemporaryStorage" grew as expected
>   (as it stores the sessions).
> 
>   I forced all session objects maintained by
"/temp_folder/session_data"
>   to be discarded.
> 
>   Creating a single new session caused almost all "TemporaryStorage"
>   pickles to be released.
> 
>   However, the ZODB caches still contained about 20.000 session
>   objects. It was very surprising (and contrary to my previous
>   message) that the storage flushed most of its pickles
>   while references to them were still in the ZODB caches.
>   This looks like a bug (but not a memory leak).
> 
>   I flushed the ZODB caches.
> 
>   I repeated the test and verified that the number of
>   pickles remaining in "TemporaryStorage" does not increase.
> 
> 
> Results:
> 
>   * no memory leak!
> 
>   * maybe something that looks like a memory leak caused by the ZODB
caches
>     which may hold up to 20.000 (with standard configuration) no
>     longer used objects
> 
>     A more senseful configuration can work around this.
> 
> 
> --
> Dieter


More information about the Zope mailing list