[ZODB-Dev] Zope memory usage

[Izak Burger]

Hi all,

I'm sure this question has been asked before, but it drives me nuts so I 
figured I'll ask again. This is a problem that has been bugging me for 
ages. Why does zope memory use never decrease? Okay, I've seen it 
decrease maybe by a couple megabyte, but never by much. It seems the 
general way to run zope is to put in some kind of monitoring, and 
restart it when memory goes out of bounds. In general it always uses 
more and more RAM until the host starts paging to disk. This sort of 
baby-sitting just seems wrong to me.

It doesn't seem to make any difference if you set the cache-size to a 
smaller number of objects or use a different number of threads. Over 
time things always go from good to bad and then on to worse. I have only 
two theories: a memory leak, or an issue with garbage collection (python 
side).

It is possible that this is not a bug, my reasoning goes like this: The 
OS exposes a "virtual memory" picture to the application, ie, the 
application does not know how much of the RAM is real and how much is 
paged out. All it does is call malloc every now and then. Combined with 
a kernel that allows overcommit (which is in general a good thing), I 
see this going only one way.

I'm hoping someone on this list has some insights into the issue.

regards,
Izak