[ZODB-Dev] Does the ZODB eat memory?

[Shane Hathaway]

Ulla Theiss wrote:
> Hello list,
> 
> in our product (an additionally in an other process) we use stand alone
> ZODB-Storages.
> 
> Opening an closing the ZODB several times after some operation, the
> memory of the python process
> increases.
> 
> We have reduced the problem to the following code. Performing it, the
> memory increases from little bit less
> than 4.000 KB to about 28.000 KB.
> 
> Have we forgotton to close anything, or made anything else wrong? Has
> anybody an idea how to fix or work
> around the problem?

Since I'm working on a memory leak today, and this might have been 
related, I decided to investigate.  I see the same memory bloat and it's 
due to uncollectable cycles.  I came up with some changes that force the 
cycles to be broken, which might be valuable for Jeremy.

First, Connection._breakcr() is incomplete.  Here is my version, which 
deletes _root_ and _db in addition to the other attributes.

     def _breakcr(self):
         try: del self._cache
         except: pass
         try: del self._incrgc
         except: pass
         try: del self.cacheGC
         except: pass
         try: del self._root_
         except: pass
         try: del self._db
         except: pass

Second, perhaps DB.close() ought to call _breakcr() on each connection.

Third, to finally get Ulla's program to stop eating memory, I had to 
call DB.cacheMinimize() before calling db.close().  I'm not sure calling 
cacheMinimize() in close() is a good idea, since cacheMinimize() can 
take quite a while to complete sometimes.  cPickleCache apparently does 
not participate in cyclic garbage collectection.

With these changes, on Python 2.2.2 with the Zope/ZODB HEAD, Ulla's 
program used a constant 3 MB of RAM.

Shane