[ZODB-Dev] sticky objects

[Simon Burton]

Hi,

Following on from my "memory bomb" quandry, I have been further
stress testing my computer with ZODB :)

I am needing to scan through a PersistantList of Persistant objects
each of which has non-persistant (2nd class) data.
As the list is built, the memory usage is low, but after a scan it pulls the whole DB into memory.

It seems that these Persistant objects become sticky, and no amount of
commit/cacheMinimize/db.close/_p_invalidate/del will get rid of them.

Below is a simple example of what I am talking about.

Am I doing something stupid ? Is this just too pathological for ZODB ?

Thanks,

Simon.



#!/usr/bin/env python2.3

from ZODB import FileStorage, DB
from persistent import Persistent
from persistent.list import PersistentList

class PItem(Persistent):
  def __init__(self, s):
    Persistent.__init__(self)
    self.s = s

def main():
  storage=FileStorage.FileStorage('test.fs')
  db = DB(storage, cache_size=400)
  connection = db.open()
  root = connection.root()

  data = PersistentList()
  
  root[0] = data

  x = 'a'*10240

  for i in range(2):
    for j in xrange(10000):
      data.append( PItem( x ) )
    get_transaction().commit()
    print len(data), "items"

  # Memory = 10 Mb
  raw_input( 'enter to scan' )
  for item in data:
    s = item.s
  # Memory = 200 Mb
  raw_input( 'enter to invalidate all' )
  for item in data:
    item._p_invalidate()
  data._p_invalidate()
  connection.cacheMinimize()

  # Memory = 200 Mb
  raw_input( 'enter to quit' )

if __name__=='__main__':
  main()



-- 
Simon Burton, B.Sc.
Licensed PO Box 8066
ANU Canberra 2601
Australia
Ph. 61 02 6249 6940
http://arrowtheory.com