[Checkins] [zopefoundation/persistent] fe2219: On deactivate release in-slots objects too

GitHub noreply at github.com
Wed Aug 10 16:41:55 CEST 2016


  Branch: refs/heads/master
  Home:   https://github.com/zopefoundation/persistent
  Commit: fe2219f45678bed2dc022d1519747031d5416e4c
      https://github.com/zopefoundation/persistent/commit/fe2219f45678bed2dc022d1519747031d5416e4c
  Author: Kirill Smelkov <kirr at nexedi.com>
  Date:   2016-07-22 (Fri, 22 Jul 2016)

  Changed paths:
    M persistent/cPersistence.c
    M persistent/persistence.py
    M persistent/tests/test_persistence.py

  Log Message:
  -----------
  On deactivate release in-slots objects too

On ._p_deactivate() and ._p_invalidate(), when an object goes to ghost
state, objects referenced by all its attributes, except related to
persistence machinery, are released, this way freeing memory (if they
were referenced only from going-to-ghost object).

That's the idea - an object in ghost state is simply a stub, which loads
its content on first access (via hooking into get/set attr) while
occupying minimal memory in not-yet-loaded state.

However the above is not completely true right now, as currently on
ghostification only object's .__dict__ is released, while in-slots objects
are retained attached to ghost object staying in RAM:

    ---- 8< ----
    from ZODB import DB
    from persistent import Persistent
    import gc

    db = DB(None)
    jar = db.open()

    class C:
  def __init__(self, v):
      self.v = v
  def __del__(self):
      print 'released (%s)' % self.v

    class P1(Persistent):
  pass

    class P2(Persistent):
  __slots__ = ('aaa')

    p1 = P1()
    jar.add(p1)
    p1.aaa = C(1)

    p2 = P2()
    jar.add(p2)
    p2.aaa = C(2)

    p1._p_invalidate()
    # "released (1)" is printed

    p2._p_invalidate()
    gc.collect()
    # "released (2)" is NOT printed     <--
    ---- 8< ----

So teach ghostify() & friends to release objects in slots to free-up
memory when an object goes to ghost state.

NOTE PyErr_Occurred() added after ghostify() calls because
pickle_slotnames() can raise an error, but we do not want to change
ghostify() prototype for backward compatibility reason - as it is used
in cPersistenceCAPIstruct.

( I hit this bug with wendelin.core which uses proxies to load
  data from DB to virtual memory manager and then deactivate proxy right
  after load has been completed:

  https://lab.nexedi.com/nexedi/wendelin.core/blob/f7803634/bigfile/file_zodb.py#L239
  https://lab.nexedi.com/nexedi/wendelin.core/blob/f7803634/bigfile/file_zodb.py#L295 )


  Commit: 2cff7d1bc751074f17292da2d9df6f0d7c335861
      https://github.com/zopefoundation/persistent/commit/2cff7d1bc751074f17292da2d9df6f0d7c335861
  Author: Tres Seaver <tseaver at palladion.com>
  Date:   2016-08-10 (Wed, 10 Aug 2016)

  Changed paths:
    M persistent/cPersistence.c
    M persistent/persistence.py
    M persistent/tests/test_persistence.py

  Log Message:
  -----------
  Merge pull request #44 from navytux/y/invalidate-slots

On deactivate release in-slots objects too


Compare: https://github.com/zopefoundation/persistent/compare/b29c0ca244a2...2cff7d1bc751


More information about the checkins mailing list