[ZODB-Dev] Disappearing volatile attributes in ZODB 3.6

[Terry Jones]

In trying to debug some code last night and today, I ran across what looks
to be a problem with ZODB 3.6. I'm not 100% sure, and I'm also not running
the SVN code (see below). I'm on Mac OS X 10.4.8 with Python 2.4.3 from the
Darwin Ports collection.

Briefly, I have a class that puts a volatile _v_xxx attribute on instances
via a method called from its __init__ method. I make 4 instances of this
class, and immediately assert(hasattr(instance, '_v_xxx')) on return. All
these assertions succeed.

A little later in my program though, another set of asserts shows that two
of the four volatile attributes are gone.

So of course I went into the debugger to try to see where/when the
attributes were disappearing. But in one long trip through the code, the
attribute I was looking for was still present.

Then I discovered that if I simply moved a copy of the assert statements up
two lines in my code, the attributes do not disappear!

E.g., with this:

    # assert(hasattr(instance, '_v_xxx'))
    func1(x)
    func2(x)
    assert(hasattr(instance, '_v_xxx'))

the assertion fails. But with this

    assert(hasattr(instance, '_v_xxx'))
    func1(x)
    func2(x)
    assert(hasattr(instance, '_v_xxx'))

everything runs just fine.

I.e., the presence or absence of a single assert(hasattr(...)) call causes
a volatile attribute to remain or to disappear. That seemed pretty odd.  I
hesitate to send mail here, having learned the hard way that to resist the
idea that "the system" is probably at fault. But I've spent probably 5
hours trying to find and analyze this - and I'm wondering if it may be a
known issue.

I should add that my class is a subclass of Persistent, but that I remove
the ZODB file storage file before each test run, so the new instances are
not being brought out of a persisted state.

Seeing as I'm running the stock 3.6 ZODB, I went looking for the latest
ZODB, and grabbed svn://svn.zope.org/repos/main/ZODB/trunk. The svn log
includes:

  r66125 | tseaver | 2006-03-22 16:43:22 +0100 (Wed, 22 Mar 2006) | 3 lines
  PersistentMapping was inadvertently pickling volatile attributes
  (http://www.zope.org/Collectors/Zope/2052).

Which sounded promising, especially seeing as my class does contain a
PersistentMapping, and this fix is not in the 3.6 tarball. So I fixed that
by hand in my local 3.6 version and made sure a new created mapping.pyc was
created in the right place, etc. But after making this change, the problem
continues.

So I went to install the latest ZODB. I found 3.7.0b3 at
http://cheeseshop.python.org/pypi/ZODB3/3.7.0b3 and I've made 3 attempts to
install this and none of them worked:

  - using python setup.py install, which gets me raise TypeError, "dist
    must be a Distribution instance", and there's not much help for this to
    be found via google.

  - using easy_install, but I couldn't figure out what argument to give it
    (I tried all of ZODB, ZODB3.7, ZODB3-3.7.0b3, ZODB3-3.7.0b3.xml)

  - using zc.buildout, which I installed via easy_install but when I tried
    reading how to use it at http://www.python.org/pypi/zc.buildout
    I ended up feeling a little overwhelmed.

I also grabbed svn://svn.zope.org/repos/main/ZODB/trunk but doing the
normal python setup.py install in their gets me the raise TypeError, "dist
must be a Distribution instance", as above.

As a result I haven't yet managed to install or test under the latest
ZODB3-3.7.0b3

So I have 2 questions:

  - Does the error I'm seeing ring any bells with ZODB developers?

  - Can someone tell me what I'm doing wrong in trying to install the
    latest ZODB?

If no-one has any suggestions, I'll try to make a small example where it
goes wrong. Apologies for not including real code above, I'm seeing this in
the middle of something much bigger and it's not easy to extract anything
digestible.

Thanks for any help / suggestions,
Terry