[ZODB-Dev] __setstate__() and _p_changed

[Greg Ward]

Hi --

I'm trying to understand how __setstate__() methods interact with
_p_changed attributes.  We write a lot of __setstate__()'s to handle
database evolution: if a class adds an attribute foo, then we add this
to the class:

    def __setstate__ (self, dict):
        self.__dict__.update(dict)
        if not hasattr(self, 'foo'):
            self.foo = None

in order to transparently add the new attribute to old objects.  So far,
this has worked great.

But now I have a class where it's *not* working, and I don't understand
why.  Here's the original __setstate__() method:

    def __setstate__ (self, dict):
        self.__dict__.update(dict)

        if not hasattr(self, '_process_name_idx'):
            self._process_name_idx = BTree.BTree()

The trouble is that _p_changed is *not* being set on this object, even
though it's obviously being changed (through a setattr).  Or rather, I
should say that _p_changed is 0 after __setstate__() completes.  Here's
an instrumented __setstate__():

    def __setstate__ (self, dict):
        self.__dict__.update(dict)
        if not hasattr(self, '_process_name_idx'):
            print "ProcessLibrary.__setstate__: creating _process_name_idx"
            print "  _p_changed =", self._p_changed
            self._process_name_idx = BTree.BTree()
            print "  _p_changed =", self._p_changed

and here's what happens when I access the object in an interactive
session:

>>> connection.root()['process_lib']
ProcessLibrary.__setstate__: creating _process_name_idx
  _p_changed = 1
  _p_changed = 1
<ProcessLibrary at 82a35f8>

Note that __setstate__() did what it said:

>>> process_lib._process_name_idx
<BTree object at 0x8282050>
>>> len(process_lib._process_name_idx)
0

but _p_changed has become false:
>>> process_lib._p_changed
0

Factors that may or may not be relevant:
  * the class in question (ProcessLibrary) is a root class, ie.
    "connection.root()['process_lib']"
    returns the sole instance of ProcessLibrary in our database
    (connection is a ZODB.Connection.Connection instance, and
    connection.root() is a PersistentMapping with eight
    "root objects" in it)
  * the attribute in question starts with "_" -- although I tried it
    without the underscore and got the same behaviour
  * the value in question is a BTree -- although I tried it with
    an ordinary dictionary and a plain integer and got the same
    behaviour

I don't think it's the attribute name or value -- if I change the attr
assignment to "self.foo = 37", I have the same problem.

I wonder if there is something I should know about __setstate__() on
"root objects", though -- we use this trick a lot, and _p_changed seems
to be properly set on non-root objects after a __setstate__() that makes
changes.  What's going on here?

        Greg
-- 
Greg Ward - software developer                gward@mems-exchange.org
MEMS Exchange                            http://www.mems-exchange.org