[ZODB-Dev] problems w/ Pickle based deep-copy of Blob objects
Rob Miller
robm at openplans.org
Tue Aug 19 15:05:48 EDT 2008
hi all,
i've been experimenting w/ getting ZODB 3.8's blob storage support to work w/
the openplans.org stack, and have hit a snag about which i could use some
feedback.
the problem is that CMFEditions, which we use to version our content, uses
Pickler.dump() and Unpickler.load() to create deep copies of objects, which
then get stored as historical versions of those objects. the heart of the
issue is probably best illustrated by an interactive session:
>>> blob
<ZODB.blob.Blob object at 0xa0c1230>
>>> blob.readers
[]
>>> blob.writers
[]
>>> stream = StringIO()
>>> p = Pickler(stream, 1)
>>> p.dump(blob)
<cPickle.Pickler object at 0x9fb9cc8>
>>> u = Unpickler(stream)
>>> stream.seek(0)
>>> copy = u.load()
>>> copy
<ZODB.blob.Blob object at 0xa0c12a8>
>>> copy.readers
>>> copy.writers
'readers' and 'writers' are defined as None on the Blob class, but they get
initialized as empty list instance variables by the __init__ and __setstate__
methods. neither of these are triggered by the deep copy process, however,
and apparently the __getstate__ and __setstate__ implementations prevent these
instance variables from propagating to the pickled copy.
this seems okay, at first, but if anything is done w/ the transaction that
triggers the copy's _p_invalidate method to be called, it blows up with a
TypeError on the following:
for ref in self.readers+self.writers:
the problem seems to go away w/ a simple change:
for ref in (self.readers or []) + (self.writers or []):
this seems safe to me; it's not actually changing the state of any objects,
and it's only going to have an impact when an object w/ no readers and writers
is trying to be invalidated. but i don't know enough about what's going on
here to feel confident that this is the right solution, that i'm not
introducing some subtle ZODB bug that's going to bite me (or someone else)
down the road.
can anyone tell me whether or not this seems safe? if this isn't the right
approach, what would you suggest? (my second guess would be to fix the
__getstate__ and __setstate__ implementation so that these instance variables
make it through the pickling process in the first place...)
any help very appreciated, thanks!
-r
More information about the ZODB-Dev
mailing list