[ZODB-Dev] [zopefoundation/ZODB] 49919d: test for POSKeyError during transaction commit

[Godefroid Chapelle]

Le 05/02/14 13:25, Jim Fulton a écrit :
>>>> Acquisition is added as a test dependency. Any hint how to replicate
>>>> >> >the bug without acquisition is welcome.
>>> >>
>>> >>Define a subclass of Persistent which emulates what Acquisition does, e.g.:
>>> >>
>>> >>   from persistent import Persistent
>>> >>   class Foo(Persistent):
>>> >>       @property
>>> >>       def _p_jar(self): # or whatever attribute trggers
>>> >>           return object()
>> >
>> >What if full replication requires a C extension module?
>> >
>> >(I hope that's not true and that it is possible to reproduce the bug
>> >using some fakes, but I haven't spent the time investigating this.)
> I'm going to dig into this.  I'm baffled by the assertion that this has
> anything to do with readCurrent.

For sure : the POSKeyError happens during connection.commit when 
checking oids stored in Connection._readCurrent mapping.

(see traceback at http://rpatterson.net/blog/poskeyerror-during-commit)

The _readCurrent mapping is populated only by calls to 
Connection.readCurrent method.

In the Plone code base, the only way I found to get that 
Connection.readCurrent method to be called is by adding a key value pair 
to a BTree.

_BTree_set C function is then called, which in turn calls readCurrent by 
inlining the PER_READCURRENT macro.

This calls the cPersistence.c readCurrent function, which in turn calls 
readCurrent method on the ZODB connection.

When setting a key value pair on a new (not already committed) instance 
of a standard BTree, readCurrent method is not called on the connection.

My understanding is that it is due to the fact that _p_jar and _p_oid 
are only set during transaction commit.

However, with a new BTree instance that also inherits from 
Acquisition.Implicit, readCurrent method is called on ZODB connection 
when setting key value pair. The only explanation I found is that this 
instance _p_jar attribute has a value (acquired in a way or another ?).

In this case, when readCurrent is called on an object created during a 
savepoint and this savepoint is rolled back, the oid is leftover in the 
Connection._readCurrent mapping. This leads to the POSKeyError when 
committing later as checkCurrentSerialInTransaction cannot check the 
object since it went away at rollback.

This brings us to the fix I propose: calls to readCurrent should not 
track objects with oid equal to z64.

AFAICS, this is inline with readCurrent API goal which is (if I 
understand well) to ensure that an object manipulated by two connections 
does not get into an incoherent state.

An object that has never been committed cannot be accessed by a second 
transaction hence does not need to participate in readCurrent dance.

This was a very long explanation which I hope will help to confirm the 
fix or to come up with a better one.

PS: keep in mind that english is not my mothertongue.
-- 
Godefroid Chapelle (aka __gotcha) http://bubblenet.be