[ZODB-Dev] POSKeyError when transaction.commit after savepoint.rollback and connection.readCurrent

[Godefroid Chapelle]

Le 17/09/13 10:44, Godefroid Chapelle a écrit :
> Hi,
>
> Plone 3 to 4 migration occasionally triggers a POSKeyError during
> transaction commit. See
> http://rpatterson.net/blog/poskeyerror-during-commit
>
> I encountered the issue yesterday.
>
> I committed a test and a fix in branch 3.10.
>
> https://github.com/zopefoundation/ZODB/commit/6457bcfd07b3b77f240cbc6d60f222f0d40e8d5d
>
>
> Before the fix, the test triggered a traceback very similar to the one
> seen when migrating Plone.
>
> As I am not an expert of ZODB internals, I'd like someone to review my
> fix even if the test suite does not break.

I confirm that I am not a ZODB internals expert :-(

The fix does avoid the issue triggered by my test.

But does NOT fix the Plone migration issue. IOW, the test is not 
reproducing the issue triggered by the Plone migration.

This is what I have found :

At the end of the Connection.commit() method, oids found in 
Connection._readCurrent mapping are checked for mismatch between the 
committed tid and the one stored in _readCurrent mapping.

If there is a mismatch, the object must be invalidated in cache.

In the Plone migration issue, when getting to the step where the 
committed tid is fetched, a POSKeyError is triggered by getTid() method.

I found out that the tid stored in _readCurrrent for this falty oid is 
equal to z64 (eight nul chars).

This is what I deduce :

the oid, tid pair (where tid is null) corresponds to an object that was 
added to the transaction but not completely cleaned up by the 
transaction rollback

(I am looking at rollback because my migration does call 
savepoint.rollback() five times and there are five oids with null tid 
leftover in Connection._readCurrent when the POSKeyError is triggered. 
But this might just be a coincidence.)

Any hint where to add the call to pop those falty oids out of 
_readCurrent during transaction rollback, or to enlighten me about 
things I misunderstand is welcome.

Regards
-- 
Godefroid Chapelle (aka __gotcha) http://bubblenet.be