[ZODB-Dev] ZODB packing error

[Tim Peters]

[Tim Peters]
>> - Transaction T1 loads an object O with a reference to persistent
>>   object P, and modifies some other attribute of O (it does not load
>>   P itself).
>>
>> - Other transactions remove the reference from O to P, and all other
>>   references to P, from the database.
>>
>> - A pack to current time is done.  Now the only reference to P lives
>>   in memory, inside T1's view of O.
>>
>> - T1 finally tries to commit.
>>
>> At this point, I see nothing to prevent O from having a "dangling
>> reference" to P -- T1 itself never loaded P into memory, and the
>> database no longer contains any trace of P, but O still references
>> it.  That probably isn't good.

[Dieter Maurer]
> Should this not give a (Write)ConflictError?
>
>   Both "T1" and the transaction removing the reference from O to
>   P modify "O".

I expect the outcome depends on the details of the conflict resolution
implemented by O.  If, e.g., O is a BTree, the removal of P (maybe that's
one of O's keys) probably wouldn't raise a ConflictError.  But in that case,
P would also be removed from the final (post-resolution) committed state, so
no dangling reference either.

Let me try to construct an actual dangling reference ... OK, this seems
easiest with a couple ZEO clients, although the same could clearly be done
with threads (it's easier to show relative timing via switching "by hand"
among ZEO clients):

First I started a ZEO server, on localhost:8888.

Now start a ZEO client:

>>> import ZODB
>>> from ZEO import ClientStorage
>>> st = ClientStorage.ClientStorage(('localhost', 8888))
>>> db = ZODB.DB(st)
>>> conn = db.open()
>>> from BTrees.OOBTree import *
>>> O = conn.root()['tree'] = OOBTree()
>>> O[1] = P = OOBTree()
>>> get_transaction().commit()

Now a second ZEO client:

>>> import ZODB
>>> from ZEO import ClientStorage
>>> st = ClientStorage.ClientStorage(('localhost', 8888))
>>> db = ZODB.DB(st)
>>> conn = db.open()
>>> O = conn.root()['tree']
>>> P = O[1]

Back to the first client:

>>> del O[1]  # so nothing current refers to P anymore
>>> get_transaction().commit()
>>> db.pack() # and everything that used to refer to P also vanishes

Back to the second client:

>>> conn.sync()
>>> O[2] = P      # this client still has a ref to P in memory
>>> get_transaction().commit()  # and adds it to the BTree
>>> st.close()

Back to the first client:

>>> st.close()

Now fsrefs.py complains about a dangling reference (oid 1 is O, oid 2 is
(was) P):

oid 0x1L <type 'BTrees._OOBTree.OOBTree'>.
last updated: 2004-04-05 20:44:05.960000, tid=0x3543A7C196DE8CCL
refers to invalid object:
        oid 0x2L missing: 'None'