[ZODB-Dev] Re: Invalid object references on brand new Data.fs using fsrefs.py

Tim Peters tim.peters at gmail.com
Wed Jul 26 21:50:20 EDT 2006


[various fsrefs.py failures in Zope 2.9.3]

[Tres]
> I can reproduce this on the Zope 2.9 branch, but not on the 2.8 branch.

I'll note that it can be reproduced easily with a standalone ZODB 3.6 too:

"""
import ZODB
import ZODB.FileStorage
import transaction
from BTrees.OOBTree import OOBTree

st = ZODB.FileStorage.FileStorage("Data.fs")
db = ZODB.DB(st)
cn = db.open()
rt = cn.root()

rt["tree"] = OOBTree()
transaction.commit()
db.close()
"""

Run that, and then:

$ python24\python.exe src\scripts\fsrefs.py Data.fs
oid 0x0L persistent.mapping.PersistentMapping
last updated: 2006-07-27 01:37:35.342000, tid=0x36715E196CAE377L
refers to invalid object:
        oid ('\x00\x00\x00\x00\x00\x00\x00\x01', None) missing: '<unknown>'

> ...
> My *guess* is that this represents a failure in the fsrefs script, since
> the appserver is functioning properly.  Perhaps it has not been updated
> to deal with the API changes between ZODB 3.4.x and ZODB 3.6.x?  The
> delta between the two is pretty small::

I expect it was actually the /entire/ checkin that made that change to
fsrefs.py:

    http://svn.zope.org/ZODB/trunk/src/ZODB/serialize.py?rev=30715&view=rev

That moved and changed the crucial get_refs() function used by
fsrefs.py too.  Unfortunately, I don't believe there are any tests for
fsrefs, and I bet it's been broken ever since that checkin.

This code in fsrefs.py doesn't make sense now:

            ref, klass = info
            if klass is None:
                # failed to unpack
                ref = info
                klass = '<unknown>'

The "failed to unpack" comment is left over from before that checkin,
where it did make sense.  The "ref = info" after it appears to be dead
wrong.  Try this patch?:

Index: src/scripts/fsrefs.py
===================================================================
--- src/scripts/fsrefs.py       (revision 69267)
+++ src/scripts/fsrefs.py       (working copy)
@@ -129,11 +129,8 @@
         data, serial = fs.load(oid, "")
         refs = get_refs(data)
         missing = [] # contains 3-tuples of oid, klass-metadata, reason
-        for info in refs:
-            ref, klass = info
+        for ref, klass in refs:
             if klass is None:
-                # failed to unpack
-                ref = info
                 klass = '<unknown>'
             if ref not in fs._index:
                 missing.append((ref, klass, "missing"))


More information about the ZODB-Dev mailing list