[ZODB-Dev] Loading objects directly by OID

[Greg Ward]

On 29 October 2001, I said:
> I'm having a problem loading an object directly by OID (ie., __getitem__
> on the Connection).

Well, I've spent all day bashing on this, up to me eyeballs in the guts
of ZODB.  Fun fun fun!  I haven't made a lot of progress, but I have
narrowed the problem down to a fairly simple anomaly.  Specifically, I
want to know:

  Is it ever legal/possible for object A's pickle to contain a reference
  to OID B, but for B to not be loadable via the storage's load()
  method?

If yes, I'm in a bit of a pickle; if no, our database is.  (Pun entirely
intentional.)

Here's a little script that demonstrates the problem:

  import ZODB
  from ZODB.FileStorage import FileStorage
  from ZEO.ClientStorage import ClientStorage
  from ZODB.DB import DB
  from ZODB.referencesf import referencesf

  oid1 = '\x00\x00\x00\x00\x00\x07\xf8\x9a'      # owning object
  oid2 = '\x00\x00\x00\x00\x00\x07\xf5\xc6'      # owned object

  storage = FileStorage("/www/var/mxdb.fs")
  (pickle, rev_id) = storage.load(oid1, None)
  referenced_oids = referencesf(pickle)
  print "%r: has %d byte pickle and %d sub-objects" \
        % (oid1, len(pickle), len(referenced_oids))

  for oid in referenced_oids:
      if oid == oid2:
          print "hello, old friend"
      try:
          (pickle, rev_id) = storage.load(oid, None)
      except KeyError:
          print "%r: load failed (KeyError)" % oid
      else:
          print "%r: loaded %d byte pickle" % (oid, len(pickle))

And here's the output of running this script on a viciously
stripped-down copy of our database:

  '\x00\x00\x00\x00\x00\x07\xf8\x9a': has 1182 byte pickle and 5 sub-objects
  '\x00\x00\x00\x00\x00\x06W\x93': loaded 271 byte pickle
  '\x00\x00\x00\x00\x00\x07\xf8\x9b': loaded 156 byte pickle
  hello, old friend
  '\x00\x00\x00\x00\x00\x07\xf5\xc6': load failed (KeyError)
  '\x00\x00\x00\x00\x00\x06W\x93': loaded 271 byte pickle
  '\x00\x00\x00\x00\x00\x07\xf8\xaf': loaded 70 byte pickle

IOW, the first object ('\x00\x00\x00\x00\x00\x07\xf8\x9a') references
an object ('\x00\x00\x00\x00\x00\x07\xf5\xc6') that cannot be loaded by
the storage.  Is this ever supposed to happen?

        Greg
-- 
Greg Ward - software developer                gward@mems-exchange.org
MEMS Exchange                            http://www.mems-exchange.org