[ZODB-Dev] Weird errors with Zope 2.7.7

[Chris Withers]

Hi Tim,

Tim Peters wrote:
> As before, this "Shouldn't load state ..." error is almost certainly due to
> a logic error in some product you're using, or in Zope.  Take this message
> as meaning exactly what it says:  something is trying to work with a
> persistent object after the Connection it came from was closed.  That's an
> app problem, not a ZODB problem.  Have you tracked down what kind of object
> (say) oid 0x6c33b4 is?  That may give a good clue.

Okay, had a spat of 6 of these in the same minute (significant?), with 
no particular known trigger, on one of the machines in a cluster this 
morning, so I went digging.

I started by writing two little methods for use in a zopectl debug session:

from binascii import unhexlify
def oid_unrepr(oidr):
     if oidr.startswith('0x'):
         oidr = oidr[2:]
     h = unhexlify(oidr)
     return '\x00'*(8-len(h))+h

def oidr2obj(app,oidr):
     return app._p_jar[oid_unrepr(oidr)]

First question, would you mind if I added oid_unrepr to ZODB.utils? I 
couldn't find this anywhere, and it's kinda handy to have ;-)

Anyway, armed with my little friends, I went digging for the types of 
the oids causing problems.
For kicks, I started off with 0x6c33b4 that you mention above. Doing the 
following in a zopectl debug session gave:

 >>>obj = oidr2obj(app,'0x6c33b4')
 >>>>>> obj
<X instance at b64d4f20>

...which is just an AT-based Plone content object.

Okay, now the 6 from today:

 >>>>>> oidr2obj(app,'0x66b0ad')

<OOBTree object at 0xb50752c0>

...upon digging further, this was the OOBTree that stores the contents 
of one of our CMFBTreeFolder2. Two identical errors for this occurred 
straight after each other on the server in question.

The rest were all roughly the same, and shortly followed the error for 
0x66b0ad:

 >>>>>> obj = oidr2obj(app,'0xab653f')
ZODB.POSException.POSKeyError: 0xab653f
 >>>>>> obj = oidr2obj(app,'0xab4e40')
ZODB.POSException.POSKeyError: 0xab4e40
 >>>>>> obj = oidr2obj(app,'0xab653f')
ZODB.POSException.POSKeyError: 0xab653f
 >>>>>> obj = oidr2obj(app,'0xab4e40')
ZODB.POSException.POSKeyError: 0xab4e40

Now, my guess, and I do mean guess, is that maybe these were objects in 
the TemporaryStorage that we serve over ZEO to back the session data 
container...

However, other than documenting what I've found, and maybe adding a 
little microutility to ZODB.utils, I haven't really gotten anywhere :-(

Any ideas?

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk