[ZODB-Dev] How to fix "CorruptedDataError: Error reading unknown oid. Found '' at 81036527"?

[Hanno Schlichting]

On Thu, Jul 14, 2011 at 9:40 PM, Jim Fulton <jim at zope.com> wrote:
> On Thu, Jul 14, 2011 at 3:23 PM, Hanno Schlichting <hanno at hannosch.eu> wrote:
>> At Jarn we've used that trick many times to repair broken internals in
>> the intid/keyreference data structures.
>
> Do you have any theories why objects are going away for you?

Not on a low-level.

I know the culprit is five.intid which basically register
zope.lifecycle event subscribers for all IPersistent objects to add
and remove intid registrations.

That code seems to get things wrong. But I've never dug into the code
to figure out under what circumstances this happens. I very much
believe this is wrong application code in five.intid. It does have to
do some funky tricks with raising NotYet exceptions and handling those
at transaction boundaries, as the intid is calculated from the p_oid.
A new object doesn't have a p_oid until the object is added to the
connection.

So this is rather tricky code dealing with low-level assumptions. But
I never got a reproducible case. So I've just fixed the invalid data
whenever I hit it and ripped out five.intid from every project where
possible. To my knowledge no such problems exist in
zope.intid/zope.keyreference.

Hanno