[ZODB-Dev] [FileStorage] Potential data loss through packing

[Jim Fulton]

On Jan 21, 2008, at 5:00 AM, Dieter Maurer wrote:

> Looking at the current (not Jims new) pack algorithm to optimize
> the reachability analysis, I recognized a behaviour that looks
> like a potential data loss through packing.

I will be applying my changes to the trunk soon.


> The potential data loss can occur when an object unreachable at
> pack time becomes reachable again after pack time.
>
> The current pack supports a single use case which can cause such
> an object resurrection: the use of backpointers (probably from  
> "undo").
>
> However, resurrection is possible by other means as well -- e.g.
> by reinstating a historical version which references objects
> meanwhile deleted.

Oooh, that's an interesting case.

> Packing can cause such objects to get lost (resulting in  
> POSKeyErrors).
>
>
> Reinstating a historical version which references to meanwhile
> deleted objects is probably quite a rare situation such
> that the potential data loss seems not to be very critical.
>
> But, potential data loss is nasty, even when the probablity is quite  
> low.


I've already forgotten what I learned looking at the current  
algorithm, although I thought it was immune to this.  I thought it  
"counted" all references after the pack time.

I'm sure the new pack algorithm is immune to this.  It would be  
helpful to design a test case to try to provoke this.

Jim

--
Jim Fulton
Zope Corporation