[ZODB-Dev] Re: [Bug] Another FileStorage packing bug

[Dieter Maurer]

Dieter Maurer wrote at 2004-1-13 07:52 +0100:
>We hit another FileStorage packing bug (ZODB 3.1; Zope 2.6.2):
>
>  Pack enters an infinite loop by messing up its file position.
>
>  In our case, it reads up to a file position near 360.000.000
>  and then falls back to a position near 280.000.000 --
>  ad infinitum...
>
>I do not yet know what causes this going back to an earlier
>position. I will report when I do...

It has not been an infinite loop.

Instead is was extremely bad runtime behaviour in connection
with "_resolve_backpointer" and "_data_find".
With the ZODB 3.1 implementation (at least) it is easy
to get quadratic runtime behaviour with respect to the number
of objects contained in a transaction.

I attach a patch (with respect to ZODB code as delivered with
Zope 2.6.2) that uses a transaction cache to avoid this
catastrophic behaviour:

  In our case, packing took about 10 min with the patch.
  Without the patch, we had to abort packing after 2 weeks (!).
  To complete the packing process, another 2 weeks would have
  been necessary.


This bug usually hides as backpointers are used only for
"Version"s and undo.

-- 
Dieter

-------------- next part --------------
A non-text attachment was scrubbed...
Name: fspack.pat
Type: application/x-patch
Size: 3305 bytes
Desc: Patch to "ZODB/fspack.py" avoiding quadratic runtime
	behaviour for 'resolve_backpointer'
Url : http://mail.zope.org/pipermail/zodb-dev/attachments/20040113/47d51af7/fspack.bin