[ZODB-Dev] Re: [Bug] Another FileStorage packing bug
Dieter Maurer
dieter at handshake.de
Tue Jan 13 05:51:35 EST 2004
Dieter Maurer wrote at 2004-1-13 07:52 +0100:
>We hit another FileStorage packing bug (ZODB 3.1; Zope 2.6.2):
>
> Pack enters an infinite loop by messing up its file position.
>
> In our case, it reads up to a file position near 360.000.000
> and then falls back to a position near 280.000.000 --
> ad infinitum...
>
>I do not yet know what causes this going back to an earlier
>position. I will report when I do...
It has not been an infinite loop.
Instead is was extremely bad runtime behaviour in connection
with "_resolve_backpointer" and "_data_find".
With the ZODB 3.1 implementation (at least) it is easy
to get quadratic runtime behaviour with respect to the number
of objects contained in a transaction.
I attach a patch (with respect to ZODB code as delivered with
Zope 2.6.2) that uses a transaction cache to avoid this
catastrophic behaviour:
In our case, packing took about 10 min with the patch.
Without the patch, we had to abort packing after 2 weeks (!).
To complete the packing process, another 2 weeks would have
been necessary.
This bug usually hides as backpointers are used only for
"Version"s and undo.
--
Dieter
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fspack.pat
Type: application/x-patch
Size: 3305 bytes
Desc: Patch to "ZODB/fspack.py" avoiding quadratic runtime
behaviour for 'resolve_backpointer'
Url : http://mail.zope.org/pipermail/zodb-dev/attachments/20040113/47d51af7/fspack.bin
More information about the ZODB-Dev
mailing list