[ZODB-Dev] fsrecover.py and fstest.py Tool with Zope 2.6.1 Btrees

[Jeremy Hylton]

On Mon, 2003-03-03 at 12:06, Kummer, Ben wrote:
> Hi all, 
> we  use Zope 2.6. binary distribution since a while without problems.
> We are checking daily the consistency of our data.fs with the script fstest.py from the utilities folder.
> After upgrading to zope 2.6.1 and packing the database we got an error:
> 183349913 object serialno 0x034b042615f93288 does not matchtransaction id 0x034b08d9f4e711aa
> 
> I tried to recover with utilities/ZODBTools/fsrecover.py and got a lot of errors like this:
> _main__.ErrorFound: bad transaction length at 268110497
> __main__.ErrorFound: invalid status, , at 268110777
> __main__.ErrorFound: bad transaction length at 268110957
> ...
> exceptions.TypeError: __init__() takes exactly 10 arguments (8 given)
> 345837294 bytes removed during recovery
> 
> The recovered data.fs got a size of 4 bytes (original length 345837298 bytes)
> 
> I read in the changelog of Zope 2.6.1: 
> > value-based consistency checker for BTrees was added.  See  the module
> >BTrees.check for the checker and other utilities   for working with BTrees!
> 
> But could not find a tool in the Btree folder to check and recover databases.
> 
> I also tried fsrecover with a fresh zope 2.6.1 with a newly created database with the same result: errors and an empty data.fs
> 
> How could i test an recover data.fs now?

Ken,

I can't really tell what has gone wrong.  The ZODB test suite doesn't
cover fstest and fsrecover, so there may be bugs there that we never
noticed.  (We should make sure we have tests for these tools!)

The fact that the recovered data.fs is exactly 4 bytes long means it
only has the magic number.  The recovery script didn't find anything it
could salvage.  The situation may not be as bad as that sounds, though,
because the recovery script was designed to a small chunk of bad data
somewhere in the file.

Any chance you can send me the data.fs before you ran recover and the
data.fs.old that was generated from the pack?  (I've got a 60GB disk on
my desktop.)

Jeremy