[ZODB-Dev] RE: Corrupt db when trying to pack

[Gerry Kirk]

Gerry Kirk
Project Manager
www.faithonline.com 
www.diocesessm.org



> -----Original Message-----
> From: Tim Peters [mailto:tim at zope.com] 
> Sent: October 4, 2004 2:18 PM
> To: 'Gerry Kirk'
> Cc: zodb-dev at zope.org
> Subject: RE: Corrupt db when trying to pack
> 
> 
> [Tim Peters]
> >> For the future, take the recommended practices listed on the link 
> >> above to heart.  For example, had you made daily backups 
> all along, 
> >> repairing "impossible" corruption in the first 300MB of a 700MB 
> >> FileStorage probably would have been straightforward.
> 
> [Gerry Kirk]
> > We do a nightly backup of the whole database. How does that make 
> > repairing easier?
> 
> As you learned at
> 
     http://zope.org/Wikis/ZODB/FileStorageBackup
>
> a FileStorage changes only by appending new data.  So, e.g., if you
drop in  
> incompatible memory chips and that ends up corrupting data in the
first 300MB of 
> a 700MB FileStorage (which "should be" impossible, precisely because a

> FileStorage changes only by appending), you can very likely construct
a new 
> database that's in perfect shape by pasting together the first 300MB
from a good 
> backup and the tail end (everything after its first
> 300MB) of your current database.  That should work fine provided you
haven't 
> packed since the backup.

Makes sense, I think. In my case, I would think the corruption is
somewhere in the middle of the file, so I would need to extract that
part out and merge the front and end parts together.

> That's about detecting damage, so is a different issue than repairing
damage.  
> If I were you, yes, I'd run fstest and fsrefs on each nightly backup
too.  This 
> is catastrophe insurance, and like all such seems like a waste of
effort -- 
> until a catastrophe occurs.

Yes, but my point is, I'm not detecting damage now, so maybe there is
nothing to repair. I tried running a pack on a copy of a backup, and got
this error:

Traceback (innermost last):
  Module ZPublisher.Publish, line 100, in publish
  Module ZPublisher.mapply, line 88, in mapply
  Module ZPublisher.Publish, line 40, in call_object
  Module App.ApplicationManager, line 428, in manage_pack
  Module ZODB.DB, line 528, in pack
  Module ZODB.FileStorage, line 1557, in pack
  Module ZODB.fspack, line 709, in pack
  Module ZODB.fspack, line 754, in copyToPacktime
  Module ZODB.fspack, line 80, in _read_num
IOError: [Errno 22] Invalid argument

Googling didn't track down what this problem is.

Thanks for all your help so far, Tim! I really appreciate it.
- Gerry