[ZODB-Dev] FileStorage database unrecoverable?

[Greg Czajkowski]

Thanks so much Shane,

You got me thinking about trying python 2.2 with
ZODB3.1.1 for 2.2. So I ran fsrecover with pack, which
worked great. So now Im running ZEO with python 2.2 on
the server and all the clients are running python 2.1
ZEO 

Thanks again for your help.
--Greg

--- Shane Hathaway <shane@zope.com> wrote:
> Greg Czajkowski wrote:
> > Hi all, I would appreciate any help.
> > 
> > my database grew over 2GB on NT causing it to
> become
> > corrupted. I am running ZODB 3.1.1 on NT.
> > ZODB.FileStorage.packed_version = 'FS21'
> > 
> > I am using FileStorage because it packs python
> > strings,lists,dict,ints in the least amount of
> space.
> > How can I prevent issues when my database grows
> larger
> > than 2GB with the ZEO client/server dying? Should
> I
> > use DirectoryStorage instead? Does it pack the
> data
> > just as efficiently? 
> > 
> ...
> > 
> > Loading the storage fails:
> > ------------------------------------------
> > storages = {'1' :
> > FileStorage(r"C:\ZODB_BACKUP\data.fs")}
> > Traceback (most recent call last):
> >   File "<interactive input>", line 1, in ?
> >   File "C:\Python21\ZODB\FileStorage.py", line
> 272, in
> > __init__
> >     r = self._restore_index()
> >   File "C:\Python21\ZODB\FileStorage.py", line
> 473, in
> > _restore_index
> >     tid=self._sane(index, pos)
> >   File "C:\Python21\ZODB\FileStorage.py", line
> 374, in
> > _sane
> >     if file.tell() < pos:
> > IOError: (0, 'Error')
> 
> Yikes, it looks like the file is actually 2^31 bytes
> or larger, making 
> it so that even file.tell() fails.  In the short
> term, if you trim the 
> file to 2147483647 bytes (2^31 - 1), you'll probably
> be able to open it 
> or use fsrecover.  In the long term, the file.tell()
> call probably ought 
> to be surrounded by an IOError catcher.  Here's a
> way to truncate in 
> Python (make a backup first ;-) ):
> 
> f = open('/path/to/data.fs', 'w+b')
> f.truncate(2147483647)
> f.close()
> 
> Once recovered, you can pack more often, move to a
> platform that 
> supports large files with Python 2.1, or move to
> Python 2.2, which 
> implements large file support on Windows.
> 
> Shane
> 


__________________________________________________
Do you Yahoo!?
Yahoo! Platinum - Watch CBS' NCAA March Madness, live on your desktop!
http://platinum.yahoo.com