[ZODB-Dev] Re: [Zope-dev] corrupted data.fs

[Chris Kratz]

Here is a little more information related to this problem.  I restored from 
the previous nights backup to another machine.  The backup happens nightly 
before packing the database.  I exported the folder from backup and imported 
it into our live site after renaming the existing folder out of the way.  
That took care of the problem.

I did some additional testing.  Out of many test cases, I was able to 
reproduce the problem.  I restored the Data.fs to a test zope.  I then packed 
it interactively (2x, 1 at 0 days, another at 5 days, both on fresh copies).  
The folder was still useable.  So I restored from backup once again and then 
interactively ran the commands that do the nightly pack from the command 
line.  After the pack using Client.py, the folder was corrupted in exactly 
the same way!!!  Doing the pack from the command line crashed with the 
following traceback:

Traceback (most recent call last):
  File "/usr/local/zope/lib/python/ZPublisher/Client.py", line 635, in ?
    main()
  File "/usr/local/zope/lib/python/ZPublisher/Client.py", line 628, in main
    headers, body = apply(f,(),kw)
  File "/usr/local/zope/lib/python/ZPublisher/Client.py", line 226, in 
__call__
    self.handleError(query, ec, em, headers, response)
  File "/usr/local/zope/lib/python/ZPublisher/Client.py", line 241, in 
handleError
    raise t, RemoteException(t,v,f,l,self.url,query,ec,em,response)
bci.ServerError: 302 (File: Unknown Line: Unknown)
302 Moved Temporarily for 
http://localhost:8080/Control_Panel/Database/manage_pack

The code in my bash script looks like this:
ZOPE_PATH="/usr/local/zope"
ZOPE_COMMAND="/usr/local/bin/python 
${ZOPE_PATH}/lib/python/ZPublisher/Client.py"
ZOPE_ADDRESS="http://localhost:80"
ZOPE_AUTH="buser:********" # a special user and password only for backups, 
user has manager access
${ZOPE_COMMAND} -u ${ZOPE_AUTH} 
${ZOPE_ADDRESS}/Control_Panel/Database/manage_pack days:float=5

So, it seems pretty clear to me that the corruption issue has to do with 
using the Client.py script to pack the database.  Does anybody know why this 
might be?

It doesn't happen every time that this particular folder in question gets 
corrupted, but it happens often (about 7 out of 10 times) when I run the pack 
from the command line.  The number of days doesn't seem to matter (it 
corrupts at 5, 4, 3, etc).  Packing interactively doesn't seem to ever cause 
a problem (even 10 or so times in a row).

Anybody have any ideas?  Perhaps there is a better way to back a database in 
batch...  Thanks,

-Chris

ps A final interesting tidbit.  The two logged entries in the Z2.log are:

127.0.0.1 - buser [06/Dec/2001:17:08:11 -0400] "POST 
/Control_Panel/Database/manage_pack HTTP/1.0" 500 8930 "" ""

192.168.1.200 - buser [06/Dec/2001:17:54:15 -0400] "POST 
/Control_Panel/Database/manage_pack HTTP/1.1" 500 16760 
"http://192.168.1.200:8080/Control_Panel/Database/manage_main" "Mozilla/5.0 
(X11; U; Linux i686; en-US; rv:0.9.6) Gecko/20011120"

The first is run from a script and the second is interactively.  The first 
corrupted the database, the second did not.
-- 
Systems Analyst/Programmer
VistaShare LLC
chris.kratz-no-spam-please-vistashare.com
(replace -no-spam-please- by @)
www.vistashare.com