[ZODB-Dev] Relstorage pack problems

[Shane Hathaway]

Santi Camps wrote:
> I attach the script we use to do the conversion.   Be free to include
> in Relstorage if you think it's useful and it is well done (as I said,
> I really don't know much about ZODB, I just mix zodbconvert.py with
> some DirectoryStorage code)

Ah-ha, you removed from zodbconvert.py the code that checks whether the 
destination already has data.  That was unnecessary and a big mistake. 
By removing that error condition, you made the script blind to the mess 
it was making.

Assuming your bad script caused your problem, it is likely that packing 
will still mess up your database, since you still probably have mixed-up 
object_state rows.  Don't pack until I've had a chance to look again.

Put the error condition back.  Here is what it looks like:

     if storage_has_data(destination):
         msg = "Error: the destination storage has data.  Try --clear."
         sys.exit(msg)

It comes just before the call to copyTransactionsFrom().  Also, your 
current storage_has_data() function simply returns True, which is 
insane.  Here is the correct method:

def storage_has_data(storage):
     i = storage.iterator()
     try:
         i[0]
     except IndexError:
         return False
     return True

It's possible that function has a bug due to recent changes in the 
storage iterator API.  If it does, I'll fix it.

Shane