[ZODB-Dev] zodbupdate branch sylvain-persistent-load

[Sylvain Viollon]

Hello,

   I did few month ago a branch of zodbupdate, which works completely 
differently than the trunk. Instead of changing the pickle code of the record, 
it unpickle it, changing references at that point, and repickle if there is 
changes.

  I did this in order to support the ZODB references that are implemented 
using persistent ID, in which a reference to a class can be expressed as a 
tuple (module, classname) instead of the class pickle opcode. That tuple 
(module, classname) can appear in the class meta pickle of a record as well. 
You can have more details about this by reading the comments of the 
serialize.py file in the ZODB code.

  If I understand correctly, those are legacy formats, that the ZODB doesn't 
use anymore to write data, but is still able to read data stored using those 
formats for backward compatibility support.

  The issue is that I want to use zodbupdate on database that have been 
created a long time ago, and I do need zodbupdate to be able to read those 
formats and correct class references in those records as well. That's the 
motivation of my changes.

  My version of zodbupdate is feature alike with the old one: 

- it is able to update records which have backward compatibility imports in 
the code, and report them at the end, even if they are not described in the 
renaming rules,

-  it is able to ignore missing modules and classes (it use ZODB.broken),

- command line invocation stays the same,

- performance-wise, it seems even to be faster (I use cPickle),

  As well, it's able to work on 'broken data.fs' which trigger POSKey errors.

  I did my testing using the infrae.com website database, which have been 
created around 2002 I think, and contain a lot of interesting cases.

  So I would like to release my work now, since I need to use it in production 
environment. Since there is lot of changes, I can release it as a fork as 
well, if it's not possible to merge it with the trunk.

  Regards,

  Sylvain, 

-- 
Sylvain Viollon -- Infrae
t +31 10 243 7051 -- http://infrae.com
Hoevestraat 10 3033GC Rotterdam -- The Netherlands