[Zope3-dev] Re: Replacing class refs in ZODB

[Garrett Smith]

Derrick Hudson wrote:
> On Fri, Feb 18, 2005 at 11:59:04AM -0500, Jim Fulton wrote:
>> Garrett Smith wrote:
>>> Jim Fulton wrote:
>>>> Garrett Smith wrote:
>>>> 
>>>>> Rather than leave aliases around forever, is there a utlity (or
>>>>> other means) one can use to update references to renamed/moved
>>>>> classes in the ZODB?
>>>> 
>>>> Not currently.  We'd like a cleaner mechanism, but haven't come up
>>>> with one yet.
> 
>>> So I take it this is non-trivial. I was hoping it would boil down
>>> to a search-and-release in the Data.fs.
>> 
>> It depends on what "this" is.  One could fairly straightforwardly
>> write a script that could convert an off-line database.  I think we
>> really want to be able to do this without taking the database
>> off-line. 
> 
> I tried to do this as a step to solve a problem I had about two weeks
> ago.  It didn't work, though.
> 
>     #!/usr/bin/python2.3 -i
> 
>     from ZODB import FileStorage, DB
>     storage = FileStorage.FileStorage( 'Data.fs' )
>     db = DB(storage)
>     conn = db.open()
>     root = conn.root()
>     app = root['Application']
> 
>     import zwiki, zwiki.wiki
>     # setup the alias for the initial load
>     sys.modules['zope.app.wiki'] = zwiki
> 
>     the_wiki = app.data['the_wiki']
>     the_wiki.__class__ = zwiki.wiki.Wiki
>     the_wiki._p_changed = True
> 
>     # save the changes:
>     import transaction ; transaction.commit()
> 
> The ZODB (probably pickle, actually) would complain that it didn't
> like the object.  I think it is because I changed the class, so some
> sanity check failed.  In hindsight I think it is the same error I had
> recently with my singleton pattern and Zope2.
> 
> If someone has some pointers for me I'm willing to read (docs, code)
> and experiment.
> 
> -D

I ran into this from Guido back in 2002:

> I don't believe a search-and-replace on a pickle can ever be safe. 
> In a binary pickle, it might interfere with length fields.  And in
> either kind of pickle, you might accidentally replace data that
> happens to look like a module name.
>
> I'd suggest something else instead: when you have a pickle
> referencing module A which has since been renamed to B, create a
> dummy module A that contains "from B import *".  Then load the
> pickle, and write it back again.  The loading should work because a
> reference to class A.C will find it (as an alias for B.C); the
> storing should store it as B.C because that's the real name of class

So, aliases can be theoretically removed, but you'd have to guarantee
that all the old serialized objects in the world have been loaded and
re-written, which is probably impossible.

 -- Garrett