[ZODB-Dev] Support for graceful ZODB Class renaming

[holger krekel]

Pieter Nagel wrote:
> On Sun, Jan 19, 2003 at 11:50:09AM +0000, Toby Dickenson wrote:
> 
> > I assume that other classes would have self.my_portfolio attributes that you 
> > would also want to rename? How do you cope with this?
> 
> By renaming the attributes :-)
> 
> I have a migration framework that copes with class renamings, class
> deletions, and attribute changes quite nicely all in one. It works like this:
> 
> The work to be done to migrate a database is specified declaratively,
> stored on a Migrator object.
> 
> The Migrator stores two things:
> 1) A mapping of "old class name" to "new class".
> 2) A list of migration phases. A migration phases consists of two
> items: a specification of all objects to operate on, and the function to
> be invoked on these.
> 
> When the Migrator migrates a database, it first sets up a classFactory on
> the database that will instantiate any pickles of classes under an old
> name in the mapping as instances of the new class. It sweeps throught the
> database, activating all the objects in it - which ensures, as side effect,
> that all instances of renamed classes are re-pickled under their new name.
> 
> Then it proceeds through all the migration phases, in order, and invokes
> migration methods on all the objects to be migrated.
> 
> Lastly, it calls 'checkIntegrity' on all objects in the database that
> implement it. This is to test that the migration did, indeed, result in
> what one expected. 
> 
> Deleted classes are handled by renaming instances of the deleted class to
> a new stub class that contains a checkIntegrity() method that always
> barfs. This is so one can be certain one actually got rid of them.
> 
> So the work the end-user needs to do to migrate a database looks like
> this:
> 
>   def migrateNewPerson(person):
>     person.fullname = person.firstname + ' ' + person.surname
>     if not (person.telephone.strip()):
>       #OOPS! Fix old data that treated whitespace numbers as being 'real'
>       person.telephone = None
> 
>   import NewPerson, Migrator, RenameAttribute
> 
>   migrator = Migrator()
>   migrator.renameClasses('SomeModule.OldPerson', NewPerson)
>   migrator.addPhase(Instances(NewPerson), migrateNewPerson)
>   migrator.addPhase(Instances(NewPerson), RenameAttribute('portfolio', 'fundHoldings'))
> 
>   migrator.migrate(aDatabaseConnection)
> 
> In practice this has served me well throughout many renamings, add-hoc
> data fixes, and computed migrations.
> 
> It still needs to be ported to ZODB4 HEAD (to cope with the disapperance
> of setClassFactory - metaclass.__new__ seems the way to go), and can still
> be optimized to not sweep through the entire database gratuitously.
> 
> But the approach of refactoring data in a production database to a sane
> state at each deployment is definitely feasible. I've done it in ZODB and
> GemStone/Smalltalk for years. It works.

interesting approach.  Do you also cope with exported data (say to
xml) which gets reimported later when the scheme has already changed? 
Do you have a history list of Migrators that would get applied? 

    holger