[ZODB-Dev] Support for graceful ZODB Class renaming

[Pieter Nagel]

On Sun, Jan 19, 2003 at 11:50:09AM +0000, Toby Dickenson wrote:

> I assume that other classes would have self.my_portfolio attributes that you 
> would also want to rename? How do you cope with this?

By renaming the attributes :-)

I have a migration framework that copes with class renamings, class
deletions, and attribute changes quite nicely all in one. It works like this:

The work to be done to migrate a database is specified declaratively,
stored on a Migrator object.

The Migrator stores two things:
1) A mapping of "old class name" to "new class".
2) A list of migration phases. A migration phases consists of two
items: a specification of all objects to operate on, and the function to
be invoked on these.

When the Migrator migrates a database, it first sets up a classFactory on
the database that will instantiate any pickles of classes under an old
name in the mapping as instances of the new class. It sweeps throught the
database, activating all the objects in it - which ensures, as side effect,
that all instances of renamed classes are re-pickled under their new name.

Then it proceeds through all the migration phases, in order, and invokes
migration methods on all the objects to be migrated.

Lastly, it calls 'checkIntegrity' on all objects in the database that
implement it. This is to test that the migration did, indeed, result in
what one expected. 

Deleted classes are handled by renaming instances of the deleted class to
a new stub class that contains a checkIntegrity() method that always
barfs. This is so one can be certain one actually got rid of them.

So the work the end-user needs to do to migrate a database looks like
this:

  def migrateNewPerson(person):
    person.fullname = person.firstname + ' ' + person.surname
    if not (person.telephone.strip()):
      #OOPS! Fix old data that treated whitespace numbers as being 'real'
      person.telephone = None

  import NewPerson, Migrator, RenameAttribute

  migrator = Migrator()
  migrator.renameClasses('SomeModule.OldPerson', NewPerson)
  migrator.addPhase(Instances(NewPerson), migrateNewPerson)
  migrator.addPhase(Instances(NewPerson), RenameAttribute('portfolio', 'fundHoldings'))

  migrator.migrate(aDatabaseConnection)

In practice this has served me well throughout many renamings, add-hoc
data fixes, and computed migrations.

It still needs to be ported to ZODB4 HEAD (to cope with the disapperance
of setClassFactory - metaclass.__new__ seems the way to go), and can still
be optimized to not sweep through the entire database gratuitously.

But the approach of refactoring data in a production database to a sane
state at each deployment is definitely feasible. I've done it in ZODB and
GemStone/Smalltalk for years. It works.

-- 
     ,_
     /_)              /| /
    /   i e t e r    / |/ a g e l
    http://www.nagel.co.za