[ZODB-Dev] Converting from 3.2 to 3.3

[Tim Peters]

[Syver Enstad]
> It seems to go well

Syver, what exactly are you doing?  The subject line says "Converting", but
you haven't revealed what steps you've taken that "converting" might mean
here.  I'm sure moving from 3.2 to 3.3 will become a pressing concern for
many more in the more-or-less near future.

> until I try to access a BTree in my root object.
>
> This is the repr string for it.
> <persistent broken IndexedCatalog.BTrees.IOBTree.IOBTree instance
> '\x00\x00\x00\x00\x00\x00\x00\x02'>
>
> How do I get around this?

Well, IndexedCatalog isn't part of ZODB (or even Zope), and you're a
"victim" of its "non-standard" importing of standard ZODB objects here.  I
think it was trying to cater to that all sorts of paths to standard ZODB
objects were planned to change (gratuitously, if you ask me) in ZODB4, so
IndexedCatalog's __init__.py tried both

    import BTrees  # ZODB3

and

    from zodb import btrees as BTrees  # ZODB4

and then the rest of the package imports BTrees from IndexedCatalog
unconditionally (the same way regardless of ZODB version).  Then
"IndexedCatalog" ends up in all the class paths, so you can't load the
objects anymore without IndexedCatalog present.  But ZODB4 has been
abandoned, so this eager attempt to live with it effectively backfired.

> I found a way around this problem:
>
> The IndexedCatalog module does the following when it is present but I
> can't use it with ZODB 3.3 so... If I make a module under 3.3 with this
> content I get it to work
>
> import BTrees
> import sys
> sys.modules['IndexedCatalog.BTrees'] = BTrees

Fooling sys.modules is certainly the easiest hack.  A good solution probably
has to wait for Christian to make a version of IndexedCatalog that doesn't
depend on ExtensionClass -- or for someone to figure out how to add Zope
2.8's ExtensionClass implementation to a ZODB 3.3 installation.

> Now I just have the warning messages about using Persistence classes,
> when looking at the output from fsdump.py I can indeed see that the
> objects are listed as from the Persistance module (PersistentList and so
> on). How can I convert the database so that it uses the new "module path"
> for Persistent and load the IOBTree from BTrees instead of relying on the
> hack above?

This is really the same question you've been asking since March:

    http://mail.zope.org/pipermail/zodb-dev/2004-March/007010.html
    http://mail.zope.org/pipermail/zope-dev/2004-April/022196.html

You don't get an answer you like because it appears there isn't a good
answer.  I expect that a one-shot conversion script would be relatively easy
to write, but that never gains traction because while it might do 100% of
the job for most people, it does 0% of the job for the rest.  So it gets
sidetracked by much harder issues involved in switching class paths "on the
fly", without bringing the app down.

As Jeremy explained in at least one of those older msgs, the database
contains an object O's class path not only in data records for transactions
that modify O, but also in all data records for objects that *reference* O.
So it's not only the objects with renamed paths that need to get fiddled,
it's also all objects that reference such objects.  That means you're not
going to get very far running a sed script over Data.fs <wink>.  Seriously,
it means there isn't a dead-simple approach that will work.