[Zope3-dev] Zope 3.3.0 - Why is the minimum generation for 'zope.app' 1? What's a good practice to update to generation 5 prior to my own evolver?

[Jeff Shell]

I'm working on moving an old customer to our updated CMS. Their
current site is running on Zope 3.1, with our new work being done on
Zope 3.3.

I've gone through their project, removing deprecation warnings, and
watched the code work against a fresh / clean ZODB. But when I moved
one of the old development databases over, things started blowing up
immediately.

It appears to be coming from our CMS core's evolver code. The latest
generation (2) removes the old zope.app.catalog instance and replaces
it and its indexes with a new zc.catalog Extent Catalog and indexes.

On the last job I worked on (the first customer developed with our
revised CMS), this evolver script ran fine. But that customer's ZODB
started out fresh on Zope 3.3.

New customer (on fresh ZODB)::

    >>> connection = debugger.db.open(); cr = connection.root()
    >>> cr['zope.app.generations'].items()
    [(u'br.cms', 2), (u'zope.app', 5)]

Old customer (old ZODB, trying to evolve to new, running on Zope 3.3).
This is the report _FROM_ the Zope 3.3 instance::

    >>> connection = debugger.db.open(); cr = connection.root()
    >>> cr['zope.app.generations'].items()
    [(u'br.cms', 1), (u'zope.app', 1)]

I would like those numbers to be 2 and 5.

But Zope's default evolution mechanism is to "evolve to minimum," and
the zope.app schema manager is configured like this::

    ZopeAppSchemaManager = SchemaManager(
        minimum_generation=1,
        generation=5,
        package_name=key)

Great. Well it turns out that generations 2 through 5 do some pretty
important things. In my case, I'm interested in generation 4 which
evolves the Registrations so that I can unregister and delete an old
catalog and replace it with zc.catalog. Unless my ZODB is updated to
at least generation 4, I can't even do this from the ZMI.

I downloaded Zope 3.3 again just to look at the docs to see if there
was any tidbit that I missed about using generations. Because this
'evolve to minimum' scenario is just raising hell. Go ahead - find an
old ZODB, run it on Zope 3.3, and try to unregister or remove a
Utility in a ++etc++site/default scenario. I doubt it will work.

Has no one else run into this? This is another element that seems very
broken: there's this code to update one's ZODB instance when moving to
a new Zope version, but it's not configured to ever execute by
default. And there isn't any "how to update your ZODB" documentation
that even mentions "you may want to add the following to your
overrides.zcml to ensure that the ZODB is updated to its current
generation, as the minimum generation may not work on your data".

Is there a reason for this?

Finally, is there any way that I can easily add guards in my own
scheme manager evolve() scripts to ensure that other evolutions have
happened first? My `br.cms generation 2` evolver depends on the new
registration system, which isn't in place on older ZODB instances
unless zope.app is at generation 4 or later.

This is a huge issue. I promised my boss that we could update and
upgrade this old customer's code in less than a day. The customer's
code now loads up without deprecation warnings on a fresh ZODB, but
I'm struggling to load up an old ZODB copy with real data, and am now
suddenly behind schedule.

We have at least three more customer code bases beyond this one that
we want to update as well. I'm trying to think of a better way to
automate this situation for us so that development and deployment can
be pretty transparent.

-- 
Jeff Shell