[Zope-CMF] GenericSetup: catalog.xml ideas

[Maurits van Rees]

Hi,

I have been busy with catalog.xml in the GenericSetup profile of a
Plone product (eXtremeManagement).  I encountered a few uses cases
there that are not met by the current implementation of
GenericSetup/ZCatalog/exportimport.py

I am using the GenericSetup from the plone 2.5 bundle, which includes:

svn://svn.zope.org/repos/main/GenericSetup/branches/1.2/ZCatalog

But I see that trunk does not have changes that are relevant to this
case.


So here are some points, with suggested changes to the _initIndexes
function in exportimport.py.  The tests even run after this, so it
seems these changes do not break anything.  But I have not *added*
tests for this.


1. Removing an index
--------------------

Currently, you can add an attribute 'remove' to an index in
catalog.xml.  This means the index will be removed.  Presumably this
is meant to remove an index that was previously used by a product but
is now not necessary anymore or even harmful.

When you apply this profile a second time, you run into an error.
Here as a test I added remove="True" to the getAssignees index:

    ...
    Module Products.GenericSetup.ZCatalog.exportimport, line 101, in _initIndexes
    Module Products.ZCatalog.ZCatalog, line 1018, in delIndex
    Module Products.ZCatalog.Catalog, line 262, in delIndex
  CatalogError: The index getAssignees does not exist

And of course it is absolutely fine that this index does not exist, as
I want it removed anyway. :)

This can be fixed with the following change to exportimport.py (around
line 94):

             if child.hasAttribute('remove'):
-                zcatalog.delIndex(idx_id)
+                # Remove index if it is there; then continue to the
+                # next index.
+                if idx_id in zcatalog.indexes():
+                    zcatalog.delIndex(idx_id)
                 continue
 

2. Changing an index
--------------------

I had a FieldIndex getAssignees, which is in any Data.fs that uses the
eXtremeManagement product.  I found out that it was better to make
this a KeywordIndex.  So I added this to my catalog.xml:

  <index name="getAssignees" meta_type="KeywordIndex">
   <indexed_attr value="getAssignees"/>
  </index>

The only effect this has is that the index is cleared: all values are
gone (which will be my third point below).  The index itself is still
a FieldIndex, not the KeywordIndex that I requested.

So I propose a new attribute that can be used in catalog.xml:
'purge'.  When this attribute is set, the index gets removed and then
readded, which will mean that in my case the index would become a
KeywordIndex like I wanted.

This can be done with the following change (around line 100):

+            if child.hasAttribute('purge'):
+                # If the index is there, remove it; this can be used
+                # to remove an index with the same name but a wrong
+                # meta_type.
+                if idx_id in zcatalog.indexes():
+                    zcatalog.delIndex(idx_id)
+
             if idx_id not in zcatalog.indexes():


This is not ideal, as it always destroys the index, even when it is
already correctly there, but it is an improvement.


3. Keeping an index
-------------------

When you mention an index in your catalog.xml, GenericSetup clears it:
after applying the profile, the index has zero values.  For me it
seems more logical to keep the index if it is already there.  In the
current situation, I have learned to work around this by adding a
dependency step with some code that reindexes those cleared indexes.

This can take a long time so I would like to avoid doing that on each
reinstall, especially when I am busy developing this product and am
often reinstalling it or reapplying the profile.

So I propose a new attribute that can be used in catalog.xml:
'keep'.  When this attribute is set and the index is already there,
just do nothing and get on with the next index.  If the index is not
there, add it.

This can be done with the following change (around line 108):

                 meta_type = str(child.getAttribute('meta_type'))
                 zcatalog.addIndex(idx_id, meta_type, extra)
+            elif child.hasAttribute('keep'):
+                continue


Or maybe some attribute can be added like 'reindex' or
'reindex_when_new', which would trigger a reindex of the index,
possibly only when the index was added this time and not when it was
already there.  This might be an attribute of the catalog tool object
itself.  This would at least avoid the need of having to add
non-GenericSetup code to force a reindex.


4. Keep when no change; purge otherwise
---------------------------------------

As a probably better alternative, it would be nice to do all this
automatically without any extra attributes.

- When the settings for the index are the same in catalog.xml and in
  the current catalog: do nothing.

- When at least one setting is different, remove the index and add it
  with the new settings.

This seems more difficult to implement though.


Does this make sense?

Or has this been debated to death already without me knowing it? :)


Thanks for any reactions,

-- 
Maurits van Rees | http://maurits.vanrees.org/ [NL]
            Work | http://zestsoftware.nl/
"Do not worry about your difficulties in computers,
 I can assure you mine are still greater."