[Checkins] SVN: zc.catalog/tags/1.4.1/ Tag 1.4.1

Dan Korostelev nadako at gmail.com
Fri Feb 27 09:44:34 EST 2009


Log message for revision 97355:
  Tag 1.4.1

Changed:
  A   zc.catalog/tags/1.4.1/
  D   zc.catalog/tags/1.4.1/CHANGES.txt
  A   zc.catalog/tags/1.4.1/CHANGES.txt
  D   zc.catalog/tags/1.4.1/buildout.cfg
  A   zc.catalog/tags/1.4.1/buildout.cfg
  D   zc.catalog/tags/1.4.1/setup.py
  A   zc.catalog/tags/1.4.1/setup.py
  D   zc.catalog/tags/1.4.1/src/zc/catalog/catalogindex.py
  A   zc.catalog/tags/1.4.1/src/zc/catalog/catalogindex.py
  D   zc.catalog/tags/1.4.1/src/zc/catalog/configure.zcml
  A   zc.catalog/tags/1.4.1/src/zc/catalog/configure.zcml
  D   zc.catalog/tags/1.4.1/src/zc/catalog/index.py
  A   zc.catalog/tags/1.4.1/src/zc/catalog/index.py
  D   zc.catalog/tags/1.4.1/src/zc/catalog/normalizedindex.txt
  A   zc.catalog/tags/1.4.1/src/zc/catalog/normalizedindex.txt
  D   zc.catalog/tags/1.4.1/src/zc/catalog/valueindex.txt
  A   zc.catalog/tags/1.4.1/src/zc/catalog/valueindex.txt

-=-
Deleted: zc.catalog/tags/1.4.1/CHANGES.txt
===================================================================
--- zc.catalog/trunk/CHANGES.txt	2009-02-27 09:39:25 UTC (rev 97335)
+++ zc.catalog/tags/1.4.1/CHANGES.txt	2009-02-27 14:44:33 UTC (rev 97355)
@@ -1,131 +0,0 @@
-=======
-CHANGES
-=======
-
-The 1.2 line (and higher) supports Zope 3.4/ZODB 3.8.  The 1.1 line supports
-Zope 3.3/ZODB 3.7.
-
-1.4.1 (unreleased)
-------------------
-
-*
-
-
-1.4.0 (2009-02-07)
-------------------
-
-Bugs fixed
-~~~~~~~~~~
-
-* Fixed a typo in ValueIndex addform and addMenuItem
-
-* Use ``zope.container`` instead of ``zope.app.container``.
-
-* Use ``zope.keyreference`` instead of ``zope.app.keyreference``.
-
-* Use ``zope.intid`` instead of ``zope.app.intid``.
-
-* Use ``zope.catalog`` instead of ``zope.app.catalog``.
-
-
-1.3.0 (2008-09-10)
-------------------
-
-Features added
-~~~~~~~~~~~~~~
-
-* Added hook point to allow extent catalog to be used with local UID sources.
-
-
-1.2.0 (2007-11-03)
-------------------
-
-Features added
-~~~~~~~~~~~~~~
-
-* Updated package meta-data.
-
-* zc.catalog now can use 64-bit BTrees ("L") as provided by ZODB 3.8.
-
-* Albertas Agejavas (alga at pov.lt) included the new CallableWrapper, for
-  when the typical Zope 3 index-by-adapter story
-  (zope.app.catalog.attribute) is unnecessary trouble, and you just want
-  to use a callable.  See callablewrapper.txt.  This can also be used for
-  other indexes based on the zope.index interfaces.
-
-* Extents now have a __len__.  The current implementation defers to the
-  standard BTree len implementation, and shares its performance
-  characteristics: it needs to wake up all of the buckets, but if all of the
-  buckets are awake it is a fairly quick operation.
-
-* A simple ISelfPoulatingExtent was added to the extentcatalog module for
-  which populating is a no-op.  This is directly useful for catalogs that
-  are used as implementation details of a component, in which objects are
-  indexed explicitly by your own calls rather than by the usual subscribers.
-  It is also potentially slightly useful as a base for other self-populating
-  extents.
-
-
-1.1.1 (2007-3-17)
------------------
-
-Bugs fixed
-~~~~~~~~~~
-
-'all_of' would return all results when one of the values had no results.
-Reported, with test and fix provided, by Nando Quintana.
-
-
-1.1 (2007-01-06)
-----------------
-
-Features removed
-~~~~~~~~~~~~~~~~
-
-The queueing of events in the extent catalog has been entirely removed.
-Subtransactions caused significant problems to the code introduced in 1.0.
-Other solutions also have significant problems, and the win of this kind
-of queueing is qustionable.  Here is a run down of the approaches rejected
-for getting the queueing to work:
-
-* _p_invalidate (used in 1.0).  Not really designed for use within a
-  transaction, and reverts to last savepoint, rather than the beginning of
-  the transaction.  Could monkeypatch savepoints to iterate over
-  precommit transaction hooks but that just smells too bad.
-
-* _p_resolveConflict.  Requires application software to exist in ZEO and
-  even ZRS installations, which is counter to our software deployment goals.
-  Also causes useless repeated writes of empty queue to database, but that's
-  not the showstopper.
-
-* vague hand-wavy ideas for separate storages or transaction managers for the
-  queue.  Never panned out in discussion.
-
-
-1.0 (2007-01-05)
-----------------
-
-Bugs fixed
-~~~~~~~~~~
-
-* adjusted extentcatalog tests to trigger (and discuss and test) the queueing
-  behavior.
-
-* fixed problem with excessive conflict errors due to queueing code.
-
-* updated stemming to work with newest version of TextIndexNG's extensions.
-
-* omitted stemming test when TextIndexNG's extensions are unavailable, so
-  tests pass without it.  Since TextIndexNG's extensions are optional, this
-  seems reasonable.
-
-* removed use of zapi in extentcatalog.
-
-
-0.2 (2006-11-22)
-----------------
-
-Features added
-~~~~~~~~~~~~~~
-
-* First release on Cheeseshop.

Copied: zc.catalog/tags/1.4.1/CHANGES.txt (from rev 97345, zc.catalog/trunk/CHANGES.txt)
===================================================================
--- zc.catalog/tags/1.4.1/CHANGES.txt	                        (rev 0)
+++ zc.catalog/tags/1.4.1/CHANGES.txt	2009-02-27 14:44:33 UTC (rev 97355)
@@ -0,0 +1,133 @@
+=======
+CHANGES
+=======
+
+The 1.2 line (and higher) supports Zope 3.4/ZODB 3.8.  The 1.1 line supports
+Zope 3.3/ZODB 3.7.
+
+1.4.1 (2009-02-27)
+------------------
+
+* Add FieldIndex-like sorting support for the ValueIndex.
+
+* Add sorting indexes support for the NormalizationWrapper.
+
+
+1.4.0 (2009-02-07)
+------------------
+
+Bugs fixed
+~~~~~~~~~~
+
+* Fixed a typo in ValueIndex addform and addMenuItem
+
+* Use ``zope.container`` instead of ``zope.app.container``.
+
+* Use ``zope.keyreference`` instead of ``zope.app.keyreference``.
+
+* Use ``zope.intid`` instead of ``zope.app.intid``.
+
+* Use ``zope.catalog`` instead of ``zope.app.catalog``.
+
+
+1.3.0 (2008-09-10)
+------------------
+
+Features added
+~~~~~~~~~~~~~~
+
+* Added hook point to allow extent catalog to be used with local UID sources.
+
+
+1.2.0 (2007-11-03)
+------------------
+
+Features added
+~~~~~~~~~~~~~~
+
+* Updated package meta-data.
+
+* zc.catalog now can use 64-bit BTrees ("L") as provided by ZODB 3.8.
+
+* Albertas Agejavas (alga at pov.lt) included the new CallableWrapper, for
+  when the typical Zope 3 index-by-adapter story
+  (zope.app.catalog.attribute) is unnecessary trouble, and you just want
+  to use a callable.  See callablewrapper.txt.  This can also be used for
+  other indexes based on the zope.index interfaces.
+
+* Extents now have a __len__.  The current implementation defers to the
+  standard BTree len implementation, and shares its performance
+  characteristics: it needs to wake up all of the buckets, but if all of the
+  buckets are awake it is a fairly quick operation.
+
+* A simple ISelfPoulatingExtent was added to the extentcatalog module for
+  which populating is a no-op.  This is directly useful for catalogs that
+  are used as implementation details of a component, in which objects are
+  indexed explicitly by your own calls rather than by the usual subscribers.
+  It is also potentially slightly useful as a base for other self-populating
+  extents.
+
+
+1.1.1 (2007-3-17)
+-----------------
+
+Bugs fixed
+~~~~~~~~~~
+
+'all_of' would return all results when one of the values had no results.
+Reported, with test and fix provided, by Nando Quintana.
+
+
+1.1 (2007-01-06)
+----------------
+
+Features removed
+~~~~~~~~~~~~~~~~
+
+The queueing of events in the extent catalog has been entirely removed.
+Subtransactions caused significant problems to the code introduced in 1.0.
+Other solutions also have significant problems, and the win of this kind
+of queueing is qustionable.  Here is a run down of the approaches rejected
+for getting the queueing to work:
+
+* _p_invalidate (used in 1.0).  Not really designed for use within a
+  transaction, and reverts to last savepoint, rather than the beginning of
+  the transaction.  Could monkeypatch savepoints to iterate over
+  precommit transaction hooks but that just smells too bad.
+
+* _p_resolveConflict.  Requires application software to exist in ZEO and
+  even ZRS installations, which is counter to our software deployment goals.
+  Also causes useless repeated writes of empty queue to database, but that's
+  not the showstopper.
+
+* vague hand-wavy ideas for separate storages or transaction managers for the
+  queue.  Never panned out in discussion.
+
+
+1.0 (2007-01-05)
+----------------
+
+Bugs fixed
+~~~~~~~~~~
+
+* adjusted extentcatalog tests to trigger (and discuss and test) the queueing
+  behavior.
+
+* fixed problem with excessive conflict errors due to queueing code.
+
+* updated stemming to work with newest version of TextIndexNG's extensions.
+
+* omitted stemming test when TextIndexNG's extensions are unavailable, so
+  tests pass without it.  Since TextIndexNG's extensions are optional, this
+  seems reasonable.
+
+* removed use of zapi in extentcatalog.
+
+
+0.2 (2006-11-22)
+----------------
+
+Features added
+~~~~~~~~~~~~~~
+
+* First release on Cheeseshop.

Deleted: zc.catalog/tags/1.4.1/buildout.cfg
===================================================================
--- zc.catalog/trunk/buildout.cfg	2009-02-27 09:39:25 UTC (rev 97335)
+++ zc.catalog/tags/1.4.1/buildout.cfg	2009-02-27 14:44:33 UTC (rev 97355)
@@ -1,9 +0,0 @@
-[buildout]
-develop = .
-parts = test
-
-[test]
-recipe = zc.recipe.testrunner
-eggs = zc.catalog [test]
-defaults = "--exit-with-status".split()
-

Copied: zc.catalog/tags/1.4.1/buildout.cfg (from rev 97353, zc.catalog/trunk/buildout.cfg)
===================================================================
--- zc.catalog/tags/1.4.1/buildout.cfg	                        (rev 0)
+++ zc.catalog/tags/1.4.1/buildout.cfg	2009-02-27 14:44:33 UTC (rev 97355)
@@ -0,0 +1,9 @@
+[buildout]
+develop = .
+parts = test
+
+[test]
+recipe = zc.recipe.testrunner
+eggs = zc.catalog [test]
+defaults = "--exit-with-status".split()
+

Deleted: zc.catalog/tags/1.4.1/setup.py
===================================================================
--- zc.catalog/trunk/setup.py	2009-02-27 09:39:25 UTC (rev 97335)
+++ zc.catalog/tags/1.4.1/setup.py	2009-02-27 14:44:33 UTC (rev 97355)
@@ -1,94 +0,0 @@
-##############################################################################
-#
-# Copyright (c) 2007 Zope Corporation and Contributors.
-# All Rights Reserved.
-#
-# This software is subject to the provisions of the Zope Public License,
-# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
-# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
-# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
-# FOR A PARTICULAR PURPOSE.
-#
-##############################################################################
-"""Setup for zc.catalog package
-
-$Id: setup.py 81038 2007-10-24 14:34:17Z srichter $
-"""
-import os
-from setuptools import setup, find_packages
-
-def read(*rnames):
-    return open(os.path.join(os.path.dirname('.'), *rnames)).read()
-
-setup(name='zc.catalog',
-      version = '1.4.1dev',
-      author='Zope Corporation and Contributors',
-      author_email='zope-dev at zope.org',
-      description="Extensions to the Zope 3 Catalog",
-      long_description=(
-          read('README.txt')
-          + '\n\n.. contents::\n\n' +
-          read('src', 'zc', 'catalog', 'valueindex.txt')
-          + '\n\n' +
-          read('src', 'zc', 'catalog', 'setindex.txt')
-          + '\n\n' +
-          read('src', 'zc', 'catalog', 'normalizedindex.txt')
-          + '\n\n' +
-          read('src', 'zc', 'catalog', 'extentcatalog.txt')
-          + '\n\n' +
-          read('src', 'zc', 'catalog', 'stemmer.txt')
-          + '\n\n' +
-          read('src', 'zc', 'catalog', 'legacy.txt')
-          + '\n\n' +
-          read('src', 'zc', 'catalog', 'globber.txt')
-          + '\n\n' +
-          read('src', 'zc', 'catalog', 'callablewrapper.txt')
-          + '\n\n' +
-          read('src', 'zc', 'catalog', 'browser', 'README.txt')
-          + '\n\n' +
-          read('CHANGES.txt')
-          ),
-      keywords = "zope3 i18n date time duration catalog index",
-      classifiers = [
-          'Development Status :: 5 - Production/Stable',
-          'Environment :: Web Environment',
-          'Intended Audience :: Developers',
-          'License :: OSI Approved :: Zope Public License',
-          'Programming Language :: Python',
-          'Natural Language :: English',
-          'Operating System :: OS Independent',
-          'Topic :: Internet :: WWW/HTTP',
-          'Framework :: Zope3'],
-      url='http://pypi.python.org/pypi/zc.catalog',
-      license='ZPL 2.1',
-      packages=find_packages('src'),
-      package_dir = {'': 'src'},
-      namespace_packages=['zc'],
-      extras_require=dict(
-           test=['zope.app.authentication',
-                 'zope.app.catalog',
-                 'zope.app.securitypolicy',
-                 'zope.app.server',
-                 'zope.app.testing',
-                 'zope.app.zcmlfiles',
-                 'zope.keyreference',
-                 'zope.intid',
-                 'zope.testbrowser',
-                 'zope.testing',
-                 ]),
-      install_requires=['ZODB3',
-                        'pytz',
-                        'setuptools',
-                        'zope.catalog',
-                        'zope.container',
-                        'zope.component',
-                        'zope.i18nmessageid',
-                        'zope.interface',
-                        'zope.publisher',
-                        'zope.schema',
-                        'zope.security',
-                        ],
-      include_package_data = True,
-      zip_safe = False,
-      )

Copied: zc.catalog/tags/1.4.1/setup.py (from rev 97353, zc.catalog/trunk/setup.py)
===================================================================
--- zc.catalog/tags/1.4.1/setup.py	                        (rev 0)
+++ zc.catalog/tags/1.4.1/setup.py	2009-02-27 14:44:33 UTC (rev 97355)
@@ -0,0 +1,95 @@
+##############################################################################
+#
+# Copyright (c) 2007 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Setup for zc.catalog package
+
+$Id: setup.py 81038 2007-10-24 14:34:17Z srichter $
+"""
+import os
+from setuptools import setup, find_packages
+
+def read(*rnames):
+    return open(os.path.join(os.path.dirname('.'), *rnames)).read()
+
+setup(name='zc.catalog',
+      version = '1.4.1',
+      author='Zope Corporation and Contributors',
+      author_email='zope-dev at zope.org',
+      description="Extensions to the Zope 3 Catalog",
+      long_description=(
+          read('README.txt')
+          + '\n\n.. contents::\n\n' +
+          read('src', 'zc', 'catalog', 'valueindex.txt')
+          + '\n\n' +
+          read('src', 'zc', 'catalog', 'setindex.txt')
+          + '\n\n' +
+          read('src', 'zc', 'catalog', 'normalizedindex.txt')
+          + '\n\n' +
+          read('src', 'zc', 'catalog', 'extentcatalog.txt')
+          + '\n\n' +
+          read('src', 'zc', 'catalog', 'stemmer.txt')
+          + '\n\n' +
+          read('src', 'zc', 'catalog', 'legacy.txt')
+          + '\n\n' +
+          read('src', 'zc', 'catalog', 'globber.txt')
+          + '\n\n' +
+          read('src', 'zc', 'catalog', 'callablewrapper.txt')
+          + '\n\n' +
+          read('src', 'zc', 'catalog', 'browser', 'README.txt')
+          + '\n\n' +
+          read('CHANGES.txt')
+          ),
+      keywords = "zope3 i18n date time duration catalog index",
+      classifiers = [
+          'Development Status :: 5 - Production/Stable',
+          'Environment :: Web Environment',
+          'Intended Audience :: Developers',
+          'License :: OSI Approved :: Zope Public License',
+          'Programming Language :: Python',
+          'Natural Language :: English',
+          'Operating System :: OS Independent',
+          'Topic :: Internet :: WWW/HTTP',
+          'Framework :: Zope3'],
+      url='http://pypi.python.org/pypi/zc.catalog',
+      license='ZPL 2.1',
+      packages=find_packages('src'),
+      package_dir = {'': 'src'},
+      namespace_packages=['zc'],
+      extras_require=dict(
+           test=['zope.app.authentication',
+                 'zope.app.catalog',
+                 'zope.app.securitypolicy',
+                 'zope.app.server',
+                 'zope.app.testing',
+                 'zope.app.zcmlfiles',
+                 'zope.keyreference',
+                 'zope.intid',
+                 'zope.testbrowser',
+                 'zope.testing',
+                 ]),
+      install_requires=['ZODB3',
+                        'pytz',
+                        'setuptools',
+                        'zope.catalog',
+                        'zope.container',
+                        'zope.component',
+                        'zope.i18nmessageid',
+                        'zope.index>=3.5.1',
+                        'zope.interface',
+                        'zope.publisher',
+                        'zope.schema',
+                        'zope.security',
+                        ],
+      include_package_data = True,
+      zip_safe = False,
+      )

Deleted: zc.catalog/tags/1.4.1/src/zc/catalog/catalogindex.py
===================================================================
--- zc.catalog/trunk/src/zc/catalog/catalogindex.py	2009-02-27 09:39:25 UTC (rev 97335)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/catalogindex.py	2009-02-27 14:44:33 UTC (rev 97355)
@@ -1,73 +0,0 @@
-#############################################################################
-#
-# Copyright (c) 2006 Zope Corporation and Contributors.
-# All Rights Reserved.
-#
-# This software is subject to the provisions of the Zope Public License,
-# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
-# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
-# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
-# FOR A PARTICULAR PURPOSE.
-#
-##############################################################################
-"""Indexes appropriate for zope.catalog
-
-$Id: catalogindex.py 2918 2005-07-19 22:12:38Z jim $
-"""
-import zope.interface
-
-import zope.catalog.attribute
-import zope.container.contained
-
-import zc.catalog.index
-import zc.catalog.interfaces
-
-class ValueIndex(zope.catalog.attribute.AttributeIndex,
-                 zc.catalog.index.ValueIndex,
-                 zope.container.contained.Contained):
-
-    zope.interface.implements(zc.catalog.interfaces.ICatalogValueIndex)
-
-class SetIndex(zope.catalog.attribute.AttributeIndex,
-               zc.catalog.index.SetIndex,
-               zope.container.contained.Contained):
-
-    zope.interface.implements(zc.catalog.interfaces.ICatalogSetIndex)
-
-class NormalizationWrapper(
-    zope.catalog.attribute.AttributeIndex,
-    zc.catalog.index.NormalizationWrapper,
-    zope.container.contained.Contained):
-
-    pass
-
-
-class CallableWrapper(zc.catalog.index.CallableWrapper,
-                      zope.container.contained.Contained):
-    zope.interface.implements(zc.catalog.interfaces.ICallableWrapper)
-
-
- at zope.interface.implementer(
-    zope.interface.implementedBy(NormalizationWrapper),
-    zc.catalog.interfaces.IValueIndex)
-def DateTimeValueIndex(
-    field_name=None, interface=None, field_callable=False,
-    resolution=2): # hour; good for per-day searches
-    ix = NormalizationWrapper(
-        field_name, interface, field_callable, zc.catalog.index.ValueIndex(),
-        zc.catalog.index.DateTimeNormalizer(resolution), False)
-    zope.interface.directlyProvides(ix, zc.catalog.interfaces.IValueIndex)
-    return ix
-
- at zope.interface.implementer(
-    zope.interface.implementedBy(NormalizationWrapper),
-    zc.catalog.interfaces.ISetIndex)
-def DateTimeSetIndex(
-    field_name=None, interface=None, field_callable=False, 
-    resolution=2): # hour; good for per-day searches
-    ix = NormalizationWrapper(
-        field_name, interface, field_callable, zc.catalog.index.SetIndex(),
-        zc.catalog.index.DateTimeNormalizer(resolution), True)
-    zope.interface.directlyProvides(ix, zc.catalog.interfaces.ISetIndex)
-    return ix

Copied: zc.catalog/tags/1.4.1/src/zc/catalog/catalogindex.py (from rev 97345, zc.catalog/trunk/src/zc/catalog/catalogindex.py)
===================================================================
--- zc.catalog/tags/1.4.1/src/zc/catalog/catalogindex.py	                        (rev 0)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/catalogindex.py	2009-02-27 14:44:33 UTC (rev 97355)
@@ -0,0 +1,75 @@
+#############################################################################
+#
+# Copyright (c) 2006 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Indexes appropriate for zope.catalog
+
+$Id: catalogindex.py 2918 2005-07-19 22:12:38Z jim $
+"""
+import zope.interface
+
+import zope.catalog.attribute
+import zope.container.contained
+import zope.index.interfaces
+
+import zc.catalog.index
+import zc.catalog.interfaces
+
+class ValueIndex(zope.catalog.attribute.AttributeIndex,
+                 zc.catalog.index.ValueIndex,
+                 zope.container.contained.Contained):
+
+    zope.interface.implements(zc.catalog.interfaces.ICatalogValueIndex)
+
+class SetIndex(zope.catalog.attribute.AttributeIndex,
+               zc.catalog.index.SetIndex,
+               zope.container.contained.Contained):
+
+    zope.interface.implements(zc.catalog.interfaces.ICatalogSetIndex)
+
+class NormalizationWrapper(
+    zope.catalog.attribute.AttributeIndex,
+    zc.catalog.index.NormalizationWrapper,
+    zope.container.contained.Contained):
+
+    pass
+
+
+class CallableWrapper(zc.catalog.index.CallableWrapper,
+                      zope.container.contained.Contained):
+    zope.interface.implements(zc.catalog.interfaces.ICallableWrapper)
+
+
+ at zope.interface.implementer(
+    zope.interface.implementedBy(NormalizationWrapper),
+    zc.catalog.interfaces.IValueIndex,
+    zope.index.interfaces.IIndexSort)
+def DateTimeValueIndex(
+    field_name=None, interface=None, field_callable=False,
+    resolution=2): # hour; good for per-day searches
+    ix = NormalizationWrapper(
+        field_name, interface, field_callable, zc.catalog.index.ValueIndex(),
+        zc.catalog.index.DateTimeNormalizer(resolution), False)
+    zope.interface.alsoProvides(ix, zc.catalog.interfaces.IValueIndex)
+    return ix
+
+ at zope.interface.implementer(
+    zope.interface.implementedBy(NormalizationWrapper),
+    zc.catalog.interfaces.ISetIndex)
+def DateTimeSetIndex(
+    field_name=None, interface=None, field_callable=False, 
+    resolution=2): # hour; good for per-day searches
+    ix = NormalizationWrapper(
+        field_name, interface, field_callable, zc.catalog.index.SetIndex(),
+        zc.catalog.index.DateTimeNormalizer(resolution), True)
+    zope.interface.alsoProvides(ix, zc.catalog.interfaces.ISetIndex)
+    return ix

Deleted: zc.catalog/tags/1.4.1/src/zc/catalog/configure.zcml
===================================================================
--- zc.catalog/trunk/src/zc/catalog/configure.zcml	2009-02-27 09:39:25 UTC (rev 97335)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/configure.zcml	2009-02-27 14:44:33 UTC (rev 97355)
@@ -1,73 +0,0 @@
-<configure
-    xmlns="http://namespaces.zope.org/zope"
-    xmlns:browser="http://namespaces.zope.org/browser"
-    i18n_domain="zope"
-    >
-
-  <class class=".extentcatalog.Catalog">
-    <factory
-        id="zc.catalog.extentcatalog"
-        />
-    <require
-        interface="zope.catalog.interfaces.ICatalogQuery"
-        permission="zope.Public"
-        />
-    <require
-        interface="zope.catalog.interfaces.ICatalogEdit"
-        permission="zope.ManageServices"
-        />
-    <require
-        interface="zope.container.interfaces.IContainer"
-        permission="zope.ManageServices"
-        />
-  </class>
-
-  <class class=".extentcatalog.FilterExtent">
-    <require
-        attributes="addable __iter__ __or__ __ror__ union __and__ __rand__
-                    intersection __sub__ difference __rsub__ rdifference
-                    __nonzero__ __contains__"
-        permission="zope.View"/>
-    <require
-        attributes="add remove discard clear"
-        permission="zope.ManageServices"
-        />
-  </class>
-
-  <class class=".catalogindex.ValueIndex">
-    <require
-        permission="zope.ManageServices"
-        interface="zope.catalog.interfaces.IAttributeIndex
-                   zope.index.interfaces.IStatistics"
-        set_schema="zope.catalog.interfaces.IAttributeIndex"
-        />
-  </class>
-
-  <class class=".catalogindex.SetIndex">
-    <require
-        permission="zope.ManageServices"
-        interface="zope.catalog.interfaces.IAttributeIndex
-                   zope.index.interfaces.IStatistics"
-        set_schema="zope.catalog.interfaces.IAttributeIndex"
-        />
-  </class>
-
-  <class class=".catalogindex.NormalizationWrapper">
-    <require
-        permission="zope.ManageServices"
-        interface="zope.catalog.interfaces.IAttributeIndex
-                   zope.index.interfaces.IStatistics"
-        set_schema="zope.catalog.interfaces.IAttributeIndex"
-        />
-  </class>
-
-  <class class="BTrees.Length.Length">
-    <require
-        permission="zope.ManageServices"
-        attributes="__call__"
-        />
-  </class>
-
-  <include package=".browser" />
-
-</configure>

Copied: zc.catalog/tags/1.4.1/src/zc/catalog/configure.zcml (from rev 97354, zc.catalog/trunk/src/zc/catalog/configure.zcml)
===================================================================
--- zc.catalog/tags/1.4.1/src/zc/catalog/configure.zcml	                        (rev 0)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/configure.zcml	2009-02-27 14:44:33 UTC (rev 97355)
@@ -0,0 +1,77 @@
+<configure
+    xmlns="http://namespaces.zope.org/zope"
+    xmlns:browser="http://namespaces.zope.org/browser"
+    xmlns:zcml="http://namespaces.zope.org/zcml"
+    i18n_domain="zope"
+    >
+
+  <class class=".extentcatalog.Catalog">
+    <factory
+        id="zc.catalog.extentcatalog"
+        />
+    <require
+        interface="zope.catalog.interfaces.ICatalogQuery"
+        permission="zope.Public"
+        />
+    <require
+        interface="zope.catalog.interfaces.ICatalogEdit"
+        permission="zope.ManageServices"
+        />
+    <require
+        interface="zope.container.interfaces.IContainer"
+        permission="zope.ManageServices"
+        />
+  </class>
+
+  <class class=".extentcatalog.FilterExtent">
+    <require
+        attributes="addable __iter__ __or__ __ror__ union __and__ __rand__
+                    intersection __sub__ difference __rsub__ rdifference
+                    __nonzero__ __contains__"
+        permission="zope.View"/>
+    <require
+        attributes="add remove discard clear"
+        permission="zope.ManageServices"
+        />
+  </class>
+
+  <class class=".catalogindex.ValueIndex">
+    <require
+        permission="zope.ManageServices"
+        interface="zope.catalog.interfaces.IAttributeIndex
+                   zope.index.interfaces.IStatistics"
+        set_schema="zope.catalog.interfaces.IAttributeIndex"
+        />
+  </class>
+
+  <class class=".catalogindex.SetIndex">
+    <require
+        permission="zope.ManageServices"
+        interface="zope.catalog.interfaces.IAttributeIndex
+                   zope.index.interfaces.IStatistics"
+        set_schema="zope.catalog.interfaces.IAttributeIndex"
+        />
+  </class>
+
+  <class class=".catalogindex.NormalizationWrapper">
+    <require
+        permission="zope.ManageServices"
+        interface="zope.catalog.interfaces.IAttributeIndex
+                   zope.index.interfaces.IStatistics"
+        set_schema="zope.catalog.interfaces.IAttributeIndex"
+        />
+  </class>
+
+  <class class="BTrees.Length.Length">
+    <require
+        permission="zope.ManageServices"
+        attributes="__call__"
+        />
+  </class>
+
+  <include
+      zcml:condition="installed zope.app.form"
+      package=".browser"
+      />
+
+</configure>

Deleted: zc.catalog/tags/1.4.1/src/zc/catalog/index.py
===================================================================
--- zc.catalog/trunk/src/zc/catalog/index.py	2009-02-27 09:39:25 UTC (rev 97335)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/index.py	2009-02-27 14:44:33 UTC (rev 97355)
@@ -1,538 +0,0 @@
-#############################################################################
-#
-# Copyright (c) 2006 Zope Corporation and Contributors.
-# All Rights Reserved.
-#
-# This software is subject to the provisions of the Zope Public License,
-# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
-# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
-# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
-# FOR A PARTICULAR PURPOSE.
-#
-##############################################################################
-"""indexes, as might be found in zope.index
-
-$Id: index.py 2918 2005-07-19 22:12:38Z jim $
-"""
-import sys
-import datetime
-import pytz.reference
-import BTrees
-import persistent
-from BTrees import Length
-from BTrees.Interfaces import IMerge
-
-from zope import component, interface
-import zope.component.interfaces
-import zope.interface.common.idatetime
-import zope.index.interfaces
-import zope.security.management
-from zope.publisher.interfaces import IRequest
-import zc.catalog.interfaces
-from zc.catalog.i18n import _
-
-
-class FamilyProperty(object):
-
-    __name__ = "family"
-
-    def __get__(self, instance, type=None):
-        if instance is None:
-            return self
-        d = instance.__dict__
-        if "family" in d:
-            return d["family"]
-        if "btreemodule" in d:
-            iftype = d["btreemodule"].split(".")[-1][:2]
-            if iftype == "IF":
-                d["family"] = BTrees.family32
-            elif iftype == "LF":
-                d["family"] = BTrees.family64
-            else:
-                raise ValueError("can't determine btree family based on"
-                                 " btreemodule of %r" % (iftype,))
-        else:
-            d["family"] = BTrees.family32
-        self._clear_old_cruft(instance)
-        return d["family"]
-
-    def __set__(self, instance, value):
-        instance.__dict__["family"] = value
-        self._clear_old_cruft(instance)
-
-    def _clear_old_cruft(self, instance):
-        d = instance.__dict__
-        if "btreemodule" in d:
-            del d["btreemodule"]
-        if "IOBTree" in d:
-            del d["IOBTree"]
-        if "BTreeAPI" in d:
-            del d["BTreeAPI"]
-
-
-class AbstractIndex(persistent.Persistent):
-
-    interface.implements(zope.index.interfaces.IInjection,
-                         zope.index.interfaces.IIndexSearch,
-                         zope.index.interfaces.IStatistics,
-                         zc.catalog.interfaces.IIndexValues,
-                         )
-
-    family = FamilyProperty()
-
-    def __init__(self, family=None):
-        if family is not None:
-            self.family = family
-        self.clear()
-
-    # These three are deprecated (they were never interface), but can
-    # all be computed from the family attribute:
-
-    @property
-    def btreemodule(self):
-        return self.family.IF.__name__
-
-    @property
-    def BTreeAPI(self):
-        return self.family.IF
-
-    @property
-    def IOBTree(self):
-        return self.family.IO.BTree
-
-    def clear(self):
-        self.values_to_documents = self.family.OO.BTree()
-        self.documents_to_values = self.family.IO.BTree()
-        self.documentCount = Length.Length(0)
-        self.wordCount = Length.Length(0)
-
-    def minValue(self, min=None):
-        if min is None:
-            return self.values_to_documents.minKey()
-        else:
-            return self.values_to_documents.minKey(min)
-
-    def maxValue(self, max=None):
-        if max is None:
-            return self.values_to_documents.maxKey()
-        else:
-            return self.values_to_documents.maxKey(max)
-
-    def values(self, min=None, max=None, excludemin=False, excludemax=False,
-               doc_id=None):
-        if doc_id is None:
-            return iter(self.values_to_documents.keys(
-                min, max, excludemin, excludemax))
-        else:
-            values = self.documents_to_values.get(doc_id)
-            if values is None:
-                return ()
-            else:
-                return iter(values.keys(min, max, excludemin, excludemax))
-
-    def containsValue(self, value):
-        return bool(self.values_to_documents.has_key(value))
-
-    def ids(self):
-        return self.documents_to_values.keys()
-
-def parseQuery(query):
-    if isinstance(query, dict):
-        if len(query) > 1:
-            raise ValueError(
-                'may only pass one of key, value pair')
-        elif not query:
-            return None, None
-        query_type, query = query.items()[0]
-        query_type = query_type.lower()
-    else:
-        raise ValueError('may only pass a dict to apply')
-    return query_type, query
-
-class ValueIndex(AbstractIndex):
-
-    interface.implements(zc.catalog.interfaces.IValueIndex)
-
-    def _add_value(self, doc_id, added):
-        values_to_documents = self.values_to_documents
-        docs = values_to_documents.get(added)
-        if docs is None:
-            values_to_documents[added] = self.family.IF.TreeSet((doc_id,))
-            self.wordCount.change(1)
-        else:
-            docs.insert(doc_id)
-
-    def index_doc(self, doc_id, value):
-        if value is None:
-            self.unindex_doc(doc_id)
-        else:
-            values_to_documents = self.values_to_documents
-            documents_to_values = self.documents_to_values
-            old = documents_to_values.get(doc_id)
-            documents_to_values[doc_id] = value
-            if old is None:
-                self.documentCount.change(1)
-            elif old != value:
-                docs = values_to_documents.get(old)
-                docs.remove(doc_id)
-                if not docs:
-                    del values_to_documents[old]
-                    self.wordCount.change(-1)
-            self._add_value(doc_id, value)
-
-    def unindex_doc(self, doc_id):
-        documents_to_values = self.documents_to_values
-        value = documents_to_values.get(doc_id)
-        if value is not None:
-            values_to_documents = self.values_to_documents
-            self.documentCount.change(-1)
-            del documents_to_values[doc_id]
-            docs = values_to_documents.get(value)
-            docs.remove(doc_id)
-            if not docs:
-                del values_to_documents[value]
-                self.wordCount.change(-1)
-
-    def apply(self, query): # any_of, any, between, none,
-        values_to_documents = self.values_to_documents
-        query_type, query = parseQuery(query)
-        if query_type is None:
-            res = None
-        elif query_type == 'any_of':
-            res = self.family.IF.multiunion(
-                [s for s in (values_to_documents.get(v) for v in query)
-                 if s is not None])
-        elif query_type == 'any':
-            if query is None:
-                res = self.family.IF.Set(self.ids())
-            else:
-                assert zc.catalog.interfaces.IExtent.providedBy(query)
-                res = query & self.family.IF.Set(self.ids())
-        elif query_type == 'between':
-            res = self.family.IF.multiunion(
-                [s for s in (values_to_documents.get(v) for v in
-                             values_to_documents.keys(*query))
-                 if s is not None])
-        elif query_type == 'none':
-            assert zc.catalog.interfaces.IExtent.providedBy(query)
-            res = query - self.family.IF.Set(self.ids())
-        else:
-            raise ValueError(
-                "unknown query type", query_type)
-        return res
-
-    def values(self, min=None, max=None, excludemin=False, excludemax=False,
-               doc_id=None):
-        if doc_id is None:
-            return iter(self.values_to_documents.keys(
-                min, max, excludemin, excludemax))
-        else:
-            value = self.documents_to_values.get(doc_id)
-            if (value is None or
-                min is not None and (
-                    value < min or excludemin and value == min) or
-                max is not None and (
-                    value > max or excludemax and value == max)):
-                return ()
-            else:
-                return (value,)
-
-class SetIndex(AbstractIndex):
-
-    interface.implements(zc.catalog.interfaces.ISetIndex)
-
-    def _add_values(self, doc_id, added):
-        values_to_documents = self.values_to_documents
-        for v in added:
-            docs = values_to_documents.get(v)
-            if docs is None:
-                values_to_documents[v] = self.family.IF.TreeSet((doc_id,))
-                self.wordCount.change(1)
-            else:
-                docs.insert(doc_id)
-
-    def index_doc(self, doc_id, value):
-        new = self.family.OO.TreeSet(v for v in value if v is not None)
-        if not new:
-            self.unindex_doc(doc_id)
-        else:
-            values_to_documents = self.values_to_documents
-            documents_to_values = self.documents_to_values
-            old = documents_to_values.get(doc_id)
-            if old is None:
-                documents_to_values[doc_id] = new
-                self.documentCount.change(1)
-                self._add_values(doc_id, new)
-            else:
-                removed = self.family.OO.difference(old, new)
-                added = self.family.OO.difference(new, old)
-                for v in removed:
-                    old.remove(v)
-                    docs = values_to_documents.get(v)
-                    docs.remove(doc_id)
-                    if not docs:
-                        del values_to_documents[v]
-                        self.wordCount.change(-1)
-                old.update(added)
-                self._add_values(doc_id, added)
-
-    def unindex_doc(self, doc_id):
-        documents_to_values = self.documents_to_values
-        values = documents_to_values.get(doc_id)
-        if values is not None:
-            values_to_documents = self.values_to_documents
-            self.documentCount.change(-1)
-            del documents_to_values[doc_id]
-            for v in values:
-                docs = values_to_documents.get(v)
-                docs.remove(doc_id)
-                if not docs:
-                    del values_to_documents[v]
-                    self.wordCount.change(-1)
-
-    def apply(self, query): # any_of, any, between, none, all_of
-        values_to_documents = self.values_to_documents
-        query_type, query = parseQuery(query)
-        if query_type is None:
-            res = None
-        elif query_type == 'any_of':
-            res = self.family.IF.Bucket()
-            for v in query:
-                _, res = self.family.IF.weightedUnion(
-                    res, values_to_documents.get(v))
-        elif query_type == 'any':
-            if query is None:
-                res = self.family.IF.Set(self.ids())
-            else:
-                assert zc.catalog.interfaces.IExtent.providedBy(query)
-                res = query & self.family.IF.Set(self.ids())
-        elif query_type == 'all_of':
-            res = None
-            values = iter(query)
-            empty = self.family.IF.TreeSet()
-            try:
-                res = values_to_documents.get(values.next(), empty)
-            except StopIteration:
-                res = empty
-            while res:
-                try:
-                    v = values.next()
-                except StopIteration:
-                    break
-                res = self.family.IF.intersection(
-                    res, values_to_documents.get(v, empty))
-        elif query_type == 'between':
-            res = self.family.IF.Bucket()
-            for v in values_to_documents.keys(*query):
-                _, res = self.family.IF.weightedUnion(
-                    res, values_to_documents.get(v))
-        elif query_type == 'none':
-            assert zc.catalog.interfaces.IExtent.providedBy(query)
-            res = query - self.family.IF.Set(self.ids())
-        else:
-            raise ValueError(
-                "unknown query type", query_type)
-        return res
-
-class NormalizationWrapper(persistent.Persistent):
-
-    interface.implements(zc.catalog.interfaces.INormalizationWrapper)
-
-    index = normalizer = None
-    collection_index = False
-
-    def documentCount(self):
-        return self.index.documentCount()
-
-    def wordCount(self):
-        return self.index.wordCount()
-
-    def clear(self):
-        """see zope.index.interfaces.IInjection.clear"""
-        return self.index.clear()
-
-    def __init__(self, index, normalizer, collection_index=False):
-        self.index = index
-        self.normalizer = normalizer
-        self.collection_index = collection_index
-
-    def index_doc(self, doc_id, value):
-        if self.collection_index:
-            self.index.index_doc(
-                doc_id, (self.normalizer.value(v) for v in value))
-        else:
-            self.index.index_doc(doc_id, self.normalizer.value(value))
-
-    def unindex_doc(self, doc_id):
-        self.index.unindex_doc(doc_id)
-
-    def apply(self, query):
-        query_type, query = parseQuery(query)
-        if query_type == 'any_of':
-            res = set()
-            for v in query:
-                res.update(self.normalizer.any(v, self.index))
-        elif query_type == 'all_of':
-            res = [self.normalizer.all(v, self.index) for v in query]
-        elif query_type == 'between':
-            query = tuple(query) # collect iterators
-            len_query = len(query)
-            max_exclude = len_query >= 4 and bool(query[3])
-            min_exclude = len_query >= 3 and bool(query[2])
-            max = len_query >= 2 and query[1] and self.normalizer.maximum(
-                query[1], self.index, max_exclude) or None
-            min = len_query >= 1 and query[0] and self.normalizer.minimum(
-                query[0], self.index, min_exclude) or None
-            res = (min, max, min_exclude, max_exclude)
-        else:
-            res = query
-        return self.index.apply({query_type: res})
-
-    def minValue(self, min=None):
-        if min is not None:
-            min = self.normalizer.minimum(min, self.index)
-        return self.index.minValue(min)
-
-    def maxValue(self, max=None):
-        if max is not None:
-            max = self.normalizer.maximum(max, self.index)
-        return self.index.maxValue(max)
-
-    def values(self, min=None, max=None, excludemin=False, excludemax=False,
-               doc_id=None):
-        if min is not None:
-            min = self.normalizer.minimum(min, self.index)
-        if max is not None:
-            max = self.normalizer.maximum(max, self.index)
-        return self.index.values(min, max, excludemin, excludemax,
-                doc_id=doc_id)
-
-    def containsValue(self, value):
-        return self.index.containsValue(value)
-
-    def ids(self):
-        return self.index.ids()
-
-
-class CallableWrapper(persistent.Persistent):
-
-    interface.implements(zc.catalog.interfaces.ICallableWrapper)
-
-    converter = None
-    index = None
-
-    def __init__(self, index, converter):
-        self.index = index
-        self.converter = converter
-
-    def index_doc(self, docid, value):
-        "See zope.index.interfaces.IInjection"
-        self.index.index_doc(docid, self.converter(value))
-
-    def __getattr__(self, name):
-        return getattr(self.index, name)
-
-
-def set_resolution(value, resolution):
-    resolution += 2
-    if resolution < 6:
-        args = []
-        args.extend(value.timetuple()[:resolution+1])
-        args.extend([0]*(6-resolution))
-        args.append(value.tzinfo)
-        value = datetime.datetime(*args)
-    return value
-
-def get_request():
-    i = zope.security.management.queryInteraction()
-    if i is not None:
-        for p in i.participations:
-            if IRequest.providedBy(p):
-                return p
-    return None
-
-def get_tz(default=pytz.reference.Local):
-    request = get_request()
-    if request is None:
-        return default
-    return zope.interface.common.idatetime.ITZInfo(request, default)
-
-def add_tz(value):
-    if type(value) is datetime.datetime:
-        if value.tzinfo is None:
-            value = value.replace(tzinfo=get_tz())
-        return value
-    else:
-        raise ValueError(value)
-
-def day_end(value):
-    return (
-        datetime.datetime.combine(
-            value, datetime.time(tzinfo=get_tz())) +
-        datetime.timedelta(days=1) - # separate for daylight savings
-        datetime.timedelta(microseconds=1))
-
-def day_begin(value):
-    return datetime.datetime.combine(
-        value, datetime.time(tzinfo=get_tz()))
-
-
-class DateTimeNormalizer(persistent.Persistent):
-
-    interface.implements(zc.catalog.interfaces.IDateTimeNormalizer)
-    def __init__(self, resolution=2):
-        self.resolution = resolution
-        # 0, 1, 2, 3, 4
-        # day, hour, minute, second, microsecond
-
-    def value(self, value):
-        if not isinstance(value, datetime.datetime) or value.tzinfo is None:
-            raise ValueError(
-                _('This index only indexes timezone-aware datetimes.'))
-        return set_resolution(value, self.resolution)
-
-    def any(self, value, index):
-        if type(value) is datetime.date:
-            start = datetime.datetime.combine(
-                value, datetime.time(tzinfo=get_tz()))
-            stop = start + datetime.timedelta(days=1)
-            return index.values(start, stop, False, True)
-        return (add_tz(value),)
-
-    def all(self, value, index):
-        return add_tz(value)
-
-    def minimum(self, value, index, exclude=False):
-        if type(value) is datetime.date:
-            if exclude:
-                return day_end(value)
-            else:
-                return day_begin(value)
-        return add_tz(value)
-
-    def maximum(self, value, index, exclude=False):
-        if type(value) is datetime.date:
-            if exclude:
-                return day_begin(value)
-            else:
-                return day_end(value)
-        return add_tz(value)
-
- at interface.implementer(
-    zope.interface.implementedBy(NormalizationWrapper),
-    zc.catalog.interfaces.IValueIndex)
-def DateTimeValueIndex(resolution=2): # 2 == minute; note that hour is good
-    # for timezone-aware per-day searches
-    ix = NormalizationWrapper(ValueIndex(), DateTimeNormalizer(resolution))
-    interface.directlyProvides(ix, zc.catalog.interfaces.IValueIndex)
-    return ix
-
- at interface.implementer(
-    zope.interface.implementedBy(NormalizationWrapper),
-    zc.catalog.interfaces.ISetIndex)
-def DateTimeSetIndex(resolution=2): # 2 == minute; note that hour is good
-    # for timezone-aware per-day searches
-    ix = NormalizationWrapper(SetIndex(), DateTimeNormalizer(resolution), True)
-    interface.directlyProvides(ix, zc.catalog.interfaces.ISetIndex)    
-    return ix

Copied: zc.catalog/tags/1.4.1/src/zc/catalog/index.py (from rev 97345, zc.catalog/trunk/src/zc/catalog/index.py)
===================================================================
--- zc.catalog/tags/1.4.1/src/zc/catalog/index.py	                        (rev 0)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/index.py	2009-02-27 14:44:33 UTC (rev 97355)
@@ -0,0 +1,551 @@
+#############################################################################
+#
+# Copyright (c) 2006 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""indexes, as might be found in zope.index
+
+$Id: index.py 2918 2005-07-19 22:12:38Z jim $
+"""
+import sys
+import datetime
+import pytz.reference
+import BTrees
+import persistent
+from BTrees import Length
+from BTrees.Interfaces import IMerge
+
+from zope import component, interface
+import zope.component.interfaces
+import zope.interface.common.idatetime
+import zope.index.interfaces
+from zope.index.field.sorting import SortingIndexMixin
+import zope.security.management
+from zope.publisher.interfaces import IRequest
+import zc.catalog.interfaces
+from zc.catalog.i18n import _
+
+
+class FamilyProperty(object):
+
+    __name__ = "family"
+
+    def __get__(self, instance, type=None):
+        if instance is None:
+            return self
+        d = instance.__dict__
+        if "family" in d:
+            return d["family"]
+        if "btreemodule" in d:
+            iftype = d["btreemodule"].split(".")[-1][:2]
+            if iftype == "IF":
+                d["family"] = BTrees.family32
+            elif iftype == "LF":
+                d["family"] = BTrees.family64
+            else:
+                raise ValueError("can't determine btree family based on"
+                                 " btreemodule of %r" % (iftype,))
+        else:
+            d["family"] = BTrees.family32
+        self._clear_old_cruft(instance)
+        return d["family"]
+
+    def __set__(self, instance, value):
+        instance.__dict__["family"] = value
+        self._clear_old_cruft(instance)
+
+    def _clear_old_cruft(self, instance):
+        d = instance.__dict__
+        if "btreemodule" in d:
+            del d["btreemodule"]
+        if "IOBTree" in d:
+            del d["IOBTree"]
+        if "BTreeAPI" in d:
+            del d["BTreeAPI"]
+
+
+class AbstractIndex(persistent.Persistent):
+
+    interface.implements(zope.index.interfaces.IInjection,
+                         zope.index.interfaces.IIndexSearch,
+                         zope.index.interfaces.IStatistics,
+                         zc.catalog.interfaces.IIndexValues,
+                         )
+
+    family = FamilyProperty()
+
+    def __init__(self, family=None):
+        if family is not None:
+            self.family = family
+        self.clear()
+
+    # These three are deprecated (they were never interface), but can
+    # all be computed from the family attribute:
+
+    @property
+    def btreemodule(self):
+        return self.family.IF.__name__
+
+    @property
+    def BTreeAPI(self):
+        return self.family.IF
+
+    @property
+    def IOBTree(self):
+        return self.family.IO.BTree
+
+    def clear(self):
+        self.values_to_documents = self.family.OO.BTree()
+        self.documents_to_values = self.family.IO.BTree()
+        self.documentCount = Length.Length(0)
+        self.wordCount = Length.Length(0)
+
+    def minValue(self, min=None):
+        if min is None:
+            return self.values_to_documents.minKey()
+        else:
+            return self.values_to_documents.minKey(min)
+
+    def maxValue(self, max=None):
+        if max is None:
+            return self.values_to_documents.maxKey()
+        else:
+            return self.values_to_documents.maxKey(max)
+
+    def values(self, min=None, max=None, excludemin=False, excludemax=False,
+               doc_id=None):
+        if doc_id is None:
+            return iter(self.values_to_documents.keys(
+                min, max, excludemin, excludemax))
+        else:
+            values = self.documents_to_values.get(doc_id)
+            if values is None:
+                return ()
+            else:
+                return iter(values.keys(min, max, excludemin, excludemax))
+
+    def containsValue(self, value):
+        return bool(self.values_to_documents.has_key(value))
+
+    def ids(self):
+        return self.documents_to_values.keys()
+
+def parseQuery(query):
+    if isinstance(query, dict):
+        if len(query) > 1:
+            raise ValueError(
+                'may only pass one of key, value pair')
+        elif not query:
+            return None, None
+        query_type, query = query.items()[0]
+        query_type = query_type.lower()
+    else:
+        raise ValueError('may only pass a dict to apply')
+    return query_type, query
+
+class ValueIndex(SortingIndexMixin, AbstractIndex):
+
+    interface.implements(zc.catalog.interfaces.IValueIndex)
+
+    # attributes used by sorting mixin
+    _sorting_num_docs_attr = 'documentCount'        # Length object
+    _sorting_fwd_index_attr = 'values_to_documents' # forward BTree index
+    _sorting_rev_index_attr = 'documents_to_values' # reverse BTree index
+
+    def _add_value(self, doc_id, added):
+        values_to_documents = self.values_to_documents
+        docs = values_to_documents.get(added)
+        if docs is None:
+            values_to_documents[added] = self.family.IF.TreeSet((doc_id,))
+            self.wordCount.change(1)
+        else:
+            docs.insert(doc_id)
+
+    def index_doc(self, doc_id, value):
+        if value is None:
+            self.unindex_doc(doc_id)
+        else:
+            values_to_documents = self.values_to_documents
+            documents_to_values = self.documents_to_values
+            old = documents_to_values.get(doc_id)
+            documents_to_values[doc_id] = value
+            if old is None:
+                self.documentCount.change(1)
+            elif old != value:
+                docs = values_to_documents.get(old)
+                docs.remove(doc_id)
+                if not docs:
+                    del values_to_documents[old]
+                    self.wordCount.change(-1)
+            self._add_value(doc_id, value)
+
+    def unindex_doc(self, doc_id):
+        documents_to_values = self.documents_to_values
+        value = documents_to_values.get(doc_id)
+        if value is not None:
+            values_to_documents = self.values_to_documents
+            self.documentCount.change(-1)
+            del documents_to_values[doc_id]
+            docs = values_to_documents.get(value)
+            docs.remove(doc_id)
+            if not docs:
+                del values_to_documents[value]
+                self.wordCount.change(-1)
+
+    def apply(self, query): # any_of, any, between, none,
+        values_to_documents = self.values_to_documents
+        query_type, query = parseQuery(query)
+        if query_type is None:
+            res = None
+        elif query_type == 'any_of':
+            res = self.family.IF.multiunion(
+                [s for s in (values_to_documents.get(v) for v in query)
+                 if s is not None])
+        elif query_type == 'any':
+            if query is None:
+                res = self.family.IF.Set(self.ids())
+            else:
+                assert zc.catalog.interfaces.IExtent.providedBy(query)
+                res = query & self.family.IF.Set(self.ids())
+        elif query_type == 'between':
+            res = self.family.IF.multiunion(
+                [s for s in (values_to_documents.get(v) for v in
+                             values_to_documents.keys(*query))
+                 if s is not None])
+        elif query_type == 'none':
+            assert zc.catalog.interfaces.IExtent.providedBy(query)
+            res = query - self.family.IF.Set(self.ids())
+        else:
+            raise ValueError(
+                "unknown query type", query_type)
+        return res
+
+    def values(self, min=None, max=None, excludemin=False, excludemax=False,
+               doc_id=None):
+        if doc_id is None:
+            return iter(self.values_to_documents.keys(
+                min, max, excludemin, excludemax))
+        else:
+            value = self.documents_to_values.get(doc_id)
+            if (value is None or
+                min is not None and (
+                    value < min or excludemin and value == min) or
+                max is not None and (
+                    value > max or excludemax and value == max)):
+                return ()
+            else:
+                return (value,)
+
+class SetIndex(AbstractIndex):
+
+    interface.implements(zc.catalog.interfaces.ISetIndex)
+
+    def _add_values(self, doc_id, added):
+        values_to_documents = self.values_to_documents
+        for v in added:
+            docs = values_to_documents.get(v)
+            if docs is None:
+                values_to_documents[v] = self.family.IF.TreeSet((doc_id,))
+                self.wordCount.change(1)
+            else:
+                docs.insert(doc_id)
+
+    def index_doc(self, doc_id, value):
+        new = self.family.OO.TreeSet(v for v in value if v is not None)
+        if not new:
+            self.unindex_doc(doc_id)
+        else:
+            values_to_documents = self.values_to_documents
+            documents_to_values = self.documents_to_values
+            old = documents_to_values.get(doc_id)
+            if old is None:
+                documents_to_values[doc_id] = new
+                self.documentCount.change(1)
+                self._add_values(doc_id, new)
+            else:
+                removed = self.family.OO.difference(old, new)
+                added = self.family.OO.difference(new, old)
+                for v in removed:
+                    old.remove(v)
+                    docs = values_to_documents.get(v)
+                    docs.remove(doc_id)
+                    if not docs:
+                        del values_to_documents[v]
+                        self.wordCount.change(-1)
+                old.update(added)
+                self._add_values(doc_id, added)
+
+    def unindex_doc(self, doc_id):
+        documents_to_values = self.documents_to_values
+        values = documents_to_values.get(doc_id)
+        if values is not None:
+            values_to_documents = self.values_to_documents
+            self.documentCount.change(-1)
+            del documents_to_values[doc_id]
+            for v in values:
+                docs = values_to_documents.get(v)
+                docs.remove(doc_id)
+                if not docs:
+                    del values_to_documents[v]
+                    self.wordCount.change(-1)
+
+    def apply(self, query): # any_of, any, between, none, all_of
+        values_to_documents = self.values_to_documents
+        query_type, query = parseQuery(query)
+        if query_type is None:
+            res = None
+        elif query_type == 'any_of':
+            res = self.family.IF.Bucket()
+            for v in query:
+                _, res = self.family.IF.weightedUnion(
+                    res, values_to_documents.get(v))
+        elif query_type == 'any':
+            if query is None:
+                res = self.family.IF.Set(self.ids())
+            else:
+                assert zc.catalog.interfaces.IExtent.providedBy(query)
+                res = query & self.family.IF.Set(self.ids())
+        elif query_type == 'all_of':
+            res = None
+            values = iter(query)
+            empty = self.family.IF.TreeSet()
+            try:
+                res = values_to_documents.get(values.next(), empty)
+            except StopIteration:
+                res = empty
+            while res:
+                try:
+                    v = values.next()
+                except StopIteration:
+                    break
+                res = self.family.IF.intersection(
+                    res, values_to_documents.get(v, empty))
+        elif query_type == 'between':
+            res = self.family.IF.Bucket()
+            for v in values_to_documents.keys(*query):
+                _, res = self.family.IF.weightedUnion(
+                    res, values_to_documents.get(v))
+        elif query_type == 'none':
+            assert zc.catalog.interfaces.IExtent.providedBy(query)
+            res = query - self.family.IF.Set(self.ids())
+        else:
+            raise ValueError(
+                "unknown query type", query_type)
+        return res
+
+class NormalizationWrapper(persistent.Persistent):
+
+    interface.implements(zc.catalog.interfaces.INormalizationWrapper)
+
+    index = normalizer = None
+    collection_index = False
+
+    def documentCount(self):
+        return self.index.documentCount()
+
+    def wordCount(self):
+        return self.index.wordCount()
+
+    def clear(self):
+        """see zope.index.interfaces.IInjection.clear"""
+        return self.index.clear()
+
+    def __init__(self, index, normalizer, collection_index=False):
+        self.index = index
+        if zope.index.interfaces.IIndexSort.providedBy(index):
+            zope.interface.alsoProvides(self, zope.index.interfaces.IIndexSort)
+        self.normalizer = normalizer
+        self.collection_index = collection_index
+
+    def index_doc(self, doc_id, value):
+        if self.collection_index:
+            self.index.index_doc(
+                doc_id, (self.normalizer.value(v) for v in value))
+        else:
+            self.index.index_doc(doc_id, self.normalizer.value(value))
+
+    def unindex_doc(self, doc_id):
+        self.index.unindex_doc(doc_id)
+
+    def apply(self, query):
+        query_type, query = parseQuery(query)
+        if query_type == 'any_of':
+            res = set()
+            for v in query:
+                res.update(self.normalizer.any(v, self.index))
+        elif query_type == 'all_of':
+            res = [self.normalizer.all(v, self.index) for v in query]
+        elif query_type == 'between':
+            query = tuple(query) # collect iterators
+            len_query = len(query)
+            max_exclude = len_query >= 4 and bool(query[3])
+            min_exclude = len_query >= 3 and bool(query[2])
+            max = len_query >= 2 and query[1] and self.normalizer.maximum(
+                query[1], self.index, max_exclude) or None
+            min = len_query >= 1 and query[0] and self.normalizer.minimum(
+                query[0], self.index, min_exclude) or None
+            res = (min, max, min_exclude, max_exclude)
+        else:
+            res = query
+        return self.index.apply({query_type: res})
+
+    def minValue(self, min=None):
+        if min is not None:
+            min = self.normalizer.minimum(min, self.index)
+        return self.index.minValue(min)
+
+    def maxValue(self, max=None):
+        if max is not None:
+            max = self.normalizer.maximum(max, self.index)
+        return self.index.maxValue(max)
+
+    def values(self, min=None, max=None, excludemin=False, excludemax=False,
+               doc_id=None):
+        if min is not None:
+            min = self.normalizer.minimum(min, self.index)
+        if max is not None:
+            max = self.normalizer.maximum(max, self.index)
+        return self.index.values(min, max, excludemin, excludemax,
+                doc_id=doc_id)
+
+    def containsValue(self, value):
+        return self.index.containsValue(value)
+
+    def ids(self):
+        return self.index.ids()
+
+    @property
+    def sort(self):
+        # delegate upstream or raise AttributeError
+        return self.index.sort
+
+class CallableWrapper(persistent.Persistent):
+
+    interface.implements(zc.catalog.interfaces.ICallableWrapper)
+
+    converter = None
+    index = None
+
+    def __init__(self, index, converter):
+        self.index = index
+        self.converter = converter
+
+    def index_doc(self, docid, value):
+        "See zope.index.interfaces.IInjection"
+        self.index.index_doc(docid, self.converter(value))
+
+    def __getattr__(self, name):
+        return getattr(self.index, name)
+
+
+def set_resolution(value, resolution):
+    resolution += 2
+    if resolution < 6:
+        args = []
+        args.extend(value.timetuple()[:resolution+1])
+        args.extend([0]*(6-resolution))
+        args.append(value.tzinfo)
+        value = datetime.datetime(*args)
+    return value
+
+def get_request():
+    i = zope.security.management.queryInteraction()
+    if i is not None:
+        for p in i.participations:
+            if IRequest.providedBy(p):
+                return p
+    return None
+
+def get_tz(default=pytz.reference.Local):
+    request = get_request()
+    if request is None:
+        return default
+    return zope.interface.common.idatetime.ITZInfo(request, default)
+
+def add_tz(value):
+    if type(value) is datetime.datetime:
+        if value.tzinfo is None:
+            value = value.replace(tzinfo=get_tz())
+        return value
+    else:
+        raise ValueError(value)
+
+def day_end(value):
+    return (
+        datetime.datetime.combine(
+            value, datetime.time(tzinfo=get_tz())) +
+        datetime.timedelta(days=1) - # separate for daylight savings
+        datetime.timedelta(microseconds=1))
+
+def day_begin(value):
+    return datetime.datetime.combine(
+        value, datetime.time(tzinfo=get_tz()))
+
+
+class DateTimeNormalizer(persistent.Persistent):
+
+    interface.implements(zc.catalog.interfaces.IDateTimeNormalizer)
+    def __init__(self, resolution=2):
+        self.resolution = resolution
+        # 0, 1, 2, 3, 4
+        # day, hour, minute, second, microsecond
+
+    def value(self, value):
+        if not isinstance(value, datetime.datetime) or value.tzinfo is None:
+            raise ValueError(
+                _('This index only indexes timezone-aware datetimes.'))
+        return set_resolution(value, self.resolution)
+
+    def any(self, value, index):
+        if type(value) is datetime.date:
+            start = datetime.datetime.combine(
+                value, datetime.time(tzinfo=get_tz()))
+            stop = start + datetime.timedelta(days=1)
+            return index.values(start, stop, False, True)
+        return (add_tz(value),)
+
+    def all(self, value, index):
+        return add_tz(value)
+
+    def minimum(self, value, index, exclude=False):
+        if type(value) is datetime.date:
+            if exclude:
+                return day_end(value)
+            else:
+                return day_begin(value)
+        return add_tz(value)
+
+    def maximum(self, value, index, exclude=False):
+        if type(value) is datetime.date:
+            if exclude:
+                return day_begin(value)
+            else:
+                return day_end(value)
+        return add_tz(value)
+
+ at interface.implementer(
+    zope.interface.implementedBy(NormalizationWrapper),
+    zope.index.interfaces.IIndexSort,
+    zc.catalog.interfaces.IValueIndex)
+def DateTimeValueIndex(resolution=2): # 2 == minute; note that hour is good
+    # for timezone-aware per-day searches
+    ix = NormalizationWrapper(ValueIndex(), DateTimeNormalizer(resolution))
+    interface.alsoProvides(ix, zc.catalog.interfaces.IValueIndex)
+    return ix
+
+ at interface.implementer(
+    zope.interface.implementedBy(NormalizationWrapper),
+    zc.catalog.interfaces.ISetIndex)
+def DateTimeSetIndex(resolution=2): # 2 == minute; note that hour is good
+    # for timezone-aware per-day searches
+    ix = NormalizationWrapper(SetIndex(), DateTimeNormalizer(resolution), True)
+    interface.alsoProvides(ix, zc.catalog.interfaces.ISetIndex)    
+    return ix

Deleted: zc.catalog/tags/1.4.1/src/zc/catalog/normalizedindex.txt
===================================================================
--- zc.catalog/trunk/src/zc/catalog/normalizedindex.txt	2009-02-27 09:39:25 UTC (rev 97335)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/normalizedindex.txt	2009-02-27 14:44:33 UTC (rev 97355)
@@ -1,324 +0,0 @@
-================
-Normalized Index
-================
-
-The index module provides a normalizing wrapper, a DateTime normalizer, and
-a set index and a value index normalized with the DateTime normalizer.
-
-The normalizing wrapper implements a full complement of index interfaces--
-zope.index.interfaces.IInjection, zope.index.interfaces.IIndexSearch,
-zope.index.interfaces.IStatistics, and zc.catalog.interfaces.IIndexValues--
-and delegates all of the behavior to the wrapped index, normalizing values
-using the normalizer before the index sees them.
-
-The normalizing wrapper currently only supports queries offered by
-zc.catalog.interfaces.ISetIndex and zc.catalog.interfaces.IValueIndex.
-
-The normalizer interface requires the following methods, as defined in the
-interface:
-
-    def value(value):
-        """normalize or check constraints for an input value; raise an error
-        or return the value to be indexed."""
-
-    def any(value, index):
-        """normalize a query value for a "any_of" search; return a sequence of
-        values."""
-
-    def all(value, index):
-        """Normalize a query value for an "all_of" search; return the value
-        for query"""
-
-    def minimum(value, index):
-        """normalize a query value for minimum of a range; return the value for
-        query"""
-
-    def maximum(value, index):
-        """normalize a query value for maximum of a range; return the value for
-        query"""
-
-The DateTime normalizer performs the following normalizations and validations.
-Whenever a timezone is needed, it tries to get a request from the current
-interaction and adapt it to zope.interface.common.idatetime.ITZInfo; failing
-that (no request or no adapter) it uses the system local timezone.
-
-- input values must be datetimes with a timezone.  They are normalized to the
-  resolution specified when the normalizer is created: a resolution of 0
-  normalizes values to days; a resolution of 1 to hours; 2 to minutes; 3 to
-  seconds; and 4 to microseconds.
-
-- 'any' values may be timezone-aware datetimes, timezone-naive datetimes,
-  or dates.  dates are converted to any value from the start to the end of the
-  given date in the found timezone, as described above.  timezone-naive
-  datetimes get the found timezone.
-
-- 'all' values may be timezone-aware datetimes or timezone-naive datetimes.
-  timezone-naive datetimes get the found timezone.
-
-- 'minimum' values may be timezone-aware datetimes, timezone-naive datetimes,
-  or dates.  dates are converted to the start of the given date in the found
-  timezone, as described above.  timezone-naive datetimes get the found
-  timezone.
-
-- 'maximum' values may be timezone-aware datetimes, timezone-naive datetimes,
-  or dates.  dates are converted to the end of the given date in the found
-  timezone, as described above.  timezone-naive datetimes get the found
-  timezone.
-
-Let's look at the DateTime normalizer first, and then an integration of it
-with the normalizing wrapper and the value and set indexes.
-
-The indexed values are parsed with 'value'.
-
-    >>> from zc.catalog.index import DateTimeNormalizer
-    >>> n = DateTimeNormalizer() # defaults to minutes
-    >>> import datetime
-    >>> import pytz
-    >>> naive_datetime = datetime.datetime(2005, 7, 15, 11, 21, 32, 104)
-    >>> date = naive_datetime.date()
-    >>> aware_datetime = naive_datetime.replace(
-    ...     tzinfo=pytz.timezone('US/Eastern'))
-    >>> n.value(naive_datetime)
-    Traceback (most recent call last):
-    ...
-    ValueError: This index only indexes timezone-aware datetimes.
-    >>> n.value(date)
-    Traceback (most recent call last):
-    ...
-    ValueError: This index only indexes timezone-aware datetimes.
-    >>> n.value(aware_datetime) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 15, 11, 21, tzinfo=<DstTzInfo 'US/Eastern'...>)
-
-If we specify a different resolution, the results are different.
-
-    >>> another = DateTimeNormalizer(1) # hours
-    >>> another.value(aware_datetime) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 15, 11, 0, tzinfo=<DstTzInfo 'US/Eastern'...>)
-
-Note that changing the resolution of an indexed value may create surprising
-results, because queries do not change their resolution.  Therefore, if you
-index something with a datetime with a finer resolution that the normalizer's,
-then searching for that datetime will not find the doc_id.
-
-Values in an 'any_of' query are parsed with 'any'.  'any' should return a
-sequence of values.  It requires an index, which we will mock up here.
-
-    >>> class DummyIndex(object):
-    ...     def values(self, start, stop, exclude_start, exclude_stop):
-    ...         assert not exclude_start and exclude_stop
-    ...         six_hours = datetime.timedelta(hours=6)
-    ...         res = []
-    ...         dt = start
-    ...         while dt < stop:
-    ...             res.append(dt)
-    ...             dt += six_hours
-    ...         return res
-    ...
-    >>> index = DummyIndex()
-    >>> tuple(n.any(naive_datetime, index)) # doctest: +ELLIPSIS
-    (datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>),)
-    >>> tuple(n.any(aware_datetime, index)) # doctest: +ELLIPSIS
-    (datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>),)
-    >>> tuple(n.any(date, index)) # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
-    (datetime.datetime(2005, 7, 15, 0, 0, tzinfo=<...Local...>),
-     datetime.datetime(2005, 7, 15, 6, 0, tzinfo=<...Local...>),
-     datetime.datetime(2005, 7, 15, 12, 0, tzinfo=<...Local...>),
-     datetime.datetime(2005, 7, 15, 18, 0, tzinfo=<...Local...>))
-
-Values in an 'all_of' query are parsed with 'all'.
-
-    >>> n.all(naive_datetime, index) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
-    >>> n.all(aware_datetime, index) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
-    >>> n.all(date, index) # doctest: +ELLIPSIS
-    Traceback (most recent call last):
-    ...
-    ValueError: ...
-
-Minimum values in a 'between' query as well as those in other methods are
-parsed with 'minimum'.  They also take an optional exclude boolean, which
-indicates whether the minimum is to be excluded.  For datetimes, it only
-makes a difference if you pass in a date.
-
-    >>> n.minimum(naive_datetime, index) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
-    >>> n.minimum(naive_datetime, index, exclude=True) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
-
-    >>> n.minimum(aware_datetime, index) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
-    >>> n.minimum(aware_datetime, index, True) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
-
-    >>> n.minimum(date, index) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 15, 0, 0, tzinfo=<...Local...>)
-    >>> n.minimum(date, index, True) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 15, 23, 59, 59, 999999, tzinfo=<...Local...>)
-
-Maximum values in a 'between' query as well as those in other methods are
-parsed with 'maximum'.  They also take an optional exclude boolean, which
-indicates whether the maximum is to be excluded.  For datetimes, it only
-makes a difference if you pass in a date.
-
-    >>> n.maximum(naive_datetime, index) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
-    >>> n.maximum(naive_datetime, index, exclude=True) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
-
-    >>> n.maximum(aware_datetime, index) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
-    >>> n.maximum(aware_datetime, index, True) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
-
-    >>> n.maximum(date, index) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 15, 23, 59, 59, 999999, tzinfo=<...Local...>)
-    >>> n.maximum(date, index, True) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 15, 0, 0, tzinfo=<...Local...>)
-
-Now let's examine these normalizers in the context of a real index.
-
-    >>> from zc.catalog.index import DateTimeValueIndex, DateTimeSetIndex
-    >>> setindex = DateTimeSetIndex() # minutes resolution
-    >>> data = [] # generate some data
-    >>> def date_gen(
-    ...     start=aware_datetime,
-    ...     count=12,
-    ...     period=datetime.timedelta(hours=10)):
-    ...     dt = start
-    ...     ix = 0
-    ...     while ix < count:
-    ...         yield dt
-    ...         dt += period
-    ...         ix += 1
-    ...
-    >>> gen = date_gen()
-    >>> count = 0
-    >>> while True:
-    ...     try:
-    ...         next = [gen.next() for i in range(6)]
-    ...     except StopIteration:
-    ...         break
-    ...     data.append((count, next[0:1]))
-    ...     count += 1
-    ...     data.append((count, next[1:3]))
-    ...     count += 1
-    ...     data.append((count, next[3:6]))
-    ...     count += 1
-    ...
-    >>> print data # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
-    [(0,
-      [datetime.datetime(2005, 7, 15, 11, 21, 32, 104, ...<...Eastern...>)]),
-     (1,
-      [datetime.datetime(2005, 7, 15, 21, 21, 32, 104, ...<...Eastern...>),
-       datetime.datetime(2005, 7, 16, 7, 21, 32, 104, ...<...Eastern...>)]),
-     (2,
-      [datetime.datetime(2005, 7, 16, 17, 21, 32, 104, ...<...Eastern...>),
-       datetime.datetime(2005, 7, 17, 3, 21, 32, 104, ...<...Eastern...>),
-       datetime.datetime(2005, 7, 17, 13, 21, 32, 104, ...<...Eastern...>)]),
-     (3,
-      [datetime.datetime(2005, 7, 17, 23, 21, 32, 104, ...<...Eastern...>)]),
-     (4,
-      [datetime.datetime(2005, 7, 18, 9, 21, 32, 104, ...<...Eastern...>),
-       datetime.datetime(2005, 7, 18, 19, 21, 32, 104, ...<...Eastern...>)]),
-     (5,
-      [datetime.datetime(2005, 7, 19, 5, 21, 32, 104, ...<...Eastern...>),
-       datetime.datetime(2005, 7, 19, 15, 21, 32, 104, ...<...Eastern...>),
-       datetime.datetime(2005, 7, 20, 1, 21, 32, 104, ...<...Eastern...>)])]
-    >>> data_dict = dict(data)
-    >>> for doc_id, value in data:
-    ...     setindex.index_doc(doc_id, value)
-    ...
-    >>> list(setindex.ids())
-    [0, 1, 2, 3, 4, 5]
-    >>> set(setindex.values()) == set(
-    ...     setindex.normalizer.value(v) for v in date_gen())
-    True
-
-For the searches, we will actually use a request and interaction, with an
-adapter that returns the Eastern timezone.  This makes the examples less
-dependent on the machine that they use.
-
-    >>> import zope.security.management
-    >>> import zope.publisher.browser
-    >>> import zope.interface.common.idatetime
-    >>> import zope.publisher.interfaces
-    >>> request = zope.publisher.browser.TestRequest()
-    >>> zope.security.management.newInteraction(request)
-    >>> from zope import interface, component
-    >>> @interface.implementer(zope.interface.common.idatetime.ITZInfo)
-    ... @component.adapter(zope.publisher.interfaces.IRequest)
-    ... def tzinfo(req):
-    ...     return pytz.timezone('US/Eastern')
-    ...
-    >>> component.provideAdapter(tzinfo)
-    >>> n.all(naive_datetime, index).tzinfo is pytz.timezone('US/Eastern')
-    True
-
-    >>> set(setindex.apply({'any_of': (datetime.date(2005, 7, 17),
-    ...                                datetime.date(2005, 7, 20),
-    ...                                datetime.date(2005, 12, 31))})) == set(
-    ...     (2, 3, 5))
-    True
-
-Note that this search is using the normalized values.
-
-    >>> set(setindex.apply({'all_of': (
-    ...     datetime.datetime(
-    ...         2005, 7, 16, 7, 21, tzinfo=pytz.timezone('US/Eastern')),
-    ...     datetime.datetime(
-    ...         2005, 7, 15, 21, 21, tzinfo=pytz.timezone('US/Eastern')),)})
-    ...     ) == set((1,))
-    True
-    >>> list(setindex.apply({'any': None}))
-    [0, 1, 2, 3, 4, 5]
-    >>> set(setindex.apply({'between': (
-    ...     datetime.datetime(2005, 4, 1, 12), datetime.datetime(2006, 5, 1))})
-    ...     ) == set((0, 1, 2, 3, 4, 5))
-    True
-    >>> set(setindex.apply({'between': (
-    ...     datetime.datetime(2005, 4, 1, 12), datetime.datetime(2006, 5, 1),
-    ...     True, True)})
-    ...     ) == set((0, 1, 2, 3, 4, 5))
-    True
-
-'between' searches should deal with dates well.
-
-    >>> set(setindex.apply({'between': (
-    ...     datetime.date(2005, 7, 16), datetime.date(2005, 7, 17))})
-    ...     ) == set((1, 2, 3))
-    True
-    >>> len(setindex.apply({'between': (
-    ...     datetime.date(2005, 7, 16), datetime.date(2005, 7, 17))})
-    ...     ) == len(setindex.apply({'between': (
-    ...     datetime.date(2005, 7, 15), datetime.date(2005, 7, 18),
-    ...     True, True)})
-    ...     )
-    True
-
-Removing docs works as usual.
-
-    >>> setindex.unindex_doc(1)
-    >>> list(setindex.ids())
-    [0, 2, 3, 4, 5]
-
-Value, Minvalue and Maxvalue can take timezone-less datetimes and dates.
-
-    >>> setindex.minValue() # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 15, 11, 21, ...<...Eastern...>)
-    >>> setindex.minValue(datetime.date(2005, 7, 17)) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 17, 3, 21, ...<...Eastern...>)
-
-    >>> setindex.maxValue() # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 20, 1, 21, ...<...Eastern...>)
-    >>> setindex.maxValue(datetime.date(2005, 7, 17)) # doctest: +ELLIPSIS
-    datetime.datetime(2005, 7, 17, 23, 21, ...<...Eastern...>)
-
-    >>> list(setindex.values(
-    ... datetime.date(2005, 7, 17), datetime.date(2005, 7, 17)))
-    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
-    [datetime.datetime(2005, 7, 17, 3, 21, ...<...Eastern...>),
-     datetime.datetime(2005, 7, 17, 13, 21, ...<...Eastern...>),
-     datetime.datetime(2005, 7, 17, 23, 21, ...<...Eastern...>)]
-
-    >>> zope.security.management.endInteraction() # TODO put in tests tearDown

Copied: zc.catalog/tags/1.4.1/src/zc/catalog/normalizedindex.txt (from rev 97345, zc.catalog/trunk/src/zc/catalog/normalizedindex.txt)
===================================================================
--- zc.catalog/tags/1.4.1/src/zc/catalog/normalizedindex.txt	                        (rev 0)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/normalizedindex.txt	2009-02-27 14:44:33 UTC (rev 97355)
@@ -0,0 +1,357 @@
+================
+Normalized Index
+================
+
+The index module provides a normalizing wrapper, a DateTime normalizer, and
+a set index and a value index normalized with the DateTime normalizer.
+
+The normalizing wrapper implements a full complement of index interfaces--
+zope.index.interfaces.IInjection, zope.index.interfaces.IIndexSearch,
+zope.index.interfaces.IStatistics, and zc.catalog.interfaces.IIndexValues--
+and delegates all of the behavior to the wrapped index, normalizing values
+using the normalizer before the index sees them.
+
+The normalizing wrapper currently only supports queries offered by
+zc.catalog.interfaces.ISetIndex and zc.catalog.interfaces.IValueIndex.
+
+The normalizer interface requires the following methods, as defined in the
+interface:
+
+    def value(value):
+        """normalize or check constraints for an input value; raise an error
+        or return the value to be indexed."""
+
+    def any(value, index):
+        """normalize a query value for a "any_of" search; return a sequence of
+        values."""
+
+    def all(value, index):
+        """Normalize a query value for an "all_of" search; return the value
+        for query"""
+
+    def minimum(value, index):
+        """normalize a query value for minimum of a range; return the value for
+        query"""
+
+    def maximum(value, index):
+        """normalize a query value for maximum of a range; return the value for
+        query"""
+
+The DateTime normalizer performs the following normalizations and validations.
+Whenever a timezone is needed, it tries to get a request from the current
+interaction and adapt it to zope.interface.common.idatetime.ITZInfo; failing
+that (no request or no adapter) it uses the system local timezone.
+
+- input values must be datetimes with a timezone.  They are normalized to the
+  resolution specified when the normalizer is created: a resolution of 0
+  normalizes values to days; a resolution of 1 to hours; 2 to minutes; 3 to
+  seconds; and 4 to microseconds.
+
+- 'any' values may be timezone-aware datetimes, timezone-naive datetimes,
+  or dates.  dates are converted to any value from the start to the end of the
+  given date in the found timezone, as described above.  timezone-naive
+  datetimes get the found timezone.
+
+- 'all' values may be timezone-aware datetimes or timezone-naive datetimes.
+  timezone-naive datetimes get the found timezone.
+
+- 'minimum' values may be timezone-aware datetimes, timezone-naive datetimes,
+  or dates.  dates are converted to the start of the given date in the found
+  timezone, as described above.  timezone-naive datetimes get the found
+  timezone.
+
+- 'maximum' values may be timezone-aware datetimes, timezone-naive datetimes,
+  or dates.  dates are converted to the end of the given date in the found
+  timezone, as described above.  timezone-naive datetimes get the found
+  timezone.
+
+Let's look at the DateTime normalizer first, and then an integration of it
+with the normalizing wrapper and the value and set indexes.
+
+The indexed values are parsed with 'value'.
+
+    >>> from zc.catalog.index import DateTimeNormalizer
+    >>> n = DateTimeNormalizer() # defaults to minutes
+    >>> import datetime
+    >>> import pytz
+    >>> naive_datetime = datetime.datetime(2005, 7, 15, 11, 21, 32, 104)
+    >>> date = naive_datetime.date()
+    >>> aware_datetime = naive_datetime.replace(
+    ...     tzinfo=pytz.timezone('US/Eastern'))
+    >>> n.value(naive_datetime)
+    Traceback (most recent call last):
+    ...
+    ValueError: This index only indexes timezone-aware datetimes.
+    >>> n.value(date)
+    Traceback (most recent call last):
+    ...
+    ValueError: This index only indexes timezone-aware datetimes.
+    >>> n.value(aware_datetime) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 15, 11, 21, tzinfo=<DstTzInfo 'US/Eastern'...>)
+
+If we specify a different resolution, the results are different.
+
+    >>> another = DateTimeNormalizer(1) # hours
+    >>> another.value(aware_datetime) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 15, 11, 0, tzinfo=<DstTzInfo 'US/Eastern'...>)
+
+Note that changing the resolution of an indexed value may create surprising
+results, because queries do not change their resolution.  Therefore, if you
+index something with a datetime with a finer resolution that the normalizer's,
+then searching for that datetime will not find the doc_id.
+
+Values in an 'any_of' query are parsed with 'any'.  'any' should return a
+sequence of values.  It requires an index, which we will mock up here.
+
+    >>> class DummyIndex(object):
+    ...     def values(self, start, stop, exclude_start, exclude_stop):
+    ...         assert not exclude_start and exclude_stop
+    ...         six_hours = datetime.timedelta(hours=6)
+    ...         res = []
+    ...         dt = start
+    ...         while dt < stop:
+    ...             res.append(dt)
+    ...             dt += six_hours
+    ...         return res
+    ...
+    >>> index = DummyIndex()
+    >>> tuple(n.any(naive_datetime, index)) # doctest: +ELLIPSIS
+    (datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>),)
+    >>> tuple(n.any(aware_datetime, index)) # doctest: +ELLIPSIS
+    (datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>),)
+    >>> tuple(n.any(date, index)) # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
+    (datetime.datetime(2005, 7, 15, 0, 0, tzinfo=<...Local...>),
+     datetime.datetime(2005, 7, 15, 6, 0, tzinfo=<...Local...>),
+     datetime.datetime(2005, 7, 15, 12, 0, tzinfo=<...Local...>),
+     datetime.datetime(2005, 7, 15, 18, 0, tzinfo=<...Local...>))
+
+Values in an 'all_of' query are parsed with 'all'.
+
+    >>> n.all(naive_datetime, index) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
+    >>> n.all(aware_datetime, index) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
+    >>> n.all(date, index) # doctest: +ELLIPSIS
+    Traceback (most recent call last):
+    ...
+    ValueError: ...
+
+Minimum values in a 'between' query as well as those in other methods are
+parsed with 'minimum'.  They also take an optional exclude boolean, which
+indicates whether the minimum is to be excluded.  For datetimes, it only
+makes a difference if you pass in a date.
+
+    >>> n.minimum(naive_datetime, index) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
+    >>> n.minimum(naive_datetime, index, exclude=True) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
+
+    >>> n.minimum(aware_datetime, index) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
+    >>> n.minimum(aware_datetime, index, True) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
+
+    >>> n.minimum(date, index) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 15, 0, 0, tzinfo=<...Local...>)
+    >>> n.minimum(date, index, True) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 15, 23, 59, 59, 999999, tzinfo=<...Local...>)
+
+Maximum values in a 'between' query as well as those in other methods are
+parsed with 'maximum'.  They also take an optional exclude boolean, which
+indicates whether the maximum is to be excluded.  For datetimes, it only
+makes a difference if you pass in a date.
+
+    >>> n.maximum(naive_datetime, index) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
+    >>> n.maximum(naive_datetime, index, exclude=True) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
+
+    >>> n.maximum(aware_datetime, index) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
+    >>> n.maximum(aware_datetime, index, True) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
+
+    >>> n.maximum(date, index) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 15, 23, 59, 59, 999999, tzinfo=<...Local...>)
+    >>> n.maximum(date, index, True) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 15, 0, 0, tzinfo=<...Local...>)
+
+Now let's examine these normalizers in the context of a real index.
+
+    >>> from zc.catalog.index import DateTimeValueIndex, DateTimeSetIndex
+    >>> setindex = DateTimeSetIndex() # minutes resolution
+    >>> data = [] # generate some data
+    >>> def date_gen(
+    ...     start=aware_datetime,
+    ...     count=12,
+    ...     period=datetime.timedelta(hours=10)):
+    ...     dt = start
+    ...     ix = 0
+    ...     while ix < count:
+    ...         yield dt
+    ...         dt += period
+    ...         ix += 1
+    ...
+    >>> gen = date_gen()
+    >>> count = 0
+    >>> while True:
+    ...     try:
+    ...         next = [gen.next() for i in range(6)]
+    ...     except StopIteration:
+    ...         break
+    ...     data.append((count, next[0:1]))
+    ...     count += 1
+    ...     data.append((count, next[1:3]))
+    ...     count += 1
+    ...     data.append((count, next[3:6]))
+    ...     count += 1
+    ...
+    >>> print data # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    [(0,
+      [datetime.datetime(2005, 7, 15, 11, 21, 32, 104, ...<...Eastern...>)]),
+     (1,
+      [datetime.datetime(2005, 7, 15, 21, 21, 32, 104, ...<...Eastern...>),
+       datetime.datetime(2005, 7, 16, 7, 21, 32, 104, ...<...Eastern...>)]),
+     (2,
+      [datetime.datetime(2005, 7, 16, 17, 21, 32, 104, ...<...Eastern...>),
+       datetime.datetime(2005, 7, 17, 3, 21, 32, 104, ...<...Eastern...>),
+       datetime.datetime(2005, 7, 17, 13, 21, 32, 104, ...<...Eastern...>)]),
+     (3,
+      [datetime.datetime(2005, 7, 17, 23, 21, 32, 104, ...<...Eastern...>)]),
+     (4,
+      [datetime.datetime(2005, 7, 18, 9, 21, 32, 104, ...<...Eastern...>),
+       datetime.datetime(2005, 7, 18, 19, 21, 32, 104, ...<...Eastern...>)]),
+     (5,
+      [datetime.datetime(2005, 7, 19, 5, 21, 32, 104, ...<...Eastern...>),
+       datetime.datetime(2005, 7, 19, 15, 21, 32, 104, ...<...Eastern...>),
+       datetime.datetime(2005, 7, 20, 1, 21, 32, 104, ...<...Eastern...>)])]
+    >>> data_dict = dict(data)
+    >>> for doc_id, value in data:
+    ...     setindex.index_doc(doc_id, value)
+    ...
+    >>> list(setindex.ids())
+    [0, 1, 2, 3, 4, 5]
+    >>> set(setindex.values()) == set(
+    ...     setindex.normalizer.value(v) for v in date_gen())
+    True
+
+For the searches, we will actually use a request and interaction, with an
+adapter that returns the Eastern timezone.  This makes the examples less
+dependent on the machine that they use.
+
+    >>> import zope.security.management
+    >>> import zope.publisher.browser
+    >>> import zope.interface.common.idatetime
+    >>> import zope.publisher.interfaces
+    >>> request = zope.publisher.browser.TestRequest()
+    >>> zope.security.management.newInteraction(request)
+    >>> from zope import interface, component
+    >>> @interface.implementer(zope.interface.common.idatetime.ITZInfo)
+    ... @component.adapter(zope.publisher.interfaces.IRequest)
+    ... def tzinfo(req):
+    ...     return pytz.timezone('US/Eastern')
+    ...
+    >>> component.provideAdapter(tzinfo)
+    >>> n.all(naive_datetime, index).tzinfo is pytz.timezone('US/Eastern')
+    True
+
+    >>> set(setindex.apply({'any_of': (datetime.date(2005, 7, 17),
+    ...                                datetime.date(2005, 7, 20),
+    ...                                datetime.date(2005, 12, 31))})) == set(
+    ...     (2, 3, 5))
+    True
+
+Note that this search is using the normalized values.
+
+    >>> set(setindex.apply({'all_of': (
+    ...     datetime.datetime(
+    ...         2005, 7, 16, 7, 21, tzinfo=pytz.timezone('US/Eastern')),
+    ...     datetime.datetime(
+    ...         2005, 7, 15, 21, 21, tzinfo=pytz.timezone('US/Eastern')),)})
+    ...     ) == set((1,))
+    True
+    >>> list(setindex.apply({'any': None}))
+    [0, 1, 2, 3, 4, 5]
+    >>> set(setindex.apply({'between': (
+    ...     datetime.datetime(2005, 4, 1, 12), datetime.datetime(2006, 5, 1))})
+    ...     ) == set((0, 1, 2, 3, 4, 5))
+    True
+    >>> set(setindex.apply({'between': (
+    ...     datetime.datetime(2005, 4, 1, 12), datetime.datetime(2006, 5, 1),
+    ...     True, True)})
+    ...     ) == set((0, 1, 2, 3, 4, 5))
+    True
+
+'between' searches should deal with dates well.
+
+    >>> set(setindex.apply({'between': (
+    ...     datetime.date(2005, 7, 16), datetime.date(2005, 7, 17))})
+    ...     ) == set((1, 2, 3))
+    True
+    >>> len(setindex.apply({'between': (
+    ...     datetime.date(2005, 7, 16), datetime.date(2005, 7, 17))})
+    ...     ) == len(setindex.apply({'between': (
+    ...     datetime.date(2005, 7, 15), datetime.date(2005, 7, 18),
+    ...     True, True)})
+    ...     )
+    True
+
+Removing docs works as usual.
+
+    >>> setindex.unindex_doc(1)
+    >>> list(setindex.ids())
+    [0, 2, 3, 4, 5]
+
+Value, Minvalue and Maxvalue can take timezone-less datetimes and dates.
+
+    >>> setindex.minValue() # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 15, 11, 21, ...<...Eastern...>)
+    >>> setindex.minValue(datetime.date(2005, 7, 17)) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 17, 3, 21, ...<...Eastern...>)
+
+    >>> setindex.maxValue() # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 20, 1, 21, ...<...Eastern...>)
+    >>> setindex.maxValue(datetime.date(2005, 7, 17)) # doctest: +ELLIPSIS
+    datetime.datetime(2005, 7, 17, 23, 21, ...<...Eastern...>)
+
+    >>> list(setindex.values(
+    ... datetime.date(2005, 7, 17), datetime.date(2005, 7, 17)))
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    [datetime.datetime(2005, 7, 17, 3, 21, ...<...Eastern...>),
+     datetime.datetime(2005, 7, 17, 13, 21, ...<...Eastern...>),
+     datetime.datetime(2005, 7, 17, 23, 21, ...<...Eastern...>)]
+
+    >>> zope.security.management.endInteraction() # TODO put in tests tearDown
+
+Sorting
+-------
+
+The normalization wrapper provides the zope.index.interfaces.IIndexSort
+interface if its upstream index provides it. For example, the
+DateTimeValueIndex will provide IIndexSort, because ValueIndex provides
+sorting. It will also delegate the ``sort`` method to the value index.
+
+    >>> from zc.catalog.index import DateTimeValueIndex
+    >>> from zope.index.interfaces import IIndexSort
+
+    >>> ix = DateTimeValueIndex()
+    >>> IIndexSort.providedBy(ix.index)
+    True
+    >>> IIndexSort.providedBy(ix)
+    True
+    >>> ix.sort.im_self is ix.index
+    True
+
+But it won't work for indexes that doesn't do sorting, for example
+DateTimeSetIndex.
+
+    >>> ix = DateTimeSetIndex()
+    >>> IIndexSort.providedBy(ix.index)
+    False
+    >>> IIndexSort.providedBy(ix)
+    False
+    >>> ix.sort
+    Traceback (most recent call last):
+    ...
+    AttributeError: 'SetIndex' object has no attribute 'sort'
+   
\ No newline at end of file

Deleted: zc.catalog/tags/1.4.1/src/zc/catalog/valueindex.txt
===================================================================
--- zc.catalog/trunk/src/zc/catalog/valueindex.txt	2009-02-27 09:39:25 UTC (rev 97335)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/valueindex.txt	2009-02-27 14:44:33 UTC (rev 97355)
@@ -1,222 +0,0 @@
-===========
-Value Index
-===========
-
-The valueindex is an index similar to, but more flexible than a standard Zope
-field index.  The index allows searches for documents that contain any of a
-set of values; between a set of values; any (non-None) values; and any empty
-values.
-
-Additionally, the index supports an interface that allows examination of the
-indexed values.
-
-It is as policy-free as possible, and is intended to be the engine for indexes
-with more policy, as well as being useful itself.
-
-On creation, the index has no wordCount, no documentCount, and is, as
-expected, fairly empty.
-
-    >>> from zc.catalog.index import ValueIndex
-    >>> index = ValueIndex()
-    >>> index.documentCount()
-    0
-    >>> index.wordCount()
-    0
-    >>> index.maxValue() # doctest: +ELLIPSIS
-    Traceback (most recent call last):
-    ...
-    ValueError:...
-    >>> index.minValue() # doctest: +ELLIPSIS
-    Traceback (most recent call last):
-    ...
-    ValueError:...
-    >>> list(index.values())
-    []
-    >>> len(index.apply({'any_of': (5,)}))
-    0
-
-The index supports indexing any value.  All values within a given index must
-sort consistently across Python versions.
-
-    >>> data = {1: 'a',
-    ...         2: 'b',
-    ...         3: 'a',
-    ...         4: 'c',
-    ...         5: 'd',
-    ...         6: 'c',
-    ...         7: 'c',
-    ...         8: 'b',
-    ...         9: 'c',
-    ... }
-    >>> for k, v in data.items():
-    ...     index.index_doc(k, v)
-    ...
-
-After indexing, the statistics and values match the newly entered content.
-
-    >>> list(index.values())
-    ['a', 'b', 'c', 'd']
-    >>> index.documentCount()
-    9
-    >>> index.wordCount()
-    4
-    >>> index.maxValue()
-    'd'
-    >>> index.minValue()
-    'a'
-    >>> list(index.ids())
-    [1, 2, 3, 4, 5, 6, 7, 8, 9]
-
-The index supports four types of query.  The first is 'any_of'.  It
-takes an iterable of values, and returns an iterable of document ids that
-contain any of the values.  The results are not weighted.
-
-    >>> list(index.apply({'any_of':('b', 'c')}))
-    [2, 4, 6, 7, 8, 9]
-    >>> list(index.apply({'any_of': ('b',)}))
-    [2, 8]
-    >>> list(index.apply({'any_of': ('d',)}))
-    [5]
-    >>> list(index.apply({'any_of':(42,)}))
-    []
-
-Another query is 'any', If the key is None, all indexed document ids with any
-values are returned.  If the key is an extent, the intersection of the extent
-and all document ids with any values is returned.
-
-    >>> list(index.apply({'any': None}))
-    [1, 2, 3, 4, 5, 6, 7, 8, 9]
-
-    >>> from zc.catalog.extentcatalog import FilterExtent
-    >>> extent = FilterExtent(lambda extent, uid, obj: True)
-    >>> for i in range(15):
-    ...     extent.add(i, i)
-    ...
-    >>> list(index.apply({'any': extent}))
-    [1, 2, 3, 4, 5, 6, 7, 8, 9]
-    >>> limited_extent = FilterExtent(lambda extent, uid, obj: True)
-    >>> for i in range(5):
-    ...     limited_extent.add(i, i)
-    ...
-    >>> list(index.apply({'any': limited_extent}))
-    [1, 2, 3, 4]
-
-The 'between' argument takes from 1 to four values.  The first is the
-minimum, and defaults to None, indicating no minimum; the second is the
-maximum, and defaults to None, indicating no maximum; the next is a boolean for
-whether the minimum value should be excluded, and defaults to False; and the
-last is a boolean for whether the maximum value should be excluded, and also
-defaults to False.  The results are not weighted.
-
-    >>> list(index.apply({'between': ('b', 'd')}))
-    [2, 4, 5, 6, 7, 8, 9]
-    >>> list(index.apply({'between': ('c', None)}))
-    [4, 5, 6, 7, 9]
-    >>> list(index.apply({'between': ('c',)}))
-    [4, 5, 6, 7, 9]
-    >>> list(index.apply({'between': ('b', 'd', True, True)}))
-    [4, 6, 7, 9]
-
-The 'none' argument takes an extent and returns the ids in the extent
-that are not indexed; it is intended to be used to return docids that have
-no (or empty) values.
-
-    >>> list(index.apply({'none': extent}))
-    [0, 10, 11, 12, 13, 14]
-
-Trying to use more than one of these at a time generates an error.
-
-    >>> index.apply({'between': (5,), 'any_of': (3,)})
-    ... # doctest: +ELLIPSIS
-    Traceback (most recent call last):
-    ...
-    ValueError:...
-
-Using none of them simply returns None.
-
-    >>> index.apply({}) # returns None
-
-Invalid query names cause ValueErrors.
-
-    >>> index.apply({'foo':()})
-    ... # doctest: +ELLIPSIS
-    Traceback (most recent call last):
-    ...
-    ValueError:...
-
-When you unindex a document, the searches and statistics should be updated.
-
-    >>> index.unindex_doc(5)
-    >>> len(index.apply({'any_of': ('d',)}))
-    0
-    >>> index.documentCount()
-    8
-    >>> index.wordCount()
-    3
-    >>> list(index.values())
-    ['a', 'b', 'c']
-    >>> list(index.ids())
-    [1, 2, 3, 4, 6, 7, 8, 9]
-
-Reindexing a document that has a changed value also is reflected in
-subsequent searches and statistic checks.
-
-    >>> list(index.apply({'any_of': ('b',)}))
-    [2, 8]
-    >>> data[8] = 'e'
-    >>> index.index_doc(8, data[8])
-    >>> index.documentCount()
-    8
-    >>> index.wordCount()
-    4
-    >>> list(index.apply({'any_of': ('e',)}))
-    [8]
-    >>> list(index.apply({'any_of': ('b',)}))
-    [2]
-    >>> data[2] = 'e'
-    >>> index.index_doc(2, data[2])
-    >>> index.documentCount()
-    8
-    >>> index.wordCount()
-    3
-    >>> list(index.apply({'any_of': ('e',)}))
-    [2, 8]
-    >>> list(index.apply({'any_of': ('b',)}))
-    []
-
-Reindexing a document for which the value is now None causes it to be removed
-from the statistics.
-
-    >>> data[3] = None
-    >>> index.index_doc(3, data[3])
-    >>> index.documentCount()
-    7
-    >>> index.wordCount()
-    3
-    >>> list(index.ids())
-    [1, 2, 4, 6, 7, 8, 9]
-
-This affects both ways of determining the ids that are and are not in the index
-(that do and do not have values).
-
-    >>> list(index.apply({'any': None}))
-    [1, 2, 4, 6, 7, 8, 9]
-    >>> list(index.apply({'any': extent}))
-    [1, 2, 4, 6, 7, 8, 9]
-    >>> list(index.apply({'none': extent}))
-    [0, 3, 5, 10, 11, 12, 13, 14]
-
-The values method can be used to examine the indexed values for a given
-document id.  For a valueindex, the "values" for a given doc_id will always
-have a length of 0 or 1.
-
-    >>> index.values(doc_id=8)
-    ('e',)
-
-And the containsValue method provides a way of determining membership in the
-values.
-
-    >>> index.containsValue('a')
-    True
-    >>> index.containsValue('q')
-    False

Copied: zc.catalog/tags/1.4.1/src/zc/catalog/valueindex.txt (from rev 97338, zc.catalog/trunk/src/zc/catalog/valueindex.txt)
===================================================================
--- zc.catalog/tags/1.4.1/src/zc/catalog/valueindex.txt	                        (rev 0)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/valueindex.txt	2009-02-27 14:44:33 UTC (rev 97355)
@@ -0,0 +1,259 @@
+===========
+Value Index
+===========
+
+The valueindex is an index similar to, but more flexible than a standard Zope
+field index.  The index allows searches for documents that contain any of a
+set of values; between a set of values; any (non-None) values; and any empty
+values.
+
+Additionally, the index supports an interface that allows examination of the
+indexed values.
+
+It is as policy-free as possible, and is intended to be the engine for indexes
+with more policy, as well as being useful itself.
+
+On creation, the index has no wordCount, no documentCount, and is, as
+expected, fairly empty.
+
+    >>> from zc.catalog.index import ValueIndex
+    >>> index = ValueIndex()
+    >>> index.documentCount()
+    0
+    >>> index.wordCount()
+    0
+    >>> index.maxValue() # doctest: +ELLIPSIS
+    Traceback (most recent call last):
+    ...
+    ValueError:...
+    >>> index.minValue() # doctest: +ELLIPSIS
+    Traceback (most recent call last):
+    ...
+    ValueError:...
+    >>> list(index.values())
+    []
+    >>> len(index.apply({'any_of': (5,)}))
+    0
+
+The index supports indexing any value.  All values within a given index must
+sort consistently across Python versions.
+
+    >>> data = {1: 'a',
+    ...         2: 'b',
+    ...         3: 'a',
+    ...         4: 'c',
+    ...         5: 'd',
+    ...         6: 'c',
+    ...         7: 'c',
+    ...         8: 'b',
+    ...         9: 'c',
+    ... }
+    >>> for k, v in data.items():
+    ...     index.index_doc(k, v)
+    ...
+
+After indexing, the statistics and values match the newly entered content.
+
+    >>> list(index.values())
+    ['a', 'b', 'c', 'd']
+    >>> index.documentCount()
+    9
+    >>> index.wordCount()
+    4
+    >>> index.maxValue()
+    'd'
+    >>> index.minValue()
+    'a'
+    >>> list(index.ids())
+    [1, 2, 3, 4, 5, 6, 7, 8, 9]
+
+The index supports four types of query.  The first is 'any_of'.  It
+takes an iterable of values, and returns an iterable of document ids that
+contain any of the values.  The results are not weighted.
+
+    >>> list(index.apply({'any_of':('b', 'c')}))
+    [2, 4, 6, 7, 8, 9]
+    >>> list(index.apply({'any_of': ('b',)}))
+    [2, 8]
+    >>> list(index.apply({'any_of': ('d',)}))
+    [5]
+    >>> list(index.apply({'any_of':(42,)}))
+    []
+
+Another query is 'any', If the key is None, all indexed document ids with any
+values are returned.  If the key is an extent, the intersection of the extent
+and all document ids with any values is returned.
+
+    >>> list(index.apply({'any': None}))
+    [1, 2, 3, 4, 5, 6, 7, 8, 9]
+
+    >>> from zc.catalog.extentcatalog import FilterExtent
+    >>> extent = FilterExtent(lambda extent, uid, obj: True)
+    >>> for i in range(15):
+    ...     extent.add(i, i)
+    ...
+    >>> list(index.apply({'any': extent}))
+    [1, 2, 3, 4, 5, 6, 7, 8, 9]
+    >>> limited_extent = FilterExtent(lambda extent, uid, obj: True)
+    >>> for i in range(5):
+    ...     limited_extent.add(i, i)
+    ...
+    >>> list(index.apply({'any': limited_extent}))
+    [1, 2, 3, 4]
+
+The 'between' argument takes from 1 to four values.  The first is the
+minimum, and defaults to None, indicating no minimum; the second is the
+maximum, and defaults to None, indicating no maximum; the next is a boolean for
+whether the minimum value should be excluded, and defaults to False; and the
+last is a boolean for whether the maximum value should be excluded, and also
+defaults to False.  The results are not weighted.
+
+    >>> list(index.apply({'between': ('b', 'd')}))
+    [2, 4, 5, 6, 7, 8, 9]
+    >>> list(index.apply({'between': ('c', None)}))
+    [4, 5, 6, 7, 9]
+    >>> list(index.apply({'between': ('c',)}))
+    [4, 5, 6, 7, 9]
+    >>> list(index.apply({'between': ('b', 'd', True, True)}))
+    [4, 6, 7, 9]
+
+The 'none' argument takes an extent and returns the ids in the extent
+that are not indexed; it is intended to be used to return docids that have
+no (or empty) values.
+
+    >>> list(index.apply({'none': extent}))
+    [0, 10, 11, 12, 13, 14]
+
+Trying to use more than one of these at a time generates an error.
+
+    >>> index.apply({'between': (5,), 'any_of': (3,)})
+    ... # doctest: +ELLIPSIS
+    Traceback (most recent call last):
+    ...
+    ValueError:...
+
+Using none of them simply returns None.
+
+    >>> index.apply({}) # returns None
+
+Invalid query names cause ValueErrors.
+
+    >>> index.apply({'foo':()})
+    ... # doctest: +ELLIPSIS
+    Traceback (most recent call last):
+    ...
+    ValueError:...
+
+When you unindex a document, the searches and statistics should be updated.
+
+    >>> index.unindex_doc(5)
+    >>> len(index.apply({'any_of': ('d',)}))
+    0
+    >>> index.documentCount()
+    8
+    >>> index.wordCount()
+    3
+    >>> list(index.values())
+    ['a', 'b', 'c']
+    >>> list(index.ids())
+    [1, 2, 3, 4, 6, 7, 8, 9]
+
+Reindexing a document that has a changed value also is reflected in
+subsequent searches and statistic checks.
+
+    >>> list(index.apply({'any_of': ('b',)}))
+    [2, 8]
+    >>> data[8] = 'e'
+    >>> index.index_doc(8, data[8])
+    >>> index.documentCount()
+    8
+    >>> index.wordCount()
+    4
+    >>> list(index.apply({'any_of': ('e',)}))
+    [8]
+    >>> list(index.apply({'any_of': ('b',)}))
+    [2]
+    >>> data[2] = 'e'
+    >>> index.index_doc(2, data[2])
+    >>> index.documentCount()
+    8
+    >>> index.wordCount()
+    3
+    >>> list(index.apply({'any_of': ('e',)}))
+    [2, 8]
+    >>> list(index.apply({'any_of': ('b',)}))
+    []
+
+Reindexing a document for which the value is now None causes it to be removed
+from the statistics.
+
+    >>> data[3] = None
+    >>> index.index_doc(3, data[3])
+    >>> index.documentCount()
+    7
+    >>> index.wordCount()
+    3
+    >>> list(index.ids())
+    [1, 2, 4, 6, 7, 8, 9]
+
+This affects both ways of determining the ids that are and are not in the index
+(that do and do not have values).
+
+    >>> list(index.apply({'any': None}))
+    [1, 2, 4, 6, 7, 8, 9]
+    >>> list(index.apply({'any': extent}))
+    [1, 2, 4, 6, 7, 8, 9]
+    >>> list(index.apply({'none': extent}))
+    [0, 3, 5, 10, 11, 12, 13, 14]
+
+The values method can be used to examine the indexed values for a given
+document id.  For a valueindex, the "values" for a given doc_id will always
+have a length of 0 or 1.
+
+    >>> index.values(doc_id=8)
+    ('e',)
+
+And the containsValue method provides a way of determining membership in the
+values.
+
+    >>> index.containsValue('a')
+    True
+    >>> index.containsValue('q')
+    False
+
+Sorting
+-------
+
+Value indexes supports sorting, just like zope.index.field.FieldIndex.
+
+    >>> index.clear()
+
+    >>> index.index_doc(1, 9)
+    >>> index.index_doc(2, 8)
+    >>> index.index_doc(3, 7)
+    >>> index.index_doc(4, 6)
+    >>> index.index_doc(5, 5)
+    >>> index.index_doc(6, 4)
+    >>> index.index_doc(7, 3)
+    >>> index.index_doc(8, 2)
+    >>> index.index_doc(9, 1)
+
+    >>> list(index.sort([4, 2, 9, 7, 3, 1, 5]))
+    [9, 7, 5, 4, 3, 2, 1]
+
+We can also specify the ``reverse`` argument to reverse results:
+
+    >>> list(index.sort([4, 2, 9, 7, 3, 1, 5], reverse=True))
+    [1, 2, 3, 4, 5, 7, 9]
+
+And as per IIndexSort, we can limit results by specifying the ``limit``
+argument:
+
+    >>> list(index.sort([4, 2, 9, 7, 3, 1, 5], limit=3)) 
+    [9, 7, 5]
+
+If we pass an id that is not indexed by this index, it won't be included
+in the result.
+
+    >>> list(index.sort([2, 10]))
+    [2]



More information about the Checkins mailing list