[Checkins] SVN: zc.catalog/tags/1.4.1/ Tag 1.4.1
Dan Korostelev
nadako at gmail.com
Fri Feb 27 09:44:34 EST 2009
Log message for revision 97355:
Tag 1.4.1
Changed:
A zc.catalog/tags/1.4.1/
D zc.catalog/tags/1.4.1/CHANGES.txt
A zc.catalog/tags/1.4.1/CHANGES.txt
D zc.catalog/tags/1.4.1/buildout.cfg
A zc.catalog/tags/1.4.1/buildout.cfg
D zc.catalog/tags/1.4.1/setup.py
A zc.catalog/tags/1.4.1/setup.py
D zc.catalog/tags/1.4.1/src/zc/catalog/catalogindex.py
A zc.catalog/tags/1.4.1/src/zc/catalog/catalogindex.py
D zc.catalog/tags/1.4.1/src/zc/catalog/configure.zcml
A zc.catalog/tags/1.4.1/src/zc/catalog/configure.zcml
D zc.catalog/tags/1.4.1/src/zc/catalog/index.py
A zc.catalog/tags/1.4.1/src/zc/catalog/index.py
D zc.catalog/tags/1.4.1/src/zc/catalog/normalizedindex.txt
A zc.catalog/tags/1.4.1/src/zc/catalog/normalizedindex.txt
D zc.catalog/tags/1.4.1/src/zc/catalog/valueindex.txt
A zc.catalog/tags/1.4.1/src/zc/catalog/valueindex.txt
-=-
Deleted: zc.catalog/tags/1.4.1/CHANGES.txt
===================================================================
--- zc.catalog/trunk/CHANGES.txt 2009-02-27 09:39:25 UTC (rev 97335)
+++ zc.catalog/tags/1.4.1/CHANGES.txt 2009-02-27 14:44:33 UTC (rev 97355)
@@ -1,131 +0,0 @@
-=======
-CHANGES
-=======
-
-The 1.2 line (and higher) supports Zope 3.4/ZODB 3.8. The 1.1 line supports
-Zope 3.3/ZODB 3.7.
-
-1.4.1 (unreleased)
-------------------
-
-*
-
-
-1.4.0 (2009-02-07)
-------------------
-
-Bugs fixed
-~~~~~~~~~~
-
-* Fixed a typo in ValueIndex addform and addMenuItem
-
-* Use ``zope.container`` instead of ``zope.app.container``.
-
-* Use ``zope.keyreference`` instead of ``zope.app.keyreference``.
-
-* Use ``zope.intid`` instead of ``zope.app.intid``.
-
-* Use ``zope.catalog`` instead of ``zope.app.catalog``.
-
-
-1.3.0 (2008-09-10)
-------------------
-
-Features added
-~~~~~~~~~~~~~~
-
-* Added hook point to allow extent catalog to be used with local UID sources.
-
-
-1.2.0 (2007-11-03)
-------------------
-
-Features added
-~~~~~~~~~~~~~~
-
-* Updated package meta-data.
-
-* zc.catalog now can use 64-bit BTrees ("L") as provided by ZODB 3.8.
-
-* Albertas Agejavas (alga at pov.lt) included the new CallableWrapper, for
- when the typical Zope 3 index-by-adapter story
- (zope.app.catalog.attribute) is unnecessary trouble, and you just want
- to use a callable. See callablewrapper.txt. This can also be used for
- other indexes based on the zope.index interfaces.
-
-* Extents now have a __len__. The current implementation defers to the
- standard BTree len implementation, and shares its performance
- characteristics: it needs to wake up all of the buckets, but if all of the
- buckets are awake it is a fairly quick operation.
-
-* A simple ISelfPoulatingExtent was added to the extentcatalog module for
- which populating is a no-op. This is directly useful for catalogs that
- are used as implementation details of a component, in which objects are
- indexed explicitly by your own calls rather than by the usual subscribers.
- It is also potentially slightly useful as a base for other self-populating
- extents.
-
-
-1.1.1 (2007-3-17)
------------------
-
-Bugs fixed
-~~~~~~~~~~
-
-'all_of' would return all results when one of the values had no results.
-Reported, with test and fix provided, by Nando Quintana.
-
-
-1.1 (2007-01-06)
-----------------
-
-Features removed
-~~~~~~~~~~~~~~~~
-
-The queueing of events in the extent catalog has been entirely removed.
-Subtransactions caused significant problems to the code introduced in 1.0.
-Other solutions also have significant problems, and the win of this kind
-of queueing is qustionable. Here is a run down of the approaches rejected
-for getting the queueing to work:
-
-* _p_invalidate (used in 1.0). Not really designed for use within a
- transaction, and reverts to last savepoint, rather than the beginning of
- the transaction. Could monkeypatch savepoints to iterate over
- precommit transaction hooks but that just smells too bad.
-
-* _p_resolveConflict. Requires application software to exist in ZEO and
- even ZRS installations, which is counter to our software deployment goals.
- Also causes useless repeated writes of empty queue to database, but that's
- not the showstopper.
-
-* vague hand-wavy ideas for separate storages or transaction managers for the
- queue. Never panned out in discussion.
-
-
-1.0 (2007-01-05)
-----------------
-
-Bugs fixed
-~~~~~~~~~~
-
-* adjusted extentcatalog tests to trigger (and discuss and test) the queueing
- behavior.
-
-* fixed problem with excessive conflict errors due to queueing code.
-
-* updated stemming to work with newest version of TextIndexNG's extensions.
-
-* omitted stemming test when TextIndexNG's extensions are unavailable, so
- tests pass without it. Since TextIndexNG's extensions are optional, this
- seems reasonable.
-
-* removed use of zapi in extentcatalog.
-
-
-0.2 (2006-11-22)
-----------------
-
-Features added
-~~~~~~~~~~~~~~
-
-* First release on Cheeseshop.
Copied: zc.catalog/tags/1.4.1/CHANGES.txt (from rev 97345, zc.catalog/trunk/CHANGES.txt)
===================================================================
--- zc.catalog/tags/1.4.1/CHANGES.txt (rev 0)
+++ zc.catalog/tags/1.4.1/CHANGES.txt 2009-02-27 14:44:33 UTC (rev 97355)
@@ -0,0 +1,133 @@
+=======
+CHANGES
+=======
+
+The 1.2 line (and higher) supports Zope 3.4/ZODB 3.8. The 1.1 line supports
+Zope 3.3/ZODB 3.7.
+
+1.4.1 (2009-02-27)
+------------------
+
+* Add FieldIndex-like sorting support for the ValueIndex.
+
+* Add sorting indexes support for the NormalizationWrapper.
+
+
+1.4.0 (2009-02-07)
+------------------
+
+Bugs fixed
+~~~~~~~~~~
+
+* Fixed a typo in ValueIndex addform and addMenuItem
+
+* Use ``zope.container`` instead of ``zope.app.container``.
+
+* Use ``zope.keyreference`` instead of ``zope.app.keyreference``.
+
+* Use ``zope.intid`` instead of ``zope.app.intid``.
+
+* Use ``zope.catalog`` instead of ``zope.app.catalog``.
+
+
+1.3.0 (2008-09-10)
+------------------
+
+Features added
+~~~~~~~~~~~~~~
+
+* Added hook point to allow extent catalog to be used with local UID sources.
+
+
+1.2.0 (2007-11-03)
+------------------
+
+Features added
+~~~~~~~~~~~~~~
+
+* Updated package meta-data.
+
+* zc.catalog now can use 64-bit BTrees ("L") as provided by ZODB 3.8.
+
+* Albertas Agejavas (alga at pov.lt) included the new CallableWrapper, for
+ when the typical Zope 3 index-by-adapter story
+ (zope.app.catalog.attribute) is unnecessary trouble, and you just want
+ to use a callable. See callablewrapper.txt. This can also be used for
+ other indexes based on the zope.index interfaces.
+
+* Extents now have a __len__. The current implementation defers to the
+ standard BTree len implementation, and shares its performance
+ characteristics: it needs to wake up all of the buckets, but if all of the
+ buckets are awake it is a fairly quick operation.
+
+* A simple ISelfPoulatingExtent was added to the extentcatalog module for
+ which populating is a no-op. This is directly useful for catalogs that
+ are used as implementation details of a component, in which objects are
+ indexed explicitly by your own calls rather than by the usual subscribers.
+ It is also potentially slightly useful as a base for other self-populating
+ extents.
+
+
+1.1.1 (2007-3-17)
+-----------------
+
+Bugs fixed
+~~~~~~~~~~
+
+'all_of' would return all results when one of the values had no results.
+Reported, with test and fix provided, by Nando Quintana.
+
+
+1.1 (2007-01-06)
+----------------
+
+Features removed
+~~~~~~~~~~~~~~~~
+
+The queueing of events in the extent catalog has been entirely removed.
+Subtransactions caused significant problems to the code introduced in 1.0.
+Other solutions also have significant problems, and the win of this kind
+of queueing is qustionable. Here is a run down of the approaches rejected
+for getting the queueing to work:
+
+* _p_invalidate (used in 1.0). Not really designed for use within a
+ transaction, and reverts to last savepoint, rather than the beginning of
+ the transaction. Could monkeypatch savepoints to iterate over
+ precommit transaction hooks but that just smells too bad.
+
+* _p_resolveConflict. Requires application software to exist in ZEO and
+ even ZRS installations, which is counter to our software deployment goals.
+ Also causes useless repeated writes of empty queue to database, but that's
+ not the showstopper.
+
+* vague hand-wavy ideas for separate storages or transaction managers for the
+ queue. Never panned out in discussion.
+
+
+1.0 (2007-01-05)
+----------------
+
+Bugs fixed
+~~~~~~~~~~
+
+* adjusted extentcatalog tests to trigger (and discuss and test) the queueing
+ behavior.
+
+* fixed problem with excessive conflict errors due to queueing code.
+
+* updated stemming to work with newest version of TextIndexNG's extensions.
+
+* omitted stemming test when TextIndexNG's extensions are unavailable, so
+ tests pass without it. Since TextIndexNG's extensions are optional, this
+ seems reasonable.
+
+* removed use of zapi in extentcatalog.
+
+
+0.2 (2006-11-22)
+----------------
+
+Features added
+~~~~~~~~~~~~~~
+
+* First release on Cheeseshop.
Deleted: zc.catalog/tags/1.4.1/buildout.cfg
===================================================================
--- zc.catalog/trunk/buildout.cfg 2009-02-27 09:39:25 UTC (rev 97335)
+++ zc.catalog/tags/1.4.1/buildout.cfg 2009-02-27 14:44:33 UTC (rev 97355)
@@ -1,9 +0,0 @@
-[buildout]
-develop = .
-parts = test
-
-[test]
-recipe = zc.recipe.testrunner
-eggs = zc.catalog [test]
-defaults = "--exit-with-status".split()
-
Copied: zc.catalog/tags/1.4.1/buildout.cfg (from rev 97353, zc.catalog/trunk/buildout.cfg)
===================================================================
--- zc.catalog/tags/1.4.1/buildout.cfg (rev 0)
+++ zc.catalog/tags/1.4.1/buildout.cfg 2009-02-27 14:44:33 UTC (rev 97355)
@@ -0,0 +1,9 @@
+[buildout]
+develop = .
+parts = test
+
+[test]
+recipe = zc.recipe.testrunner
+eggs = zc.catalog [test]
+defaults = "--exit-with-status".split()
+
Deleted: zc.catalog/tags/1.4.1/setup.py
===================================================================
--- zc.catalog/trunk/setup.py 2009-02-27 09:39:25 UTC (rev 97335)
+++ zc.catalog/tags/1.4.1/setup.py 2009-02-27 14:44:33 UTC (rev 97355)
@@ -1,94 +0,0 @@
-##############################################################################
-#
-# Copyright (c) 2007 Zope Corporation and Contributors.
-# All Rights Reserved.
-#
-# This software is subject to the provisions of the Zope Public License,
-# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
-# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
-# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
-# FOR A PARTICULAR PURPOSE.
-#
-##############################################################################
-"""Setup for zc.catalog package
-
-$Id: setup.py 81038 2007-10-24 14:34:17Z srichter $
-"""
-import os
-from setuptools import setup, find_packages
-
-def read(*rnames):
- return open(os.path.join(os.path.dirname('.'), *rnames)).read()
-
-setup(name='zc.catalog',
- version = '1.4.1dev',
- author='Zope Corporation and Contributors',
- author_email='zope-dev at zope.org',
- description="Extensions to the Zope 3 Catalog",
- long_description=(
- read('README.txt')
- + '\n\n.. contents::\n\n' +
- read('src', 'zc', 'catalog', 'valueindex.txt')
- + '\n\n' +
- read('src', 'zc', 'catalog', 'setindex.txt')
- + '\n\n' +
- read('src', 'zc', 'catalog', 'normalizedindex.txt')
- + '\n\n' +
- read('src', 'zc', 'catalog', 'extentcatalog.txt')
- + '\n\n' +
- read('src', 'zc', 'catalog', 'stemmer.txt')
- + '\n\n' +
- read('src', 'zc', 'catalog', 'legacy.txt')
- + '\n\n' +
- read('src', 'zc', 'catalog', 'globber.txt')
- + '\n\n' +
- read('src', 'zc', 'catalog', 'callablewrapper.txt')
- + '\n\n' +
- read('src', 'zc', 'catalog', 'browser', 'README.txt')
- + '\n\n' +
- read('CHANGES.txt')
- ),
- keywords = "zope3 i18n date time duration catalog index",
- classifiers = [
- 'Development Status :: 5 - Production/Stable',
- 'Environment :: Web Environment',
- 'Intended Audience :: Developers',
- 'License :: OSI Approved :: Zope Public License',
- 'Programming Language :: Python',
- 'Natural Language :: English',
- 'Operating System :: OS Independent',
- 'Topic :: Internet :: WWW/HTTP',
- 'Framework :: Zope3'],
- url='http://pypi.python.org/pypi/zc.catalog',
- license='ZPL 2.1',
- packages=find_packages('src'),
- package_dir = {'': 'src'},
- namespace_packages=['zc'],
- extras_require=dict(
- test=['zope.app.authentication',
- 'zope.app.catalog',
- 'zope.app.securitypolicy',
- 'zope.app.server',
- 'zope.app.testing',
- 'zope.app.zcmlfiles',
- 'zope.keyreference',
- 'zope.intid',
- 'zope.testbrowser',
- 'zope.testing',
- ]),
- install_requires=['ZODB3',
- 'pytz',
- 'setuptools',
- 'zope.catalog',
- 'zope.container',
- 'zope.component',
- 'zope.i18nmessageid',
- 'zope.interface',
- 'zope.publisher',
- 'zope.schema',
- 'zope.security',
- ],
- include_package_data = True,
- zip_safe = False,
- )
Copied: zc.catalog/tags/1.4.1/setup.py (from rev 97353, zc.catalog/trunk/setup.py)
===================================================================
--- zc.catalog/tags/1.4.1/setup.py (rev 0)
+++ zc.catalog/tags/1.4.1/setup.py 2009-02-27 14:44:33 UTC (rev 97355)
@@ -0,0 +1,95 @@
+##############################################################################
+#
+# Copyright (c) 2007 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Setup for zc.catalog package
+
+$Id: setup.py 81038 2007-10-24 14:34:17Z srichter $
+"""
+import os
+from setuptools import setup, find_packages
+
+def read(*rnames):
+ return open(os.path.join(os.path.dirname('.'), *rnames)).read()
+
+setup(name='zc.catalog',
+ version = '1.4.1',
+ author='Zope Corporation and Contributors',
+ author_email='zope-dev at zope.org',
+ description="Extensions to the Zope 3 Catalog",
+ long_description=(
+ read('README.txt')
+ + '\n\n.. contents::\n\n' +
+ read('src', 'zc', 'catalog', 'valueindex.txt')
+ + '\n\n' +
+ read('src', 'zc', 'catalog', 'setindex.txt')
+ + '\n\n' +
+ read('src', 'zc', 'catalog', 'normalizedindex.txt')
+ + '\n\n' +
+ read('src', 'zc', 'catalog', 'extentcatalog.txt')
+ + '\n\n' +
+ read('src', 'zc', 'catalog', 'stemmer.txt')
+ + '\n\n' +
+ read('src', 'zc', 'catalog', 'legacy.txt')
+ + '\n\n' +
+ read('src', 'zc', 'catalog', 'globber.txt')
+ + '\n\n' +
+ read('src', 'zc', 'catalog', 'callablewrapper.txt')
+ + '\n\n' +
+ read('src', 'zc', 'catalog', 'browser', 'README.txt')
+ + '\n\n' +
+ read('CHANGES.txt')
+ ),
+ keywords = "zope3 i18n date time duration catalog index",
+ classifiers = [
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'License :: OSI Approved :: Zope Public License',
+ 'Programming Language :: Python',
+ 'Natural Language :: English',
+ 'Operating System :: OS Independent',
+ 'Topic :: Internet :: WWW/HTTP',
+ 'Framework :: Zope3'],
+ url='http://pypi.python.org/pypi/zc.catalog',
+ license='ZPL 2.1',
+ packages=find_packages('src'),
+ package_dir = {'': 'src'},
+ namespace_packages=['zc'],
+ extras_require=dict(
+ test=['zope.app.authentication',
+ 'zope.app.catalog',
+ 'zope.app.securitypolicy',
+ 'zope.app.server',
+ 'zope.app.testing',
+ 'zope.app.zcmlfiles',
+ 'zope.keyreference',
+ 'zope.intid',
+ 'zope.testbrowser',
+ 'zope.testing',
+ ]),
+ install_requires=['ZODB3',
+ 'pytz',
+ 'setuptools',
+ 'zope.catalog',
+ 'zope.container',
+ 'zope.component',
+ 'zope.i18nmessageid',
+ 'zope.index>=3.5.1',
+ 'zope.interface',
+ 'zope.publisher',
+ 'zope.schema',
+ 'zope.security',
+ ],
+ include_package_data = True,
+ zip_safe = False,
+ )
Deleted: zc.catalog/tags/1.4.1/src/zc/catalog/catalogindex.py
===================================================================
--- zc.catalog/trunk/src/zc/catalog/catalogindex.py 2009-02-27 09:39:25 UTC (rev 97335)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/catalogindex.py 2009-02-27 14:44:33 UTC (rev 97355)
@@ -1,73 +0,0 @@
-#############################################################################
-#
-# Copyright (c) 2006 Zope Corporation and Contributors.
-# All Rights Reserved.
-#
-# This software is subject to the provisions of the Zope Public License,
-# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
-# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
-# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
-# FOR A PARTICULAR PURPOSE.
-#
-##############################################################################
-"""Indexes appropriate for zope.catalog
-
-$Id: catalogindex.py 2918 2005-07-19 22:12:38Z jim $
-"""
-import zope.interface
-
-import zope.catalog.attribute
-import zope.container.contained
-
-import zc.catalog.index
-import zc.catalog.interfaces
-
-class ValueIndex(zope.catalog.attribute.AttributeIndex,
- zc.catalog.index.ValueIndex,
- zope.container.contained.Contained):
-
- zope.interface.implements(zc.catalog.interfaces.ICatalogValueIndex)
-
-class SetIndex(zope.catalog.attribute.AttributeIndex,
- zc.catalog.index.SetIndex,
- zope.container.contained.Contained):
-
- zope.interface.implements(zc.catalog.interfaces.ICatalogSetIndex)
-
-class NormalizationWrapper(
- zope.catalog.attribute.AttributeIndex,
- zc.catalog.index.NormalizationWrapper,
- zope.container.contained.Contained):
-
- pass
-
-
-class CallableWrapper(zc.catalog.index.CallableWrapper,
- zope.container.contained.Contained):
- zope.interface.implements(zc.catalog.interfaces.ICallableWrapper)
-
-
- at zope.interface.implementer(
- zope.interface.implementedBy(NormalizationWrapper),
- zc.catalog.interfaces.IValueIndex)
-def DateTimeValueIndex(
- field_name=None, interface=None, field_callable=False,
- resolution=2): # hour; good for per-day searches
- ix = NormalizationWrapper(
- field_name, interface, field_callable, zc.catalog.index.ValueIndex(),
- zc.catalog.index.DateTimeNormalizer(resolution), False)
- zope.interface.directlyProvides(ix, zc.catalog.interfaces.IValueIndex)
- return ix
-
- at zope.interface.implementer(
- zope.interface.implementedBy(NormalizationWrapper),
- zc.catalog.interfaces.ISetIndex)
-def DateTimeSetIndex(
- field_name=None, interface=None, field_callable=False,
- resolution=2): # hour; good for per-day searches
- ix = NormalizationWrapper(
- field_name, interface, field_callable, zc.catalog.index.SetIndex(),
- zc.catalog.index.DateTimeNormalizer(resolution), True)
- zope.interface.directlyProvides(ix, zc.catalog.interfaces.ISetIndex)
- return ix
Copied: zc.catalog/tags/1.4.1/src/zc/catalog/catalogindex.py (from rev 97345, zc.catalog/trunk/src/zc/catalog/catalogindex.py)
===================================================================
--- zc.catalog/tags/1.4.1/src/zc/catalog/catalogindex.py (rev 0)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/catalogindex.py 2009-02-27 14:44:33 UTC (rev 97355)
@@ -0,0 +1,75 @@
+#############################################################################
+#
+# Copyright (c) 2006 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Indexes appropriate for zope.catalog
+
+$Id: catalogindex.py 2918 2005-07-19 22:12:38Z jim $
+"""
+import zope.interface
+
+import zope.catalog.attribute
+import zope.container.contained
+import zope.index.interfaces
+
+import zc.catalog.index
+import zc.catalog.interfaces
+
+class ValueIndex(zope.catalog.attribute.AttributeIndex,
+ zc.catalog.index.ValueIndex,
+ zope.container.contained.Contained):
+
+ zope.interface.implements(zc.catalog.interfaces.ICatalogValueIndex)
+
+class SetIndex(zope.catalog.attribute.AttributeIndex,
+ zc.catalog.index.SetIndex,
+ zope.container.contained.Contained):
+
+ zope.interface.implements(zc.catalog.interfaces.ICatalogSetIndex)
+
+class NormalizationWrapper(
+ zope.catalog.attribute.AttributeIndex,
+ zc.catalog.index.NormalizationWrapper,
+ zope.container.contained.Contained):
+
+ pass
+
+
+class CallableWrapper(zc.catalog.index.CallableWrapper,
+ zope.container.contained.Contained):
+ zope.interface.implements(zc.catalog.interfaces.ICallableWrapper)
+
+
+ at zope.interface.implementer(
+ zope.interface.implementedBy(NormalizationWrapper),
+ zc.catalog.interfaces.IValueIndex,
+ zope.index.interfaces.IIndexSort)
+def DateTimeValueIndex(
+ field_name=None, interface=None, field_callable=False,
+ resolution=2): # hour; good for per-day searches
+ ix = NormalizationWrapper(
+ field_name, interface, field_callable, zc.catalog.index.ValueIndex(),
+ zc.catalog.index.DateTimeNormalizer(resolution), False)
+ zope.interface.alsoProvides(ix, zc.catalog.interfaces.IValueIndex)
+ return ix
+
+ at zope.interface.implementer(
+ zope.interface.implementedBy(NormalizationWrapper),
+ zc.catalog.interfaces.ISetIndex)
+def DateTimeSetIndex(
+ field_name=None, interface=None, field_callable=False,
+ resolution=2): # hour; good for per-day searches
+ ix = NormalizationWrapper(
+ field_name, interface, field_callable, zc.catalog.index.SetIndex(),
+ zc.catalog.index.DateTimeNormalizer(resolution), True)
+ zope.interface.alsoProvides(ix, zc.catalog.interfaces.ISetIndex)
+ return ix
Deleted: zc.catalog/tags/1.4.1/src/zc/catalog/configure.zcml
===================================================================
--- zc.catalog/trunk/src/zc/catalog/configure.zcml 2009-02-27 09:39:25 UTC (rev 97335)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/configure.zcml 2009-02-27 14:44:33 UTC (rev 97355)
@@ -1,73 +0,0 @@
-<configure
- xmlns="http://namespaces.zope.org/zope"
- xmlns:browser="http://namespaces.zope.org/browser"
- i18n_domain="zope"
- >
-
- <class class=".extentcatalog.Catalog">
- <factory
- id="zc.catalog.extentcatalog"
- />
- <require
- interface="zope.catalog.interfaces.ICatalogQuery"
- permission="zope.Public"
- />
- <require
- interface="zope.catalog.interfaces.ICatalogEdit"
- permission="zope.ManageServices"
- />
- <require
- interface="zope.container.interfaces.IContainer"
- permission="zope.ManageServices"
- />
- </class>
-
- <class class=".extentcatalog.FilterExtent">
- <require
- attributes="addable __iter__ __or__ __ror__ union __and__ __rand__
- intersection __sub__ difference __rsub__ rdifference
- __nonzero__ __contains__"
- permission="zope.View"/>
- <require
- attributes="add remove discard clear"
- permission="zope.ManageServices"
- />
- </class>
-
- <class class=".catalogindex.ValueIndex">
- <require
- permission="zope.ManageServices"
- interface="zope.catalog.interfaces.IAttributeIndex
- zope.index.interfaces.IStatistics"
- set_schema="zope.catalog.interfaces.IAttributeIndex"
- />
- </class>
-
- <class class=".catalogindex.SetIndex">
- <require
- permission="zope.ManageServices"
- interface="zope.catalog.interfaces.IAttributeIndex
- zope.index.interfaces.IStatistics"
- set_schema="zope.catalog.interfaces.IAttributeIndex"
- />
- </class>
-
- <class class=".catalogindex.NormalizationWrapper">
- <require
- permission="zope.ManageServices"
- interface="zope.catalog.interfaces.IAttributeIndex
- zope.index.interfaces.IStatistics"
- set_schema="zope.catalog.interfaces.IAttributeIndex"
- />
- </class>
-
- <class class="BTrees.Length.Length">
- <require
- permission="zope.ManageServices"
- attributes="__call__"
- />
- </class>
-
- <include package=".browser" />
-
-</configure>
Copied: zc.catalog/tags/1.4.1/src/zc/catalog/configure.zcml (from rev 97354, zc.catalog/trunk/src/zc/catalog/configure.zcml)
===================================================================
--- zc.catalog/tags/1.4.1/src/zc/catalog/configure.zcml (rev 0)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/configure.zcml 2009-02-27 14:44:33 UTC (rev 97355)
@@ -0,0 +1,77 @@
+<configure
+ xmlns="http://namespaces.zope.org/zope"
+ xmlns:browser="http://namespaces.zope.org/browser"
+ xmlns:zcml="http://namespaces.zope.org/zcml"
+ i18n_domain="zope"
+ >
+
+ <class class=".extentcatalog.Catalog">
+ <factory
+ id="zc.catalog.extentcatalog"
+ />
+ <require
+ interface="zope.catalog.interfaces.ICatalogQuery"
+ permission="zope.Public"
+ />
+ <require
+ interface="zope.catalog.interfaces.ICatalogEdit"
+ permission="zope.ManageServices"
+ />
+ <require
+ interface="zope.container.interfaces.IContainer"
+ permission="zope.ManageServices"
+ />
+ </class>
+
+ <class class=".extentcatalog.FilterExtent">
+ <require
+ attributes="addable __iter__ __or__ __ror__ union __and__ __rand__
+ intersection __sub__ difference __rsub__ rdifference
+ __nonzero__ __contains__"
+ permission="zope.View"/>
+ <require
+ attributes="add remove discard clear"
+ permission="zope.ManageServices"
+ />
+ </class>
+
+ <class class=".catalogindex.ValueIndex">
+ <require
+ permission="zope.ManageServices"
+ interface="zope.catalog.interfaces.IAttributeIndex
+ zope.index.interfaces.IStatistics"
+ set_schema="zope.catalog.interfaces.IAttributeIndex"
+ />
+ </class>
+
+ <class class=".catalogindex.SetIndex">
+ <require
+ permission="zope.ManageServices"
+ interface="zope.catalog.interfaces.IAttributeIndex
+ zope.index.interfaces.IStatistics"
+ set_schema="zope.catalog.interfaces.IAttributeIndex"
+ />
+ </class>
+
+ <class class=".catalogindex.NormalizationWrapper">
+ <require
+ permission="zope.ManageServices"
+ interface="zope.catalog.interfaces.IAttributeIndex
+ zope.index.interfaces.IStatistics"
+ set_schema="zope.catalog.interfaces.IAttributeIndex"
+ />
+ </class>
+
+ <class class="BTrees.Length.Length">
+ <require
+ permission="zope.ManageServices"
+ attributes="__call__"
+ />
+ </class>
+
+ <include
+ zcml:condition="installed zope.app.form"
+ package=".browser"
+ />
+
+</configure>
Deleted: zc.catalog/tags/1.4.1/src/zc/catalog/index.py
===================================================================
--- zc.catalog/trunk/src/zc/catalog/index.py 2009-02-27 09:39:25 UTC (rev 97335)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/index.py 2009-02-27 14:44:33 UTC (rev 97355)
@@ -1,538 +0,0 @@
-#############################################################################
-#
-# Copyright (c) 2006 Zope Corporation and Contributors.
-# All Rights Reserved.
-#
-# This software is subject to the provisions of the Zope Public License,
-# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
-# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
-# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
-# FOR A PARTICULAR PURPOSE.
-#
-##############################################################################
-"""indexes, as might be found in zope.index
-
-$Id: index.py 2918 2005-07-19 22:12:38Z jim $
-"""
-import sys
-import datetime
-import pytz.reference
-import BTrees
-import persistent
-from BTrees import Length
-from BTrees.Interfaces import IMerge
-
-from zope import component, interface
-import zope.component.interfaces
-import zope.interface.common.idatetime
-import zope.index.interfaces
-import zope.security.management
-from zope.publisher.interfaces import IRequest
-import zc.catalog.interfaces
-from zc.catalog.i18n import _
-
-
-class FamilyProperty(object):
-
- __name__ = "family"
-
- def __get__(self, instance, type=None):
- if instance is None:
- return self
- d = instance.__dict__
- if "family" in d:
- return d["family"]
- if "btreemodule" in d:
- iftype = d["btreemodule"].split(".")[-1][:2]
- if iftype == "IF":
- d["family"] = BTrees.family32
- elif iftype == "LF":
- d["family"] = BTrees.family64
- else:
- raise ValueError("can't determine btree family based on"
- " btreemodule of %r" % (iftype,))
- else:
- d["family"] = BTrees.family32
- self._clear_old_cruft(instance)
- return d["family"]
-
- def __set__(self, instance, value):
- instance.__dict__["family"] = value
- self._clear_old_cruft(instance)
-
- def _clear_old_cruft(self, instance):
- d = instance.__dict__
- if "btreemodule" in d:
- del d["btreemodule"]
- if "IOBTree" in d:
- del d["IOBTree"]
- if "BTreeAPI" in d:
- del d["BTreeAPI"]
-
-
-class AbstractIndex(persistent.Persistent):
-
- interface.implements(zope.index.interfaces.IInjection,
- zope.index.interfaces.IIndexSearch,
- zope.index.interfaces.IStatistics,
- zc.catalog.interfaces.IIndexValues,
- )
-
- family = FamilyProperty()
-
- def __init__(self, family=None):
- if family is not None:
- self.family = family
- self.clear()
-
- # These three are deprecated (they were never interface), but can
- # all be computed from the family attribute:
-
- @property
- def btreemodule(self):
- return self.family.IF.__name__
-
- @property
- def BTreeAPI(self):
- return self.family.IF
-
- @property
- def IOBTree(self):
- return self.family.IO.BTree
-
- def clear(self):
- self.values_to_documents = self.family.OO.BTree()
- self.documents_to_values = self.family.IO.BTree()
- self.documentCount = Length.Length(0)
- self.wordCount = Length.Length(0)
-
- def minValue(self, min=None):
- if min is None:
- return self.values_to_documents.minKey()
- else:
- return self.values_to_documents.minKey(min)
-
- def maxValue(self, max=None):
- if max is None:
- return self.values_to_documents.maxKey()
- else:
- return self.values_to_documents.maxKey(max)
-
- def values(self, min=None, max=None, excludemin=False, excludemax=False,
- doc_id=None):
- if doc_id is None:
- return iter(self.values_to_documents.keys(
- min, max, excludemin, excludemax))
- else:
- values = self.documents_to_values.get(doc_id)
- if values is None:
- return ()
- else:
- return iter(values.keys(min, max, excludemin, excludemax))
-
- def containsValue(self, value):
- return bool(self.values_to_documents.has_key(value))
-
- def ids(self):
- return self.documents_to_values.keys()
-
-def parseQuery(query):
- if isinstance(query, dict):
- if len(query) > 1:
- raise ValueError(
- 'may only pass one of key, value pair')
- elif not query:
- return None, None
- query_type, query = query.items()[0]
- query_type = query_type.lower()
- else:
- raise ValueError('may only pass a dict to apply')
- return query_type, query
-
-class ValueIndex(AbstractIndex):
-
- interface.implements(zc.catalog.interfaces.IValueIndex)
-
- def _add_value(self, doc_id, added):
- values_to_documents = self.values_to_documents
- docs = values_to_documents.get(added)
- if docs is None:
- values_to_documents[added] = self.family.IF.TreeSet((doc_id,))
- self.wordCount.change(1)
- else:
- docs.insert(doc_id)
-
- def index_doc(self, doc_id, value):
- if value is None:
- self.unindex_doc(doc_id)
- else:
- values_to_documents = self.values_to_documents
- documents_to_values = self.documents_to_values
- old = documents_to_values.get(doc_id)
- documents_to_values[doc_id] = value
- if old is None:
- self.documentCount.change(1)
- elif old != value:
- docs = values_to_documents.get(old)
- docs.remove(doc_id)
- if not docs:
- del values_to_documents[old]
- self.wordCount.change(-1)
- self._add_value(doc_id, value)
-
- def unindex_doc(self, doc_id):
- documents_to_values = self.documents_to_values
- value = documents_to_values.get(doc_id)
- if value is not None:
- values_to_documents = self.values_to_documents
- self.documentCount.change(-1)
- del documents_to_values[doc_id]
- docs = values_to_documents.get(value)
- docs.remove(doc_id)
- if not docs:
- del values_to_documents[value]
- self.wordCount.change(-1)
-
- def apply(self, query): # any_of, any, between, none,
- values_to_documents = self.values_to_documents
- query_type, query = parseQuery(query)
- if query_type is None:
- res = None
- elif query_type == 'any_of':
- res = self.family.IF.multiunion(
- [s for s in (values_to_documents.get(v) for v in query)
- if s is not None])
- elif query_type == 'any':
- if query is None:
- res = self.family.IF.Set(self.ids())
- else:
- assert zc.catalog.interfaces.IExtent.providedBy(query)
- res = query & self.family.IF.Set(self.ids())
- elif query_type == 'between':
- res = self.family.IF.multiunion(
- [s for s in (values_to_documents.get(v) for v in
- values_to_documents.keys(*query))
- if s is not None])
- elif query_type == 'none':
- assert zc.catalog.interfaces.IExtent.providedBy(query)
- res = query - self.family.IF.Set(self.ids())
- else:
- raise ValueError(
- "unknown query type", query_type)
- return res
-
- def values(self, min=None, max=None, excludemin=False, excludemax=False,
- doc_id=None):
- if doc_id is None:
- return iter(self.values_to_documents.keys(
- min, max, excludemin, excludemax))
- else:
- value = self.documents_to_values.get(doc_id)
- if (value is None or
- min is not None and (
- value < min or excludemin and value == min) or
- max is not None and (
- value > max or excludemax and value == max)):
- return ()
- else:
- return (value,)
-
-class SetIndex(AbstractIndex):
-
- interface.implements(zc.catalog.interfaces.ISetIndex)
-
- def _add_values(self, doc_id, added):
- values_to_documents = self.values_to_documents
- for v in added:
- docs = values_to_documents.get(v)
- if docs is None:
- values_to_documents[v] = self.family.IF.TreeSet((doc_id,))
- self.wordCount.change(1)
- else:
- docs.insert(doc_id)
-
- def index_doc(self, doc_id, value):
- new = self.family.OO.TreeSet(v for v in value if v is not None)
- if not new:
- self.unindex_doc(doc_id)
- else:
- values_to_documents = self.values_to_documents
- documents_to_values = self.documents_to_values
- old = documents_to_values.get(doc_id)
- if old is None:
- documents_to_values[doc_id] = new
- self.documentCount.change(1)
- self._add_values(doc_id, new)
- else:
- removed = self.family.OO.difference(old, new)
- added = self.family.OO.difference(new, old)
- for v in removed:
- old.remove(v)
- docs = values_to_documents.get(v)
- docs.remove(doc_id)
- if not docs:
- del values_to_documents[v]
- self.wordCount.change(-1)
- old.update(added)
- self._add_values(doc_id, added)
-
- def unindex_doc(self, doc_id):
- documents_to_values = self.documents_to_values
- values = documents_to_values.get(doc_id)
- if values is not None:
- values_to_documents = self.values_to_documents
- self.documentCount.change(-1)
- del documents_to_values[doc_id]
- for v in values:
- docs = values_to_documents.get(v)
- docs.remove(doc_id)
- if not docs:
- del values_to_documents[v]
- self.wordCount.change(-1)
-
- def apply(self, query): # any_of, any, between, none, all_of
- values_to_documents = self.values_to_documents
- query_type, query = parseQuery(query)
- if query_type is None:
- res = None
- elif query_type == 'any_of':
- res = self.family.IF.Bucket()
- for v in query:
- _, res = self.family.IF.weightedUnion(
- res, values_to_documents.get(v))
- elif query_type == 'any':
- if query is None:
- res = self.family.IF.Set(self.ids())
- else:
- assert zc.catalog.interfaces.IExtent.providedBy(query)
- res = query & self.family.IF.Set(self.ids())
- elif query_type == 'all_of':
- res = None
- values = iter(query)
- empty = self.family.IF.TreeSet()
- try:
- res = values_to_documents.get(values.next(), empty)
- except StopIteration:
- res = empty
- while res:
- try:
- v = values.next()
- except StopIteration:
- break
- res = self.family.IF.intersection(
- res, values_to_documents.get(v, empty))
- elif query_type == 'between':
- res = self.family.IF.Bucket()
- for v in values_to_documents.keys(*query):
- _, res = self.family.IF.weightedUnion(
- res, values_to_documents.get(v))
- elif query_type == 'none':
- assert zc.catalog.interfaces.IExtent.providedBy(query)
- res = query - self.family.IF.Set(self.ids())
- else:
- raise ValueError(
- "unknown query type", query_type)
- return res
-
-class NormalizationWrapper(persistent.Persistent):
-
- interface.implements(zc.catalog.interfaces.INormalizationWrapper)
-
- index = normalizer = None
- collection_index = False
-
- def documentCount(self):
- return self.index.documentCount()
-
- def wordCount(self):
- return self.index.wordCount()
-
- def clear(self):
- """see zope.index.interfaces.IInjection.clear"""
- return self.index.clear()
-
- def __init__(self, index, normalizer, collection_index=False):
- self.index = index
- self.normalizer = normalizer
- self.collection_index = collection_index
-
- def index_doc(self, doc_id, value):
- if self.collection_index:
- self.index.index_doc(
- doc_id, (self.normalizer.value(v) for v in value))
- else:
- self.index.index_doc(doc_id, self.normalizer.value(value))
-
- def unindex_doc(self, doc_id):
- self.index.unindex_doc(doc_id)
-
- def apply(self, query):
- query_type, query = parseQuery(query)
- if query_type == 'any_of':
- res = set()
- for v in query:
- res.update(self.normalizer.any(v, self.index))
- elif query_type == 'all_of':
- res = [self.normalizer.all(v, self.index) for v in query]
- elif query_type == 'between':
- query = tuple(query) # collect iterators
- len_query = len(query)
- max_exclude = len_query >= 4 and bool(query[3])
- min_exclude = len_query >= 3 and bool(query[2])
- max = len_query >= 2 and query[1] and self.normalizer.maximum(
- query[1], self.index, max_exclude) or None
- min = len_query >= 1 and query[0] and self.normalizer.minimum(
- query[0], self.index, min_exclude) or None
- res = (min, max, min_exclude, max_exclude)
- else:
- res = query
- return self.index.apply({query_type: res})
-
- def minValue(self, min=None):
- if min is not None:
- min = self.normalizer.minimum(min, self.index)
- return self.index.minValue(min)
-
- def maxValue(self, max=None):
- if max is not None:
- max = self.normalizer.maximum(max, self.index)
- return self.index.maxValue(max)
-
- def values(self, min=None, max=None, excludemin=False, excludemax=False,
- doc_id=None):
- if min is not None:
- min = self.normalizer.minimum(min, self.index)
- if max is not None:
- max = self.normalizer.maximum(max, self.index)
- return self.index.values(min, max, excludemin, excludemax,
- doc_id=doc_id)
-
- def containsValue(self, value):
- return self.index.containsValue(value)
-
- def ids(self):
- return self.index.ids()
-
-
-class CallableWrapper(persistent.Persistent):
-
- interface.implements(zc.catalog.interfaces.ICallableWrapper)
-
- converter = None
- index = None
-
- def __init__(self, index, converter):
- self.index = index
- self.converter = converter
-
- def index_doc(self, docid, value):
- "See zope.index.interfaces.IInjection"
- self.index.index_doc(docid, self.converter(value))
-
- def __getattr__(self, name):
- return getattr(self.index, name)
-
-
-def set_resolution(value, resolution):
- resolution += 2
- if resolution < 6:
- args = []
- args.extend(value.timetuple()[:resolution+1])
- args.extend([0]*(6-resolution))
- args.append(value.tzinfo)
- value = datetime.datetime(*args)
- return value
-
-def get_request():
- i = zope.security.management.queryInteraction()
- if i is not None:
- for p in i.participations:
- if IRequest.providedBy(p):
- return p
- return None
-
-def get_tz(default=pytz.reference.Local):
- request = get_request()
- if request is None:
- return default
- return zope.interface.common.idatetime.ITZInfo(request, default)
-
-def add_tz(value):
- if type(value) is datetime.datetime:
- if value.tzinfo is None:
- value = value.replace(tzinfo=get_tz())
- return value
- else:
- raise ValueError(value)
-
-def day_end(value):
- return (
- datetime.datetime.combine(
- value, datetime.time(tzinfo=get_tz())) +
- datetime.timedelta(days=1) - # separate for daylight savings
- datetime.timedelta(microseconds=1))
-
-def day_begin(value):
- return datetime.datetime.combine(
- value, datetime.time(tzinfo=get_tz()))
-
-
-class DateTimeNormalizer(persistent.Persistent):
-
- interface.implements(zc.catalog.interfaces.IDateTimeNormalizer)
- def __init__(self, resolution=2):
- self.resolution = resolution
- # 0, 1, 2, 3, 4
- # day, hour, minute, second, microsecond
-
- def value(self, value):
- if not isinstance(value, datetime.datetime) or value.tzinfo is None:
- raise ValueError(
- _('This index only indexes timezone-aware datetimes.'))
- return set_resolution(value, self.resolution)
-
- def any(self, value, index):
- if type(value) is datetime.date:
- start = datetime.datetime.combine(
- value, datetime.time(tzinfo=get_tz()))
- stop = start + datetime.timedelta(days=1)
- return index.values(start, stop, False, True)
- return (add_tz(value),)
-
- def all(self, value, index):
- return add_tz(value)
-
- def minimum(self, value, index, exclude=False):
- if type(value) is datetime.date:
- if exclude:
- return day_end(value)
- else:
- return day_begin(value)
- return add_tz(value)
-
- def maximum(self, value, index, exclude=False):
- if type(value) is datetime.date:
- if exclude:
- return day_begin(value)
- else:
- return day_end(value)
- return add_tz(value)
-
- at interface.implementer(
- zope.interface.implementedBy(NormalizationWrapper),
- zc.catalog.interfaces.IValueIndex)
-def DateTimeValueIndex(resolution=2): # 2 == minute; note that hour is good
- # for timezone-aware per-day searches
- ix = NormalizationWrapper(ValueIndex(), DateTimeNormalizer(resolution))
- interface.directlyProvides(ix, zc.catalog.interfaces.IValueIndex)
- return ix
-
- at interface.implementer(
- zope.interface.implementedBy(NormalizationWrapper),
- zc.catalog.interfaces.ISetIndex)
-def DateTimeSetIndex(resolution=2): # 2 == minute; note that hour is good
- # for timezone-aware per-day searches
- ix = NormalizationWrapper(SetIndex(), DateTimeNormalizer(resolution), True)
- interface.directlyProvides(ix, zc.catalog.interfaces.ISetIndex)
- return ix
Copied: zc.catalog/tags/1.4.1/src/zc/catalog/index.py (from rev 97345, zc.catalog/trunk/src/zc/catalog/index.py)
===================================================================
--- zc.catalog/tags/1.4.1/src/zc/catalog/index.py (rev 0)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/index.py 2009-02-27 14:44:33 UTC (rev 97355)
@@ -0,0 +1,551 @@
+#############################################################################
+#
+# Copyright (c) 2006 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""indexes, as might be found in zope.index
+
+$Id: index.py 2918 2005-07-19 22:12:38Z jim $
+"""
+import sys
+import datetime
+import pytz.reference
+import BTrees
+import persistent
+from BTrees import Length
+from BTrees.Interfaces import IMerge
+
+from zope import component, interface
+import zope.component.interfaces
+import zope.interface.common.idatetime
+import zope.index.interfaces
+from zope.index.field.sorting import SortingIndexMixin
+import zope.security.management
+from zope.publisher.interfaces import IRequest
+import zc.catalog.interfaces
+from zc.catalog.i18n import _
+
+
+class FamilyProperty(object):
+
+ __name__ = "family"
+
+ def __get__(self, instance, type=None):
+ if instance is None:
+ return self
+ d = instance.__dict__
+ if "family" in d:
+ return d["family"]
+ if "btreemodule" in d:
+ iftype = d["btreemodule"].split(".")[-1][:2]
+ if iftype == "IF":
+ d["family"] = BTrees.family32
+ elif iftype == "LF":
+ d["family"] = BTrees.family64
+ else:
+ raise ValueError("can't determine btree family based on"
+ " btreemodule of %r" % (iftype,))
+ else:
+ d["family"] = BTrees.family32
+ self._clear_old_cruft(instance)
+ return d["family"]
+
+ def __set__(self, instance, value):
+ instance.__dict__["family"] = value
+ self._clear_old_cruft(instance)
+
+ def _clear_old_cruft(self, instance):
+ d = instance.__dict__
+ if "btreemodule" in d:
+ del d["btreemodule"]
+ if "IOBTree" in d:
+ del d["IOBTree"]
+ if "BTreeAPI" in d:
+ del d["BTreeAPI"]
+
+
+class AbstractIndex(persistent.Persistent):
+
+ interface.implements(zope.index.interfaces.IInjection,
+ zope.index.interfaces.IIndexSearch,
+ zope.index.interfaces.IStatistics,
+ zc.catalog.interfaces.IIndexValues,
+ )
+
+ family = FamilyProperty()
+
+ def __init__(self, family=None):
+ if family is not None:
+ self.family = family
+ self.clear()
+
+ # These three are deprecated (they were never interface), but can
+ # all be computed from the family attribute:
+
+ @property
+ def btreemodule(self):
+ return self.family.IF.__name__
+
+ @property
+ def BTreeAPI(self):
+ return self.family.IF
+
+ @property
+ def IOBTree(self):
+ return self.family.IO.BTree
+
+ def clear(self):
+ self.values_to_documents = self.family.OO.BTree()
+ self.documents_to_values = self.family.IO.BTree()
+ self.documentCount = Length.Length(0)
+ self.wordCount = Length.Length(0)
+
+ def minValue(self, min=None):
+ if min is None:
+ return self.values_to_documents.minKey()
+ else:
+ return self.values_to_documents.minKey(min)
+
+ def maxValue(self, max=None):
+ if max is None:
+ return self.values_to_documents.maxKey()
+ else:
+ return self.values_to_documents.maxKey(max)
+
+ def values(self, min=None, max=None, excludemin=False, excludemax=False,
+ doc_id=None):
+ if doc_id is None:
+ return iter(self.values_to_documents.keys(
+ min, max, excludemin, excludemax))
+ else:
+ values = self.documents_to_values.get(doc_id)
+ if values is None:
+ return ()
+ else:
+ return iter(values.keys(min, max, excludemin, excludemax))
+
+ def containsValue(self, value):
+ return bool(self.values_to_documents.has_key(value))
+
+ def ids(self):
+ return self.documents_to_values.keys()
+
+def parseQuery(query):
+ if isinstance(query, dict):
+ if len(query) > 1:
+ raise ValueError(
+ 'may only pass one of key, value pair')
+ elif not query:
+ return None, None
+ query_type, query = query.items()[0]
+ query_type = query_type.lower()
+ else:
+ raise ValueError('may only pass a dict to apply')
+ return query_type, query
+
+class ValueIndex(SortingIndexMixin, AbstractIndex):
+
+ interface.implements(zc.catalog.interfaces.IValueIndex)
+
+ # attributes used by sorting mixin
+ _sorting_num_docs_attr = 'documentCount' # Length object
+ _sorting_fwd_index_attr = 'values_to_documents' # forward BTree index
+ _sorting_rev_index_attr = 'documents_to_values' # reverse BTree index
+
+ def _add_value(self, doc_id, added):
+ values_to_documents = self.values_to_documents
+ docs = values_to_documents.get(added)
+ if docs is None:
+ values_to_documents[added] = self.family.IF.TreeSet((doc_id,))
+ self.wordCount.change(1)
+ else:
+ docs.insert(doc_id)
+
+ def index_doc(self, doc_id, value):
+ if value is None:
+ self.unindex_doc(doc_id)
+ else:
+ values_to_documents = self.values_to_documents
+ documents_to_values = self.documents_to_values
+ old = documents_to_values.get(doc_id)
+ documents_to_values[doc_id] = value
+ if old is None:
+ self.documentCount.change(1)
+ elif old != value:
+ docs = values_to_documents.get(old)
+ docs.remove(doc_id)
+ if not docs:
+ del values_to_documents[old]
+ self.wordCount.change(-1)
+ self._add_value(doc_id, value)
+
+ def unindex_doc(self, doc_id):
+ documents_to_values = self.documents_to_values
+ value = documents_to_values.get(doc_id)
+ if value is not None:
+ values_to_documents = self.values_to_documents
+ self.documentCount.change(-1)
+ del documents_to_values[doc_id]
+ docs = values_to_documents.get(value)
+ docs.remove(doc_id)
+ if not docs:
+ del values_to_documents[value]
+ self.wordCount.change(-1)
+
+ def apply(self, query): # any_of, any, between, none,
+ values_to_documents = self.values_to_documents
+ query_type, query = parseQuery(query)
+ if query_type is None:
+ res = None
+ elif query_type == 'any_of':
+ res = self.family.IF.multiunion(
+ [s for s in (values_to_documents.get(v) for v in query)
+ if s is not None])
+ elif query_type == 'any':
+ if query is None:
+ res = self.family.IF.Set(self.ids())
+ else:
+ assert zc.catalog.interfaces.IExtent.providedBy(query)
+ res = query & self.family.IF.Set(self.ids())
+ elif query_type == 'between':
+ res = self.family.IF.multiunion(
+ [s for s in (values_to_documents.get(v) for v in
+ values_to_documents.keys(*query))
+ if s is not None])
+ elif query_type == 'none':
+ assert zc.catalog.interfaces.IExtent.providedBy(query)
+ res = query - self.family.IF.Set(self.ids())
+ else:
+ raise ValueError(
+ "unknown query type", query_type)
+ return res
+
+ def values(self, min=None, max=None, excludemin=False, excludemax=False,
+ doc_id=None):
+ if doc_id is None:
+ return iter(self.values_to_documents.keys(
+ min, max, excludemin, excludemax))
+ else:
+ value = self.documents_to_values.get(doc_id)
+ if (value is None or
+ min is not None and (
+ value < min or excludemin and value == min) or
+ max is not None and (
+ value > max or excludemax and value == max)):
+ return ()
+ else:
+ return (value,)
+
+class SetIndex(AbstractIndex):
+
+ interface.implements(zc.catalog.interfaces.ISetIndex)
+
+ def _add_values(self, doc_id, added):
+ values_to_documents = self.values_to_documents
+ for v in added:
+ docs = values_to_documents.get(v)
+ if docs is None:
+ values_to_documents[v] = self.family.IF.TreeSet((doc_id,))
+ self.wordCount.change(1)
+ else:
+ docs.insert(doc_id)
+
+ def index_doc(self, doc_id, value):
+ new = self.family.OO.TreeSet(v for v in value if v is not None)
+ if not new:
+ self.unindex_doc(doc_id)
+ else:
+ values_to_documents = self.values_to_documents
+ documents_to_values = self.documents_to_values
+ old = documents_to_values.get(doc_id)
+ if old is None:
+ documents_to_values[doc_id] = new
+ self.documentCount.change(1)
+ self._add_values(doc_id, new)
+ else:
+ removed = self.family.OO.difference(old, new)
+ added = self.family.OO.difference(new, old)
+ for v in removed:
+ old.remove(v)
+ docs = values_to_documents.get(v)
+ docs.remove(doc_id)
+ if not docs:
+ del values_to_documents[v]
+ self.wordCount.change(-1)
+ old.update(added)
+ self._add_values(doc_id, added)
+
+ def unindex_doc(self, doc_id):
+ documents_to_values = self.documents_to_values
+ values = documents_to_values.get(doc_id)
+ if values is not None:
+ values_to_documents = self.values_to_documents
+ self.documentCount.change(-1)
+ del documents_to_values[doc_id]
+ for v in values:
+ docs = values_to_documents.get(v)
+ docs.remove(doc_id)
+ if not docs:
+ del values_to_documents[v]
+ self.wordCount.change(-1)
+
+ def apply(self, query): # any_of, any, between, none, all_of
+ values_to_documents = self.values_to_documents
+ query_type, query = parseQuery(query)
+ if query_type is None:
+ res = None
+ elif query_type == 'any_of':
+ res = self.family.IF.Bucket()
+ for v in query:
+ _, res = self.family.IF.weightedUnion(
+ res, values_to_documents.get(v))
+ elif query_type == 'any':
+ if query is None:
+ res = self.family.IF.Set(self.ids())
+ else:
+ assert zc.catalog.interfaces.IExtent.providedBy(query)
+ res = query & self.family.IF.Set(self.ids())
+ elif query_type == 'all_of':
+ res = None
+ values = iter(query)
+ empty = self.family.IF.TreeSet()
+ try:
+ res = values_to_documents.get(values.next(), empty)
+ except StopIteration:
+ res = empty
+ while res:
+ try:
+ v = values.next()
+ except StopIteration:
+ break
+ res = self.family.IF.intersection(
+ res, values_to_documents.get(v, empty))
+ elif query_type == 'between':
+ res = self.family.IF.Bucket()
+ for v in values_to_documents.keys(*query):
+ _, res = self.family.IF.weightedUnion(
+ res, values_to_documents.get(v))
+ elif query_type == 'none':
+ assert zc.catalog.interfaces.IExtent.providedBy(query)
+ res = query - self.family.IF.Set(self.ids())
+ else:
+ raise ValueError(
+ "unknown query type", query_type)
+ return res
+
+class NormalizationWrapper(persistent.Persistent):
+
+ interface.implements(zc.catalog.interfaces.INormalizationWrapper)
+
+ index = normalizer = None
+ collection_index = False
+
+ def documentCount(self):
+ return self.index.documentCount()
+
+ def wordCount(self):
+ return self.index.wordCount()
+
+ def clear(self):
+ """see zope.index.interfaces.IInjection.clear"""
+ return self.index.clear()
+
+ def __init__(self, index, normalizer, collection_index=False):
+ self.index = index
+ if zope.index.interfaces.IIndexSort.providedBy(index):
+ zope.interface.alsoProvides(self, zope.index.interfaces.IIndexSort)
+ self.normalizer = normalizer
+ self.collection_index = collection_index
+
+ def index_doc(self, doc_id, value):
+ if self.collection_index:
+ self.index.index_doc(
+ doc_id, (self.normalizer.value(v) for v in value))
+ else:
+ self.index.index_doc(doc_id, self.normalizer.value(value))
+
+ def unindex_doc(self, doc_id):
+ self.index.unindex_doc(doc_id)
+
+ def apply(self, query):
+ query_type, query = parseQuery(query)
+ if query_type == 'any_of':
+ res = set()
+ for v in query:
+ res.update(self.normalizer.any(v, self.index))
+ elif query_type == 'all_of':
+ res = [self.normalizer.all(v, self.index) for v in query]
+ elif query_type == 'between':
+ query = tuple(query) # collect iterators
+ len_query = len(query)
+ max_exclude = len_query >= 4 and bool(query[3])
+ min_exclude = len_query >= 3 and bool(query[2])
+ max = len_query >= 2 and query[1] and self.normalizer.maximum(
+ query[1], self.index, max_exclude) or None
+ min = len_query >= 1 and query[0] and self.normalizer.minimum(
+ query[0], self.index, min_exclude) or None
+ res = (min, max, min_exclude, max_exclude)
+ else:
+ res = query
+ return self.index.apply({query_type: res})
+
+ def minValue(self, min=None):
+ if min is not None:
+ min = self.normalizer.minimum(min, self.index)
+ return self.index.minValue(min)
+
+ def maxValue(self, max=None):
+ if max is not None:
+ max = self.normalizer.maximum(max, self.index)
+ return self.index.maxValue(max)
+
+ def values(self, min=None, max=None, excludemin=False, excludemax=False,
+ doc_id=None):
+ if min is not None:
+ min = self.normalizer.minimum(min, self.index)
+ if max is not None:
+ max = self.normalizer.maximum(max, self.index)
+ return self.index.values(min, max, excludemin, excludemax,
+ doc_id=doc_id)
+
+ def containsValue(self, value):
+ return self.index.containsValue(value)
+
+ def ids(self):
+ return self.index.ids()
+
+ @property
+ def sort(self):
+ # delegate upstream or raise AttributeError
+ return self.index.sort
+
+class CallableWrapper(persistent.Persistent):
+
+ interface.implements(zc.catalog.interfaces.ICallableWrapper)
+
+ converter = None
+ index = None
+
+ def __init__(self, index, converter):
+ self.index = index
+ self.converter = converter
+
+ def index_doc(self, docid, value):
+ "See zope.index.interfaces.IInjection"
+ self.index.index_doc(docid, self.converter(value))
+
+ def __getattr__(self, name):
+ return getattr(self.index, name)
+
+
+def set_resolution(value, resolution):
+ resolution += 2
+ if resolution < 6:
+ args = []
+ args.extend(value.timetuple()[:resolution+1])
+ args.extend([0]*(6-resolution))
+ args.append(value.tzinfo)
+ value = datetime.datetime(*args)
+ return value
+
+def get_request():
+ i = zope.security.management.queryInteraction()
+ if i is not None:
+ for p in i.participations:
+ if IRequest.providedBy(p):
+ return p
+ return None
+
+def get_tz(default=pytz.reference.Local):
+ request = get_request()
+ if request is None:
+ return default
+ return zope.interface.common.idatetime.ITZInfo(request, default)
+
+def add_tz(value):
+ if type(value) is datetime.datetime:
+ if value.tzinfo is None:
+ value = value.replace(tzinfo=get_tz())
+ return value
+ else:
+ raise ValueError(value)
+
+def day_end(value):
+ return (
+ datetime.datetime.combine(
+ value, datetime.time(tzinfo=get_tz())) +
+ datetime.timedelta(days=1) - # separate for daylight savings
+ datetime.timedelta(microseconds=1))
+
+def day_begin(value):
+ return datetime.datetime.combine(
+ value, datetime.time(tzinfo=get_tz()))
+
+
+class DateTimeNormalizer(persistent.Persistent):
+
+ interface.implements(zc.catalog.interfaces.IDateTimeNormalizer)
+ def __init__(self, resolution=2):
+ self.resolution = resolution
+ # 0, 1, 2, 3, 4
+ # day, hour, minute, second, microsecond
+
+ def value(self, value):
+ if not isinstance(value, datetime.datetime) or value.tzinfo is None:
+ raise ValueError(
+ _('This index only indexes timezone-aware datetimes.'))
+ return set_resolution(value, self.resolution)
+
+ def any(self, value, index):
+ if type(value) is datetime.date:
+ start = datetime.datetime.combine(
+ value, datetime.time(tzinfo=get_tz()))
+ stop = start + datetime.timedelta(days=1)
+ return index.values(start, stop, False, True)
+ return (add_tz(value),)
+
+ def all(self, value, index):
+ return add_tz(value)
+
+ def minimum(self, value, index, exclude=False):
+ if type(value) is datetime.date:
+ if exclude:
+ return day_end(value)
+ else:
+ return day_begin(value)
+ return add_tz(value)
+
+ def maximum(self, value, index, exclude=False):
+ if type(value) is datetime.date:
+ if exclude:
+ return day_begin(value)
+ else:
+ return day_end(value)
+ return add_tz(value)
+
+ at interface.implementer(
+ zope.interface.implementedBy(NormalizationWrapper),
+ zope.index.interfaces.IIndexSort,
+ zc.catalog.interfaces.IValueIndex)
+def DateTimeValueIndex(resolution=2): # 2 == minute; note that hour is good
+ # for timezone-aware per-day searches
+ ix = NormalizationWrapper(ValueIndex(), DateTimeNormalizer(resolution))
+ interface.alsoProvides(ix, zc.catalog.interfaces.IValueIndex)
+ return ix
+
+ at interface.implementer(
+ zope.interface.implementedBy(NormalizationWrapper),
+ zc.catalog.interfaces.ISetIndex)
+def DateTimeSetIndex(resolution=2): # 2 == minute; note that hour is good
+ # for timezone-aware per-day searches
+ ix = NormalizationWrapper(SetIndex(), DateTimeNormalizer(resolution), True)
+ interface.alsoProvides(ix, zc.catalog.interfaces.ISetIndex)
+ return ix
Deleted: zc.catalog/tags/1.4.1/src/zc/catalog/normalizedindex.txt
===================================================================
--- zc.catalog/trunk/src/zc/catalog/normalizedindex.txt 2009-02-27 09:39:25 UTC (rev 97335)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/normalizedindex.txt 2009-02-27 14:44:33 UTC (rev 97355)
@@ -1,324 +0,0 @@
-================
-Normalized Index
-================
-
-The index module provides a normalizing wrapper, a DateTime normalizer, and
-a set index and a value index normalized with the DateTime normalizer.
-
-The normalizing wrapper implements a full complement of index interfaces--
-zope.index.interfaces.IInjection, zope.index.interfaces.IIndexSearch,
-zope.index.interfaces.IStatistics, and zc.catalog.interfaces.IIndexValues--
-and delegates all of the behavior to the wrapped index, normalizing values
-using the normalizer before the index sees them.
-
-The normalizing wrapper currently only supports queries offered by
-zc.catalog.interfaces.ISetIndex and zc.catalog.interfaces.IValueIndex.
-
-The normalizer interface requires the following methods, as defined in the
-interface:
-
- def value(value):
- """normalize or check constraints for an input value; raise an error
- or return the value to be indexed."""
-
- def any(value, index):
- """normalize a query value for a "any_of" search; return a sequence of
- values."""
-
- def all(value, index):
- """Normalize a query value for an "all_of" search; return the value
- for query"""
-
- def minimum(value, index):
- """normalize a query value for minimum of a range; return the value for
- query"""
-
- def maximum(value, index):
- """normalize a query value for maximum of a range; return the value for
- query"""
-
-The DateTime normalizer performs the following normalizations and validations.
-Whenever a timezone is needed, it tries to get a request from the current
-interaction and adapt it to zope.interface.common.idatetime.ITZInfo; failing
-that (no request or no adapter) it uses the system local timezone.
-
-- input values must be datetimes with a timezone. They are normalized to the
- resolution specified when the normalizer is created: a resolution of 0
- normalizes values to days; a resolution of 1 to hours; 2 to minutes; 3 to
- seconds; and 4 to microseconds.
-
-- 'any' values may be timezone-aware datetimes, timezone-naive datetimes,
- or dates. dates are converted to any value from the start to the end of the
- given date in the found timezone, as described above. timezone-naive
- datetimes get the found timezone.
-
-- 'all' values may be timezone-aware datetimes or timezone-naive datetimes.
- timezone-naive datetimes get the found timezone.
-
-- 'minimum' values may be timezone-aware datetimes, timezone-naive datetimes,
- or dates. dates are converted to the start of the given date in the found
- timezone, as described above. timezone-naive datetimes get the found
- timezone.
-
-- 'maximum' values may be timezone-aware datetimes, timezone-naive datetimes,
- or dates. dates are converted to the end of the given date in the found
- timezone, as described above. timezone-naive datetimes get the found
- timezone.
-
-Let's look at the DateTime normalizer first, and then an integration of it
-with the normalizing wrapper and the value and set indexes.
-
-The indexed values are parsed with 'value'.
-
- >>> from zc.catalog.index import DateTimeNormalizer
- >>> n = DateTimeNormalizer() # defaults to minutes
- >>> import datetime
- >>> import pytz
- >>> naive_datetime = datetime.datetime(2005, 7, 15, 11, 21, 32, 104)
- >>> date = naive_datetime.date()
- >>> aware_datetime = naive_datetime.replace(
- ... tzinfo=pytz.timezone('US/Eastern'))
- >>> n.value(naive_datetime)
- Traceback (most recent call last):
- ...
- ValueError: This index only indexes timezone-aware datetimes.
- >>> n.value(date)
- Traceback (most recent call last):
- ...
- ValueError: This index only indexes timezone-aware datetimes.
- >>> n.value(aware_datetime) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 15, 11, 21, tzinfo=<DstTzInfo 'US/Eastern'...>)
-
-If we specify a different resolution, the results are different.
-
- >>> another = DateTimeNormalizer(1) # hours
- >>> another.value(aware_datetime) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 15, 11, 0, tzinfo=<DstTzInfo 'US/Eastern'...>)
-
-Note that changing the resolution of an indexed value may create surprising
-results, because queries do not change their resolution. Therefore, if you
-index something with a datetime with a finer resolution that the normalizer's,
-then searching for that datetime will not find the doc_id.
-
-Values in an 'any_of' query are parsed with 'any'. 'any' should return a
-sequence of values. It requires an index, which we will mock up here.
-
- >>> class DummyIndex(object):
- ... def values(self, start, stop, exclude_start, exclude_stop):
- ... assert not exclude_start and exclude_stop
- ... six_hours = datetime.timedelta(hours=6)
- ... res = []
- ... dt = start
- ... while dt < stop:
- ... res.append(dt)
- ... dt += six_hours
- ... return res
- ...
- >>> index = DummyIndex()
- >>> tuple(n.any(naive_datetime, index)) # doctest: +ELLIPSIS
- (datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>),)
- >>> tuple(n.any(aware_datetime, index)) # doctest: +ELLIPSIS
- (datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>),)
- >>> tuple(n.any(date, index)) # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
- (datetime.datetime(2005, 7, 15, 0, 0, tzinfo=<...Local...>),
- datetime.datetime(2005, 7, 15, 6, 0, tzinfo=<...Local...>),
- datetime.datetime(2005, 7, 15, 12, 0, tzinfo=<...Local...>),
- datetime.datetime(2005, 7, 15, 18, 0, tzinfo=<...Local...>))
-
-Values in an 'all_of' query are parsed with 'all'.
-
- >>> n.all(naive_datetime, index) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
- >>> n.all(aware_datetime, index) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
- >>> n.all(date, index) # doctest: +ELLIPSIS
- Traceback (most recent call last):
- ...
- ValueError: ...
-
-Minimum values in a 'between' query as well as those in other methods are
-parsed with 'minimum'. They also take an optional exclude boolean, which
-indicates whether the minimum is to be excluded. For datetimes, it only
-makes a difference if you pass in a date.
-
- >>> n.minimum(naive_datetime, index) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
- >>> n.minimum(naive_datetime, index, exclude=True) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
-
- >>> n.minimum(aware_datetime, index) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
- >>> n.minimum(aware_datetime, index, True) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
-
- >>> n.minimum(date, index) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 15, 0, 0, tzinfo=<...Local...>)
- >>> n.minimum(date, index, True) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 15, 23, 59, 59, 999999, tzinfo=<...Local...>)
-
-Maximum values in a 'between' query as well as those in other methods are
-parsed with 'maximum'. They also take an optional exclude boolean, which
-indicates whether the maximum is to be excluded. For datetimes, it only
-makes a difference if you pass in a date.
-
- >>> n.maximum(naive_datetime, index) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
- >>> n.maximum(naive_datetime, index, exclude=True) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
-
- >>> n.maximum(aware_datetime, index) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
- >>> n.maximum(aware_datetime, index, True) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
-
- >>> n.maximum(date, index) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 15, 23, 59, 59, 999999, tzinfo=<...Local...>)
- >>> n.maximum(date, index, True) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 15, 0, 0, tzinfo=<...Local...>)
-
-Now let's examine these normalizers in the context of a real index.
-
- >>> from zc.catalog.index import DateTimeValueIndex, DateTimeSetIndex
- >>> setindex = DateTimeSetIndex() # minutes resolution
- >>> data = [] # generate some data
- >>> def date_gen(
- ... start=aware_datetime,
- ... count=12,
- ... period=datetime.timedelta(hours=10)):
- ... dt = start
- ... ix = 0
- ... while ix < count:
- ... yield dt
- ... dt += period
- ... ix += 1
- ...
- >>> gen = date_gen()
- >>> count = 0
- >>> while True:
- ... try:
- ... next = [gen.next() for i in range(6)]
- ... except StopIteration:
- ... break
- ... data.append((count, next[0:1]))
- ... count += 1
- ... data.append((count, next[1:3]))
- ... count += 1
- ... data.append((count, next[3:6]))
- ... count += 1
- ...
- >>> print data # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
- [(0,
- [datetime.datetime(2005, 7, 15, 11, 21, 32, 104, ...<...Eastern...>)]),
- (1,
- [datetime.datetime(2005, 7, 15, 21, 21, 32, 104, ...<...Eastern...>),
- datetime.datetime(2005, 7, 16, 7, 21, 32, 104, ...<...Eastern...>)]),
- (2,
- [datetime.datetime(2005, 7, 16, 17, 21, 32, 104, ...<...Eastern...>),
- datetime.datetime(2005, 7, 17, 3, 21, 32, 104, ...<...Eastern...>),
- datetime.datetime(2005, 7, 17, 13, 21, 32, 104, ...<...Eastern...>)]),
- (3,
- [datetime.datetime(2005, 7, 17, 23, 21, 32, 104, ...<...Eastern...>)]),
- (4,
- [datetime.datetime(2005, 7, 18, 9, 21, 32, 104, ...<...Eastern...>),
- datetime.datetime(2005, 7, 18, 19, 21, 32, 104, ...<...Eastern...>)]),
- (5,
- [datetime.datetime(2005, 7, 19, 5, 21, 32, 104, ...<...Eastern...>),
- datetime.datetime(2005, 7, 19, 15, 21, 32, 104, ...<...Eastern...>),
- datetime.datetime(2005, 7, 20, 1, 21, 32, 104, ...<...Eastern...>)])]
- >>> data_dict = dict(data)
- >>> for doc_id, value in data:
- ... setindex.index_doc(doc_id, value)
- ...
- >>> list(setindex.ids())
- [0, 1, 2, 3, 4, 5]
- >>> set(setindex.values()) == set(
- ... setindex.normalizer.value(v) for v in date_gen())
- True
-
-For the searches, we will actually use a request and interaction, with an
-adapter that returns the Eastern timezone. This makes the examples less
-dependent on the machine that they use.
-
- >>> import zope.security.management
- >>> import zope.publisher.browser
- >>> import zope.interface.common.idatetime
- >>> import zope.publisher.interfaces
- >>> request = zope.publisher.browser.TestRequest()
- >>> zope.security.management.newInteraction(request)
- >>> from zope import interface, component
- >>> @interface.implementer(zope.interface.common.idatetime.ITZInfo)
- ... @component.adapter(zope.publisher.interfaces.IRequest)
- ... def tzinfo(req):
- ... return pytz.timezone('US/Eastern')
- ...
- >>> component.provideAdapter(tzinfo)
- >>> n.all(naive_datetime, index).tzinfo is pytz.timezone('US/Eastern')
- True
-
- >>> set(setindex.apply({'any_of': (datetime.date(2005, 7, 17),
- ... datetime.date(2005, 7, 20),
- ... datetime.date(2005, 12, 31))})) == set(
- ... (2, 3, 5))
- True
-
-Note that this search is using the normalized values.
-
- >>> set(setindex.apply({'all_of': (
- ... datetime.datetime(
- ... 2005, 7, 16, 7, 21, tzinfo=pytz.timezone('US/Eastern')),
- ... datetime.datetime(
- ... 2005, 7, 15, 21, 21, tzinfo=pytz.timezone('US/Eastern')),)})
- ... ) == set((1,))
- True
- >>> list(setindex.apply({'any': None}))
- [0, 1, 2, 3, 4, 5]
- >>> set(setindex.apply({'between': (
- ... datetime.datetime(2005, 4, 1, 12), datetime.datetime(2006, 5, 1))})
- ... ) == set((0, 1, 2, 3, 4, 5))
- True
- >>> set(setindex.apply({'between': (
- ... datetime.datetime(2005, 4, 1, 12), datetime.datetime(2006, 5, 1),
- ... True, True)})
- ... ) == set((0, 1, 2, 3, 4, 5))
- True
-
-'between' searches should deal with dates well.
-
- >>> set(setindex.apply({'between': (
- ... datetime.date(2005, 7, 16), datetime.date(2005, 7, 17))})
- ... ) == set((1, 2, 3))
- True
- >>> len(setindex.apply({'between': (
- ... datetime.date(2005, 7, 16), datetime.date(2005, 7, 17))})
- ... ) == len(setindex.apply({'between': (
- ... datetime.date(2005, 7, 15), datetime.date(2005, 7, 18),
- ... True, True)})
- ... )
- True
-
-Removing docs works as usual.
-
- >>> setindex.unindex_doc(1)
- >>> list(setindex.ids())
- [0, 2, 3, 4, 5]
-
-Value, Minvalue and Maxvalue can take timezone-less datetimes and dates.
-
- >>> setindex.minValue() # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 15, 11, 21, ...<...Eastern...>)
- >>> setindex.minValue(datetime.date(2005, 7, 17)) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 17, 3, 21, ...<...Eastern...>)
-
- >>> setindex.maxValue() # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 20, 1, 21, ...<...Eastern...>)
- >>> setindex.maxValue(datetime.date(2005, 7, 17)) # doctest: +ELLIPSIS
- datetime.datetime(2005, 7, 17, 23, 21, ...<...Eastern...>)
-
- >>> list(setindex.values(
- ... datetime.date(2005, 7, 17), datetime.date(2005, 7, 17)))
- ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
- [datetime.datetime(2005, 7, 17, 3, 21, ...<...Eastern...>),
- datetime.datetime(2005, 7, 17, 13, 21, ...<...Eastern...>),
- datetime.datetime(2005, 7, 17, 23, 21, ...<...Eastern...>)]
-
- >>> zope.security.management.endInteraction() # TODO put in tests tearDown
Copied: zc.catalog/tags/1.4.1/src/zc/catalog/normalizedindex.txt (from rev 97345, zc.catalog/trunk/src/zc/catalog/normalizedindex.txt)
===================================================================
--- zc.catalog/tags/1.4.1/src/zc/catalog/normalizedindex.txt (rev 0)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/normalizedindex.txt 2009-02-27 14:44:33 UTC (rev 97355)
@@ -0,0 +1,357 @@
+================
+Normalized Index
+================
+
+The index module provides a normalizing wrapper, a DateTime normalizer, and
+a set index and a value index normalized with the DateTime normalizer.
+
+The normalizing wrapper implements a full complement of index interfaces--
+zope.index.interfaces.IInjection, zope.index.interfaces.IIndexSearch,
+zope.index.interfaces.IStatistics, and zc.catalog.interfaces.IIndexValues--
+and delegates all of the behavior to the wrapped index, normalizing values
+using the normalizer before the index sees them.
+
+The normalizing wrapper currently only supports queries offered by
+zc.catalog.interfaces.ISetIndex and zc.catalog.interfaces.IValueIndex.
+
+The normalizer interface requires the following methods, as defined in the
+interface:
+
+ def value(value):
+ """normalize or check constraints for an input value; raise an error
+ or return the value to be indexed."""
+
+ def any(value, index):
+ """normalize a query value for a "any_of" search; return a sequence of
+ values."""
+
+ def all(value, index):
+ """Normalize a query value for an "all_of" search; return the value
+ for query"""
+
+ def minimum(value, index):
+ """normalize a query value for minimum of a range; return the value for
+ query"""
+
+ def maximum(value, index):
+ """normalize a query value for maximum of a range; return the value for
+ query"""
+
+The DateTime normalizer performs the following normalizations and validations.
+Whenever a timezone is needed, it tries to get a request from the current
+interaction and adapt it to zope.interface.common.idatetime.ITZInfo; failing
+that (no request or no adapter) it uses the system local timezone.
+
+- input values must be datetimes with a timezone. They are normalized to the
+ resolution specified when the normalizer is created: a resolution of 0
+ normalizes values to days; a resolution of 1 to hours; 2 to minutes; 3 to
+ seconds; and 4 to microseconds.
+
+- 'any' values may be timezone-aware datetimes, timezone-naive datetimes,
+ or dates. dates are converted to any value from the start to the end of the
+ given date in the found timezone, as described above. timezone-naive
+ datetimes get the found timezone.
+
+- 'all' values may be timezone-aware datetimes or timezone-naive datetimes.
+ timezone-naive datetimes get the found timezone.
+
+- 'minimum' values may be timezone-aware datetimes, timezone-naive datetimes,
+ or dates. dates are converted to the start of the given date in the found
+ timezone, as described above. timezone-naive datetimes get the found
+ timezone.
+
+- 'maximum' values may be timezone-aware datetimes, timezone-naive datetimes,
+ or dates. dates are converted to the end of the given date in the found
+ timezone, as described above. timezone-naive datetimes get the found
+ timezone.
+
+Let's look at the DateTime normalizer first, and then an integration of it
+with the normalizing wrapper and the value and set indexes.
+
+The indexed values are parsed with 'value'.
+
+ >>> from zc.catalog.index import DateTimeNormalizer
+ >>> n = DateTimeNormalizer() # defaults to minutes
+ >>> import datetime
+ >>> import pytz
+ >>> naive_datetime = datetime.datetime(2005, 7, 15, 11, 21, 32, 104)
+ >>> date = naive_datetime.date()
+ >>> aware_datetime = naive_datetime.replace(
+ ... tzinfo=pytz.timezone('US/Eastern'))
+ >>> n.value(naive_datetime)
+ Traceback (most recent call last):
+ ...
+ ValueError: This index only indexes timezone-aware datetimes.
+ >>> n.value(date)
+ Traceback (most recent call last):
+ ...
+ ValueError: This index only indexes timezone-aware datetimes.
+ >>> n.value(aware_datetime) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 15, 11, 21, tzinfo=<DstTzInfo 'US/Eastern'...>)
+
+If we specify a different resolution, the results are different.
+
+ >>> another = DateTimeNormalizer(1) # hours
+ >>> another.value(aware_datetime) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 15, 11, 0, tzinfo=<DstTzInfo 'US/Eastern'...>)
+
+Note that changing the resolution of an indexed value may create surprising
+results, because queries do not change their resolution. Therefore, if you
+index something with a datetime with a finer resolution that the normalizer's,
+then searching for that datetime will not find the doc_id.
+
+Values in an 'any_of' query are parsed with 'any'. 'any' should return a
+sequence of values. It requires an index, which we will mock up here.
+
+ >>> class DummyIndex(object):
+ ... def values(self, start, stop, exclude_start, exclude_stop):
+ ... assert not exclude_start and exclude_stop
+ ... six_hours = datetime.timedelta(hours=6)
+ ... res = []
+ ... dt = start
+ ... while dt < stop:
+ ... res.append(dt)
+ ... dt += six_hours
+ ... return res
+ ...
+ >>> index = DummyIndex()
+ >>> tuple(n.any(naive_datetime, index)) # doctest: +ELLIPSIS
+ (datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>),)
+ >>> tuple(n.any(aware_datetime, index)) # doctest: +ELLIPSIS
+ (datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>),)
+ >>> tuple(n.any(date, index)) # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
+ (datetime.datetime(2005, 7, 15, 0, 0, tzinfo=<...Local...>),
+ datetime.datetime(2005, 7, 15, 6, 0, tzinfo=<...Local...>),
+ datetime.datetime(2005, 7, 15, 12, 0, tzinfo=<...Local...>),
+ datetime.datetime(2005, 7, 15, 18, 0, tzinfo=<...Local...>))
+
+Values in an 'all_of' query are parsed with 'all'.
+
+ >>> n.all(naive_datetime, index) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
+ >>> n.all(aware_datetime, index) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
+ >>> n.all(date, index) # doctest: +ELLIPSIS
+ Traceback (most recent call last):
+ ...
+ ValueError: ...
+
+Minimum values in a 'between' query as well as those in other methods are
+parsed with 'minimum'. They also take an optional exclude boolean, which
+indicates whether the minimum is to be excluded. For datetimes, it only
+makes a difference if you pass in a date.
+
+ >>> n.minimum(naive_datetime, index) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
+ >>> n.minimum(naive_datetime, index, exclude=True) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
+
+ >>> n.minimum(aware_datetime, index) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
+ >>> n.minimum(aware_datetime, index, True) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
+
+ >>> n.minimum(date, index) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 15, 0, 0, tzinfo=<...Local...>)
+ >>> n.minimum(date, index, True) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 15, 23, 59, 59, 999999, tzinfo=<...Local...>)
+
+Maximum values in a 'between' query as well as those in other methods are
+parsed with 'maximum'. They also take an optional exclude boolean, which
+indicates whether the maximum is to be excluded. For datetimes, it only
+makes a difference if you pass in a date.
+
+ >>> n.maximum(naive_datetime, index) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
+ >>> n.maximum(naive_datetime, index, exclude=True) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Local...>)
+
+ >>> n.maximum(aware_datetime, index) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
+ >>> n.maximum(aware_datetime, index, True) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 15, 11, 21, 32, 104, tzinfo=<...Eastern...>)
+
+ >>> n.maximum(date, index) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 15, 23, 59, 59, 999999, tzinfo=<...Local...>)
+ >>> n.maximum(date, index, True) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 15, 0, 0, tzinfo=<...Local...>)
+
+Now let's examine these normalizers in the context of a real index.
+
+ >>> from zc.catalog.index import DateTimeValueIndex, DateTimeSetIndex
+ >>> setindex = DateTimeSetIndex() # minutes resolution
+ >>> data = [] # generate some data
+ >>> def date_gen(
+ ... start=aware_datetime,
+ ... count=12,
+ ... period=datetime.timedelta(hours=10)):
+ ... dt = start
+ ... ix = 0
+ ... while ix < count:
+ ... yield dt
+ ... dt += period
+ ... ix += 1
+ ...
+ >>> gen = date_gen()
+ >>> count = 0
+ >>> while True:
+ ... try:
+ ... next = [gen.next() for i in range(6)]
+ ... except StopIteration:
+ ... break
+ ... data.append((count, next[0:1]))
+ ... count += 1
+ ... data.append((count, next[1:3]))
+ ... count += 1
+ ... data.append((count, next[3:6]))
+ ... count += 1
+ ...
+ >>> print data # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+ [(0,
+ [datetime.datetime(2005, 7, 15, 11, 21, 32, 104, ...<...Eastern...>)]),
+ (1,
+ [datetime.datetime(2005, 7, 15, 21, 21, 32, 104, ...<...Eastern...>),
+ datetime.datetime(2005, 7, 16, 7, 21, 32, 104, ...<...Eastern...>)]),
+ (2,
+ [datetime.datetime(2005, 7, 16, 17, 21, 32, 104, ...<...Eastern...>),
+ datetime.datetime(2005, 7, 17, 3, 21, 32, 104, ...<...Eastern...>),
+ datetime.datetime(2005, 7, 17, 13, 21, 32, 104, ...<...Eastern...>)]),
+ (3,
+ [datetime.datetime(2005, 7, 17, 23, 21, 32, 104, ...<...Eastern...>)]),
+ (4,
+ [datetime.datetime(2005, 7, 18, 9, 21, 32, 104, ...<...Eastern...>),
+ datetime.datetime(2005, 7, 18, 19, 21, 32, 104, ...<...Eastern...>)]),
+ (5,
+ [datetime.datetime(2005, 7, 19, 5, 21, 32, 104, ...<...Eastern...>),
+ datetime.datetime(2005, 7, 19, 15, 21, 32, 104, ...<...Eastern...>),
+ datetime.datetime(2005, 7, 20, 1, 21, 32, 104, ...<...Eastern...>)])]
+ >>> data_dict = dict(data)
+ >>> for doc_id, value in data:
+ ... setindex.index_doc(doc_id, value)
+ ...
+ >>> list(setindex.ids())
+ [0, 1, 2, 3, 4, 5]
+ >>> set(setindex.values()) == set(
+ ... setindex.normalizer.value(v) for v in date_gen())
+ True
+
+For the searches, we will actually use a request and interaction, with an
+adapter that returns the Eastern timezone. This makes the examples less
+dependent on the machine that they use.
+
+ >>> import zope.security.management
+ >>> import zope.publisher.browser
+ >>> import zope.interface.common.idatetime
+ >>> import zope.publisher.interfaces
+ >>> request = zope.publisher.browser.TestRequest()
+ >>> zope.security.management.newInteraction(request)
+ >>> from zope import interface, component
+ >>> @interface.implementer(zope.interface.common.idatetime.ITZInfo)
+ ... @component.adapter(zope.publisher.interfaces.IRequest)
+ ... def tzinfo(req):
+ ... return pytz.timezone('US/Eastern')
+ ...
+ >>> component.provideAdapter(tzinfo)
+ >>> n.all(naive_datetime, index).tzinfo is pytz.timezone('US/Eastern')
+ True
+
+ >>> set(setindex.apply({'any_of': (datetime.date(2005, 7, 17),
+ ... datetime.date(2005, 7, 20),
+ ... datetime.date(2005, 12, 31))})) == set(
+ ... (2, 3, 5))
+ True
+
+Note that this search is using the normalized values.
+
+ >>> set(setindex.apply({'all_of': (
+ ... datetime.datetime(
+ ... 2005, 7, 16, 7, 21, tzinfo=pytz.timezone('US/Eastern')),
+ ... datetime.datetime(
+ ... 2005, 7, 15, 21, 21, tzinfo=pytz.timezone('US/Eastern')),)})
+ ... ) == set((1,))
+ True
+ >>> list(setindex.apply({'any': None}))
+ [0, 1, 2, 3, 4, 5]
+ >>> set(setindex.apply({'between': (
+ ... datetime.datetime(2005, 4, 1, 12), datetime.datetime(2006, 5, 1))})
+ ... ) == set((0, 1, 2, 3, 4, 5))
+ True
+ >>> set(setindex.apply({'between': (
+ ... datetime.datetime(2005, 4, 1, 12), datetime.datetime(2006, 5, 1),
+ ... True, True)})
+ ... ) == set((0, 1, 2, 3, 4, 5))
+ True
+
+'between' searches should deal with dates well.
+
+ >>> set(setindex.apply({'between': (
+ ... datetime.date(2005, 7, 16), datetime.date(2005, 7, 17))})
+ ... ) == set((1, 2, 3))
+ True
+ >>> len(setindex.apply({'between': (
+ ... datetime.date(2005, 7, 16), datetime.date(2005, 7, 17))})
+ ... ) == len(setindex.apply({'between': (
+ ... datetime.date(2005, 7, 15), datetime.date(2005, 7, 18),
+ ... True, True)})
+ ... )
+ True
+
+Removing docs works as usual.
+
+ >>> setindex.unindex_doc(1)
+ >>> list(setindex.ids())
+ [0, 2, 3, 4, 5]
+
+Value, Minvalue and Maxvalue can take timezone-less datetimes and dates.
+
+ >>> setindex.minValue() # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 15, 11, 21, ...<...Eastern...>)
+ >>> setindex.minValue(datetime.date(2005, 7, 17)) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 17, 3, 21, ...<...Eastern...>)
+
+ >>> setindex.maxValue() # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 20, 1, 21, ...<...Eastern...>)
+ >>> setindex.maxValue(datetime.date(2005, 7, 17)) # doctest: +ELLIPSIS
+ datetime.datetime(2005, 7, 17, 23, 21, ...<...Eastern...>)
+
+ >>> list(setindex.values(
+ ... datetime.date(2005, 7, 17), datetime.date(2005, 7, 17)))
+ ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+ [datetime.datetime(2005, 7, 17, 3, 21, ...<...Eastern...>),
+ datetime.datetime(2005, 7, 17, 13, 21, ...<...Eastern...>),
+ datetime.datetime(2005, 7, 17, 23, 21, ...<...Eastern...>)]
+
+ >>> zope.security.management.endInteraction() # TODO put in tests tearDown
+
+Sorting
+-------
+
+The normalization wrapper provides the zope.index.interfaces.IIndexSort
+interface if its upstream index provides it. For example, the
+DateTimeValueIndex will provide IIndexSort, because ValueIndex provides
+sorting. It will also delegate the ``sort`` method to the value index.
+
+ >>> from zc.catalog.index import DateTimeValueIndex
+ >>> from zope.index.interfaces import IIndexSort
+
+ >>> ix = DateTimeValueIndex()
+ >>> IIndexSort.providedBy(ix.index)
+ True
+ >>> IIndexSort.providedBy(ix)
+ True
+ >>> ix.sort.im_self is ix.index
+ True
+
+But it won't work for indexes that doesn't do sorting, for example
+DateTimeSetIndex.
+
+ >>> ix = DateTimeSetIndex()
+ >>> IIndexSort.providedBy(ix.index)
+ False
+ >>> IIndexSort.providedBy(ix)
+ False
+ >>> ix.sort
+ Traceback (most recent call last):
+ ...
+ AttributeError: 'SetIndex' object has no attribute 'sort'
+
\ No newline at end of file
Deleted: zc.catalog/tags/1.4.1/src/zc/catalog/valueindex.txt
===================================================================
--- zc.catalog/trunk/src/zc/catalog/valueindex.txt 2009-02-27 09:39:25 UTC (rev 97335)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/valueindex.txt 2009-02-27 14:44:33 UTC (rev 97355)
@@ -1,222 +0,0 @@
-===========
-Value Index
-===========
-
-The valueindex is an index similar to, but more flexible than a standard Zope
-field index. The index allows searches for documents that contain any of a
-set of values; between a set of values; any (non-None) values; and any empty
-values.
-
-Additionally, the index supports an interface that allows examination of the
-indexed values.
-
-It is as policy-free as possible, and is intended to be the engine for indexes
-with more policy, as well as being useful itself.
-
-On creation, the index has no wordCount, no documentCount, and is, as
-expected, fairly empty.
-
- >>> from zc.catalog.index import ValueIndex
- >>> index = ValueIndex()
- >>> index.documentCount()
- 0
- >>> index.wordCount()
- 0
- >>> index.maxValue() # doctest: +ELLIPSIS
- Traceback (most recent call last):
- ...
- ValueError:...
- >>> index.minValue() # doctest: +ELLIPSIS
- Traceback (most recent call last):
- ...
- ValueError:...
- >>> list(index.values())
- []
- >>> len(index.apply({'any_of': (5,)}))
- 0
-
-The index supports indexing any value. All values within a given index must
-sort consistently across Python versions.
-
- >>> data = {1: 'a',
- ... 2: 'b',
- ... 3: 'a',
- ... 4: 'c',
- ... 5: 'd',
- ... 6: 'c',
- ... 7: 'c',
- ... 8: 'b',
- ... 9: 'c',
- ... }
- >>> for k, v in data.items():
- ... index.index_doc(k, v)
- ...
-
-After indexing, the statistics and values match the newly entered content.
-
- >>> list(index.values())
- ['a', 'b', 'c', 'd']
- >>> index.documentCount()
- 9
- >>> index.wordCount()
- 4
- >>> index.maxValue()
- 'd'
- >>> index.minValue()
- 'a'
- >>> list(index.ids())
- [1, 2, 3, 4, 5, 6, 7, 8, 9]
-
-The index supports four types of query. The first is 'any_of'. It
-takes an iterable of values, and returns an iterable of document ids that
-contain any of the values. The results are not weighted.
-
- >>> list(index.apply({'any_of':('b', 'c')}))
- [2, 4, 6, 7, 8, 9]
- >>> list(index.apply({'any_of': ('b',)}))
- [2, 8]
- >>> list(index.apply({'any_of': ('d',)}))
- [5]
- >>> list(index.apply({'any_of':(42,)}))
- []
-
-Another query is 'any', If the key is None, all indexed document ids with any
-values are returned. If the key is an extent, the intersection of the extent
-and all document ids with any values is returned.
-
- >>> list(index.apply({'any': None}))
- [1, 2, 3, 4, 5, 6, 7, 8, 9]
-
- >>> from zc.catalog.extentcatalog import FilterExtent
- >>> extent = FilterExtent(lambda extent, uid, obj: True)
- >>> for i in range(15):
- ... extent.add(i, i)
- ...
- >>> list(index.apply({'any': extent}))
- [1, 2, 3, 4, 5, 6, 7, 8, 9]
- >>> limited_extent = FilterExtent(lambda extent, uid, obj: True)
- >>> for i in range(5):
- ... limited_extent.add(i, i)
- ...
- >>> list(index.apply({'any': limited_extent}))
- [1, 2, 3, 4]
-
-The 'between' argument takes from 1 to four values. The first is the
-minimum, and defaults to None, indicating no minimum; the second is the
-maximum, and defaults to None, indicating no maximum; the next is a boolean for
-whether the minimum value should be excluded, and defaults to False; and the
-last is a boolean for whether the maximum value should be excluded, and also
-defaults to False. The results are not weighted.
-
- >>> list(index.apply({'between': ('b', 'd')}))
- [2, 4, 5, 6, 7, 8, 9]
- >>> list(index.apply({'between': ('c', None)}))
- [4, 5, 6, 7, 9]
- >>> list(index.apply({'between': ('c',)}))
- [4, 5, 6, 7, 9]
- >>> list(index.apply({'between': ('b', 'd', True, True)}))
- [4, 6, 7, 9]
-
-The 'none' argument takes an extent and returns the ids in the extent
-that are not indexed; it is intended to be used to return docids that have
-no (or empty) values.
-
- >>> list(index.apply({'none': extent}))
- [0, 10, 11, 12, 13, 14]
-
-Trying to use more than one of these at a time generates an error.
-
- >>> index.apply({'between': (5,), 'any_of': (3,)})
- ... # doctest: +ELLIPSIS
- Traceback (most recent call last):
- ...
- ValueError:...
-
-Using none of them simply returns None.
-
- >>> index.apply({}) # returns None
-
-Invalid query names cause ValueErrors.
-
- >>> index.apply({'foo':()})
- ... # doctest: +ELLIPSIS
- Traceback (most recent call last):
- ...
- ValueError:...
-
-When you unindex a document, the searches and statistics should be updated.
-
- >>> index.unindex_doc(5)
- >>> len(index.apply({'any_of': ('d',)}))
- 0
- >>> index.documentCount()
- 8
- >>> index.wordCount()
- 3
- >>> list(index.values())
- ['a', 'b', 'c']
- >>> list(index.ids())
- [1, 2, 3, 4, 6, 7, 8, 9]
-
-Reindexing a document that has a changed value also is reflected in
-subsequent searches and statistic checks.
-
- >>> list(index.apply({'any_of': ('b',)}))
- [2, 8]
- >>> data[8] = 'e'
- >>> index.index_doc(8, data[8])
- >>> index.documentCount()
- 8
- >>> index.wordCount()
- 4
- >>> list(index.apply({'any_of': ('e',)}))
- [8]
- >>> list(index.apply({'any_of': ('b',)}))
- [2]
- >>> data[2] = 'e'
- >>> index.index_doc(2, data[2])
- >>> index.documentCount()
- 8
- >>> index.wordCount()
- 3
- >>> list(index.apply({'any_of': ('e',)}))
- [2, 8]
- >>> list(index.apply({'any_of': ('b',)}))
- []
-
-Reindexing a document for which the value is now None causes it to be removed
-from the statistics.
-
- >>> data[3] = None
- >>> index.index_doc(3, data[3])
- >>> index.documentCount()
- 7
- >>> index.wordCount()
- 3
- >>> list(index.ids())
- [1, 2, 4, 6, 7, 8, 9]
-
-This affects both ways of determining the ids that are and are not in the index
-(that do and do not have values).
-
- >>> list(index.apply({'any': None}))
- [1, 2, 4, 6, 7, 8, 9]
- >>> list(index.apply({'any': extent}))
- [1, 2, 4, 6, 7, 8, 9]
- >>> list(index.apply({'none': extent}))
- [0, 3, 5, 10, 11, 12, 13, 14]
-
-The values method can be used to examine the indexed values for a given
-document id. For a valueindex, the "values" for a given doc_id will always
-have a length of 0 or 1.
-
- >>> index.values(doc_id=8)
- ('e',)
-
-And the containsValue method provides a way of determining membership in the
-values.
-
- >>> index.containsValue('a')
- True
- >>> index.containsValue('q')
- False
Copied: zc.catalog/tags/1.4.1/src/zc/catalog/valueindex.txt (from rev 97338, zc.catalog/trunk/src/zc/catalog/valueindex.txt)
===================================================================
--- zc.catalog/tags/1.4.1/src/zc/catalog/valueindex.txt (rev 0)
+++ zc.catalog/tags/1.4.1/src/zc/catalog/valueindex.txt 2009-02-27 14:44:33 UTC (rev 97355)
@@ -0,0 +1,259 @@
+===========
+Value Index
+===========
+
+The valueindex is an index similar to, but more flexible than a standard Zope
+field index. The index allows searches for documents that contain any of a
+set of values; between a set of values; any (non-None) values; and any empty
+values.
+
+Additionally, the index supports an interface that allows examination of the
+indexed values.
+
+It is as policy-free as possible, and is intended to be the engine for indexes
+with more policy, as well as being useful itself.
+
+On creation, the index has no wordCount, no documentCount, and is, as
+expected, fairly empty.
+
+ >>> from zc.catalog.index import ValueIndex
+ >>> index = ValueIndex()
+ >>> index.documentCount()
+ 0
+ >>> index.wordCount()
+ 0
+ >>> index.maxValue() # doctest: +ELLIPSIS
+ Traceback (most recent call last):
+ ...
+ ValueError:...
+ >>> index.minValue() # doctest: +ELLIPSIS
+ Traceback (most recent call last):
+ ...
+ ValueError:...
+ >>> list(index.values())
+ []
+ >>> len(index.apply({'any_of': (5,)}))
+ 0
+
+The index supports indexing any value. All values within a given index must
+sort consistently across Python versions.
+
+ >>> data = {1: 'a',
+ ... 2: 'b',
+ ... 3: 'a',
+ ... 4: 'c',
+ ... 5: 'd',
+ ... 6: 'c',
+ ... 7: 'c',
+ ... 8: 'b',
+ ... 9: 'c',
+ ... }
+ >>> for k, v in data.items():
+ ... index.index_doc(k, v)
+ ...
+
+After indexing, the statistics and values match the newly entered content.
+
+ >>> list(index.values())
+ ['a', 'b', 'c', 'd']
+ >>> index.documentCount()
+ 9
+ >>> index.wordCount()
+ 4
+ >>> index.maxValue()
+ 'd'
+ >>> index.minValue()
+ 'a'
+ >>> list(index.ids())
+ [1, 2, 3, 4, 5, 6, 7, 8, 9]
+
+The index supports four types of query. The first is 'any_of'. It
+takes an iterable of values, and returns an iterable of document ids that
+contain any of the values. The results are not weighted.
+
+ >>> list(index.apply({'any_of':('b', 'c')}))
+ [2, 4, 6, 7, 8, 9]
+ >>> list(index.apply({'any_of': ('b',)}))
+ [2, 8]
+ >>> list(index.apply({'any_of': ('d',)}))
+ [5]
+ >>> list(index.apply({'any_of':(42,)}))
+ []
+
+Another query is 'any', If the key is None, all indexed document ids with any
+values are returned. If the key is an extent, the intersection of the extent
+and all document ids with any values is returned.
+
+ >>> list(index.apply({'any': None}))
+ [1, 2, 3, 4, 5, 6, 7, 8, 9]
+
+ >>> from zc.catalog.extentcatalog import FilterExtent
+ >>> extent = FilterExtent(lambda extent, uid, obj: True)
+ >>> for i in range(15):
+ ... extent.add(i, i)
+ ...
+ >>> list(index.apply({'any': extent}))
+ [1, 2, 3, 4, 5, 6, 7, 8, 9]
+ >>> limited_extent = FilterExtent(lambda extent, uid, obj: True)
+ >>> for i in range(5):
+ ... limited_extent.add(i, i)
+ ...
+ >>> list(index.apply({'any': limited_extent}))
+ [1, 2, 3, 4]
+
+The 'between' argument takes from 1 to four values. The first is the
+minimum, and defaults to None, indicating no minimum; the second is the
+maximum, and defaults to None, indicating no maximum; the next is a boolean for
+whether the minimum value should be excluded, and defaults to False; and the
+last is a boolean for whether the maximum value should be excluded, and also
+defaults to False. The results are not weighted.
+
+ >>> list(index.apply({'between': ('b', 'd')}))
+ [2, 4, 5, 6, 7, 8, 9]
+ >>> list(index.apply({'between': ('c', None)}))
+ [4, 5, 6, 7, 9]
+ >>> list(index.apply({'between': ('c',)}))
+ [4, 5, 6, 7, 9]
+ >>> list(index.apply({'between': ('b', 'd', True, True)}))
+ [4, 6, 7, 9]
+
+The 'none' argument takes an extent and returns the ids in the extent
+that are not indexed; it is intended to be used to return docids that have
+no (or empty) values.
+
+ >>> list(index.apply({'none': extent}))
+ [0, 10, 11, 12, 13, 14]
+
+Trying to use more than one of these at a time generates an error.
+
+ >>> index.apply({'between': (5,), 'any_of': (3,)})
+ ... # doctest: +ELLIPSIS
+ Traceback (most recent call last):
+ ...
+ ValueError:...
+
+Using none of them simply returns None.
+
+ >>> index.apply({}) # returns None
+
+Invalid query names cause ValueErrors.
+
+ >>> index.apply({'foo':()})
+ ... # doctest: +ELLIPSIS
+ Traceback (most recent call last):
+ ...
+ ValueError:...
+
+When you unindex a document, the searches and statistics should be updated.
+
+ >>> index.unindex_doc(5)
+ >>> len(index.apply({'any_of': ('d',)}))
+ 0
+ >>> index.documentCount()
+ 8
+ >>> index.wordCount()
+ 3
+ >>> list(index.values())
+ ['a', 'b', 'c']
+ >>> list(index.ids())
+ [1, 2, 3, 4, 6, 7, 8, 9]
+
+Reindexing a document that has a changed value also is reflected in
+subsequent searches and statistic checks.
+
+ >>> list(index.apply({'any_of': ('b',)}))
+ [2, 8]
+ >>> data[8] = 'e'
+ >>> index.index_doc(8, data[8])
+ >>> index.documentCount()
+ 8
+ >>> index.wordCount()
+ 4
+ >>> list(index.apply({'any_of': ('e',)}))
+ [8]
+ >>> list(index.apply({'any_of': ('b',)}))
+ [2]
+ >>> data[2] = 'e'
+ >>> index.index_doc(2, data[2])
+ >>> index.documentCount()
+ 8
+ >>> index.wordCount()
+ 3
+ >>> list(index.apply({'any_of': ('e',)}))
+ [2, 8]
+ >>> list(index.apply({'any_of': ('b',)}))
+ []
+
+Reindexing a document for which the value is now None causes it to be removed
+from the statistics.
+
+ >>> data[3] = None
+ >>> index.index_doc(3, data[3])
+ >>> index.documentCount()
+ 7
+ >>> index.wordCount()
+ 3
+ >>> list(index.ids())
+ [1, 2, 4, 6, 7, 8, 9]
+
+This affects both ways of determining the ids that are and are not in the index
+(that do and do not have values).
+
+ >>> list(index.apply({'any': None}))
+ [1, 2, 4, 6, 7, 8, 9]
+ >>> list(index.apply({'any': extent}))
+ [1, 2, 4, 6, 7, 8, 9]
+ >>> list(index.apply({'none': extent}))
+ [0, 3, 5, 10, 11, 12, 13, 14]
+
+The values method can be used to examine the indexed values for a given
+document id. For a valueindex, the "values" for a given doc_id will always
+have a length of 0 or 1.
+
+ >>> index.values(doc_id=8)
+ ('e',)
+
+And the containsValue method provides a way of determining membership in the
+values.
+
+ >>> index.containsValue('a')
+ True
+ >>> index.containsValue('q')
+ False
+
+Sorting
+-------
+
+Value indexes supports sorting, just like zope.index.field.FieldIndex.
+
+ >>> index.clear()
+
+ >>> index.index_doc(1, 9)
+ >>> index.index_doc(2, 8)
+ >>> index.index_doc(3, 7)
+ >>> index.index_doc(4, 6)
+ >>> index.index_doc(5, 5)
+ >>> index.index_doc(6, 4)
+ >>> index.index_doc(7, 3)
+ >>> index.index_doc(8, 2)
+ >>> index.index_doc(9, 1)
+
+ >>> list(index.sort([4, 2, 9, 7, 3, 1, 5]))
+ [9, 7, 5, 4, 3, 2, 1]
+
+We can also specify the ``reverse`` argument to reverse results:
+
+ >>> list(index.sort([4, 2, 9, 7, 3, 1, 5], reverse=True))
+ [1, 2, 3, 4, 5, 7, 9]
+
+And as per IIndexSort, we can limit results by specifying the ``limit``
+argument:
+
+ >>> list(index.sort([4, 2, 9, 7, 3, 1, 5], limit=3))
+ [9, 7, 5]
+
+If we pass an id that is not indexed by this index, it won't be included
+in the result.
+
+ >>> list(index.sort([2, 10]))
+ [2]
More information about the Checkins
mailing list