[Zope-CVS] CVS: Products/ZCTextIndex - QueryParser.py:

Jeremy Hylton jeremy@zope.com
Thu, 2 May 2002 22:43:18 -0400

Update of /cvs-repository/Products/ZCTextIndex
In directory cvs.zope.org:/tmp/cvs-serv32468

Modified Files:
      Tag: TextIndexDS9-branch
Log Message:
Add a terms() function to the QueryParser module that returns a
sequence of atoms contained in the query.  This is needed to compute
W(q) for the cosine rule.

Note that there are odd interactions with NOT nodes and the
computation of W(q).  We need to think more about what to do in the
presence of NOT, but it seems possible that we'll need a terms()-like
function that returns all terms that are not excluded, i.e. all AND
and OR terms but no NOT terms.  Tim noted that we need to handle cases
like "a AND b and NOT (c or NOT d)" which should presumably return a,
b, d (or should it).

I'm not wedded to the implementation. I guess I was feeling functional
tonight, because reduce seemed like the simplest thing that could
possibly work.

=== Products/ZCTextIndex/QueryParser.py => ===
+import operator
 import re
 # Create unique symbols for token types.
@@ -160,3 +161,14 @@
 class AtomNode(ParseTreeNode):
     _nodeType = _ATOM
+def terms(node):
+    """Return all the ATOM nodes in a parse tree node."""
+    # XXX This is used in the test suite to find the non-NOT terms,
+    # which works only because the test queries don't have NOT terms.
+    # We'll need a more general mechanism to extra the terms if
+    # query_weight is factored into the scoring.
+    if isinstance(node, AtomNode):
+        return [node.getValue()]
+    else:
+        return reduce(operator.add, [terms(v) for v in node.getValue()])