[Zope-Checkins] CVS: Zope/lib/python/Products/PluginIndexes/TextIndexNG - TextIndexNG.py:1.2.2.14

[Andreas Jung]

Update of /cvs-repository/Zope/lib/python/Products/PluginIndexes/TextIndexNG
In directory cvs.zope.org:/tmp/cvs-serv20481

Modified Files:
      Tag: ajung-textindexng-branch
	TextIndexNG.py 
Log Message:
minor cleanup


=== Zope/lib/python/Products/PluginIndexes/TextIndexNG/TextIndexNG.py 1.2.2.13 => 1.2.2.14 ===
         # Split the text into a list of words
         # The splitterfunc just returns an iterator-like object.
-        # For performance reasons it might be better when the 
-        # splitter returns the list of splitted words.
 
         words = self._v_splitterfunc(source,encoding=encoding).split()
 
@@ -369,8 +367,7 @@
 
 
         # Check if we want proximity searches. If yes, we need to create
-        # a list containing the proximity representations of the of     
-        # the words
+        # a list containing the proximity representations of the words     
 
         if self.useProximity:
             proximity_words =  self._v_proximityfunc(words)
@@ -384,10 +381,10 @@
         if self._v_stemmerfunc:
             words = self._v_stemmerfunc(words)
 
-        # we collect all wordIds for performance reasons in a list
-        # and update the backward index once instead of inserting
-        # every single wordId
 
+        # We pass the list of words to the corresponding lexicon
+        # and obtain a list of wordIds. The "old" TextIndex iterated
+        # over every single words (overhead).
 
         widLst = self._v_getWordIdList(words)
         assert len(widLst)==len(words)