[ZCM] [ZC] 304/ 1 Request "ZCatalog Optimisation"

[Collector: Zope Bugs and Patches ...]

Issue #304 Update (Request) "ZCatalog Optimisation"
 Status Accepted, Zope/feature+solution medium
To followup, visit:
  http://collector.zope.org/Zope/304

==============================================================
= Request - Entry #1 by htrd on Mar 19, 2002 8:17 am

 Status: Pending => Accepted

 Supporters added: htrd


Uploaded:  "catalogopt.diff"
 - http://collector.zope.org/Zope/304/catalogopt.diff/view
This patch includes three significant optimisations for ZCatalog.

I had one search that was taking much longer than it really should have done; roughly 10s. These combined optimisations take it down to 400ms.


1. ZCatalog implements search differently depending on the relative size of the results set, compared to the number of buckets in the sort index. This optimisation was previously not applied to a match-everything search results set. This patch refactors the sort implementation into a separate _build_sorted_results function.

This reduced the search time from 10s to 5s

2. After sorting, the results list needs to be crunched down from a list of 2-tuples (sort-value, result) into a simple list of results. This previously was using map/lambda. This has been replaced with a list comprehension.

This reduced the search time from 5s to 2s


3. The inner loop of _build_sorted_results is run once per result. Moving several immutable attribute lookups out of the inner loop has reduced the search time from 2s to 0.4s


==============================================================