[Zope-dev] ZCatalog scalability

[Michael Bernstein]

After comsidering the feedback I got from the previous
'Massive scalability' thread, I decided to split my queries
into two areas: Rack scalability and ZCatalog scalability.
This email deals with the latter.

Partial match (wildcard) searches have already been
identified as a resource hog, depending on the size of the
result list. I am more than willing to give up wildcards in
my application for performance.

What I am interested in for my application are two things:

- ZTopics populated using one or more keyword indexes

- Full text search on a single computed attribute that
concatenates several fields including the aforementioned
keyword index fields and a few simple string attributes
(title, caption, description, etc.)

I need to know how far the ZCatalog will scale using this
indexing and search strategy. Does anyone have anectodal or
benchmark data to suggest if (and when) I will hit a 'wall'
regarding the number of objects being indexed and searched?
Some anectodal data suggests that single field indexing will
scale easily to 60,000 objects, what about hundreds of
thousands or even millions of objects?

Also, is there a way to disable wildcards in full text
searches?

Thanks,

Michael Bernstein.