[Zope3-dev] catalog 'all documents' abstraction

Jim Fulton jim at zope.com
Wed Aug 31 09:54:53 EDT 2005


Martijn Faassen wrote:
> Gary Poster wrote:
> 
>> On Aug 30, 2005, at 1:57 PM, Martijn Faassen wrote:
> 
> [snip]
> 
>>> It would be helpful if someone could explain the motivations behind
>>>  the extent catalog, by the way -- this information seems to be 
>>> missing in zc.catalog. Am I at all on the right track with my 
>>> thinking on it?
> 
> 
>> It should be pointed out initially that the son-of-queued-catalog
>> code doesn't have anything to do with extents.  I think Jim wants
>> that factored out when we have time so that can be a mix-and-match 
>> capability.  I think you are asking about extents themselves, though.
>>
> Okay, I didn't realize yet glancing at this that this is *also*
> son-of-queued catalog. Interesting. I'll glance at it some more. :)

Actually, it isn't.  I should have thought to menion this earlier.
This code just provides a within-transaction queue, so that multiple
modifications to an object within a transaction cause only one indexing
operation.

Queued catalog uses a persistent queue to allow a separate dedicated
thread to do indexing to reduce conflicts and to provide greater aggregation
of index operations to reduce database writes.  Queued catalog needs to
be ported still. It *also* needs some repair to a bug in it's queue
conflict-resolution algorithm to improve it's effectiveness. Tests
wouldn't hurt either. :)

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org


More information about the Zope3-dev mailing list