[ZODB-Dev] Re: Community opinion about search+filter

[Dieter Maurer]

Ross Patterson wrote at 2007-3-15 14:25 -0700:
> ...
>I recently became obsessed with this problem and sketched out an
>architecture for presorted indexes.  I thought I'd take this
>opportunity to get some review of what I came to.
>
>>From my draft initial README:
>
>> Presort provides intids which assure the corresponding documents will
>> be presorted in any BTrees objects where the intid is used as a key.
>
>> Presorted intids exist alongside normal intids.  Intids are
>> distributed over the range of integers available on a platform so as
>> to avoid moving presorted intids whenever possible, but eventually a
>> given presorted intid may need to be placed in between two other
>> consecutive presorted intids.  When this happens, one or more
>> presorted intids will have to be moved.  Normal intids are unchanging
>> as normal.
>
>> Presort also provides a facility for updating objects who store
>> presorted intids when a presorted intid is moved, such as in indexes.
>> It also provides for indexes that map a given query to the appropriate
>> presorted intid result set and for catalogs that use the appropriate
>> presorted intid utility to lookup the real objects for the results.
>
>Would this be a viable approach? 

It would be a special case where an object's rank depends only
on the object and not the query (see my earlier response to Martijn).

"IncrementalSearch[2]" can very efficiently generate initial
sorted batches in such cases.

> Would it be generally useful?

Only when your application needs only very few different orders (for
the largest result sets): you need different presorted ids and a complete
index set using them  for each such order.

Thus, useful for special case applications with large data sets
but not useful for all applications or when only small data sets are
involved.



-- 
Dieter