[Zope3-dev] catalog 'all documents' abstraction

[Jim Fulton]

Martijn Faassen wrote:
> Jim Fulton wrote:
> 
>> Martijn Faassen wrote:
> 
...
>> In general, I'd like the catalog to remain fairly small and free of 
>> logic.
>> I wanted to say this in the other thread you started on cataloging, but
>> didn't get to it.  Ideally, a catalog wouldn't have any query logic
>> at all.  People should be able to invovate on query algorithms without
>> affecting catalogs.
> 
> 
> This is already clear;

Cool. Just making sure. :)

 > I've been trying to do so in the project I'm
> working on, though I'm more focusing on a sensible query language (well, 
> of Python objects) than performance algorithms.

Cool.  I really wish someone would pick up on Aaron Waters Gadfly 2 work.
I strongly suspect that there are some interesting bits to mine there.


> At the same time I believe Zope 3 *does* need query systems built in 
> eventually. While it's fine to allow people to design their own query 
> languages and algorithms, not everybody is able to do this, and those 
> who are able to don't always want to.

Agreed.

 > Even if they did, I don't want us
> to end up with 5 different query systems either.

Hm. I wouldn't mind a bit of they addressed different needs.

> So, while I agree that a query language in the core should not exclude 
> someone from building something better, I do believe a catalog query 
> language package is needed in the core.

I agree that a good query mechanism should be in the core, but it should be
a separate component. It should not be in the catalog.

> (To be absolutely clear: I also think the RDF avenues being explored are 
> very interesting, and I don't want to imply that this is not an 
> interesting direction, but I do think we need something for the plain 
> catalog too)

Yup

>>> Hm, perhaps this isn't ideal either, as this would get hairy in case 
>>> of a query that spans multiple catalogs -- which catalog will be 
>>> asked in that case for a list of all documents? 
>>
>>
>> I think in the particular case of "not", you have to have an implicit or
>> explicit set that you are subtracting something from.  The "right" set is
>> application specific.  In any case, I think the query logic should be
>> in separate query components.
> 
> 
> I agree that the catalog should remain nice and simple.
> 
> That said, catalogs right now already have an implicit concept of 
> 'everything indexed', which for instance is already used for 
> re-indexing, it's just not made available to someone who wants to build 
> something on it.

Welllll, it has a concept of of everything that could be indexed.
It generally won't index everything in the intids.

> The extent catalog makes this more explicit by defining an extent, so 
> perhaps this is the way to go.

I suspect so.

 > The extent could be a query parameter to
> help the query engine figure out what to do in case of 'not'. For simple 
> use cases, the extent can be constructed from the intid utility, perhaps.
> 
> It would be helpful if someone could explain the motivations behind the 
> extent catalog, by the way -- this information seems to be missing in 
> zc.catalog. Am I at all on the right track with my thinking on it?

What information?

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org