[Zope3-dev] catalog 'all documents' abstraction
Martijn Faassen
faassen at infrae.com
Tue Aug 30 13:57:52 EDT 2005
Jim Fulton wrote:
> Martijn Faassen wrote:
[snip]
> > ). I also think however that it's the wrong
>
>> place the ask for this information, as this doesn't work with the
>> extentcatalog.
>
> Well, it depends on what you meant by "indexed" above. Different indexes
> index different objects. The extent catalog tried to define an extent for
> which it makes sense to apply different (not) operations.
And is the idea that multiple extent catalogs can share an extent?
(By the way you say 'the extent catalog tried', does this mean something
else is being considered now?)
>> The catalog itself seems like the wrong place to ask as well however,
>> as things would get hairy in the case of a query over multiple
>> catalogs -- which catalog would be asked for all ids that are indexed?
>
> In general, I'd like the catalog to remain fairly small and free of logic.
> I wanted to say this in the other thread you started on cataloging, but
> didn't get to it. Ideally, a catalog wouldn't have any query logic
> at all. People should be able to invovate on query algorithms without
> affecting catalogs.
This is already clear; I've been trying to do so in the project I'm
working on, though I'm more focusing on a sensible query language (well,
of Python objects) than performance algorithms.
At the same time I believe Zope 3 *does* need query systems built in
eventually. While it's fine to allow people to design their own query
languages and algorithms, not everybody is able to do this, and those
who are able to don't always want to. Even if they did, I don't want us
to end up with 5 different query systems either.
So, while I agree that a query language in the core should not exclude
someone from building something better, I do believe a catalog query
language package is needed in the core.
(To be absolutely clear: I also think the RDF avenues being explored are
very interesting, and I don't want to imply that this is not an
interesting direction, but I do think we need something for the plain
catalog too)
>> Hm, perhaps this isn't ideal either, as this would get hairy in case
>> of a query that spans multiple catalogs -- which catalog will be asked
>> in that case for a list of all documents?
>
> I think in the particular case of "not", you have to have an implicit or
> explicit set that you are subtracting something from. The "right" set is
> application specific. In any case, I think the query logic should be
> in separate query components.
I agree that the catalog should remain nice and simple.
That said, catalogs right now already have an implicit concept of
'everything indexed', which for instance is already used for
re-indexing, it's just not made available to someone who wants to build
something on it.
The extent catalog makes this more explicit by defining an extent, so
perhaps this is the way to go. The extent could be a query parameter to
help the query engine figure out what to do in case of 'not'. For simple
use cases, the extent can be constructed from the intid utility, perhaps.
It would be helpful if someone could explain the motivations behind the
extent catalog, by the way -- this information seems to be missing in
zc.catalog. Am I at all on the right track with my thinking on it?
Regards,
Martijn
More information about the Zope3-dev
mailing list