[ZODB-Dev] Increasing MAX_BUCKET_SIZE for IISet, etc

[Hanno Schlichting]

On Thu, Jan 27, 2011 at 11:09 AM, Matt Hamilton <matth at netsight.co.uk> wrote:
> Hanno Schlichting <hanno <at> hannosch.eu> writes:
>
> There still seem to be instances in which the entire set is loaded.  This
> could be an artifact of the fact I am clearing the ZODB cache before each
> ]test, which I think seems to be clearing the query plan.

Yes. The queryplan is stored in a volatile attribute, so clearing the
zodb cache will throw away the plan. The queryplan version integrated
into Zope 2.13 stores the plan in a module global with thread locks
around it.

> Speaking of
> which I saw in the query plan code, some hook to load a pre-defined query
> plan... but I can't see exactly how you supply this plan or in what format
> it is. Do you use this feature?

You get a plan representation by calling:

http://localhost:8080/Plone/@@catalogqueryplan-prioritymap

Then add an environment variable pointing to a variable inside a module:

[instance]
recipe = plone.recipe.zope2instance
environment-vars =
    CATALOGQUERYPLAN my.customer.module.queryplan

Create that module and put the dump in it. it should start with something like:

# query plan dumped at 'Mon May 24 01:33:28 2010'

queryplan = {
  '/Plone/portal_catalog': {
...
}

You can keep updating this plan with some new data from the dump once
in a while.

Ideally this plan should be persisted in the database at certain
intervals, but we haven't implemented that yet. You don't want to
persist the plan in every request doing a catalog query.

Hanno