[ZODB-Dev] Re: Serious ZODB time reversal

[Dieter Maurer]

Andreas Kaiser wrote at 2004-9-2 13:12 +0200:
>...
>> The behaviour drastically improved after we banned relevance ranking
>> information.
>
>Could you please provide a little more information on how to "bann 
>relevance ranking information"? I hardly know anything about ZODB 
>internals (never heard of "relevance ranking information" at all), so 
>any pointers to relevant documentation are appreciated.

Andreas Jung, the TextIndexNG2 author, is a former colleague of us.
We still work closely together.

When Andreas got aware of the problems with relevance ranking,
he implemented a so called "Stupid Storage" for "TextIndexNG2"
which does not store relevance ranking information.
We banned it by using this storage.

The new feature will probably be publicly available in the next "TextIndexNG2"
release.


You might be able to use my "Managable WordIndex".

  <http://www.dieter.handshake.de/pyprojects/zope#ManagableIndex>

As is is drastically restricted and (almost) requires use of
"AdvancedQuery" (from the same page), this may not be the case, though.


"ZCTextIndex" may allow you to control whether to use relevance ranking
or not. I did not check this, however. You must find out yourself.

> ...
>Yes, they don´t seem to cause any inconsitencies, the only problem is 
>the enormous cpu time consumed, which makes the application rather 
>unuasble for users who don´t have the "Manager" role associated.

This suggests that you are using the wrong type of
index for "expires" and "effective".

Use a "DateRangeIndex" or (even better) a
"Managable RangeIndex with type "DateInteger".

>> I expect some bug in the "TemporaryStorage" implementation.

I checked and recognized that I was wrong.

There is no serious problem -- just an exchange of "was" and "now".
This means: "was" is the serial at commit time, "now" that one at
load time (which is not later than the commit time serial).

>Would replacing "TemporaryStorage" by e.g. "FileStorage" help in any 
>way?

All signs are that there is some problem with Sessions/TemporaryStorage.
We just do not yet know what precisely the problem is.

However, that the "was serial" is later than the "now serial" is
not a serious problem -- contrary to what I had expected.

> Would there be any side-effects?

Sure. There is a reason that sessions use their own special
storage: the temporary storage is significantly more efficient -- especially
for high write rates.


-- 
Dieter