[ZODB-Dev] Re: What makes the ZODB slow?

[Dieter Maurer]

Chris Withers wrote at 2006-6-26 14:00 +0100:
> ...
>Ah, conflict errors, the bane of any ZODB app. Makes me wonder why other 
>optimistic concurrency databases (I believe PostGres is one of these?) 
>don't seem to exhibit the same problems...

"PostGres" does use looks, lots of them and for different purposes.

The only thing for which "Postgres" does not use locks is reading.
For this is uses MVCC (which we meanwhile adapted for the ZODB
to get rid of "ReadConflictError"s).

And even when locks are used, conflicts arise (they take on the
form of deadlocks). I have seen several of them with Postgres
-- not as deadlocks but as "concurrent update failed".

>> In a log with 402 ConflictErrors, 225 are on BTrees (_IIBTree.IITreeSet,
>> _IOBTree.IOBucket, _OOBTree.OOBTree, _OOBTree.OOBucket all feature). We
>> assume these all relate to catalog indexing.
>
>Now, here's the crux of the matter. 99% of conflicts I see happen in the 
>indexing stuff (ie: ZCatalog) and I really think these shouldn't. Search 
>should always be accurate, but may be slow. I reckon if we could sort 
>out a way that conflicts in indexing objects can better be dealt with, 
>99% of the problem goes away..

Most of our "ConflictError"s come from the session machinery -- because
conflict resolution works there only in a very limited way
(due to limited history availability).

>> Of the rest, 147 of the 177 are either Products.Transience.Transience.Increaser
>> or Products.Transience.Transience.Length2

Yes, these are our hits -- despite the fact that our "increaser"
is much more intelligent (and increases only rarely and not 
on each access)



-- 
Dieter