[ZODB-Dev] Re: What makes the ZODB slow?

[Florent Guillaume]

Jean Jordaan wrote:
>> The ZODB is actually very fast. [...]
>>
>> So you're probably observing slowness in the frameworks on top of it.
> 
> I'll believe this anytime :-]
> 
> In our case, a transaction may be a workflow state change on say 50 objects.
> Two or three people try a transaction like that within a couple of seconds
> of one another, and ConflictErrors crop up.
> 
> In a log with 402 ConflictErrors, 225 are on BTrees (_IIBTree.IITreeSet,
> _IOBTree.IOBucket, _OOBTree.OOBTree, _OOBTree.OOBucket all feature). We
> assume these all relate to catalog indexing.
> 
> Of the rest, 147 of the 177 are either Products.Transience.Transience.Increaser
> or Products.Transience.Transience.Length2
> 
> The role the framework (Plone, unsurprisingly) is playing in this case, is
> that it leans hard on the catalog during a transaction lasting a number of
> seconds.
> 
> To mitigate this, we want to create a savepoint and then commit more often
> while iterating and changing workflow, rolling back to the savepoint if
> necessary.
> 

BTrees perform best when keys' prefixes are randomly distributed.
So if your application generates keys like 'foo001', 'foo002',... you'll 
get lots of conflicts. Same for consecutive integers in IOBTree.

Florent

-- 
Florent Guillaume, Nuxeo (Paris, France)   Director of R&D
+33 1 40 33 71 59   http://nuxeo.com   fg at nuxeo.com