[ZODB-Dev] Re: ZODB Benchmarks

[Dieter Maurer]

Roché Compaan wrote at 2008-2-3 09:15 +0200:
> ...
>I have tried different commit intervals. The published results are for a
>commit interval of 100, iow 100 inserts per commit.
>
>> Your profile looks very surprising:
>> 
>>   I would expect that for a single insertion, typically
>>   one persistent object (the bucket where the insertion takes place)
>>   is changed. About every 15 inserts, 3 objects are changed (the bucket
>>   is split) about every 15*125 inserts, 5 objects are changed
>>   (split of bucket and its container).
>>   But the mean value of objects changed in a transaction is 20
>>   in your profile.
>>   The changed objects typically have about 65 subobjects. This
>>   fits with "OOBucket"s.
>
>It was very surprising to me too since the insertion is so basic. I
>simply assign a Persistent object with 1 string attribute that is 1K in
>size to a key in a OOBTree. I mentioned this earlier on the list and I
>thought that Jim's explanation was sufficient when he said that the
>persistent_id method is called for all objects including simple types
>like strings, ints, etc. I don't know if it explains all the calls that
>add up to a mean value of 20 though. I guess the calls are being made by
>the cPickle module, but I don't have the experience to investigate this.

The number of "persitent_id" calls suggests that a written
persistent object has a mean value of 65 subobjects -- which
fits well will OOBuckets.

However, when the profile is for commits with 100 insertions each,
then the number of written persistent objects is far too small.
In fact, we would expect about 200 persistent object writes per transaction:
the 100 new persistent objects assigned plus about as many buckets
changed by these insertions.

> ....
>The keys that I lookup are completely random so it is probably the case
>that the lookup causes disk lookups all the time. If this is the case,
>is 230ms not still to slow?

Unreasonably slow in fact.

A tree with size 10**7 does likely not have a depth larger than 4
(internal nodes should typically have at least 125 entries, leaves should have
at least 15 -- a tree of depth 4 thus can have about 125**3*15 = 29.x * 10**6).
Therefore, one would expect at most 4 disk accesses.

On my (6 year old) computer, a disk access can take up to 30 ms.



-- 
Dieter