[ZODB-Dev] PersistentMapping good for large numbers of objects?
Shane Hathaway
shane at zope.com
Tue Jul 22 19:21:31 EDT 2003
Tim Peters wrote:
> [Shane Hathaway]
>
>>How about a combination of the above suggestions: every operation that
>>mutates a BTree could look for an attribute of the BTree called
>>"length".
>
> Wow -- I've been looking for ways to slow BTrees, and this one didn't even
> occur to me <wink>.
I don't think we're thinking of the same idea. :-) Only Python-level
operations would invoke the callback, and only if the callback exists.
Along with being more reliable and less to remember, I think it could
actually be faster than maintaining an independent length object like we
do now, since we often fall into the update() trap discussed earlier.
>>If it's there, the mutating operation could call length.changed
>>(deltaLength). Also, the __len__ slot could delegate to the length
>>object. That way, we don't have to wrap the BTree just to get a
>>minor (but nice) feature.
>
> A practical snag is that "a BTree" doesn't exist as such -- it's a
> (potentially very large) collection of BTree and Bucket nodes, and nothing
> at all distinguishes the root BTree node (which alone is "the BTree" from a
> program's POV, but that's an illusion, and mutations *actually* occur in
> Buckets, not in BTree nodes).
Yep. So what I have in mind is passing around an "int *deltaLength" in
BTree mutating functions. The mutators would modify *deltaLength as
they work, then at the end of the operation (involving many nodes), that
deltaLength (which exists only on the stack) would be given to some
other object. It's similar to the way BTrees used to work: they stored
a length in the root BTree node.
This assumes that for every BTree-mutating operation, there is a
function that gets called externally but never internally. The external
function is where we'd call length.change(). It's common for Python
extensions to separate external and internal functions, but I haven't
looked at whether the BTrees code uses that pattern. If it doesn't,
this idea is no good. ;-)
Shane
More information about the ZODB-Dev
mailing list