[Grok-dev] Re: Iterativly emptying a grok.container shows weird
Philipp von Weitershausen
philipp at weitershausen.de
Tue Jun 12 11:12:28 EDT 2007
Brandon Craig Rhodes wrote:
>> Volker might like it more if the container had 100,000 items in it,
>> where iterating over the keys rather than getting them all back as a
>> list would be much faster.
> Nonsense. If Volker is unhappy getting a huge list from .keys(), he
> can call .iterkeys() instead. Please review the Python interface for
> mapping types; it says explicitly that .keys() returns:
> "a copy of a's list of keys"
> They key word there is "copy" - the caller has a right to expect that
> the sequence of keys he has been returned is a "copy", that is, an
> object that now stands independent of the dictionary-like object that
> generated it.
> Making the decision to turn .keys() into .iterkeys() unilaterally
> destroys the whole point of dictionary-like objects supporting both
> methods, and, as Volker rightly complains, it breaks many common
> Python idioms for working with dictionaries.
When the BTree mapping type was conceived, Python didn't know iterators
and there was no iterkeys() method in the mapping interface. BTrees
store large mappings that possibly contain very large objects
efficiently. They do that by storing each node of the search tree as a
separate object. That way, when you're accessing an item in the mapping,
it only has to deserialize the nodes along the search path, and it
doesn't have to deserialize no other object except for the one you're
getting from the mapping. Tests have shown that this is relevant once
the number of items in your mapping exceeds about 100 (which isn't that
So, it might appear as nonsense, but it's a combination of efficiency
* keys() returns an iterable and not a list because it would be very
inefficient to traverse the whole BTree to clobber up a list if you're
only going to batch over a fraction of them.
* It's not called iterkeys() because our BTree implementation in the
ZODB has been around for about 10 years.
http://worldcookery.com -- Professional Zope documentation and training
More information about the Grok-dev