[ZODB-Dev] Weird KeyError with OOBTree

[Laurence Rowe]

On 16 August 2010 13:13, Tres Seaver <tseaver at palladion.com> wrote:
> Hanno Schlichting wrote:
>> On Mon, Aug 16, 2010 at 12:14 PM, Pedro Ferreira
>> <jose.pedro.ferreira at cern.ch> wrote:
>>> Could this be some problem with using persistent objects as keys in a BTree?
>>> Some comparison problem?
>>
>> I'm not entirely sure about this, but I think using persistent objects
>> as keys isn't supported. Looking at the code, I doubt using anything
>> expect simple types like unicode strings or tuples of simple types
>> will work without further work.
>>
>>>From what I can see in the code, BTree's use functions like
>> PyObject_Compare to compare different keys. Persistent doesn't
>> implement any special compare function and falls back to the standard
>> hash algorithm for an object. This happens to be its memory address.
>> The memory address obviously changes over time and the same address
>> gets reused for different objects.
>>
>> I think implementing a stable hash function for your type could make
>> this work though.
>>
>> The ZODB gods correct me please :)
>
> Btrees require "comparability", rather than "hashability":  your
> persistent type needs to define a total ordering[1], which typically
> means defining '__cmp__' for your class.  You could also define just
> '__eq__' and '__lt__', but '__cmp__' is slightly more efficient.
>
>
> [1]http://www.zodb.org/documentation/guide/modules.html#total-ordering-and-persistence

While ZODB 3.8 makes it possible to use Persistent objects as keys in
a BTree, it's almost certainly a bad idea as a lookup will incur many
more object loads while traversing the BTree as the Persistent keys
will have to be loaded before they can be compared. Consider using one
of these alternatives instead:

* Set the IOTreeSet as an attribute directly on the persistent object.
* Use http://pypi.python.org/pypi/zope.intid and use the intid for the
key. (This uses http://pypi.python.org/pypi/zope.keyreference which
uses the referenced object's oid and database name to perform the
comparison, avoiding the need to load the persistent object.)

Laurence