[Zope-Coders] Re: BTrees, Unicode, and Python 2.2

[Andreas Jung]

Despite yesterdays discussion I unable to understand the following
behaviour;

The unittest fails with the following setUp() function:

    def setUp(self):
        """ setup an OOBTree with some unicode strings """

        self.s = unicode('dreit\xe4gigen','latin1')

        self.data = [('alien', 284708388),
                ('fox', 284708387),
                ('future', 284708388),
                ('k\xf6nnten', 284708389),
                ('quick', 284708387),
                ('zerst\xf6rt', 284708389),
                (unicode('dreit\xe4gigen','latin1'), 284708391)]

        self.tree = OOBTree()
        for k,v in self.data:   self.tree[k]=v

But it works with the following (last line of self.data changed):

    def setUp(self):
        """ setup an OOBTree with some unicode strings """

        self.s = unicode('dreit\xe4gigen','latin1')

        self.data = [('alien', 284708388),
                ('fox', 284708387),
                ('future', 284708388),
                ('k\xf6nnten', 284708389),
                ('quick', 284708387),
                ('zerst\xf6rt', 284708389),
                (self.s, 284708391)]

        self.tree = OOBTree()
        for k,v in self.data:   self.tree[k]=v

Any explanations for this behaviour ?

Andreas




----- Original Message -----
From: "Jeremy Hylton" <jeremy@zope.com>
To: <jim@zope.com>; <andreas@zope.com>
Cc: <zope-coders@zope.org>
Sent: Tuesday, October 16, 2001 15:43
Subject: BTrees, Unicode, and Python 2.2


> I'd like to see StandaloneZODB work with Python 2.1 and Python 2.2.
> I've been using Python 2.2 for development and it's worked fine.  Some
> recent test failures in testBTreesUnicode, though, suggests that we've
> got a 2.1 / 2.2 issue to straighten out.
>
> testBTreesUnicode fails on Python 2.2 with this error:
> ERROR: check every item of the tree
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "lib.linux-i686-2.2/BTrees/tests/testBTreesUnicode.py", line 21, in
setUp    for k,v in self.data:
> UnicodeError: ASCII decoding error: ordinal not in range(128)
>
> The specific error is caused by passing a Unicode key to an OOBTree.
> I can't tell what specifically is going wrong, but expect the error is
> raised in C inside PyObject_Compare().
>
> I see a different error with Python 2.1:
>
> ERROR: try to access unicode keys in tree
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "lib.linux-i686-2.1/BTrees/tests/testBTreesUnicode.py", line 36, in
test2    self.assertEqual(self.tree[self.s], self.data[-1][1])
> KeyError: <unprintable instance object>
>
> So the behavior of the BTrees seems to depend on something that has
> changed between Python versions.  I'm not really sure what we want the
> BTrees code to do, so I'm not sure how to go about fixing these
> errors.
>
> We want the tests to succeed with Python 2.1 and Python 2.2, but I'm
> not sure how to achieve that.
>
> Jeremy
>
>