[Zope3-dev] Potential bug of intid

[Jim Fulton]

Tadashi Matsumoto wrote:
> Hello
> 
> I was looking into the following code of intid's __init__.py.
> 
> 
>      91     def _generateId(self):
>      92         """Generate an id which is not yet taken.
>      93 
>      94         This tries to allocate sequential ids so they fall into the
>      95         same BTree bucket, and randomizes if it stumbles upon a
>      96         used one.
>      97         """
>      98         while True:
>      99             if self._v_nextid is None:
>     100                 self._v_nextid = random.randint(0, 2**31)
>     101             uid = self._v_nextid
>     102             self._v_nextid += 1
>     103             if uid not in self.refs:
>     104                 return uid
>     105             self._v_nextid = None
>     106 
>     107     def register(self, ob):
>     108         # Note that we'll still need to keep this proxy removal.
>     109         ob = removeSecurityProxy(ob)
>     110         key = IKeyReference(ob)
>     111 
>     112         if key in self.ids:
>     113             return self.ids[key]
>     114         uid = self._generateId()
>     115         self.refs[uid] = key
>     116         self.ids[key] = uid
>     117         return uid
>     118 
> 
> 
> 2**31 used at the line 100 is a long integer, and there is a possibility
> of generating long integer. long interger can not be used with Btree
> module.
> 
> In fact,
> 
> 
>>>>from BTrees import IOBTree
>>>>t=IOBTree.IOBTree()
>>>>t[2**30] = 'abc'
>>>>t[2**31] = 'efg'
> 
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: expected integer key

Good catch!


> I think it is better to use, for example, 2**30 instead of 2**31.

I think there are better strategies.  This stragey would throw away
half of the key space.

> By the way, there is a probability (less than 1/2**31 percent) of
> generating same intids, if thread switching occurs between the line
> 114 and 115.

Yup, although, as Gary points out, this will be caught later.

This could should really have used the BTree insert API to combine
the test and insert.  It's actually rather silly to have separated
_generateId into a separate method, as it is used only once.

As far as dealing with longs, a simple strategy would be to simply
catch TypeError when calling insert. If we get a TypeError, we should
pick a new random starting point.

Please report this problem in the collector.
http://www.zope.org/Collectors/Zope3-dev.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org