[ZODB-Dev] BTrees q [Fwd: [Zope-dev] More Transience weirdness in2.7.1b1]

[Tim Peters]

[Chris McDonough]
...

>> ... Zope-2.7.1b1 (Python 2.3.3, Fedora Core 1) ...
...
>>    Traceback (innermost last):
...
>>      Module Products.Transience.Transience, line 555, in _gc
>>    RuntimeError: the bucket being iterated changed size

...

> I am wondering how the "RuntimeError:  bucket being iterated over" error
> could get raised given that

...

> Line 555 is the line:
>
>    "for key in list(self._data.keys(None, max_ts)):"

Assuming list() is the builtin list(), it seems it could only happen if the
BTree changed size *while* list() was running.  IOW, the explicit "for" loop
looks irrelevant to me, it's the implicit loop inside the implementation of
list() that must be complaining.  But I don't see how that's plausibly
possible.

Of what data type are the keys of this BTree (I'm assuming self._data is
some flavor of BTree)?

> It doesn't appear that I am mutating the state of the _data btree when
> iterating over its keys, values or items.

It shouldn't make any difference what you're doing in the loop guts, because
the 'for' loop is iterating over a Python list (or so it appears), not over
a BTree.

> If I'm wrong, I trust you'll let me know. ;-)

Sorry, it's mysterious to me too!

> Guesses:  Could the BTreeItems_seek code raise this error if the data
> structure was being accessed by another thread via a different database
> connection simultaneously?

That shouldn't matter.

> Could a suppressed conflict error at a past transaction commit time
> leave the database in such a state where a subsequent run through this
> code could wind up in this state?

Under theory that the exception is getting raised during the list() call, I
can't think of anything that account for it.  My head is inching toward
compiler optimization bugs, or a flaky memory chip (etc).

> FWIW, I have a high-concurrency test rig that exercises this code which I
> have run for 24+ hours without bumping in to the same issue, so I'm
> flying a bit blind here.

Not as blind as I am, pal -- no clue.  I'll try to look more later (buried
under other stuff right now).