[ZODB-Dev] Re: BTrees q [Fwd: [Zope-dev] More Transience weirdness in 2.7.1b1]

[Chris McDonough]

On Wed, 2004-06-02 at 17:53, Casey Duncan wrote:
> This error occurs if the current offset in the current bucket in
> BTreeItems_seek is either negative or greater than the largest index in
> the bucket. The code assumes that this can only happen if the bucket was
> mutated between calls to BTreeItems_seek. Since you are just iterating
> it, no such change could be occuring (unless the reference is shared
> between threads somehow), so I suspect foul play.
> 
> It might be iteresting to instrument the BTreeItemsTemplate.c in the
> BTreeItems_seek() function to spew the values of i, currentoffset and
> currentbucket->len when it blows up. Then you could see whether it dies
> at the beginning, end or somewhere in the middle and what bucket and
> index it is on when it does.

I will try to do this.  Even if it turns out to be unhelpful, it can't
hurt...

> > FWIW, I have a high-concurrency test rig that exercises this code
> > which I have run for 24+ hours without bumping in to the same issue,
> > so I'm flying a bit blind here.
> 
> Does you test rig run multiple threads? Is ths BTree referenced from any
> non-persistent global data structure (which might allow sharing across
> threads)?

The test rig uses the Zope publisher via HTTP.  It kicks off many
threads to do simultaneous requests and on each request more-or-less
randomly chooses to either write to session data, read from session
data, raise an error, or start a new session.  So unless there's a bug
in Zope's publisher itself that might cause that to happen, it's
unlikely.  The sessioning machinery itself doesn't keep any copies of
instance data (in a global) that could be shared across
threads/connections.

> Interesting that restarting Zope cured this error. There must be some
> inconsistency in the cached state then. Calling BTree.keys() will give
> you a fresh BTreeItems object each time (with reset currentoffset and
> currentbucket values), so unless something is hosed in the BTree state
> itself, calling it again should start fresh.

It's not that interesting because it turns out that the data was in a
TemporaryStorage, so when it got shut down, the database essentially
vanished and needed to be reconstructed.  Steve is trying it now under
FileStorage.

- C