[ZODB-Dev] [Persistent] STICKY mechanism unsafe

[Dieter Maurer]

We currently see occational SIGSEGVs in "BTrees/BucketTemplate.c:_bucket_get".
I am not yet sure but it looks as if the object had been deactivated
during the "BUCKET_SEARCH".

Trying to analyse the problem, I had a close look at the STICKY
mechanism of "persistent.Persistent" which should prevent
accidental deactivation -- and found it unsafe.


"STICKY" is one of the states on persistent objects -- beside
"GHOST", "UPTODATE" and "CHANGED". It is mostly equivalent to
"UPTODATE" but prevents deactivation (but not invalidation).

It is used by C extensions that may release the GIL or call back
to Python (which may indirectly release the GIL).
Their typical usage pattern is

      if (obj->state == GHOST) obj->unghostify();
      if (obj->state == UPTODATE) obj->state = STICKY;
      ... do whatever needs to be done with "obj" ...
      if (obj->state == STICKY) obj->state = UPTODATE

This usage pattern obviously breaks when a similar
code sequence is executed for "obj" while "... do whatever .."
is executed as it resets "STICKY" too early (in the "nested" code
sequence rather than the original one).

This may happen in several ways:

 1. "... do whatever ..." does it explicitly

 2. "obj" is accessed from a different thread

 3. "obj" is accessed from a Python callback

1. might be considered a bug in "... do whatever ..." -- although one
that is not easily avoidable.

2. is a general problem. Not only "STICKY" is unsafe against concurrent
use -- the complete state model of "Persistent" is.
We might explicitly state that the concurrent use of persistent objects
is unsafe and check against it.

With respect to STICKY, all three cases can be detected
by prepending "if (obj->state == STICKY) ERROR;".

1. and 3. (but obviously not 2.) could be handled by
implementing "STICKY" not by a bit but by a counter.



-- 
Dieter