[ZODB-Dev] Database Read Conflicts -- neither safe nor robust

[Dieter Maurer]

The ZODB tries to implement serialized transactions through
detection of read conflicts.

When I am right, then the algorithm is:

  A committing transaction informs all connections about
  modified objects. Receiving a modification notice,
  the connection invalidates the object.
  When a connection loads an invalidated object from ZODB,
  then it raises a ReadConflictError.

This is neither safe nor robust.

To see why it is not safe consider transactions T1 and T2
and object O. O is written by T1 and read by T2.
Now consider the following event sequences

  1.  T1 and T2 start
      T1 writes O and commits
      T2 reads O -- ReadConflictError

  2.  T1 and T2 start
      T2 reads O (and makes decisions based on the read value)
      T1 writes O and commits
      T2 commits

With respect to serializability both sequences are equivalent
but they are handled differently by ZODB. In fact, 2 may
violate serializability.


Why is it not robust?
Assume, you have a writer and a reader with the reader just
interested to read consistent data (a typical Zope application
displaying search results, for example) running in parallel.
Then the reader is plagued with ReadConflictErrors.

While we have quite a good way to handle write conflicts
(application level conflict resolution), the mechanism
to tackle read conflicts ("_p_independent") seems to be too weak
and therefore not widely supported by Zope's data structures
(such a BTrees).


How to make it safer?
To get true serializability, we need to make a freshness check for
read data at transaction commit time. For this, read objects would register
with the transaction and be checked at commit.


How to make it more robust?
Versioned storages could implement serialization level "READ_COMMITTED".
For this, the transaction would register its start time.
When an invalid object should be read from ZODB, then the
object's state valid at the transactions start time would be read.
Storages unable to implement this would raise a ReadConflictError.


Dieter