[ZODB-Dev] RE: [Zope-Annce] ZODB 3.2.4 release candidate 1released

[Tim Peters]

[Chris McDonough]
> ReadConflictErrors are inevitable but also unpredictable.  Any attribute
> access on a persistent object can lead to a ReadConflictError so the
> error can potentially get thrown any time you're accessing persistent
> state.  What looks like a simple attribute access can have (in your
> words) "enormously complex side effects", ReadConflictError being one.
> Unlike other kinds of errors, they are raised at times that are literally
> impossible to predict (as they depend on the state of other threads and
> processes), so they influence the code path in particularly wacky ways.
> As such, it's highly unlikely that the developer can anticipate the set
> of trailing statements that remain unexecuted after the system encounters
> a ReadConflictError.

ReadConflictError prevents the transaction from committing, so the partially
unknown in-memory changes to persistent objects don't matter -- they stay in
memory, and get thrown away when a commit() attempt is finally made.  I
understand that persistent changes outside ZODB's persistence system may
also get made based on faulty understanding in such cases, but nobody has a
scheme even pretending to get anywhere near that in the presence of overly
embracing exception suppression.

> However, "more obvious" operations, which tend to raise "more normal"
> errors (like AttributeError or KeyError or whatnot) are typically
> anticipated and dealt with by the developer, because it's more obvious
> *when* they will happen.   A developer is never expected to catch
> ReadConflictError but he is almost certainly expected to catch a KeyError
> and deal with it appropriately if it's possible that it will get raised
> in the normal course of operations.  "Appropriately" may mean attempting
> to bubble the error up to the publisher (in which case, to be really sure
> that nothing wacky gets committed to the database due to a suppressed
> exception, the developer should mark the current transaction as
> uncommittable, but of course no one except me seems to even want to think
> about doing that now ;-), or it may mean just executing a different
> codepath and continuing on.  In either case, the developer has some
> influence over the codepath and thus the "trailing unexecuted code".

Maybe, but those aren't the kinds of examples Florent or Dieter described in
their msgs.  They're showing me bare excepts that suppress everything.

> This way of thinking about persistent state inconsistencies is very
> helpful, BTW.  It's awfully clear just how fragile application data state
> consistency is!  I'm constantly amazed this thing works at all. ;-)

Ya -- I'd like to see you prove that it does <wink>.

>> Even then, ReadConflictErrors are less harmful than other exceptions,
>> because suppressing one of those leaves the transaction in an
>> uncommittable state despite the suppression.  But a suppressed
>> AttributeError (etc) has no effect on whether commit() will succeed.

> In this case it's up to recognize

Missing word.  Up to Jeremy!  It's his fault then.

BTW, which case is "this case"?  This is the "normal errors" case?

> that the state is inconsistent and ensure that the transaction does not
> commit by himself.
>
> It's a bit easier to do this for "normal" errors than it is for
> ReadConflictErrors if only because they occur more predictably.

I'm not following.  You don't have to do anything to make a
ReadConflictError stop commit() from succeeding.  How are normal errors
easier than that?

> The only way to prevent the transaction from committing in the current
> crop of Zopes is registering a TM that raises during vote.

Or to have experienced a ReadConflictError.

> Exceptions that are not caught by bare excepts would be another
> mechanism for doing this.

If that's truly a solution, then another solution could be derived from it
by banning bare excepts, and banning the catching of the specific exceptions
you wish a bare except wouldn't catch.  That could be done today.  I don't
expect there will ever be an exception that bare except won't catch (but
wouldn't be surprised if bare except got deprecated).