[Zope-dev] Session Errors (read conflicts)

[Chris McDonough]

On Mon, 2003-03-17 at 20:34, John Eikenberry wrote:
> John Eikenberry wrote:
> 
> > Toby Dickenson wrote:
> > 
> > > Read conflicts occur if a change is committed in between the start of a 
> > > transaction, and the transaction needing to load the object. A workaround to 
> > > reduce the number of read conflicts is to touch the objects that are likely 
> > > to change early in the transaction.
> > 
> > Thanks for the tip. My sessions test script had a sleep() call right after
> > the transaction commit, thus making it right at the beginning of the next
> > transaction. Moving it to right before the commit (after all the session
> > tests) pretty much eliminated all the read conflict errors. Just get a
> > standard ConflictError occasionally now (not nearly as often).
> 
> This turned out to be better than I thought. After nearly 3 hours of
> constant abuse I have yet to see a ReadConflictError and have yet to get
> either the load() or get() KeyErrors, previously I could force one of these
> in a 2-3 minutes of abuse.
> 
> Our live sessions code uses the sessions about half to two-thirds of the
> way through the transaction. Given what can happen in that first half,
> there is easily plenty of time for read conflicts. I think I might be able
> to move our session use to the beginning of the transaction (just storing
> stuff in the REQUEST object for later usage) and hopefully fix our issue.

Interesting, and I'm very glad you got it working reliably.

But I'm not sure I understand why touching an object at the start of a
transaction would fix the consistency problem.  If a object A is read at
the beginning of a transaction that uses the data from object A during
write of object B and later object A changes and is invalidated by
another transaction (before the first finishes), no conflict errors will
be raised.  Isn't there the potential for the first transaction to write
inconsistent data into object B?  In my mind, this is the same thing as
using a "low conflict connection" (ie. you are essentially "turning off"
read conflicts, albeit by circumstance rather than by code).

I guess the question boils down to: why *don't* you see consistency
problems now?  I would expect to continue to see the "get" errors you
reported earlier. 

I think this exact problem is being discussed on ZODB-Dev under the
title "Database Read Conflicts -- neither safe nor robust".

> The KeyErrors happen under similar circumstances to the ReadConflictErrors.
> The significant difference being that the KeyErrors happen after the
> transience timeout has occured. When I am running with the
> LowConflictConnection disabled the KeyErrors occur in the
> Connection.setstate method at line 509, before the ReadConflictError check.

Is this the same KeyError you reported as coming out of
TemporaryStorage.load?

> So say you have 2 threads; A and B.  If A starts first, hits the session
> code and triggers a _housekeep() call during which time B has started but
> has not reached the Sessions code until after _housekeep() has finished.
> When it does reach the sessions code, you get the KeyErrors. 

Would you mind restating that?  I think this is important, and I'm not
sure I understand the second sentence above.

Thanks!

- C