[ZODB-Dev] Weird ConflictErrors with Zope 2.8.4 - It's been a while ; -)

[Dieter Maurer]

Chris Withers wrote at 2006-1-24 13:41 +0000:
>Okay, now that we have 2.8.4 in place, we get proper reporting of 
>ConflictErrors and today we started seeing one happening over and over 
>again which looked roughly as follows:
>
>Traceback (most recent call last):
>   File "lib/python/Products/Transience/Transience.py", line 844, in 
>new_or_existing
>     item = self.get(key, _marker)
>   File "lib/python/Products/Transience/Transience.py", line 491, in get
>     item = self._move_item(k, current_ts, default)
>   File "lib/python/Products/Transience/Transience.py", line 346, in 
>_move_item
>     if getattr(self._data[current_ts][k], 'setLastAccessed', None):
>   File "lib/python/ZODB/Connection.py", line 704, in setstate
>     self._setstate(obj)
> ...
>     return self.rpc.call("loadEx", oid, version)
>   File "lib/python/ZEO/zrpc/connection.py", line 536, in call
>     raise inst # error raised by server
>ConflictError: database conflict error (oid 0x010f0d)

We see that this is a ReadConflictError on access to
"self._data[current_ts][k]".
We do not know whether "self._data", "self._data[current_ts]" or
"self._data[current_ts][k]" makes the problem.

>The things which got me worried is that it was always the same oid that 
>was conflicting

Much more worrying for me is that the exception comes from ZEO:

  ZEO should *NEVER* raise "ReadConflictError" exceptions!


Okay, I see: "TemporaryStorage" raises "ConflictError"
when other storages would raise "POSKeyError": something
tries to access an "oid" that was recently garbage collected (i.e.
deleted).

What you see may be a result of "tempstorage" not implementing
faithfull MVCC...


However, when the client gets the "ConflictError", it should
also have received the invalidation message for the object
with the broken reference. The request retrial (caused by
the "Conflict") should therefore not repeat the "ConflictError".

> ...
>Now, I wanted to find out what the troublesome object behind 0x010f0d 
>was, so I fired up and debug zope client, and was distrubed by the result:
>
> >>> app.temp_folder._p_jar[p64(0x010f0d)]
>Traceback (most recent call last):
> ...
>     raise inst # error raised by server
>ZODB.POSException.ConflictError: database conflict error (oid 0x010f0d)

The object with this OID has been deleted; thus, this will
stay until you get a "KeyError" instead (when
"_recently_gc_oids" no longer contains it).

> ...
>However, even now, an hour or so later, if I fire up a debug client and 
>try and look up that oid, I get a ConflictError. Any idea what's going on?

See above.

You will never again be able to load this oid without exception
(as it was deleted).

-- 
Dieter