[Zope] Zope 2.8 and conflict errors

Maciej Wisniowski maciej.wisniowski at coig.katowice.pl
Mon Aug 1 06:21:24 EDT 2005


Hi!

Does anybody tried 2.8 and can say something about conflict errors?

I've done few tests and results are not good.

We've checked one of our applications on Zope 2.7.2 and the same
application on Zope 2.8. We have simulated 50-65 users executing the
same path in application. There was total count 1500 pages visited
during test.

With Zope 2.7.2 we've received about 39 internal server errors
HTTP/1.1 500.

Errors in Zope 2.7.2 are mainly ReadConflictErrors with
DCOracle2.DA.Procedure like:

------
2005-08-01T09:54:08 INFO(0) TM:65540 Failed to abort object
Traceback (most recent call last):
  File "/opt/Zope/2.7.2/lib/python/ZODB/Transaction.py", line 393, in 
_commit_error
    j.abort(o, self)
  File "/opt/Zope/2.7.2/lib/python/ZODB/Connection.py", line 561, in 
setstate
    invalid = self._is_invalidated(obj)
  File "/opt/Zope/2.7.2/lib/python/ZODB/Connection.py", line 590, in 
_is_invalidated
    raise ReadConflictError(object=obj)
ReadConflictError: database read conflict error (oid 00000000000299de, 
class Products.DCOracle2.DA.Procedure)
------

Other errors are like "Shouldn't load state xxx when
connection is closed" but these are known bugs in older ZODB version.


Zope 2.8 supports MVCC so we hoped that the number of
errors decrease but... with Zope 2.8 we get about 129 internal
server errors HTTP/1.1 500 and additionally a lot of 402 errors 
(unauthorized).

Errors 402 are because of internal server errors on login page
(testing application continues test without checking if user has been
properly logged in).

In the event.log there are no conflict errors saved :-/ ,
only unauthorized. We get the error from client responses
and from console.

I don't know what happens. May it be because of using old
data.fs file (from 2.7.2) with Zope 2.8?

Anybody has similiar experiences? Is Zope 2.8 useable??


Below few errors from the console (runzope) and from user browser:

----------------------------------------------------------------
2005-08-01 10:46:22 INFO ZODB conflict error at /crProd/acl_users/login 
(16 conflicts since startup at 2005-08-01T10:45:58)
2005-08-01 10:46:22 BLATHER Conflict traceback
Traceback (most recent call last):
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in 
publish
    request, bind=1)
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply
    if debug is not None: return debug(object,args,context)
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in 
call_object
    result=apply(object,args) # Type s<cr> to step into published object.
  File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 
311, in __call__
    return self._bindAndExec(args, kw, None)
  File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 
348, in _bindAndExec
    return self._exec(bound_data, args, kw)
  File 
"/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", 
line 323, in _exec
    result = f(*args, **kw)
  File "Script (Python)", line 3, in login
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 
1214, in __getattr__
    v = self.get(key, default, returnTaints=returnTaints)
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 
1174, in get
    if callable(v): v = v()
  File 
"/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", 
line 93, in getSessionData
    return self._getSessionDataObject(key)
  File 
"/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", 
line 180, in _getSessionDataObject
    ob = container.new_or_existing(key)
  File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
line 844, in new_or_existing
    item = self.get(key, _marker)
  File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
line 491, in get
    item = self._move_item(k, current_ts, default)
  File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
line 297, in _move_item
    abucket = self._data.get(ts, None) # XXX ReadConflictError hotspot
  File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in 
setstate
    self._setstate(obj)
  File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in 
_setstate
    self._load_before_or_conflict(obj)
  File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in 
_load_before_or_conflict
    raise ReadConflictError(object=obj)
ReadConflictError: database read conflict error (oid 0x0a, class 
BTrees._IOBTree.IOBTree)
----------------------------------------------------------------

error in /crProd/acl_users/login - is with request.SESSION statement.

----------------------------------------------------------------
2005-08-01 10:46:21 BLATHER Conflict traceback
Traceback (most recent call last):
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in 
publish
    request, bind=1)
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply
    if debug is not None: return debug(object,args,context)
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in 
call_object
    result=apply(object,args) # Type s<cr> to step into published object.
  File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 
311, in __call__
    return self._bindAndExec(args, kw, None)
  File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 
348, in _bindAndExec
    return self._exec(bound_data, args, kw)
  File 
"/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", 
line 323, in _exec
    result = f(*args, **kw)
  File "Script (Python)", line 3, in login
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 
1214, in __getattr__
    v = self.get(key, default, returnTaints=returnTaints)
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 
1174, in get
    if callable(v): v = v()
  File 
"/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", 
line 93, in getSessionData
    return self._getSessionDataObject(key)
  File 
"/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", 
line 180, in _getSessionDataObject
    ob = container.new_or_existing(key)
  File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
line 844, in new_or_existing
    item = self.get(key, _marker)
  File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
line 491, in get
    item = self._move_item(k, current_ts, default)
  File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
line 280, in _move_item
    self._housekeep(current_ts)
  File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
line 939, in _housekeep
    self._finalize(now)
  File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
line 547, in _finalize
    last_finalized = self._last_finalized_timeslice()
  File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
line 1076, in __call__
    return self.value
  File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in 
setstate
    self._setstate(obj)
  File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in 
_setstate
    self._load_before_or_conflict(obj)
  File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in 
_load_before_or_conflict
    raise ReadConflictError(object=obj)
ReadConflictError: database read conflict error (oid 0x09, class 
Products.Transience.Transience.Increaser)
-----------------------------------------------------------

or

-----------------------------------------------------------
2005-08-01 10:46:21 BLATHER Conflict traceback
Traceback (most recent call last):
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in 
publish
    request, bind=1)
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply
    if debug is not None: return debug(object,args,context)
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in 
call_object
    result=apply(object,args) # Type s<cr> to step into published object.
  File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 
311, in __call__
    return self._bindAndExec(args, kw, None)
  File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 
348, in _bindAndExec
    return self._exec(bound_data, args, kw)
  File 
"/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", 
line 323, in _exec
    result = f(*args, **kw)
  File "Script (Python)", line 3, in login
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 
1214, in __getattr__
    v = self.get(key, default, returnTaints=returnTaints)
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 
1174, in get
    if callable(v): v = v()
  File 
"/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", 
line 93, in getSessionData
    return self._getSessionDataObject(key)
  File 
"/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", 
line 180, in _getSessionDataObject
    ob = container.new_or_existing(key)
  File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
line 847, in new_or_existing
    self[key] = item
  File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
line 435, in __setitem__
    length = self._length() # XXX ReadConflictError hotspot
  File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
line 1119, in __call__
    return self.value
  File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in 
setstate
    self._setstate(obj)
  File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in 
_setstate
    self._load_before_or_conflict(obj)
  File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in 
_load_before_or_conflict
    raise ReadConflictError(object=obj)
ReadConflictError: database read conflict error (oid 0x08, class 
Products.Transience.Transience.Length2)
-----------------------------------------------------------

or

-----------------------------------------------------------
2005-08-01 10:46:21 BLATHER Conflict traceback
Traceback (most recent call last):
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in 
publish
    request, bind=1)
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply
    if debug is not None: return debug(object,args,context)
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in 
call_object
    result=apply(object,args) # Type s<cr> to step into published object.
  File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 
311, in __call__
    return self._bindAndExec(args, kw, None)
  File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 
348, in _bindAndExec
    return self._exec(bound_data, args, kw)
  File 
"/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", 
line 323, in _exec
    result = f(*args, **kw)
  File "Script (Python)", line 3, in login
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 
1214, in __getattr__
    v = self.get(key, default, returnTaints=returnTaints)
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 
1174, in get
    if callable(v): v = v()
  File 
"/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", 
line 93, in getSessionData
    return self._getSessionDataObject(key)
  File 
"/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", 
line 180, in _getSessionDataObject
    ob = container.new_or_existing(key)
  File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
line 844, in new_or_existing
    item = self.get(key, _marker)
  File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
line 491, in get
    item = self._move_item(k, current_ts, default)
  File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", 
line 297, in _move_item
    abucket = self._data.get(ts, None) # XXX ReadConflictError hotspot
  File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in 
setstate
    self._setstate(obj)
  File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in 
_setstate
    self._load_before_or_conflict(obj)
  File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in 
_load_before_or_conflict
    raise ReadConflictError(object=obj)
ReadConflictError: database read conflict error (oid 0x0a, class 
BTrees._IOBTree.IOBTree)
-----------------------------------------------------------

or

-----------------------------------------------------------
2005-08-01 10:47:55 INFO ZODB conflict error at 
/crProd/cr/centralne/procName (87 conflicts since startup at 
2005-08-01T10:45:58)
2005-08-01 10:47:55 BLATHER Conflict traceback
Traceback (most recent call last):
  File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 119, in 
publish
    transactions_manager.commit()
  File "/opt/Zope/2.8.0/lib/python/Zope2/App/startup.py", line 215, in 
commit
    transaction.commit()
  File "/opt/Zope/2.8.0/lib/python/transaction/_manager.py", line 84, in 
commit
    self.get().commit(sub)
  File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line 
382, in commit
    self._saveCommitishError() # This raises!
  File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line 
380, in commit
    self._commitResources()
  File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line 
428, in _commitResources
    rm.tpc_vote(self)
  File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 637, in 
tpc_vote
    s = vote(transaction)
  File "/opt/Zope/2.8.0/lib/python/ZEO/ClientStorage.py", line 893, in 
tpc_vote
    return self._check_serials()
  File "/opt/Zope/2.8.0/lib/python/ZEO/ClientStorage.py", line 877, in 
_check_serials
    raise s
ConflictError: database conflict error (oid 0x0299de, class 
Products.DCOracle2.DA.Procedure, serial this txn started with 
0x035f0726c7bfca88 2005-08-01 08:38:46.816215, serial currently 
committed 0x035f072fed78ebee 2005-08-01 08:47:55.657582)
-----------------------------------------------------------

User receives errors like:
-----------------------------------------------------------------
Response:
HTTP/1.1 500 Internal Server Error
Server: Zope/(Zope 2.8.0-final, python 2.3.5, linux2) ZServer/1.1
Date: Fri, 29 Jul 2005 12:53:00 GMT
Bobo-Exception-Line: 877
Content-Length: 2020
Bobo-Exception-Value: See the server error log for details
Content-Language: pl
Bobo-Exception-File: ClientStorage.py
Bobo-Exception-Type: ZODB.POSException.ConflictError

(...)

<TD WIDTH="90%">
  <H2>Site Error</H2>
  <P>An error was encountered while publishing this resource.
  </P>
  <P><STRONG>ZODB.POSException.ConflictError</STRONG></P>
  Sorry, a site error occurred.<p><p>Traceback (innermost last):
<ul>

Module ZPublisher.Publish, line 187, in publish_module_standard
<li>  Module ZPublisher.Publish, line 161, in publish</li>
<li>  Module ZPublisher.Publish, line 161, in publish</li>
<li>  Module ZPublisher.Publish, line 161, in publish</li>
<li>  Module ZPublisher.Publish, line 151, in publish</li>
<li>  Module Zope2.App.startup, line 158, in zpublisher_exception_hook</li>
<li>  Module ZPublisher.Publish, line 119, in publish</li>
<li>  Module Zope2.App.startup, line 215, in commit</li>
<li>  Module transaction._manager, line 84, in commit</li>
<li>  Module transaction._transaction, line 382, in commit</li>
<li>  Module transaction._transaction, line 380, in commit</li>
<li>  Module transaction._transaction, line 428, in _commitResources</li>
<li>  Module ZODB.Connection, line 637, in tpc_vote</li>
<li>  Module ZEO.ClientStorage, line 893, in tpc_vote</li>
<li>  Module ZEO.ClientStorage, line 877, in _check_serials</li>
</ul>ConflictError: database conflict error (oid 0x0299de, class 
Products.DCOracle2.DA.Procedure, serial this txn started with 
0x035ef744f3d9b366 2005-07-29 12:52:57.152436, serial currently 
committed 0x035ef744fe4886cc 2005-07-29 12:52:59.597650)
</p>
  <HR NOSHADE>
  <P>Troubleshooting Suggestions</P>
(...)
-----------------------------------------------------------------


-- 
Maciej Wisniowski



More information about the Zope mailing list