[Zope-DB] Zope database connectivity
dieter at handshake.de
Thu Sep 28 14:20:15 EDT 2006
Maciej Wisniowski wrote at 2006-9-27 22:40 +0200:
>> It can lead to nasty, apparently non-deterministic effects.
>> You may lose your database connection mid transaction
>> and part of what you think is a transaction (i.e. atomical) can be lost.
>What are the circumstances under which this can occur? I've found something
>similiar (?) when I was dealing with stored procedures in DCOracle2:
Several events must occur together such that this happens (that's
why the problem is not yet fixed).
While the ZODB does not promise any definite lifetime for
"_v_" variables (they may disappear at any time),
the current implementation calls cache garbage collection
only at savepoint and transaction boundaries.
During cache garbage collection objects may be deactivated
which causes their content (including "_v_" variables) to be deleted.
While the ZODB provides other means to deactivate
objects than cache garbage collection, there are rarely used
(especially on a DA object).
Thus, one condition is that savepoints (formerly "subtransaction"s)
However, some Zope operations, e.g. large cataloguing operations
use savepoints (in order to reduce memory footprint by
early cache garbage collection), without you beeing aware of.
There is not problem when the DA is deactivated when it was
not used in the same transaction. If however, it is use in this
transaction, then it is used rather recently. Cache garbage collection
first deactivates the least recently used objects.
A second condition is therefore, that your transaction accesses
huge numbers of objects (more than the cache size) since it accessed
the DA and then it performs a savepoint.
If these conditions are met, you will lose your connection. Still
nothing bad happens as the transaction still holds a reference to it
and may eventually commit/abort it at transaction end time.
If however, the same request accesses the DA again, a new connection
will be opened. This new connection will not see the effects
performed via the former connection and may deadlock on
access to resources locked by the former connection.
In these rare cases, you will have problems.
And as they are very rare and apparently non-deterministic,
they are almost impossible to analyse (and quite difficult to
>When you have a request that causes the new _v_database_connection to be
>created and this new connection is used to execute stored procedure which
>causes exception. Then... transaction is aborted and resource
>managers' abort() functions are called.
>I've found that newly created _v_database_connection was destroyed
>before(!) ResourceManager's (used to handle relational db connection)
>abort() function was called, so there was no rollback to database... huh...
This might be possible but need not be a problem:
The order in which operations occur at commit time can be controlled
via the resource manager's "sortKey" (or similarly spelled).
As the sort order between ZODB resources and relational
resources is not really relevant, it may well be
that the ZODB connection is aborted before the relational connection.
In this case, you may see the loss of the "_v_" variable
(caused either by the cache garbage collection or the invalidation
of the DA object) before the relational "abort".
However, the transaction holds an additional (beside the "_v_" attribute)
reference to the connection and can commit/abort it despite
the loss of the "_v_" attribute.
More information about the Zope-DB