[Checkins] [zopefoundation/ZODB] 2ed12f: Fix possible data corruption after FileStorage is ...
GitHub
noreply at github.com
Fri Apr 29 00:20:11 CEST 2016
Branch: refs/heads/3.10
Home: https://github.com/zopefoundation/ZODB
Commit: 2ed12f35d26cef06dda56cd6a83adf601ead2aae
https://github.com/zopefoundation/ZODB/commit/2ed12f35d26cef06dda56cd6a83adf601ead2aae
Author: Julien Muchembled <jm at nexedi.com>
Date: 2016-04-28 (Thu, 28 Apr 2016)
Changed paths:
M src/ZODB/FileStorage/FileStorage.py
M src/ZODB/tests/testFileStorage.py
Log Message:
-----------
Fix possible data corruption after FileStorage is truncated to roll back a transaction
Multi-threaded IO support, which is new to ZODB 3.10, allows clients to read
data (load & loadBefore) even after tpc_vote has started to write a new
transaction to disk. This is done by using different 'file' objects.
Issues start when a transaction is rolled back after data has been appended
(using the writing file object). Truncating is not enough because the FilePool
may have been used concurrently to read the end of the last transaction:
file objects have their own read buffers which, in this case, may also contain
the beginning of the aborted transaction.
So a solution is to invalidate read buffers whenever they may contain wrong
data. This patch does it on truncation, which happens rarely enough to not
affect performance.
We discovered this bug in the following conditions:
- ZODB splitted in several FileStorage
- many conflicts in the first committed DB, but always resolved
- unresolved conflict in another DB
If the transaction is replayed with success (no more conflict in the other DB),
a subsequent load of the object that could be resolved in the first DB may, for
example, return a wrong serial (tid of the aborted transaction) if the layout
of the committed transaction matches that of the aborted one.
The bug usually manifests with POSKeyError & CorruptedDataError exceptions in
ZEO logs, for example while trying to resolve a conflict (and restarting the
transaction does not help, causing Site Errors in Zope). But theorically,
this could also cause silent corruption or unpickling errors at client side.
(cherry picked from commit 028b1922090cb831d8fc0e0ebdfc1d00574f0e40)
Conflicts:
src/ZODB/FileStorage/FileStorage.py
Commit: 64f3b58f843cc73e5b0cebd2f3089b592d78c914
https://github.com/zopefoundation/ZODB/commit/64f3b58f843cc73e5b0cebd2f3089b592d78c914
Author: Julien Muchembled <jm at nexedi.com>
Date: 2016-04-28 (Thu, 28 Apr 2016)
Changed paths:
M src/CHANGES.txt
Log Message:
-----------
Changelog for PR #52
Compare: https://github.com/zopefoundation/ZODB/compare/b9887679b3da...64f3b58f843c
More information about the checkins
mailing list