[Zope-Checkins] CVS: ZODB/src/ZODB/FileStorage - fspack.py:1.12
jeremy at zope.com
Mon Mar 29 21:43:34 EST 2004
Update of /cvs-repository/ZODB/src/ZODB/FileStorage
In directory cvs.zope.org:/tmp/cvs-serv23751
Add presumed speed optimization for pack().
Only open the file for unbuffered I/O after finishing the first phase
of pack. The first phase gets its end-of-file position from the main
thread, so there's no possibility of reading a 'c' record.
Timings on Linux are inconclusive, but it seems like using buffered
I/O for the initial phase should be faster.
=== ZODB/src/ZODB/FileStorage/fspack.py 1.11 => 1.12 ===
--- ZODB/src/ZODB/FileStorage/fspack.py:1.11 Thu Mar 18 18:41:47 2004
+++ ZODB/src/ZODB/FileStorage/fspack.py Mon Mar 29 21:43:25 2004
@@ -416,14 +416,8 @@
# progress after it).
def __init__(self, path, stop, la, lr, cla, clr, current_size):
self._name = path
- # Caution: It's critical that the file be opened in unbuffered mode.
- # The code used to leave off the trailing 0 argument, and then on
- # every platform except native Windows it was observed that we could
- # read stale data from the tail end of the file -- keep in mind that
- # transactions can still be in progress throughout much of packing,
- # and are written to the same physical file but via a distinct Python
- # file object.
- self._file = open(path, "rb", 0)
+ self._file = open(path, "rb")
+ self._path = path
self._stop = stop
self.locked = 0
self.file_end = current_size
@@ -493,6 +487,19 @@
self.locked = 1
+ # Re-open the file in unbuffered mode.
+ # The main thread may write new transactions to the file,
+ # which creates the possibility that we will read a status
+ # 'c' transaction into the pack thread's stdio buffer even
+ # though we're acquiring the commit lock. Transactions
+ # can still be in progress throughout much of packing, and
+ # are written to the same physical file but via a distinct
+ # Python file object. The code used to leave off the
+ # trailing 0 argument, and then on every platform except
+ # native Windows it was observed that we could read stale
+ # data from the tail end of the file.
+ self._file = open(self._path, "rb", 0)
self.file_end = self._file.tell()
More information about the Zope-Checkins