[Checkins] [zopefoundation/ZODB] 2115c9: Avoid seeking on import (#188)
GitHub
noreply at github.com
Fri Feb 16 00:28:51 CET 2018
Branch: refs/heads/master
Home: https://github.com/zopefoundation/ZODB
Commit: 2115c90aa09844554dc90ed468d35fce1121be9e
https://github.com/zopefoundation/ZODB/commit/2115c90aa09844554dc90ed468d35fce1121be9e
Author: Jim Fulton <jim at jimfulton.info>
Date: 2018-02-15 (Thu, 15 Feb 2018)
Changed paths:
M src/ZODB/ExportImport.py
M src/ZODB/utils.py
Log Message:
-----------
Avoid seeking on import (#188)
* Don't seek on imports
(other than the possible seek for custom importers)
We were seeking to handle blob markers. This has two major drawbacks:
1. It wasn't possible to use a non-seekable file. A use case for
export/import is to copy database data. An intermediate file, and
associated I/O, could be avoided using a pipe, but pipes aren't
seekable.
2. Seeks cause file-buffer data to be discarded, making IO far more
expensive.
We didn't really need blob markers, because the preceeding blob data
records serve as markers. (Now we're stuck with them for backward
compatibility.)
* Make cp's buffer size larger and configurable.
* Use the storage temprary directory when importing blobs
To avoid an extra copy.
Also, allow the copy (cp) buffer sie to be overridden on export.
(I see no obvious way to plumb it on import. :( )
* Oops, need to use a binary literal (Python 3)
* Respond to PR comments
More information about the checkins
mailing list