[Checkins] [zopefoundation/ZODB] 2115c9: Avoid seeking on import (#188)

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/zopefoundation/ZODB
  Commit: 2115c90aa09844554dc90ed468d35fce1121be9e
      https://github.com/zopefoundation/ZODB/commit/2115c90aa09844554dc90ed468d35fce1121be9e
  Author: Jim Fulton <jim at jimfulton.info>
  Date:   2018-02-15 (Thu, 15 Feb 2018)

  Changed paths:
    M src/ZODB/ExportImport.py
    M src/ZODB/utils.py

  Log Message:
  -----------
  Avoid seeking on import (#188)

* Don't seek on imports

(other than the possible seek for custom importers)

We were seeking to handle blob markers. This has two major drawbacks:

1. It wasn't possible to use a non-seekable file.  A use case for
   export/import is to copy database data.  An intermediate file, and
   associated I/O, could be avoided using a pipe, but pipes aren't
   seekable.

2. Seeks cause file-buffer data to be discarded, making IO far more
   expensive.

We didn't really need blob markers, because the preceeding blob data
records serve as markers.  (Now we're stuck with them for backward
compatibility.)

* Make cp's buffer size larger and configurable.

* Use the storage temprary directory when importing blobs

To avoid an extra copy.

Also, allow the copy (cp) buffer sie to be overridden on export.

(I see no obvious way to plumb it on import. :( )

* Oops, need to use a binary literal (Python 3)

* Respond to PR comments