[ZODB-Dev] repozo full backup and index files

[Hanno Schlichting]

On Fri, May 14, 2010 at 10:24 AM, Christian Theune <ct at gocept.com> wrote:
> Hmm. If the full backup is just a regular FS file then you could start
> with the naive approach and just open/close it once after performing a
> backup as that would create the index file.

Sure. That would be an easy but also rather inefficient way. In my
case it takes only about 15 minutes of rather heavy I/O to create the
file, but for larger files this time usually goes up.

>From what I understand repozo does the following:

1. it opens the real file storage file as a read-only FileStorage
2. calculates the byte position of end of the last complete transaction
3. closes the file storage
4. opens the fs file as a normal binary file
5. copies over all bytes up to the calculated position into a temp file
6. closes the fs file and temp file and renames the temp file
according to some timestamp

What I'm wondering is, if you could copy or otherwise create the index
as part of the second step. In step 5 we only deal with bytes and
cannot parse those, but in step 2 we have a proper ZODB FileStorage
object and can to some degree decide which transaction we want.

Hanno