[ZODB-Dev] Storage iterator() method

[Barry A. Warsaw]

>>>>> "JH" == Jeremy Hylton <jeremy@zope.com> writes:

    JH> I don't think it was designed for that purpose, but I think
    JH> the implementations in berkeley storage and file storage
    JH> should work well enough.  FileStorage implements the iterator
    JH> by opening a new Python file object (read-only) on the Data.fs
    JH> and reading that.  The only potential gotcha, and I'd have to
    JH> check the code to be sure it exists, is that it could read a
    JH> partial transaction if another thread is in the midst of a
    JH> commit when the next record is pulled off the iterator; if it
    JH> did, I expect it would raise an exception.

    JH> Berkeley should be even safer, since it is just reading
    JH> transactions out of the transaction log.  The storage uses
    JH> berkeley transactions for storage transactions, so clients
    JH> using an iterator should get ACID semantics.

Other potential gotchas are transaction destructive operations such as
non-transactional undo and pack.

A pack operation in the middle of an iteration could pose problems,
although off-hand I'm not sure what would happen for FileStorage.  For
Berkeley Full storage, if the transaction that the iterator currently
pointed to got packed away, you'll probably get a KeyError which the
internal iterator object transforms into an IndexError, so iteration
would stop (early) at that point.

Berkeley Full shouldn't be affected by non-transactional undo because
it doesn't support it. ;)  If one thread undo'd while another is
iterating, the iterator should just happily spew out the new
transactions when it gets to them.

-Barry