[ZODB-Dev] Feature Request 2381 : Persistent object iterator in Storage

[Barry A. Warsaw]

>>>>> "GW" == Greg Ward <gward@mems-exchange.org> writes:

    GW> ...which brings me to my next point: what am I supposed to
    GW> *do* with the module you posted?  Install it somewhere?  Just
    GW> read it?  Is it supposed to be abundantly clear from reading
    GW> that interface how I iterate over a FileStorage or
    GW> bsddb3Storage database?  (It isn't.)

Heh, I was just about to say "just read it", but I think it also helps
to have an example usage to ponder.  Look at BaseStorage's
copyTransactionsFrom() method.  This is the only use of the iterator
protocol in the field, AFAIK.  Below is the entirety of the script
that I use to copy a database.

    GW> Is ClientStorage going to support this interface someday?
    GW> Would it be easy to add, or would it require protocol
    GW> modifications?

I don't know.  Jeremy (when he gets back) is probably the best person
to answer this.

>>>>> "JDH" == John D Heintz <jheintz@isogen.com> writes:

    JDH> Is this API going to be added to BaseStorage someday? (That
    JDH> would handle ClientStorage as well as OracleStorage)

BaseStorage has copyTransactionsFrom() which is a consumer of this
interface (so all inheriting storages can populate themselves from an
iterator protocol supporting storage).  But it seems difficult to
implement the protocol in a generic enough way to include it in
BaseStorage.

    JDH> Now that I've looked more closely it seems to satify my
    JDH> initial request plus a lot more, great!

Cool!

    JDH> Question: If I am using this interface to make changes to the
    JDH> db/storage as I iterate, then I will need to make sure that I
    JDH> stop iterating when I get a RecordsIterator with tid greater
    JDH> than the timestamp when I started iterating?

I think so.  BaseStorage.copyTransactionsFrom() appears to have some
tests for transactions being out of order (but it assumes transaction
id's are valid timestamps -- not an assumption, I think, guaranteed by
the API).  The iterator protocol /is/ supposed to feed you
transactions in order from earliest to latest.

-Barry