[ZODB-Dev] Storage iterator() method

[Toby Dickenson]

On Thursday 25 Jul 2002 4:25 pm, Barry A. Warsaw wrote:
> >>>>> "JH" =3D=3D Jeremy Hylton <jeremy@zope.com> writes:
>
>     JH> I don't think it was designed for that purpose, but I think
>     JH> the implementations in berkeley storage and file storage
>     JH> should work well enough.  FileStorage implements the iterator
>     JH> by opening a new Python file object (read-only) on the Data.fs
>     JH> and reading that.  The only potential gotcha, and I'd have to
>     JH> check the code to be sure it exists, is that it could read a
>     JH> partial transaction if another thread is in the midst of a
>     JH> commit when the next record is pulled off the iterator; if it
>     JH> did, I expect it would raise an exception.

It might be nice to use something similar in a 'tail -f' manner, particul=
arly=20
for implementing a replication capability, but Im sure it is asking too m=
uch=20
to ask the storage to provide that.

I can get consistent behaviour between FileStorage and Berkeley/Full by=20
passing an upper-bound tid/timestamp that forces iteration to stop before=
 it=20
reaches the problem area.

> Other potential gotchas are transaction destructive operations such as
> non-transactional undo

does anyone use that?

> and pack.
>
> A pack operation in the middle of an iteration could pose problems,
> although off-hand I'm not sure what would happen for FileStorage.

FileStorage doesnt modify its original file during packing, so I suspect =
this=20
should all be happy. =20

>  For
> Berkeley Full storage, if the transaction that the iterator currently
> pointed to got packed away, you'll probably get a KeyError which the
> internal iterator object transforms into an IndexError, so iteration
> would stop (early) at that point.

Eeek.