[ZODB-Dev] replicated ZEO idea

[Toby Dickenson]

On Sunday 15 Sep 2002 7:28 pm, sean.upton@uniontrib.com wrote:
> I like the idea of an iterator style transaction copier, using ZEO itse=
lf.
>
> Months back, I was thinking about doing something similar (but at a muc=
h
> higher level, using something like a deepcopy or the Zope import/export
> mechanism) using a dedicated ZEO client to do this kind of thing.  This
> seems like a much more appropriate way to do this.  Right now my likely
> approach for replication has swung the pendulum to the other end and I'=
m
> thinking of replicating the underlying storage, using Directory Storage
> snapshots 8) and find+cpio+nfs.  Something in-between seems a bit more
> graceful because of the ability to be storage agnostic.

The problem with DirectoryStorage+find+cpio is that 'find' needs to stat =
every=20
file in every directory. That needs disk I/O and cpu time proportional to=
=20
database size. That feels bad in principcal, but I guess in practice it m=
ight=20
not be a problem for a daily backup of your size storage.

Hmmmm, what about a script which is a ZEO client, and uses the iterator m=
ethod=20
to output the filenames of the DirectoryStorage files which have particip=
ated=20
in transactions committed since a timestamp. That could be used in place =
of=20
'find'. It would user cpu time and disk IO proportional to the amount of =
data=20
committed since the previous replication event.