[ZODB-Dev] Replicating data over disconnected servers.

[Michel Pelletier]

On Fri, 25 May 2001, Christian Robottom Reis wrote:

> On Thu, 24 May 2001, Tim Cook wrote:
>
> > May I guess that an export/import solution doesn't work for your
> > situation?
> > Or are you attempting something more dynamic?
>
> You mean a full export/import of the whole database? Hmmmm. Well, the
> problem is mainly that there is data (objects) on side A and on side B,
> and only part of that data needs to be updated.

Why not just run a ZEO client on side A and B connected to the same
database?  That would replicate your objects.  It sounds like you want to
sync two ZEO *servers*.  There is a project on this already:

http://dev.zope.org/Wikis/DevSite/Projects/ZEOReplicatedStorage/FrontPage

> There are semantics on
> these updates that I have to preserve, and that's why I'm wondering what
> the right solution is. That, and me not understanding very well what ZODB
> does with the actual id()s of the objects it's restoring -- I must assume
> there's some reference rewriting done on-the-fly, but Jim hasn't said so
> yet.

I can't speak for Jim, but I wouldn't want all those ZODB guts too
exposed, it's all implementation specific to ZODB.  By exceeding the ZODB
interface and going into the guts of it you risk future breakage and we
certainly don't want to look like we're making any promises on how ZODB
works on the inside.

> > Maybe some insight on how you designed your site would help.
>
> Well, it's not a site, to start off with. It's a distributed, open source,
> Point of Sales/Enterprise solution with commerce (not e-commerce, now) as
> it's main focus. There is a central server that sits in an office that
> syncronizes with many stores periodically; these run their own servers and
> are busy capturing sales objects and updating inventory objects.
>
> Now the inventory object must be maintained sincronized with the main
> database, which means updates must flow to and from that database to/from
> the stores. Since these updates could conflict, there has to be conflict
> resolution and so forth. _Apart from that_, my object contain references
> to many other objects, so closure has to kept in the sense that these
> references shouldn't break. Object must preserve referential integrity
> when moving from store to server.

If you can do it, I advise you to run a ZEO client at each store.  If you
can, you can forget about all this.  ZODB does conflict resolution,
caching, keeps all that id() and reference gunk completely under the hood
for you and a whole bunch of other stuff.  With minimal hacking you could
probably come up with a robust ZEO client storage that works fairly well
even in disconnected mode by caching and reconnecting to "sync" at
discreet intervals.  Who knows, maybe it can even do that now.  I know
that it can serve up cached objects even when the connection i down.

-Michel