[Zope3-dev] Synching objects with underlying datastore

[Garrett Smith]

Kapil Thangavelu wrote:
> On Aug 20, 2004, at 8:45 AM, Garrett Smith wrote:
>> I know when the RDB changes at the (Zope) application level. I can
>> even get a list of objects that are effected by the RDB change. I
>> want to avoid, however, synching all objects, since only a subset
>> are actually cached (in memory). I.e. I don't want to cause an
>> object to be loaded (__setstate__ called) if I don't have to.
>> 
> its not clear at what level your gluing together these rdb attributes
> with zodb persistent objects, ie if their stored in volatiles, or are
> persisted along with the rest of the object, or how  object attributes
> map onto rdb values, nor what application level infrastructure you
> have available, or even what rdbms is being used. say for a simple
> example, that these attributes are persisted along with the rest of
> the object and that there is a 1-1 object to row ratio with the row
> providing the rdb object attributes, and your using postgres. then if
> you had some sort of 'object hub' which allowed resolution of zodb
> objects based on their  row peer primary key, you could have an
> daemon listening to postgres async notifications, possibly batching
> them (depending on application sync requirements), and then sending
> invalidation messages to an application method in zope (via xmlrpc or
> some rest method) in the form of a set of rdb primary keys which the
> application method in turn uses to resolve the zodb objects, and
> update their attribute values.

I do have a list of the objects that are invalidated, so I think I'm
pretty close. If my thinking is correct, the update alogrithm is:

  for object in invalidated_objects:
    object._p_jar.sync()

My concern is that iterating *all* invalidated objects is unncecessary
and potentially very expensive.

- There'd be no point to calling calling _p_jar.sync() for unloaded
objects -- they're really not invalidated.

- By reading an object via the iteration, isn't each object's
__setstate__ method called (if it hasn't already been loaded by the
ZODB)?

What I'd like, then, is:

  for some_id in potentially_invalidated_objects:
    if loaded_by_zodb(some_id):
       get_object(some_id)._p_jar.sync()

If this isn't possible, perhaps an alternative approach would be
something like:

  for data_manager in zodb:
    data_manager.sync()

Btw, thanks again for your input.

Also, I realize this topic is better suited for the zodb list -- if it
goes much longer, I'll move it over there :-)

 --Garrett