ObjectHub (was Re: [Zope3-dev] Big import blocks)

[Gary Poster]

Guido van Rossum wrote:
> Could you try that again without using Zope 2 analogies?  Being
> ignorant of Zope 2 stuff (for better and for worse :-), definitions in
> the form of comparisons to Zope 2 features don't help me.

:-)  OK, revision follows.  The Zope 2 references are contained to one 
section now, and the general description can stand on its own, I think. 
  It has various improvements beyond the removal of Zope 2 references, 
and should replace the version Paul put in the wiki (Thanks!).  Changes 
are confined to the first four or five paragraphs.  I then have further 
response to your email below.

ObjectHub goal:

   Provide a unique id (UID) for objects in traversal context.  This id 
is called a hubId.

HubId closest Zope 2 analog:

   A ZCatalog RUID.  Primary differences are that it is consistent 
(i.e., does not have any semantics of changing on reindexing) and it is 
exposed (i.e., it is globally available rather than a private 
implementation detail).

General description:

   The ObjectHub keeps track of objects in their traversal context by 
mapping hubIds to paths.  Since traversal context is an important part 
of the mapping, if the same object were mounted in two locations and 
both locations were registered by the ObjectHub, then the ObjectHub 
would consider them to be have two separate hubIds.

   The ObjectHub's hubIds are kept valid through object moves, and are 
removed for deletions.  The Zope 3 event system receives the various 
object change events and delivers them to the ObjectHub for this purpose.

   If the object is registered with the ObjectHub, the hub makes any 
necessary alterations to its own data on the basis of the change and 
also publishes a parallel hub event to the event service.  The hub event 
is similar to the original object event but uses the hubId as object 
identifier instead of the path.

   ObjectHub subscribers (clients) can then make any necessary changes 
to their data structure on the basis of the change.  They can also 
choose to raise an exception if the change violates an assertion in 
their data structures (such as breaking a required relationship) and 
thus turn back the whole transaction.

   ObjectHub clients are typically index-like objects, keeping track of 
object content, metadata, or relationships.  The ObjectHub itself is 
fairly simple, but it relies on the more complicated event service, and 
it is part of the larger set of cataloging machinery.  Cataloging is not 
yet done, and is waiting (last I heard) on SteveA's query service and 
possibly then on my object query plugin to Steve's service.  I know I've 
become utterly swamped and have had to pull away from Zope 3 
temporarily, but I think Steve was close to finishing his query service 
a while back.  He's polishing the hub itself and some of the services 
and other infrastructure on which it depends now, I think.

Future explorations:

   hubIds are intended to behave as unique ids.  Because hubids are only 
unique to one ObjectHub, having another ObjectHub in your site is a 
significant and possibly dangerous configuration decision.  These 
patterns have only begun to be explored and are recorded in the Zope 3 
wiki under the proposals section as "Advanced Object Hub Usage" or 
something similar.

Conclusion:

   If you simply want to refer to an object and have something else be 
responsible for keeping track of the object through any moves, the 
ObjectHub should be your tool for many such use cases.

   Some use cases will want more of a ZODB OID available in application 
space, in which the id is the same if it is the same object mounted in 
different locations.  These use cases have not yet been thoroughly 
explored in Zope 3 to my knowledge.

> AFAIK, the ObjectHub service doesn't ever look at the objects -- both
> its internal data structures and its API are defined in terms of
> paths, and whether or not there's an object at that path doesn't
> actually concern the ObjectHub.

The interface methods do accept contextually wrapped objects in addition 
to paths.  Moreover, while internal data structures do refer exclusively 
to paths, the presence of an object at the given path *does* concern the 
ObjectHub, because it has the responsibility in the interface to convert 
a hubid to an object, not just a path.

In fact, I have recently begun to think that this suggests we need to 
factor the interfaces better, such that the basic interface merely maps 
objects to hubIds, and  the current ObjectHub interface is a subclass 
that adds paths to the mix.  This might be a more general interface that 
could also be a super class of a repository-type object that also mapped 
id to object but without the contextual path information.  There are 
some wrinkles, though, and I haven't had to time to think it through.

> I wonder if the name isn't causing more confusion than clarity.
> Certainly the "hub" part never made sense to me: this suggests that it
> is a pretty fundamental central data structure, while in fact it is
> only used for cataloging/searching.  (Which is important, but far from
> fundamental IMO.)  And the "object" part of the name seems to be
> factually wrong.

I thought my description might raise these sorts of questions, and upon 
revision I see it is partly my fault.  As I hope I made more clear above 
than in the first email, the connection to objects is essential.  I'm 
ambivalent about the "hub" bit.

> It's really about UIDs (Unique IDs) for paths.  I wonder if a name
> like "uid service" wouldn't make more sense?

That would be better, maybe, despite my niggles with your reasons.

I'm slow to get excited about name changes, these days, though I 
understand their impetus.

Gary