[Zope3-dev] ObjectHub: Kill the plugins!

[Gary Poster]

On Sun, 2002-10-20 at 08:51, Steve Alexander wrote:
> Gary Poster wrote:
> > I haven't had much time to write this out, so let me be brief, and
> > expand as requested later:
> > 
> > We have now, in this neck of the service manager woods, an EventService
> > and an ObjectHub (now also a service; client of the Event Service).  
> 
> Will you be able to check this into CVS soon?

Yes.  Not railroading my ObjectHub decisions onto CVS was important to
me, and thus my flurry of ObjectHub emails.  I think I have enough
consensus now to proceed.

> > In addition, I propose
> > 
> >  * a MetadataService (client of the object hub; takes a hubId and
> > returns a dict of whatever metadata we've collected about that object;
> > this collection mechanism being presumably via an adapter, i.e. a more
> > sophisticated and non-pseudocode version of (given hubid)
> >  myObj = ObjectHub.getObject(hubid)
> >  MetaDataService.myIOBTree[hubId] = getAdapter(myObj, ICachedMetadata)()
> > where the called adapter returns the desired dict of info gathered from
> > myObj),
> > 
> >  * an IndexService (client again of the object hub, and perhaps to a
> > lesser degree of the MetadataService, that accepts TBD (To Be
> > Determined) query objects, as discussed somewhere or another on the
> > wiki, and returns TBD query results; contains addable indexes), 
> > 
> >  * and various query utilities (clients of the IndexService that each
> > take a given variety of query--SQL-like, form-like, etc.-- and translate
> > them into the ever-popular TBD query objects, which it passes to the
> > IndexService to then return that result to the caller.
> 
> Ewww... too many services!

Everything is a balance...on the bright side, even the names themselves
tell you what to put where.

> For me, these all come under the heading of "QueryService".
> There are many advantages to having metadata, indexing and querying 
> handled as plugins inside the same service. You can build a range of 
> queries out of simple reusable parts, rather than have monolithic 
> "indexes". You can also make the facilities of external indexing 
> systems, such as Lucene or MySQL, available as a group of related 
> plugins to provide such metadata, indexing and normalisation services, 
> which can be used in combination with other sources of data.

Yes.  Configurability and usability of your approach--the approach we
have agreed upon in the past, because I do see its many strengths--have
begun to concern me.  In my mind it looks like a rats nest of
configurations, in which you don't know what plugin is solving what part
of the equation, because they all can be doing several parts of the
job.  While the separation of query interface from indexes from metadata
seems to have a clear disadvantage in terms of efficiency, and perhaps
even has an unpleasant artificiality to it in comparison to the
implementation, at least the contracts are clear and the configuration
is easily imaginable.

> There are also advantages to using a single service of collaborating 
> plugins when you want to handle cascading deletes/renames -- when you 
> delete or rename a folder, you can send just one event out to the 
> object-hub, and the single HubId event sent to a plugin that knows about 
> containment. This plugin can notify the other objecthub subscribers of 
> the new location (or deletion) of all registered objects that are 
> transitively contained within that folder.

That doesn't require plugins, but subscribers that can return a response
to the object hub.  This is a significant use case we need to address
soon, but it essentially comes down to (a) the subscriber being able to
know what objecthub sent it the message, so it can send one back, and
(b) a method on the objecthub for subscribers to call in that situation
(some kind of notify variation).  We would probably need (b) anyway, and
(a) can be solved in a number of ways that are not tied to plugins.

...
 
> I have a design for the Query service, and Query objects, and 
> collaborating plugins. I discussed these at length with people at the 
> EuroPython conference.

Understood.  I am eager to be convinced, and happy to be outvoted on
this issue, as I know your design has many impressive technical merits.

> As soon as you check in your latest ObjectHub work, I'd like to start 
> work on the Query service. I think I need to see the current state of 
> the ObjectHub to work out how the original Query service design works 
> now it has been split off from the core objecthub.

Gotcha.  :-)  I'll try to have working version 0.1 in late tonight (EST
as usual).  It's been a few days since I worked on it.  I have the tests
from the generic ObjectHub ported, but the placeful one needs a lot more
tests, sooner rather than later...

Thanks

Gary