[Zope3-dev] Re: [Z3lab] Support for relations in Zope

[Jean-Marc Orliaguet]

Dario Lopez-Kästen wrote:

> Jim Fulton wrote:
>
>> Jean-Marc Orliaguet wrote:
>>
>>> Janko Hauser wrote:
>>>
>>>> I'm in support of most of your points, although I do not see that
>>>> the catalog is actually proposed as an alternative. Jim's
>>>> suggestions were more directed to store things in each other. Or
>>>> reference them somehow, but not by an indirection with a central
>>>> index.
>>>
>>
>>
>> Thanks Janko. :)
>>
>>
>>> My intuition is that if you store "things in each other" you have
>>> implicitly created an index as soon as the objects get cataloged.
>>
>>
>>
>> I don't see how.  You only have indexes for the information that gets
>> cataloged. Janko summarized my position very well.
>>
>> I'll note that I'm not opposed to "relations".  I'm not even opposed
>> to a centralized store for relations.  I am inclined to limit their use
>> to where they are needed and in leveraging the component architecture
>> to make it easy to change one's mind.
>>
>
> Caveat: I have not really followed the thread to it's full extent, nor
> have I done any significant work on/with Zope3 yet; consequently
> (sp?), I  am not sure if the "things" mentioned above are general
> things, i.e. arbitrary objects, or only "things" specific to portlets,
> and the models related to this particular problem. Also I am no ZODB
> expert.
>
> With that in mind, I offer my sincere apologies if I am beating a dead
> horse or missing the point entirely. The discussion above triggered my
> curiosity about relation-support in Zope, being the reason I write
> this :-)
>
> So here is my take on relations and Zope:
>
> In general, I would like to see Zope have some way to, or provide
> basic support for applications to, create relations between objects -
> the term "relation" is in this context defined losely as the most
> basic idea of relation. No need for ACID, foreing constraints, etc.
> Just plain old simple "X is related to Y, and it is expressed as Z"
> without any particular meaning of what the relation X-Y means - that
> interpretation would be up to the particular application being built,
> and Z would be the actiual "implementation", oway of actually creating
> the X-Y relation.
>
> I have several use cases where the basic Zope relationship between
> objects, containment, is too limited and just cannot be applied.
>
> The ZODB, AFAIK, supports only relations by interpreting containment
> as a kind of relation, and the way to access an object is to traverse
> to it from the root. While it is possible to create relations between
> objects allready in Zope, these objects either cannot be
> moved/deleted, nor can they be renamed if we want to maintain the
> relation.
>
> I believe that if Zope is to provide support for relations (which I
> would like to see :) it needs to have more  plumbing to help sort out
> some practical problems that exist.
>
> To avoid the "oops, object moved - relation broke" problem, a simple,
> yet I belive totally sufficient, solution is to implement a
> centralised index of _all_ objects. Each and every object in the ZODB,
> without exception, is indexed and may be referenced by looking it up
> in the index. This implies at least the following changes:
>
> - this "master" index is maintained automagically. It is always there
> and is being constantly updated, whether you want/use it or not.
>
> - objects will have to be tagged with a unique internal id, UUID,
> read-only, somewhat similar to the concept of ROW-IDs in relational
> databases. This is not the id of an object as we know it now, where we
> can rename an object, thus changing it's ID. This is a
> system-maintained, read-only id, guaranteed to be unique in the ZODB
> (read "unique across mounted ZODB storages" as well). This  UUID is
> what the master index uses to reference objects.
>
> - all object manipulation commands need to be enhanced so as to
> support the "master" index. If I create/rename/move/delete/and-so-on
> an object, then the "master" index is updated accordingly,
> transparently. Note that the master index not necessarily needs to
> support undoing (in fact it may just be plain wrong for it to support
> undoing). It just has to reflect the current state of the ZODB and no
> more.
>
> Using this we can then create relations (implementation agnostic)
> between arbitrary objects with the help of the "master" index  using
> the objects' UUID to reference them. The relation will hold through
> any change in the object, except the deletion on an object. With the
> help of the "master" index we have the information needed to build and
> enforce constraints, build and maintain integrity rules, etc.
>
> If the idea of having this at the Zope Level is too overwhelming, then
> perhaps it is easier to create special container objects (ie. folders)
> that may be used as the starting point for applications. Inside these
> folders, the above applies.
>
> Of course I have no idea of how hard this would be to actually implement.
>
> There are several current zope-projects/solutions that IMHO would have
> benefitted from having access to at least part of the above plumbing,
> and for instance both CPS and Plone have implemented some of the above
> ideas in different ways.
>
> I do not believe that it would not be necessary to implement the above
> as a new kind of ZODB storage. I rather see it as something that is
> created and maintained at the application level (the application in
> this context being zope) or perhaps between the storage and
> application level. Best would be to have the ZODB maintain it as part
> of it's design, but I guess that would be too much work.
>
> The actual nature of relations then can be left to the
> desinger/implementor to desing. For my particualar needs, it would be
> nice to have a standard relation-object that is configurable to create
> and maintain several kinds of relations, but I am perfectly happy with
> maintaining my own kind of relation-objects (or equivalent).
>
> These relation-objects could then be used both in component level
> design (for example when Jean-Marc needs to maintain relationships in
> CPSSkins) and in application level design (for instance when I need to
> maintain groups of objects where each object may be anyplace in the
> ZODB and in several groups simultaniously).
>
> The nice part of this is that this does not dispute nor question the
> standard "containment" realtion that we now have implicitly in the
> ZODB, though I am not sure if that particular kind of relationship
> should be expressed as a relation-object.
>
> What are your thoughts?
>
> Thanks,
>
> /dario
>

FYI, maybe slightly off-topic, there is a new CPSRelation product:
http://svn.nuxeo.org/trac/pub/browser/CPSRelation/trunk/

that can use RDF as a backend or store the relations in an IOBTree graph.

If I got it right it stores relations of the type:

a -> (b, c, d, ...)

mapping integer uids to tuples of uids

and indexes the inverse relations to be able to do fast reverse lookups:

b -> a, c -> a, d -> a ...

that is one type of relation that works well to store information about
documents, relations between documents, resource descriptions...  It
doesn't work well to store the following information:

slot _ has portlets _ (portlet1, portlet2, portlet3, ...)
perspective _ has portlets _ (portlet1, portlet3, ...)

because an intersection on possibly huge sets is needed to get the
information about (slot, *, perspective)

this is to point out that different applications, or parts of different
applications need to store different kinds of relations in different ways:

* Jim also mentionned using annotations, this is OK for me if the
information stored really is *metadata*, and is not vital to make the
rendering engine work. Such information could be: the name of the user
who last modified a portlet, or updated a style, maybe store the
previous version of the content object to be able to do a simple undo
(revert changes, ...). Basically, the idea is that if you lose this
information, they will still be a page rendered with the right colors,
etc.. :-)


* It appears to me that the marker interface machinery in Zope3 is
sometimes misused in order to store information about things that had
better be stored in a relation store. cf the zope.app.pagelet
implementation:

<... tal:repeat="pagelets
pagelets:zope.app.demo.pagelet.interfaces.IDemoSlot">


* Containment is perfect for tree-like structures (DOM, for instance),
there is no discussion about that


* The relations used in cpsskins are of a third type, they are used to
connect instances of classes (portlets, styles, ...)  using predefined
predicates, when there is no way of using an adapter and assume that a
given portlet necessarily has a style for instance, but if a given
portlet has a given style associated, then it will have it in the
predefined way almost with an adapter-like sort of lookup. This is
somewhere between the component architecture and RDF.

/JM