[Zope3-dev] [DRAFT] local portlets and perspectives

[Jean-Marc Orliaguet]

Gary Poster wrote:

>
> On Aug 24, 2005, at 2:24 PM, Jean-Marc Orliaguet wrote:
>
>>
>> Hi!
>>
>> Somehow related to the discussion on optimizing catalog queries, I  have
>> been thinking about how to best implement local portlets in  cpsskins in
>> terms of scalability, performance and functionality. The  implementation
>> is heavily dependent on being able to performance effective catalog
>> queries (it is OK to wait 2 seconds in an ECMS to search 1 million
>> documents, but it is not OK to have to wait 2 seconds to render a page
>> containing portlets).
>>
>> Here is the proposal:
>> http://www.z3lab.org/sections/blogs/jean-marc-orliaguet/
>> 2005_08_24_local-portlets
>>
>> It is built on the notion of "Perspective" (see the link) and on the
>> idea of querying the catalog using triadic relations instead of  joining
>> sets of query results based on dyadic predicates (such as with RDF).
>
>

Hi Gary!

> I need to read more to have a more complete reply (though I'm hoping 
> Benji will do it for me, since he knows more about CPSSkins than I), 
> but here are a few responses, based on reading it and talking it over 
> with Benji.
>
> Your overall model is interesting to me because it goes a good way to 
> unifying the various portlet use cases.  I know of six categories: 
> page designer wants to fill slots for a site; page designer wants to 
> fill slots for a section of a site; page designer wants to fill slots 
> for a page on a site; user wants to fill slots for a site; user wants 
> to fill slots for a section of a site; and user wants to fill slots 
> for a page on a site.  The section and page use cases might be 
> collapsed technically, but have a different feel to a user, IMO, and 
> so should be separated.  Combining these into a unified model is 
> impressive and cool.
>
> I worry that the description does not get into the possible UI 
> expectation differences between a page designer and a user, and 
> between working on a site or section versus a single page.  These 
> seem like tricky UI bits to get right to me.  I'd be interested in 
> seeing a discussion of your thoughts of the UI from this perspective, 
> especially in regards to showing a user, within an editing screen, 
> which portlets are being edited from a page perspective, from a per-
> user perspective, from a section perspective, and so on.  It seems 
> like it could cause information overload.
>

I'll get back to this part in a mail later since the answer is longer.
But basically:

- there is a separation between page designer (theme designer) space and
user space.  Page designers add portlets into cells and users add
portlets to slots. A same theme can have several pages, and pages can be
associates to sections of the site. A page is a page (not a
perspective). Perspectives are only for portlets in slots.

- the main idea with perspectives is that you can group contextual
information into a few categories:
  1) let us say for instance: the category of users that have just
logged in for the first time. These users would see the site from "the
first login perspective", maybe be presented with a "Welcome" portlet.

  2) a "perspective designer" (in this case an application designer)
would in a blog application create a "Blog perspective" for users, where
they'd see the blog categories, the archived blogs, etc. Currently this
is done in CPSBlog by creating local portlets for every user which is
not very efficient in terms of scalability.

 3) in a webmail application the "perspective designer" would set up an
INBOX portlet, a toolbox portlet for sending mail, etc.

 as a result the end user would move from a "blog" perspective to a
"webmail" perspective even maybe by staying in the same folder (which is
not easy to implement with the notion portlets associated to sections of
a site)

This shows that the notion of sections of a site is not appropriate for
local portlets, it is better to use a more general concept such as the
one as Perspective. Also if there are many sections in a site, there are
in comparison very few perspectives.

In fact when it comes to the question of the visibility of portlets
(visible in some folders of the site) this will be done by using a
visibility filter (the infrastructure is already implemented cf.
http://www.z3lab.org/sections/front-page/white-papers/draft-renderer/downloadFile/attachedFile/renderer-architecture.png)


Hence for a given user there won't typically be more than 4 or 5
perspectives in the portal.

I have prototype soon ready, that will become clearer with an animation.

> The 'perspective' idea is a good concept for the discussion (and I 
> used the word above), but you claim that it is a single element in a 
> triadic relationship, then list several perspectives, each of which 
> could influence the lookup.  This means to me that it is not triadic 
> but 'n-ary'.  Moreover, it seems a lot more like a graph walk, for 
> which we don't need new code in CPSSkins, than a new n-ary data 
> index.  RDFLib or any similar graph implementation with typed 
> relationships would get the efficiency you want, as far as I can 
> tell, with a lot less new code to maintain.
>
> Gary
>

that's true but if there are at most 5 perspectives for a user in the
entire portal, there will for a same page be at most 1 or 2 combined
perspectives (the user's own perspective and the application's
perspective) so this is not a problem really for the catalog queries.

I've just updated the relation storage code:
http://svn.nuxeo.org/trac/pub/file/z3lab/cpsskins/branches/jmo-ajax-integration/storage/relations.py

this is how the elements of the relation are indexed:

--------------------------------------------------------

    def _indexRelates(self, relation, id):
        """Index the relation in the relates catalog.
        - the relate's identifier and its role in the relation (1, 2, 3)
          is used as the index key.
        - relations are refered to by their id (their name in the container)
        """
        position = 1
        id = unicode(id)
        # index the relates by position (meant for wildcard queries)
        for relate in relation:
            key = (str(relate), position)
            self._relates.setdefault(key, []).append(id)
            position += 1
        # index the relates as a tuple
        # (meant for queries without wildcards)
        self._relates.setdefault(tuple(map(str, relation)), []).append(id)
        self._relates.p_changed = 1

--------------------------------------------------------

the idea is to be able to do wildcard queries such as:

- give me all the portlets that use this style
- give me all the portlets located in this slot
- ...

I think that Michel uses a triple BTree store in zemantic to index all
three elements (subject, predicate, object), the idea here is the same
except that relations can be triadic, (which gives 4 elements to index)
this is why I index the relates with _indexRelates() and the predicate
with _indexPredicate().

Also most importantly the elements of the relation are also indexed as
one unique tuple, which means that I do can a direct lookup in the
catalog if *all* the elements are specified during the query.

This is the crucial point (and it really has to do with optimization
through application design):

when designing the application:
- wildcard queries in editor mode are allowed
- wildcard queries in page rendering mode are not allowed

this is why I'm using the triadic relation for
slots/portlets/perspective, i.e. to be able to do a direct lookup in the
catalog when rendering slots.

the relevant code is:

def search(self, predicate=None, first=None, second=None, third=None):
   ...
        relate_index = self._relates
        predicate_index = self._predicates

        # if all the relates are specified do a direct lookup
        if predicate is not None:
            arity = len(predicate)
            relates = (first, second, third)[:arity]
            if not None in relates:
                if relate_index.has_key(relates):
                    return relate_index[relates]


more later..

/JM