[Zope3-dev] Vocabulariy issues

Jim Fulton jim at zope.com
Sat Sep 25 18:40:35 EDT 2004


Zope 3 has a potentially very cool framework named "vocabulary":

   http://svn.zope.org/Zope3/trunk/doc/schema/vocabularies.txt?view=markup
   http://svn.zope.org/Zope3/trunk/src/zope/schema/interfaces.py?view=log

Vocabularies represent sets of values.  They exist to support choice fields
in schemas, which constrain values to be a choice from a set of values.

I think we don't understand vocabularies well enough yet.  The evidence for
this is the set of APIs surrounding vocabularies, which are very complex.
(Usually complexity in an API reflects poor understanding of a problem.)

(I'm working on a proposal for implementing PAS for Zope 3.  I need to work
out how to provide searching capabilities for PAS plugins.  I've spend
a lot of time trying to figure out the vocabulary frameworks to support
this application.  I think I've managed it, with Gary's help.)

Let's start with some basic issues.

1. A vocabulary is defined as a set of terms.  Terms are objects having
    'value', 'token' and 'title' attributes.  The 'token' attribute is a
    a printable ASCII string that provides an attribute for a value to be used
    in forms.  This is clearly UI-specific and shouldn't be in the underlying
    model.  The 'title' attribute is a bit suspicious as well, as it is
    primarily useful for displaying values in forms.

    I think it would be better to simplify vocabularies to represent sets
    of values, rather than terms.  Vocabulary views can then be used to
    provide access to UI information, such as ids and titles.

2. Choice fields require IVocabulary objects, which means that they can
    only be used with vocabularies that support iteration.  Large
    vocabularies shouldn't attempt to support iteration. (See below.)

One of the greatest benefits of vocabularies is their potential to support
very large (possibly infinite) sets of objects.  Attempted support of large
vocabularies is a source of major complexity in the current framework. This
support centers around the concept of queries and query views.  Sadly, despite
the apparent support, vocabulary queries and query views don't actually work
with the current widgets. (I know they did work once, but they don't work
now.)

Before discussing the query issues, let's step back and look at the problem
that queries want to solve:

For large vocabularies it isn't practical to list the values for selection.
Rather, we need some way to query the set of values to find values we might
want to use.  Different vocabularies will need different query interfaces,
depending on the mechanisms (e.g. database indexes) available.  Having
gotten some search results, we can use them to update selected values.
How we update the selected values depends on the way the vocabulary is
being used.  Consider 3 common cases:

a. We have a simple choice field.  We need to select a single value
    from the vocabulary.  After doing a search, we can choose to use
    one of the serach results as the selected value.

b. We have a field that is defined as a set of choices.  In this case,
    we need to select multiple values.  After doing a search, we might
    choose to add some of the search results to the list of selected
    values.  Note that the selected values will often include values
    not in the search results.  We can also choose to remove values from
    the list of selected values.

c. We have a field that's defined as a sequence (e.g. List or Tuple)
    of choices.  This case is similar to the set case (b) above, but with
    the added requirement to sort the selected values.

In all cases, we end up with a 3-part UI for a widget:

A. We have an input form for collecting search criteria.

B. We have a results display.  This part of the UI might
    be ommitted in the absense of a search.

C. We have UI displaying selected values.  This might include
    controls for deselecting items.

In a GUI, parts A and B above would typically be provided in a
separate dialog.  It's worth noting that part B will depend on
the number of values that can be selected.

Theoretically, vocabularies support large sets ov values
through "queries".  IVocabulary defines a getQuery method for
getting a query object for a vocabulary.  The separate query
object is supposed provide the necessary search support.  For
browser based UIs, an IVocabularyQueryView interface provides an
API for delegating implemention of parts A and B described above.
Implementation code in 'zope.app.form.bowser.itemswidgets' seems
to support query veies, but the support is incomplete (decoy).

Now, continuing with issues related to querying:

3. The views in itemswidgets are designed and registered for
    IVocabulary, rather than IBaseVocabulary. In particular,
    they rely on vocabulary iteration and create selects lists
    listing all vocabularies.  They should, instead, maintain
    data copied from query results when query views are used.

4. The views in itemswidgets never call the performAction
    method defined by IVocabularyQueryView and, therefore can't
    information from search results.

Really, choice or choice-based widgets need to have very different
behavior in the presense of large vocabularies.

I really think we can simplify all of this quite a bit.
One challenge is backward compatability. There's quite a bit
of code that has been written for the existing vocabulary APIs.

I propose to create a new vocabulary-like framwork to replace the current
one.  For the sake of discussion, lets call these sources
(of data from which we can choose).  I'm open to alternate name
suggestions.

- Define the following interfaces:

   class ISource(Interface):

       def __contains__(value):
           """Returns True if the value is available from the source"""

   class IIterableSource(ISource):

       def __iter__():
           """Iterate over the values in the source"""

       def __len__():
           """Return the number of values in the source"""

   class ITerm(Interface):

       value = Attribute(
           "value",
           "The value used to represent vocabulary term in a field.")

       title = TextLine(title=_(u"Title"))

       id = ASCII(
           title=u"Id",
           description=u"""Unique identifier for a value

           The value of this attribute must be a non-empty 7-bit string.
           Control characters are not allowed.
           """)

   class ITerms(Interface):

       def getTerm(value):
           """Return the ITerm object for the given value

           LookupError is raised if the value isn't in the source
           """

       def getValue(id):
           """Return a valie for a given identifier

           LookupError is raised if there isn't a value in the source.
           """

   class ISourceQueryView(Interface):
       """View support for querying non-iterable vocabularies
       """

       def setName(name):
           """Set the name used to compute the form field names
           """

       def value(value):
           """Update a selection value to reflect use of query results

           The value should be modified *only* if the user has
           elected to apply serach results to the selected value.

           Return a new value, reflecting updates, if any.
           """

       def renderInput():
           """Return a rendering of the input portion of the widget."""

       def renderResults(value):
           """Return a rendering of the results portion of the widget.

           `value` is the current value represented by the widget.
           """

- When we want either a widget for a choice field or a widget for a
   collection of choice fields, we'll use a multi-view widget on the
   field and the field's source.

- We will use different widgets for sources and iterable sources.

   For iterable sources, the widgets will use select lists on the
   values obtained through iteration.

   For non-iterable sources, the widgets will get source query views
   (ISourceQueryView) and use them to implements parts A and B of the
   UI described earlier.  The source query views will be multi-views on
   the source and the field. The extra indirection through source query
   views allows the implementations of UI parts A and B to be separated
   from the implementation of part C.  The implementation of part C can
   be fairly generic.

- Choice widgets will use 'ITerms' views on the choice sources to
   implement 'ITerms'.

- We'll support both sources and vocabularies by making
   vocabularies specialized sources.  In fact, for iterable sources,
   we may just reuse the existing iterable vocabulary support by
   adapting iterable sources to iterable vocabularies (as views).
   We'll remove the non-working (decoy) query support from the current
   vocabulary widgets.

Thoughts?

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org



More information about the Zope3-dev mailing list