[Zope3-dev] Extensibility and ZConfig

[Fred L. Drake, Jr.]

=================================
Extended Section Types in ZConfig
=================================

ZConfig tries very hard to break the world down into carefully
described pieces, with "abstract types" to describe things that are
similar in nature but different in detail (such as different types of
storages), and "section types" to describe specific kinds of things
(for example, a FileStorage).

For convenience and modularity, an application schema can "import"
collections of types from different packages and use those to build
the top-level schema.  This doesn't allow for new section types which
are implementations for existing abstract types, and that's what we
need to enable new kinds of storages (DirectoryStorage) or databases
(APE).


Why Isn't It There Already?
---------------------------

Ok, so we have imports, why can't any code that runs in Zope be used
to provide implementations for abstract types?  Isn't that enough?

Here's the scoop: importing a collection of types loads types based on
a package; the types that the package can provide are loaded, and
that's it.  Third-party code that implements an abstract type defined
in a component isn't known by the component, so the concrete section
types implemented by the third-party code are not loaded as a result
of the import.

To add types that aren't available until 3rd-party code is combined
with Zope in an instance, we need a way to import type collections
based on information in the configuration of the instance (what
third-party code is installed).


Possible Approaches
-------------------

There are several possible approaches, two of which are both
reasonable and easy to use (of those that we've come up with so far).

Configuration-based Imports
~~~~~~~~~~~~~~~~~~~~~~~~~~~

The basic issue of needing to load additional schema components can be
handled by adding a mechanism to allow schema imports to be performed
by the configuration file.  This can be done by adding a ``%import``
directive similar to the ``%define`` and ``%include`` directives.
This would allow additional types to be loaded and used from the
configuration, without introducing any name conflicts with the base
schema (since the base would be loaded before the configuration file
is loaded).

For example, this configuration file::

  # import some additional components:
  %import toby.directorystorage

  <zodb>
    <directorystorage>
      base-path  /my/instance/var/dir-storage
      ...
    </directorystorage>
    ...
  </zodb>

Causes a schema component (similar to that found in the ZODB package)
to be loaded.  This new component would be located by importing the
toby.directorystorage package and searching for a ``component.xml``
file to load along the package's ``__path__`` (``component.xml`` is
the default name used by the schema definition's ``import`` when it
isn't specified using a ``file`` attribute).

With this approach, additional components must be imported before the
types defined in them are used, but that should not be a problem.  If
a type is used before the required import, it will be reported by
ZConfig as an unknown type name, just like a type name that's
completely made up or that contains a typo.

The downside of this approach is that we introduce ``%import`` in the
configuration syntax, which looks quite similar to ``%include``.


Dotted Section Type Names
~~~~~~~~~~~~~~~~~~~~~~~~~

This is an alternate approach that Jim and I came up with a month or
so ago.  No separate ``%import`` directive is required.

Instead of allowing the user to specify additional imports, the
configuration data can imply additional imports by using dotted
section names.  Names that contain dots are broken into two parts at
the last dot: ``alpha.beta.gamma`` would be split into ``alpha.beta``
and ``gamma``.  This would imply the import of the ``component.xml``
from the ``alpha.beta`` package, and refer to the ``gamma`` type
defined there.

For this to make sense, every type defined in the ``alpha.beta``
package would end up having dotted names, which could be very
annoying.  Otherwise, the name of the ``gamma`` type could be spelled
two different ways after the initial import:  ``alpha.beta.gamma`` and
``gamma``.

Just to be thorough, the example presented above, converted to use
this syntax, would be::

  <zodb>
    <toby.directorystorage.directorystorage>
      base-path  /my/instance/var/dir-storage
      ...
    </toby.directorystorage.directorystorage>
    ...
  </zodb>

I think this weird handling of names kills this approach.


  -Fred

-- 
Fred L. Drake, Jr.  <fred at zope.com>
PythonLabs at Zope Corporation