[Zope3-dev] ZCML, Paths, and the Things You Actually Want To Change At Configuration Time

[Jeff Shell]

"No sir, I am a real horse"

Another frustrating aspect of ZCML is that the things you actually
WANT to have sys-admin style control over the configuration of seem to
get buried. I'm talking about things where the configuration is always
specific to the running environment, things like:

* RDB adapters (database name, connection/login information, etc)
* Mailers (hostname, port) and Delivery systems (queue directory, etc)
* Additional browser Resources (ie, using a special directory outside
of instance home as a drop-zone for a webcam, so that a zope
application can just refer to the resource like @@/webcam/latest.jpg
or whatever).

These are the things that can easily get buried in source code, in the
midst of enormous ZCML files or lost within a large package hierarchy.
And it's not until you move from a development server to staging (or
back) that you realize "oh crap, can't load up, that directory doesn't
exist here!")

Is there any good pattern for dealing with this? The best we can think
of, right now, for our policy at Bottlerocket is to have an
INSTALL.txt file that provides descriptions / examples of expected
mailers, rdb connectors, etc, so that they can be put into
overrides.zcml (I guess that's the right place?) and set up easily for
their target environment. We've also talked of having an etc/ or
environments/ directory in a package/application that would house ZCML
files that configured the machine-specific resources, would keep them
in source control, and in each deployment the includes would include
something like:

  <include package="example.environments" file="staging.zcml" />

This is an area where I would like to see ZCML really provide help for
configurations - where the application's configurations (sigh) could
specify "expects mailer utility with name 'example-mailer'" or
"expects an RDB adapter with the name 'example_db'", and at startup
time the ZCML could complain if at the end of all of the meta,
configure, and overrides loading it never came across an
'example-mailer' declaration and could say "The Example package
requires an email utility with the name 'example-mailer'" - set one up
by putting something like this in overrides.zcml to configure mail
support."... something that'd make it obvious that a machine specific
resource was missed, or not configured. These are the little things we
forget about (especially when more than a couple of months have gone
by since active application development), and whenever code has to be
moved between machines, we deal with all of these bits blowing up
because they were just forgotten.

Speaking of mailers. The path support in ZCML is lousy and fights
against customization and environmental configuration. Tools like the
Queued mail delivery should *probably* be putting things in
$INSTANCE_HOME/var. At least, that would be a good option to have.
Then one could configure some of those things in the application's
configuration and generally have it work anywhere. As it stands right
now, the queue directory and other such paths are local to the
configuration. And I had many a CVS update blow up because I didn't
have a 'mail/...' directory inside a package I had checked out. It
would be nice, I think, to expose some sort of environmental variables
to the Path field in ZCML to refer to the instance home and maybe to
other variables that could be set up in ZConfig (other common roots),
since ZConfig actually supports the ability to define new variables
for things like that.

I also suggested, a long time ago, to support a way to address paths
relative to packages. I remember in early Zope 3 experiments making a
little product, and then trying to make a product that built on it. In
one product, I'd have something like:

package: example1:
<browser:page class="Foo" template="foo.pt" ... />

In another package, I subclassed Foo and wanted to register that as an
alternate view.

class SubFoo(example1.Foo):
    ...

<browser:page class="SubFoo" template="???" />

You can't refer to foo.pt here! You have to copy it! This really bit
me when it came to using the browser:*form directives in a base
framework we used to build our apps, and then tried to use in other
packages. This is one of the HUGE reasons why I grew to hate the
browser:* directives and found them to hinder reuse much more than
they ever helped it. But anyways, I thought (and still think) there
should be - if this kind of ZCML is to continue - a way to address
package relative paths. I swear someone made an implementation of this
at some point.

<browser:page class="SubFoo" template="example1:foo.pt"/>

As far as I can tell, the 'Path' schema field is not much more than a
string. For a 'configuration' language, that's rough.

--
Jeff Shell