[Zope3-dev] RFC: zopeproject

Tue Jul 17 19:13:51 EDT 2007

For the past weeks I've been toying around with egg-based installs of 
Zope and flexible WSGI-based deployment using PasteDeploy [1]. Both 
essentially challenge the concept of instances as we know them.

What is an instance?
--------------------

* Start/stop scripts for a pluggable application called Zope

* A collection of "plug-ins" for Zope-the-application (in Zope 2 these
   were called Products, in Zope 3 we say packages) that make up the web
   application.

* Configuration of Zope-the-application (site.zcml)

* Server configuration (zope.conf)

So, an instance actually defines the web application by keeping 
"plug-ins" in lib/python or Products and by saying how they work 
together (site.zcml and package-includes). And then instances also 
define how they're deployed (again site.zcml, and zope.conf).

Pluggable app vs. libraries
---------------------------

While the idea of Zope as pluggable-application may have its value to 
some people, a library approach is much more natural to most Python 
developers.

Imagine you're writing a GUI application. Without question you'd use 
some sort of GUI toolkit (e.g. wxPython). Would you expect you would 
have to hook into the the "wxPython application" as a plug-in? Isn't it 
more natural that you simply write your application and just use the 
wxPython *library* wherever necessary? Indeed it seems so, and that's 
probably why it really is that way.

Zope 3 has now been successfully split up into separate pieces: 
individual libraries. I'd therefore like to propose an alternate 
approach to developing web applications with Zope: the library one, 
rather than the pluggable app one.

The good news is that Zope is already there (thanks to the split-up). We 
just need to slightly change the way we develop applications with it.

Writing an application using the Zope libraries
-----------------------------------------------

How would you actually write an application that *uses* Zope instead of 
plugging into it? Simple. You do what any other Python developer does: 
start a package. Then proceed as follows:

1. Since you're using Zope you'll put all the kinds of Zope-based 
components in that package that you know: ZODB-persistent classes, 
views, Page Templates, etc. The more Zope components you use, the more 
dependencies your application will grow, which can all be tracked using 
the a standard setup.py script that we know from distutils/setuptools.

2. You configure the application's components. Surely you want the Zope 
publisher, the Zope security machinery, etc. So before configuring the 
applications own components, you want to load those components from Zope 
that you're reusing. That means your package's configure.zcml file will 
start off by including lots of Zope bits and pieces. Then it might 
configure all the components that are specific to your application.

3. You make the application startable. A GUI application would simply 
have a main() routine. A modern web application has a WSGI application 
factory. So, you'd write a WSGI application factory that does a few but 
necessary steps (usually that only involves parsing zope.conf and 
site.zcml) and then hands over the work to Zope's own 
WSGIPublisherApplication.

(If that does sound like a lot of typing to you, don't worry. We'll 
solve that problem later.)

The application is a single package!
------------------------------------

So what have we got now? A Python package that defines a WSGI-capable 
web application. And it's self-consistent! The whole web application is 
actually defined in that one package plus setup.py (which is needed for 
dependencies).

If you compare that to the old instance-based approach, this is a 
*major* leap forward. With traditional instances, your application was 
made up of whichever plugins you had in lib/python or Products and 
whether or not they were enabled in site.zcml or package-includes. So 
developing on an application such as Plone led to "interesting" 
solutions like having the Products directory be a subversion checkout 
with lots of externals. I've seen similar solutions in the Zope 3 world, 
too.

With eggs and and a library approach to Zope, this is a thing of the 
past: you simply install that one package and it contains a runnable, 
configured application. And because it lists Zope's libraries as 
dependencies, they will be installed automatically as well. So you no 
longer install Zope and then add the application code. You install the 
application, and Zope happens to be pulled in as a consequence. And if 
your application package specifies a specific version of Zope, people 
won't have problems by accidentally having installed the wrong version 
of Zope.

Deploying the application
-------------------------

Thanks to eggs, the application is easily installed, including its 
dependencies. Thanks to WSGI and PasteDeploy, the application is also 
easily deployed:

1. Write a configuration file (e.g. deploy.ini) that connects the 
application to a WSGI server, like so::

   [app:main]
   use = egg:MyApp

   [server:main]
   use = egg:Paste#http
   host = 127.0.0.1
   port = 8080

2. Write a site.zcml file that includes your application package and 
contains deploy-specific ZCML directives that don't make sense in the 
application's configuration. A good example is the a default 
administrator user account.

3. Write a zope.conf file that configures the ZODB database.

After those three simple steps, you can then lauch the application by 
invoking Paste::

   $ paster serve deploy.ini

As we've seen, the application definition is clearly separated from the 
deployment. The application is concealed in the package, the 
deployment-specific stuff is in three configuration files (deploy.ini, 
site.zcml, zope.conf).

Automation
----------

What I've sketched out here sounds like a lot of typing. Indeed it is. 
At least with instances, you get some automation so you won't have to 
start from scratch. This where zopeproject [2] enters.  It basically is 
mkzopeinstance for the library-based approach. It doesn't create an 
instance but

a) a Python package with common Zope library dependencies, a 
configure.zcml that includes the most common Zope libraries and a WSGI 
application factory that pretty much does the standard things upon 
startup (load zope.conf, load site.zcml, load ZODB databases).

b) a sample deployment of that application, in other words, a sample 
deploy.ini, zope.conf, site.zcml.

It also downloads the standard Zope library dependencies. After running 
zopeproject, you'll have a fully functional Zope-based web application 
(much like you do after running mkzopeinstance). That means with 
zopeproject your Zope server will be up and running just as quickly (if 
not even quicker since it even downloads and installs Zope for you).

Feedback
--------

As much as I see the library-driven approach replace the instance 
approach, I see zopeproject as the way to get started with an 
application instead of mkzopeinstance. I would like to ask everyone 
who's interested in checking it out and give feedback, both on these 
ideas and on zopeproject specifically.

[1] http://pythonpaste.org/deploy
[2] http://cheeseshop.python.org/pypi/zopeproject

-- 
http://worldcookery.com -- Professional Zope documentation and training