[Zope3-dev] Zope 3 Source organization

[Martijn Faassen]

Jim Fulton wrote:
> Everbody else I talk to complains about the separation of presentation,
> interfaces, and application code.  Typically, people who work on
> application, interface, and presentation code have to work on
> three areas of the tree.  Often people are forced to edit three
> files with the same name and maintain two related configuration files.

Note that before the reorganization this was not incredibly much
better. Views for instance still were in their separate directory,
meaning two .zcml files were needed. Not having to edit the interfaces
in a different source tree helped somewhat, though of course the interfaces
were scattered across little files at the time, which has been improved
somewhat now.

> What do folks think about this?  Would people like to see a change?
> Suggestions?

We need to focus not just on the files and the hierarchies but on what's in 
the files. ZCML statements with 7 or more parameters for instance are just too
much. I'd prefer say, three or four. I don't know how that would be 
accomplished, but currently the amount of copy & paste that is going
on is just too high. I should be able to fit it in my brain better.
Complicated hierarchies are part of the problem, but just part of it.

I suspect the problems arise because we have a lot of small objects and
code fragments interacting with each other; the patterns in which they dance 
are frequently complicated and hard to remember. Where Zope 2 has overly
fat objects and interfaces and a problem as a result, Zope 3 has overly
complicated patterns and interfaces where none are yet needed. Code
evolves from ill-defined patterns that are closely tied together to
more pluggability as it turns out to be needed. With Zope 3 at present
one is forced to support endless pluggability right from the start.

Admittedly his is far more common with Zope 3 framework code than Zope 3
application code -- but since the Zope 3 framework is in many ways a Zope 3 
application it still has me worried. One object with one view is
currently pretty understandable. But what about two objects that interoperate
with two views? Three of them that have to work together? Three objects
with views currently require:

  * three content objects in possibly three different locations in the tree

  * three interfaces in possibly three different locations in the
    interfaces tree

  * three views in possibly three different locations in the view tree

  * three ZCML factory directives, possibly in three locations in the tree.

  * three ZCML add view directives, possibly in three locations in the tree.

  * three ZCML edit view directives, possibly in three locations in the tree.

  * If schema/forms are not used, page templates for the views.

  * Also menu hookup directives.

  * Am I forgetting things?
 
The locations in the tree aggravate this problem, but it's already rather
cumbersome if you just had one content.py, one interfaces.py and one
view.py and one configure.zcml.

A suggestion that I already made to you in private is giving each class its
own interface that is unique to itself. Often now interfaces have
to be created in the interfaces tree and we have to import them all
over the place, refer to them from ZCML, etc, while we are writing
*very specific application code*. We have to bear the full brunt of the
component architecture when we're not altogether sure whether this
interface is reusable at all. This is really bad for prototyping; 
programmers can't be expected to come up with reusable interfaces
right away.

So I propose the following:
 
   * each class implements its own unique marker interface.

   * this interface is just 'there' for importing from some obvious
     location.

   * It's also easy to refer to this interface from ZCML.

This way it's possible to hook up things directly, in effect through class,
but does not require much extra machinery or new ZCML directives to make
this work (as we introduce a marker interface).

If we then later on notice that there is something reusable going on, 
you can:
 
  * now introduce the actual handwritten interface.

  * make sure by altering code that the classes now implement this interface.

  * change around the various directives in ZCML so they now hook up to this
    interface.

I realize that this makes it more likely that functionality is added
through changing code, not through addition of new code. While I think
that goal is admirable and useful in gluing different components written
by multiple independent people together, in reality components often
evolve in sync and are written by the same group, and they evolve by
their code changing. This is a normal and acceptible programming practice.

The advantage of this proposal is that the whole interface tree
remains where it is, but can still be forgotten about when writing
new code. It cuts down on one typical location in the interface tree,
view tree, content tree triad.

Possibly a similar pattern can be followed for default views that
are just 'there'. I don't know what that would look like, but it might
leverage schema/forms without requiring the extensive directives.

The goal of these proposals is to make it possible for the programmer to
just focus on the actual problem at hand and solve that instead of
setting up all this infrastructure that has to be tied together first
before Zope 3 will even start. The solution that the programmer wrote won't 
be reusable from the start, but very importantly it gets the job done quickly.

There's a tradeoff in programming; you can get the job done in a nice
pluggable reusable way, or you can get the job done *quickly*. Zope 3
is very good at the former, but it should improve the latter a lot.
Often it has been said that various ZMI features will solve this,
and I'm sure they will help, but the same must the case from Python.

Note that I'm probably overstating the complexity in this post for
the sake of argument. The complexity to non-framework programmers is
probably not that bad. If you write a Zope 3 extension, often
interfaces and views *are* together in a far more shallow hierarchy.
A simple proposal would be to do this for more core code as well; 
move zope.app.content and associated interface and view tree into
zopeproducts.content, for instance.

Regards,

Martijn