[Zope3-dev] defining interfaces

[Steve Alexander]

Garrett Smith wrote:
> Looking through the Z3 codelines, it's hard to see a consistent pattern
> wrt interface organization.

The "interfaces" modules and packages are organised according to 
projects. Perhaps it would be easier to see if it were more obvious 
which packages are considered to be projects?

Here are all the interfaces modules and packages in the Zope 3 CVS HEAD.

./persistence/interfaces.py
./transaction/interfaces.py
./zodb/btrees/interfaces.py
./zodb/interfaces.py
./zodb/code/interfaces.py
./zodb/storage/interfaces.py
./zodb/zeo/interfaces.py
./zodb/zeo/zrpc/interfaces.py
./zope/app/interfaces
./zope/app/interfaces/index/interfaces.py
./zope/i18n/interfaces.py
./zope/component/interfaces.py
./zope/configuration/interfaces.py
./zope/exceptions/interfaces.py
./zope/interface/common/interfaces.py
./zope/interface/interfaces.py
./zope/pagetemplate/interfaces.py
./zope/proxy/interfaces
./zope/publisher/interfaces
./zope/schema/interfaces.py
./zope/security/interfaces.py
./zope/server/interfaces
./zope/tal/interfaces.py

 From this, I can see that there are three "large" projects, as 
identified by their "interfaces" packages:

./zope/app/interfaces
./zope/proxy/interfaces
./zope/publisher/interfaces
./zope/server/interfaces

There's one module that shouldn't be there, and should be renamed:

./zope/app/interfaces/index/interfaces.py

In a previous email, I discussed the zodb package, and how it perhaps 
should look like this:

./zodb/interfaces

rather than like this:

./zodb/btrees/interfaces.py
./zodb/interfaces.py
./zodb/code/interfaces.py
./zodb/storage/interfaces.py
./zodb/zeo/interfaces.py
./zodb/zeo/zrpc/interfaces.py

The interface package does something a bit different:

./zope/interface/common/interfaces.py
./zope/interface/interfaces.py

The latter is the module of interface interfaces, such as IInterface and 
IAttribute.

The former is where various interfaces for builtins are defined.


> I'm assuming that no one wants another massive file reorganization,

I don't believe anyone has suggested that there's something wrong with 
the current organisation of interfaces within projects. The problem is 
that this organisation isn't clearly documented.


> so I'm thinking in terms of what non-core Zope 3 developers should follow.
 >
> Preliminary Definition of Project:
> 
> 
>>a body of code that is within some package that may be 
>>independently maintained, and that well-defined dependencies on other 
>>code and other projects
> 
> 
> Procedure for Small Projects (no subpackages, no subprojects):
> 
> - Define all interfaces in a file "interfaces.py"

This must be a procedure for small projects that are in their own package.

  /foo/                 The project
  /foo/interfaces.py    The project's interfaces
  /foo/__init__.py      Let's make this into a package
  /foo/stuff1.py        Here's some more implementation
  /foo/stuff2.py


> Procedure for Large Projects (at least one subpackage or subproject):
> 
> - Define all interfaces in a subpackage named "interfaces"

If there are interfaces that are pertinent to subpackages.


> - Maintain a "mirror" between the top-most package directory and the
> interface subpackage directory. I.e. when a subpackage is added to the
> project, add a subpackage to "interfaces" of the same name.

Usually, putting interfaces in a new module inside the 'interfaces' 
package is sufficient. You only need a new package of the same name if 
you think you'll have loads of interfaces in there.


> - Default interface implementations should be defined in files/modules
> of the same name as their respective interfaces.

I don't know what you mean by a default interface implementation.
Interfaces have many implementations. There is no "default".


> I realize the second procedure deviates a bit from your last message,
> but here's where I'm coming from:
> 
> - Trying to keep things drop-dead-simple, without oversimplifying.

That sounds like a good goal.


> In my
> experience, users just want to know how to do something...the fewer
> things to ponder/weigh, the better.

In that case, I suggest that people always define interfaces in a single 
module file called "interfaces.py" in their project's package.

If this gets unweildy in the future, they can easily split this out into 
a package. They can use the __init__.py in the package to keep the API 
of project.interfaces exactly the same, so they won't need to change 
code that uses the interfaces as a result of this organisation.

There's no point starting out with something that is more complex than 
it needs to be.


> - I'm not thinking in terms of how things are now, but in terms of how
> new developers should be advised.

New developers should be advised to keep things simple, until that 
simple thing no longer benefits them.


> - The simplest delineation between project types is "small" and "large".
> Large is characterized as having subpackages/subprojects. Sure, there
> might be exceptions, so this is presented as a guideline. Ultimately,
> the developer can figure out what's small and what's large.

You can see from the analysis of projects in Zope 3 at the top of this 
message that most projects in Zope 3 are "small".

The only time I'd want to start with a "large" project lay-out is if I'm 
working with a team of people who are going to take independent 
responsibility for various sub-projects. But then, I've got 
sub-projects, and sub-projects are projects... In which case, let's call 
each "subproject" a "project", talk about having a bunch of small 
projects inside a package.


> - Once the decision has been made to go with a "large" project
> structure, the developer should maintain a mirror within the
> "interfaces" package, as much as is possible. My thinking here is that
> this approach:

Yes.

>   - simplifies the thinking for the developer -- file for file,
> directory for directory, things stay mirrored.

No. Not "file for file" or "directory for directory". Often, a module is 
sufficient to keep the interfaces for a whole package. There is no point 
using lots of small files in a directory when one larger file would be 
simpler.

There are good reasons for this:

   * Flat is better than nested.
   * In one module, you have just one set of imports, in many modules you
     have many imports (including imports between the modules).


>   - simplifies things for the reader/user of the project -- file for
> file, directory for directory, the reader/user knows what expect.

If you use modules in preference to packages, the arrangement looks the 
same, so the reader still knows what to expect. There's less to look 
for, and more of the documentation is available in one place. (For some 
purposes, interfaces are documentation.)


> I've spent some time in the Java world where this is big issue.

I spent a lot of time in the Java world. I'm so glad I'm on Planet 
Python now :-)


> It's
> important that the user of a "project" be able to track down default
> implementations for interfaces and vise versa.

I think this is because in Java interfaces are often defined as "dead 
chickens" that are defined because they must be defined to satisfy 
strong type-checking.

In Zope, we define interfaces so that we can loosely couple components 
by using adapters and services.


> In Java, these class types are almost always in the same package and
 > follow a naming convention:
> 
>    mypackage
>    -----------------------
>    Foo - interface
>    DefaultFoo - default implementation

They are in the same package because they are not really intended to be 
used separately. There often aren't facilities available to easily use 
alternative implementations.

In Zope, interfaces are in a different package, and all together, 
because many people want to use the interfaces without being concerned 
with the implementations of those interfaces. An example of such a 
person is someone writing Browser presentation code.


> Very simple indeed. My recommendation above is based the (naive?)
> assumption that a similarly clear approach is possible/good for Zope 3.

Before the great renaming in December, Zope modules were arranged 
somewhat as you describe the typical Java case:

   * One class or interface per file.
   * Names like IFooClass and FooClass for an interface and its first
     or nearest implementation.
   * A deep package hierarchy.

With the new organisation of names, we have instead:

   * Many classes or interfaces to a file.
   * Fewer files, so we can have more interfaces / classes to a package.
   * Fewer sub-packages, so a flatter and more understandable package
     hierarchy.
   * Shorter import statements, and vastly fewer imports per module.
   * Interfaces clearly separated from implementation clearly separated
     from presentation code.

--
Steve Alexander