[Zope-dev] Re: [Zope] Multi tier partitioning meaningful in some Zope apps ?

[Greg Ward]

On 09 December 1999, Patrick Phalen said:
> This is worth further discussion, expansion, IMO.

Hey, I'm always happy to rant about something that I have only a bit of
experience with.  At least that way I can use ignorance as an excuse to
bow out when things get too heated.

> I'm curious if you (Greg) are finding that your External Methods that
> "deal with stuff like translating Web/HTTP/Zope peculiarities" are
> shaping up to be reusable code. I'd also like to see an example or two
> to see how you're approaching this, if possible. I wonder if we could
> have a few base classes which would take care of the majority of these
> situations.

In a word: no.  As I said, I have only a few weeks' experience with
Zope, so I have not given a moment's thought to making Yet Another Grand
Framework that encapsulates my pet theory about using Zope as a tool for
putting my Python objects on the web quickly and not-too-painfully.  (I
was going to say "painlessly", but in all honesty that word does not
apply when DTML scoping and namespace rules are anywhere in sight.)

I'm happy to explain the architecture that we're devising for putting
our Python objects on the web, though.  (For the record, "we" means the
MEMS Exchange, a rather commercially-oriented research project that
lives at CNRI -- meaning we have loads of Python expertise around.  We
all love Python, Linux, and free software; we think Zope is pretty neat
but has some serious problems -- DTML being the big one.  What we're
trying to do is build a distributed manufacturing network for
semiconductor processing, in particular for fabricating MEMS devices
[aka microsystems, aka micromachines].  MEMS fabrication borrows a lot
of technology from IC fabrication, but puts it together in weird and
wonderful ways to build mechanical components in addition to electronic
circuitry.)

If you're into thinking in terms of "n tiers", I think we have 4 of
them, but "n tiers" is a pretty fuzzy concept.  The bottom tier is what
a certain class of wanker would call "business objects": we have a bunch
of classes to represent fabrication processes, customer orders, user
accounts (a person that logs in to our web site), customer accounts (an
organization that we send bills to), fabrication sites, fabrication
equipment, etc.  (Our persistence "layer" is a jumbled mish-mash of SQL,
raw pickle files, text files, and DBM files full of pickles.  Mostly the
persistence code is in the same classes as the "business logic", so I
consider it all one tier.)

Tier 2 is the user interface.  There are actually three sub-layers here
(which is why I'm not sure if we have 3 or 4 or 5 or what "tiers").
Closest to the core, fundamental classes (aka "business logic") are a
bunch of functions that might belong in those fundamental classes, or
they might just be artifacts of our current user interface -- hence,
they stand alone.  Call that layer 2a. This is straight Python code that
has nothing to do with Zope and knows nothing about the web.  Farthest
from the core (layer 2c) is all of our DTML: presentation, interaction,
and as little logic as possible to make interface meaningful and usable.
(Turns out that you still need a lot of logic to make a meaningful,
usable web interface, and IMHO doing this stretches DTML to its limits.)

In the middle -- layer 2b, if you will -- are our external methods,
which provide an interface between the DTML and both the convenience
functions of layer 2a and the core "tier 1" classes.  (Sometimes parts
of the user interface need the convenience functions, and some don't.)
Of course, the beauty of Zope and DTML is that from our highest-level
user interface components -- the DTML methods and documents -- we can go
straight down to the fundamental classes, calling methods on the objects
that represent fabrication processes, user accounts, fab equipment,
etc.  Having the whole system based on a single modern, dynamic,
object-oriented language is an enormous win for ease-of-understanding
and rapid development.  

Tier 3 is Zope itself, including ZODB -- the only thing we (currently)
use ZODB for is what Zope uses it for, ie. to store all those DTML
methods and documents and folders and external methods.  So for now,
ZODB and Zope are indistinguishable in our architecture.

Tier 4 is of course the user's web browser, warts 'n all.

Now, back to your original question: what do those external methods look
like?  Well, they tend to be fairly tightly tied to the DTML methods (or
HTML hyperlinks, or form-actions, or whatever) that invoke them.  If
DTML method "foo" calls external method "bar", then "bar" is written
with a full understanding of what's in foo's namespace.  If an HTML form
invokes external method "blah", then "blah" is keenly aware of the form
variables it expects to see in REQUEST.form.  Etc., etc.  I see no need
to invent Yet Another Grand Framework, because Zope does that for me.  I
think some of the details are stupid and annoying, but hey -- at least
it's a Grand Framework, which is what web development really needs.

> I'd certainly welcome further thoughts/comments from others on this
> approach, particularly pros/cons on the "big leap of faith" part in the
> last paragraph.

I was referring to ZODB; two aspects in particular:

  * involuntary use: all my user interface code (DTML methods and
    documents) goes into this big mysterious object database.
    I have nothing against object databases, and I'm glad to
    see a free one for Python.  But dammit, to me this stuff isn't
    data, it's *CODE* -- and code is far too precious to entrust
    to a database.  I want it in a filesystem with full CVS control
    over it.  I want to be able to back it up with tar or afio,
    one file at a time (or rather, restore it one file at a time,
    if need be).  I don't want my code locked up in this big
    box of data.

  * voluntary use: object database are good.  Python is good.  Why
    can't I put my Python objects into an object database, dammit?
    Well, I suppose I can, now that there is ZODB.  However, it's a bit
    new and -- AFAIK -- has really only been put through its paces in
    the context of Zope.  More importantly is the limitiation of one
    data store per Python interpreter.  I want to draw a strict line
    between "my" objects (those "business objects" that represent the
    stuff important to what we're doing) and Zope's objects (all those
    DTML things that I really wish were in the filesystem where I can
    see them).  Seems to me that the best way to do that is to give
    Zope a data store, and to give me a data store.  Oops.  Can't
    get there from here... sigh...

Enough ranting for now.  Guess I should do as AMK recommended and join
zope-dev... sigh... I'll *never* get any work done now...

        Greg
-- 
Greg Ward - software developer                    gward@cnri.reston.va.us
Corporation for National Research Initiatives    
1895 Preston White Drive                           voice: +1-703-620-8990
Reston, Virginia, USA  20191-5434                    fax: +1-703-620-0913