[Zope-dev] DISCUSS: XHTML Templates proposal

[Paul Everitt]

Howdy folks.  As advertised in my last email, below is our Fishbowl
Proposal on a new template architecture.  Some notes:

1) This mostly replaces DTML.  As mentioned, the goal isn't necessarily
to get DTML removed from Zope.  Rather, most of the audience will see it
little or none of the time.

2) This proposal has gotten more internal DC scrutiny from a
requirements perspective than anything we've done.  The proposal has
been rewritten four times.

3) The original version of this forced the connection between
reinventing the data tier (documents) and the presentation tier
(templates/pages/stylesheets).  We eventually found the way to decouple
the connection, but we still need to write a documents proposal.

4) What this really, really needs right now is extensive user
documentation with screenshots written *before* the software.  This is
where we got stuck.  Docs are the fastest way to confront the myriad of
usage details.

5) Our proposal is a little more ambitious than HiperDOM (we have a
concept of "components" that are hinted to but not explained).  But for
now, HiperDOM is a good basis for exploring XHTML Templates.

6) I'm still not convinced template is the right word.  Unfortunately
I'm less convinced at any alternatives.  Whatever is chosen, it must
feel "normal" to the vast majority of the audience.

Cheers!

--Paul

XHTML Template Architecture

  Zope should increase usability and promote separation of tiers by
  moving to a presentation and data architecture.  Site Designers will
  author and round-trip presentation objects in familiar tools.
  Combined with the Document Architecture, this proposal establishes
  the fundamental content and presentation architecture for Zope.

Note on status

  The great folks at Hiperglocia have already produced something quite 
  similar to what is discussed here.  Their Zope product is called 
  "HiperDOM", http://www.zope.org/Members/lalo/HiperDOM/.

Note On Jargon

  The choice of term for the presentation object has been contentious.
  Right now the list of choices include: template, view, page, or
  stylesheet.  This proposal doesn't make the decision on the jargon.
  Rather, the tier is usually refered to as the presentation.  When a
  choice has to be made, such as the Architecture section, Template is
  used as the temporary choice.

Problem

  Zope currently has a model of DTML Documents and DTML Methods as the
  basic data and presentation tiers.  However, this model is
  confusing, doesn't promote separation of tiers, and doesn't work
  with popular web design/authoring tools.

  For the presentation tier, Site Designers are faced with an idiom in
  which well-formed HTML is split between two "files"
  (standard_html_header and standard_html_footer).  Also, the
  presentation mockups are "taken away" by programmers who insert a
  bunch of alien tags that don't present anything on the screen of the
  web design tool.

  Worse, DTML is fundamentally impossible to edit directly by classic
  tools due to the "source vs. rendered" issue.  That is, when a Site
  Designer wants to edit a piece of DTML, they will get the *rendered*
  version of the DTML for editing, where all the DTML tags are
  expanded into HTML.

  In summary, there is a tremendous usability issue, both conceptual
  and in practice.

  There are additional problems:

    o DTML is considered a "proprietary" language

    o Data needs to have different presentations used under different
    conditions

    o Consulting practices need a way for customers to start seeing
    screen results immediately without having the programmer "steal"
    the templates by converting them to something alien

Goals

  The goals of the Presentation Architecture are:

    o Clear separation of presentation, data, and logic with clear
    concepts in Zope for these tiers and audiences

    o Increase usability by allowing round trip presentation with
    common tools and familiar concepts

    o Allow those presentation tools to work by having well-formed
    markup (e.g. no separation into header and footer)

    o Supplant "proprietary" markup language (DTML) in presentation
    tier by leveraging standards (XHTML) (Note: "eliminate" isn't the
    target as it might not be feasible in 100% of the cases)

    o Match the appropriate level of capability currently in DTML
    usage

    o Simple presentation reuse within reach of standard tools

Solution

  Zope should move to an architecture with clear separation of
  presentation, data, and logic, with clear concepts in Zope for these
  tiers and audiences.

  The presentation tier will get a tremendous usability increase by
  allowing round trip presentation with common tools.  This also
  ensures that Site Designers finish with the same stuff they started
  with, meaning the programmers don't come in and cast their work into
  "code".

  The architecture should make sure those presentation tools are
  effective by having well-formed markup (e.g. no separation into
  header and footer).

  Zope should supplant "proprietary" markup language (DTML) in
  presentation tier by leveraging standards (XHTML).  It might not be
  possible to 100% remove DTML in 100% of the cases, but it should be
  a rarity instead of the rule.

  The new presentation architecture should match the appropriate level
  of capability currently in DTML usage.  For instance, common idioms
  like browse-by-batch should be possible.

  The new presentation architecture should leverage this new XHTML
  approach, XSLT, and perhaps DTML.

  Site Designers want simple reuse of content within reach of standard
  tools.  For instance, the Site Designer wants a standard copyright
  "asset" on the bottom of every page.

  The presentation architecture should not know about the details of
  the data tier or the logic tier.  Rather, the presentation
  architecture will just provide a view onto the model expressed in
  the object system, where the model uses standard APIs.

  A **page** is the result of applying presentation to data in the
  object system.  A page is a particular result of a URL when viewed
  under certain conditions.

  Ultimately, data such as documents will be matched to a presentation
  according to implicit and explicit rules.  The simple case will be
  simple (invisible); the complex, custom cases will be
  straightforward.  In the initial rev of the architecture, only one
  presentation will be implicit for a resource.  Other presentations
  of that resource will require explicit traversal to obtain them.

Architecture

  The Presentation Architecture initiative will deprecate the
  presentation aspects of DTML Document and DTML Method objects.
  Instead, a new kind of object, code-named Template, will be used.
  Templates have two varieties: XHTML Templates for Site Designers
  that use HTML layout tools, and XSLT Templates for XML programmers
  requiring high-end processing control.

  Site Designers will create XHTML Template objects that control
  look-and-feel across numerous content resources in a site.  The
  Template object will allow multiple formats.  XSLT will be used for
  high-end presentation oriented towards XML programmers.  DTML might
  be retained as a template format for certain presentation tasks.

  However, the focus of the new Template architecture will be a new
  format based on XHTML.  This XHTML Template will be focused on
  giving Site Designers round trip presentation authoring using the
  dominant tools such as Dreamweaver.

  XHTML Templates will be complete, well-formed HTML that look like a
  resource will look when the XHTML Template is applied to it.  There
  will be no "source vs. rendered" issue.

  In this Presentation Architecture, Site Designers can start mocking
  up a Zope site by editing real HTML and graphics in Zope from the
  very beginning.  There will be nothing Zope-ish in what they are
  doing.

  As the mockups turn into live content, certain regions of the XHTML
  Template will be "connected" to the live content.  This is done by
  indicating the object data that will be inserted when the XHTML
  Template is applied to content.

  However, there will still be sample content in the XHTML Template
  that lets the presentation task continue.  This sample content gets
  discarded when the XHTML Template is applied to a piece of content
  such as a Document; the connected regions in the Document replace
  the sample content in the XHTML Template.

  These connectors will exhibit themselves by simply adding Zope
  attributes to existing, valid HTML tags.  These attributes will
  follow the XHTML extension facilities of namespaces and will be
  silently preserved by authoring tools.  The attributes indicate that
  this region of the XHTML Template will be replaced by content from
  the Document being viewed and from other content in the object
  system.

  The connectors will allow different "modes" that provide the
  alternation and conditional facilities present in DTML.  These modes
  are just simple attributes to extend the semantics of existing XHTML
  tags.

  The XHTML Template architecture will facilitate re-use by allowing
  blocks in the XHTML Template to be named as a
  (component/library/asset) block.  As the Site Designer authors and
  updates the block of XHTML, Zope saves the named block as a separate
  object in Zope.  This named object can then be used in other XHTML
  Templates.

  Internally, XHTML Templates will use the standard XML parser in Zope
  instead of maintaining a separate parser.  It is possible that the
  XHTML Template will be broken into a DOM and stored in the ZODB as
  separate database objects for different nodes.

Scope

  Defining the document architecture has been moved to a separate
  proposal.

  XHTML Templates will be focused on the presentation needs and skill
  level of Site Designers.  Highly complex presentation tasks that
  blur into programming tasks will be left to the domain of XSLT
  Templates.  Thus, the XHTML presentation format is scoped by
  focusing on usability for the Site Designer, non-programmer
  audience.  XSLT Templates will be aimed at XML professionals and at
  highly-complex presentation layout needs.

  The XHTML Template will not do transforms.  If a Site Designer needs
  to enforce any rules on markup in content, some other step will have
  to be taken to ensure the Site Designer is handed acceptible,
  "stripped" content.

  DTML has permitted the rendering on non-markup such as SMTP
  messages.  The XHTML Template does not try to satisfy this job.
  Other objects, such as DTML Methods or Python Methods, would have to
  handle this chore.

  Clarifying the "DTML namespace" confusion felt by authors is not in
  the scope of this proposal.

  While providing and documenting integration with a few important
  tools will be done, this project won't integrate every tool.

  Later projects will deal with the rules that allow implicit binding
  of presentations to resources.

Risks

  *Role of DTML*.  What is the role, if any, of DTML in the future?

  *Replacing magic with magic*.  If too much magical stuff happens
   (e.g. how data gets implicitly bound in different ways to
   presentation, the component idea creating new stuff in the
   background) for people to handle, we're just replacing the
   confusing old magic with the confusing new magic.

Deliverables

  The first deliverable will be rich user documentation.  This will
  supplement use cases.

  The second deliverable will be a documented API and explanatory text
  to go along with the API.

  The third deliverable will be an alpha-quality implementation on a
  branch.