[Zope3-dev] Mail delivery failed: returning message to sender
Mail Delivery System
Mailer-Daemon at python.org
Tue Feb 17 07:58:06 EST 2004
This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:
corey at streamreel.net
SMTP error from remote mailer after RCPT TO:<corey at streamreel.net>:
host iris2.directnic.com [204.251.10.82]: 550 5.7.1 No such recipient
------ This is a copy of the message, including all the headers. ------
Return-path: <zope3-dev at zope.org>
Received: from cache1.zope.org ([12.155.117.38])
by mail.python.org with esmtp (Exim 4.22)
id 1At4my-00007m-5E; Tue, 17 Feb 2004 07:57:24 -0500
From: zope3-dev at zope.org (srichter)
Reply-To: zope3-dev at zope.org
To: ;
Subject: [LifeOfRequest] (new) first draft (generated from LaTeX file)
Message-ID: <20040217075724EST at dev.zope.org>
X-BeenThere: zope3-dev at zope.org
X-Zwiki-Version: 0.21.1
Precedence: bulk
List-Id: <zope3-dev at zope.org>
List-Post: <mailto:zope3-dev at zope.org>
List-Subscribe: <http://dev.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/LifeOfRequest/subscribeform>
List-Unsubscribe: <http://dev.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/LifeOfRequest/subscribeform>
List-Archive: <http://dev.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/LifeOfRequest>
List-Help: <http://dev.zope.org/Wikis/DevSite/Projects/ComponentArchitecture>
Date: Tue, 17 Feb 2004 07:57:24 -0500
X-Spam-Status: OK (default 0.000)
The Life of a Request
Status
IsDraft
Authors
StephanRichter
Difficulty
Contributor Level
Skills
- You should be well-familiar with the Zope 3 framework.
- Having a general idea of internet server design is also very
helpful.
Problem/Task
When developing Zope 3 applications, the coder is commonly dealing
with the 'request' object to create views without thinking much
about the details on how the request gets into the view and what happens
with the response that is constructed in it. And this is fine, since it
is often not necessary to know. But sometimes one needs to write a custom
server or change the behavior of the publisher. In these cases it is good
to know the general design of the Zope servers and publishers. This
recipe takes you on the journey through the life of a request using the
browser (special HTTP) request as an example.
Recipe
What is a Request
The term "request" appears often when talking about Zope 3. But what
is a request? In technically concrete situations we usually refer to
objects that implement 'IRequest' . These objects are responsible to
embed protocol-specific details and represent the protocol semantics to
provide them for usage by presentation components.
It is not enough to only think of request objects. For me, anything
that the client sends to the server after connection negotiations is
considered a request. So on the very lowest level, when the user agent
(in this case the classic Web browser) sends the HTTP string::
GET /index.html HTTP/1.1
it could be considered a request (raw) as well.
Finding the Origin of the Request
Now that we have an idea what the request is, let's take a more
technical approach and find out how connections are handled technically
and how a request is born in the midst of much redirection and many
confusing abstraction layers.
When a server starts up, it binds a socket to an address on the
local machine 'bind(address)' ) and then starts to listen for
connections by calling 'listen(backlog)' . When an incoming
connection is detected, the 'accept()' method is called, which
returns a connection object and the address of the computer to which the
connection was made. All of this is part of the standard Python 'socket'
library and is documented in 'zope.server.interfaces.ISocket'
interface.
The server, which is mainly an 'IDispatcher' implementation, has a
simple interface to handle the specific events, by calling its
corresponding 'handle' method. A complete list of all events that
are managed this way is given in the 'IDisplatcherEventHandler'
interface. So when a connection comes in, 'handle' is called, which
is overridden in the 'zope.server.serverbase.ServerBase' class
around line 130. This method tries to get the connection by calling 'accept()'
(see previous paragraph). If the connection was successfully created, it
is used to create a 'ServerChannel' , which is the next level of
abstraction. Most of the other dispatcher functionality is provided by
the standard 'async.dispatcher' , which fully implements 'IDispatcher'
.
At this stage the channel is taking over, which is just another
dispatcher. So the channel starts to collect the incoming data (see 'received(data)'
) and sends it right away to the request parser, which is an instance of
a class specified as 'parser' . Obviously, this class will be
different for every server implementation. The way the parser functions
is not that important. All we have to know is that it implements 'IStreamConsumer'
, which has a way of saying when it has completed parsing a request. Once
all of the data is received, the 'IServerChannel' method 'receivedCompleteRequest(req)'
is called, whose goal is to schedule the request to be executed. But only
one request of the channel can be running at a time. So if the cahnnel is
busy, then we need to queue the request, until the running task is
completed.
Whenever the channel becomes available (see 'end' , the next request
from the queue is taken and convered to an 'ITask' object. The task
is then immediately sent to the server for execution using 'IServer.addTask(task)'
. There it is added to a task dispatcher (see 'ITaskDispatcher' ),
which schedules the task for execution. But why all this redirection
through a task and a task dispatcher? Until now, all the code ran on a
single thread. But in order to scale the servers better and to support
long-running requests without blocking the entire server, it is necessary
to be able to start several threads to handle the requests. It is now up
to the 'ITaskDispatcher' implementation to decide how to spread the
requests. Theoretically, it could even consult other computers to execute
a task. By default, we use the 'zope.server.taskthreads.ThreadedTaskDispatcher'
though. Using its 'setThreadCount(count)' method, the Zope startup
code is able specify the maximum amount of threads running at a time.
Once, it is the task's turn to be serviced, the task dispatcher
calls 'ITask.service()' which should finally execute the request.
Specifically, when the 'HTTPTask' is serviced, the method 'executeRequest(task)'
of the 'HTTPServer' is called. The 'zope.server.http.publisherhttpserver.PublisherHTTPServer'
, which is the one used for Zope 3, creates a 'IHTTPRequest' object
from the task and publishes the request with 'zope.publisher.publish(request)'
. The server has an attribute 'request' in which the request class
that is used to create the request is stored.
So what did the system accomplish so far? We have taken an incoming
connection, read all the incoming data and parsed it, scheduled it for
execution and finally created a request that was published with the Zope
3 publisher. Except for the last step, there was nothing Zope-specific
about this code, so that all of this could be replaced by any other Web
server, like Twisted.
** Diagram of interactions and sequence diagram. **
The Request and the Publisher
With the birth of the request and its start to walk the path of a
request's life by entering the publisher using 'zope.publisher.publish.publish()'
, it also enters a Zope-pure domain. In fact, Zope does not really care
how a request was created, as long as it implements 'IRequest' . For
example, when functional tests are executed, they create the request
purely from fictional data and pass it through the publisher to analyze
the response afterwards.
From a high-level point of view, the publisher's 'publish()' method
is responsible of interpreting the request's information and case the
correct actions. It starts out by traversing the given object path to an
actual object, then call the desired method and finally writing the
result in the response. Everything else in this method is about handling
errors and exceptions as well as providing enough hooks for other
components to step in.
The 'publish()' method is so central to the entire Zope 3 framework,
that we will now go through it very carefully trying to understand each
step's purpose. Whereever necessary we will take a rest and examine side
paths closer. It might be of advantage to open the 'zope.publisher.publish'
module at this point, so that it is easier to follow the text.
The work of the 'publish()' method starts with a infinite
while-loop. The first step inside the loop is to get the publication.
The publication provides the publisher with hooks to accomplish
application-specific tasks, related to data storages, transactions and
security. The default implementation is 'DefaultPublication' , which
is located in 'zope.publisher.base' and can be used by software that
do not make use of the entire Zope framework. For Zope 3, however, there
is a specific Zope implementation in 'zope.app.publication.zopepublication'
.
Now, wrapped inside three 'try/except' statements, we tell the
request to look at its data and process what every needs to be processed.
In the case of a browser request, like the one we use as example, the 'processInputs()'
tries to parse all HTML form data that might have been sent by the
browser and convert it to Python objects.
The next step is to convert the request's object path to an object,
a process known as traversal. Besides calling all the event-hooks, the
first step of the traversal processs is to determine the application or,
in other words, the object root. For a common Zope 3 installation, the
application is of course the root of the ZODB. Then we use the request's
'traverse(object)' method to get to the desired object. Let's have a
closer look at this method for the 'BrowserRequest' .
First of all we notice that the 'BrowserRequest' 's traverse method
does not do any of the heavy lifting, but only covers a few
browser-specific corner cases, like picking a default view and using the
HTML form data (by inspecting form variable names for the suffix
":method") for possible additional traversal steps. It turns out that the
'BaseRequest' 's traverse method does all the work. At the beginning
of the method there are several private attributes that are being pulled
into the local namespace and setup.
-'publication' : It is simply the 'publication' object from
before, which gives us access to the application-specific
functionality.
-'_' : A simple stack (i.e. list) of names that must be traversed.
These names came from the parsed path of the URL. For example
"/path/to/foo/bar/index.html" would be parsed to '['path', 'to', 'foo', 'bar', 'index.html]'
.
-'_' : This is a list of names that have been already successfully
traversed. The names are simply the entries coming from '_' .
-'_' : This variable keeps track of the last object that was found
in the traversal process.
Now we just work through the traversal stack until it has been
completely emptied. The intersting call here is the 'publication.traverseName(request, object, name'
) which tries to determine the next object using the name from the
traversal stack and the request. The 'traverseName()' method can be
very complex. The Zope 3 application version, found in 'zope.app.publication.publicationtraverse.PublicationTraverse'
, must be able to handle namespaces ("++namespace++"), views ("@@") and
pluggable traverser lookups, so that objects can implement their own
traversers. To discuss the details of this method be beyond the goal of
this recipe.
If everything goes well, and no exception was raised, meaning that
the object specified in the path was found, the 'traverse()' method
returns the found object and we are back in the publisher's 'publish()'
function. The next step is to execute the object.
Calling the object assumes that the object is callable in the first
place. Therefore, the traversal process should always end in a view or a
method on a view. But since all common content objects have
browser-specific default views, we are guaranteed that the object is
callable. For other presentation types, similar default options exist.
Even though the object is formally executed by calling 'publication.callObject(request, object)'
, eventually 'mapply()' is called, which is defined in the 'zope.publisher.publish'
module. 'mapply()' does not just call the object, but takes great
care of determining the argument and finding values for them.
When an object is called, it can either write the result directly to
the request's response object or return a result object. In the latter
case, the 'publish()' method adds the result to the body of the
response. Here it is assumed that the result object is a unicode string.
For the Zope application the 'afterCall(request)' execution is of
importance, since it commits the transaction to the ZODB. This process
can can cause a failure, so it is very important that we do not return
any data to the server until the transaction is committed.
When all this has successfully finished, we call 'outputBody()' on
the response, which sends the data out to the world going through the
tsak, channel and eventually through the socket to the connected machine.
Note that the 'output(data)' method, which is called from 'outputBody'
, is responsible for converting the internally kept unicode strings to
valid ASCII using an encoding. If no encoding was specified, "UTF-8" is
used by default.
Once the response has sent out its data, the request is closed by
calling 'close()' on itself, which releases all locks on resources.
This will also finish the running task, close the channel and eventually
disconnect the socket. This marks the end of the request.
Let's now look at some of the possible failure scenarios. The most
common failure is a ZODB write conflict, in which case we simply want to
rollback the transaction and retry the request again. But where does the
'Retry' error come from, when the ZODB raises a 'ConflictError' ? A
quick look in the publication's 'handleException()' method reveals,
that if a write conflict error is detected, it is logged and afterwards a
'Retry' exception is raised, so that the next exception handler is
used. Here we simply reset the request and the response and allow the
pulishing process to start all over again (remember, we have an
everlasting while loop over all of this code).
In general, though, exceptions are handled by the 'handleException()'
method, which logs the error and even allows them to be formatted in the
appropriate output format using a view. See the recipe on "Changing
Standard Exception Views" for details on how to define your own views on
exceptions.
** Diagram of interactions and sequence diagram. **
This concludes our journey through the life of a request. Sometimes
I intentionally ignored details to stay focused and not confuse the
reader. The interested reader will find that the interfaces of the
various involved components serve well as further documentation,
especially for the publisher.
--
forwarded from http://dev.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/LifeOfRequest
More information about the Zope3-dev
mailing list