[Zope3-dev] Twisted as server

[Shane Hathaway]

On Mon, 11 Feb 2002 sean.upton@uniontrib.com wrote:

> Isn't Twisted an asynchronous framework servicing multiple requests with an
> event loop?  I don't claim to know much about Twisted, but I would think
> that it wouldn't work to integrate a threaded HTTP server into Twisted due
> to architectural differences at a core level?

You're thinking in the right direction.  Zope fundamentally requires
threading since any request might take 1 millisecond or it might take 1
hour.  But event-driven communication tends to be more efficient than
thread-bound communication, especially when there are a lot of slow
connections.

So Zope2's ZServer tries to bridge the gap.  It has an event-driven core
that drives application threads.  The strategy is largely
successful--ZServer is capable of serving 10 fast, hungry clients as well
as 1000 slow clients, even though there are normally only 4 application
threads.

ZServer has some very rough edges, though.  Its code is difficult to read
so it's hard to say how well it buffers data, how quickly it can push data
out the socket, whether it complies with standards, and whether it ever
blocks while waiting for I/O.

So Zope3 has a new HTTP server with all new code that uses ideas from
ZServer.  The new server operates in *both* asynchronous and threaded
mode, which allows application threads to push data out without waiting
for the main loop.  There are unit tests that verify standards support,
verify the server opens temp files when necessary instead of flooding RAM,
verify correct pipelining, check for leaked sockets, and other things.

I don't think Twisted has this kind of capability.  But I suggest that
Twisted *could* have these capabilities.  It would take some refactoring
because a single-threaded event loop is not the right strategy for Zope in
general.  I would be happy to explain how to start.

> It sounds like it might be nice to see someone implement a Twisted HTTP
> interface to Z3X, and compare its performance alongside what already
> exists...  And the option to use either might be appealing too - if nothing
> other than to use Twisted for SMTP/NNTP/IRC/IM stuff... I can think of uses
> for NNTP and IM applications I could integrate with Zope already...

There already is a Twisted interface to Zope 3X.  It's in the Twisted
package.  You can compare them, but be careful not to just use "ab" since
that won't tell you what happens when a request comes in that takes a long
time but not a lot of CPU.

Shane