[ZODB-Dev] ZEO ClientStorage's use of the asyncore main loop (was [Fwd: Re: [Zope3-dev] Twisted Publisher and Zope 2])

[Jim Fulton]

Here is a draft of a note I plan to send to the zodb list soon:

The ZEO client storage currently relies on a host application to
provide an ayncore main loop.  This causes two major headaches:

1. The ZEO client storage has to have two modes of operation,
     depending on whether the host application's main loop has started.

2. The ZEO client storage doesn't work optimally if there isn't a
     main loop.  In particular, without an asyncore main loop, a
     ZEO client storage won't see messages from the ZEO server until
     the client storage tries to read or write some data from or to
     the ZEO storage server.

Starting in this month's release, Zope 3 is moving away from the
asyncore-based "ZServer" network (HTTP, FTP) server architecture.
I expect Zope 2 to follow in the June release.  This means that Zope
no-longer provides the main loop that the ZEO client storage expects.

Tim and I have discussed this for some time.  We think an
asynchronous I/O approach is still appropriate, to handle
asynchronous messages from servers to clients, but we need
to get away from expecting a server to provide the asyncore
main loop needed by ZEO. Rather, ZEO should provide its own
main loop running in a separate thread. (Or, possibly, each
client storage should have its own thread.) Among other things,
this would allow us to get rid of the horribly complex dual-mode
code we have now.

This should be a project for the next development cycle feeding
the June release.

In the short term, I'll probably have to add some Zope 3.2 code
to start an asyncore main loop for ZEO's use.

Thoughts?

(Note that we've had lots of frustration with asyncore.  It is
    tempting to get rid of that at the same time.  However, we still
    need an asynchronous I/O framework.  This would probably requite
    a switch to Twisted.  This might be worthwhile, but would almost
    certainly entail a lot of risk.  I'm not at all sure the benefit
    is worth it unless there were some able volunteers who wanted to
    work on it.)

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org


-------------- next part --------------
An embedded message was scrubbed...
From: =?windows-1252?Q?Dario_Lopez-K=E4sten?= <dario at ita.chalmers.se>
Subject: Re: [Zope3-dev] Twisted Publisher and Zope 2
Date: Thu, 08 Dec 2005 14:45:41 +0100
Size: 4450
Url: http://mail.zope.org/pipermail/zodb-dev/attachments/20051208/e9a47edb/AttachedMessage.eml