[ZODB-Dev] Really want to refactor ZEO's networking archiecture

[David Pratt]

Hi Jim and Roderigo! This is encouraging news. I have been putting much 
thought into this also. There is a downside to refactoring ZEO in that 
there are many folks heavily dependent on it and also reasonably happy 
with it.

Secondly, zeo has a specific place and relationship. When I initiated 
the twisted zeo discussion about a week ago, it was with the hope of 
drawing interest to some possible changes. I was thinking about 
security, my own positive experiences with twisted, and how things could 
be modified to bring tighter integration. That said, the general 
experience of twisted and zope has been short (since 3.2) and this is a 
departure that has risks that I respect.

I was encouraged to challenge my own thinking on what may be needed. 
Before committing, I am planning to build and test a few ideas to 
explore this further without confining the scope. I don't know where 
this is leading me just yet.

On the notion of multiple independent loops for twisted, I believe there 
are possibilities, some interested folks, and a patch I found that could 
be evaluated. The work would be murky as Glyph has pointed out - but he 
has offered to review tests for this as well.

Many thanks,

Regards,
David


Jim Fulton wrote:
> 
> I think that my next big project might be to
> refactor ZEO's networking architecture.
> 
> Wonder what the main reason is?
> 
> - Provide more secure connections? Nope
> - Leverage Twisted? Nope
> - Get rid if the insane async/sync client madness? Nope
> - Get better storage-server performance? Nope
> - Allow simultaneous syncronous calls from the same client? Nope
> 
> I'd like to do all of these things, but the main reason to refactor
> ZEO's networking architecture is to make it testable.
> 
> Writing ZEO tests now requires actually starting servers and
> clients.  This is nuts.  IMO, application code shouldn't
> touch sockets.  Application code, like ZODB and ZEO should
> be insulated from actual network APIs.  They use simpler APIs
> that are easy to interccept and control.
> 
> I've been thinking of a someone general networking API with these aims
> in mind, but I realized that the ZEO existing frameworks could probably
> be refactored in a more limited way to do this.  We'll see.
> 
> If anyone is interested in workingon this with me, let me know.
> 
> Jim
>