[Zope-dev] Handling of ECONNRESET socket error in asyncore.py

[Chris McDonough]

On Sat, 2002-08-24 at 13:47, Oliver Bleutgen wrote:
> Yes, and where does it put his output to? There's no way to get it 
> directly to the browser. Therefore, the typical situation is 
> pdf-generation for later delivery (via email etc.) or whatnot, and that 
> doesn't need to waste one of zope's worker threads, they seem too 
> precious for me.

Well, sometimes there isn't any output or the output is not useful;
there are many times you'd like to run a method for its side effects. 
There's no uniformly sane way to tell whether a caller has called a
method for its side effects or whether they've called it in order to
obtain its results.

> I don't see backwards compatibility compromised. If the job of a method 
> is to send its results to the browser, connection reset means it can't 
> fullfil it's job.

Absolutely.  The trick is being able to determine whether that is the
job of the method or not, or whether the method was called for its side
effects.

It's obvious that more often than not a method is called for its return
value rather than for its side effects.  It's hard to tell when this is
the case, but let's make the leap of faith and assume you could.  You'll
still need to figure out how to tell it to stop processing.

It would seem obvious to say "just send the thread a signal to tell it
to stop processing".  But this isn't possible because signals in Python
are always caught by the main thread. 
(http://www.python.org/doc/lib/module-signal.html).  Additionally, even
if you could send it a signal, there is no sane way to force it to stop
processing.  "Killing" a thread is a feature that Python doesn't have
and may never have:
http://sourceforge.net/tracker/index.php?func=detail&aid=221115&group_id=5470&atid=105470

So, given that we can't kill threads or sanely do much to effect their
execution from the outside, maybe we could make our threads do it
cooperatively.  This would require some deep dark voodoo in the Python
interpreter.  When a Zope worker thread got a hold of the interpreter
lock, it could be instructed to check somewhere for state related to the
its associated connection.  If the state had a flag in it that said
"stop processing", it could be tricked into executing some bytecode that
caused it to raise a non-catchable exception (it would kill itself).  I
don't feel competent enough to write such a beast, but maybe others do.

However, IMHO, it's less work and requires less explanation to make it
the responsibility of the Zope programmer.  Execution could be effected
by requiring him to sprinkle his code with a few checks in opportune
places that caused processing to stop happening if and only if
ECONNRESET had been raised previously.  This is why I proposed the
REQUEST.RESPONSE.isClientConnected() method.  It also has the benefit of
being noninvasive and an add-on rather than a change to existing code.

> If you don't need to send something to the browser, 
> you better should have offloaded the work to a special thread/process 
> anyway. Maybe the administrative tasks you mentioned above could just 
> ignore the connection reset instead of getting stopped, but show me 
> someone who has pressed the stop button, reloaded or went to another 
> page while a database pack was in progress, and I show you a very brave 
> person.

Wow, I must be a superhero then. I always knew I was in this business
for the glory. ;-)

> It's just super counterintuitive that the work continues after I pressed 
> stop.

I agree.  Do you have any suggestions about how to make it stop
processing (conditionally) besides the one I've already given?  I'm out
of ideas.

> I'm not sure why they talk about this, but it might have to do with the 
> special example they give. I.e. checking if the client is connected 
> before kicking of an expensive database query. I think their recipe is 
> to handle the fact that the script also doesn't get stopped immediately 
> when the connection is closed by the client, but that doesn't mean that 
> it runs indefinately.
> 
> As for if I'm sure that apache+mod_perl works different than zope, I 
> tested that myself and wrote it to the list:

OK, sounds reasonable... I think this may be a big win for
multiprocess-model servers rather than threaded servers.  It's easy to
kill a process, but threads are a pain in the ass. ;-)

- C