[Zope] Important Fix for Zope 2.2.x

Evan Simpson evan@digicool.com
Sun, 10 Dec 2000 11:19:29 -0500


This is a multi-part message in MIME format.

------=_NextPart_000_0052_01C0629B.1033D860
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

Thanks to Jeff Ragsdale, we've finally been able to kill a
longstanding bug that allows POST requests to interfere with
each other.  Symptoms include corrupted or aborted File and
Image uploads, and stupid-log messages about
"AttributeError: data" killing threads.

The attached HTTPServer.py is valid for all Zope 2.2.x
versions.  I am posting the patched file for earlier Zope
versions separately.

PLEASE BACK UP <Zope>/ZServer/HTTPServer.py, then replace it
with the attached file.

Cheers,

Evan @ digicool & 4-am

------=_NextPart_000_0052_01C0629B.1033D860
Content-Type: text/plain;
	name="HTTPServer.py"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filename="HTTPServer.py"

#########################################################################=
#####=0A=
# =0A=
# Zope Public License (ZPL) Version 1.0=0A=
# -------------------------------------=0A=
# =0A=
# Copyright (c) Digital Creations.  All rights reserved.=0A=
# =0A=
# This license has been certified as Open Source(tm).=0A=
# =0A=
# Redistribution and use in source and binary forms, with or without=0A=
# modification, are permitted provided that the following conditions are=0A=
# met:=0A=
# =0A=
# 1. Redistributions in source code must retain the above copyright=0A=
#    notice, this list of conditions, and the following disclaimer.=0A=
# =0A=
# 2. Redistributions in binary form must reproduce the above copyright=0A=
#    notice, this list of conditions, and the following disclaimer in=0A=
#    the documentation and/or other materials provided with the=0A=
#    distribution.=0A=
# =0A=
# 3. Digital Creations requests that attribution be given to Zope=0A=
#    in any manner possible. Zope includes a "Powered by Zope"=0A=
#    button that is installed by default. While it is not a license=0A=
#    violation to remove this button, it is requested that the=0A=
#    attribution remain. A significant investment has been put=0A=
#    into Zope, and this effort will continue if the Zope community=0A=
#    continues to grow. This is one way to assure that growth.=0A=
# =0A=
# 4. All advertising materials and documentation mentioning=0A=
#    features derived from or use of this software must display=0A=
#    the following acknowledgement:=0A=
# =0A=
#      "This product includes software developed by Digital Creations=0A=
#      for use in the Z Object Publishing Environment=0A=
#      (http://www.zope.org/)."=0A=
# =0A=
#    In the event that the product being advertised includes an=0A=
#    intact Zope distribution (with copyright and license included)=0A=
#    then this clause is waived.=0A=
# =0A=
# 5. Names associated with Zope or Digital Creations must not be used to=0A=
#    endorse or promote products derived from this software without=0A=
#    prior written permission from Digital Creations.=0A=
# =0A=
# 6. Modified redistributions of any form whatsoever must retain=0A=
#    the following acknowledgment:=0A=
# =0A=
#      "This product includes software developed by Digital Creations=0A=
#      for use in the Z Object Publishing Environment=0A=
#      (http://www.zope.org/)."=0A=
# =0A=
#    Intact (re-)distributions of any official Zope release do not=0A=
#    require an external acknowledgement.=0A=
# =0A=
# 7. Modifications are encouraged but must be packaged separately as=0A=
#    patches to official Zope releases.  Distributions that do not=0A=
#    clearly separate the patches from the original work must be clearly=0A=
#    labeled as unofficial distributions.  Modifications which do not=0A=
#    carry the name Zope may be packaged in any form, as long as they=0A=
#    conform to all of the clauses above.=0A=
# =0A=
# =0A=
# Disclaimer=0A=
# =0A=
#   THIS SOFTWARE IS PROVIDED BY DIGITAL CREATIONS ``AS IS'' AND ANY=0A=
#   EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE=0A=
#   IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR=0A=
#   PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL DIGITAL CREATIONS OR ITS=0A=
#   CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,=0A=
#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT=0A=
#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF=0A=
#   USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND=0A=
#   ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,=0A=
#   OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT=0A=
#   OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF=0A=
#   SUCH DAMAGE.=0A=
# =0A=
# =0A=
# This software consists of contributions made by Digital Creations and=0A=
# many individuals on behalf of Digital Creations.  Specific=0A=
# attributions are listed in the accompanying credits file.=0A=
# =0A=
#########################################################################=
#####=0A=
=0A=
"""=0A=
Medusa HTTP server for Zope=0A=
=0A=
changes from Medusa's http_server=0A=
=0A=
    Request Threads -- Requests are processed by threads from a thread=0A=
    pool.=0A=
    =0A=
    Output Handling -- Output is pushed directly into the producer=0A=
    fifo by the request-handling thread. The HTTP server does not do=0A=
    any post-processing such as chunking.=0A=
=0A=
    Pipelineable -- This is needed for protocols such as HTTP/1.1 in=0A=
    which mutiple requests come in on the same channel, before=0A=
    responses are sent back. When requests are pipelined, the client=0A=
    doesn't wait for the response before sending another request. The=0A=
    server must ensure that responses are sent back in the same order=0A=
    as requests are received.=0A=
    =0A=
""" =0A=
import sys=0A=
import regex=0A=
import string=0A=
import os=0A=
import types=0A=
import thread=0A=
import time=0A=
from cStringIO import StringIO=0A=
=0A=
from PubCore import handle=0A=
from HTTPResponse import make_response=0A=
from ZPublisher.HTTPRequest import HTTPRequest=0A=
=0A=
from medusa.http_server import http_server, http_channel=0A=
from medusa import counter, producers, asyncore, max_sockets=0A=
from medusa.default_handler import split_path, unquote, get_header=0A=
from medusa.asyncore import compact_traceback, dispatcher=0A=
=0A=
from ZServer import CONNECTION_LIMIT, ZOPE_VERSION, ZSERVER_VERSION=0A=
=0A=
from zLOG import LOG, register_subsystem, BLATHER, INFO, WARNING, ERROR=0A=
import DebugLogger=0A=
=0A=
register_subsystem('ZServer HTTPServer')=0A=
=0A=
CONTENT_LENGTH =3D regex.compile('Content-Length: =
\([0-9]+\)',regex.casefold)=0A=
CONNECTION =3D regex.compile ('Connection: \(.*\)', regex.casefold)=0A=
=0A=
# maps request some headers to environment variables.=0A=
# (those that don't start with 'HTTP_')=0A=
header2env=3D{'content-length'    : 'CONTENT_LENGTH',=0A=
            'content-type'      : 'CONTENT_TYPE',=0A=
            'connection'        : 'CONNECTION_TYPE',=0A=
            }=0A=
=0A=
class zhttp_collector:=0A=
    def __init__(self, handler, request, size):=0A=
        self.handler =3D handler=0A=
        self.request =3D request=0A=
        if size > 524288:=0A=
            # write large upload data to a file=0A=
            from tempfile import TemporaryFile=0A=
            self.data =3D TemporaryFile('w+b')=0A=
        else:=0A=
            self.data =3D StringIO()=0A=
        request.channel.set_terminator(size)=0A=
        request.collector=3Dself=0A=
=0A=
    # put and post collection methods=0A=
    #=0A=
    def collect_incoming_data (self, data):=0A=
        self.data.write(data)=0A=
=0A=
    def found_terminator(self):=0A=
        # reset collector=0A=
        self.request.channel.set_terminator('\r\n\r\n')=0A=
        self.request.collector=3DNone=0A=
        # finish request=0A=
        self.data.seek(0)=0A=
        r=3Dself.request=0A=
        d=3Dself.data=0A=
        del self.request=0A=
        del self.data=0A=
        self.handler.continue_request(d,r)=0A=
=0A=
class zhttp_handler:=0A=
    "A medusa style handler for zhttp_server"=0A=
        =0A=
    def __init__ (self, module, uri_base=3DNone, env=3DNone):=0A=
        """Creates a zope_handler=0A=
        =0A=
        module -- string, the name of the module to publish=0A=
        uri_base -- string, the base uri of the published module=0A=
                    defaults to '/<module name>' if not given.=0A=
        env -- dictionary, environment variables to be overridden.       =
 =0A=
                    Replaces standard variables with supplied ones.=0A=
        """=0A=
        =0A=
        self.module_name=3Dmodule=0A=
        self.env_override=3Denv or {}=0A=
        self.hits =3D counter.counter()=0A=
        # if uri_base is unspecified, assume it=0A=
        # starts with the published module name=0A=
        #=0A=
        if uri_base is None:=0A=
            uri_base=3D'/%s' % module=0A=
        elif uri_base =3D=3D '':=0A=
            uri_base=3D'/'=0A=
        else:=0A=
            if uri_base[0] !=3D '/':=0A=
              uri_base=3D'/'+uri_base=0A=
            if uri_base[-1] =3D=3D '/':=0A=
              uri_base=3Duri_base[:-1]=0A=
        self.uri_base=3Duri_base=0A=
        uri_regex=3D'%s.*' % self.uri_base=0A=
        self.uri_regex =3D regex.compile(uri_regex)=0A=
=0A=
    def match(self, request):=0A=
        uri =3D request.uri=0A=
        if self.uri_regex.match(uri) =3D=3D len(uri):=0A=
            return 1=0A=
        else:=0A=
            return 0=0A=
=0A=
    def handle_request(self,request):=0A=
        self.hits.increment()=0A=
=0A=
        DebugLogger.log('B', id(request), '%s %s' % =
(string.upper(request.command), request.uri))=0A=
=0A=
        size=3Dget_header(CONTENT_LENGTH, request.header)=0A=
        if size and size !=3D '0':=0A=
            size=3Dstring.atoi(size)=0A=
            zhttp_collector(self, request, size)=0A=
        else:=0A=
            sin=3DStringIO()=0A=
            self.continue_request(sin,request)=0A=
=0A=
    def get_environment(self, request,=0A=
                        # These are strictly performance hackery...=0A=
                        split=3Dstring.split,=0A=
                        strip=3Dstring.strip,=0A=
                        join =3Dstring.join,=0A=
                        upper=3Dstring.upper,=0A=
                        lower=3Dstring.lower,=0A=
                        h2ehas=3Dheader2env.has_key,=0A=
                        h2eget=3Dheader2env.get,=0A=
                        workdir=3Dos.getcwd(),=0A=
                        ospath=3Dos.path,=0A=
                        ):=0A=
        [path, params, query, fragment] =3D split_path(request.uri)=0A=
        while path and path[0] =3D=3D '/':=0A=
            path =3D path[1:]=0A=
        if '%' in path:=0A=
            path =3D unquote(path)=0A=
        if query:=0A=
            # ZPublisher doesn't want the leading '?'=0A=
            query =3D query[1:]=0A=
=0A=
        server=3Drequest.channel.server=0A=
        env =3D {}=0A=
        env['REQUEST_METHOD']=3Dupper(request.command)=0A=
        env['SERVER_PORT']=3Dstr(server.port)=0A=
        env['SERVER_NAME']=3Dserver.server_name=0A=
        env['SERVER_SOFTWARE']=3Dserver.SERVER_IDENT=0A=
        env['SERVER_PROTOCOL']=3Drequest.version=0A=
        env['channel.creation_time']=3Drequest.channel.creation_time=0A=
        if self.uri_base=3D=3D'/':=0A=
            env['SCRIPT_NAME']=3D''=0A=
            env['PATH_INFO']=3D'/' + path=0A=
        else:=0A=
            env['SCRIPT_NAME'] =3D self.uri_base=0A=
            try:=0A=
                path_info=3Dsplit(path,self.uri_base[1:],1)[1]=0A=
            except:=0A=
                path_info=3D''=0A=
            env['PATH_INFO']=3Dpath_info=0A=
        env['PATH_TRANSLATED']=3Dospath.normpath(ospath.join(=0A=
                workdir, env['PATH_INFO']))=0A=
        if query:=0A=
            env['QUERY_STRING'] =3D query=0A=
        env['GATEWAY_INTERFACE']=3D'CGI/1.1'=0A=
        env['REMOTE_ADDR']=3Drequest.channel.addr[0]=0A=
=0A=
        # If we're using a resolving logger, try to get the=0A=
        # remote host from the resolver's cache.=0A=
        if hasattr(server.logger, 'resolver'):=0A=
            dns_cache=3Dserver.logger.resolver.cache=0A=
            if dns_cache.has_key(env['REMOTE_ADDR']):=0A=
                remote_host=3Ddns_cache[env['REMOTE_ADDR']][2]=0A=
                if remote_host is not None:=0A=
                    env['REMOTE_HOST']=3Dremote_host=0A=
=0A=
        env_has=3Denv.has_key=0A=
        for header in request.header:=0A=
            key,value=3Dsplit(header,":",1)=0A=
            key=3Dlower(key)=0A=
            value=3Dstrip(value)=0A=
            if h2ehas(key) and value:=0A=
                env[h2eget(key)]=3Dvalue=0A=
            else:=0A=
                key=3D'HTTP_%s' % upper(join(split(key, "-"), "_"))=0A=
                if value and not env_has(key):=0A=
                    env[key]=3Dvalue=0A=
        env.update(self.env_override)=0A=
        return env=0A=
=0A=
    def continue_request(self, sin, request):=0A=
        "continue handling request now that we have the stdin"=0A=
       =0A=
        s=3Dget_header(CONTENT_LENGTH, request.header)=0A=
        if s:=0A=
            s=3Dstring.atoi(s)=0A=
        else:=0A=
            s=3D0    =0A=
        DebugLogger.log('I', id(request), s)=0A=
=0A=
        env=3Dself.get_environment(request)=0A=
        zresponse=3Dmake_response(request,env)=0A=
        zrequest=3DHTTPRequest(sin, env, zresponse)=0A=
        request.channel.current_request=3DNone=0A=
        request.channel.queue.append((self.module_name, zrequest, =
zresponse))=0A=
        request.channel.work()=0A=
=0A=
    def status(self):=0A=
        return producers.simple_producer("""=0A=
            <li>Zope Handler=0A=
            <ul>=0A=
            <li><b>Published Module:</b> %s=0A=
            <li><b>Hits:</b> %s=0A=
            </ul>""" %(self.module_name, self.hits)=0A=
            )=0A=
=0A=
=0A=
=0A=
class zhttp_channel(http_channel):=0A=
    "http channel"=0A=
=0A=
    closed=3D0=0A=
    zombie_timeout=3D100*60 # 100 minutes=0A=
    =0A=
    def __init__(self, server, conn, addr):=0A=
        http_channel.__init__(self, server, conn, addr)=0A=
        self.queue=3D[]=0A=
        self.working=3D0=0A=
        =0A=
    def push(self, producer, send=3D1):=0A=
        # this is thread-safe when send is false=0A=
        # note, that strings are not wrapped in =0A=
        # producers by default=0A=
        if self.closed:=0A=
            return=0A=
        self.producer_fifo.push(producer)=0A=
        if send: self.initiate_send()=0A=
        =0A=
    push_with_producer=3Dpush=0A=
=0A=
    def work(self):=0A=
        "try to handle a request"=0A=
        if not self.working:=0A=
            if self.queue:=0A=
                self.working=3D1=0A=
                try: module_name, request, response=3Dself.queue.pop(0)=0A=
                except: return=0A=
                handle(module_name, request, response)=0A=
=0A=
    def close(self):=0A=
        self.closed=3D1=0A=
        while self.queue:=0A=
            self.queue.pop()=0A=
        if self.current_request is not None:=0A=
            self.current_request.channel=3DNone # break circ refs=0A=
            self.current_request=3DNone=0A=
        while self.producer_fifo:=0A=
            p=3Dself.producer_fifo.first()=0A=
            if p is not None and type(p) !=3D types.StringType:=0A=
                p.more() # free up resources held by producer=0A=
            self.producer_fifo.pop()=0A=
        dispatcher.close(self)=0A=
=0A=
    def done(self):=0A=
        "Called when a publishing request is finished"=0A=
        self.working=3D0=0A=
        self.work()=0A=
=0A=
    def kill_zombies(self):=0A=
        now =3D int (time.time())=0A=
        for channel in asyncore.socket_map.values():=0A=
            if channel.__class__ =3D=3D self.__class__:=0A=
                if (now - channel.creation_time) > =
channel.zombie_timeout:=0A=
                    channel.close()=0A=
=0A=
=0A=
class zhttp_server(http_server):    =0A=
    "http server"=0A=
    =0A=
    SERVER_IDENT=3D'Zope/%s ZServer/%s' % (ZOPE_VERSION,ZSERVER_VERSION)=0A=
    =0A=
    channel_class =3D zhttp_channel=0A=
=0A=
    def readable(self):=0A=
        return self.accepting and \=0A=
                len(asyncore.socket_map) < CONNECTION_LIMIT=0A=
=0A=
    def listen(self, num):=0A=
        # override asyncore limits for nt's listen queue size=0A=
        self.accepting =3D 1=0A=
        return self.socket.listen (num)=0A=
=0A=

------=_NextPart_000_0052_01C0629B.1033D860--