[Zope3-checkins] SVN: Zope3/branches/3.2/ Added new machinery that allows published methods to just return

Jim Fulton jim at zope.com
Fri Dec 23 14:51:23 EST 2005


Log message for revision 41002:
  Added new machinery that allows published methods to just return
  files.
  
  Also:
  
  - IResult is now a private interface
  
  - When we look up an IResult, we use a multi-adapter call with the
      request.  This means that result adapters have access to the
      request and response, which would allow a significant
      simplification of the result API.  This is why we made it
      private now, so we can change it later.
  

Changed:
  U   Zope3/branches/3.2/doc/CHANGES.txt
  U   Zope3/branches/3.2/src/zope/app/configure.zcml
  U   Zope3/branches/3.2/src/zope/app/publisher/http.zcml
  A   Zope3/branches/3.2/src/zope/app/wsgi/configure.zcml
  A   Zope3/branches/3.2/src/zope/app/wsgi/fileresult.py
  A   Zope3/branches/3.2/src/zope/app/wsgi/fileresult.txt
  U   Zope3/branches/3.2/src/zope/app/wsgi/tests.py
  U   Zope3/branches/3.2/src/zope/publisher/http.py
  U   Zope3/branches/3.2/src/zope/publisher/httpresults.txt
  U   Zope3/branches/3.2/src/zope/publisher/interfaces/http.py

-=-
Modified: Zope3/branches/3.2/doc/CHANGES.txt
===================================================================
--- Zope3/branches/3.2/doc/CHANGES.txt	2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/doc/CHANGES.txt	2005-12-23 19:51:22 UTC (rev 41002)
@@ -110,32 +110,19 @@
 
       - addMenuItem directive supports a `layer` attribute.
 
-      - Added a new API, zope.publisher.interfaces.http.IResult. See
-        the file httpresults.txt in the zope.publisher package for
-        details.
+      - Changed the Publisher Response API.
 
-      - Formalized the Publisher Response API.
+        + Large results can now ne handled efeciently by returning
+          files rather than strings.  See the file httpresults.txt in
+          the zope.publisher package.
 
-        + Until now the publisher made assumptions about the form of ouput of
-          a publishing process. Either the called method returned a string
-          (regular or unicode) or the response's write() method was used
-          directly to write the data. Those models do not work well with some
-          protocols. Thus, now the publisher deals with result objects. Those
-          are generally not well defined, but for HTTP they must implement the
-          IResult interface.
+        + The unused response.write method is no-longer supported.
 
         + HTTP responses provide two new methods that make reading the output
           easier: `consumeBody()` and `consumeBodyIter()`. Either method can
           be only called once. After that the output iterator is used and
           empty.
 
-        + The WSGI specification specifically has some provisions in it that
-          supported our use of writing directly to the output stream. However,
-          this method of providing an output is strongly discouraged. Instead,
-          the application should return an iterable. Using the new IResult
-          implementation in the HTTP publisher, we can now return such an
-          iterable.
-
         + When a retry is issued in the publisher, then a new request is
           created. This means that the request (including its response) that
           were passed into `publish()` are not necessarily the same that are

Modified: Zope3/branches/3.2/src/zope/app/configure.zcml
===================================================================
--- Zope3/branches/3.2/src/zope/app/configure.zcml	2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/app/configure.zcml	2005-12-23 19:51:22 UTC (rev 41002)
@@ -71,6 +71,7 @@
   <include package="zope.app.applicationcontrol" />
   <include package="zope.app.dublincore" />
   <include package="zope.app.introspector" />
+  <include package="zope.app.wsgi" />
 
 
   <!-- Content types -->

Modified: Zope3/branches/3.2/src/zope/app/publisher/http.zcml
===================================================================
--- Zope3/branches/3.2/src/zope/app/publisher/http.zcml	2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/app/publisher/http.zcml	2005-12-23 19:51:22 UTC (rev 41002)
@@ -15,7 +15,7 @@
   </content>
 
   <class class="zope.publisher.http.DirectResult">
-    <allow interface="zope.publisher.interfaces.http.IResult" />
+    <allow interface="zope.publisher.http.IResult" />
   </class>
 
 </configure>

Added: Zope3/branches/3.2/src/zope/app/wsgi/configure.zcml
===================================================================
--- Zope3/branches/3.2/src/zope/app/wsgi/configure.zcml	2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/app/wsgi/configure.zcml	2005-12-23 19:51:22 UTC (rev 41002)
@@ -0,0 +1,5 @@
+<configure xmlns="http://namespaces.zope.org/zope">
+   <adapter factory=".fileresult.FileResult" />
+   <adapter factory=".fileresult.TemporaryFileResult" />
+</configure>
+


Property changes on: Zope3/branches/3.2/src/zope/app/wsgi/configure.zcml
___________________________________________________________________
Name: svn:eol-style
   + native

Added: Zope3/branches/3.2/src/zope/app/wsgi/fileresult.py
===================================================================
--- Zope3/branches/3.2/src/zope/app/wsgi/fileresult.py	2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/app/wsgi/fileresult.py	2005-12-23 19:51:22 UTC (rev 41002)
@@ -0,0 +1,74 @@
+##############################################################################
+#
+# Copyright (c) 2005 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""IResult adapters for files.
+
+$Id$
+"""
+
+import tempfile
+from zope import component, interface
+import zope.publisher.interfaces.http
+import zope.publisher.http
+from zope.publisher.http import DirectResult
+from zope.security.proxy import removeSecurityProxy
+
+class FallbackWrapper:
+
+    def __init__(self, f):
+        self.close = f.close
+        self._file = f
+
+    def __iter__(self):
+        f = self._file
+        while 1:
+            v = f.read(32768)
+            if v:
+                yield v
+            else:
+                break
+
+ at component.adapter(file, zope.publisher.interfaces.http.IHTTPRequest)
+ at interface.implementer(zope.publisher.http.IResult)
+def FileResult(f, request):
+    f = removeSecurityProxy(f)
+    headers = ()
+    if request.response.getHeader('content-length') is None:
+        f.seek(0, 2)
+        size = f.tell()
+        f.seek(0)
+        headers += (('Content-Length', str(size)), )
+        
+    wrapper = request.environment.get('wsgi.file_wrapper')
+    if wrapper is not None:
+        f = wrapper(f)
+    else:
+        f = FallbackWrapper(f)
+    return DirectResult(f, headers)
+
+# We need to provide an adapter for temporary files *if* they are different
+# than regular files. Whether they are is system dependent. Sigh.
+# If temporary files are the same type, we'll create a fake type just
+# to make the registration work.
+_tfile = tempfile.TemporaryFile()
+_tfile.close()
+_tfile = _tfile.__class__
+if _tfile is file:
+    # need a fake one. Sigh
+    class _tfile:
+        pass
+
+ at component.adapter(_tfile, zope.publisher.interfaces.http.IHTTPRequest)
+ at interface.implementer(zope.publisher.http.IResult)
+def TemporaryFileResult(f, request):
+    return FileResult(f, request)


Property changes on: Zope3/branches/3.2/src/zope/app/wsgi/fileresult.py
___________________________________________________________________
Name: svn:keywords
   + Id
Name: svn:eol-style
   + native

Added: Zope3/branches/3.2/src/zope/app/wsgi/fileresult.txt
===================================================================
--- Zope3/branches/3.2/src/zope/app/wsgi/fileresult.txt	2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/app/wsgi/fileresult.txt	2005-12-23 19:51:22 UTC (rev 41002)
@@ -0,0 +1,98 @@
+File results
+============
+
+The file results adapters provide adapters from Python file objects
+to and from temporary file objects to zope.publisher.http.IResult. They also have
+the property that they can handle security proxied files and unproxy
+them in the result. Finally, if the request has a wsgi.file_wrapper
+environment variable, then that is used to wrap the file in the
+result.
+
+Lets look at an example with a regular file object:
+
+    >>> from zope import component
+    >>> import zope.app.wsgi.fileresult
+    >>> component.provideAdapter(zope.app.wsgi.fileresult.FileResult)
+    >>> component.provideAdapter(zope.app.wsgi.fileresult.TemporaryFileResult)
+
+    >>> import tempfile
+    >>> dir = tempfile.mkdtemp()
+    >>> import os
+    >>> f = open(os.path.join(dir, 'f'), 'w+b')
+    >>> f.write('One\nTwo\nThree\nHa ha! I love to count!\n')
+    >>> from zope.security.checker import ProxyFactory
+    >>> from zope.publisher.http import IResult
+    >>> from zope.publisher.browser import TestRequest
+    >>> request = TestRequest()
+    >>> result = component.getMultiAdapter((ProxyFactory(f), request), IResult)
+    >>> for line in result.body:
+    ...     print line
+    One
+    Two
+    Three
+    Ha ha! I love to count!
+
+    >>> result.headers
+    (('Content-Length', '38'),)
+    
+
+We'll see something similar with a temporary file:
+
+    >>> t = tempfile.TemporaryFile()
+    >>> t.write('Three\nTwo\nOne\nHa ha! I love to count down!\n')
+    >>> result = component.getMultiAdapter((ProxyFactory(t), request), IResult)
+    >>> for line in result.body:
+    ...     print line
+    Three
+    Two
+    One
+    Ha ha! I love to count down!
+
+    >>> result.headers
+    (('Content-Length', '43'),)
+
+        
+If we provide a custom file wrapper:
+
+    >>> class Wrapper:
+    ...     def __init__(self, file):
+    ...         self.file = file
+ 
+    >>> request = TestRequest(environ={'wsgi.file_wrapper': Wrapper})
+    >>> result = component.getMultiAdapter((ProxyFactory(f), request), IResult)
+    >>> result.body.__class__ is Wrapper
+    True
+    >>> result.body.file is f
+    True
+
+    >>> result.headers
+    (('Content-Length', '38'),)
+
+    >>> result = component.getMultiAdapter((ProxyFactory(t), request), IResult)
+    >>> result.body.__class__ is Wrapper
+    True
+    >>> result.body.file is t
+    True
+
+    >>> result.headers
+    (('Content-Length', '43'),)
+
+Normally, the file given to FileResult must be seekable and the entire
+file is used.  The adapters figure out the file size to determine a
+content length and seek to the beginning of the file.
+
+You can suppress this behavior by setting the content length yourself:
+
+    >>> request = TestRequest()
+    >>> request.response.setHeader('content-length', '10')
+    >>> f.seek(7)
+    >>> result = component.getMultiAdapter((ProxyFactory(t), request), IResult)
+    >>> print f.tell()
+    7
+
+    >>> result.headers
+    ()
+
+Note, that you should really only use file returns for large results.
+Files use file descriptors which can be somewhat scarece resources on
+some systems.  Only use them when you needs them.


Property changes on: Zope3/branches/3.2/src/zope/app/wsgi/fileresult.txt
___________________________________________________________________
Name: svn:eol-style
   + native

Modified: Zope3/branches/3.2/src/zope/app/wsgi/tests.py
===================================================================
--- Zope3/branches/3.2/src/zope/app/wsgi/tests.py	2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/app/wsgi/tests.py	2005-12-23 19:51:22 UTC (rev 41002)
@@ -15,8 +15,14 @@
 
 $Id$
 """
+import tempfile
 import unittest
+
+from zope import component, interface
 from zope.testing import doctest
+
+import zope.app.testing.functional
+import zope.publisher.interfaces.browser
 from zope.app.testing import placelesssetup
 from zope.app.publication.requestpublicationregistry import factoryRegistry
 from zope.app.publication.requestpublicationfactories import BrowserFactory
@@ -25,10 +31,78 @@
     placelesssetup.setUp(test)
     factoryRegistry.register('GET', '*', 'browser', 0, BrowserFactory())
 
+
+
+class FileView:
+
+    interface.implements(zope.publisher.interfaces.browser.IBrowserPublisher)
+    component.adapts(interface.Interface,
+                     zope.publisher.interfaces.browser.IBrowserRequest)
+
+    def __init__(self, _, request):
+        self.request = request
+
+    def browserDefault(self, *_):
+        return self, ()
+
+    def __call__(self):
+        self.request.response.setHeader('content-type', 'text/plain')
+        f = tempfile.TemporaryFile()
+        f.write("Hello\nWorld!\n")
+        return f
+
+
+def test_file_returns():
+    """We want to make sure that file returns work
+
+Let's register a view that returns a temporary file and make sure that
+nothing bad happens. :)
+
+    >>> component.provideAdapter(FileView, name='test-file-view.html')
+    >>> from zope.security import checker
+    >>> checker.defineChecker(
+    ...     FileView,
+    ...     checker.NamesChecker(['browserDefault', '__call__']),
+    ...     )
+
+    >>> from zope.testbrowser import Browser
+    >>> browser = Browser()
+    >>> browser.handleErrors = False
+    >>> browser.open('http://localhost/@@test-file-view.html')
+    >>> browser.headers['content-type']
+    'text/plain'
+
+    >>> browser.headers['content-length']
+    '13'
+
+    >>> print browser.contents
+    Hello
+    World!
+    <BLANKLINE>
+
+Clean up:
+
+    >>> checker.undefineChecker(FileView)
+    >>> component.provideAdapter(
+    ...     None,
+    ...     (interface.Interface,
+    ...      zope.publisher.interfaces.browser.IBrowserRequest),
+    ...     zope.publisher.interfaces.browser.IBrowserPublisher,
+    ...     'test-file-view.html',
+    ...     )
+
+
+"""
+
 def test_suite():
+
+    functional_suite = doctest.DocTestSuite()
+    functional_suite.layer = zope.app.testing.functional.Functional
+
     return unittest.TestSuite((
+        functional_suite,
         doctest.DocFileSuite(
-            'README.txt',
+            'README.txt', 'fileresult.txt',
             setUp=setUp,
             tearDown=placelesssetup.tearDown,
             optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS),

Modified: Zope3/branches/3.2/src/zope/publisher/http.py
===================================================================
--- Zope3/branches/3.2/src/zope/publisher/http.py	2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/publisher/http.py	2005-12-23 19:51:22 UTC (rev 41002)
@@ -24,8 +24,9 @@
 import logging
 from tempfile import TemporaryFile
 
+from zope import component, interface
+
 from zope.deprecation import deprecation
-from zope.interface import implements
 
 from zope.publisher import contenttype
 from zope.publisher.interfaces.http import IHTTPCredentials
@@ -34,7 +35,7 @@
 from zope.publisher.interfaces.http import IHTTPPublisher
 
 from zope.publisher.interfaces import Redirect
-from zope.publisher.interfaces.http import IHTTPResponse, IResult
+from zope.publisher.interfaces.http import IHTTPResponse
 from zope.publisher.interfaces.http import IHTTPApplicationResponse
 from zope.publisher.interfaces.logginginfo import ILoggingInfo
 from zope.i18n.interfaces import IUserPreferredCharsets
@@ -251,7 +252,9 @@
     values will be looked up in the order: environment variables,
     other variables, form data, and then cookies.
     """
-    implements(IHTTPCredentials, IHTTPRequest, IHTTPApplicationRequest)
+    interface.implements(IHTTPCredentials,
+                         IHTTPRequest,
+                         IHTTPApplicationRequest)
 
     __slots__ = (
         '__provides__',   # Allow request to directly provide interfaces
@@ -593,8 +596,32 @@
         d.update(self._cookies)
         return d.keys()
 
+
+class IResult(interface.Interface):
+    """HTTP result.
+
+    WARNING! This is a PRIVATE interface and VERY LIKELY TO CHANGE!
+
+    The result provides the result in a form suitable for delivery to HTTP
+    clients.
+
+    IMPORTANT: The result object may be held indefinitely by a server and may
+    be accessed by arbitrary threads. For that reason the result should not
+    hold on to any application resources and should be prepared to be invoked
+    from any thread.
+    """
+
+    headers = interface.Attribute(
+        'A sequence of tuples of result headers, such as '
+        '"Content-Type" and "Content-Length", etc.')
+
+    body = interface.Attribute(
+        'An iterable that provides the body data of the response.')
+
+
+
 class HTTPResponse(BaseResponse):
-    implements(IHTTPResponse, IHTTPApplicationResponse)
+    interface.implements(IHTTPResponse, IHTTPApplicationResponse)
 
     __slots__ = (
         'authUser',             # Authenticated user string
@@ -771,16 +798,21 @@
 
 
     def setResult(self, result):
-        r = IResult(result, None)
-        if r is None:
-            if isinstance(result, basestring):
-                body, headers = self._implicitResult(result)
-                r = DirectResult((body,), headers)
-            elif result is None:
-                body, headers = self._implicitResult('')
-                r = DirectResult((body,), headers)
-            else:
-                raise TypeError('The result should be adaptable to IResult.')
+        if IResult.providedBy(result):
+            r = result
+        else:
+            r = component.queryMultiAdapter((result, self._request), IResult)
+            if r is None:
+                if isinstance(result, basestring):
+                    body, headers = self._implicitResult(result)
+                    r = DirectResult((body,), headers)
+                elif result is None:
+                    body, headers = self._implicitResult('')
+                    r = DirectResult((body,), headers)
+                else:
+                    raise TypeError(
+                        'The result should be adaptable to IResult.')
+
         self._result = r
         self._headers.update(dict([(k, [v]) for (k, v) in r.headers]))
         if not self._status_set:
@@ -937,7 +969,7 @@
 
 
 class HTTPCharsets(object):
-    implements(IUserPreferredCharsets)
+    interface.implements(IUserPreferredCharsets)
 
     def __init__(self, request):
         self.request = request
@@ -1009,7 +1041,7 @@
     application to specify all headers related to the content, such as the
     content type and length.
     """
-    implements(IResult)
+    interface.implements(IResult)
 
     def __init__(self, body, headers=()):
         self.body = body
@@ -1023,3 +1055,4 @@
     including content type and length.
     """
     return DirectResult((body,), headers)
+

Modified: Zope3/branches/3.2/src/zope/publisher/httpresults.txt
===================================================================
--- Zope3/branches/3.2/src/zope/publisher/httpresults.txt	2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/publisher/httpresults.txt	2005-12-23 19:51:22 UTC (rev 41002)
@@ -30,67 +30,15 @@
 Returning large amounts of data without storing the data in memory
 ------------------------------------------------------------------
 
-Starting in Zope 3.2, a published object (e.g. a view or view method)
-can return any object as long as it is is adaptable to
-zope.publisher.interfaces.http.IResult::
+To return a large result, you should write the result to a temporary
+file (tempfile.TemporaryFile) and return the temporary file.
+Alternatively, if the data you want to return is already in a
+(non-temporary) file, just open and return that file.  The publisher
+(actually an adapter used by the publisher) will handle a returned
+file very efficiently.  
 
-  class IResult(Interface):
-      """HTTP result.
+The publisher will compute the response content length from the file
+automatically. It is up to applications to set the content type.
+It will also take care of positioning the file to it's beginning, 
+so applications don't need to do this beforehand.
 
-      The result provides the result in a form suitable for delivery to HTTP
-      clients.
-
-      IMPORTANT: The result object may be held indefinitely by a server and may
-      be accessed by arbitrary threads. For that reason the result should not
-      hold on to any application resources and should be prepared to be invoked
-      from any thread.
-      """
-
-      headers = Attribute('A sequence of tuples of result headers, such as'
-                          '"Content-Type" and "Content-Length", etc.')
-
-      body = Attribute('An iterable that provides the body data of the'
-                       'response.')
-
-The result object has headers and an iterable body.  The ability to
-supply headers in a result is useful for adapters that compute headers
-by inspecting a the object being adapted.
-
-There is a helper class, zope.publisher.http.DirectResult that can be
-used to compute result objects.
-
-When an published object returns a string. the string is inspected to
-determine response headers (like content type and content length) and
-a result is created using DirectResult.
-
-If you want to return a large amont of data, you can create a result
-object yourself.  A good way to do this is to copy the data to a 
-temporary file and return an iterator to that::
-
-   import tempfile
-   file = tempfile.TemporaryFile()
-
-   # ... write data to the file ...
-
-   def fileiterator(file, bufsize=8192):
-       while 1:
-           data = file.read(bufsize)
-           if data:
-               yield data
-           else:
-               break
-
-       file.close()
-
-   return DirectResult(fileiterator(file), 
-                       [('Content-Length', mydatalength),
-                        ('Content-Type', mydatatype),
-                       ])
-
-We should provide some helper objects that automate more of this, and
-we probably will in later revisions.
-
-IMPORTANT NOTE: the iterator that you pass to DirectResult must *not*
-use any application resources.  When the iterator is called,
-application resoures may have been released or be in use by another
-thread. 

Modified: Zope3/branches/3.2/src/zope/publisher/interfaces/http.py
===================================================================
--- Zope3/branches/3.2/src/zope/publisher/interfaces/http.py	2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/publisher/interfaces/http.py	2005-12-23 19:51:22 UTC (rev 41002)
@@ -373,7 +373,7 @@
         """
 
     def setResult(result):
-        """Sets the response result value that is adaptable to ``IResult``.
+        """Sets the response result value to a string or a file.
         """
 
     def consumeBody():
@@ -389,23 +389,3 @@
         Note that this function can be only requested once, since it is
         constructed from the result.
         """
-
-
-class IResult(Interface):
-    """HTTP result.
-
-    The result provides the result in a form suitable for delivery to HTTP
-    clients.
-
-    IMPORTANT: The result object may be held indefinitely by a server and may
-    be accessed by arbitrary threads. For that reason the result should not
-    hold on to any application resources and should be prepared to be invoked
-    from any thread.
-    """
-
-    headers = Attribute('A sequence of tuples of result headers, such as'
-                        '"Content-Type" and "Content-Length", etc.')
-
-    body = Attribute('An iterable that provides the body data of the'
-                     'response.')
-



More information about the Zope3-Checkins mailing list