[Zope3-dev] Returning large amounts of data to a client

[Jim Fulton]

Peter Mayne wrote:
> I have a resource class that returns binary data (PNG images in this 
> case) to the client, along the following lines:
> 
> class MyResource(BrowserView, Resource):
>     implements(IBrowserPublisher)
> 
> ...
> 
>     def GET(self):
>         request = self.request
>         response = request.response
>         response.setHeader('Content-Type', CONTENT_TYPE)
>         data = /generate binary data here/
> 
>         return data
> 
> This is fine when the amount of data is small, but what if there are 
> many bytes involved.
> 
> The obvious thing to try was
> 
>         response.write(data)
>         return None
> 
> and guessing that I can write a sequence of (say) 64KB chunks so I don't 
> use too much memory, but I get
> 
>   File "C:\opt\Python23\Lib\site-packages\zope\server\buffers.py", line 
> 186, in append
>     self.strbuf = strbuf + s
> UnicodeDecodeError: 'ascii' codec can't decode byte 0x89 in position 
> 283: ordinal not in range(128)
> 
> which implies that ZX3 is expecting (possibly UTF-8) encoded characters, 
> rather than binary data, when writing to the response. Furthermore, I 
> possibly wouldn't want to buffer the large amount of data, which is what 
> the classes in buffers.py seem to do.
> 
> What is the right thing to do when returning potentially large amounts 
> of data to a client?

You've found a bug in the publisher.  It has nothing to do with
the amound of data involved.  I assume the value of "data" in the example
above is a regular Python string. For some reason, the strbuf variable
is being set to a unicode value and Python is doing an implicit unicode
conversion. Please report this as a bug.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org