[Zope3-dev] Returning large amounts of data to a client

[Shane Hathaway]

On Tuesday 04 January 2005 20:35 pm, Peter Mayne wrote:
>    File "C:\opt\Python23\Lib\site-packages\zope\server\buffers.py", line
> 186, in append
>      self.strbuf = strbuf + s
> UnicodeDecodeError: 'ascii' codec can't decode byte 0x89 in position
> 283: ordinal not in range(128)
>
> which implies that ZX3 is expecting (possibly UTF-8) encoded characters,
> rather than binary data, when writing to the response. Furthermore, I
> possibly wouldn't want to buffer the large amount of data, which is what
> the classes in buffers.py seem to do.
>
> What is the right thing to do when returning potentially large amounts
> of data to a client?

That's the right way.  However, buffers.py wasn't designed to handle unicode 
data; the data is supposed to be encoded in an 8-bit byte stream before it 
reaches that point.  I wonder what's sending it unicode.

The current architecture buffers data so that ZODB connections can be freed as 
quickly as possible.  When buffers get large they automatically move to a 
temporary file, so hopefully the buffers won't consume a lot of RAM.

Shane