[Zope3-dev] Re: Returning large amounts of data to a client

[Peter Mayne]

Peter Mayne wrote:
> 
> The obvious thing to try was
> 
>         response.write(data)
>         return None
> 
> and guessing that I can write a sequence of (say) 64KB chunks so I don't 
> use too much memory, but I get
> 
>   File "C:\opt\Python23\Lib\site-packages\zope\server\buffers.py", line 
> 186, in append
>     self.strbuf = strbuf + s
> UnicodeDecodeError: 'ascii' codec can't decode byte 0x89 in position 
> 283: ordinal not in range(128)

Shane Hathaway wrote:
 > The current architecture buffers data so that ZODB connections can be
 > freed as quickly as possible.  When buffers get large they
 > automatically move to a temporary file, so hopefully the buffers won't
 > consume a lot of RAM.

I see your point, but if I'm generating something big (large PDF, video 
stream, etc) that isn't coming from the instance database (either 
generated dynamically or, I don't think I want it rebuffered. Do I?

Jim Fulton wrote:
 > You've found a bug in the publisher.  It has nothing to do with
 > the amound of data involved.  I assume the value of "data" in the
 > example above is a regular Python string.

Yes. In this case it's a fairly small PNG image, but having written the 
code for my own ZCML resource tag, I want to see how easy it is to use 
it for a filesystem access resource, which would involve potentially 
large files.

 > For some reason, the strbuf
 > variable is being set to a unicode value and Python is doing an
 > implicit unicode conversion. Please report this as a bug.

http://collector.zope.org/Zope3-dev/345

PJDM
-- 
Peter Mayne
Spherion Technology Solutions
Canberra, ACT, Australia
"I'm after rebellion, I'll settle for lies" - Blue Oyster Cult