[Zope3-dev] Viewing/downloading large files

[Jim Fulton]

Garrett Smith wrote:
>>From the looks of the z/a/file/File code, viewing a file will cause all of the file contents to be loaded into memory (specifically, I'm looking at line 45 of z/a/file/browser/file.py).
> 
> Is there a way to provide a file-like object (no pun intended, i.e. something with streaming capabilities) to the response rather than a single block of bytes?

Yes, this *should* be possible.  File would need to grow a method to
get at the underlying chunk data, and a view would need to be provided that
used request.response.write to output the data.  Christian Theune and I
looked at this in early 2004 and, at the time, though, there seemed to
be a pub in the publisher that would have prevented this from being useful.
Unfortunately, I don't remember the details.  At the time, it wasn't important
enough to fix, but it's too bad that it hasn't been fixed by now.


There are 2 developments looming that would affect this:

1. ZODB is growing Blob support.  This will both provide much better
    handling of fairly large files at the ZODB lavel and suggests a more
    file-like API.

2. I'd really like to look at making the publisher more WSGI friendly
    for 3.2.  In particular, I'd like to provide a way for a published
    object to provide an iterator that could be served efficiently, rather
    than using response.write. In fact, I'd like to deprecate response.write.

I'd be happy to help with this if someone wants to work on it.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org