[ZODB-Dev] Relstorage Blob support and Oracle

[Shane Hathaway]

On 06/09/2011 06:32 AM, Martijn Pieters wrote:
> We've looked over the RelsStorage ZODB Blob storage implementation and
> came to the conclusion that the current use of blob chunks is
> unnecessary in Oracle when using the cx_Oracle database connector. Not
> splitting ZODB Blobs into chunks may have performance benefits on the
> Oracle side (especially on 11g) as Oracle can then use read-ahead more
> efficiently for the larger BLOBs while streaming these back to the
> client.

Well, the ZODB blob interfaces make it impossible to actually stream 
blobs from the database.  The entire blob has to be read into cache 
before the client can get it, so I'm not sure what you will gain.

> I'll be refactoring the blob support currently found in RelStorage
> 1.5b2 to just store one blob in one row, using the cx_Oracle
> LOB.read() and .write() methods, which let you read and write to blobs
> in chunks to avoid memory overload, and I'll reuse the blob chunk size
> to determine how much we read / write per iteration.
>
> I am currently leaning towards dropping the chunk column in the Oracle
> schema altogether; it certainly won't hold any other value than
> integer "0" after my refactor. Any reason to keep it, other than that
> others whom already are using 1.5b2 on an Oracle database will now
> have to drop that column again (or set it to a default value of "0" on
> insert)? Should the code support reading blob chunks still?

No, please don't change the schema.  I know all of the supported 
databases support a form of streaming, but they are all wildly 
different.  I suggest that chunking is much simpler and has little 
overhead compared with streaming.

Shane