[ZODB-Dev] Relstorage Blob support and Oracle

[Martijn Pieters]

On Tue, Jun 14, 2011 at 22:22, Martijn Pieters <mj at zopatista.com> wrote:
> I am considering altering the ON COMMIT behaviour of the
> temp_blob_chunk table to TRUNCATE to extend this behaviour to all
> remaining blob rows on commit. However, such events should be rare
> enough, and there is a `vacuumlo` command that can find orphaned blob
> objects in the pg_largeobjects table and clean these out.

I've now made this change. This means that the temp_blob_chunk table
is dropped when a storage is opened, not on commit, and instead it's
contents are truncated on commit and any oids left in there that
didn't make it to blob_chunk are unlinked.

> I'm also writing a unittest that'll exercise the blob storage a little
> more, by creating a file larger than the maximum storable in one
> chunk, then upload, commit and re-download to see if the data survived
> intact. This'll be set to test level 2 to not clog up the normal test
> cycle as generating a 4GB+ random file can take a little while! :-)

See http://zope3.pov.lt/trac/changeset/121949/relstorage/branches/postgres_blob_oid

The tests run at test level 2, meaning that they do not run by
default. You need to either set the test level to 2 (-a 2) or use the
--all switch to run all levels regardless. The PostgreSQL version
(2GB+ blob size) takes a healthy quarter of an hour on my laptop to
complete one such test.

> Last but not least I'll need to write a migration script for those
> users of RelStorage 1.5.0b2 already in production.

I have figured out how to do this (see
http://archives.postgresql.org/pgsql-general/2009-01/msg00771.php). I
won't try to consolidate the blob_chunks, so you'll end up with 1MB
oid blobs but that's fine as the download code can handle those
perfectly.

The query just needs to be written and tested.

-- 
Martijn Pieters