[ZODB-Dev] Relstorage Blob support and Oracle

[Martijn Pieters]

On Mon, Jun 13, 2011 at 10:47, Shane Hathaway <shane at hathawaymix.org> wrote:
>> I am kind of wondering as to why a base64 encoding is needed at all.
>> The psycopg supports the required encoding / decoding steps natively
>> for both escape formats, and will do so transparently. See:
>>
>>    http://initd.org/psycopg/docs/usage.html#index-7
>>
>> Not encoding ZODB pickles for the object_state table from and to
>> base64 would save a lot of space (base64 encoding expands data to 133%
>> it's original size, leading to up to 1/3rd more space required to
>> store a Zope ZODB in PostgreSQL than in other databases.
>
> No, the base 64 encoding is only used for database communication.  In
> all databases RelStorage supports, objects are stored in binary form
> with no encoding.
>
> Early on in the project I found out by measurement that the standard
> binary encoding used by psycopg was a lot slower than base64 encoding.
> I was trying to get PostgreSQL to match the MySQL numbers I was seeing.
>  I didn't quite accomplish that, but base64 brought it closer.

Right, that's an excellent reason to keep the base64 encoding then!
Perhaps this should be documented in the package, or is there a design
decisions doc somewhere else that we can add to (or where I should
have looked and missed this)? :-)

> WRT using PostgreSQL OIDs for blobs: the extra burden of deleting unused
> OIDs doesn't seem worth the extra effort to me.  I would be happy to
> hear that you found a reasonably simple way to do it.  My guess is that
> an on-delete trigger will be the best way.

Already solved with a trigger, see

  http://zope3.pov.lt/trac/changeset/121922/relstorage/branches/postgres_blob_oid

:-)

The trigger is really really simple; the example I first found had to
deal with UPDATE too, making things a little more complicated, but
RelStorage never does that; it's either INSERT or DELETE.

-- 
Martijn Pieters