[ZODB-Dev] Relstorage Blob support and Oracle

[Shane Hathaway]

On 06/12/2011 01:39 PM, Martijn Pieters wrote:
> On Sun, Jun 12, 2011 at 16:40, Hanno Schlichting<hanno at hannosch.eu>  wrote:
>> Looking at the most recent docs for the bytea type [1] there's two
>> encoding schemes. The new default in 9.0+ is called hex and doesn't
>> suffer from the same problems as the old "escape" format, which
>> couldn't represent some data - which in turn prompted Shane to do the
>> base64 encoding.
>
> I am kind of wondering as to why a base64 encoding is needed at all.
> The psycopg supports the required encoding / decoding steps natively
> for both escape formats, and will do so transparently. See:
>
>    http://initd.org/psycopg/docs/usage.html#index-7
>
> Not encoding ZODB pickles for the object_state table from and to
> base64 would save a lot of space (base64 encoding expands data to 133%
> it's original size, leading to up to 1/3rd more space required to
> store a Zope ZODB in PostgreSQL than in other databases.

No, the base 64 encoding is only used for database communication.  In 
all databases RelStorage supports, objects are stored in binary form 
with no encoding.

Early on in the project I found out by measurement that the standard 
binary encoding used by psycopg was a lot slower than base64 encoding. 
I was trying to get PostgreSQL to match the MySQL numbers I was seeing. 
  I didn't quite accomplish that, but base64 brought it closer.

WRT using PostgreSQL OIDs for blobs: the extra burden of deleting unused 
OIDs doesn't seem worth the extra effort to me.  I would be happy to 
hear that you found a reasonably simple way to do it.  My guess is that 
an on-delete trigger will be the best way.

Shane