[ZODB-Dev] Build compression into ZODB 3.11?

[Jim Fulton]

On Wed, Mar 14, 2012 at 7:46 PM, Stephan Richter
<stephan.richter at gmail.com> wrote:
> On Wednesday, March 14, 2012 01:47:54 PM Jim Fulton wrote:
>> I'm pretty happy with how zc.zlibstorage has worked out.
>>
>> Should I build this into ZODB 3.11?
>
> Hi Jim,
>
> I am +1 as well. Since we are also dealing with lots of sensitive data, we
> sometimes also encrypt the pickles before storage. Would it be possible to
> have any arbitrary hook to modify the pickle before sotrage and then on the
> way back out?

It is already possible.

See ZODB.tests.hexstorage and all the tests that use it.  Basically, the ZODB
APIs have hooks for adding layered transformations.  There's also a convention
for recording transformations by adding a '.C' prefix to transformed
records, where
C is a 1-letter code indicating which transformation was was applied.
(zlibstorage uses 'z', so the prefix is '.z'.)

I need to think about how to integrate compression into 3.11. At a
minimum, I'd like
to arrange that existing zlib-transformed records are recognized without
having to create a special storage to make exploratory database connections
easier.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton