[ZODB-Dev] Optimizing RelStorage Packing for large DBs

[Jim Fulton]

On Mon, Nov 18, 2013 at 12:00 PM, Jens W. Klein <jens at bluedynamics.com> wrote:
> Hi Jim,
>
> thanks for the hint (also in the other post). I looked at zc.zodbdgc and
> took some inspiration from it. As far as I understand it stores the incoming
> references in a separate filestorage backend.

This is just a temporary file to avoid storing all the data in memory.

> So this works similar to my
> impelmentation but uses the ZODB infrastructure. I dont see how I make
> zc.zodbdgc play with Relstorage and since it works on the abstracted ZODB
> level using pickles

I don't know what you're saying, since I don't know what "it" refers to.

zodbdgc works with storages.  relstorage conforms to the storage API.
It's possible some changes would be needed, but they should be minor.

> I suspected it to be not fast enough for so many obejcts

No idea why, or what "fast enough" is.  We use it on a database with
~200 million objects.

> - so I skipped this alternative.

Good luck.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton