[ZODB-Dev] Some interesting (to some:) numbers

[Roel Bruggink]

On Sun, May 9, 2010 at 8:33 PM, Jim Fulton <jim at zope.com> wrote:

> Our recent discussion of compression made me curious so I did some
> analysis of pickle sizes in one of our large databases. This is for a
> content management system.  The database is packed weekly.  It doesn't
> include media, which are in blobs.
>
> There were ~19 million transaction in the database and around 130
> million data records. About 60% of the size was taken up by BTrees.
> Compressing pickles using zlib with default compression reduced the
> pickle sizes by ~58%. The average uncompressed record size was 1163
> bytes.  The average compressed size was ~493 bytes.
>
> This is probably enough of a savings to make compression interesting.
>
> Jim
>
> --
> Jim Fulton
> _______________________________________________
> For more information about ZODB, see the ZODB Wiki:
> http://www.zope.org/Wikis/ZODB/
>
> ZODB-Dev mailing list  -  ZODB-Dev at zope.org
> https://mail.zope.org/mailman/listinfo/zodb-dev
>


That's really interesting! Did you notice any issues performance wise, or
didn't you check that yet?
I wonder what DirectoryStorage on btrfs with transparent compression can do
btw.

-- 
Roel Bruggink
http://www.fourdigits.nl/mensen/roel-bruggink

Four Digits BV
http://www.fourdigits.nl
Willemsplein 44, 6811 KD, Arnhem
tel: +31(0)26 4422700 fax: +31(0)84 2206117
KVK 091621370000 BTW 8161.22.234.B01
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.zope.org/pipermail/zodb-dev/attachments/20100509/3ad8b5e9/attachment.html