<div class="gmail_quote">On Sun, May 9, 2010 at 8:33 PM, Jim Fulton <span dir="ltr"><<a href="mailto:jim@zope.com">jim@zope.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Our recent discussion of compression made me curious so I did some<br>
analysis of pickle sizes in one of our large databases. This is for a<br>
content management system. The database is packed weekly. It doesn't<br>
include media, which are in blobs.<br>
<br>
There were ~19 million transaction in the database and around 130<br>
million data records. About 60% of the size was taken up by BTrees.<br>
Compressing pickles using zlib with default compression reduced the<br>
pickle sizes by ~58%. The average uncompressed record size was 1163<br>
bytes. The average compressed size was ~493 bytes.<br>
<br>
This is probably enough of a savings to make compression interesting.<br>
<br>
Jim<br>
<font color="#888888"><br>
--<br>
Jim Fulton<br>
_______________________________________________<br>
For more information about ZODB, see the ZODB Wiki:<br>
<a href="http://www.zope.org/Wikis/ZODB/" target="_blank">http://www.zope.org/Wikis/ZODB/</a><br>
<br>
ZODB-Dev mailing list - <a href="mailto:ZODB-Dev@zope.org">ZODB-Dev@zope.org</a><br>
<a href="https://mail.zope.org/mailman/listinfo/zodb-dev" target="_blank">https://mail.zope.org/mailman/listinfo/zodb-dev</a><br>
</font></blockquote></div><br><br><div>That's really interesting! Did you notice any issues performance wise, or didn't you check that yet?</div><div>I wonder what DirectoryStorage on btrfs with transparent compression can do btw.<br>
<div><br>-- <br>Roel Bruggink<br><a href="http://www.fourdigits.nl/mensen/roel-bruggink">http://www.fourdigits.nl/mensen/roel-bruggink</a><br><br>Four Digits BV<br><a href="http://www.fourdigits.nl">http://www.fourdigits.nl</a><br>
Willemsplein 44, 6811 KD, Arnhem<br>tel: +31(0)26 4422700 fax: +31(0)84 2206117<br>KVK 091621370000 BTW 8161.22.234.B01<br>
</div></div>