[Checkins] SVN: zc.zlibstorage/branches/dev/ Added documentation.
Jim Fulton
jim at zope.com
Thu May 20 15:39:15 EDT 2010
Log message for revision 112582:
Added documentation.
Changed:
U zc.zlibstorage/branches/dev/README.txt
U zc.zlibstorage/branches/dev/setup.py
A zc.zlibstorage/branches/dev/src/zc/zlibstorage/README.txt
-=-
Modified: zc.zlibstorage/branches/dev/README.txt
===================================================================
--- zc.zlibstorage/branches/dev/README.txt 2010-05-20 19:39:12 UTC (rev 112581)
+++ zc.zlibstorage/branches/dev/README.txt 2010-05-20 19:39:14 UTC (rev 112582)
@@ -1,13 +1 @@
-zlib Storage
-************
-
-A ZODB wrapper storage that compresses data records using zlib.
-
-
-Changes
-*******
-
-0.1 (yyyy-mm-dd)
-================
-
-Initial release
+See src/zc/zlibstorage/README.txt.
Modified: zc.zlibstorage/branches/dev/setup.py
===================================================================
--- zc.zlibstorage/branches/dev/setup.py 2010-05-20 19:39:12 UTC (rev 112581)
+++ zc.zlibstorage/branches/dev/setup.py 2010-05-20 19:39:14 UTC (rev 112582)
@@ -1,6 +1,6 @@
##############################################################################
#
-# Copyright (c) Zope Foundation and Contributors.
+# Copyright (c) 2010 Zope Foundation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
@@ -14,21 +14,25 @@
name, version = 'zc.zlibstorage', '0'
install_requires = ['ZODB3 >=3.10.0b1', 'setuptools']
-extras_require = dict(test=['zope.testing'])
+extras_require = dict(test=['zope.testing', 'manuel'])
entry_points = """
"""
from setuptools import setup
+import os
+long_description = open(os.path.join('src', *name.split('.')+['README.txt'])
+ ).read()
+
setup(
author = 'Jim Fulton',
author_email = 'jim at zope.com',
license = 'ZPL 2.1',
name = name, version = version,
- long_description=open('README.txt').read(),
- description = open('README.txt').read().strip().split('\n')[0],
+ long_description=long_description,
+ description = long_description.split('\n')[1],
packages = [name.split('.')[0], name],
namespace_packages = [name.split('.')[0]],
package_dir = {'': 'src'},
Added: zc.zlibstorage/branches/dev/src/zc/zlibstorage/README.txt
===================================================================
--- zc.zlibstorage/branches/dev/src/zc/zlibstorage/README.txt (rev 0)
+++ zc.zlibstorage/branches/dev/src/zc/zlibstorage/README.txt 2010-05-20 19:39:14 UTC (rev 112582)
@@ -0,0 +1,225 @@
+=============================================================
+ZODB storage wrapper for zlib compression of database records
+=============================================================
+
+The ``zc.zlibstorage`` package provides ZODB storage wrapper
+implementations that provide compression of database records.
+
+.. contents::
+
+Usage
+=====
+
+The primary storage is ``zc.zlibstorage.ZlibStorage``. It is used as
+a wrapper around a lower-level storage. From Python, is is
+constructed by passing another storage, as in::
+
+ import ZODB.FileStorage, zc.zlibstorage
+
+ storage = zc.zlibstorage.ZlibStorage(
+ ZODB.FileStorage.FileStorage('data.fs'))
+
+.. -> src
+
+ >>> import zlib
+ >>> exec src
+ >>> data = 'x'*100
+ >>> storage.transform_record_data(data) == '.z'+zlib.compress(data)
+ True
+ >>> storage.close()
+
+When using a ZODB configuration file, the zlibstorage tag is used::
+
+ %import zc.zlibstorage
+
+ <zodb>
+ <zlibstorage>
+ <filestorage>
+ path data.fs
+ </filestorage>
+ </zlibstorage>
+ </zodb>
+
+.. -> src
+
+ >>> import ZODB.config
+ >>> db = ZODB.config.databaseFromString(src)
+ >>> db.storage.transform_record_data(data) == '.z'+zlib.compress(data)
+ True
+ >>> db.close()
+
+Note the ``%import`` used to load the definition of the
+``zlibstorage`` tag.
+
+Use with ZEO
+============
+
+When used with a ZEO ClientStorage, you'll need to use a server zlib
+storage on the storage server. This is necessary so that server
+operations that need to get at uncompressed record data can do so.
+This is accomplished using the ``serverzlibstorage`` tag in your ZEO
+server configuration file::
+
+ %import zc.zlibstorage
+
+ <zeo>
+ address 8100
+ </zeo>
+
+ <serverzlibstorage>
+ <filestorage>
+ path data.fs
+ </filestorage>
+ </serverzlibstorage>
+
+.. -> src
+
+ >>> src = src[:src.find('<zeo>')]+src[src.find('</zeo>')+7:]
+
+ >>> storage = ZODB.config.storageFromString(src)
+ >>> storage.transform_record_data(data) == '.z'+zlib.compress(data)
+ True
+ >>> storage.__class__.__name__
+ 'ServerZlibStorage'
+
+ >>> storage.close()
+
+Applying compression on the client this way is attractive because, in
+addition to reducing the size of stored database records on the
+server, you also reduce the size of records sent from the server to the
+client and the size of records stored in the client's ZEO cache.
+
+Decompressing only
+==================
+
+By default, records are compressed when written to the storage and
+uncompressed when read from the storage. A ``compress`` option can be
+used to disable compression of records but still uncompress compressed
+records if they are encountered. Here's an example from in Python::
+
+ import ZODB.FileStorage, zc.zlibstorage
+
+ storage = zc.zlibstorage.ZlibStorage(
+ ZODB.FileStorage.FileStorage('data.fs'),
+ compress=False)
+
+.. -> src
+
+ >>> exec src
+ >>> storage.transform_record_data(data) == data
+ True
+ >>> storage.close()
+
+and using the configurationb syntax::
+
+ %import zc.zlibstorage
+
+ <zodb>
+ <zlibstorage>
+ compress false
+ <filestorage>
+ path data.fs
+ </filestorage>
+ </zlibstorage>
+ </zodb>
+
+.. -> src
+
+ >>> db = ZODB.config.databaseFromString(src)
+ >>> db.storage.transform_record_data(data) == data
+ True
+ >>> db.close()
+
+This option is useful when deploying the storage when there are
+multiple clients. If you don't want to update all of the clients at
+once, you can gradually update all of the clients with a zlib storage
+that doesn't do compression, but recognizes compressed records. Then,
+in a second phase, you can update the clients to compress records, at
+which point, all of the clients will be able to read the compressed
+records produced.
+
+Compressing entire databases
+============================
+
+One way to compress all of the records in a database is to copy data
+from an uncompressed database to a comprressed once, as in::
+
+ import ZODB.FileStorage, zc.zlibstorage
+
+ orig = ZODB.FileStorage.FileStorage('data.fs')
+ new = zc.zlibstorage.ZlibStorage(
+ ZODB.FileStorage.FileStorage('data.fs-copy'))
+ new.copyTransactionsFrom(orig)
+
+ orig.close()
+ new.close()
+
+.. -> src
+
+ >>> conn = ZODB.connection('data.fs', create=True)
+ >>> conn.root.a = conn.root().__class__([(i,i) for i in range(1000)])
+ >>> conn.root.b = conn.root().__class__([(i,i) for i in range(2000)])
+ >>> import transaction
+ >>> transaction.commit()
+ >>> conn.close()
+
+ >>> exec(src)
+
+ >>> new = zc.zlibstorage.ZlibStorage(
+ ... ZODB.FileStorage.FileStorage('data.fs-copy'))
+ >>> conn = ZODB.connection(new)
+ >>> dict(conn.root.a) == dict([(i,i) for i in range(1000)])
+ True
+ >>> dict(conn.root.b) == dict([(i,i) for i in range(2000)])
+ True
+
+ >>> import ZODB.utils
+ >>> for i in range(3):
+ ... if not new.base.load(ZODB.utils.p64(i))[0][:2] == '.z':
+ ... print 'oops', i
+ >>> len(new)
+ 3
+
+ >>> conn.close()
+
+Record prefix
+=============
+
+Compressed records have a prefix of ".z". This allows a database to
+have a mix of compressed and uncompressed records.
+
+Stand-alone Compression and decompression functions
+===================================================
+
+In anticipation of wanting to plug the compression and decompression
+logic into other tools without creating storages, the functions used
+to compress and uncompress data records are available as
+``zc.zlibstorage`` module-level functions:
+
+``compress(data)``
+ Compress the given data if:
+
+ - it is a string more than 20 characters in length,
+ - it doesn't start with the compressed-record marker, ``'.z'``, and
+ - the compressed size is less the original.
+
+ The compressed (or original) data are returned.
+
+``decompress(data)``
+ Decompress the data if it is compressed.
+
+ The decompressed (or original) data are returned.
+
+.. basic sanity check :)
+
+ >>> _ = (zc.zlibstorage.compress, zc.zlibstorage.decompress)
+
+.. Hide changes for now
+
+ Changes
+ =======
+
+ 0.1.0 2010-05-20
+ ----------------
+
+ Initial release
Property changes on: zc.zlibstorage/branches/dev/src/zc/zlibstorage/README.txt
___________________________________________________________________
Added: svn:eol-style
+ native
More information about the checkins
mailing list