[Checkins] SVN: relstorage/tags/1.1c1/ tagged 1.1c1

Shane Hathaway shane at hathawaymix.org
Sat Aug 2 20:12:34 EDT 2008


Log message for revision 89242:
  tagged 1.1c1

Changed:
  A   relstorage/tags/1.1c1/
  D   relstorage/tags/1.1c1/CHANGES.txt
  A   relstorage/tags/1.1c1/CHANGES.txt
  D   relstorage/tags/1.1c1/README.txt
  A   relstorage/tags/1.1c1/README.txt
  D   relstorage/tags/1.1c1/poll-invalidation-1-zodb-3-7-1.patch
  A   relstorage/tags/1.1c1/poll-invalidation-1-zodb-3-7-1.patch
  D   relstorage/tags/1.1c1/poll-invalidation-1-zodb-3-8-0.patch
  A   relstorage/tags/1.1c1/poll-invalidation-1-zodb-3-8-0.patch
  D   relstorage/tags/1.1c1/relstorage/
  A   relstorage/tags/1.1c1/relstorage/

-=-
Copied: relstorage/tags/1.1c1 (from rev 88770, relstorage/branches/1.1)

Deleted: relstorage/tags/1.1c1/CHANGES.txt
===================================================================
--- relstorage/branches/1.1/CHANGES.txt	2008-07-23 18:06:37 UTC (rev 88770)
+++ relstorage/tags/1.1c1/CHANGES.txt	2008-08-03 00:12:31 UTC (rev 89242)
@@ -1,187 +0,0 @@
-
-RelStorage 1.1c1
-
-- Added optional memcache integration.  This is useful when the connection
-  to the relational database has high latency.
-
-- Made it possible to set the pack and memcache options in zope.conf.
-
-- Log more info when a KeyError occurs within RelStorage.
-
-
-RelStorage 1.1b2
-
-- Made the MySQL locks database-specific rather than server-wide.  This is
-  important for multi-database configurations.
-
-- In the PostgreSQL adapter, made the pack lock fall back to table locking
-  rather than advisory locks for PostgreSQL 8.1.
-
-- Changed a query for following object references (used during packing)
-  to work around a MySQL performance bug.  Thanks to Anton Stonor for
-  discovering this.
-
-
-RelStorage 1.1b1
-
-- Fixed the use of setup.py without setuptools.  Thanks to Chris Withers.
-
-- Fixed type coercion of the transaction extension field.  This fixes
-  an issue with converting databases.  Thanks to Kevin Smith for
-  discovering this.
-
-- Added logging to the pack code to help diagnose performance issues.
-
-- Additions to the object_ref table are now periodically committed
-  during pre_pack so that the work is not lost if pre_pack fails.
-
-- Modified the pack code to pack one transaction at a time and
-  release the commit lock frequently.  This should help large pack
-  operations.
-
-- Fixed buildout-based installation of the zodbconvert script.  Thanks to
-  Jim Fulton.
-
-
-RelStorage 1.0.1
-
-- The speedtest script failed if run on a test database that has no tables.
-  Now the script creates the tables if needed.  Thanks to Flavio Coelho
-  for discovering this.
-
-- Reworked the auto-reconnect logic so that applications never see
-  temporary database disconnects if possible.  Thanks to Rigel Di Scala
-  for pointing out this issue.
-
-- Improved the log messages explaining database connection failures.
-
-- Moved poll_invalidations to the common adapter base class, reducing the
-  amount of code to maintain.
-
-
-RelStorage 1.0
-
-- Added a utility for converting between storages called zodbconvert.
-
-
-RelStorage 1.0c1
-
-- The previous fix for non-ASCII characters was incorrect.  Now transaction
-  metadata is stored as raw bytes.  A schema migration is required; see
-  notes/migrate-1.0-beta.txt.
-
-- Integrated setuptools and made an egg.
-
-
-RelStorage 1.0 beta
-
-- Renamed to reflect expanding database support.
-
-- Added support for Oracle 10g.
-
-- Major overhaul with many scalability and reliability improvements,
-  particularly in the area of packing.
-
-- Moved to svn.zope.org and switched to ZPL 2.1.
-
-- Made two-phase commit optional in both Oracle and PostgreSQL.  They
-  both use commit_lock in such a way that the commit is not likely to
-  fail in the second phase.
-
-- Switched most database transaction isolation levels from serializable
-  to read committed.  It turns out that commit_lock already provides
-  the serializability guarantees we need, so it is safe to take advantage
-  of the potential speed gains.  The one major exception is the load
-  connection, which requires an unchanging view of the database.
-
-- Stored objects are now buffered in a database table rather than a file.
-
-- Stopped using the LISTEN and NOTIFY statements in PostgreSQL since
-  they are not strictly transactional in the sense we require.
-
-- Started using a prepared statement in PostgreSQL for getting the
-  newest transaction ID quickly.
-
-- Removed the code in the Oracle adapter for retrying connection attempts.
-  (It is better to just reconfigure Oracle.)
-
-- Added support for MySQL 5.0.
-
-- Added the poll_interval option.  It reduces the frequency of database
-  polls, but it also increases the potential for conflict errors on
-  servers with high write volume.
-
-- Implemented the storage iterator protocol, making it possible to copy
-  transactions to and from FileStorage and other RelStorage instances.
-
-- Fixed a bug that caused OIDs to be reused after importing transactions.
-  Added a corresponding test.
-
-- Made it possible to disable garbage collection during packing.
-  Exposed the option in zope.conf.
-
-- Valery Suhomlinov discovered a problem with non-ASCII data in transaction
-  metadata.  The problem has been fixed for all supported databases.
-
-
-PGStorage 0.4
-
-- Began using the PostgreSQL LISTEN and NOTIFY statements as a shortcut
-  for invalidation polling.
-
-- Removed the commit_order code.  The commit_order idea was intended to
-  allow concurrent commits, but that idea is a little too ambitious while
-  other more important ideas are being tested.  Something like it may
-  come later.
-
-- Improved connection management: only one database connection is
-  held continuously open per storage instance.
-
-- Reconnect to the database automatically.
-
-- Removed test mode.
-
-- Switched from using a ZODB.Connection subclass to a ZODB patch.  The
-  Connection class changes in subtle ways too often to subclass reliably;
-  a patch is much safer.
-
-- PostgreSQL 8.1 is now a dependency because PGStorage uses two phase commit.
-
-- Fixed an undo bug.  Symptom: attempting to examine the undo log revealed
-  broken pickles.  Cause: the extension field was not being wrapped in
-  psycopg2.Binary upon insert.  Solution: used psycopg2.Binary.
-  Unfortunately, this doesn't fix existing transactions people have
-  committed.  If anyone has any data to keep, fixing the old transactions
-  should be easy.
-
-- Moved from a private CVS repository to Sourceforge.
-  See http://pgstorage.sourceforge.net .  Also switched to the MIT license.
-
-- David Pratt added a basic getSize() implementation so that the Zope
-  management interface displays an estimate of the size of the database.
-
-- Turned PGStorage into a top-level package.  Python generally makes
-  top-level packages easier to install.
-
-
-PGStorage 0.3
-
-- Made compatible with Zope 3, although an undo bug apparently remains.
-
-
-PGStorage 0.2
-
-- Fixed concurrent commits, which were generating deadlocks.  Fixed by
-  adding a special table, "commit_lock", which is used for
-  synchronizing increments of commit_seq (but only at final commit.)
-  If you are upgrading from version 0.1, you need to change your
-  database using the 'psql' prompt:
-
-    create table commit_lock ();
-
-- Added speed tests and an OpenDocument spreadsheet comparing
-  FileStorage / ZEO with PGStorage.  PGStorage wins at reading objects
-  and writing a lot of small transactions, while FileStorage / ZEO
-  wins at writing big transactions.  Interestingly, they tie when
-  writing a RAM disk.
-

Copied: relstorage/tags/1.1c1/CHANGES.txt (from rev 89241, relstorage/branches/1.1/CHANGES.txt)
===================================================================
--- relstorage/tags/1.1c1/CHANGES.txt	                        (rev 0)
+++ relstorage/tags/1.1c1/CHANGES.txt	2008-08-03 00:12:31 UTC (rev 89242)
@@ -0,0 +1,202 @@
+
+Next Release
+
+- Normalized poll-invalidation patches as Solaris' patch command would not
+  accept the current format. The patches now apply with:
+  patch -d lib/python/ZODB -p0 < poll-invalidation-1-zodb-3-X-X.patch
+
+- In MySQL, Use DROP TABLE IF EXISTS instead of TRUNCATE to clear 'temp_store'
+  because:
+  - TRUNCATE has one page of caveats in the MySQL documentation.
+  - TEMPORARY TABLEs have half a page of caveats when it comes to
+    replication.
+  - The end result is that 'temp_store' may not exist on the
+    replication slave at the exact same time(s) it exists on the
+    master.
+
+RelStorage 1.1c1
+
+- Added optional memcache integration.  This is useful when the connection
+  to the relational database has high latency.
+
+- Made it possible to set the pack and memcache options in zope.conf.
+
+- Log more info when a KeyError occurs within RelStorage.
+
+
+RelStorage 1.1b2
+
+- Made the MySQL locks database-specific rather than server-wide.  This is
+  important for multi-database configurations.
+
+- In the PostgreSQL adapter, made the pack lock fall back to table locking
+  rather than advisory locks for PostgreSQL 8.1.
+
+- Changed a query for following object references (used during packing)
+  to work around a MySQL performance bug.  Thanks to Anton Stonor for
+  discovering this.
+
+
+RelStorage 1.1b1
+
+- Fixed the use of setup.py without setuptools.  Thanks to Chris Withers.
+
+- Fixed type coercion of the transaction extension field.  This fixes
+  an issue with converting databases.  Thanks to Kevin Smith for
+  discovering this.
+
+- Added logging to the pack code to help diagnose performance issues.
+
+- Additions to the object_ref table are now periodically committed
+  during pre_pack so that the work is not lost if pre_pack fails.
+
+- Modified the pack code to pack one transaction at a time and
+  release the commit lock frequently.  This should help large pack
+  operations.
+
+- Fixed buildout-based installation of the zodbconvert script.  Thanks to
+  Jim Fulton.
+
+
+RelStorage 1.0.1
+
+- The speedtest script failed if run on a test database that has no tables.
+  Now the script creates the tables if needed.  Thanks to Flavio Coelho
+  for discovering this.
+
+- Reworked the auto-reconnect logic so that applications never see
+  temporary database disconnects if possible.  Thanks to Rigel Di Scala
+  for pointing out this issue.
+
+- Improved the log messages explaining database connection failures.
+
+- Moved poll_invalidations to the common adapter base class, reducing the
+  amount of code to maintain.
+
+
+RelStorage 1.0
+
+- Added a utility for converting between storages called zodbconvert.
+
+
+RelStorage 1.0c1
+
+- The previous fix for non-ASCII characters was incorrect.  Now transaction
+  metadata is stored as raw bytes.  A schema migration is required; see
+  notes/migrate-1.0-beta.txt.
+
+- Integrated setuptools and made an egg.
+
+
+RelStorage 1.0 beta
+
+- Renamed to reflect expanding database support.
+
+- Added support for Oracle 10g.
+
+- Major overhaul with many scalability and reliability improvements,
+  particularly in the area of packing.
+
+- Moved to svn.zope.org and switched to ZPL 2.1.
+
+- Made two-phase commit optional in both Oracle and PostgreSQL.  They
+  both use commit_lock in such a way that the commit is not likely to
+  fail in the second phase.
+
+- Switched most database transaction isolation levels from serializable
+  to read committed.  It turns out that commit_lock already provides
+  the serializability guarantees we need, so it is safe to take advantage
+  of the potential speed gains.  The one major exception is the load
+  connection, which requires an unchanging view of the database.
+
+- Stored objects are now buffered in a database table rather than a file.
+
+- Stopped using the LISTEN and NOTIFY statements in PostgreSQL since
+  they are not strictly transactional in the sense we require.
+
+- Started using a prepared statement in PostgreSQL for getting the
+  newest transaction ID quickly.
+
+- Removed the code in the Oracle adapter for retrying connection attempts.
+  (It is better to just reconfigure Oracle.)
+
+- Added support for MySQL 5.0.
+
+- Added the poll_interval option.  It reduces the frequency of database
+  polls, but it also increases the potential for conflict errors on
+  servers with high write volume.
+
+- Implemented the storage iterator protocol, making it possible to copy
+  transactions to and from FileStorage and other RelStorage instances.
+
+- Fixed a bug that caused OIDs to be reused after importing transactions.
+  Added a corresponding test.
+
+- Made it possible to disable garbage collection during packing.
+  Exposed the option in zope.conf.
+
+- Valery Suhomlinov discovered a problem with non-ASCII data in transaction
+  metadata.  The problem has been fixed for all supported databases.
+
+
+PGStorage 0.4
+
+- Began using the PostgreSQL LISTEN and NOTIFY statements as a shortcut
+  for invalidation polling.
+
+- Removed the commit_order code.  The commit_order idea was intended to
+  allow concurrent commits, but that idea is a little too ambitious while
+  other more important ideas are being tested.  Something like it may
+  come later.
+
+- Improved connection management: only one database connection is
+  held continuously open per storage instance.
+
+- Reconnect to the database automatically.
+
+- Removed test mode.
+
+- Switched from using a ZODB.Connection subclass to a ZODB patch.  The
+  Connection class changes in subtle ways too often to subclass reliably;
+  a patch is much safer.
+
+- PostgreSQL 8.1 is now a dependency because PGStorage uses two phase commit.
+
+- Fixed an undo bug.  Symptom: attempting to examine the undo log revealed
+  broken pickles.  Cause: the extension field was not being wrapped in
+  psycopg2.Binary upon insert.  Solution: used psycopg2.Binary.
+  Unfortunately, this doesn't fix existing transactions people have
+  committed.  If anyone has any data to keep, fixing the old transactions
+  should be easy.
+
+- Moved from a private CVS repository to Sourceforge.
+  See http://pgstorage.sourceforge.net .  Also switched to the MIT license.
+
+- David Pratt added a basic getSize() implementation so that the Zope
+  management interface displays an estimate of the size of the database.
+
+- Turned PGStorage into a top-level package.  Python generally makes
+  top-level packages easier to install.
+
+
+PGStorage 0.3
+
+- Made compatible with Zope 3, although an undo bug apparently remains.
+
+
+PGStorage 0.2
+
+- Fixed concurrent commits, which were generating deadlocks.  Fixed by
+  adding a special table, "commit_lock", which is used for
+  synchronizing increments of commit_seq (but only at final commit.)
+  If you are upgrading from version 0.1, you need to change your
+  database using the 'psql' prompt:
+
+    create table commit_lock ();
+
+- Added speed tests and an OpenDocument spreadsheet comparing
+  FileStorage / ZEO with PGStorage.  PGStorage wins at reading objects
+  and writing a lot of small transactions, while FileStorage / ZEO
+  wins at writing big transactions.  Interestingly, they tie when
+  writing a RAM disk.
+

Deleted: relstorage/tags/1.1c1/README.txt
===================================================================
--- relstorage/branches/1.1/README.txt	2008-07-23 18:06:37 UTC (rev 88770)
+++ relstorage/tags/1.1c1/README.txt	2008-08-03 00:12:31 UTC (rev 89242)
@@ -1,232 +0,0 @@
-
-
-Overview
-========
-
-  RelStorage is a storage implementation for ZODB that stores pickles in a
-relational database.  PostgreSQL 8.1 and above (via psycopg2), MySQL 5.0.x (via
-MySQLdb), and Oracle 10g (via cx_Oracle) are currently supported.
-
-  RelStorage replaces the PGStorage project.
-
-  See:
-
-     http://wiki.zope.org/ZODB/RelStorage         (wiki)
-     http://shane.willowrise.com/                 (blog)
-     http://pypi.python.org/pypi/RelStorage       (PyPI entry and downloads)
-
-
-Features
-========
-
-  * It is a drop-in replacement for FileStorage and ZEO.
-  * There is a simple way to convert FileStorage to RelStorage and back again. 
-You can also convert a RelStorage instance to a different relational database.
-  * Designed for high volume sites: multiple ZODB instances can share the same
-database. This is similar to ZEO, but RelStorage does not require ZEO.
-  * According to some tests, RelStorage handles high concurrency better than
-the standard combination of ZEO and FileStorage.
-  * Whereas FileStorage takes longer to start as the database grows due to an
-in-memory index of all objects, RelStorage starts quickly regardless of
-database size.
-  * Supports undo and packing.
-  * Free, open source (ZPL 2.1)
-
-
-Installation in Zope
-====================
-
-  You can install RelStorage using easy_install::
-
-    easy_install RelStorage
-
-  If you are not using easy_install (part of the setuptools package), you can
-get the latest release at PyPI (http://pypi.python.org/pypi/RelStorage), then
-place the relstorage package in the lib/python directory of either the
-SOFTWARE_HOME or the INSTANCE_HOME.  You can do this with the following
-command::
-
-    python2.4 setup.py install --install-lib=${INSTANCE_HOME}/lib/python
-
-  Before you can use RelStorage, ZODB must have the invalidation polling patch
-applied.  Get it from Subversion (http://svn.zope.org/relstorage/trunk/). 
-There are two versions of the patch: one for ZODB 3.7.1 (which is part of Zope
-2.10.5) and one for ZODB 3.8.0 (which is part of Zope 2.11).  The patch has no
-effect on ZODB except when using RelStorage.  Hopefully, a future release of
-ZODB will include the feature.
-
-  You need the Python database adapter that corresponds with your database. 
-Install psycopg2, MySQLdb 1.2.2+, or cx_Oracle 4.3+.  Note that Debian Etch
-ships MySQLdb 1.2.1, but that version has a bug in BLOB handling that manifests
-itself only with certain character set configurations.  MySQLdb 1.2.2 fixes the
-bug.  Also, MySQL 5.1.23 has major bugs that lead to loss of data when packing,
-so MySQL 5.1 is not recommended at this time.
-
-  Finally, modify etc/zope.conf of your Zope instance.  Remove the main mount
-point and add one of the following blocks.  For PostgreSQL::
-
-    %import relstorage
-    <zodb_db main>
-      mount-point /
-      <relstorage>
-        <postgresql>
-          # The dsn is optional, as are each of the parameters in the dsn.
-          dsn dbname='zodb' user='username' host='localhost' password='pass'
-        </postgresql>
-      </relstorage>
-    </zodb_db>
-
-  For MySQL::
-
-    %import relstorage
-    <zodb_db main>
-      mount-point /
-      <relstorage>
-        <mysql>
-          # Most of the options provided by MySQLdb are available.
-          # See component.xml.
-          db zodb
-        </mysql>
-      </relstorage>
-    </zodb_db>
-
-  For Oracle (10g XE in this example)::
-
-    %import relstorage
-    <zodb_db main>
-      mount-point /
-      <relstorage>
-        <oracle>
-          user username
-          password pass
-          dsn XE
-        </oracle>
-     </relstorage>
-    </zodb_db>
-
-Migrating from FileStorage
-==========================
-
-  You can convert a FileStorage instance to RelStorage and back using a utility
-called ZODBConvert.  See http://wiki.zope.org/ZODB/ZODBConvert .
-
-
-Migrating from PGStorage
-========================
-
-  The following script migrates your database from PGStorage to RelStorage 1.0
-beta:
-
-    migrate.sql_
-
-    .. _migrate.sql:
-http://svn.zope.org/*checkout*/relstorage/trunk/notes/migrate.sql
-
-  After you do this, you still need to migrate from 1.0 beta to the latest
-release.
-
-
-Migrating to a new version of RelStorage
-========================================
-
-  Sometimes RelStorage needs a schema modification along with a software
-upgrade.  Hopefully, this will not often be necessary.
-
-  To migrate from version 1.0 beta to version 1.0c1, see:
-
-    migrate-1.0-beta.txt_
-
-    .. _migrate-1.0-beta.txt:
-http://svn.zope.org/*checkout*/relstorage/trunk/notes/migrate-1.0-beta.txt
-
-  To migrate from version 1.0.1 to version 1.1b1, see:
-
-    migrate-1.0.1.txt_
-
-    .. _migrate-1.0.1.txt:
-http://svn.zope.org/*checkout*/relstorage/branches/1.1/notes/migrate-1.0.1.txt
-
-
-Optional Features
-=================
-
-  poll-interval
-
-    This option is useful if you need to reduce database traffic.  If set,
-RelStorage will poll the database for changes less often.  A setting of 1 to 5
-seconds should be sufficient for most systems.  Fractional seconds are allowed.
-
-    While this setting should not affect database integrity, it increases the
-probability of basing transactions on stale data, leading to conflicts.  Thus a
-nonzero setting can hurt the performance of servers with high write volume.
-
-    To enable this feature, add a line similar to "poll-interval 2" inside a
-<relstorage> section of zope.conf.
-
-  pack-gc
-
-    If pack-gc is false, pack operations do not perform garbage collection. 
-Garbage collection is enabled by default.
-
-    If garbage collection is disabled, pack operations keep at least one
-revision of every object.  With garbage collection disabled, the pack code does
-not need to follow object references, making packing conceivably much faster. 
-However, some of that benefit may be lost due to an ever increasing number of
-unused objects.
-
-    Disabling garbage collection is also a hack that ensures inter-database
-references never break.
-
-    To disable garbage collection, add the line "pack-gc no" inside a
-<relstorage> section of zope.conf.
-
-
-Development
-===========
-
-  You can checkout from Subversion using the following command::
-
-    svn co svn://svn.zope.org/repos/main/relstorage/trunk RelStorage
-
-  You can also browse the code:
-
-    http://svn.zope.org/relstorage/trunk/
-
-  The best place to discuss development of RelStorage is on the zodb-dev
-mailing list.
-
-
-
-Probable FAQs
-==============
-
-  Q: How can I help?
-
-    A: The best way to help is to test and to provide database-specific
-expertise.  Ask questions about RelStorage on the zodb-dev mailing list.
-
-  Q: Can I perform SQL queries on the data in the database?
-
-    A: No.  Like FileStorage and DirectoryStorage, RelStorage stores the data
-as pickles, making it hard for anything but ZODB to interpret the data.  An
-earlier project called Ape attempted to store data in a truly relational way,
-but it turned out that Ape worked too much against ZODB principles and
-therefore could not be made reliable enough for production use.  RelStorage, on
-the other hand, is much closer to an ordinary ZODB storage, and is therefore
-much safer for production use.
-
-  Q: How does RelStorage performance compare with FileStorage?
-
-    A: According to benchmarks, RelStorage with PostgreSQL is often faster than
-FileStorage, especially under high concurrency.
-
-  Q: Why should I choose RelStorage?
-
-    A: Because RelStorage is a fairly small layer that builds on world-class
-databases.  These databases have proven reliability and scalability, along with
-numerous support options.
-
-  Q: Can RelStorage replace ZRS (Zope Replication Services)?
-
-    A: In theory, yes.  With RelStorage, you can use the replication features
-native to your database.  However, this capability has not yet been tested.

Copied: relstorage/tags/1.1c1/README.txt (from rev 89241, relstorage/branches/1.1/README.txt)
===================================================================
--- relstorage/tags/1.1c1/README.txt	                        (rev 0)
+++ relstorage/tags/1.1c1/README.txt	2008-08-03 00:12:31 UTC (rev 89242)
@@ -0,0 +1,280 @@
+
+
+Overview
+========
+
+  RelStorage is a storage implementation for ZODB that stores pickles in a
+relational database.  PostgreSQL 8.1 and above (via psycopg2), MySQL 5.0.x (via
+MySQLdb), and Oracle 10g (via cx_Oracle) are currently supported.
+
+  RelStorage replaces the PGStorage project.
+
+  See:
+
+     http://wiki.zope.org/ZODB/RelStorage         (wiki)
+     http://shane.willowrise.com/                 (blog)
+     http://pypi.python.org/pypi/RelStorage       (PyPI entry and downloads)
+
+
+Features
+========
+
+  * It is a drop-in replacement for FileStorage and ZEO.
+  * There is a simple way to convert FileStorage to RelStorage and back again. 
+You can also convert a RelStorage instance to a different relational database.
+  * Designed for high volume sites: multiple ZODB instances can share the same
+database. This is similar to ZEO, but RelStorage does not require ZEO.
+  * According to some tests, RelStorage handles high concurrency better than
+the standard combination of ZEO and FileStorage.
+  * Whereas FileStorage takes longer to start as the database grows due to an
+in-memory index of all objects, RelStorage starts quickly regardless of
+database size.
+  * Supports undo and packing.
+  * Free, open source (ZPL 2.1)
+
+
+Installation in Zope
+====================
+
+  You can install RelStorage using easy_install::
+
+    easy_install RelStorage
+
+  If you are not using easy_install (part of the setuptools package), you can
+get the latest release at PyPI (http://pypi.python.org/pypi/RelStorage), then
+place the relstorage package in the lib/python directory of either the
+SOFTWARE_HOME or the INSTANCE_HOME.  You can do this with the following
+command::
+
+    python2.4 setup.py install --install-lib=${INSTANCE_HOME}/lib/python
+
+  Before you can use RelStorage, ZODB must have the invalidation polling patch
+applied.  Get it from Subversion (http://svn.zope.org/relstorage/trunk/). 
+There are two versions of the patch: one for ZODB 3.7.1 (which is part of Zope
+2.10.5) and one for ZODB 3.8.0 (which is part of Zope 2.11).  The patch has no
+effect on ZODB except when using RelStorage.  Hopefully, a future release of
+ZODB will include the feature.
+
+  You need the Python database adapter that corresponds with your database. 
+Install psycopg2, MySQLdb 1.2.2+, or cx_Oracle 4.3+.  Note that Debian Etch
+ships MySQLdb 1.2.1, but that version has a bug in BLOB handling that manifests
+itself only with certain character set configurations.  MySQLdb 1.2.2 fixes the
+bug.  Also, MySQL 5.1.23 has major bugs that lead to loss of data when packing,
+so MySQL 5.1 is not recommended at this time.
+
+  Finally, modify etc/zope.conf of your Zope instance.  Remove the main mount
+point and add one of the following blocks.  For PostgreSQL::
+
+    %import relstorage
+    <zodb_db main>
+      mount-point /
+      <relstorage>
+        <postgresql>
+          # The dsn is optional, as are each of the parameters in the dsn.
+          dsn dbname='zodb' user='username' host='localhost' password='pass'
+        </postgresql>
+      </relstorage>
+    </zodb_db>
+
+  For MySQL::
+
+    %import relstorage
+    <zodb_db main>
+      mount-point /
+      <relstorage>
+        <mysql>
+          # Most of the options provided by MySQLdb are available.
+          # See component.xml.
+          db zodb
+        </mysql>
+      </relstorage>
+    </zodb_db>
+
+  For Oracle (10g XE in this example)::
+
+    %import relstorage
+    <zodb_db main>
+      mount-point /
+      <relstorage>
+        <oracle>
+          user username
+          password pass
+          dsn XE
+        </oracle>
+     </relstorage>
+    </zodb_db>
+
+Migrating from FileStorage
+==========================
+
+  You can convert a FileStorage instance to RelStorage and back using a utility
+called ZODBConvert.  See http://wiki.zope.org/ZODB/ZODBConvert .
+
+
+Migrating from PGStorage
+========================
+
+  The following script migrates your database from PGStorage to RelStorage 1.0
+beta:
+
+    migrate.sql_
+
+    .. _migrate.sql:
+http://svn.zope.org/*checkout*/relstorage/trunk/notes/migrate.sql
+
+  After you do this, you still need to migrate from 1.0 beta to the latest
+release.
+
+
+Migrating to a new version of RelStorage
+========================================
+
+  Sometimes RelStorage needs a schema modification along with a software
+upgrade.  Hopefully, this will not often be necessary.
+
+  To migrate from version 1.0 beta to version 1.0c1, see:
+
+    migrate-1.0-beta.txt_
+
+    .. _migrate-1.0-beta.txt:
+http://svn.zope.org/*checkout*/relstorage/trunk/notes/migrate-1.0-beta.txt
+
+  To migrate from version 1.0.1 to version 1.1, see:
+
+    migrate-1.0.1.txt_
+
+    .. _migrate-1.0.1.txt:
+http://svn.zope.org/*checkout*/relstorage/branches/1.1/notes/migrate-1.0.1.txt
+
+
+Optional Features
+=================
+
+    Specify these options in zope.conf.
+
+    poll-interval
+
+        Defer polling the database for the specified maximum time interval.
+        Set to 0 (the default) to always poll.  Fractional seconds are
+        allowed.
+
+        Use this to lighten the database load on servers with high read
+        volume and low write volume.  A setting of 1-5 seconds is sufficient
+        for most systems.
+
+        While this setting should not affect database integrity,
+        it increases the probability of basing transactions on stale data,
+        leading to conflicts.  Thus a nonzero setting can hurt
+        the performance of servers with high write volume.
+
+    pack-gc
+
+        If pack-gc is false, pack operations do not perform
+        garbage collection.  Garbage collection is enabled by default.
+
+        If garbage collection is disabled, pack operations keep at least one
+        revision of every object.  With garbage collection disabled, the
+        pack code does not need to follow object references, making
+        packing conceivably much faster.  However, some of that benefit
+        may be lost due to an ever increasing number of unused objects.
+
+        Disabling garbage collection is also a hack that ensures
+        inter-database references never break.
+
+    pack-batch-timeout
+
+        Packing occurs in batches of transactions; this specifies the
+        timeout in seconds for each batch.  Note that some database
+        configurations have unpredictable I/O performance
+        and might stall much longer than the timeout.
+        The default timeout is 5.0 seconds.
+
+    pack-duty-cycle
+
+        After each batch, the pack code pauses for a time to
+        allow concurrent transactions to commit.  The pack-duty-cycle
+        specifies what fraction of time should be spent on packing.
+        For example, if the duty cycle is 0.75, then 75% of the time
+        will be spent packing: a 6 second pack batch
+        will be followed by a 2 second delay.  The duty cycle should
+        be greater than 0.0 and less than or equal to 1.0.  Specify
+        1.0 for no delay between batches.
+
+        The default is 0.5.  Raise it to finish packing faster; lower it
+        to reduce the effect of packing on transaction commit performance.
+
+    pack-max-delay
+
+        This specifies a maximum delay between pack batches.  Sometimes
+        the database takes an extra long time to finish a pack batch; at
+        those times it is useful to cap the delay imposed by the
+        pack-duty-cycle.  The default is 20 seconds.
+
+    cache-servers
+
+        Specifies a list of memcache servers.  Enabling memcache integration
+        is useful if the connection to the relational database has high
+        latency and the connection to memcache has significantly lower
+        latency.  On the other hand, if the connection to the relational
+        database already has low latency, memcache integration may actually
+        hurt overall performance.
+
+        Provide a list of host:port pairs, separated by whitespace.
+        "127.0.0.1:11211" is a common setting.  The default is to disable
+        memcache integration.
+
+    cache-module-name
+
+        Specifies which Python memcache module to use.  The default is
+        "memcache", a pure Python module.  An alternative module is
+        "cmemcache".  This setting has no effect unless cache-servers is set.
+
+Development
+===========
+
+  You can checkout from Subversion using the following command::
+
+    svn co svn://svn.zope.org/repos/main/relstorage/trunk RelStorage
+
+  You can also browse the code:
+
+    http://svn.zope.org/relstorage/trunk/
+
+  The best place to discuss development of RelStorage is on the zodb-dev
+mailing list.
+
+
+
+Probable FAQs
+==============
+
+  Q: How can I help?
+
+    A: The best way to help is to test and to provide database-specific
+expertise.  Ask questions about RelStorage on the zodb-dev mailing list.
+
+  Q: Can I perform SQL queries on the data in the database?
+
+    A: No.  Like FileStorage and DirectoryStorage, RelStorage stores the data
+as pickles, making it hard for anything but ZODB to interpret the data.  An
+earlier project called Ape attempted to store data in a truly relational way,
+but it turned out that Ape worked too much against ZODB principles and
+therefore could not be made reliable enough for production use.  RelStorage, on
+the other hand, is much closer to an ordinary ZODB storage, and is therefore
+much safer for production use.
+
+  Q: How does RelStorage performance compare with FileStorage?
+
+    A: According to benchmarks, RelStorage with PostgreSQL is often faster than
+FileStorage, especially under high concurrency.
+
+  Q: Why should I choose RelStorage?
+
+    A: Because RelStorage is a fairly small layer that builds on world-class
+databases.  These databases have proven reliability and scalability, along with
+numerous support options.
+
+  Q: Can RelStorage replace ZRS (Zope Replication Services)?
+
+    A: In theory, yes.  With RelStorage, you can use the replication features
+native to your database.  However, this capability has not yet been tested.

Deleted: relstorage/tags/1.1c1/poll-invalidation-1-zodb-3-7-1.patch
===================================================================
--- relstorage/branches/1.1/poll-invalidation-1-zodb-3-7-1.patch	2008-07-23 18:06:37 UTC (rev 88770)
+++ relstorage/tags/1.1c1/poll-invalidation-1-zodb-3-7-1.patch	2008-08-03 00:12:31 UTC (rev 89242)
@@ -1,94 +0,0 @@
-diff -r 34747fbd09ec Connection.py
---- a/Connection.py	Tue Nov 20 21:57:31 2007 -0700
-+++ b/Connection.py	Fri Jan 11 21:19:00 2008 -0700
-@@ -75,8 +75,14 @@ class Connection(ExportImport, object):
-         """Create a new Connection."""
- 
-         self._db = db
--        self._normal_storage = self._storage = db._storage
--        self.new_oid = db._storage.new_oid
-+        storage = db._storage
-+        m = getattr(storage, 'bind_connection', None)
-+        if m is not None:
-+            # Use a storage instance bound to this connection.
-+            storage = m(self)
-+
-+        self._normal_storage = self._storage = storage
-+        self.new_oid = storage.new_oid
-         self._savepoint_storage = None
- 
-         self.transaction_manager = self._synch = self._mvcc = None
-@@ -170,6 +176,12 @@ class Connection(ExportImport, object):
-         # Multi-database support
-         self.connections = {self._db.database_name: self}
- 
-+        # Allow the storage to decide whether invalidations should
-+        # propagate between connections.  If the storage provides MVCC
-+        # semantics, it is better to not propagate invalidations between
-+        # connections.
-+        self._propagate_invalidations = getattr(
-+            self._storage, 'propagate_invalidations', True)
- 
-     def add(self, obj):
-         """Add a new object 'obj' to the database and assign it an oid."""
-@@ -267,6 +279,11 @@ class Connection(ExportImport, object):
-             self.transaction_manager.unregisterSynch(self)
-             self._synch = None
- 
-+        # If the storage wants to know, tell it this connection is closing.
-+        m = getattr(self._storage, 'connection_closing', None)
-+        if m is not None:
-+            m()
-+
-         if primary:
-             for connection in self.connections.values():
-                 if connection is not self:
-@@ -295,6 +312,10 @@ class Connection(ExportImport, object):
- 
-     def invalidate(self, tid, oids):
-         """Notify the Connection that transaction 'tid' invalidated oids."""
-+        if not self._propagate_invalidations:
-+            # The storage disabled inter-connection invalidation.
-+            return
-+
-         self._inv_lock.acquire()
-         try:
-             if self._txn_time is None:
-@@ -438,8 +459,23 @@ class Connection(ExportImport, object):
-         self._registered_objects = []
-         self._creating.clear()
- 
-+    def _poll_invalidations(self):
-+        """Poll and process object invalidations provided by the storage.
-+        """
-+        m = getattr(self._storage, 'poll_invalidations', None)
-+        if m is not None:
-+            # Poll the storage for invalidations.
-+            invalidated = m()
-+            if invalidated is None:
-+                # special value: the transaction is so old that
-+                # we need to flush the whole cache.
-+                self._cache.invalidate(self._cache.cache_data.keys())
-+            elif invalidated:
-+                self._cache.invalidate(invalidated)
-+
-     # Process pending invalidations.
-     def _flush_invalidations(self):
-+        self._poll_invalidations()
-         self._inv_lock.acquire()
-         try:
-             # Non-ghostifiable objects may need to read when they are
-diff -r 34747fbd09ec DB.py
---- a/DB.py	Tue Nov 20 21:57:31 2007 -0700
-+++ b/DB.py	Wed Nov 28 18:33:12 2007 -0700
-@@ -260,6 +260,10 @@ class DB(object):
-             storage.store(z64, None, file.getvalue(), '', t)
-             storage.tpc_vote(t)
-             storage.tpc_finish(t)
-+        if hasattr(storage, 'connection_closing'):
-+            # Let the storage release whatever resources it used for loading
-+            # the root object.
-+            storage.connection_closing()
- 
-         # Multi-database setup.
-         if databases is None:

Copied: relstorage/tags/1.1c1/poll-invalidation-1-zodb-3-7-1.patch (from rev 89241, relstorage/branches/1.1/poll-invalidation-1-zodb-3-7-1.patch)
===================================================================
--- relstorage/tags/1.1c1/poll-invalidation-1-zodb-3-7-1.patch	                        (rev 0)
+++ relstorage/tags/1.1c1/poll-invalidation-1-zodb-3-7-1.patch	2008-08-03 00:12:31 UTC (rev 89242)
@@ -0,0 +1,96 @@
+Index: Connection.py
+===================================================================
+--- Connection.py	(revision 87280)
++++ Connection.py	(working copy)
+@@ -75,8 +75,14 @@
+         """Create a new Connection."""
+ 
+         self._db = db
+-        self._normal_storage = self._storage = db._storage
+-        self.new_oid = db._storage.new_oid
++        storage = db._storage
++        m = getattr(storage, 'bind_connection', None)
++        if m is not None:
++            # Use a storage instance bound to this connection.
++            storage = m(self)
++
++        self._normal_storage = self._storage = storage
++        self.new_oid = storage.new_oid
+         self._savepoint_storage = None
+ 
+         self.transaction_manager = self._synch = self._mvcc = None
+@@ -170,6 +176,12 @@
+         # Multi-database support
+         self.connections = {self._db.database_name: self}
+ 
++        # Allow the storage to decide whether invalidations should
++        # propagate between connections.  If the storage provides MVCC
++        # semantics, it is better to not propagate invalidations between
++        # connections.
++        self._propagate_invalidations = getattr(
++            self._storage, 'propagate_invalidations', True)
+ 
+     def add(self, obj):
+         """Add a new object 'obj' to the database and assign it an oid."""
+@@ -267,6 +279,11 @@
+             self.transaction_manager.unregisterSynch(self)
+             self._synch = None
+ 
++        # If the storage wants to know, tell it this connection is closing.
++        m = getattr(self._storage, 'connection_closing', None)
++        if m is not None:
++            m()
++
+         if primary:
+             for connection in self.connections.values():
+                 if connection is not self:
+@@ -295,6 +312,10 @@
+ 
+     def invalidate(self, tid, oids):
+         """Notify the Connection that transaction 'tid' invalidated oids."""
++        if not self._propagate_invalidations:
++            # The storage disabled inter-connection invalidation.
++            return
++
+         self._inv_lock.acquire()
+         try:
+             if self._txn_time is None:
+@@ -438,8 +459,23 @@
+         self._registered_objects = []
+         self._creating.clear()
+ 
++    def _poll_invalidations(self):
++        """Poll and process object invalidations provided by the storage.
++        """
++        m = getattr(self._storage, 'poll_invalidations', None)
++        if m is not None:
++            # Poll the storage for invalidations.
++            invalidated = m()
++            if invalidated is None:
++                # special value: the transaction is so old that
++                # we need to flush the whole cache.
++                self._cache.invalidate(self._cache.cache_data.keys())
++            elif invalidated:
++                self._cache.invalidate(invalidated)
++
+     # Process pending invalidations.
+     def _flush_invalidations(self):
++        self._poll_invalidations()
+         self._inv_lock.acquire()
+         try:
+             # Non-ghostifiable objects may need to read when they are
+Index: DB.py
+===================================================================
+--- DB.py	(revision 87280)
++++ DB.py	(working copy)
+@@ -260,6 +260,10 @@
+             storage.store(z64, None, file.getvalue(), '', t)
+             storage.tpc_vote(t)
+             storage.tpc_finish(t)
++        if hasattr(storage, 'connection_closing'):
++            # Let the storage release whatever resources it used for loading
++            # the root object.
++            storage.connection_closing()
+ 
+         # Multi-database setup.
+         if databases is None:

Deleted: relstorage/tags/1.1c1/poll-invalidation-1-zodb-3-8-0.patch
===================================================================
--- relstorage/branches/1.1/poll-invalidation-1-zodb-3-8-0.patch	2008-07-23 18:06:37 UTC (rev 88770)
+++ relstorage/tags/1.1c1/poll-invalidation-1-zodb-3-8-0.patch	2008-08-03 00:12:31 UTC (rev 89242)
@@ -1,95 +0,0 @@
-diff -r 72bf2fd94b66 src/ZODB/Connection.py
---- a/src/ZODB/Connection.py	Wed Jan 30 23:23:05 2008 -0700
-+++ b/src/ZODB/Connection.py	Wed Jan 30 23:38:51 2008 -0700
-@@ -90,8 +90,15 @@ class Connection(ExportImport, object):
-         self.connections = {self._db.database_name: self}
- 
-         self._version = version
--        self._normal_storage = self._storage = db._storage
--        self.new_oid = db._storage.new_oid
-+
-+        storage = db._storage
-+        m = getattr(storage, 'bind_connection', None)
-+        if m is not None:
-+            # Use a storage instance bound to this connection.
-+            storage = m(self)
-+        self._normal_storage = self._storage = storage
-+        self.new_oid = storage.new_oid
-+
-         self._savepoint_storage = None
- 
-         # Do we need to join a txn manager?
-@@ -151,6 +158,12 @@ class Connection(ExportImport, object):
-         # in the cache on abort and in other connections on finish.
-         self._modified = []
- 
-+        # Allow the storage to decide whether invalidations should
-+        # propagate between connections.  If the storage provides MVCC
-+        # semantics, it is better to not propagate invalidations between
-+        # connections.
-+        self._propagate_invalidations = getattr(
-+            self._storage, 'propagate_invalidations', True)
- 
-         # _invalidated queues invalidate messages delivered from the DB
-         # _inv_lock prevents one thread from modifying the set while
-@@ -297,6 +310,11 @@ class Connection(ExportImport, object):
-         if self._opened:
-             self.transaction_manager.unregisterSynch(self)
- 
-+        # If the storage wants to know, tell it this connection is closing.
-+        m = getattr(self._storage, 'connection_closing', None)
-+        if m is not None:
-+            m()
-+
-         if primary:
-             for connection in self.connections.values():
-                 if connection is not self:
-@@ -328,6 +346,10 @@ class Connection(ExportImport, object):
- 
-     def invalidate(self, tid, oids):
-         """Notify the Connection that transaction 'tid' invalidated oids."""
-+        if not self._propagate_invalidations:
-+            # The storage disabled inter-connection invalidation.
-+            return
-+
-         self._inv_lock.acquire()
-         try:
-             if self._txn_time is None:
-@@ -469,8 +491,23 @@ class Connection(ExportImport, object):
-         self._registered_objects = []
-         self._creating.clear()
- 
-+    def _poll_invalidations(self):
-+        """Poll and process object invalidations provided by the storage.
-+        """
-+        m = getattr(self._storage, 'poll_invalidations', None)
-+        if m is not None:
-+            # Poll the storage for invalidations.
-+            invalidated = m()
-+            if invalidated is None:
-+                # special value: the transaction is so old that
-+                # we need to flush the whole cache.
-+                self._cache.invalidate(self._cache.cache_data.keys())
-+            elif invalidated:
-+                self._cache.invalidate(invalidated)
-+
-     # Process pending invalidations.
-     def _flush_invalidations(self):
-+        self._poll_invalidations()
-         self._inv_lock.acquire()
-         try:
-             # Non-ghostifiable objects may need to read when they are
-diff -r 72bf2fd94b66 src/ZODB/DB.py
---- a/src/ZODB/DB.py	Wed Jan 30 23:23:05 2008 -0700
-+++ b/src/ZODB/DB.py	Wed Jan 30 23:38:51 2008 -0700
-@@ -284,6 +284,10 @@ class DB(object):
-             storage.store(z64, None, file.getvalue(), '', t)
-             storage.tpc_vote(t)
-             storage.tpc_finish(t)
-+        if hasattr(storage, 'connection_closing'):
-+            # Let the storage release whatever resources it used for loading
-+            # the root object.
-+            storage.connection_closing()
- 
-         # Multi-database setup.
-         if databases is None:

Copied: relstorage/tags/1.1c1/poll-invalidation-1-zodb-3-8-0.patch (from rev 89241, relstorage/branches/1.1/poll-invalidation-1-zodb-3-8-0.patch)
===================================================================
--- relstorage/tags/1.1c1/poll-invalidation-1-zodb-3-8-0.patch	                        (rev 0)
+++ relstorage/tags/1.1c1/poll-invalidation-1-zodb-3-8-0.patch	2008-08-03 00:12:31 UTC (rev 89242)
@@ -0,0 +1,97 @@
+Index: Connection.py
+===================================================================
+--- Connection.py	(revision 87666)
++++ Connection.py	(working copy)
+@@ -90,8 +90,15 @@
+         self.connections = {self._db.database_name: self}
+ 
+         self._version = version
+-        self._normal_storage = self._storage = db._storage
+-        self.new_oid = db._storage.new_oid
++
++        storage = db._storage
++        m = getattr(storage, 'bind_connection', None)
++        if m is not None:
++            # Use a storage instance bound to this connection.
++            storage = m(self)
++        self._normal_storage = self._storage = storage
++        self.new_oid = storage.new_oid
++
+         self._savepoint_storage = None
+ 
+         # Do we need to join a txn manager?
+@@ -151,6 +158,12 @@
+         # in the cache on abort and in other connections on finish.
+         self._modified = []
+ 
++        # Allow the storage to decide whether invalidations should
++        # propagate between connections.  If the storage provides MVCC
++        # semantics, it is better to not propagate invalidations between
++        # connections.
++        self._propagate_invalidations = getattr(
++            self._storage, 'propagate_invalidations', True)
+ 
+         # _invalidated queues invalidate messages delivered from the DB
+         # _inv_lock prevents one thread from modifying the set while
+@@ -297,6 +310,11 @@
+         if self._opened:
+             self.transaction_manager.unregisterSynch(self)
+ 
++        # If the storage wants to know, tell it this connection is closing.
++        m = getattr(self._storage, 'connection_closing', None)
++        if m is not None:
++            m()
++
+         if primary:
+             for connection in self.connections.values():
+                 if connection is not self:
+@@ -328,6 +346,10 @@
+ 
+     def invalidate(self, tid, oids):
+         """Notify the Connection that transaction 'tid' invalidated oids."""
++        if not self._propagate_invalidations:
++            # The storage disabled inter-connection invalidation.
++            return
++
+         self._inv_lock.acquire()
+         try:
+             if self._txn_time is None:
+@@ -469,8 +491,23 @@
+         self._registered_objects = []
+         self._creating.clear()
+ 
++    def _poll_invalidations(self):
++        """Poll and process object invalidations provided by the storage.
++        """
++        m = getattr(self._storage, 'poll_invalidations', None)
++        if m is not None:
++            # Poll the storage for invalidations.
++            invalidated = m()
++            if invalidated is None:
++                # special value: the transaction is so old that
++                # we need to flush the whole cache.
++                self._cache.invalidate(self._cache.cache_data.keys())
++            elif invalidated:
++                self._cache.invalidate(invalidated)
++
+     # Process pending invalidations.
+     def _flush_invalidations(self):
++        self._poll_invalidations()
+         self._inv_lock.acquire()
+         try:
+             # Non-ghostifiable objects may need to read when they are
+Index: DB.py
+===================================================================
+--- DB.py	(revision 87666)
++++ DB.py	(working copy)
+@@ -284,6 +284,10 @@
+             storage.store(z64, None, file.getvalue(), '', t)
+             storage.tpc_vote(t)
+             storage.tpc_finish(t)
++        if hasattr(storage, 'connection_closing'):
++            # Let the storage release whatever resources it used for loading
++            # the root object.
++            storage.connection_closing()
+ 
+         # Multi-database setup.
+         if databases is None:

Copied: relstorage/tags/1.1c1/relstorage (from rev 89241, relstorage/branches/1.1/relstorage)



More information about the Checkins mailing list