[Checkins] SVN: relstorage/trunk/ Moved to GitHub.
Shane Hathaway
cvs-admin at zope.org
Sun Mar 17 21:51:50 UTC 2013
Log message for revision 130136:
Moved to GitHub.
Changed:
D relstorage/trunk/CHANGES.txt
D relstorage/trunk/COPYRIGHT.txt
D relstorage/trunk/HISTORY.txt
D relstorage/trunk/LICENSE.txt
D relstorage/trunk/MANIFEST.in
A relstorage/trunk/MOVED_TO_GITHUB
D relstorage/trunk/README.txt
D relstorage/trunk/bootstrap.py
D relstorage/trunk/buildout-oracle.cfg
D relstorage/trunk/buildout.cfg
D relstorage/trunk/notes/
D relstorage/trunk/poll-invalidation-zodb-3-7.patch
D relstorage/trunk/poll-invalidation-zodb-3-8.patch
D relstorage/trunk/relstorage/
D relstorage/trunk/repltest/
D relstorage/trunk/scripts/
D relstorage/trunk/setup.py
-=-
Deleted: relstorage/trunk/CHANGES.txt
===================================================================
--- relstorage/trunk/CHANGES.txt 2013-03-17 00:50:26 UTC (rev 130135)
+++ relstorage/trunk/CHANGES.txt 2013-03-17 21:51:49 UTC (rev 130136)
@@ -1,368 +0,0 @@
-Next Release
-------------
-
-- zodbconvert: Add an --incremental option to the zodbconvert script,
- letting you convert additional transactions at a later date, or
- update a non-live copy of your database, copying over missing
- transactions.
-
-- Replication: Added the ro-replica-conf option, which tells RelStorage
- to use a read-only database replica for load connections. This makes
- it easy for RelStorage clients to take advantage of read-only
- database replicas.
-
-- Replication: When the database connection is stale (such as when
- RelStorage switches to an asynchronous replica that is not yet up to
- date), RelStorage will now raise ReadConflictError by default.
- Ideally, the application will react to the error by transparently
- retrying the transaction, while the database gets up to date. A
- subsequent transaction will no longer be stale.
-
-- Replication: Added the revert-when-stale option. When this option is
- true and the database connection is stale, RelStorage reverts the
- ZODB connection to the stale state rather than raise
- ReadConflictError. This option is intended for highly available,
- read-only ZODB clients. This option would probably confuse users of
- read-write ZODB clients, whose changes would sometimes seem to be
- temporarily reverted.
-
-- Caching: Use the database name as the cache-prefix by default. This
- will hopefully help people who accidentally use a single cache for
- multiple databases.
-
-1.5.1 (2011-11-12)
-------------------
-
-- Packing: Lowered garbage collection object reference finding log level to
- debug; this stage takes mere seconds, even in large sites, but could produce
- 10s of thousands of lines of log output.
-
-- RelStorage was opening a test database connection (and was leaving it
- idle in a transaction with recent ZODB versions that support
- IMVCCStorage.) RelStorage no longer opens that test connection.
-
-- zodbconvert: Avoid holding a list of all transactions in memory.
-
-- Just after installing the database schema, verify the schema was
- created correctly. This affects MySQL in particular.
-
-1.5.0 (2011-06-30)
-------------------
-
-- PostgreSQL: Fixed another minor compatibility issue with PostgreSQL 9.0.
- Packing raised an error when the client used old an version of libpq.
-
-- Delete empty transactions in batches of 1000 rows instead of all in one
- go, to prevent holding the transaction lock for longer than absolutely
- necessary.
-
-- Oracle: Fix object reference downloading performance for large RelStorage
- databases during the garbage collection phase of a pack.
-
-- Oracle, PostgreSQL: Switch to storing ZODB blob in chunks up to 4GB
- (the maximum supported by cx_Oracle) or 2GB (PostgreSQL maximum blob size)
- to maximize blob reading and writing performance.
-
- The PostgreSQL blob_chunk schema changed to support this, see
- notes/migrate-to-1.5.txt to update existing databases.
-
-- zodbconvert: When copying a database containing blobs, ensure the source
- blob file exists long enough to copy it.
-
-1.5.0b2 (2011-03-02)
---------------------
-
-- Better packing based on experience with large databases. Thanks
- to Martijn Pieters!
-
- - Added more feedback to the packing process. It'll now report
- during batch commit how much of the total work has been
- completed, but at most every .1% of the total number of
- transactions or objects to process.
-
- - Renamed the --dry-run option to --prepack and added a
- --use-prepack-state to zodbpack. With these 2 options the
- pre-pack and pack phases can be run separately, allowing re-use
- of the pre-pack analysis data or even delegating the pre-pack
- phase off to a separate server.
-
- - Replaced the packing duty cycle with a nowait locking strategy.
- The pack operation will now request the commit lock but pauses if
- it is already taken. It releases the lock after every batch
- (defaulting to 1 second processing). This makes the packing
- process faster while at the same time yielding to regular ZODB
- commits when busy.
-
- - Do not hold the commit lock during pack cleanup while deleting
- rows from the object reference tables; these tables are
- pack-specific and regular ZODB commits never touch these.
-
-- Added an option to control schema creation / updating on startup.
- Setting the ``create-schema`` option to false will let you use
- RelStorage without a schema update.
-
-- Fixed compatibility with PostgreSQL 9.0, which is capable of
- returning a new 'hex' type to the client. Some builds of psycopg2
- return garbage or raise an error when they see the new type. The fix
- was to encode more SQL query responses using base 64.
-
-- With the new shared-blob-dir option set to false, it was possible
- for a thread to read a partially downloaded blob. Fixed. Thanks for
- the report from Maurits van Rees.
-
-- Support for "shared-blob-dir false" now requires ZODB 3.9 or better.
- The code in the ZODB 3.8 version of ZODB.blob is not compatible with
- BlobCacheLayout, leading to blob filename collisions.
-
-1.5.0b1 (2011-02-05)
---------------------
-
-- Added a state_size column to object_state, making it possible
- to query the size of objects without loading the state. The new
- column is intended for gathering statistics. A schema migration
- is required.
-
-- Added more logging during zodbconvert to show that something is
- happening and give an indication of how far along the process is.
-
-- Fixed a missing import in the blob cache cleanup code.
-
-- Added a --dry-run option to zodbpack.
-
-- Replaced the graph traversal portion of the pack code with
- a more efficient implementation using Python sets (instead of SQL).
- The new code is much faster for packing databases with deeply
- nested objects.
-
-1.5.0a1 (2010-10-21)
---------------------
-
-- Added an option to store ZODB blobs in the database. The option is
- called "shared-blob-dir" and it behaves very much like the ZEO
- option of the same name. Blobs stored in the database are broken
- into chunks to reduce the impact on RAM.
-
-- Require setuptools or distribute. Plain distutils is not sufficient.
-
-1.4.2 (2011-02-04)
-------------------
-
-- Fixed compatibility with ZODB 3.10. As reported by Jürgen Herrmann,
- there was a problem with conflict errors. The RelStorage patch of the
- sync() method now works with ZODB 3.10.
-
-- Fixed a bug in packing history-free databases. If changes were
- made to the database during the pack, the pack code could delete
- too many objects. Thanks to Chris Withers for writing test code
- that revealed the bug. A schema migration is required for history-free
- databases; see notes/migration-to-1.4.txt.
-
-- Enabled logging to stderr in zodbpack.
-
-1.4.1 (2010-10-21)
-------------------
-
-- Oracle: always connect in threaded mode. Without threaded mode,
- clients of Oracle 11g sometimes segfault.
-
-1.4.0 (2010-09-30)
-------------------
-
-- Made compatible with ZODB 3.10.0b7.
-
-- Enabled ketama and compression in pylibmc_wrapper. Both options
- are better for clusters. [Helge Tesdal]
-
-- Oracle: Use a more optimal query for POSKeyError logging. [Helge Tesdal]
-
-- Fixed a NameError that occurred when getting the history of an
- object where transaction extended info was set. [Helge Tesdal]
-
-1.4.0c4 (2010-09-17)
---------------------
-
-- Worked around an Oracle RAC bug: apparently, in a RAC environment,
- the read-only transaction mode does not isolate transactions in the
- manner specified by the documentation, so Oracle users now have to
- use serializable isolation like everyone else. It's slower but more
- reliable.
-
-- Use the client time instead of the database server time as a factor
- in the transaction ID. RelStorage was using the database server time
- to reduce the need for synchronized clocks, but in practice, that
- policy broke tests and did not really avoid the need to synchronize
- clocks. Also, the effect of unsynchronized clocks is predictable
- and manageable: you'll get bunches of transactions with sequential
- timestamps.
-
-- If the database returns an object from the future (which should never
- happen), generate a ReadConflictError, hopefully giving the application
- a chance to recover. The most likely causes of this are a broken
- database and threading bugs.
-
-1.4.0c3 (2010-07-31)
---------------------
-
-- Always update the RelStorage cache when opening a database connection for
- loading, even when no ZODB Connection is using the storage. Otherwise,
- code that used the storage interface directly could cause the cache
- to fall out of sync; the effects would be seen in the next
- ZODB.Connection.
-
-- Added a ZODB monkey patch that passes the "force" parameter to the
- sync method. This should help the poll-interval option do its job
- better.
-
-1.4.0c2 (2010-07-28)
---------------------
-
-- Fixed a subtle bug in the cache code that could lead to an
- AssertionError indicating a cache inconsistency. The inconsistency
- was caused by after_poll(), which was ignoring the randomness of
- the order of the list of recent changes, leading it to sometimes
- put the wrong transfer ID in the "delta_after" dicts. Also expanded
- the AssertionError with debugging info, since cache inconsistency
- can still be caused by database misconfiguration and mismatched
- client versions.
-
-- Oracle: updated the migration notes. The relstorage_util package
- is not needed after all.
-
-1.4.0c1 (2010-06-19)
---------------------
-
-- History-preserving storages now replace objects on restore instead of
- just inserting them. This should solve problems people were
- having with the zodbconvert utility.
-
-- Oracle: call the DBMS_LOCK.REQUEST function directly instead of using
- a small package named ``relstorage_util``. The ``relstorage_util``
- package was designed as a secure way to access the DBMS_LOCK package,
- but the package turned out to be confusing to DBAs and provided no
- real security advantage. People who have already deployed
- RelStorage 1.4.x on Oracle need to do the following:
-
- GRANT EXECUTE ON DBMS_LOCK TO <zodb_user>;
-
- You can also drop the ``relstorage_util`` package. Keep the
- ``relstorage_op`` package.
-
-- Made compatible with ZODB 3.10.
-
-- MySQL: specify the transaction isolation mode for every connection,
- since the default is apparently not necessarily "read committed"
- anymore.
-
-1.4.0b3 (2010-02-02)
---------------------
-
-- Auto-reconnect in new_oid().
-
-1.4.0b2 (2010-01-30)
---------------------
-
-- Include all test subpackages in setup.py.
-
-- Raise an error if MySQL reverts to MyISAM rather than using the InnoDB
- storage engine.
-
-1.4.0b1 (2009-11-17)
---------------------
-
-- Added the keep-history option. Set it to false to keep no history.
- (Packing is still required for garbage collection and blob deletion.)
-
-- Added the replica-conf and replica-timeout options. Set replica-conf
- to a filename containing the location of database replicas. Changes
- to the file take effect at transaction boundaries.
-
-- Expanded the option documentation in README.txt.
-
-- Revised the way RelStorage uses memcached. Minimized the number of
- trips to both the cache server and the database.
-
-- Added an in-process pickle cache that serves a function similar to the
- ZEO cache.
-
-- Added a wrapper module for pylibmc.
-
-- Store operations now use multi-insert and multi-delete SQL
- statements to reduce the effect of network latency.
-
-- Renamed relstorage.py to storage.py to overcome import issues.
- Also moved the Options class to options.py.
-
-- Updated the patch for ZODB 3.7 and 3.8 to fix an issue with
- blobs and subtransactions.
-
-- Divided the implementation of database adapters into many small
- objects, making the adapter code more modular. Added interfaces
- that describe the duties of each part.
-
-- Oracle: Sped up restore operations by sending short blobs inline.
-
-- Oracle: Use a timeout on commit locks. This requires installation
- of a small PL/SQL package that can access DBMS_LOCK. See README.txt.
-
-- Oracle: Used PL/SQL bulk insert operations to improve write
- performance.
-
-- PostgreSQL: use the documented ALTER SEQUENCE RESTART WITH
- statement instead of ALTER SEQUENCE START WITH.
-
-- Moved MD5 sum computation to the adapters so they can choose not
- to use MD5.
-
-- Changed loadSerial to load from the store connection only if the
- load connection can not provide the object requested.
-
-- Stopped wrapping database disconnect exceptions. Now the code
- catches and handles them directly.
-
-- Use the store connection rather than the load connection for OID
- allocation.
-
-- Detect and handle backward time travel, which can happen after
- failover to an out-of-date asynchronous slave database. For
- simplicity, invalidate the whole ZODB cache when this happens.
-
-- Replaced the speed test script with a separately distributed package,
- ``zodbshootout``.
-
-- Added the ``zodbpack`` script.
-
-1.3.0b1 (2009-09-04)
---------------------
-
-- Added support for a blob directory. No BlobStorage wrapper is needed.
- Cluster nodes will need to use a shared filesystem such as NFS or
- SMB/CIFS.
-
-- Added the blob-dir parameter to the ZConfig schema and README.txt.
-
-
-1.2.0 (2009-09-04)
-------------------
-
-- In Oracle, trim transaction descriptions longer than 2000 bytes.
-
-- When opening the database for the first time, don't issue a warning
- about the inevitable POSKeyError on the root OID.
-
-- If RelStorage tries to unpickle a corrupt object state during packing,
- it will now report the oid and tid in the log.
-
-
-1.2.0b2 (2009-05-05)
---------------------
-
-- RelStorage now implements IMVCCStorage, making it compatible with
- ZODB 3.9.0b1 and above.
-
-- Removed two-phase commit support from the PostgreSQL adapter. The
- feature turned out to be unnecessary.
-
-- Added MySQL 5.1.34 and above to the list of supportable databases.
-
-- Fixed minor test failures under Windows. Windows is now a supportable
- platform.
Deleted: relstorage/trunk/COPYRIGHT.txt
===================================================================
--- relstorage/trunk/COPYRIGHT.txt 2013-03-17 00:50:26 UTC (rev 130135)
+++ relstorage/trunk/COPYRIGHT.txt 2013-03-17 21:51:49 UTC (rev 130136)
@@ -1 +0,0 @@
-Zope Foundation and Contributors
Deleted: relstorage/trunk/HISTORY.txt
===================================================================
--- relstorage/trunk/HISTORY.txt 2013-03-17 00:50:26 UTC (rev 130135)
+++ relstorage/trunk/HISTORY.txt 2013-03-17 21:51:49 UTC (rev 130136)
@@ -1,279 +0,0 @@
-
-1.1.3 (2009-02-04)
-------------------
-
-- In rare circumstances, ZODB can legitimately commit an object twice in a
- single transaction. Fixed RelStorage to accept that.
-
-- Auto reconnect to Oracle sometimes did not work because cx_Oracle was
- raising a different kind of exception than expected. Fixed.
-
-- Included LICENSE.txt in the source distribution.
-
-
-1.1.2 (2009-01-27)
-------------------
-
-- When both cache-servers and poll-interval are set, we now poll the
- cache for changes on every request. This makes it possible to use
- a high poll-interval to reduce the database polling burden, yet
- every client can see changes immediately.
-
-- Added the pack-dry-run option, which causes pack operations to only
- populate the pack tables with the list of objects and states to pack,
- but not actually pack.
-
-- Refined the pack algorithm. It was not removing as many object states
- as it should have. As a bonus, there is now a method of adapters called
- fill_object_refs(), which could be useful for debugging. It ensures the
- object_ref table is fully populated.
-
-- Began using zc.buildout for development.
-
-- Increased automated test coverage.
-
-- Fixed KeyError reporting to not trip over a related KeyError while logging.
-
-
-1.1.1 (2008-12-27)
-------------------
-
-- Worked around MySQL performance bugs in packing. Used temporary
- tables and another column in the pack_object table. The
- other databases may benefit from the optimization as well.
-
-- Applied an optimization using setinputsizes() to the Oracle code,
- bringing write speed back up to where it was in version 1.0.
-
-
-1.1 (2008-12-19)
-----------------
-
-- Normalized poll-invalidation patches as Solaris' patch command would not
- accept the current format. The patches now apply with:
- patch -d lib/python/ZODB -p0 < poll-invalidation-1-zodb-3-X-X.patch
-
-- In MySQL, Use DROP TABLE IF EXISTS instead of TRUNCATE to clear 'temp_store'
- because:
-
- - TRUNCATE has one page of caveats in the MySQL documentation.
- - TEMPORARY TABLEs have half a page of caveats when it comes to
- replication.
- - The end result is that 'temp_store' may not exist on the
- replication slave at the exact same time(s) it exists on the
- master.
-
-- Implemented the database size query in MySQL, based on a patch from
- Kazuhiko Shiozaki. Thanks!
-
-- Optimized Oracle object retrieval by causing BLOBs to be sent inline
- when possible, based on a patch by Helge Tesdal. By default, the
- optimization is activated automatically when cx_Oracle 5 is used.
-
-- Updated the storage iterator code to be compatible with ZODB 3.9.
- The RelStorage tests now pass with the shane-poll-invalidations branch
- of ZODB 3.9.
-
-- Added a translation of README.txt to Brazilian Portuguese by
- Rogerio Ferreira. Thanks!
-
-
-1.1c1
------
-
-- Added optional memcache integration. This is useful when the connection
- to the relational database has high latency.
-
-- Made it possible to set the pack and memcache options in zope.conf.
-
-- Log more info when a KeyError occurs within RelStorage.
-
-
-1.1b2
------
-
-- Made the MySQL locks database-specific rather than server-wide. This is
- important for multi-database configurations.
-
-- In the PostgreSQL adapter, made the pack lock fall back to table locking
- rather than advisory locks for PostgreSQL 8.1.
-
-- Changed a query for following object references (used during packing)
- to work around a MySQL performance bug. Thanks to Anton Stonor for
- discovering this.
-
-
-1.1b1
------
-
-- Fixed the use of setup.py without setuptools. Thanks to Chris Withers.
-
-- Fixed type coercion of the transaction extension field. This fixes
- an issue with converting databases. Thanks to Kevin Smith for
- discovering this.
-
-- Added logging to the pack code to help diagnose performance issues.
-
-- Additions to the object_ref table are now periodically committed
- during pre_pack so that the work is not lost if pre_pack fails.
-
-- Modified the pack code to pack one transaction at a time and
- release the commit lock frequently. This should help large pack
- operations.
-
-- Fixed buildout-based installation of the zodbconvert script. Thanks to
- Jim Fulton.
-
-
-1.0.1
------
-
-- The speedtest script failed if run on a test database that has no tables.
- Now the script creates the tables if needed. Thanks to Flavio Coelho
- for discovering this.
-
-- Reworked the auto-reconnect logic so that applications never see
- temporary database disconnects if possible. Thanks to Rigel Di Scala
- for pointing out this issue.
-
-- Improved the log messages explaining database connection failures.
-
-- Moved poll_invalidations to the common adapter base class, reducing the
- amount of code to maintain.
-
-
-1.0
----
-
-- Added a utility for converting between storages called zodbconvert.
-
-
-1.0c1
------
-
-- The previous fix for non-ASCII characters was incorrect. Now transaction
- metadata is stored as raw bytes. A schema migration is required; see
- notes/migrate-1.0-beta.txt.
-
-- Integrated setuptools and made an egg.
-
-
-1.0 beta
---------
-
-- Renamed to reflect expanding database support.
-
-- Added support for Oracle 10g.
-
-- Major overhaul with many scalability and reliability improvements,
- particularly in the area of packing.
-
-- Moved to svn.zope.org and switched to ZPL 2.1.
-
-- Made two-phase commit optional in both Oracle and PostgreSQL. They
- both use commit_lock in such a way that the commit is not likely to
- fail in the second phase.
-
-- Switched most database transaction isolation levels from serializable
- to read committed. It turns out that commit_lock already provides
- the serializability guarantees we need, so it is safe to take advantage
- of the potential speed gains. The one major exception is the load
- connection, which requires an unchanging view of the database.
-
-- Stored objects are now buffered in a database table rather than a file.
-
-- Stopped using the LISTEN and NOTIFY statements in PostgreSQL since
- they are not strictly transactional in the sense we require.
-
-- Started using a prepared statement in PostgreSQL for getting the
- newest transaction ID quickly.
-
-- Removed the code in the Oracle adapter for retrying connection attempts.
- (It is better to just reconfigure Oracle.)
-
-- Added support for MySQL 5.0.
-
-- Added the poll_interval option. It reduces the frequency of database
- polls, but it also increases the potential for conflict errors on
- servers with high write volume.
-
-- Implemented the storage iterator protocol, making it possible to copy
- transactions to and from FileStorage and other RelStorage instances.
-
-- Fixed a bug that caused OIDs to be reused after importing transactions.
- Added a corresponding test.
-
-- Made it possible to disable garbage collection during packing.
- Exposed the option in zope.conf.
-
-- Valery Suhomlinov discovered a problem with non-ASCII data in transaction
- metadata. The problem has been fixed for all supported databases.
-
-
-PGStorage history
-=================
-
-0.4
----
-
-- Began using the PostgreSQL LISTEN and NOTIFY statements as a shortcut
- for invalidation polling.
-
-- Removed the commit_order code. The commit_order idea was intended to
- allow concurrent commits, but that idea is a little too ambitious while
- other more important ideas are being tested. Something like it may
- come later.
-
-- Improved connection management: only one database connection is
- held continuously open per storage instance.
-
-- Reconnect to the database automatically.
-
-- Removed test mode.
-
-- Switched from using a ZODB.Connection subclass to a ZODB patch. The
- Connection class changes in subtle ways too often to subclass reliably;
- a patch is much safer.
-
-- PostgreSQL 8.1 is now a dependency because PGStorage uses two phase commit.
-
-- Fixed an undo bug. Symptom: attempting to examine the undo log revealed
- broken pickles. Cause: the extension field was not being wrapped in
- psycopg2.Binary upon insert. Solution: used psycopg2.Binary.
- Unfortunately, this doesn't fix existing transactions people have
- committed. If anyone has any data to keep, fixing the old transactions
- should be easy.
-
-- Moved from a private CVS repository to Sourceforge.
- See http://pgstorage.sourceforge.net . Also switched to the MIT license.
-
-- David Pratt added a basic getSize() implementation so that the Zope
- management interface displays an estimate of the size of the database.
-
-- Turned PGStorage into a top-level package. Python generally makes
- top-level packages easier to install.
-
-
-0.3
----
-
-- Made compatible with Zope 3, although an undo bug apparently remains.
-
-
-0.2
----
-
-- Fixed concurrent commits, which were generating deadlocks. Fixed by
- adding a special table, "commit_lock", which is used for
- synchronizing increments of commit_seq (but only at final commit.)
- If you are upgrading from version 0.1, you need to change your
- database using the 'psql' prompt:
-
- create table commit_lock ();
-
-- Added speed tests and an OpenDocument spreadsheet comparing
- FileStorage / ZEO with PGStorage. PGStorage wins at reading objects
- and writing a lot of small transactions, while FileStorage / ZEO
- wins at writing big transactions. Interestingly, they tie when
- writing a RAM disk.
-
Deleted: relstorage/trunk/LICENSE.txt
===================================================================
--- relstorage/trunk/LICENSE.txt 2013-03-17 00:50:26 UTC (rev 130135)
+++ relstorage/trunk/LICENSE.txt 2013-03-17 21:51:49 UTC (rev 130136)
@@ -1,44 +0,0 @@
-Zope Public License (ZPL) Version 2.1
-
-A copyright notice accompanies this license document that identifies the
-copyright holders.
-
-This license has been certified as open source. It has also been designated as
-GPL compatible by the Free Software Foundation (FSF).
-
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are met:
-
-1. Redistributions in source code must retain the accompanying copyright
-notice, this list of conditions, and the following disclaimer.
-
-2. Redistributions in binary form must reproduce the accompanying copyright
-notice, this list of conditions, and the following disclaimer in the
-documentation and/or other materials provided with the distribution.
-
-3. Names of the copyright holders must not be used to endorse or promote
-products derived from this software without prior written permission from the
-copyright holders.
-
-4. The right to distribute this software or to use it for any purpose does not
-give you the right to use Servicemarks (sm) or Trademarks (tm) of the
-copyright
-holders. Use of them is covered by separate agreement with the copyright
-holders.
-
-5. If any files are modified, you must cause the modified files to carry
-prominent notices stating that you changed the files and the date of any
-change.
-
-Disclaimer
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY EXPRESSED
-OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
-OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
-EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY DIRECT, INDIRECT,
-INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
-PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
-LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
-EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Deleted: relstorage/trunk/MANIFEST.in
===================================================================
--- relstorage/trunk/MANIFEST.in 2013-03-17 00:50:26 UTC (rev 130135)
+++ relstorage/trunk/MANIFEST.in 2013-03-17 21:51:49 UTC (rev 130136)
@@ -1,2 +0,0 @@
-include MANIFEST.in *.txt *.py *.patch
-recursive-include relstorage *.py *.xml *.txt
Added: relstorage/trunk/MOVED_TO_GITHUB
===================================================================
--- relstorage/trunk/MOVED_TO_GITHUB (rev 0)
+++ relstorage/trunk/MOVED_TO_GITHUB 2013-03-17 21:51:49 UTC (rev 130136)
@@ -0,0 +1 @@
+https://github.com/zodb/relstorage
Deleted: relstorage/trunk/README.txt
===================================================================
--- relstorage/trunk/README.txt 2013-03-17 00:50:26 UTC (rev 130135)
+++ relstorage/trunk/README.txt 2013-03-17 21:51:49 UTC (rev 130136)
@@ -1,766 +0,0 @@
-
-RelStorage is a storage implementation for ZODB that stores pickles in
-a relational database. PostgreSQL 8.1 and above (via psycopg2), MySQL
-5.0.32+ / 5.1.34+ (via MySQLdb 1.2.2 and above), and Oracle 10g and 11g
-(via cx_Oracle) are currently supported. RelStorage replaced the
-PGStorage project.
-
-.. contents::
-
-
-Features
-========
-
-* It is a drop-in replacement for FileStorage and ZEO.
-* There is a simple way to convert FileStorage to RelStorage and back again.
- You can also convert a RelStorage instance to a different relational database.
-* Designed for high volume sites: multiple ZODB instances can share the same
- database. This is similar to ZEO, but RelStorage does not require ZEO.
-* According to some tests, RelStorage handles high concurrency better than
- the standard combination of ZEO and FileStorage.
-* Whereas FileStorage takes longer to start as the database grows due to an
- in-memory index of all objects, RelStorage starts quickly regardless of
- database size.
-* Supports undo, packing, and filesystem-based ZODB blobs.
-* Both history-preserving and history-free storage are available.
-* Capable of failover to replicated SQL databases.
-* Free, open source (ZPL 2.1)
-
-
-Installation
-============
-
-You can install RelStorage using easy_install::
-
- easy_install RelStorage
-
-RelStorage requires a version of ZODB that is aware of MVCC storages.
-ZODB 3.9 supports RelStorage without any patches. ZODB 3.7 and 3.8 can
-support RelStorage if you first apply a patch to ZODB. You can get
-versions of ZODB with the patch already applied here:
-
- http://packages.willowrise.org
-
-The patches are also included in the source distribution of RelStorage.
-
-You need the Python database adapter that corresponds with your database.
-Install psycopg2, MySQLdb 1.2.2+, or cx_Oracle 4.3+.
-
-Configuring Your Database
--------------------------
-
-You need to configure a database and user account for RelStorage.
-RelStorage will populate the database with its schema the first time it
-connects.
-
-PostgreSQL
-~~~~~~~~~~
-
-If you installed PostgreSQL from a binary package, you probably have a
-user account named ``postgres``. Since PostgreSQL respects the name of
-the logged-in user by default, switch to the ``postgres`` account to
-create the RelStorage user and database. Even ``root`` does not have
-the PostgreSQL privileges that the ``postgres`` account has. For
-example::
-
- $ sudo su - postgres
- $ createuser --pwprompt zodbuser
- $ createdb -O zodbuser zodb
-
-New PostgreSQL accounts often require modifications to ``pg_hba.conf``,
-which contains host-based access control rules. The location of
-``pg_hba.conf`` varies, but ``/etc/postgresql/8.4/main/pg_hba.conf`` is
-common. PostgreSQL processes the rules in order, so add new rules
-before the default rules rather than after. Here is a sample rule that
-allows only local connections by ``zodbuser`` to the ``zodb``
-database::
-
- local zodb zodbuser md5
-
-PostgreSQL re-reads ``pg_hba.conf`` when you ask it to reload its
-configuration file::
-
- /etc/init.d/postgresql reload
-
-MySQL
-~~~~~
-
-Use the ``mysql`` utility to create the database and user account. Note
-that the ``-p`` option is usually required. You must use the ``-p``
-option if the account you are accessing requires a password, but you
-should not use the ``-p`` option if the account you are accessing does
-not require a password. If you do not provide the ``-p`` option, yet
-the account requires a password, the ``mysql`` utility will not prompt
-for a password and will fail to authenticate.
-
-Most users can start the ``mysql`` utility with the following shell
-command, using any login account::
-
- $ mysql -u root -p
-
-Here are some sample SQL statements for creating the user and database::
-
- CREATE USER 'zodbuser'@'localhost' IDENTIFIED BY 'mypassword';
- CREATE DATABASE zodb;
- GRANT ALL ON zodb.* TO 'zodbuser'@'localhost';
- FLUSH PRIVILEGES;
-
-Oracle
-~~~~~~
-
-Initial setup will require ``SYS`` privileges. Using Oracle 10g XE, you
-can start a ``SYS`` session with the following shell commands::
-
- $ su - oracle
- $ sqlplus / as sysdba
-
-You need to create a database user and grant execute privileges on
-the DBMS_LOCK package to that user.
-Here are some sample SQL statements for creating the database user
-and granting the required permissions::
-
- CREATE USER zodb IDENTIFIED BY mypassword;
- GRANT CONNECT, RESOURCE, CREATE TABLE, CREATE SEQUENCE TO zodb;
- GRANT EXECUTE ON DBMS_LOCK TO zodb;
-
-Configuring Plone
------------------
-
-To install RelStorage in Plone, see the instructions in the following
-article:
-
- http://shane.willowrise.com/archives/how-to-install-plone-with-relstorage-and-mysql/
-
-Plone uses the ``plone.recipe.zope2instance`` Buildout recipe to
-generate zope.conf, so the easiest way to configure RelStorage in a
-Plone site is to set the ``rel-storage`` parameter in ``buildout.cfg``.
-The ``rel-storage`` parameter contains options separated by newlines,
-with these values:
-
- * ``type``: any database type supported (``postgresql``, ``mysql``,
- or ``oracle``)
- * RelStorage options like ``cache-servers`` and ``poll-interval``
- * Adapter-specific options
-
-An example::
-
- rel-storage =
- type mysql
- db plone
- user plone
- passwd PASSWORD
-
-Configuring Zope 2
-------------------
-
-To integrate RelStorage in Zope 2, specify a RelStorage backend in
-``etc/zope.conf``. Remove the main mount point and add one of the
-following blocks. For PostgreSQL::
-
- %import relstorage
- <zodb_db main>
- mount-point /
- <relstorage>
- <postgresql>
- # The dsn is optional, as are each of the parameters in the dsn.
- dsn dbname='zodb' user='username' host='localhost' password='pass'
- </postgresql>
- </relstorage>
- </zodb_db>
-
-For MySQL::
-
- %import relstorage
- <zodb_db main>
- mount-point /
- <relstorage>
- <mysql>
- # Most of the options provided by MySQLdb are available.
- # See component.xml.
- db zodb
- </mysql>
- </relstorage>
- </zodb_db>
-
-For Oracle (10g XE in this example)::
-
- %import relstorage
- <zodb_db main>
- mount-point /
- <relstorage>
- <oracle>
- user username
- password pass
- dsn XE
- </oracle>
- </relstorage>
- </zodb_db>
-
-To add ZODB blob support, provide a blob-dir option that specifies
-where to store the blobs. For example::
-
- %import relstorage
- <zodb_db main>
- mount-point /
- <relstorage>
- blob-dir ./blobs
- <postgresql>
- dsn dbname='zodb' user='username' host='localhost' password='pass'
- </postgresql>
- </relstorage>
- </zodb_db>
-
-Configuring ``repoze.zodbconn``
--------------------------------
-
-To use RelStorage with ``repoze.zodbconn``, a package that makes ZODB
-available to WSGI applications, create a configuration file with
-contents similar to the following::
-
- %import relstorage
- <zodb main>
- <relstorage>
- <mysql>
- db zodb
- </mysql>
- </relstorage>
- cache-size 100000
- </zodb>
-
-``repoze.zodbconn`` expects a ZODB URI. Use a URI of the form
-``zconfig://path/to/configuration#main``.
-
-
-Included Utilities
-==================
-
-``zodbconvert``
----------------
-
-RelStorage comes with a script named ``zodbconvert`` that converts
-databases between formats. Use it to convert a FileStorage instance to
-RelStorage and back, or to convert between different kinds of
-RelStorage instances, or to convert other kinds of storages that
-support the storage iterator protocol.
-
-When converting between two history-preserving databases (note that
-FileStorage uses a history-preserving format), ``zodbconvert``
-preserves all objects and transactions, meaning you can still use the
-ZODB undo feature after the conversion, and you can convert back using
-the same process in reverse. When converting from a history-free
-database to either a history-free database or a history-preserving
-database, ``zodbconvert`` retains all data, but the converted
-transactions will not be undoable. When converting from a
-history-preserving storage to a history-free storage, ``zodbconvert``
-drops all historical information during the conversion.
-
-How to use ``zodbconvert``
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Create a ZConfig style configuration file that specifies two storages,
-one named "source", the other "destination". The configuration file
-format is very much like zope.conf. Then run ``zodbconvert``, providing
-the name of the configuration file as a parameter.
-
-The utility does not modify the source storage. Before copying the
-data, the utility verifies the destination storage is completely empty.
-If the destination storage is not empty, the utility aborts without
-making any changes to the destination. (Adding transactions to an
-existing database is complex and out of scope for ``zodbconvert``.)
-
-Here is a sample ``zodbconvert`` configuration file::
-
- <filestorage source>
- path /zope/var/Data.fs
- </filestorage>
-
- <relstorage destination>
- <mysql>
- db zodb
- </mysql>
- </relstorage>
-
-This configuration file specifies that the utility should copy all of
-the transactions from Data.fs to a MySQL database called "zodb". If you
-want to reverse the conversion, exchange the names "source" and
-"destination". All storage types and storage options available in
-zope.conf are also available in this configuration file.
-
-Options for ``zodbconvert``
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
- ``--clear``
- Clears all data from the destination storage before copying. Use
- this only if you are certain the destination has no useful data.
- Currently only works when the destination is a RelStorage instance.
-
- ``--dry-run``
- Opens both storages and analyzes what would be copied, but does not
- actually copy.
-
-
-``zodbpack``
-------------
-
-RelStorage also comes with a script named ``zodbpack`` that packs any
-ZODB storage that allows concurrent connections (including RelStorage
-and ZEO, but not including FileStorage). Use ``zodbpack`` in ``cron``
-scripts. Pass the script the name of a configuration file that lists
-the storages to pack, in ZConfig format. An example configuration file::
-
- <relstorage>
- pack-gc true
- <mysql>
- db zodb
- </mysql>
- </relstorage>
-
-Options for ``zodbpack``
-~~~~~~~~~~~~~~~~~~~~~~~~
-
- ``--days`` or ``-d``
- Specifies how many days of historical data to keep. Defaults to 0,
- meaning no history is kept. This is meaningful even for
- history-free storages, since unreferenced objects are not removed
- from the database until the specified number of days have passed.
-
- ``--prepack``
- Instructs the storage to only run the pre-pack phase of the pack but not
- actually delete anything. This is equivalent to specifying
- ``pack-prepack-only true`` in the storage options.
-
- ``--use-prepack-state``
- Instructs the storage to only run the deletion (packing) phase, skipping
- the pre-pack analysis phase. This is equivalento to specifying
- ``pack-skip-prepack true`` in the storage options.
-
-
-Migrating to a new version of RelStorage
-========================================
-
-Sometimes RelStorage needs a schema modification along with a software
-upgrade. Hopefully, this will not often be necessary.
-
-Migration to RelStorage version 1.5 requires a schema upgrade.
-See `migrate-to-1.5.txt`_.
-
-.. _`migrate-to-1.5.txt`: http://svn.zope.org/*checkout*/relstorage/trunk/notes/migrate-to-1.5.txt
-
-Migration to RelStorage version 1.4.2 requires a schema upgrade if
-you are using a history-free database (meaning keep-history is false).
-See `migrate-to-1.4.txt`_.
-
-.. _`migrate-to-1.4.txt`: http://svn.zope.org/*checkout*/relstorage/trunk/notes/migrate-to-1.4.txt
-
-See the `notes subdirectory`_ if you are upgrading from an older version.
-
-.. _`notes subdirectory`: http://svn.zope.org/relstorage/trunk/notes/
-
-
-RelStorage Options
-==================
-
-Specify these options in zope.conf, as parameters for the
-``relstorage.storage.RelStorage`` constructor, or as attributes of a
-``relstorage.options.Options`` instance. In the latter two cases, use
-underscores instead of dashes in the option names.
-
-``name``
- The name of the storage. Defaults to a descriptive name that
- includes the adapter connection parameters, except the database
- password.
-
-``read-only``
- If true, only reads may be executed against the storage.
-
-``blob-dir``
- If supplied, the storage will provide ZODB blob support; this
- option specifies the name of the directory to hold blob data.
- The directory will be created if it does not exist. If no value
- (or an empty value) is provided, then no blob support will be
- provided.
-
-``shared-blob-dir``
- If true (the default), the blob directory is assumed to be
- shared among all clients using NFS or similar; blob data will
- be stored only on the filesystem and not in the database. If
- false, blob data is stored in the relational database and the
- blob directory holds a cache of blobs. When this option is
- false, the blob directory should not be shared among clients.
-
- This option must be true when using ZODB 3.8, because ZODB 3.8
- is not compatible with the file layout required for a blob
- cache. Use ZODB 3.9 or later if you want to store blobs in
- the relational database.
-
-``blob-cache-size``
- Maximum size of the blob cache, in bytes. If empty (the
- default), the cache size isn't checked and the blob directory
- will grow without bounds. This should be either empty or
- significantly larger than the largest blob you store. At least
- 1 gigabyte is recommended for typical databases. More is
- recommended if you store large files such as videos, CD/DVD
- images, or virtual machine images.
-
- This option allows suffixes such as "mb" or "gb".
- This option is ignored if shared-blob-dir is true.
-
-``blob-cache-size-check``
- Blob cache check size as percent of blob-cache-size: "10" means
- "10%". The blob cache size will be checked when this many bytes
- have been loaded into the cache. Defaults to 10% of the blob
- cache size.
-
- This option is ignored if shared-blob-dir is true.
-
-``blob-chunk-size``
- When ZODB blobs are stored in MySQL, RelStorage breaks them into
- chunks to minimize the impact on RAM. This option specifies the chunk
- size for new blobs. On PostgreSQL and Oracle, this value is used as
- the memory buffer size for blob upload and download operations. The
- default is 1048576 (1 mebibyte).
-
- This option allows suffixes such as "mb" or "gb".
- This option is ignored if shared-blob-dir is true.
-
-``keep-history``
- If this option is set to true (the default), the adapter
- will create and use a history-preserving database schema
- (like FileStorage). A history-preserving schema supports
- ZODB-level undo, but also grows more quickly and requires extensive
- packing on a regular basis.
-
- If this option is set to false, the adapter will create and
- use a history-free database schema. Undo will not be supported,
- but the database will not grow as quickly. The database will
- still require regular garbage collection (which is accessible
- through the database pack mechanism.)
-
- This option must not change once the database schema has
- been installed, because the schemas for history-preserving and
- history-free storage are different. If you want to convert
- between a history-preserving and a history-free database, use
- the ``zodbconvert`` utility to copy to a new database.
-
-``replica-conf``
- If this option is provided, it specifies a text file that
- contains a list of database replicas the adapter can choose
- from. For MySQL and PostgreSQL, put in the replica file a list
- of ``host:port`` or ``host`` values, one per line. For Oracle,
- put in a list of DSN values. Blank lines and lines starting
- with ``#`` are ignored.
-
- The adapter prefers the first replica specified in the file. If
- the first is not available, the adapter automatically tries the
- rest of the replicas, in order. If the file changes, the
- adapter will drop existing SQL database connections and make
- new connections when ZODB starts a new transaction.
-
-``ro-replica-conf``
- Like the ``replica-conf`` option, but the referenced text file
- provides a list of database replicas to use only for read-only
- load connections. This allows RelStorage to load objects from
- read-only database replicas, while using read-write replicas
- for all other database interactions.
-
- If this option is not provided, load connections will fall back
- to the replica pool specified by ``replica-conf``. If
- ``ro-replica-conf`` is provided but ``replica-conf`` is not,
- RelStorage will use replicas for load connections but not for
- other database interactions.
-
- Note that if read-only replicas are asynchronous, the next
- interaction after a write operation might not be up to date.
- When that happens, RelStorage will log a "backward time travel"
- warning and clear the ZODB cache to prevent consistency errors.
- This will likely result in temporarily reduced performance as
- the ZODB cache is repopulated.
-
-``replica-timeout``
- If this option has a nonzero value, when the adapter selects
- a replica other than the primary replica, the adapter will
- try to revert to the primary replica after the specified
- timeout (in seconds). The default is 600, meaning 10 minutes.
-
-``revert-when-stale``
- Specifies what to do when a database connection is stale.
- This is especially applicable to asynchronously replicated
- databases: RelStorage could switch to a replica that is not
- yet up to date.
-
- When ``revert-when-stale`` is ``false`` (the default) and the
- database connection is stale, RelStorage will raise a
- ReadConflictError if the application tries to read or write
- anything. The application should react to the
- ReadConflictError by retrying the transaction after a delay
- (possibly multiple times.) Once the database catches
- up, a subsequent transaction will see the update and the
- ReadConflictError will not occur again.
-
- When ``revert-when-stale`` is ``true`` and the database connection
- is stale, RelStorage will log a warning, clear the affected
- ZODB connection cache (to prevent consistency errors), and let
- the application continue with database state from
- an earlier transaction. This behavior is intended to be useful
- for highly available, read-only ZODB clients. Enabling this
- option on ZODB clients that read and write the database is
- likely to cause confusion for users whose changes
- seem to be temporarily reverted.
-
-``poll-interval``
- Defer polling the database for the specified maximum time interval,
- in seconds. Set to 0 (the default) to always poll. Fractional
- seconds are allowed. Use this to lighten the database load on
- servers with high read volume and low write volume.
-
- The poll-interval option works best in conjunction with
- the cache-servers option. If both are enabled, RelStorage will
- poll a single cache key for changes on every request.
- The database will not be polled unless the cache indicates
- there have been changes, or the timeout specified by poll-interval
- has expired. This configuration keeps clients fully up to date,
- while removing much of the polling burden from the database.
- A good cluster configuration is to use memcache servers
- and a high poll-interval (say, 60 seconds).
-
- This option can be used without the cache-servers option,
- but a large poll-interval without cache-servers increases the
- probability of basing transactions on stale data, which does not
- affect database consistency, but does increase the probability
- of conflict errors, leading to low performance.
-
-``pack-gc``
- If pack-gc is false, pack operations do not perform
- garbage collection. Garbage collection is enabled by default.
-
- If garbage collection is disabled, pack operations keep at least one
- revision of every object. With garbage collection disabled, the
- pack code does not need to follow object references, making
- packing conceivably much faster. However, some of that benefit
- may be lost due to an ever increasing number of unused objects.
-
- Disabling garbage collection is also a hack that ensures
- inter-database references never break.
-
-``pack-prepack-only``
- If pack-prepack-only is true, pack operations perform a full analysis
- of what to pack, but no data is actually removed. After a pre-pack,
- the pack_object, pack_state, and pack_state_tid tables are filled
- with the list of object states and objects that would have been
- removed. If pack-gc is true, the object_ref table will also be fully
- populated. The object_ref table can be queried to discover references
- between stored objects.
-
-``pack-skip-prepack``
- If pack-skip-prepack is true, the pre-pack phase is skipped and it
- is assumed the pack_object, pack_state and pack_state_tid tables have
- been filled already. Thus packing will only affect records already
- targeted for packing by a previous pre-pack analysis run.
-
- Use this option together with pack-prepack-only to split packing into
- distinct phases, where each phase can be run during different
- timeslots, or where a pre-pack analysis is run on a copy of the
- database to alleviate a production database load.
-
-``pack-batch-timeout``
- Packing occurs in batches of transactions; this specifies the
- timeout in seconds for each batch. Note that some database
- configurations have unpredictable I/O performance
- and might stall much longer than the timeout.
- The default timeout is 1.0 seconds.
-
-``pack-commit-busy-delay``
- Before each pack batch, the commit lock is requested. If the lock is
- already held by for a regular commit, packing is paused for a short
- while. This option specifies how long the pack process should be
- paused before attempting to get the commit lock again.
- The default delay is 5.0 seconds.
-
-``cache-servers``
- Specifies a list of memcached servers. Using memcached with
- RelStorage improves the speed of frequent object accesses while
- slightly reducing the speed of other operations.
-
- Provide a list of host:port pairs, separated by whitespace.
- "127.0.0.1:11211" is a common setting. Some memcached modules,
- such as pylibmc, allow you to specify a path to a Unix socket
- instead of a host:port pair.
-
- The default is to disable memcached integration.
-
-``cache-module-name``
- Specifies which Python memcache module to use. The default is
- "relstorage.pylibmc_wrapper", which requires pylibmc. An
- alternative module is "memcache", a pure Python module. If you
- use the memcache module, use at least version 1.47. This
- option has no effect unless cache-servers is set.
-
-``cache-prefix``
- The prefix for all keys in the cache. All clients using a
- database should use the same cache-prefix. Defaults to the
- database name. (For example, in PostgreSQL, the database
- name is determined by executing ``SELECT current_database()``.)
- Set this if you have multiple databases with the same name.
-
-``cache-local-mb``
- RelStorage caches pickled objects in memory, similar to a ZEO
- cache. This cache is shared between threads. This option
- configures the approximate maximum amount of memory the cache
- should consume, in megabytes. It defaults to 10. Set to
- 0 to disable the in-memory cache.
-
-``cache-delta-size-limit``
- This is an advanced option. RelStorage uses a system of
- checkpoints to improve the cache hit rate. This option
- configures how many objects should be stored before creating a
- new checkpoint. The default is 10000.
-
-``commit-lock-timeout``
- During commit, RelStorage acquires a database-wide lock. This
- option specifies how long to wait for the lock before
- failing the attempt to commit. The default is 30 seconds.
-
- The MySQL and Oracle adapters support this option. The
- PostgreSQL adapter currently does not.
-
-``commit-lock-id``
- During commit, RelStorage acquires a database-wide lock. This
- option specifies the lock ID. This option currently applies
- only to the Oracle adapter.
-
-``create-schema``
- Normally, RelStorage will create or update the database schema on
- start-up. Set this option to false if you need to connect to a
- RelStorage database without automatic creation or updates.
-
-Adapter Options
-===============
-
-PostgreSQL Adapter Options
---------------------------
-
-The PostgreSQL adapter accepts:
-
-``dsn``
- Specifies the data source name for connecting to PostgreSQL.
- A PostgreSQL DSN is a list of parameters separated with
- whitespace. A typical DSN looks like::
-
- dbname='zodb' user='username' host='localhost' password='pass'
-
- If dsn is omitted, the adapter will connect to a local database with
- no password. Both the user and database name will match the
- name of the owner of the current process.
-
-MySQL Adapter Options
----------------------
-
-The MySQL adapter accepts most parameters supported by the MySQL-python
-library, including:
-
-``host``
- string, host to connect
-``user``
- string, user to connect as
-``passwd``
- string, password to use
-``db``
- string, database to use
-``port``
- integer, TCP/IP port to connect to
-``unix_socket``
- string, location of unix_socket (UNIX-ish only)
-``conv``
- mapping, maps MySQL FIELD_TYPE.* to Python functions which convert a
- string to the appropriate Python type
-``connect_timeout``
- number of seconds to wait before the connection attempt fails.
-``compress``
- if set, gzip compression is enabled
-``named_pipe``
- if set, connect to server via named pipe (Windows only)
-``init_command``
- command which is run once the connection is created
-``read_default_file``
- see the MySQL documentation for mysql_options()
-``read_default_group``
- see the MySQL documentation for mysql_options()
-``client_flag``
- client flags from MySQLdb.constants.CLIENT
-``load_infile``
- int, non-zero enables LOAD LOCAL INFILE, zero disables
-
-Oracle Adapter Options
-----------------------
-
-The Oracle adapter accepts:
-
-``user``
- The Oracle account name
-``password``
- The Oracle account password
-``dsn``
- The Oracle data source name. The Oracle client library will
- normally expect to find the DSN in /etc/oratab.
-
-Development
-===========
-
-You can check out from Subversion using the following command::
-
- svn co svn://svn.zope.org/repos/main/relstorage/trunk RelStorage
-
-You can also browse the code:
-
- http://svn.zope.org/relstorage/trunk/
-
-The best place to discuss development of RelStorage is on the zodb-dev
-mailing list.
-
-
-
-FAQs
-====
-
-Q: How can I help improve RelStorage?
-
- A: The best way to help is to test and to provide database-specific
- expertise. Ask questions about RelStorage on the zodb-dev mailing list.
-
-Q: Can I perform SQL queries on the data in the database?
-
- A: No. Like FileStorage and DirectoryStorage, RelStorage stores the data
- as pickles, making it hard for anything but ZODB to interpret the data. An
- earlier project called Ape attempted to store data in a truly relational
- way, but it turned out that Ape worked too much against ZODB principles and
- therefore could not be made reliable enough for production use. RelStorage,
- on the other hand, is much closer to an ordinary ZODB storage, and is
- therefore more appropriate for production use.
-
-Q: How does RelStorage performance compare with FileStorage?
-
- A: According to benchmarks, RelStorage with PostgreSQL is often faster than
- FileStorage, especially under high concurrency.
-
-Q: Why should I choose RelStorage?
-
- A: Because RelStorage is a fairly small layer that builds on world-class
- databases. These databases have proven reliability and scalability, along
- with numerous support options.
-
-Q: Can RelStorage replace ZRS (Zope Replication Services)?
-
- A: Yes, RelStorage inherits the asynchronous master/slave replication
- capability of MySQL and other databases. RelStorage has also been
- proven to work with Oracle RAC.
-
-Q: How do I set up an environment to run the RelStorage tests?
-
- A: See README.txt in the relstorage/tests directory.
-
-
-Project URLs
-============
-
-* http://pypi.python.org/pypi/RelStorage (PyPI entry and downloads)
-* http://shane.willowrise.com/ (blog)
Deleted: relstorage/trunk/bootstrap.py
===================================================================
--- relstorage/trunk/bootstrap.py 2013-03-17 00:50:26 UTC (rev 130135)
+++ relstorage/trunk/bootstrap.py 2013-03-17 21:51:49 UTC (rev 130136)
@@ -1,54 +0,0 @@
-##############################################################################
-#
-# Copyright (c) 2007 Zope Foundation and Contributors.
-# All Rights Reserved.
-#
-# This software is subject to the provisions of the Zope Public License,
-# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
-# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
-# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
-# FOR A PARTICULAR PURPOSE.
-#
-##############################################################################
-"""Bootstrap a buildout-based project
-
-Simply run this script in a directory containing a buildout.cfg.
-The script accepts buildout command-line options, so you can
-use the -c option to specify an alternate configuration file.
-
-$Id: bootstrap.py,v 1.1 2009/01/10 15:13:39 shane Exp $
-"""
-
-import os, shutil, sys, tempfile, urllib2
-
-tmpeggs = tempfile.mkdtemp()
-
-ez = {}
-exec urllib2.urlopen('http://peak.telecommunity.com/dist/ez_setup.py'
- ).read() in ez
-ez['use_setuptools'](to_dir=tmpeggs, download_delay=0)
-
-# use pkg_resources from the package just downloaded.
-del sys.modules['pkg_resources']
-import pkg_resources
-
-cmd = 'from setuptools.command.easy_install import main; main()'
-if sys.platform == 'win32':
- cmd = '"%s"' % cmd # work around spawn lamosity on windows
-
-ws = pkg_resources.working_set
-assert os.spawnle(
- os.P_WAIT, sys.executable, sys.executable,
- '-c', cmd, '-mqNxd', tmpeggs, 'zc.buildout',
- dict(os.environ,
- PYTHONPATH=
- ws.find(pkg_resources.Requirement.parse('setuptools')).location
- ),
- ) == 0
-
-ws.add_entry(tmpeggs)
-ws.require('zc.buildout')
-import zc.buildout.buildout
-zc.buildout.buildout.main(sys.argv[1:] + ['bootstrap'])
-shutil.rmtree(tmpeggs)
Deleted: relstorage/trunk/buildout-oracle.cfg
===================================================================
--- relstorage/trunk/buildout-oracle.cfg 2013-03-17 00:50:26 UTC (rev 130135)
+++ relstorage/trunk/buildout-oracle.cfg 2013-03-17 21:51:49 UTC (rev 130136)
@@ -1,3 +0,0 @@
-[buildout]
-extends = buildout.cfg
-eggs += cx_Oracle
Deleted: relstorage/trunk/buildout.cfg
===================================================================
--- relstorage/trunk/buildout.cfg 2013-03-17 00:50:26 UTC (rev 130135)
+++ relstorage/trunk/buildout.cfg 2013-03-17 21:51:49 UTC (rev 130136)
@@ -1,37 +0,0 @@
-[buildout]
-develop = .
-base-parts = test python omelette coverage-test coverage-report
-parts = ${buildout:base-parts}
-eggs = relstorage
- psycopg2
- MySQL-python
- ZODB3 [test]
- zope.testrunner
-
-[test]
-recipe = zc.recipe.testrunner
-eggs = ${buildout:eggs}
-
-[python]
-recipe = zc.recipe.egg
-eggs = ${buildout:eggs}
-interpreter = py
-
-[omelette]
-recipe = collective.recipe.omelette
-eggs = ${buildout:eggs}
-ignores = setuptools
-
-[coverage-test]
-recipe = zc.recipe.testrunner
-eggs = ${buildout:eggs}
-defaults = ['--coverage', 'coverage']
-
-[coverage-report]
-recipe = zc.recipe.egg
-eggs =
- ${buildout:eggs}
- z3c.coverage
-scripts = coveragereport
-arguments = ('coverage', 'coverage/report')
-
Deleted: relstorage/trunk/poll-invalidation-zodb-3-7.patch
===================================================================
--- relstorage/trunk/poll-invalidation-zodb-3-7.patch 2013-03-17 00:50:26 UTC (rev 130135)
+++ relstorage/trunk/poll-invalidation-zodb-3-7.patch 2013-03-17 21:51:49 UTC (rev 130136)
@@ -1,96 +0,0 @@
-Index: Connection.py
-===================================================================
---- Connection.py (revision 104552)
-+++ Connection.py (working copy)
-@@ -75,8 +75,14 @@
- """Create a new Connection."""
-
- self._db = db
-- self._normal_storage = self._storage = db._storage
-- self.new_oid = db._storage.new_oid
-+ storage = db._storage
-+ m = getattr(storage, 'bind_connection', None)
-+ if m is not None:
-+ # Use a storage instance bound to this connection.
-+ storage = m(self)
-+
-+ self._normal_storage = self._storage = storage
-+ self.new_oid = storage.new_oid
- self._savepoint_storage = None
-
- self.transaction_manager = self._synch = self._mvcc = None
-@@ -170,6 +176,12 @@
- # Multi-database support
- self.connections = {self._db.database_name: self}
-
-+ # Allow the storage to decide whether invalidations should
-+ # propagate between connections. If the storage provides MVCC
-+ # semantics, it is better to not propagate invalidations between
-+ # connections.
-+ self._propagate_invalidations = getattr(
-+ self._storage, 'propagate_invalidations', True)
-
- def add(self, obj):
- """Add a new object 'obj' to the database and assign it an oid."""
-@@ -267,6 +279,11 @@
- self.transaction_manager.unregisterSynch(self)
- self._synch = None
-
-+ # If the storage wants to know, tell it this connection is closing.
-+ m = getattr(self._storage, 'connection_closing', None)
-+ if m is not None:
-+ m()
-+
- if primary:
- for connection in self.connections.values():
- if connection is not self:
-@@ -438,8 +455,23 @@
- self._registered_objects = []
- self._creating.clear()
-
-+ def _poll_invalidations(self):
-+ """Poll and process object invalidations provided by the storage.
-+ """
-+ m = getattr(self._storage, 'poll_invalidations', None)
-+ if m is not None:
-+ # Poll the storage for invalidations.
-+ invalidated = m()
-+ if invalidated is None:
-+ # special value: the transaction is so old that
-+ # we need to flush the whole cache.
-+ self._cache.invalidate(self._cache.cache_data.keys())
-+ elif invalidated:
-+ self._cache.invalidate(invalidated)
-+
- # Process pending invalidations.
- def _flush_invalidations(self):
-+ self._poll_invalidations()
- self._inv_lock.acquire()
- try:
- # Non-ghostifiable objects may need to read when they are
-@@ -698,6 +730,10 @@
- """Indicate confirmation that the transaction is done."""
-
- def callback(tid):
-+ if not self._propagate_invalidations:
-+ # The storage disabled inter-connection invalidation.
-+ return
-+
- d = dict.fromkeys(self._modified)
- self._db.invalidate(tid, d, self)
- # It's important that the storage calls the passed function
-Index: DB.py
-===================================================================
---- DB.py (revision 104552)
-+++ DB.py (working copy)
-@@ -260,6 +260,10 @@
- storage.store(z64, None, file.getvalue(), '', t)
- storage.tpc_vote(t)
- storage.tpc_finish(t)
-+ if hasattr(storage, 'connection_closing'):
-+ # Let the storage release whatever resources it used for loading
-+ # the root object.
-+ storage.connection_closing()
-
- # Multi-database setup.
- if databases is None:
Deleted: relstorage/trunk/poll-invalidation-zodb-3-8.patch
===================================================================
--- relstorage/trunk/poll-invalidation-zodb-3-8.patch 2013-03-17 00:50:26 UTC (rev 130135)
+++ relstorage/trunk/poll-invalidation-zodb-3-8.patch 2013-03-17 21:51:49 UTC (rev 130136)
@@ -1,96 +0,0 @@
-Index: Connection.py
-===================================================================
---- Connection.py (revision 104546)
-+++ Connection.py (working copy)
-@@ -90,8 +90,15 @@
- self.connections = {self._db.database_name: self}
-
- self._version = version
-- self._normal_storage = self._storage = db._storage
-- self.new_oid = db._storage.new_oid
-+
-+ storage = db._storage
-+ m = getattr(storage, 'bind_connection', None)
-+ if m is not None:
-+ # Use a storage instance bound to this connection.
-+ storage = m(self)
-+ self._normal_storage = self._storage = storage
-+ self.new_oid = storage.new_oid
-+
- self._savepoint_storage = None
-
- # Do we need to join a txn manager?
-@@ -151,6 +158,12 @@
- # in the cache on abort and in other connections on finish.
- self._modified = []
-
-+ # Allow the storage to decide whether invalidations should
-+ # propagate between connections. If the storage provides MVCC
-+ # semantics, it is better to not propagate invalidations between
-+ # connections.
-+ self._propagate_invalidations = getattr(
-+ self._storage, 'propagate_invalidations', True)
-
- # _invalidated queues invalidate messages delivered from the DB
- # _inv_lock prevents one thread from modifying the set while
-@@ -297,6 +310,11 @@
- if self._opened:
- self.transaction_manager.unregisterSynch(self)
-
-+ # If the storage wants to know, tell it this connection is closing.
-+ m = getattr(self._storage, 'connection_closing', None)
-+ if m is not None:
-+ m()
-+
- if primary:
- for connection in self.connections.values():
- if connection is not self:
-@@ -469,8 +487,23 @@
- self._registered_objects = []
- self._creating.clear()
-
-+ def _poll_invalidations(self):
-+ """Poll and process object invalidations provided by the storage.
-+ """
-+ m = getattr(self._storage, 'poll_invalidations', None)
-+ if m is not None:
-+ # Poll the storage for invalidations.
-+ invalidated = m()
-+ if invalidated is None:
-+ # special value: the transaction is so old that
-+ # we need to flush the whole cache.
-+ self._cache.invalidate(self._cache.cache_data.keys())
-+ elif invalidated:
-+ self._cache.invalidate(invalidated)
-+
- # Process pending invalidations.
- def _flush_invalidations(self):
-+ self._poll_invalidations()
- self._inv_lock.acquire()
- try:
- # Non-ghostifiable objects may need to read when they are
-@@ -748,6 +781,9 @@
- """Indicate confirmation that the transaction is done."""
-
- def callback(tid):
-+ if not self._propagate_invalidations:
-+ # The storage disabled inter-connection invalidation.
-+ return
- d = dict.fromkeys(self._modified)
- self._db.invalidate(tid, d, self)
- # It's important that the storage calls the passed function
-Index: DB.py
-===================================================================
---- DB.py (revision 104546)
-+++ DB.py (working copy)
-@@ -284,6 +284,10 @@
- storage.store(z64, None, file.getvalue(), '', t)
- storage.tpc_vote(t)
- storage.tpc_finish(t)
-+ if hasattr(storage, 'connection_closing'):
-+ # Let the storage release whatever resources it used for loading
-+ # the root object.
-+ storage.connection_closing()
-
- # Multi-database setup.
- if databases is None:
Deleted: relstorage/trunk/setup.py
===================================================================
--- relstorage/trunk/setup.py 2013-03-17 00:50:26 UTC (rev 130135)
+++ relstorage/trunk/setup.py 2013-03-17 21:51:49 UTC (rev 130136)
@@ -1,85 +0,0 @@
-##############################################################################
-#
-# Copyright (c) 2008 Zope Foundation and Contributors.
-# All Rights Reserved.
-#
-# This software is subject to the provisions of the Zope Public License,
-# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
-# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
-# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
-# FOR A PARTICULAR PURPOSE.
-#
-##############################################################################
-"""A backend for ZODB that stores pickles in a relational database."""
-
-VERSION = "1.5.1dev"
-
-# The choices for the Trove Development Status line:
-# Development Status :: 5 - Production/Stable
-# Development Status :: 4 - Beta
-# Development Status :: 3 - Alpha
-
-classifiers = """\
-Intended Audience :: Developers
-License :: OSI Approved :: Zope Public License
-Programming Language :: Python
-Topic :: Database
-Topic :: Software Development :: Libraries :: Python Modules
-Operating System :: Microsoft :: Windows
-Operating System :: Unix
-"""
-
-import os
-from setuptools import setup
-
-doclines = __doc__.split("\n")
-
-def read_file(*path):
- base_dir = os.path.dirname(__file__)
- file_path = (base_dir, ) + tuple(path)
- return file(os.path.join(*file_path)).read()
-
-setup(
- name="RelStorage",
- version=VERSION,
- author="Zope Foundation and Contributors",
- maintainer="Shane Hathaway",
- maintainer_email="shane at hathawaymix.org",
- url="http://pypi.python.org/pypi/RelStorage",
- packages=[
- 'relstorage',
- 'relstorage.adapters',
- 'relstorage.adapters.tests',
- 'relstorage.tests',
- 'relstorage.tests.blob',
- ],
- package_data={
- 'relstorage': ['component.xml'],
- },
- license="ZPL 2.1",
- platforms=["any"],
- description=doclines[0],
- classifiers=filter(None, classifiers.split("\n")),
- long_description = (
- read_file("README.txt") + "\n\n" +
- "Change History\n" +
- "==============\n\n" +
- read_file("CHANGES.txt")),
- zip_safe=False, # otherwise ZConfig can't see component.xml
- install_requires=[
- 'ZODB3>=3.7.0',
- 'zope.interface',
- 'zc.lockfile',
- ],
- extras_require={
- 'mysql': ['MySQL-python>=1.2.2'],
- 'postgresql': ['psycopg2>=2.0'],
- 'oracle': ['cx_Oracle>=4.3.1'],
- },
- entry_points = {'console_scripts': [
- 'zodbconvert = relstorage.zodbconvert:main',
- 'zodbpack = relstorage.zodbpack:main',
- ]},
- test_suite='relstorage.tests.alltests.make_suite',
-)
More information about the checkins
mailing list