[Zodb-checkins] CVS: ZODB3 - NEWS.txt:1.33.6.1
Jeremy Hylton
jeremy at zope.com
Tue Sep 23 00:05:12 EDT 2003
Update of /cvs-repository/ZODB3
In directory cvs.zope.org:/tmp/cvs-serv9422
Modified Files:
Tag: Zope-2_7-branch
NEWS.txt
Log Message:
Update with news for the upcoming 3.2b3 release, and
backfill with old news from the 3.1 series.
=== ZODB3/NEWS.txt 1.33 => 1.33.6.1 ===
--- ZODB3/NEWS.txt:1.33 Mon Jun 16 17:43:55 2003
+++ ZODB3/NEWS.txt Tue Sep 23 00:05:07 2003
@@ -1,3 +1,107 @@
+What's new in ZODB3 3.2 beta 3
+==============================
+Release date: DD-MMM-2003
+
+Note: The changes listed for this release include changes also made in
+ZODB 3.1.x releases and ported to the 3.2 release.
+
+This version of ZODB 3.2 is not compatible with Python 2.1. Early
+versions were explicitly designed to be compatible with Zope 2.6.
+That plan has been dropped, because Zope 2.7 is already in beta
+release.
+
+Several of the classes in ZEO and ZODB now inherit from object, making
+them new-style classes. The primary motivation for the change was to
+make it easier to debug memory leaks. We don't expect any behavior to
+change as a result.
+
+A new feature to allow removal of connection pools for versions was
+ported from Zope 2.6. This feature is needed by Zope to avoid denial
+of service attacks that allow a client to create an arbitrary number
+of version pools.
+
+Fixed several critical ZEO bugs.
+
+- If a storage server fails or times out between the vote and the
+ finish, the ZEO cache could get populated with objects that didn't
+ make it to the storage server.
+
+- If a client loses its connection to the server near the end of a
+ transaction, it is now guaranteed to get a ClientDisconnected error
+ even if it reconnects before the transaction finishes. This is
+ necessary because the server will always abort the transaction.
+ In some cases, the client would never see an error for the aborted
+ transaction.
+
+- In tpc_finish(), reordered the calls so that the server's tpc_finish()
+ is called (and must succeed) before we update the ZEO client cache.
+
+- The storage name is now prepended to the sort key, to ensure a
+ unique global sort order if storages are named uniquely. This
+ can prevent deadlock in some unusual cases.
+
+Fixed several serious flaws in the implementation of the ZEO
+authentication protocol.
+
+- The smac layer would accept a message without a MAC even after the
+ session key was established.
+
+- The client never initialized its session key, so it never checked
+ incoming messages or created MACs for outgoing messags.
+
+- The smac layer used a single HMAC instance for sending and receiving
+ messages. This approach could only work if client and server were
+ guaranteed to process all messages in the same total order, which
+ could only happen in simple scenarios like unit tests.
+
+Fixed a bug in ExtensionClass when comparing ExtensionClass instances.
+The code could raise RuntimeWarning under Python 2.3, and produce
+incorrect results on 64-bit platforms.
+
+Fixed bug in BDBStorage that cause lead to DBRunRecoveryErrors when a
+transaction was aborted after performing operations like commit
+version or undo that create new references to existing pickles.
+
+Fixed a bug in Connection.py that caused it to fail with an
+AttributeError if close() was called after the database was closed.
+
+The test suite leaves fewer log files behind, although it still leaves
+a lot of junk. The test.py script puts each tests temp files in a
+separate directory, so it is easier to see which tests are causing
+problems. Unfortunately, it is still to tedious to figure out why the
+identified tests are leaving files behind.
+
+This release contains the latest and greatest version of the
+BDBStorage. This storage has still not seen testing in a production
+environment, but it represents the current best design and most recent
+code culled from various branches where development has occurred.
+
+The Tools directory contains a number of small improvements, a few new
+tools, and README.txt that catalogs the tools. Many of the tools are
+installed by setup.py; those scripts will now have a #! line set
+automatically on Unix.
+
+Fixed bugs in Tools/repozo.py, including a timing-dependent one that
+could cause the following invocation of repozo to do a full backup when
+an incremental backup would have sufficed.
+
+A pair of new scripts from Jim Fulton can be used to synthesize
+workloads and measure ZEO performance: see zodbload.py and
+zeoserverlog.py in the Tools directory. Note that these require
+Zope.
+
+Tools/checkbtrees.py was strengthened in two ways:
+
+- In addition to running the _check() method on each BTree B found,
+ BTrees.check.check(B) is also run. The check() function was written
+ after checkbtrees.py, and identifies kinds of damage B._check()
+ cannot find.
+
+- Cycles in the object graph no longer lead to unbounded output.
+ Note that preventing this requires remembering the oid of each
+ persistent object found, which increases the memory needed by the
+ script.
+
What's new in ZODB3 3.2 beta 2
==============================
Release date: 16-Jun-2003
@@ -307,6 +411,229 @@
The Sync extension was removed from ExtensionClass, because it was not
used by ZODB.
+
+What's new in ZODB3 3.1.4?
+==========================
+Release date: 11-Sep-2003
+
+A new feature to allow removal of connection pools for versions was
+ported from Zope 2.6. This feature is needed by Zope to avoid denial
+of service attacks that allow a client to create an arbitrary number
+of version pools.
+
+A pair of new scripts from Jim Fulton can be used to synthesize
+workloads and measure ZEO performance: see zodbload.py and
+zeoserverlog.py in the Tools directory. Note that these require
+Zope.
+
+Tools/checkbtrees.py was strengthened in two ways:
+
+- In addition to running the _check() method on each BTree B found,
+ BTrees.check.check(B) is also run. The check() function was written
+ after checkbtrees.py, and identifies kinds of damage B._check()
+ cannot find.
+
+- Cycles in the object graph no longer lead to unbounded output.
+ Note that preventing this requires remembering the oid of each
+ persistent object found, which increases the memory needed by the
+ script.
+
+What's new in ZODB3 3.1.3?
+==========================
+Release date: 18-Aug-2003
+
+Fixed several critical ZEO bugs.
+
+- If a storage server fails or times out between the vote and the
+ finish, the ZEO cache could get populated with objects that didn't
+ make it to the storage server.
+
+- If a client loses its connection to the server near the end of a
+ transaction, it is now guaranteed to get a ClientDisconnected error
+ even if it reconnects before the transaction finishes. This is
+ necessary because the server will always abort the transaction.
+ In some cases, the client would never see an error for the aborted
+ transaction.
+
+- In tpc_finish(), reordered the calls so that the server's tpc_finish()
+ is called (and must succeed) before we update the ZEO client cache.
+
+- The storage name is now prepended to the sort key, to ensure a
+ unique global sort order if storages are named uniquely. This
+ can prevent deadlock in some unusual cases.
+
+A variety of fixes and improvements to Berkeley storage (aka BDBStorage)
+were back-ported from ZODB 4. This release now contains the most
+current version of the Berkeley storage code. Many tests have been
+back-ported, but not all.
+
+Modified the Windows tests to wait longer at the end of ZEO tests for
+the server to shut down. Before Python 2.3, there is no waitpid() on
+Windows, and, thus, no way to know if the server has shut down. The
+change makes the Windows ZEO tests much less likely to fail or hang,
+at the cost of increasing the time needed to run the tests.
+
+Fixed a bug in ExtensionClass when comparing ExtensionClass instances.
+The code could raise RuntimeWarning under Python 2.3, and produce
+incorrect results on 64-bit platforms.
+
+Fixed bugs in Tools/repozo.py, including a timing-dependent one that
+could cause the following invocation of repozo to do a full backup when
+an incremental backup would have sufficed.
+
+Added Tools/README.txt that explains what each of the scripts in the
+Tools directory does.
+
+There were many small changes and improvements to the test suite.
+
+What's new in ZODB3 3.1.2 final?
+================================
+
+Fixed bug in FileStorage pack that caused it to fail if it encountered
+an old undo record (status "u").
+
+Fixed several bugs in FileStorage pack that could cause OverflowErrors
+for storages > 2 GB.
+
+Fixed memory leak in TimeStamp.laterThan() that only occurred when it
+had to create a new TimeStamp.
+
+Fixed two BTree bugs that were fixed on the head a while ago:
+
+ - bug in fsBTree that would cause byValue searches to end early.
+ (fsBTrees are never used this way, but it was still a bug.)
+
+ - bug that lead to segfault if BTree was mutated via deletion
+ while it was being iterated over.
+
+What's new in ZODB3 3.1.2 beta 2?
+=================================
+
+Fixed critical race conditions in ZEO's cache consistency code that
+could cause invalidations to be lost or stale data to be written to
+the cache. These bugs can lead to data loss or data corruption.
+These bugs are relatively unlikely to be provoked in sites with few
+conflicts, but the possibility of failure existed any time an object
+was loaded and stored concurrently.
+
+Fixed a bug in conflict resolution that failed to ghostify an object
+if it was involved in a conflict. (This code may be redundant, but it
+has been fixed regardless.)
+
+The ZEO server was fixed so that it does not perform any I/O until all
+of a transactions' invalidations are queued. If it performs I/O in the
+middle of sending invalidations, it would be possible to overlap a
+load from a client with the invalidation being sent to it.
+
+The ZEO cache now handles invalidations atomically. This is the same
+sort of bug that is described in the 3.1.2b1 section below, but it
+affects the ZEO cache.
+
+Fixed several serious bugs in fsrecover that caused it to fail
+catastrophically in certain cases because it thought it had found a
+checkpoint (status "c") record when it was in the middle of the file.
+
+What's new in ZODB3 3.1.2 beta 1?
+=================================
+
+ZODB
+----
+
+Invalidations are now processed atomically. Each transaction will see
+all the changes caused by an earlier transaction or none of them.
+Before this patch, it was possible for a transaction to see invalid
+data because it saw only a subset of the invalidations. This is the
+most likely cause of reported BTrees corruption, where keys were
+stored in the wrong bucket. When a BTree bucket splits, the bucket
+and the bucket's parent are both modified. If a transaction sees the
+invalidation for the bucket but not the parent, the BTree in memory
+will be internally inconsistent and keys can be put in the wrong
+bucket. The atomic invalidation fix prevents this problem.
+
+A number of minor reference count fixes in the object cache were
+fixed. That's the cPickleCache.c file.
+
+It was possible for a transaction that failed in tpc_finish() to lose
+the traceback that caused the failure. The transaction code was fixed
+to report the original error as well as any errors that occur while
+trying to recover from the original error.
+
+ZEO
+---
+
+A ZEO client will not read from its cache during cache verification.
+This fix was necessary to prevent the client from reading inconsistent
+data.
+
+The isReadOnly() method of a ZEO client was fixed to return the false
+when the client is connected to a read-only fallback server.
+
+The sync() method of ClientStorage and the pending() method of a zrpc
+connection now do both input and output.
+
+The short_repr() function used to generate log messages was fixed so
+that it does not blow up creating a repr of very long tuples.
+
+Storages
+--------
+
+FileStorage has a new pack() implementation that fixes several
+reported problems that could lead to data loss.
+
+Two small bugs were fixed in DemoStorage. undoLog() did not handle
+its arguments correctly and pack() could accidentally delete objects
+created in versions.
+
+Fixed trivial bug in fsrecover that prevented it from working at all.
+
+FileStorage will use fsync() on Windows starting with Python 2.2.3.
+
+FileStorage's commit version was fixed. It used to stop after the
+first object, leaving all the other objects in the version.
+
+BTrees
+------
+
+Trying to store an object of a non-integer type into an IIBTree
+or OIBTree could leave the bucket in a variety of insane states. For
+example, trying
+
+ b[obj] = "I'm a string, not an integer"
+
+where b is an OIBTree. This manifested as a refcount leak in the test
+suite, but could have been much worse (most likely in real life is that
+a seemingly arbitrary existing key would "go missing").
+
+When deleting the first child of a BTree node with more than one
+child, a reference to the second child leaked. This could cause
+the entire bucket chain to leak (not be collected as garbage
+despite not being referenced anymore).
+
+Other minor BTree leak scenarios were also fixed.
+
+Other
+-----
+
+Comparing a Missing.Value object to a C type that provide its own
+comparison operation could lead to a segfault when the Missing.Value
+was on the right-hand side of the comparison operator. The Missing
+class was fixed so that its coercion and comparison operations are
+safe.
+
+Tools
+-----
+
+Four tools are now installed by setup.py: fsdump.py, fstest.py,
+repozo.py, and zeopack.py.
+
+What's new in ZODB3 3.1.1 final?
+================================
+Release date: 11-Feb-2003
+
+Tools
+-----
+
+Updated repozo.py tool
What's new in ZODB3 3.1.1 beta 2?
=================================
More information about the Zodb-checkins
mailing list