[Zodb-checkins] SVN: ZODB/trunk/src/ Merge rev 29891 from the 3.4 branch.

Tim Peters tim.one at comcast.net
Thu Apr 7 00:48:06 EDT 2005


Log message for revision 29896:
  Merge rev 29891 from the 3.4 branch.
  
  Merge the ZODB part of Zope/branches/jim-fix-zclasses.
  
  Here are checkin msgs from the branch relating to ZODB code:
  
      r29872 | jim | 2005-04-04 07:04:39 -0400 (Mon, 04 Apr 2005) | 3 lines
      Changed paths:
         M /Zope/branches/jim-fix-zclasses/lib/python/ZODB/serialize.py
  
      For instances of persistent classes, save a class-module/clas-name
      tuple if the class has a non-empty module string.
  
      ------------------------------------------------------------------------
      r29871 | jim | 2005-04-04 07:04:33 -0400 (Mon, 04 Apr 2005) | 6 lines
      Changed paths:
         M /Zope/branches/jim-fix-zclasses/lib/python/ZODB/Connection.py
  
      Changed sub-transaction abort code to not invalidate created objects.
      There's really no point, because created objects will be unreachable
      after invalidating old objects.  Also, if a created object is
      non-ghostifiable, it will try to load it's state again, and then
      either it would fail or it would load non-committed state.
  
      ------------------------------------------------------------------------
      r29870 | jim | 2005-04-04 07:04:27 -0400 (Mon, 04 Apr 2005) | 3 lines
      Changed paths:
         A /Zope/branches/jim-fix-zclasses/lib/python/ZODB/persistentclass.py
         A /Zope/branches/jim-fix-zclasses/lib/python/ZODB/persistentclass.txt
         A /Zope/branches/jim-fix-zclasses/lib/python/ZODB/tests/testpersistentclass.py
  
      Added ZClass-independent test of (and possible base class for)
      persistent-class support machinery.
  
      ------------------------------------------------------------------------
      r29867 | jim | 2005-04-04 07:03:48 -0400 (Mon, 04 Apr 2005) | 2 lines
      Changed paths:
         M /Zope/branches/jim-fix-zclasses/lib/python/transaction/_manager.py
  
      Added missing arguments to commit and abort.
  
      ------------------------------------------------------------------------
      r29774 | jim | 2005-04-01 06:24:27 -0500 (Fri, 01 Apr 2005) | 3 lines
      Changed paths:
         M /Zope/branches/jim-fix-zclasses/lib/python/transaction/__init__.py
  
      Changed to use methods of a threaded manager directly, rather than
      through wrapper functions.
  
      ------------------------------------------------------------------------
      r29773 | jim | 2005-04-01 06:24:25 -0500 (Fri, 01 Apr 2005) | 3 lines
      Changed paths:
         M /Zope/branches/jim-fix-zclasses/lib/python/transaction/_manager.py
  
      Added commit and abort methods to transaction managers.
      This makes direcr use of managers simpler.
  
      ------------------------------------------------------------------------
      r29101 | jim | 2005-02-10 07:44:52 -0500 (Thu, 10 Feb 2005) | 2 lines
      Changed paths:
         M /Zope/branches/jim-fix-zclasses/lib/python/ZODB/Connection.py
  
      Removed an unused attribute.
  
      ------------------------------------------------------------------------
      r29100 | jim | 2005-02-10 07:42:35 -0500 (Thu, 10 Feb 2005) | 3 lines
      Changed paths:
         M /Zope/branches/jim-fix-zclasses/lib/python/transaction/_transaction.py
  
      Changed an XXX comment to an ordinary comment, explaining the relevent
      issue.
  
      ------------------------------------------------------------------------
      r29099 | jim | 2005-02-10 07:41:58 -0500 (Thu, 10 Feb 2005) | 3 lines
      Changed paths:
         M /Zope/branches/jim-fix-zclasses/lib/python/ZODB/Connection.py
  
      Changed some XXX comments to ordinary comments explaining the relevent
      issues.
  
      ------------------------------------------------------------------------
      r29071 | jim | 2005-02-07 07:36:05 -0500 (Mon, 07 Feb 2005) | 3 lines
      Changed paths:
         M /Zope/branches/jim-fix-zclasses/lib/python/transaction/_transaction.py
  
      Added a sanity check to avoid registration of objects without a
      manager. (Perhaps this should be an assert.)
  
      ------------------------------------------------------------------------
      r29067 | jim | 2005-02-07 07:35:56 -0500 (Mon, 07 Feb 2005) | 15 lines
      Changed paths:
         M /Zope/branches/jim-fix-zclasses/lib/python/ZODB/Connection.py
  
      Changed the strategy for handling invalidation of classes.
      No-longer use setklassstate.  Instead, just call _p_invalidate, as
      with any other object.  This changes didn't break any tests, so I
      assume that this was untested. :(
  
      Changed the strategy for invalidating objects.  Non-ghostifiable
      objects will load their state when they are invalidated.  We have to
      worry about other invalidations that come in while this is happening.
      See the comment in _flush_invalidations.
  
      We need to force all invalidations to take this into account, by going
      through _flush_invalidations.  I haven't done this yet, but I have
      left some XXX comments in places where it needs to be done to remind
      myself that this needs to be done.
  
      ------------------------------------------------------------------------
      r29066 | jim | 2005-02-07 07:35:54 -0500 (Mon, 07 Feb 2005) | 8 lines
      Changed paths:
         M /Zope/branches/jim-fix-zclasses/lib/python/persistent/cPickleCache.c
  
      Changed the strategy for handling invalidation of classes.
      No-longer use setklassstate.  Instead, just call _p_invalidate, as
      with any other object.  This changes didn't break any tests, so I
      assume that this was untested. :(
  
      Change invalidation to not swallow errors. (Swallowing errors here was a
      travesty!)
  

Changed:
  U   ZODB/trunk/src/Persistence/tests/test_ExtensionClass.py
  U   ZODB/trunk/src/Persistence/tests/test_mapping.py
  U   ZODB/trunk/src/ZODB/Connection.py
  A   ZODB/trunk/src/ZODB/persistentclass.py
  A   ZODB/trunk/src/ZODB/persistentclass.txt
  U   ZODB/trunk/src/ZODB/serialize.py
  U   ZODB/trunk/src/ZODB/tests/sampledm.py
  U   ZODB/trunk/src/ZODB/tests/testBroken.py
  U   ZODB/trunk/src/ZODB/tests/testConnection.py
  U   ZODB/trunk/src/ZODB/tests/testFileStorage.py
  U   ZODB/trunk/src/ZODB/tests/testSerialize.py
  U   ZODB/trunk/src/ZODB/tests/testSubTransaction.py
  U   ZODB/trunk/src/ZODB/tests/test_cache.py
  U   ZODB/trunk/src/ZODB/tests/test_datamanageradapter.py
  U   ZODB/trunk/src/ZODB/tests/testmvcc.py
  A   ZODB/trunk/src/ZODB/tests/testpersistentclass.py
  U   ZODB/trunk/src/persistent/cPickleCache.c
  U   ZODB/trunk/src/persistent/tests/test_PickleCache.py
  U   ZODB/trunk/src/persistent/tests/test_overriding_attrs.py
  U   ZODB/trunk/src/persistent/tests/test_persistent.py
  U   ZODB/trunk/src/persistent/tests/test_pickle.py
  U   ZODB/trunk/src/persistent/tests/test_wref.py
  U   ZODB/trunk/src/transaction/__init__.py
  U   ZODB/trunk/src/transaction/_manager.py
  U   ZODB/trunk/src/transaction/_transaction.py
  U   ZODB/trunk/src/transaction/tests/test_SampleDataManager.py
  U   ZODB/trunk/src/transaction/tests/test_register_compat.py
  U   ZODB/trunk/src/transaction/tests/test_transaction.py
  U   ZODB/trunk/src/transaction/tests/test_util.py

-=-
Modified: ZODB/trunk/src/Persistence/tests/test_ExtensionClass.py
===================================================================
--- ZODB/trunk/src/Persistence/tests/test_ExtensionClass.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/Persistence/tests/test_ExtensionClass.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -16,7 +16,7 @@
 $Id$
 """
 
-from doctest import DocTestSuite
+from zope.testing.doctest import DocTestSuite
 import pickle
 
 from Persistence import Persistent

Modified: ZODB/trunk/src/Persistence/tests/test_mapping.py
===================================================================
--- ZODB/trunk/src/Persistence/tests/test_mapping.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/Persistence/tests/test_mapping.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -15,7 +15,7 @@
 $Id$
 """
 import unittest
-from doctest import DocTestSuite
+from zope.testing.doctest import DocTestSuite
 from Persistence import PersistentMapping
 
 def test_basic_functionality():

Modified: ZODB/trunk/src/ZODB/Connection.py
===================================================================
--- ZODB/trunk/src/ZODB/Connection.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/ZODB/Connection.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -120,9 +120,9 @@
         # will execute atomically by virtue of the GIL.  But some storage
         # might generate oids where hash or compare invokes Python code.  In
         # that case, the GIL can't save us.
+
         self._inv_lock = threading.Lock()
         self._invalidated = d = {}
-        self._invalid = d.has_key
 
         # We intend to prevent committing a transaction in which
         # ReadConflictError occurs.  _conflicts is the set of oids that
@@ -221,6 +221,26 @@
                 del obj._p_jar
                 del obj._p_oid
             else:
+
+                # Note: If we invalidate a non-ghostifiable object
+                # (i.e. a persistent class), the object will
+                # immediately reread it's state.  That means that the
+                # following call could result in a call to
+                # self.setstate, which, of course, must suceed.
+                # In general, it would be better if the read could be
+                # delayed until the start of the next transaction.  If
+                # we read at the end of a transaction and if the
+                # object was invalidated during this transaction, then
+                # we'll read non-current data, which we'll discard
+                # later in transaction finalization.  Unfortnately, we
+                # can only delay the read if this abort corresponds to
+                # a top-level-transaction abort.  We can't tell if
+                # this is a top-level-transaction abort, so we have to
+                # go ahead and invalidate now. Fortunately, it's
+                # pretty unlikely that the object we are invalidating
+                # was invalidated by another thread, so the risk of a
+                # reread is pretty low.
+
                 self._cache.invalidate(oid)
 
         self._tpc_cleanup()
@@ -387,6 +407,22 @@
         self._storage = self._tmp
         self._tmp = None
 
+        # Note: If we invalidate a non-ghostifiable object (i.e. a
+        # persistent class), the object will immediately reread it's
+        # state.  That means that the following call could result in a
+        # call to self.setstate, which, of course, must succeed.  In
+        # general, it would be better if the read could be delayed
+        # until the start of the next transaction.  If we read at the
+        # end of a transaction and if the object was invalidated
+        # during this transaction, then we'll read non-current data,
+        # which we'll discard later in transaction finalization.  We
+        # could, theoretically queue this invalidation by calling
+        # self.invalidate.  Unfortunately, attempts to make that
+        # change resulted in mysterious test failures.  It's pretty
+        # unlikely that the object we are invalidating was invalidated
+        # by another thread, so the risk of a reread is pretty low.
+        # It's really not worth the effort to pursue this.
+
         self._cache.invalidate(src._index.keys())
         self._invalidate_creating(src._creating)
 
@@ -415,12 +451,44 @@
     def _flush_invalidations(self):
         self._inv_lock.acquire()
         try:
-            self._cache.invalidate(self._invalidated)
-            self._invalidated.clear()
+
+            # Non-ghostifiable objects may need to read when they are
+            # invalidated, so, we'll quickly just replace the
+            # invalidating dict with a new one.  We'll then process
+            # the invalidations after freeing the lock *and* after
+            # reseting the time.  This means that invalidations will
+            # happen after the start of the transactions.  They are
+            # subject to conflict errors and to reading old data,
+
+            # TODO: There is a potential problem lurking for persistent
+            # classes.  Suppose we have an invlidation of a persistent
+            # class and of an instance.  If the instance is
+            # invalidated first and if the invalidation logic uses
+            # data read from the class, then the invalidation could
+            # be performed with state data.  Or, suppose that there
+            # are instances of the class that are freed as a result of
+            # invalidating some object.  Perhaps code in their __del__
+            # uses class data.  Really, the only way to properly fix
+            # this is to, in fact, make classes ghostifiable.  Then
+            # we'd have to reimplement attribute lookup to check the
+            # class state and, if necessary, activate the class.  It's
+            # much worse than that though, because we'd also need to
+            # deal with slots.  When a class is ghostified, we'd need
+            # to replace all of the slot operations with versions that
+            # reloaded the object when caled. It's hard to say which
+            # is better for worse.  For now, it seems the risk of
+            # using a class while objects are being invalidated seems
+            # small enough t be acceptable.
+
+            invalidated = self._invalidated
+            self._invalidated = {}
             self._txn_time = None
         finally:
             self._inv_lock.release()
-        # Now is a good time to collect some garbage
+
+        self._cache.invalidate(invalidated)
+
+        # Now is a good time to collect some garbage.
         self._cache.incrgc()
 
     def root(self):
@@ -532,10 +600,26 @@
         self._tpc_cleanup()
 
     def tpc_abort(self, transaction):
-        """Abort a transaction."""
         if self._import:
             self._import = None
         self._storage.tpc_abort(transaction)
+
+        # Note: If we invalidate a non-justifiable object (i.e. a
+        # persistent class), the object will immediately reread it's
+        # state.  That means that the following call could result in a
+        # call to self.setstate, which, of course, must succeed.  In
+        # general, it would be better if the read could be delayed
+        # until the start of the next transaction.  If we read at the
+        # end of a transaction and if the object was invalidated
+        # during this transaction, then we'll read non-current data,
+        # which we'll discard later in transaction finalization.  We
+        # could, theoretically queue this invalidation by calling
+        # self.invalidate.  Unfortunately, attempts to make that
+        # change resulted in mysterious test failures.  It's pretty
+        # unlikely that the object we are invalidating was invalidated
+        # by another thread, so the risk of a reread is pretty low.
+        # It's really not worth the effort to pursue this.
+
         self._cache.invalidate(self._modified)
         self._invalidate_creating()
         while self._added:
@@ -630,7 +714,9 @@
         # because we have to check again after the load anyway.
 
         if (obj._p_oid in self._invalidated
-            and not myhasattr(obj, "_p_independent")):
+            and not myhasattr(obj, "_p_independent")
+            and not self._invalidated
+            ):
             # If the object has _p_independent(), we will handle it below.
             self._load_before_or_conflict(obj)
             return
@@ -913,4 +999,3 @@
         if dt is not DEPRECATED_ARGUMENT:
             deprecated36("cacheMinimize() dt= is ignored.")
         self._cache.minimize()
-

Copied: ZODB/trunk/src/ZODB/persistentclass.py (from rev 29891, ZODB/branches/3.4/src/ZODB/persistentclass.py)


Property changes on: ZODB/trunk/src/ZODB/persistentclass.py
___________________________________________________________________
Name: svn:keywords
   + Id
Name: svn:eol-style
   + native

Copied: ZODB/trunk/src/ZODB/persistentclass.txt (from rev 29891, ZODB/branches/3.4/src/ZODB/persistentclass.txt)


Property changes on: ZODB/trunk/src/ZODB/persistentclass.txt
___________________________________________________________________
Name: svn:eol-style
   + native

Modified: ZODB/trunk/src/ZODB/serialize.py
===================================================================
--- ZODB/trunk/src/ZODB/serialize.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/ZODB/serialize.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -32,14 +32,19 @@
 
 The class description can be in a variety of formats, in part to
 provide backwards compatibility with earlier versions of Zope.  The
-two current formats for class description are:
+four current formats for class description are:
 
     1. type(obj)
     2. type(obj), obj.__getnewargs__()
+    3. (module name, class name), None
+    7. (module name, class name), obj.__getnewargs__()
 
 The second of these options is used if the object has a __getnewargs__()
 method.  It is intended to support objects like persistent classes that have
-custom C layouts that are determined by arguments to __new__().
+custom C layouts that are determined by arguments to __new__().  The
+third and fourth (#3 & #7) apply to instances of a persistent class (which
+means the class itself is persistent, not that it's a subclass of
+Persistent).
 
 The type object is usually stored using the standard pickle mechanism, which
 involves the pickle GLOBAL opcode (giving the type's module and name as
@@ -59,17 +64,17 @@
 
 Earlier versions of Zope supported several other kinds of class
 descriptions.  The current serialization code reads these descriptions, but
-does not write them.  The four earlier formats are:
+does not write them.  The three earlier formats are:
 
-    3. (module name, class name), None
     4. (module name, class name), __getinitargs__()
     5. class, None
     6. class, __getinitargs__()
 
 Formats 4 and 6 are used only if the class defines a __getinitargs__()
-method.  Formats 5 and 6 are used if the class does not have a __module__
-attribute (I'm not sure when this applies, but I think it occurs for some
-but not all ZClasses).
+method, but we really can't tell them apart from formats 7 and 2
+(respectively).  Formats 5 and 6 are used if the class does not have a
+__module__ attribute (I'm not sure when this applies, but I think it occurs
+for some but not all ZClasses).
 
 
 Persistent references
@@ -100,6 +105,8 @@
 from ZODB.broken import Broken
 from ZODB.POSException import InvalidObjectReference
 
+_oidtypes = str, type(None)
+
 # Might to update or redo coptimizations to reflect weakrefs:
 # from ZODB.coptimizations import new_persistent_id
 
@@ -147,10 +154,10 @@
         ...     _p_jar = jar
         >>> writer = ObjectWriter(O)
 
-        Normally, object references include the oid and a cached
-        reference to the class.  Having the class available allows
-        fast creation of the ghost, avoiding requiring an additional
-        database lookup.
+        Normally, object references include the oid and a cached named
+        reference to the class.  Having the class information
+        available allows fast creation of the ghost, avoiding
+        requiring an additional database lookup.
 
         >>> bob = P('bob')
         >>> oid, cls = writer.persistent_id(bob)
@@ -282,8 +289,16 @@
             # It's possible that __getnewargs__ is degenerate and
             # returns (), but we don't want to have to deghostify
             # the object to find out.
+
+            # Note that this has the odd effect that, if the class has
+            # __getnewargs__ of its own, we'll lose the optimization
+            # of caching the class info.
+
             return oid
 
+        # Note that we never get here for persistent classes.
+        # We'll use driect refs for normal classes.
+
         return oid, klass
 
     def serialize(self, obj):
@@ -291,10 +306,25 @@
         # We don't want to be fooled by proxies.
         klass = type(obj)
 
+        # We want to serialize persistent classes by name if they have
+        # a non-None non-empty module so as not to have a direct
+        # ref. This is important when copying.  We probably want to
+        # revisit this in the future.
         newargs = getattr(obj, "__getnewargs__", None)
-        if newargs is None:
+        if (isinstance(getattr(klass, '_p_oid', 0), _oidtypes)
+              and klass.__module__):
+            # This is a persistent class with a non-empty module.  This
+            # uses pickle format #3 or #7.
+            klass = klass.__module__, klass.__name__
+            if newargs is None:
+                meta = klass, None
+            else:
+                meta = klass, newargs()
+        elif newargs is None:
+            # Pickle format #1.
             meta = klass
         else:
+            # Pickle format #2.
             meta = klass, newargs()
 
         return self._dump(meta, obj.__getstate__())

Modified: ZODB/trunk/src/ZODB/tests/sampledm.py
===================================================================
--- ZODB/trunk/src/ZODB/tests/sampledm.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/ZODB/tests/sampledm.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -405,7 +405,7 @@
 
 
 def test_suite():
-    from doctest import DocTestSuite
+    from zope.testing.doctest import DocTestSuite
     return DocTestSuite()
 
 if __name__ == '__main__':

Modified: ZODB/trunk/src/ZODB/tests/testBroken.py
===================================================================
--- ZODB/trunk/src/ZODB/tests/testBroken.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/ZODB/tests/testBroken.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -20,7 +20,7 @@
 import unittest
 import persistent
 import transaction
-from doctest import DocTestSuite
+from zope.testing.doctest import DocTestSuite
 from ZODB.tests.util import DB
 
 def test_integration():

Modified: ZODB/trunk/src/ZODB/tests/testConnection.py
===================================================================
--- ZODB/trunk/src/ZODB/tests/testConnection.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/ZODB/tests/testConnection.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -13,7 +13,7 @@
 ##############################################################################
 """Unit tests for the Connection class."""
 
-import doctest
+from zope.testing import doctest
 import unittest
 import warnings
 

Modified: ZODB/trunk/src/ZODB/tests/testFileStorage.py
===================================================================
--- ZODB/trunk/src/ZODB/tests/testFileStorage.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/ZODB/tests/testFileStorage.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -485,7 +485,7 @@
     """
 
 def test_suite():
-    import doctest
+    from zope.testing import doctest
 
     suite = unittest.TestSuite()
     for klass in [FileStorageTests, Corruption.FileStorageCorruptTests,

Modified: ZODB/trunk/src/ZODB/tests/testSerialize.py
===================================================================
--- ZODB/trunk/src/ZODB/tests/testSerialize.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/ZODB/tests/testSerialize.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -121,7 +121,7 @@
 
 
 def test_suite():
-    import doctest
+    from zope.testing import doctest
     suite = unittest.makeSuite(SerializerTestCase)
     suite.addTest(doctest.DocTestSuite("ZODB.serialize"))
     return suite

Modified: ZODB/trunk/src/ZODB/tests/testSubTransaction.py
===================================================================
--- ZODB/trunk/src/ZODB/tests/testSubTransaction.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/ZODB/tests/testSubTransaction.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -132,7 +132,7 @@
 
 """
 
-import doctest
+from zope.testing import doctest
 
 def test_suite():
     return doctest.DocTestSuite()

Modified: ZODB/trunk/src/ZODB/tests/test_cache.py
===================================================================
--- ZODB/trunk/src/ZODB/tests/test_cache.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/ZODB/tests/test_cache.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -13,7 +13,7 @@
 ##############################################################################
 """Test behavior of Connection plus cPickleCache."""
 
-import doctest
+from zope.testing import doctest
 
 from persistent import Persistent
 import transaction

Modified: ZODB/trunk/src/ZODB/tests/test_datamanageradapter.py
===================================================================
--- ZODB/trunk/src/ZODB/tests/test_datamanageradapter.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/ZODB/tests/test_datamanageradapter.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -15,7 +15,7 @@
 $Id$
 """
 import unittest
-from doctest import DocTestSuite
+from zope.testing.doctest import DocTestSuite
 from transaction._transaction import DataManagerAdapter
 from ZODB.tests.sampledm import DataManager
 

Modified: ZODB/trunk/src/ZODB/tests/testmvcc.py
===================================================================
--- ZODB/trunk/src/ZODB/tests/testmvcc.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/ZODB/tests/testmvcc.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -358,7 +358,7 @@
 
 """
 
-import doctest
+from zope.testing import doctest
 
 def test_suite():
     return doctest.DocTestSuite()

Copied: ZODB/trunk/src/ZODB/tests/testpersistentclass.py (from rev 29891, ZODB/branches/3.4/src/ZODB/tests/testpersistentclass.py)


Property changes on: ZODB/trunk/src/ZODB/tests/testpersistentclass.py
___________________________________________________________________
Name: svn:keywords
   + Id
Name: svn:eol-style
   + native

Modified: ZODB/trunk/src/persistent/cPickleCache.c
===================================================================
--- ZODB/trunk/src/persistent/cPickleCache.c	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/persistent/cPickleCache.c	2005-04-07 04:48:06 UTC (rev 29896)
@@ -115,7 +115,6 @@
     int klass_count;                     /* count of persistent classes */
     PyObject *data;                      /* oid -> object dict */
     PyObject *jar;                       /* Connection object */
-    PyObject *setklassstate;             /* ??? */
     int cache_size;                      /* target number of items in cache */
 
     /* Most of the time the ring contains only:
@@ -331,58 +330,48 @@
     return lockgc(self, 0);
 }
 
-static void
+static int
 _invalidate(ccobject *self, PyObject *key)
 {
-    static PyObject *_p_invalidate;
-    PyObject *v = PyDict_GetItem(self->data, key);
+    static PyObject *_p_invalidate = NULL;
+    PyObject *meth, *v;
 
-    if (!_p_invalidate) {
+    v = PyDict_GetItem(self->data, key);
+    if (v == NULL)
+	return 0;
+
+    if (_p_invalidate == NULL)
+      {
 	_p_invalidate = PyString_InternFromString("_p_invalidate");
-	if (!_p_invalidate) {
+	if (_p_invalidate == NULL)
+          {
 	    /* It doesn't make any sense to ignore this error, but
 	       the caller ignores all errors.
+
+               TODO: and why does it do that? This should be fixed
 	    */
-	    PyErr_Clear();
-	    return;
-	}
-    }
+	    return -1;
+          }
+      }
 
-    if (!v)
-	return;
-    if (PyType_Check(v)) {
-        /* This looks wrong, but it isn't. We use strong references to types
-           because they don't have the ring members.
+    if (v->ob_refcnt <= 1 && PyType_Check(v)) {
+      /* This looks wrong, but it isn't. We use strong references to types
+         because they don't have the ring members.
 
-           The result is that we *never* remove classes unless
-           they are modified.
+         The result is that we *never* remove classes unless
+         they are modified.  We can fix this by using wekrefs uniformly.
+      */
+      self->klass_count--;
+      return PyDict_DelItem(self->data, key);
+    }
 
-         */
-	if (v->ob_refcnt <= 1) {
-	    self->klass_count--;
-	    if (PyDict_DelItem(self->data, key) < 0)
-		PyErr_Clear();
-	}
-	else {
-	    v = PyObject_CallFunction(self->setklassstate, "O", v);
-	    if (v)
-		Py_DECREF(v);
-	    else
-		PyErr_Clear();
-	}
-    } else {
-	PyObject *meth, *err;
+    meth = PyObject_GetAttr(v, _p_invalidate);
+    if (meth == NULL)
+      return -1;
 
-	meth = PyObject_GetAttr(v, _p_invalidate);
-	if (!meth) {
-	    PyErr_Clear();
-	    return;
-	}
-	err = PyObject_CallObject(meth, NULL);
-	Py_DECREF(meth);
-	if (!err)
-	    PyErr_Clear();
-    }
+    v = PyObject_CallObject(meth, NULL);
+    Py_DECREF(meth);
+    return v == NULL ? -1 : 0;
 }
 
 static PyObject *
@@ -391,16 +380,23 @@
   PyObject *key, *v;
   int i = 0;
 
-  if (PyDict_Check(inv)) {
+  if (PyDict_Check(inv))
+    {
       while (PyDict_Next(inv, &i, &key, &v))
-	  _invalidate(self, key);
+        {
+	  if (_invalidate(self, key) < 0)
+            return NULL;
+        }
       PyDict_Clear(inv);
-  }
+    }
   else {
       if (PyString_Check(inv))
-	  _invalidate(self, inv);
+        {
+	  if (_invalidate(self, inv) < 0)
+            return NULL;
+        }
       else {
-	  int l;
+	  int l, r;
 
 	  l = PyObject_Length(inv);
 	  if (l < 0)
@@ -409,8 +405,10 @@
 	      key = PySequence_GetItem(inv, i);
 	      if (!key)
 		  return NULL;
-	      _invalidate(self, key);
+	      r = _invalidate(self, key);
 	      Py_DECREF(key);
+              if (r < 0)
+                return NULL;
 	  }
 	  /* Dubious:  modifying the input may be an unexpected side effect. */
 	  PySequence_DelSlice(inv, 0, l);
@@ -669,7 +667,7 @@
     if (!PyArg_ParseTuple(args, "O|i", &jar, &cache_size))
 	return -1;
 
-    self->setklassstate = self->jar = NULL;
+    self->jar = NULL;
     self->data = PyDict_New();
     if (self->data == NULL) {
 	Py_DECREF(self);
@@ -686,11 +684,6 @@
     non-ghost objects.
     */
     PyObject_GC_UnTrack((void *)self->data);
-    self->setklassstate = PyObject_GetAttrString(jar, "setklassstate");
-    if (self->setklassstate == NULL) {
-	Py_DECREF(self);
-	return -1;
-    }
     self->jar = jar;
     Py_INCREF(jar);
     self->cache_size = cache_size;
@@ -708,7 +701,6 @@
 {
     Py_XDECREF(self->data);
     Py_XDECREF(self->jar);
-    Py_XDECREF(self->setklassstate);
     PyObject_GC_Del(self);
 }
 
@@ -755,7 +747,6 @@
     }
 
     Py_XDECREF(self->jar);
-    Py_XDECREF(self->setklassstate);
 
     while (PyDict_Next(self->data, &pos, &k, &v)) {
 	Py_INCREF(v);
@@ -765,7 +756,6 @@
     Py_XDECREF(self->data);
     self->data = NULL;
     self->jar = NULL;
-    self->setklassstate = NULL;
     return 0;
 }
 
@@ -794,7 +784,6 @@
     }
 
     VISIT(self->jar);
-    VISIT(self->setklassstate);
 
     here = self->ring_home.r_next;
 

Modified: ZODB/trunk/src/persistent/tests/test_PickleCache.py
===================================================================
--- ZODB/trunk/src/persistent/tests/test_PickleCache.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/persistent/tests/test_PickleCache.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -40,7 +40,7 @@
 
     """
 
-from doctest import DocTestSuite
+from zope.testing.doctest import DocTestSuite
 import unittest
 
 def test_suite():

Modified: ZODB/trunk/src/persistent/tests/test_overriding_attrs.py
===================================================================
--- ZODB/trunk/src/persistent/tests/test_overriding_attrs.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/persistent/tests/test_overriding_attrs.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -398,5 +398,5 @@
 
 
 def test_suite():
-    from doctest import DocTestSuite
+    from zope.testing.doctest import DocTestSuite
     return DocTestSuite()

Modified: ZODB/trunk/src/persistent/tests/test_persistent.py
===================================================================
--- ZODB/trunk/src/persistent/tests/test_persistent.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/persistent/tests/test_persistent.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -11,11 +11,9 @@
 # FOR A PARTICULAR PURPOSE.
 #
 ##############################################################################
-
+from zope.testing import doctest
 from persistent import Persistent
 
-from zope.testing.doctestunit import DocFileSuite
-
 class P(Persistent):
     def __init__(self):
         self.x = 0
@@ -23,4 +21,4 @@
         self.x += 1
 
 def test_suite():
-    return DocFileSuite("persistent.txt", globs={"P": P})
+    return doctest.DocFileSuite("persistent.txt", globs={"P": P})

Modified: ZODB/trunk/src/persistent/tests/test_pickle.py
===================================================================
--- ZODB/trunk/src/persistent/tests/test_pickle.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/persistent/tests/test_pickle.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -269,7 +269,7 @@
 
     """
 
-from doctest import DocTestSuite
+from zope.testing.doctest import DocTestSuite
 import unittest
 
 def test_suite():

Modified: ZODB/trunk/src/persistent/tests/test_wref.py
===================================================================
--- ZODB/trunk/src/persistent/tests/test_wref.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/persistent/tests/test_wref.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -15,7 +15,7 @@
 $Id$
 """
 import unittest
-from doctest import DocTestSuite
+from zope.testing.doctest import DocTestSuite
 
 def test_suite():
     return DocTestSuite('persistent.wref')

Modified: ZODB/trunk/src/transaction/__init__.py
===================================================================
--- ZODB/trunk/src/transaction/__init__.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/transaction/__init__.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -20,19 +20,11 @@
 from transaction._manager import TransactionManager, ThreadTransactionManager
 
 manager = ThreadTransactionManager()
+get = manager.get
+begin = manager.begin
+commit = manager.commit
+abort = manager.abort
 
-def get():
-    return manager.get()
-
-def begin():
-    return manager.begin()
-
-def commit(sub=False):
-    manager.get().commit(sub)
-
-def abort(sub=False):
-    manager.get().abort(sub)
-
 def get_transaction():
     from ZODB.utils import deprecated36
     deprecated36("""   use transaction.get() instead of get_transaction().

Modified: ZODB/trunk/src/transaction/_manager.py
===================================================================
--- ZODB/trunk/src/transaction/_manager.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/transaction/_manager.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -61,7 +61,13 @@
     def unregisterSynch(self, synch):
         self._synchs.remove(synch)
 
-class ThreadTransactionManager(object):
+    def commit(self, sub=False):
+        self.get().commit(sub)
+
+    def abort(self, sub=False):
+        self.get().abort(sub)
+
+class ThreadTransactionManager(TransactionManager):
     """Thread-aware transaction manager.
 
     Each thread is associated with a unique transaction.

Modified: ZODB/trunk/src/transaction/_transaction.py
===================================================================
--- ZODB/trunk/src/transaction/_transaction.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/transaction/_transaction.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -54,8 +54,10 @@
 
 The second argument to tpc_begin() indicates that a subtransaction
 commit is beginning (if it is true).  In a subtransaction, there is no
-tpc_vote() call (I don't know why not).  The tpc_finish()
-or tpc_abort() call applies just to that subtransaction.
+tpc_vote() call, because sub-transactions don't need 2-phase commit.
+If a sub-transaction abort or commit fails, we can abort the outer
+transaction.  The tpc_finish() or tpc_abort() call applies just to
+that subtransaction.
 
 Once a resource manager is involved in a subtransaction, all
 subsequent transactions will be treated as subtransactions until
@@ -244,6 +246,8 @@
         # commit protocol.
 
         manager = getattr(obj, "_p_jar", obj)
+        if manager is None:
+            raise ValueError("Register with no manager")
         adapter = self._adapters.get(manager)
         if adapter is None:
             if myhasattr(manager, "commit_sub"):

Modified: ZODB/trunk/src/transaction/tests/test_SampleDataManager.py
===================================================================
--- ZODB/trunk/src/transaction/tests/test_SampleDataManager.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/transaction/tests/test_SampleDataManager.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -405,7 +405,7 @@
 
 
 def test_suite():
-    from doctest import DocTestSuite
+    from zope.testing.doctest import DocTestSuite
     return DocTestSuite()
 
 if __name__ == '__main__':

Modified: ZODB/trunk/src/transaction/tests/test_register_compat.py
===================================================================
--- ZODB/trunk/src/transaction/tests/test_register_compat.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/transaction/tests/test_register_compat.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -148,7 +148,7 @@
         obj.abort()
         self.aborted.append(obj)
 
-import doctest
+from zope.testing import doctest
 
 def test_suite():
     return doctest.DocTestSuite()

Modified: ZODB/trunk/src/transaction/tests/test_transaction.py
===================================================================
--- ZODB/trunk/src/transaction/tests/test_transaction.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/transaction/tests/test_transaction.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -46,16 +46,12 @@
 class TransactionTests(unittest.TestCase):
 
     def setUp(self):
-        self.orig_tm = transaction.manager
-        transaction.manager = transaction.TransactionManager()
-        self.sub1 = DataObject()
-        self.sub2 = DataObject()
-        self.sub3 = DataObject()
-        self.nosub1 = DataObject(nost=1)
+        self.txn_mgr = transaction.TransactionManager()
+        self.sub1 = DataObject(self.txn_mgr)
+        self.sub2 = DataObject(self.txn_mgr)
+        self.sub3 = DataObject(self.txn_mgr)
+        self.nosub1 = DataObject(self.txn_mgr, nost=1)
 
-    def tearDown(self):
-        transaction.manager = self.orig_tm
-
     # basic tests with two sub trans jars
     # really we only need one, so tests for
     # sub1 should identical to tests for sub2
@@ -64,7 +60,7 @@
         self.sub1.modify()
         self.sub2.modify()
 
-        transaction.commit()
+        self.txn_mgr.commit()
 
         assert self.sub1._p_jar.ccommit_sub == 0
         assert self.sub1._p_jar.ctpc_finish == 1
@@ -74,13 +70,13 @@
         self.sub1.modify()
         self.sub2.modify()
 
-        transaction.abort()
+        self.txn_mgr.abort()
 
         assert self.sub2._p_jar.cabort == 1
 
     def testTransactionNote(self):
 
-        t = transaction.get()
+        t = self.txn_mgr.get()
 
         t.note('This is a note.')
         self.assertEqual(t.description, 'This is a note.')
@@ -94,12 +90,12 @@
         self.sub1.modify()
         self.sub2.modify()
 
-        transaction.commit(1)
+        self.txn_mgr.commit(1)
 
         assert self.sub1._p_jar.ctpc_vote == 0
         assert self.sub1._p_jar.ctpc_finish == 1
 
-        transaction.commit()
+        self.txn_mgr.commit()
 
         assert self.sub1._p_jar.ccommit_sub == 1
         assert self.sub1._p_jar.ctpc_vote == 1
@@ -109,8 +105,8 @@
         self.sub1.modify()
         self.sub2.modify()
 
-        transaction.commit(1)
-        transaction.abort()
+        self.txn_mgr.commit(1)
+        self.txn_mgr.abort()
 
         assert self.sub1._p_jar.ctpc_vote == 0
         assert self.sub1._p_jar.cabort == 0
@@ -118,12 +114,12 @@
 
     def testMultipleSubTransactionCommitCommit(self):
         self.sub1.modify()
-        transaction.commit(1)
+        self.txn_mgr.commit(1)
 
         self.sub2.modify()
         # reset a flag on the original to test it again
         self.sub1.ctpc_finish = 0
-        transaction.commit(1)
+        self.txn_mgr.commit(1)
 
         # this is interesting.. we go through
         # every subtrans commit with all subtrans capable
@@ -135,7 +131,7 @@
         # add another before we do the entire txn commit
         self.sub3.modify()
 
-        transaction.commit()
+        self.txn_mgr.commit()
 
         # we did an implicit sub commit, is this impl artifact?
         assert self.sub3._p_jar.ccommit_sub == 1
@@ -161,12 +157,12 @@
         # add it
         self.sub1.modify()
 
-        transaction.commit(1)
+        self.txn_mgr.commit(1)
 
         # add another
         self.sub2.modify()
 
-        transaction.commit(1)
+        self.txn_mgr.commit(1)
 
         assert self.sub1._p_jar.ctpc_vote == 0
         assert self.sub1._p_jar.ctpc_finish > 0
@@ -175,10 +171,10 @@
         self.sub3.modify()
 
         # abort the sub transaction
-        transaction.abort(1)
+        self.txn_mgr.abort(1)
 
         # commit the container transaction
-        transaction.commit()
+        self.txn_mgr.commit()
 
         assert self.sub3._p_jar.cabort == 1
         assert self.sub1._p_jar.ccommit_sub == 1
@@ -190,7 +186,7 @@
 
         self.nosub1.modify()
 
-        transaction.commit()
+        self.txn_mgr.commit()
 
         assert self.nosub1._p_jar.ctpc_finish == 1
 
@@ -198,7 +194,7 @@
 
         self.nosub1.modify()
 
-        transaction.abort()
+        self.txn_mgr.abort()
 
         assert self.nosub1._p_jar.ctpc_finish == 0
         assert self.nosub1._p_jar.cabort == 1
@@ -221,7 +217,7 @@
         self.sub1.modify(tracing='sub')
         self.nosub1.modify(tracing='nosub')
 
-        transaction.commit(1)
+        self.txn_mgr.commit(1)
 
         assert self.sub1._p_jar.ctpc_finish == 1
 
@@ -229,7 +225,7 @@
         # in a subtrans
         assert self.nosub1._p_jar.ctpc_finish == 0
 
-        transaction.abort()
+        self.txn_mgr.abort()
 
         assert self.nosub1._p_jar.cabort == 1
         assert self.sub1._p_jar.cabort_sub == 1
@@ -239,11 +235,11 @@
         self.sub1.modify()
         self.nosub1.modify()
 
-        transaction.commit(1)
+        self.txn_mgr.commit(1)
 
         assert self.nosub1._p_jar.ctpc_vote == 0
 
-        transaction.commit()
+        self.txn_mgr.commit()
 
         #assert self.nosub1._p_jar.ccommit_sub == 0
         assert self.nosub1._p_jar.ctpc_vote == 1
@@ -276,12 +272,12 @@
         # add it
         self.sub1.modify()
 
-        transaction.commit(1)
+        self.txn_mgr.commit(1)
 
         # add another
         self.nosub1.modify()
 
-        transaction.commit(1)
+        self.txn_mgr.commit(1)
 
         assert self.sub1._p_jar.ctpc_vote == 0
         assert self.nosub1._p_jar.ctpc_vote == 0
@@ -291,7 +287,7 @@
         self.sub2.modify()
 
         # commit the container transaction
-        transaction.commit()
+        self.txn_mgr.commit()
 
         # we did an implicit sub commit
         assert self.sub2._p_jar.ccommit_sub == 1
@@ -316,7 +312,7 @@
         self.sub2.modify()
 
         try:
-            transaction.abort()
+            self.txn_mgr.abort()
         except TestTxnException: pass
 
         assert self.nosub1._p_jar.cabort == 1
@@ -330,7 +326,7 @@
         self.sub1.modify(nojar=1)
 
         try:
-            transaction.commit()
+            self.txn_mgr.commit()
         except TestTxnException: pass
 
         assert self.nosub1._p_jar.ctpc_finish == 0
@@ -345,7 +341,7 @@
         self.sub1.modify(nojar=1)
 
         try:
-            transaction.commit()
+            self.txn_mgr.commit()
         except TestTxnException: pass
 
         assert self.nosub1._p_jar.ctpc_finish == 0
@@ -371,7 +367,7 @@
         self.sub1.modify(nojar=1)
 
         try:
-            transaction.commit()
+            self.txn_mgr.commit()
         except TestTxnException: pass
 
         assert self.nosub1._p_jar.ctpc_abort == 1
@@ -385,7 +381,7 @@
         self.sub1.modify(nojar=1)
 
         try:
-            transaction.commit()
+            self.txn_mgr.commit()
         except TestTxnException:
             pass
 
@@ -402,19 +398,19 @@
         # they come out of the dictionary.
 
         self.sub1.modify()
-        transaction.commit(1)
+        self.txn_mgr.commit(1)
 
         self.nosub1.modify()
 
         self.sub2._p_jar = SubTransactionJar(errors='commit_sub')
         self.sub2.modify(nojar=1)
 
-        transaction.commit(1)
+        self.txn_mgr.commit(1)
 
         self.sub3.modify()
 
         try:
-            transaction.commit()
+            self.txn_mgr.commit()
         except TestTxnException:
             pass
 
@@ -441,17 +437,17 @@
 
         self.sub1._p_jar = SubTransactionJar(errors='commit_sub')
         self.sub1.modify(nojar=1)
-        transaction.commit(1)
+        self.txn_mgr.commit(1)
 
         self.nosub1.modify()
         self.sub2._p_jar = SubTransactionJar(errors='abort_sub')
         self.sub2.modify(nojar=1)
-        transaction.commit(1)
+        self.txn_mgr.commit(1)
 
         self.sub3.modify()
 
         try:
-            transaction.commit()
+            self.txn_mgr.commit()
         except TestTxnException, err:
             pass
         else:
@@ -500,7 +496,8 @@
 
 class DataObject:
 
-    def __init__(self, nost=0):
+    def __init__(self, txn_mgr, nost=0):
+        self.txn_mgr = txn_mgr
         self.nost = nost
         self._p_jar = None
 
@@ -510,7 +507,7 @@
                 self._p_jar = NoSubTransactionJar(tracing=tracing)
             else:
                 self._p_jar = SubTransactionJar(tracing=tracing)
-        transaction.get().register(self)
+        self.txn_mgr.get().register(self)
 
 class TestTxnException(Exception):
     pass
@@ -635,7 +632,7 @@
     """
 
 def test_suite():
-    from doctest import DocTestSuite
+    from zope.testing.doctest import DocTestSuite
     return unittest.TestSuite((
         DocTestSuite(),
         unittest.makeSuite(TransactionTests),

Modified: ZODB/trunk/src/transaction/tests/test_util.py
===================================================================
--- ZODB/trunk/src/transaction/tests/test_util.py	2005-04-07 03:49:30 UTC (rev 29895)
+++ ZODB/trunk/src/transaction/tests/test_util.py	2005-04-07 04:48:06 UTC (rev 29896)
@@ -16,7 +16,7 @@
 $Id$
 """
 import unittest
-from doctest import DocTestSuite
+from zope.testing.doctest import DocTestSuite
 
 def test_suite():
     return DocTestSuite('transaction.util')



More information about the Zodb-checkins mailing list