[Checkins] SVN: mongopersist/trunk/ - Feature: A new ``IConflictHandler`` interface now controls all aspects of
Stephen Richter
cvs-admin at zope.org
Thu Mar 29 21:47:08 UTC 2012
Log message for revision 124797:
- Feature: A new ``IConflictHandler`` interface now controls all aspects of
conflict resolution. The following implementations are provided:
* ``NoCheckConflictHandler``: This handler does nothing and when used, the
system behaves as before when the ``detect_conflicts`` flag was set to
``False``.
* ``SimpleSerialConflictHandler``: This handler uses serial numbers on each
document to keep track of versions and then to detect conflicts. When a
conflict is detected, a ``ConflictError`` is raised. This handler is
identical to ``detect_conflicts`` being set to ``True``.
* ``ResolvingSerialConflictHandler``: Another serial handler, but it has the
ability to resolve a conflict. For this to happen, a persistent object
must implement ``_p_resolveConflict(orig_state, cur_state, new_state)``,
which returns the new, merged state.
As a result, the ``detect_conflicts`` flag of the data manager was removed
and replaced with the ``conflict_handler`` attribute. One can pass in the
``conflict_handler_factory`` to the data manager constructor. The factory
needs to expect on argument, the data manager.
Changed:
U mongopersist/trunk/CHANGES.txt
U mongopersist/trunk/src/mongopersist/README.txt
A mongopersist/trunk/src/mongopersist/conflict.py
U mongopersist/trunk/src/mongopersist/datamanager.py
U mongopersist/trunk/src/mongopersist/interfaces.py
U mongopersist/trunk/src/mongopersist/serialize.py
A mongopersist/trunk/src/mongopersist/tests/test_conflict.py
U mongopersist/trunk/src/mongopersist/tests/test_datamanager.py
U mongopersist/trunk/src/mongopersist/tests/test_serialize.py
-=-
Modified: mongopersist/trunk/CHANGES.txt
===================================================================
--- mongopersist/trunk/CHANGES.txt 2012-03-29 17:25:32 UTC (rev 124796)
+++ mongopersist/trunk/CHANGES.txt 2012-03-29 21:47:03 UTC (rev 124797)
@@ -5,6 +5,28 @@
0.7.0 (2012-03-??)
------------------
+- Feature: A new ``IConflictHandler`` interface now controls all aspects of
+ conflict resolution. The following implementations are provided:
+
+ * ``NoCheckConflictHandler``: This handler does nothing and when used, the
+ system behaves as before when the ``detect_conflicts`` flag was set to
+ ``False``.
+
+ * ``SimpleSerialConflictHandler``: This handler uses serial numbers on each
+ document to keep track of versions and then to detect conflicts. When a
+ conflict is detected, a ``ConflictError`` is raised. This handler is
+ identical to ``detect_conflicts`` being set to ``True``.
+
+ * ``ResolvingSerialConflictHandler``: Another serial handler, but it has the
+ ability to resolve a conflict. For this to happen, a persistent object
+ must implement ``_p_resolveConflict(orig_state, cur_state, new_state)``,
+ which returns the new, merged state.
+
+ As a result, the ``detect_conflicts`` flag of the data manager was removed
+ and replaced with the ``conflict_handler`` attribute. One can pass in the
+ ``conflict_handler_factory`` to the data manager constructor. The factory
+ needs to expect on argument, the data manager.
+
- Feature: ``ConflictError`` has now a much more meaningful API. Instead of
just referencing the object and different serials, it now actual has the
original, current and new state documents.
Modified: mongopersist/trunk/src/mongopersist/README.txt
===================================================================
--- mongopersist/trunk/src/mongopersist/README.txt 2012-03-29 17:25:32 UTC (rev 124796)
+++ mongopersist/trunk/src/mongopersist/README.txt 2012-03-29 21:47:03 UTC (rev 124797)
@@ -526,13 +526,13 @@
Let's reset the database and create a data manager with enabled conflict
detection:
- >>> from mongopersist import datamanager
+ >>> from mongopersist import conflict, datamanager
>>> conn.drop_database(DBNAME)
>>> dm2 = datamanager.MongoDataManager(
... conn,
... default_database=DBNAME,
... root_database=DBNAME,
- ... detect_conflicts=True)
+ ... conflict_handler_factory=conflict.SimpleSerialConflictHandler)
Now we add a person and see that the serial got stored.
Added: mongopersist/trunk/src/mongopersist/conflict.py
===================================================================
--- mongopersist/trunk/src/mongopersist/conflict.py (rev 0)
+++ mongopersist/trunk/src/mongopersist/conflict.py 2012-03-29 21:47:03 UTC (rev 124797)
@@ -0,0 +1,127 @@
+##############################################################################
+#
+# Copyright (c) 2012 Zope Foundation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Mongo Persistence Conflict Handler Implementations"""
+from __future__ import absolute_import
+import struct
+import zope.interface
+from mongopersist import interfaces, serialize
+
+def p64(v):
+ """Pack an integer or long into a 8-byte string"""
+ return struct.pack(">Q", v)
+
+def u64(v):
+ """Unpack an 8-byte string into a 64-bit long integer."""
+ return struct.unpack(">Q", v)[0]
+
+def create_conflict_error(obj, orig_doc, cur_doc, new_doc):
+ return interfaces.ConflictError(None, obj, orig_doc, cur_doc, new_doc)
+
+class NoCheckConflictHandler(object):
+ zope.interface.implements(interfaces.IConflictHandler)
+
+ def __init__(self, datamanager):
+ self.datamanager = datamanager
+
+ def on_before_set_state(self, obj, state):
+ pass
+
+ def on_before_store(self, obj, state):
+ pass
+
+ def on_after_store(self, obj, state):
+ pass
+
+ def on_modified(self, obj):
+ pass
+
+ def has_conflicts(self, objs):
+ return False
+
+ def check_conflicts(self, objs):
+ pass
+
+
+class SerialConflictHandler(object):
+ zope.interface.implements(interfaces.IResolvingConflictHandler)
+
+ field_name = '_py_serial'
+ conflict_error_factory = staticmethod(create_conflict_error)
+
+ def __init__(self, datamanager):
+ self.datamanager = datamanager
+
+ def on_before_set_state(self, obj, state):
+ obj._p_serial = p64(state.pop(self.field_name, 0))
+
+ def on_before_store(self, obj, state):
+ state[self.field_name] = u64(getattr(obj, '_p_serial', 0)) + 1
+ obj._p_serial = p64(state[self.field_name])
+
+ def on_after_store(self, obj, state):
+ pass
+
+ def on_modified(self, obj):
+ pass
+
+ def resolve(self, obj, orig_doc, cur_doc, new_doc):
+ raise NotImplementedError
+
+ def check_conflict(self, obj):
+ # This object is not even added to the database yet, so there
+ # cannot be a conflict.
+ if obj._p_oid is None:
+ return
+ coll = self.datamanager._get_collection_from_object(obj)
+ cur_doc = coll.find_one(obj._p_oid.id, fields=(self.field_name,))
+ if cur_doc is None:
+ return
+ if cur_doc.get(self.field_name, 0) != u64(obj._p_serial):
+ orig_doc = self.datamanager._original_states.get(obj._p_oid)
+ cur_doc = coll.find_one(obj._p_oid.id)
+ new_doc = self.datamanager._writer.get_full_state(obj)
+ resolved = self.resolve(obj, orig_doc, cur_doc, new_doc)
+ if not resolved:
+ return self.conflict_error_factory(
+ obj, orig_doc, cur_doc, new_doc)
+
+ def has_conflicts(self, objs):
+ for obj in objs:
+ if self.check_conflict(obj) is not None:
+ return True
+ return False
+
+ def check_conflicts(self, objs):
+ for obj in objs:
+ err = self.check_conflict(obj)
+ if err is not None:
+ raise err
+
+
+class SimpleSerialConflictHandler(SerialConflictHandler):
+
+ def resolve(self, obj, orig_doc, cur_doc, new_doc):
+ return False
+
+
+class ResolvingSerialConflictHandler(SerialConflictHandler):
+
+ def resolve(self, obj, orig_doc, cur_doc, new_doc):
+ if hasattr(obj, '_p_resolveConflict'):
+ doc = obj._p_resolveConflict(orig_doc, cur_doc, new_doc)
+ if doc is not None:
+ doc[self.field_name] = cur_doc[self.field_name]
+ self.datamanager._reader.set_ghost_state(obj, doc)
+ return True
+ return False
Property changes on: mongopersist/trunk/src/mongopersist/conflict.py
___________________________________________________________________
Added: svn:keywords
+ Id
Modified: mongopersist/trunk/src/mongopersist/datamanager.py
===================================================================
--- mongopersist/trunk/src/mongopersist/datamanager.py 2012-03-29 17:25:32 UTC (rev 124796)
+++ mongopersist/trunk/src/mongopersist/datamanager.py 2012-03-29 21:47:03 UTC (rev 124797)
@@ -24,16 +24,13 @@
import zope.interface
from zope.exceptions import exceptionformatter
-from mongopersist import interfaces, serialize
+from mongopersist import conflict, interfaces, serialize
MONGO_ACCESS_LOGGING = False
COLLECTION_LOG = logging.getLogger('mongopersist.collection')
LOG = logging.getLogger(__name__)
-def create_conflict_error(obj, orig_doc, cur_doc, new_doc):
- return interfaces.ConflictError(None, obj, orig_doc, cur_doc, new_doc)
-
def process_spec(collection, spec):
try:
adapter = interfaces.IMongoSpecProcessor(None)
@@ -173,14 +170,14 @@
class MongoDataManager(object):
zope.interface.implements(interfaces.IMongoDataManager)
- detect_conflicts = False
default_database = 'mongopersist'
name_map_collection = 'persistence_name_map'
- conflict_error_factory = staticmethod(create_conflict_error)
+ conflict_handler = None
- def __init__(self, conn, detect_conflicts=None, default_database=None,
+ def __init__(self, conn, default_database=None,
root_database=None, root_collection=None,
- name_map_collection=None, conflict_error_factory=None):
+ name_map_collection=None,
+ conflict_handler_factory=conflict.NoCheckConflictHandler):
self._conn = conn
self._reader = serialize.ObjectReader(self)
self._writer = serialize.ObjectWriter(self)
@@ -193,14 +190,12 @@
self._needs_to_join = True
self._object_cache = {}
self.annotations = {}
- if detect_conflicts is not None:
- self.detect_conflicts = detect_conflicts
+ if self.conflict_handler is None:
+ self.conflict_handler = conflict_handler_factory(self)
if default_database is not None:
self.default_database = default_database
if name_map_collection is not None:
self.name_map_collection = name_map_collection
- if conflict_error_factory is not None:
- self.conflict_error_factory = conflict_error_factory
self.transaction_manager = transaction.manager
self.root = Root(self, root_database, root_collection)
@@ -211,33 +206,6 @@
db_name, coll_name = self._writer.get_collection_name(obj)
return self._get_collection(db_name, coll_name)
- def _check_conflict(self, obj, can_raise=True):
- # This object is not even added to the database yet, so there
- # cannot be a conflict.
- if obj._p_oid is None:
- return None if can_raise else False
- coll = self._get_collection_from_object(obj)
- new_doc = coll.find_one(obj._p_oid.id, fields=('_py_serial',))
- if new_doc is None:
- return None if can_raise else False
- if new_doc.get('_py_serial', 0) != serialize.u64(obj._p_serial):
- if can_raise:
- orig_doc = self._original_states.get(obj._p_oid)
- cur_doc = coll.find_one(obj._p_oid.id)
- raise self.conflict_error_factory(
- obj, orig_doc, cur_doc, new_doc)
- else:
- return True
- return None if can_raise else False
-
- def _check_conflicts(self):
- if not self.detect_conflicts:
- return
- # Check each modified object to see whether Mongo has a new version of
- # the object.
- for obj in self._registered_objects:
- self._check_conflict(obj)
-
def _flush_objects(self):
# Now write every registered object, but make sure we write each
# object just once.
@@ -275,7 +243,7 @@
def flush(self):
# Check for conflicts.
- self._check_conflicts()
+ self.conflict_handler.check_conflicts(self._registered_objects)
# Now write every registered object, but make sure we write each
# object just once.
self._flush_objects()
@@ -335,10 +303,12 @@
self.transaction_manager.get().join(self)
self._needs_to_join = False
- if obj is not None and obj not in self._registered_objects:
- self._registered_objects.append(obj)
- if obj is not None and obj not in self._modified_objects:
- self._modified_objects.append(obj)
+ if obj is not None:
+ if obj not in self._registered_objects:
+ self._registered_objects.append(obj)
+ if obj not in self._modified_objects:
+ self._modified_objects.append(obj)
+ self.conflict_handler.on_modified(obj)
def abort(self, transaction):
# Aborting the transaction requires three steps:
@@ -364,8 +334,7 @@
'Original state not found while aborting: %r (%s)',
obj, db_ref.id if db_ref else '')
continue
- if (self.detect_conflicts and
- self._check_conflict(obj, can_raise=False)):
+ if self.conflict_handler.has_conflicts([obj]):
# If we have a conflict, we are not going to reset to the
# original state. (This is a policy that should be made
# pluggable.)
@@ -378,7 +347,7 @@
self.reset()
def commit(self, transaction):
- self._check_conflicts()
+ self.conflict_handler.check_conflicts(self._registered_objects)
def tpc_begin(self, transaction):
pass
Modified: mongopersist/trunk/src/mongopersist/interfaces.py
===================================================================
--- mongopersist/trunk/src/mongopersist/interfaces.py 2012-03-29 17:25:32 UTC (rev 124796)
+++ mongopersist/trunk/src/mongopersist/interfaces.py 2012-03-29 21:47:03 UTC (rev 124797)
@@ -63,6 +63,55 @@
class CircularReferenceError(Exception):
pass
+class IConflictHandler(zope.interface.Interface):
+
+ datamanager = zope.interface.Attribute(
+ """The datamanager for which to conduct the conflict resolution.""")
+
+ def on_before_set_state(obj, state):
+ """Method called just before the object's state is set."""
+
+ def on_before_store(obj, state):
+ """Method called just before the object state is written to MongoDB."""
+
+ def on_after_store(obj, state):
+ """Method called right after the object state was written to MongoDB."""
+
+ def on_modified(obj):
+ """Method called when an object is registered as modified."""
+
+ def has_conflicts(objs):
+ """Checks whether any of the passed in objects have conflicts.
+
+ Returns False if conflicts were found, otherwise True is returned.
+
+ While calling this method, the conflict handler may try to resolve
+ conflicts.
+ """
+
+ def check_conflicts(self, objs):
+ """Checks whether any of the passed in objects have conflicts.
+
+ Raises a ``ConflictError`` for the first object with a conflict.
+
+ While calling this method, the conflict handler may try to resolve
+ conflicts.
+ """
+
+class IResolvingConflictHandler(IConflictHandler):
+ """A conflict handler that is able to resolve conflicts."""
+
+ def resolve(obj, orig_doc, cur_doc, new_doc):
+ """Tries to resolve a conflict.
+
+ This is usually done through some comparison of the states. The method
+ returns ``True`` if the conflict was resolved and ``False`` otherwise.
+
+ It is the responsibility of this method to modify the object and data
+ manager models, so that the resolution is valid in the next step.
+ """
+
+
class IObjectSerializer(zope.interface.Interface):
"""An object serializer allows for custom serialization output for
objects."""
@@ -134,8 +183,8 @@
root = zope.interface.Attribute(
"""Get the root object, which is a mapping.""")
- detect_conflicts = zope.interface.Attribute(
- """A flag, when set it enables write conflict detection.""")
+ conflict_handler = zope.interface.Attribute(
+ """An ``IConflictHandler`` instance that handles all conflicts.""")
def get_collection(db_name, coll_name):
"""Return the collection for the given DB and collection names."""
Modified: mongopersist/trunk/src/mongopersist/serialize.py
===================================================================
--- mongopersist/trunk/src/mongopersist/serialize.py 2012-03-29 17:25:32 UTC (rev 124796)
+++ mongopersist/trunk/src/mongopersist/serialize.py 2012-03-29 21:47:03 UTC (rev 124797)
@@ -14,7 +14,6 @@
"""Object Serialization for Mongo/BSON"""
from __future__ import absolute_import
import copy_reg
-import struct
import lru
import persistent.interfaces
@@ -32,14 +31,6 @@
SERIALIZERS = []
OID_CLASS_LRU = lru.LRUCache(20000)
-def p64(v):
- """Pack an integer or long into a 8-byte string"""
- return struct.pack(">Q", v)
-
-def u64(v):
- """Unpack an 8-byte string into a 64-bit long integer."""
- return struct.unpack(">Q", v)[0]
-
def get_dotted_name(obj):
return obj.__module__+'.'+obj.__name__
@@ -211,6 +202,20 @@
return self.get_non_persistent_state(obj, seen)
+ def get_full_state(self, obj):
+ doc = self.get_state(obj.__getstate__())
+ # Add a persistent type info, if necessary.
+ if getattr(obj, '_p_mongo_store_type', False):
+ doc['_py_persistent_type'] = get_dotted_name(obj.__class__)
+ # A hook, so that the conflict handler can modify the state document
+ # if needed.
+ self._jar.conflict_handler.on_before_store(obj, doc)
+ # Add the object id.
+ if obj._p_oid is not None:
+ doc['_id'] = obj._p_oid.id
+ # Return the full state document
+ return doc
+
def store(self, obj, ref_only=False):
__traceback_info__ = (obj, ref_only)
@@ -229,12 +234,11 @@
doc = self.get_state(obj.__getstate__())
if getattr(obj, '_p_mongo_store_type', False):
doc['_py_persistent_type'] = get_dotted_name(obj.__class__)
- # If conflict detection is turned on, store a serial number for the
- # document.
- if self._jar.detect_conflicts:
- doc['_py_serial'] = u64(getattr(obj, '_p_serial', 0)) + 1
- obj._p_serial = p64(doc['_py_serial'])
+ # A hook, so that the conflict handler can modify the state document
+ # if needed.
+ self._jar.conflict_handler.on_before_store(obj, doc)
+
if obj._p_oid is None:
doc_id = coll.insert(doc)
obj._p_jar = self._jar
@@ -245,6 +249,11 @@
else:
doc['_id'] = obj._p_oid.id
coll.save(doc)
+
+ # A hook, so that the conflict handler can modify the object or state
+ # document after an object was stored.
+ self._jar.conflict_handler.on_after_store(obj, doc)
+
return obj._p_oid
@@ -389,14 +398,17 @@
# Remove unwanted attributes.
state_doc.pop('_id')
state_doc.pop('_py_persistent_type', None)
- # Store the serial, if conflict detection is enabled.
- if self._jar.detect_conflicts:
- obj._p_serial = p64(state_doc.pop('_py_serial', 0))
+ # Allow the conflict handler to modify the object or state document
+ # before it is set on the object.
+ self._jar.conflict_handler.on_before_set_state(obj, state_doc)
# Now convert the document to a proper Python state dict.
state = dict(self.get_object(state_doc, obj))
# Now store the original state. It is assumed that the state dict is
# not modified later.
- self._jar._original_states[obj._p_oid] = doc
+ # Make sure that we never set the original state multiple times, even
+ # if reassigning the state within the same transaction.
+ if obj._p_oid not in self._jar._original_states:
+ self._jar._original_states[obj._p_oid] = doc
# Set the state.
obj.__setstate__(state)
Added: mongopersist/trunk/src/mongopersist/tests/test_conflict.py
===================================================================
--- mongopersist/trunk/src/mongopersist/tests/test_conflict.py (rev 0)
+++ mongopersist/trunk/src/mongopersist/tests/test_conflict.py 2012-03-29 21:47:03 UTC (rev 124797)
@@ -0,0 +1,332 @@
+##############################################################################
+#
+# Copyright (c) 2011 Zope Foundation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Mongo Tests"""
+import doctest
+import persistent
+import transaction
+
+from mongopersist import conflict, datamanager, interfaces, testing
+
+class Foo(persistent.Persistent):
+ def __init__(self, name=None):
+ self.name = name
+ def __repr__(self):
+ return '<%s %r>' %(self.__class__.__name__, self.name)
+
+class MergerList(persistent.Persistent):
+ def __init__(self, list=None):
+ self.list = list
+ def __repr__(self):
+ return '<%s %r>' %(self.__class__.__name__, self.list)
+ def _p_resolveConflict(self, orig, cur, new):
+ merged = orig.copy()
+ merged['list'] = sorted(list(set(cur['list']).union(set(new['list']))))
+ return merged
+
+def doctest_create_conflict_error():
+ r"""create_conflict_error(): General Test
+
+ Simple helper function to create a conflict error.
+
+ >>> foo = Foo()
+
+ >>> conflict.create_conflict_error(
+ ... foo, {'_py_serial': 1}, {'_py_serial': 2}, {'_py_serial': 3})
+ ConflictError: database conflict error
+ (oid None, class Foo, orig serial 1, cur serial 2, new serial 3)
+ """
+
+def doctest_NoCheckConflictHandler_basic():
+ r"""class NoCheckConflictHandler: basic
+
+ This conflict handler does absolutely nothing to resolve conflicts. It is
+ the default conflict handler of the library.
+
+ >>> handler = conflict.NoCheckConflictHandler(dm)
+
+ Let's check the event methods:
+
+ >>> obj = Foo('one')
+ >>> state = {'name': 'one'}
+
+ >>> handler.on_before_set_state(obj, state)
+ >>> obj, state
+ (<Foo 'one'>, {'name': 'one'})
+
+ >>> handler.on_before_store(obj, state)
+ >>> obj, state
+ (<Foo 'one'>, {'name': 'one'})
+
+ >>> handler.on_after_store(obj, state)
+ >>> obj, state
+ (<Foo 'one'>, {'name': 'one'})
+
+ >>> handler.on_modified(obj)
+ >>> obj, state
+ (<Foo 'one'>, {'name': 'one'})
+
+ Let's check the conflict checking methods:
+
+ >>> handler.has_conflicts([obj])
+ False
+ >>> handler.check_conflicts([obj])
+ """
+
+def doctest_NoCheckConflictHandler_full():
+ r"""class NoCheckConflictHandler: Full conflict test.
+
+ This test demonstrates the conflict resolution behavior of the
+ ``NoCheckConflictHandler`` conflict handler during a real session.
+
+ First let's create an initial state:
+
+ >>> dm.reset()
+ >>> foo_ref = dm.insert(Foo('one'))
+ >>> dm.reset()
+
+ >>> coll = dm._get_collection_from_object(Foo())
+ >>> coll.find_one({})
+ {u'_id': ObjectId('4f5c114f37a08e2cac000000'), u'name': u'one'}
+
+ 1. Transaction A loads the object:
+
+ >>> foo_A = dm.load(foo_ref)
+ >>> foo_A.name
+ u'one'
+
+ 2. Transaction B comes along and modifies Foos data and commits:
+
+ >>> dm_B = datamanager.MongoDataManager(
+ ... conn, default_database=DBNAME, root_database=DBNAME,
+ ... conflict_handler_factory=conflict.NoCheckConflictHandler)
+
+ >>> foo_B = dm_B.load(foo_ref)
+ >>> foo_B.name = 'eins'
+ >>> dm_B.tpc_finish(None)
+
+ >>> coll.find_one({})
+ {u'_id': ObjectId('4f5c114f37a08e2cac000000'), u'name': u'eins'}
+
+ 3. Transaction A modifies Foo and the data is flushed:
+
+ >>> foo_A.name = '1'
+ >>> dm.flush()
+
+ >>> coll.find_one({})
+ {u'_id': ObjectId('4f5c114f37a08e2cac000000'), u'name': u'1'}
+ """
+
+def doctest_SimpleSerialConflictHandler_basic():
+ r"""class SimpleSerialConflictHandler: basic
+
+ This conflict handler detects conflicts by comparing serial numbers and
+ always raises a ``ConflictError`` error.
+
+ >>> handler = conflict.SimpleSerialConflictHandler(dm)
+ >>> obj = Foo('one')
+
+ Before the object state is set, the serial is extracted from the state and
+ set on the object:
+
+ >>> state = {'name': 'one', '_py_serial': 5}
+ >>> handler.on_before_set_state(obj, state)
+ >>> obj._p_serial
+ '\x00\x00\x00\x00\x00\x00\x00\x05'
+ >>> state
+ {'name': 'one'}
+
+ Before the object state is stored in Mongo, we add the serial by taking
+ the current one and add 1 to it:
+
+ >>> state = {'name': 'one'}
+ >>> handler.on_before_store(obj, state)
+ >>> state
+ {'_py_serial': 6, 'name': 'one'}
+
+ The event handlers after store and on modification do not need to do
+ anything:
+
+ >>> state = {'name': 'one'}
+ >>> handler.on_after_store(obj, state)
+ >>> obj, state
+ (<Foo 'one'>, {'name': 'one'})
+
+ >>> handler.on_modified(obj)
+ >>> obj
+ <Foo 'one'>
+
+ Let's check the conflict checking methods now. Initially, there are no
+ conflicts:
+
+ >>> handler.has_conflicts([obj])
+ False
+ >>> handler.check_conflicts([obj])
+
+ We can force a conflict by setting back the serial on the object:
+
+ >>> db_ref = dm.insert(obj)
+ >>> dm.reset()
+ >>> obj._p_serial = conflict.p64(3)
+
+ >>> handler.has_conflicts([obj])
+ True
+ >>> handler.check_conflicts([obj])
+ Traceback (most recent call last):
+ ...
+ ConflictError: database conflict error ...
+ """
+
+def doctest_SimpleSerialConflictHandler_full():
+ r"""class SimpleSerialConflictHandler: Full conflict test.
+
+ This test demonstrates the conflict resolution behavior of the
+ ``SimpleSerialConflictHandler`` conflict handler during a real session.
+
+ First let's create an initial state:
+
+ >>> dm.conflict_handler = conflict.SimpleSerialConflictHandler(dm)
+ >>> dm.reset()
+ >>> foo_ref = dm.insert(Foo('one'))
+ >>> dm.reset()
+
+ >>> coll = dm._get_collection_from_object(Foo())
+ >>> coll.find_one({})
+ {u'_id': ObjectId('...'), u'_py_serial': 1, u'name': u'one'}
+
+ 1. Transaction A loads the object:
+
+ >>> foo_A = dm.load(foo_ref)
+ >>> foo_A.name
+ u'one'
+
+ 2. Transaction B comes along and modifies Foos data and commits:
+
+ >>> dm_B = datamanager.MongoDataManager(
+ ... conn, default_database=DBNAME, root_database=DBNAME,
+ ... conflict_handler_factory=conflict.SimpleSerialConflictHandler)
+
+ >>> foo_B = dm_B.load(foo_ref)
+ >>> foo_B.name = 'eins'
+ >>> dm_B.tpc_finish(None)
+
+ >>> coll.find_one({})
+ {u'_id': ObjectId('...'), u'_py_serial': 2, u'name': u'eins'}
+
+ 3. Transaction A modifies Foo and the data is flushed. At this point a
+ conflict is detected and reported:
+
+ >>> foo_A.name = '1'
+ >>> dm.flush()
+ Traceback (most recent call last):
+ ...
+ ConflictError: database conflict error
+ (oid DBRef('mongopersist.tests.test_conflict.Foo',
+ ObjectId('4f74bf0237a08e3085000002'),
+ 'mongopersist_test'),
+ class Foo, orig serial 1, cur serial 2, new serial 2)
+ """
+
+def doctest_ResolvingSerialConflictHandler_basic():
+ r"""class ResolvingSerialConflictHandler: basic
+
+ This conflict handler detects conflicts by comparing serial numbers and
+ allows objects to resolve conflicts by calling their
+ ``_p_resolveConflict()`` method. Otherwise the handler is identical to the
+ simple serial handler.
+
+ >>> handler = conflict.ResolvingSerialConflictHandler(dm)
+ >>> l1 = MergerList([1, 2, 3])
+ >>> l1.list
+ [1, 2, 3]
+
+ >>> handler.resolve(
+ ... l1,
+ ... orig_doc = {'list': [1, 2, 3], '_id': 1, '_py_serial': 0},
+ ... cur_doc = {'list': [1, 2, 3, 4], '_id': 1, '_py_serial': 1},
+ ... new_doc = {'list': [1, 2, 3, 5], '_id': 1, '_py_serial': 0})
+ True
+ >>> l1.list
+ [1, 2, 3, 4, 5]
+
+ Resolving always fails, if there is no ``_p_resolveConflict()`` method:
+
+ >>> foo = Foo('one')
+ >>> handler.resolve(
+ ... foo,
+ ... orig_doc = {'name': 'one', '_id': 1, '_py_serial': 0},
+ ... cur_doc = {'name': 'eins', '_id': 1, '_py_serial': 1},
+ ... new_doc = {'name': '1', '_id': 1, '_py_serial': 0})
+ False
+
+ """
+
+def doctest_ResolvingSerialConflictHandler_full():
+ r"""class ResolvingSerialConflictHandler: Full conflict test.
+
+ This test demonstrates the conflict resolution behavior of the
+ ``ResolvingSerialConflictHandler`` conflict handler during a real session.
+
+ First let's create an initial state:
+
+ >>> dm.conflict_handler = conflict.ResolvingSerialConflictHandler(dm)
+ >>> dm.reset()
+ >>> ml = MergerList([1, 2, 3])
+ >>> ml_ref = dm.insert(ml)
+ >>> dm.reset()
+
+ >>> coll = dm._get_collection_from_object(ml)
+ >>> coll.find_one({})
+ {u'list': [1, 2, 3], u'_id': ObjectId('...'), u'_py_serial': 1}
+
+ 1. Transaction A loads the object:
+
+ >>> ml_A = dm.load(ml_ref)
+ >>> ml_A.list
+ [1, 2, 3]
+
+ 2. Transaction B comes along, adds a new item to the list and commits:
+
+ >>> dm_B = datamanager.MongoDataManager(
+ ... conn, default_database=DBNAME, root_database=DBNAME,
+ ... conflict_handler_factory=conflict.ResolvingSerialConflictHandler)
+
+ >>> ml_B = dm_B.load(ml_ref)
+ >>> ml_B.list.append(4)
+ >>> dm_B.tpc_finish(None)
+
+ >>> coll.find_one({})
+ {u'list': [1, 2, 3, 4], u'_id': ObjectId('...'), u'_py_serial': 2}
+
+ 3. Transaction A adds also an item and the data is flushed. At this point a
+ conflict is detected, reported and resolved:
+
+ >>> ml_A.list.append(5)
+ >>> ml_A._p_changed = True
+ >>> ml_A.list
+ [1, 2, 3, 5]
+ >>> dm.flush()
+ >>> ml_A.list
+ [1, 2, 3, 4, 5]
+ >>> ml_A._p_serial
+ '\x00\x00\x00\x00\x00\x00\x00\x03'
+
+ >>> coll.find_one({})
+ {u'list': [1, 2, 3, 4, 5], u'_id': ObjectId('...'), u'_py_serial': 3}
+ """
+
+def test_suite():
+ return doctest.DocTestSuite(
+ setUp=testing.setUp, tearDown=testing.tearDown,
+ checker=testing.checker,
+ optionflags=testing.OPTIONFLAGS)
Property changes on: mongopersist/trunk/src/mongopersist/tests/test_conflict.py
___________________________________________________________________
Added: svn:keywords
+ Id
Modified: mongopersist/trunk/src/mongopersist/tests/test_datamanager.py
===================================================================
--- mongopersist/trunk/src/mongopersist/tests/test_datamanager.py 2012-03-29 17:25:32 UTC (rev 124796)
+++ mongopersist/trunk/src/mongopersist/tests/test_datamanager.py 2012-03-29 21:47:03 UTC (rev 124797)
@@ -17,7 +17,7 @@
import transaction
from pymongo import dbref, objectid
-from mongopersist import interfaces, testing, datamanager
+from mongopersist import conflict, interfaces, testing, datamanager
class Foo(persistent.Persistent):
def __init__(self, name=None):
@@ -26,19 +26,6 @@
class Bar(persistent.Persistent):
_p_mongo_sub_object = True
-def doctest_create_conflict_error():
- r"""create_conflict_error(): General Test
-
- Simple helper function to create a conflict error.
-
- >>> foo = Foo()
-
- >>> datamanager.create_conflict_error(
- ... foo, {'_py_serial': 1}, {'_py_serial': 2}, {'_py_serial': 3})
- ConflictError: database conflict error
- (oid None, class Foo, orig serial 1, cur serial 2, new serial 3)
- """
-
def doctest_Root():
r"""Root: General Test
@@ -171,12 +158,10 @@
>>> dm = datamanager.MongoDataManager(
... conn,
- ... detect_conflicts=True,
... default_database = DBNAME,
... root_database = DBNAME,
... root_collection = 'proot',
- ... name_map_collection = 'coll_pypath_map',
- ... conflict_error_factory = datamanager.create_conflict_error)
+ ... name_map_collection = 'coll_pypath_map')
There are two convenience methods that let you serialize and de-serialize
objects explicitly:
@@ -225,7 +210,7 @@
We also want to test the effects of conflict detection:
- >>> dm.detect_conflicts = True
+ >>> dm.conflict_handler = conflict.SimpleSerialConflictHandler(dm)
Let's now add an object to the database and reset the manager like it is
done at the end of a transaction:
@@ -345,7 +330,7 @@
This test ensures that if the datamanager has conflict detection turned
on, all the needed helper fields are written.
- >>> dm.detect_conflicts = True
+ >>> dm.conflict_handler = conflict.SimpleSerialConflictHandler(dm)
>>> foo = Foo('foo')
>>> foo_ref = dm.insert(foo)
@@ -598,7 +583,6 @@
({u'_id': ObjectId('4f5c114f37a08e2cac000000'), u'name': u'one'},
{u'_id': ObjectId('4f5c114f37a08e2cac000001'), u'name': u'two'},
{u'_id': ObjectId('4f5c114f37a08e2cac000002'), u'name': u'3'})
-
"""
def doctest_MongoDataManager_abort_conflict_detection():
@@ -612,7 +596,7 @@
First let's create an initial state:
- >>> dm.detect_conflicts = True
+ >>> dm.conflict_handler = conflict.SimpleSerialConflictHandler(dm)
>>> dm.reset()
>>> foo_ref = dm.insert(Foo('one'))
>>> dm.reset()
@@ -628,8 +612,9 @@
2. Transaction B comes along and modifies the object as well and commits:
>>> dm_B = datamanager.MongoDataManager(
- ... conn, detect_conflicts=True,
- ... default_database=DBNAME, root_database=DBNAME)
+ ... conn,
+ ... default_database=DBNAME, root_database=DBNAME,
+ ... conflict_handler_factory=conflict.SimpleSerialConflictHandler)
>>> foo_B = dm_B.load(foo_ref)
>>> foo_B.name = 'Eins'
@@ -668,7 +653,7 @@
This method finishes the two-phase commit. So let's store a simple object:
>>> foo = Foo()
- >>> dm.detect_conflicts = True
+ >>> dm.conflict_handler = conflict.SimpleSerialConflictHandler(dm)
>>> dm._registered_objects = [foo]
>>> dm.tpc_finish(transaction.get())
>>> foo._p_serial
Modified: mongopersist/trunk/src/mongopersist/tests/test_serialize.py
===================================================================
--- mongopersist/trunk/src/mongopersist/tests/test_serialize.py 2012-03-29 17:25:32 UTC (rev 124796)
+++ mongopersist/trunk/src/mongopersist/tests/test_serialize.py 2012-03-29 21:47:03 UTC (rev 124797)
@@ -19,7 +19,7 @@
from pymongo import binary, dbref, objectid
-from mongopersist import testing, serialize
+from mongopersist import conflict, serialize, testing
class Top(persistent.Persistent):
_p_mongo_collection = 'Top'
@@ -347,7 +347,7 @@
manager can then use the serial to detect whether a competing transaction
has written to the document.
- >>> dm.detect_conflicts = True
+ >>> dm.conflict_handler = conflict.SimpleSerialConflictHandler(dm)
>>> writer = serialize.ObjectWriter(dm)
>>> top = Top()
@@ -640,7 +640,7 @@
def doctest_ObjectReader_set_ghost_state():
r"""ObjectReader: set_ghost_state()
- >>> dm.detect_conflicts = True
+ >>> dm.conflict_handler = conflict.SimpleSerialConflictHandler(dm)
>>> writer = serialize.ObjectWriter(dm)
>>> top = Top()
More information about the checkins
mailing list