[Checkins] SVN: mongopersist/trunk/ - Performance: The Zope Container fast load via find() did not work correctly,

[Stephen Richter]

Log message for revision 124835:
  - Performance: The Zope Container fast load via find() did not work correctly,
    since setstate() did not change the state from ghost to active and thus the
    state was loaded again from MongoDB and set on the object. Now we use the
    new ``_latest_states`` cache to lookup a document when ``setstate()`` is
    called through the proper channels.
  
  Between this and the previous checkin, fast loading is now 30 times 
  faster than before.
  
  

Changed:
  U   mongopersist/trunk/CHANGES.txt
  U   mongopersist/trunk/src/mongopersist/datamanager.py
  U   mongopersist/trunk/src/mongopersist/zope/container.py

-=-
Modified: mongopersist/trunk/CHANGES.txt
===================================================================
--- mongopersist/trunk/CHANGES.txt	2012-04-01 01:27:20 UTC (rev 124834)
+++ mongopersist/trunk/CHANGES.txt	2012-04-01 01:58:30 UTC (rev 124835)
@@ -51,6 +51,12 @@
   only one type of objects and where the documents do not store the type
   (i.e. it is stored in the name map collection).
 
+- Performance: The Zope Container fast load via find() did not work correctly,
+  since setstate() did not change the state from ghost to active and thus the
+  state was loaded again from MongoDB and set on the object. Now we use the
+  new ``_latest_states`` cache to lookup a document when ``setstate()`` is
+  called through the proper channels.
+
 - Bug: We have seen several occasions in production where we suddenly lost
   some state in some documents, which prohibited the objects from being
   loadable again. The cause was that the ``_original_states`` attribute did not

Modified: mongopersist/trunk/src/mongopersist/datamanager.py
===================================================================
--- mongopersist/trunk/src/mongopersist/datamanager.py	2012-04-01 01:27:20 UTC (rev 124834)
+++ mongopersist/trunk/src/mongopersist/datamanager.py	2012-04-01 01:58:30 UTC (rev 124835)
@@ -294,6 +294,12 @@
         if self._needs_to_join:
             self.transaction_manager.get().join(self)
             self._needs_to_join = False
+        # If the doc is None, but it has been loaded before, we look it
+        # up. This acts as a great hook for optimizations that load many
+        # documents at once. They can now dump the states into the
+        # _latest_states dictionary.
+        if doc is None:
+            doc = self._latest_states.get(obj._p_oid, None)
         self._reader.set_ghost_state(obj, doc)
         self._loaded_objects.append(obj)
 

Modified: mongopersist/trunk/src/mongopersist/zope/container.py
===================================================================
--- mongopersist/trunk/src/mongopersist/zope/container.py	2012-04-01 01:27:20 UTC (rev 124834)
+++ mongopersist/trunk/src/mongopersist/zope/container.py	2012-04-01 01:58:30 UTC (rev 124835)
@@ -154,8 +154,9 @@
         dbref = pymongo.dbref.DBRef(
             self._m_collection, doc['_id'],
             self._m_database or self._m_jar.default_database)
-        obj = self._m_jar._reader.get_ghost(dbref)
-        self._m_jar.setstate(obj, doc)
+        # Stick the doc into the _latest_states:
+        self._m_jar._latest_states[dbref] = doc
+        obj = self._m_jar.load(dbref)
         obj._v_key = doc[self._m_mapping_key]
         obj._v_parent = self
         return obj