[Checkins] SVN: persistent/trunk/ Move main narrative doctests to 'docs/usage.rst'.
Tres Seaver
tseaver at palladion.com
Thu Feb 17 18:26:52 EST 2011
Log message for revision 120417:
Move main narrative doctests to 'docs/usage.rst'.
Changed:
A persistent/trunk/docs/glossary.rst
U persistent/trunk/docs/index.rst
U persistent/trunk/docs/using.rst
U persistent/trunk/persistent/tests/persistent.txt
-=-
Added: persistent/trunk/docs/glossary.rst
===================================================================
--- persistent/trunk/docs/glossary.rst (rev 0)
+++ persistent/trunk/docs/glossary.rst 2011-02-17 23:26:51 UTC (rev 120417)
@@ -0,0 +1,44 @@
+.. _glossary:
+
+Glossary
+========
+
+.. glossary::
+ :sorted:
+
+ data manager
+ The object responsible for storing and loading an object's
+ :term:`pickled data` in a backing store. Also called a :term:`jar`.
+
+ jar
+ Alias for :term:`data manager`: short for "pickle jar", because
+ it traditionally holds the :term:`pickled data` of persistent objects.
+
+ object cache
+ An MRU cache for objects associated with a given :term:`data manager`.
+
+ ghost
+ An object whose :term:`pickled data` has not yet been loaded from its
+ :term:`jar`. Accessing or mutating any of its attributes causes
+ that data to be loaded, which is referred to as :term:`activation`.
+
+ volatile attribute
+ Attributes of a persistent object which are *not* caputured as part
+ of its :term:`pickled data`. These attributes thus disappear during
+ :term:`deactivation` or :term:`invalidation`.
+
+ pickled data
+ The serialized data of a persistent object, stored in and retrieved
+ from a backing store by a :term:`data manager`.
+
+ activation
+ Moving an object from the ``GHOST`` state to the ``UPTODATE`` state,
+ load its :term:`pickled data` from its :term:`jar`.
+
+ deactivation
+ Moving an object from the ``UPTODATE`` state to the ``GHOST`` state,
+ discarding its :term:`pickled data`.
+
+ invalidation
+ Moving an object from either the ``UPTODATE`` state or the ``CHANGED``
+ state to the ``GHOST`` state, discarding its :term:`pickled data`.
Modified: persistent/trunk/docs/index.rst
===================================================================
--- persistent/trunk/docs/index.rst 2011-02-17 18:41:52 UTC (rev 120416)
+++ persistent/trunk/docs/index.rst 2011-02-17 23:26:51 UTC (rev 120417)
@@ -12,6 +12,7 @@
using
api
+ glossary
Indices and tables
==================
Modified: persistent/trunk/docs/using.rst
===================================================================
--- persistent/trunk/docs/using.rst 2011-02-17 18:41:52 UTC (rev 120416)
+++ persistent/trunk/docs/using.rst 2011-02-17 23:26:51 UTC (rev 120417)
@@ -1,11 +1,7 @@
Using :mod:`persistent` in your application
===========================================
-.. note::
- This document is under construction. More basic documentation will
- eventually appear here.
-
Inheriting from :class:`persistent.Persistent`
----------------------------------------------
@@ -13,45 +9,436 @@
is mix-in interitance. Instances whose classes derive from
:class:`persistent.Persistent` are automatically capable of being
created as :term:`ghost` instances, being associated with a database
-connection (called :term:`jar`), and notifying the connection when they
-have been changed.
+connection (called the :term:`jar`), and notifying the connection when
+they have been changed.
-Overriding the attribute protocol
----------------------------------
+Relationship to a Data Manager and its Cache
+--------------------------------------------
-Subclasses can override the attribute-management methods provided by
-:class:`persistent.Persistent`. For the `__getattr__` method, the behavior
-is like that for regular Python classes and for earlier versions of ZODB 3.
+Except immediately after their creation, persistent objects are normally
+associated with a :term:`data manager` (also referred to as a :term:`jar`).
+An object's data manager is stored in its ``_p_jar`` attribute.
+The data manager is responsible for loading and saving the state of the
+persistent object to some sort of backing store, including managing any
+interactions with transaction machinery.
-When overriding `__getattribute__`, the derived class implementation
-**must** first call :meth:`persistent.Persistent._p_getattr`, passing the
-name being accessed. This method ensures that the object is activated,
-if needed, and handles the "special" attributes which do not require
-activation (e.g., ``_p_oid``, ``__class__``, ``__dict__``, etc.)
-If ``_p_getattr`` returns ``True``, the derived class implementation
-**must** delegate to the base class implementation for the attribute.
+Each data manager maintains an :term:`object cache`, which keeps track of
+the currently loaded objects, as well as any objects they reference which
+have not yet been loaded: such an object is called a :term:`ghost`.
+The cache is stored on the data manager in its ``_cache`` attribute.
-When overriding `__setattr__`, the derived class implementation
-**must** first call :meth:`persistent.Persistent._p_setattr`, passing the
-name being accessed and the value. This method ensures that the object is
-activated, if needed, and handles the "special" attributes which do not
-require activation (``_p_*``). If ``_p_setattr`` returns ``True``, the
-derived implementation must assume that the attribute value has been set by
-the base class.
+A persistent object remains in the ghost state until the application
+attempts to access or mutate one of its attributes: at that point, the
+object requests that its data manager load its state. The persistent
+object also notifies the cache that it has been loaded, as well as on
+each subsequent attribute access. The cache keeps a "most-recently-used"
+list of its objects, and removes objects in least-recently-used order
+when it is asked to reduce its working set.
-When overriding `__detattr__`, the derived class implementation
-**must** first call :meth:`persistent.Persistent._p_detattr`, passing the
-name being accessed. This method ensures that the object is
-activated, if needed, and handles the "special" attributes which do not
-require activation (``_p_*``). If ``_p_delattr`` returns ``True``, the
-derived implementation must assume that the attribute has been deleted
-base class.
+The examples below use a stub data manager class, and its stub cache class:
+.. doctest::
+ >>> class Cache(object):
+ ... def __init__(self):
+ ... self._mru = []
+ ... def mru(self, oid):
+ ... self._mru.append(oid)
+ >>> from zope.interface import implements
+ >>> from persistent.interfaces import IPersistentDataManager
+ >>> class DM(object):
+ ... implements(IPersistentDataManager)
+ ... def __init__(self):
+ ... self._cache = Cache()
+ ... self.registered = 0
+ ... def register(self, ob):
+ ... self.registered += 1
+ ... def setstate(self, ob):
+ ... ob.__setstate__({'x': 42})
-More Examples
--------------
+.. note::
+ Notic that the ``DM`` class always sets the ``x`` attribute to the value
+ ``42`` when activating an object.
-Detailed examples are provided in the test module,
-`persistent.tests.test_overriding_attrs`.
+
+Persistent objects without a Data Manager
+-----------------------------------------
+
+Before aersistent instance has been associtated with a a data manager (
+i.e., its ``_p_jar`` is still ``None``).
+
+The examples below use a class, ``P``, defined as:
+
+.. doctest::
+
+ >>> from persistent import Persistent
+ >>> from persistent.interfaces import GHOST, UPTODATE, CHANGED
+ >>> class P(Persistent):
+ ... def __init__(self):
+ ... self.x = 0
+ ... def inc(self):
+ ... self.x += 1
+
+Instances of the derived ``P`` class which are not (yet) assigned to
+a :term:`data manager` behave as other Python instances, except that
+they have some extra attributes:
+
+.. doctest::
+
+ >>> p = P()
+ >>> p.x
+ 0
+
+The :attr:`_p_changed` attribute is a three-state flag: it can be
+one of ``None`` (the object is not loaded), ``False`` (the object has
+not been changed since it was loaded) or ``True`` (the object has been
+changed). Until the object is assigned a :term:`jar`, this attribute
+will always be ``False``.
+
+.. doctest::
+
+ >>> p._p_changed
+ False
+
+The :attr:`_p_state` attribute is an integaer, representing which of the
+"persistent lifecycle" states the object is in. Until the object is assigned
+a :term:`jar`, this attribute will always be ``0`` (the ``UPTODATE``
+constant):
+
+.. doctest::
+
+ >>> p._p_state == UPTODATE
+ True
+
+The :attr:`_p_jar` attribute is the object's :term:`data manager`. Since
+it has not yet been assigned, its value is ``None``:
+
+.. doctest::
+
+ >>> print p._p_jar
+ None
+
+The :attr:`_p_oid` attribute is the :term:`object id`, a unique value
+normally assigned by the object's :term:`data manager`. Since the object
+has not yet been associated with its :term:`jar`, its value is ``None``:
+
+.. doctest::
+
+ >>> print p._p_oid
+ None
+
+Without a data manager, modifying a persistent object has no effect on
+its ``_p_state`` or ``_p_changed``.
+
+.. doctest::
+
+ >>> p.inc()
+ >>> p.inc()
+ >>> p.x
+ 2
+ >>> p._p_changed
+ False
+ >>> p._p_state
+ 0
+
+Try all sorts of different ways to change the object's state:
+
+.. doctest::
+
+ >>> p._p_deactivate()
+ >>> p._p_state
+ 0
+ >>> p._p_changed
+ False
+ >>> p._p_changed = True
+ >>> p._p_changed
+ False
+ >>> p._p_state
+ 0
+ >>> del p._p_changed
+ >>> p._p_changed
+ False
+ >>> p._p_state
+ 0
+ >>> p.x
+ 2
+
+
+Associating an Object with a Data Manager
+-----------------------------------------
+
+Once associated with a data manager, a persistent object's behavior changes:
+
+.. doctest::
+
+ >>> p = P()
+ >>> dm = DM()
+ >>> p._p_oid = "00000012"
+ >>> p._p_jar = dm
+ >>> p._p_changed
+ False
+ >>> p._p_state
+ 0
+ >>> p.__dict__
+ {'x': 0}
+ >>> dm.registered
+ 0
+
+Modifying the object marks it as changed and registers it with the data
+manager. Subsequent modifications don't have additional side-effects.
+
+.. doctest::
+
+ >>> p.inc()
+ >>> p.x
+ 1
+ >>> p.__dict__
+ {'x': 1}
+ >>> p._p_changed
+ True
+ >>> p._p_state
+ 1
+ >>> dm.registered
+ 1
+ >>> p.inc()
+ >>> p._p_changed
+ True
+ >>> p._p_state
+ 1
+ >>> dm.registered
+ 1
+
+Object which register themselves with the data manager are candidates
+for storage to the backing store at a later point in time.
+
+
+Explicitly controlling ``_p_state``
+-----------------------------------
+
+Persistent objects expose three methods for moving an object into and out
+of the "ghost" state:: :meth:`persistent.Persistent._p_activate`,
+:meth:`persistent.Persistent._p_activate_p_deactivate`, and
+:meth:`persistent.Persistent._p_invalidate`:
+
+.. doctest::
+
+ >>> p = P()
+ >>> p._p_oid = '00000012'
+ >>> p._p_jar = DM()
+
+After being assigned a jar, the object is initially in the ``UPTODATE``
+state:
+
+.. doctest::
+
+ >>> p._p_state
+ 0
+
+From that state, ``_p_deactivate`` rests the object to the ``GHOST`` state:
+
+.. doctest::
+
+ >>> p._p_deactivate()
+ >>> p._p_state
+ -1
+
+From the ``GHOST`` state, ``_p_activate`` reloads the object's data and
+moves it to the ``UPTODATE`` state:
+
+.. doctest::
+
+ >>> p._p_activate()
+ >>> p._p_state
+ 0
+ >>> p.x
+ 42
+
+Changing the object puts it in the ``CHANGED`` state:
+
+.. doctest::
+
+ >>> p.inc()
+ >>> p.x
+ 43
+ >>> p._p_state
+ 1
+
+Attempting to deactivate in the ``CHANGED`` state is a no-op:
+
+.. doctest::
+
+ >>> p._p_deactivate()
+ >>> p.__dict__
+ {'x': 43}
+ >>> p._p_changed
+ True
+ >>> p._p_state
+ 1
+
+``_p_invalidate`` forces objects into the ``GHOST`` state; it works even on
+objects in the ``CHANGED`` state, which is the key difference between
+deactivation and invalidation:
+
+.. doctest::
+
+ >>> p._p_invalidate()
+ >>> p.__dict__
+ {}
+ >>> p._p_state
+ -1
+
+You can manually reset the ``_p_changed`` field to ``False``: in this case,
+the object changes to the ``UPTODATE`` state but retains its modifications:
+
+.. doctest::
+
+ >>> p.inc()
+ >>> p.x
+ 43
+ >>> p._p_changed = False
+ >>> p._p_state
+ 0
+ >>> p._p_changed
+ False
+ >>> p.x
+ 43
+
+For an object in the "ghost" state, assigning ``True`` (or any value which is
+coercible to ``True``) to its ``_p_changed`` attributes activates the object,
+which is exactly the same as calling ``_p_activate``:
+
+.. doctest::
+
+ >>> p._p_invalidate()
+ >>> p._p_state
+ -1
+ >>> p._p_changed = True
+ >>> p._p_changed
+ True
+ >>> p._p_state
+ 1
+ >>> p.x
+ 42
+
+
+The pickling protocol
+---------------------
+
+Because persistent objects need to control how they are pickled and
+unpickled, the :class:`persistent.Persistent` base class overrides
+the implementations of ``__getstate__()`` and ``__setstate__()``:
+
+.. doctest::
+
+ >>> p = P()
+ >>> dm = DM()
+ >>> p._p_oid = "00000012"
+ >>> p._p_jar = dm
+ >>> p.__getstate__()
+ {'x': 0}
+ >>> p._p_state
+ 0
+
+Calling ``__setstate__`` always leaves the object in the uptodate state.
+
+.. doctest::
+
+ >>> p.__setstate__({'x': 5})
+ >>> p._p_state
+ 0
+
+A :term:`volatile attribute` is an attribute those whose name begins with a
+special prefix (``_v__``). Unlike normal attributes, volatile attributes do
+not get stored in the object's :term:`pickled data`.
+
+.. doctest::
+
+ >>> p._v_foo = 2
+ >>> p.__getstate__()
+ {'x': 5}
+
+Assigning to volatile attributes doesn't cause the object to be marked as
+changed:
+
+.. doctest::
+
+ >>> p._p_state
+ 0
+
+The ``_p_serial`` attribute is not affected by calling setstate.
+
+.. doctest::
+
+ >>> p._p_serial = "00000012"
+ >>> p.__setstate__(p.__getstate__())
+ >>> p._p_serial
+ '00000012'
+
+
+Estimated Object Size
+---------------------
+
+We can store a size estimation in ``_p_estimated_size``. Its default is 0.
+The size estimation can be used by a cache associated with the data manager
+to help in the implementation of its replacement strategy or its size bounds.
+
+.. doctest::
+
+ >>> p._p_estimated_size
+ 0
+ >>> p._p_estimated_size = 1000
+ >>> p._p_estimated_size
+ 1024
+
+Huh? Why is the estimated size coming out different than what we put
+in? The reason is that the size isn't stored exactly. For backward
+compatibility reasons, the size needs to fit in 24 bits, so,
+internally, it is adjusted somewhat.
+
+Of course, the estimated size must not be negative.
+
+.. doctest::
+
+ >>> p._p_estimated_size = -1
+ Traceback (most recent call last):
+ ....
+ ValueError: _p_estimated_size must not be negative
+
+
+Overriding the attribute protocol
+---------------------------------
+
+Subclasses which override the attribute-management methods provided by
+:class:`persistent.Persistent`, but must obey some constraints:
+
+:meth:`__getattribute__``
+ When overriding ``__getattribute__``, the derived class implementation
+ **must** first call :meth:`persistent.Persistent._p_getattr`, passing the
+ name being accessed. This method ensures that the object is activated,
+ if needed, and handles the "special" attributes which do not require
+ activation (e.g., ``_p_oid``, ``__class__``, ``__dict__``, etc.)
+ If ``_p_getattr`` returns ``True``, the derived class implementation
+ **must** delegate to the base class implementation for the attribute.
+
+:meth:`__setattr__`
+ When overriding ``__setattr__``, the derived class implementation
+ **must** first call :meth:`persistent.Persistent._p_setattr`, passing the
+ name being accessed and the value. This method ensures that the object is
+ activated, if needed, and handles the "special" attributes which do not
+ require activation (``_p_*``). If ``_p_setattr`` returns ``True``, the
+ derived implementation must assume that the attribute value has been set by
+ the base class.
+
+:meth:`__detattr__`
+ When overriding ``__detattr__``, the derived class implementation
+ **must** first call :meth:`persistent.Persistent._p_detattr`, passing the
+ name being accessed. This method ensures that the object is
+ activated, if needed, and handles the "special" attributes which do not
+ require activation (``_p_*``). If ``_p_delattr`` returns ``True``, the
+ derived implementation must assume that the attribute has been deleted
+ base class.
+
+:meth:`__getattr__`
+ For the `__getattr__` method, the behavior is like that for regular Python
+ classes and for earlier versions of ZODB 3.
Modified: persistent/trunk/persistent/tests/persistent.txt
===================================================================
--- persistent/trunk/persistent/tests/persistent.txt 2011-02-17 18:41:52 UTC (rev 120416)
+++ persistent/trunk/persistent/tests/persistent.txt 2011-02-17 23:26:51 UTC (rev 120417)
@@ -1,8 +1,11 @@
Tests for `persistent.Persistent`
=================================
-This document is an extended doc test that covers the basics of the
-Persistent base class. The test expects a class named `P` to be
+This document covers "edge case" tests for the Persistent base class.
+It should be replaced with normal unit tests. (The meat of the narrative
+documentation is now in ``docs/usage`` of the distribution).
+
+The test expects a class named `P` to be
provided in its globals. The `P` class implements the `Persistent`
interface.
@@ -42,236 +45,6 @@
>>> from persistent import Persistent
-Test Persistent without Data Manager
-------------------------------------
-
-First do some simple tests of a Persistent instance that does not have
-a data manager (``_p_jar``).
-
- >>> p = P()
- >>> p.x
- 0
- >>> p._p_changed
- False
- >>> p._p_state
- 0
- >>> p._p_jar
- >>> p._p_oid
-
-Verify that modifications have no effect on ``_p_state`` of ``_p_changed``.
-
- >>> p.inc()
- >>> p.inc()
- >>> p.x
- 2
- >>> p._p_changed
- False
- >>> p._p_state
- 0
-
-Try all sorts of different ways to change the object's state.
-
- >>> p._p_deactivate()
- >>> p._p_state
- 0
- >>> p._p_changed = True
- >>> p._p_state
- 0
- >>> del p._p_changed
- >>> p._p_changed
- False
- >>> p._p_state
- 0
- >>> p.x
- 2
-
-We can store a size estimation in ``_p_estimated_size``. Its default is 0.
-The size estimation can be used by a cache associated with the data manager
-to help in the implementation of its replacement strategy or its size bounds.
-Of course, the estimated size must not be negative.
-
- >>> p._p_estimated_size
- 0
- >>> p._p_estimated_size = 1000
- >>> p._p_estimated_size
- 1024
-
-Huh? Why is the estimated size coming out different than what we put
-in? The reason is that the size isn't stored exactly. For backward
-compatibility reasons, the size needs to fit in 24 bits, so,
-internally, it is adjusted somewhat.
-
- >>> p._p_estimated_size = -1
- Traceback (most recent call last):
- ....
- ValueError: _p_estimated_size must not be negative
-
-
-
-
-
-Test Persistent with Data Manager
----------------------------------
-
-Next try some tests of an object with a data manager. The `DM` class is
-a simple testing stub.
-
- >>> p = P()
- >>> dm = DM()
- >>> p._p_oid = "00000012"
- >>> p._p_jar = dm
- >>> p._p_changed
- 0
- >>> dm.called
- 0
-
-Modifying the object marks it as changed and registers it with the data
-manager. Subsequent modifications don't have additional side-effects.
-
- >>> p.inc()
- >>> p._p_changed
- 1
- >>> dm.called
- 1
- >>> p.inc()
- >>> p._p_changed
- 1
- >>> dm.called
- 1
-
-It's not possible to deactivate a modified object.
-
- >>> p._p_deactivate()
- >>> p._p_changed
- 1
-
-It is possible to invalidate it. That's the key difference between
-deactivation and invalidation.
-
- >>> p._p_invalidate()
- >>> p._p_state
- -1
-
-Now that the object is a ghost, any attempt to modify it will require that it
-be unghosted first. The test data manager has the odd property that it sets
-the object's ``x`` attribute to ``42`` when it is unghosted.
-
- >>> p.inc()
- >>> p.x
- 43
- >>> dm.called
- 2
-
-You can manually reset the changed field to ``False``, although it's not clear
-why you would want to do that. The object changes to the ``UPTODATE`` state
-but retains its modifications.
-
- >>> p._p_changed = False
- >>> p._p_state
- 0
- >>> p._p_changed
- False
- >>> p.x
- 43
-
- >>> p.inc()
- >>> p._p_changed
- True
- >>> dm.called
- 3
-
-``__getstate__()`` and ``__setstate__()``
------------------------------------------
-
-The next several tests cover the ``__getstate__()`` and ``__setstate__()``
-implementations.
-
- >>> p = P()
- >>> state = p.__getstate__()
- >>> isinstance(state, dict)
- True
- >>> state['x']
- 0
- >>> p._p_state
- 0
-
-Calling setstate always leaves the object in the uptodate state?
-(I'm not entirely clear on this one.)
-
- >>> p.__setstate__({'x': 5})
- >>> p._p_state
- 0
-
-Assigning to a volatile attribute has no effect on the object state.
-
- >>> p._v_foo = 2
- >>> p.__getstate__()
- {'x': 5}
- >>> p._p_state
- 0
-
-The ``_p_serial`` attribute is not affected by calling setstate.
-
- >>> p._p_serial = "00000012"
- >>> p.__setstate__(p.__getstate__())
- >>> p._p_serial
- '00000012'
-
-
-Change Ghost test
------------------
-
-If an object is a ghost and its ``_p_changed`` is set to ``True`` (any true
-value), it should activate (unghostify) the object. This behavior is new in
-ZODB 3.6; before then, an attempt to do ``ghost._p_changed = True`` was
-ignored.
-
- >>> p = P()
- >>> p._p_jar = DM()
- >>> p._p_oid = 1
- >>> p._p_deactivate()
- >>> p._p_changed # None
- >>> p._p_state # ghost state
- -1
- >>> p._p_changed = True
- >>> p._p_changed
- 1
- >>> p._p_state # changed state
- 1
- >>> p.x
- 42
-
-
-Activate, deactivate, and invalidate
-------------------------------------
-
-Some of these tests are redundant, but are included to make sure there
-are explicit and simple tests of ``_p_activate()``, ``_p_deactivate()``, and
-``_p_invalidate()``.
-
- >>> p = P()
- >>> p._p_oid = 1
- >>> p._p_jar = DM()
- >>> p._p_deactivate()
- >>> p._p_state
- -1
- >>> p._p_activate()
- >>> p._p_state
- 0
- >>> p.x
- 42
- >>> p.inc()
- >>> p.x
- 43
- >>> p._p_state
- 1
- >>> p._p_invalidate()
- >>> p._p_state
- -1
- >>> p.x
- 42
-
-
Test failures
-------------
More information about the checkins
mailing list