[Zodb-checkins] CVS: ZODB/src/persistent - dict.py:1.1.2.1 interfaces.py:1.1.2.1 wref.py:1.1.2.1

Jeremy Hylton jeremy at zope.com
Thu Feb 12 15:51:13 EST 2004


Update of /cvs-repository/ZODB/src/persistent
In directory cvs.zope.org:/tmp/cvs-serv22637

Added Files:
      Tag: zope3-zodb3-devel-branch
	dict.py interfaces.py wref.py 
Log Message:
Copy files from persistence to persistent.


=== Added File ZODB/src/persistent/dict.py ===
##############################################################################
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
"""Python implementation of persistent container type

$Id: dict.py,v 1.1.2.1 2004/02/12 20:51:12 jeremy Exp $
"""

import persistent
from UserDict import IterableUserDict

__metaclass__ = type

class PersistentDict(persistent.Persistent, IterableUserDict):
    """A persistent wrapper for mapping objects.

    This class allows wrapping of mapping objects so that object
    changes are registered.  As a side effect, mapping objects may be
    subclassed.
    """

    # IterableUserDict provides all of the mapping behavior.  The
    # PersistentDict class is responsible marking the persistent
    # state as changed when a method actually changes the state.  At
    # the mapping API evolves, we may need to add more methods here.

    __super_delitem = IterableUserDict.__delitem__
    __super_setitem = IterableUserDict.__setitem__
    __super_clear = IterableUserDict.clear
    __super_update = IterableUserDict.update
    __super_setdefault = IterableUserDict.setdefault
    __super_popitem = IterableUserDict.popitem

    __super_p_init = persistent.Persistent.__init__
    __super_init = IterableUserDict.__init__

    def __init__(self, dict=None):
        self.__super_init(dict)
        self.__super_p_init()

    def __delitem__(self, key):
        self.__super_delitem(key)
        self._p_changed = True

    def __setitem__(self, key, v):
        self.__super_setitem(key, v)
        self._p_changed = True

    def clear(self):
        self.__super_clear()
        self._p_changed = True

    def update(self, b):
        self.__super_update(b)
        self._p_changed = True

    def setdefault(self, key, failobj=None):
        # We could inline all of UserDict's implementation into the
        # method here, but I'd rather not depend at all on the
        # implementation in UserDict (simple as it is).
        if not self.has_key(key):
            self._p_changed = True
        return self.__super_setdefault(key, failobj)

    def popitem(self):
        self._p_changed = True
        return self.__super_popitem()


=== Added File ZODB/src/persistent/interfaces.py ===
##############################################################################
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
from zope.interface import Interface
from zope.interface import Attribute

class IPersistent(Interface):
    """Python persistent interface

    A persistent object can be in one of several states:

    - Unsaved

      The object has been created but not saved in a data manager.

      In this state, the _p_changed attribute is non-None and false
      and the _p_jar attribute is None.

    - Saved

      The object has been saved and has not been changed since it was saved.

      In this state, the _p_changed attribute is non-None and false
      and the _p_jar attribute is set to a data manager.

    - Sticky

      This state is identical to the up-to-date state except that the
      object cannot transition to the ghost state. This is a special
      state used by C methods of persistent objects to make sure that
      state is not unloaded in the middle of computation.

      In this state, the _p_changed attribute is non-None and false
      and the _p_jar attribute is set to a data manager.

      There is, currently, no official way to detect whether an object
      is in the sticky state.

    - Changed

      The object has been changed.

      In this state, the _p_changed attribute is true
      and the _p_jar attribute is set to a data manager.

    - Ghost

      the object is in memory but its state has not been loaded from
      the database (or has been unloaded).  In this state, the object
      doesn't contain any data.

    The following state transactions are possible:

    - Unsaved -> Saved

      This transition occurs when an object is saved in the
      database. This usually happens when an unsaved object is added
      to (e.g. as an attribute or item of) a saved (or changed) object
      and the transaction is committed.

    - Saved  -> Changed
      Sticky -> Changed

      This transition occurs when someone sets an attribute or sets
      _p_changed to a true value on an up-to-date or sticky
      object. When the transition occurs, the persistent object is
      required to call the register method on its data manager,
      passing itself as the only argument.

    - Saved -> Sticky

      This transition occurs when C code marks the object as sticky to
      prevent its deactivation and transition to the ghost state.

    - Saved -> Ghost

      This transition occurs when an saved object is deactivated, by:
      calling _p_deactivate, setting _p_changed to None, or deleting
      _p_changed.

    - Sticky -> Saved

      This transition occurs when C code unmarks the object as sticky to
      allow its deactivation and transition to the ghost state.

    - Changed -> Saved

      This transition occurs when a transaction is committed.
      The data manager affects the transaction by setting _p_changed
      to a true value.

    - Changed -> Ghost

      This transition occurs when a transaction is aborted.
      The data manager affects the transaction by deleting _p_changed.

    - Ghost -> Saved

      This transition occurs when an attribute or operation of a ghost
      is accessed and the object's state is loaded from the database.

    Note that there is a separate C API that is not included here.
    The C API requires a specific data layout and defines the sticky
    state that is used to prevent object deactivation while in C
    routines.

    """

    _p_jar=Attribute(
        """The data manager for the object

        The data manager implements the IPersistentDataManager interface.
        If there is no data manager, then this is None.
        """)

    _p_oid=Attribute(
        """The object id

        It is up to the data manager to assign this.
        The special value None is reserved to indicate that an object
        id has not been assigned.
        """)

    _p_changed=Attribute(
        """The persistent state of the object

        This is one of:

        None -- The object is a ghost. It is not active.

        false -- The object is saved (or has never been saved).

        true -- The object has been modified.

        The object state may be changed by assigning this attribute,
        however, assigning None is ignored if the object is not in the
        up-to-date state.

        Note that an object can change to the modified state only if
        it has a data manager. When such a state change occurs, the
        'register' method of the data manager is called, passing the
        persistent object.

        Deleting this attribute forces deactivation independent of
        existing state.

        Note that an attribute is used for this to allow optimized
        cache implementations.
        """)

    _p_serial=Attribute(
        """The object serial number

        This is an arbitrary object.
        """)

    _p_atime=Attribute(
        """The integer object access time, in seconds, modulus one day

        XXX When does a day start, the current implementation appears
        to use gmtime, but this hasn't be explicitly specified.

        XXX Why just one day?
        """)

    def __getstate__():
        """Get the object state data

        The state should not include persistent attributes ("_p_name")
        """

    def __setstate__(state):
        """Set the object state data

        Note that this does not affect the object's persistent state.
        """

    def _p_activate():
        """Activate the object

        Change the object to the up-to-date state if it is a ghost.
        """

    def _p_deactivate():
        """Deactivate the object

        If possible, change an object in the up-to-date state to the
        ghost state.  It may not be possible to make some persistent
        objects ghosts.
        """

class IPersistentNoReadConflicts(IPersistent):
    def _p_independent():
        """Hook for subclasses to prevent read conflict errors

        A specific persistent object type can define this method and
        have it return true if the data manager should ignore read
        conflicts for this object.
        """
class IPersistentDataManager(Interface):
    """Provide services for managing persistent state.

    This interface is used by a persistent object to interact with its
    data manager in the context of a transaction.
    """

    def setstate(object):
        """Load the state for the given object.

        The object should be in the deactivated (ghost) state.
        The object's state will be set and the object will end up
        in the up-to-date state.

        The object must implement the IPersistent interface.
        """

    def register(object):
        """Register a IPersistent with the current transaction.

        This method provides some insulation of the persistent object
        from details of transaction management. For example, it allows
        the use of per-database-connection rather than per-thread
        transaction managers.

        A persistent object should not register with its data manager
        more than once during a single transaction.  XXX should is too
        wishy-washy; we should probably guarantee that this is true,
        and it might be.
        """

    def mtime(object):
        """Return the modification time of the object.

        The modification time may not be known, in which case None
        is returned.
        """

class ICache(Interface):
    """In-memory object cache

    The cache serves two purposes.  It peforms pointer swizzling, and
    it keeps a bounded set of recently used but otherwise unreferenced
    in objects to avoid the cost of re-loading them.

    Pointer swizzling is the process of converting between persistent
    object ids and Python object ids.  When a persistent object is
    serialized, its references to other persistent objects are
    represented as persitent object ids (oids).  When the object is
    unserialized, the oids are converted into references to Python
    objects.  If several different serialized objects refer to the
    same object, they must all refer to the same object when they are
    unserialized.

    A cache stores persistent objects, but it treats ghost objects and
    non-ghost or active objects differently.  It has weak references
    to ghost objects, because ghost objects are only stored in the
    cache to satisfy the pointer swizzling requirement.  It has strong
    references to active objects, because it caches some number of
    them even if they are unreferenced.

    The cache keeps some number of recently used but otherwise
    unreferenced objects in memory.  We assume that there is a good
    chance the object will be used again soon, so keeping it memory
    avoids the cost of recreating the object.
    
    An ICache implementation is intended for use by an
    IPersistentDataManager.
    """

    def get(oid):
        """Return the object from the cache or None."""

    def set(oid, obj):
        """Store obj in the cache under oid.

        obj must implement IPersistent
        """

    def remove(oid):
        """Remove oid from the cache if it exists."""

    def invalidate(oids):
        """Make all of the objects in oids ghosts.

        `oids` is an iterable object that yields oids.
        
        The cache must attempt to change each object to a ghost by
        calling _p_deactivate().

        If an oid is not in the cache, ignore it.
        """

    def clear():
        """Invalidate all the active objects."""

    def activate(oid):
        """Notification that object oid is now active.

        The caller is notifying the cache of a state change.

        Raises LookupError if oid is not in cache.
        """

    def shrink():
        """Remove excess active objects from the cache."""

    def statistics():
        """Return dictionary of statistics about cache size.
        
        Contains at least the following keys:
        active -- number of active objects
        ghosts -- number of ghost objects
        """


=== Added File ZODB/src/persistent/wref.py ===
##############################################################################
#
# Copyright (c) 2003 Zope Corporation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
"""ZODB-based persistent weakrefs

$Id: wref.py,v 1.1.2.1 2004/02/12 20:51:12 jeremy Exp $
"""

from persistent import Persistent

WeakRefMarker = object()

class WeakRef(object):
    """Persistent weak references

    Persistent weak references are used much like Python weak references.
    The major difference is that you can't (at least not currently)
    specify an object to be called when the object is removed from the
    database.

    Here's an example. We'll start by creating a persistent object and
    a refernce to it:

    >>> import persistent.list
    >>> import zodb.tests.util
    >>> ob = persistent.list.PersistentList()
    >>> ref = WeakRef(ob)
    >>> ref() is ob
    True

    The hash of the ref if the same as the hash of the refernced object:

    >>> hash(ref) == hash(ob)
    True

    Two refs to the same object are equal:

    >>> WeakRef(ob) == ref
    True
    
    >>> ob2 = persistent.list.PersistentList([1])
    >>> WeakRef(ob2) == ref
    False

    Lets save the refernce and the refernced object in a database:

    >>> db = zodb.tests.util.DB()
    
    >>> conn1 = db.open()
    >>> conn1.root()['ob'] = ob
    >>> conn1.root()['ref'] = ref
    >>> zodb.tests.util.commit()

    If we oprn a new connection, sure enough, we can use the reference:

    >>> conn2 = db.open()
    >>> conn2.root()['ref']() is conn2.root()['ob']
    True
    >>> hash(conn2.root()['ref']) == hash(conn2.root()['ob'])
    True

    But if we delete the referenced object and pack:

    >>> del conn2.root()['ob']
    >>> zodb.tests.util.commit()
    >>> zodb.tests.util.pack(db)

    And then look in a new connection:

    >>> conn3 = db.open()
    >>> conn3.root()['ob']
    Traceback (most recent call last):
    ...
    KeyError: 'ob'

    Trying to dereference the refernce returns None:
    
    >>> conn3.root()['ref']()
    
    Trying to get a hash, raises a type error:

    >>> hash(conn3.root()['ref'])
    Traceback (most recent call last):
    ...
    TypeError: Weakly-referenced object has gone away

    Always explicitly close databases: :)
    
    >>> db.close()

    """

    # We set _p_oid to a merker so that the serialization system can
    # provide special handling of weakrefs.
    _p_oid = WeakRefMarker

    def __init__(self, ob):
        self._v_ob = ob
        self.oid = ob._p_oid
        self.dm = ob._p_jar

    def __call__(self):
        try:
            return self._v_ob
        except AttributeError:
            try:
                self._v_ob = self.dm[self.oid]
            except KeyError:
                return None
            return self._v_ob

    def __hash__(self):
        self = self()
        if self is None:
            raise TypeError('Weakly-referenced object has gone away')
        return hash(self)

    def __eq__(self, other):
        self = self()
        if self is None:
            raise TypeError('Weakly-referenced object has gone away')
        other = other()
        if other is None:
            raise TypeError('Weakly-referenced object has gone away')

        return self == other
    
            
class PersistentWeakKeyDictionary(Persistent):
    """Persistent weak key dictionary

    This is akin to WeakKeyDictionaries. Note, however, that removal
    of items is extremely lazy. See below.

    We'll start by creating a PersistentWeakKeyDictionary and adding
    some persistent objects to it.

    >>> d = PersistentWeakKeyDictionary()
    >>> import zodb.tests.util
    >>> p1 = zodb.tests.util.P('p1')
    >>> p2 = zodb.tests.util.P('p2')
    >>> p3 = zodb.tests.util.P('p3')
    >>> d[p1] = 1
    >>> d[p2] = 2
    >>> d[p3] = 3

    We'll create an extra persustent object that's not in the dict:

    >>> p4 = zodb.tests.util.P('p4')

    Now we'll excercise iteration and item access:

    >>> l = [(str(k), d[k], d.get(k)) for k in d]
    >>> l.sort()
    >>> l
    [('P(p1)', 1, 1), ('P(p2)', 2, 2), ('P(p3)', 3, 3)]

    And the containment operator:

    >>> [p in d for p in [p1, p2, p3, p4]]
    [True, True, True, False]

    We can add the dict and the refernced objects to a database:
    
    >>> db = zodb.tests.util.DB()
    
    >>> conn1 = db.open()
    >>> conn1.root()['p1'] = p1
    >>> conn1.root()['d'] = d
    >>> conn1.root()['p2'] = p2
    >>> conn1.root()['p3'] = p3
    >>> zodb.tests.util.commit()

    And things still work, as before:

    >>> l = [(str(k), d[k], d.get(k)) for k in d]
    >>> l.sort()
    >>> l
    [('P(p1)', 1, 1), ('P(p2)', 2, 2), ('P(p3)', 3, 3)]
    >>> [p in d for p in [p1, p2, p3, p4]]
    [True, True, True, False]

    Likewise, we can read the objects from another connection and
    things still work.

    >>> conn2 = db.open()
    >>> d = conn2.root()['d']
    >>> p1 = conn2.root()['p1']
    >>> p2 = conn2.root()['p2']
    >>> p3 = conn2.root()['p3']
    >>> l = [(str(k), d[k], d.get(k)) for k in d]
    >>> l.sort()
    >>> l
    [('P(p1)', 1, 1), ('P(p2)', 2, 2), ('P(p3)', 3, 3)]
    >>> [p in d for p in [p1, p2, p3, p4]]
    [True, True, True, False]

    Now, we'll delete one of the objects from the database, but *not*
    from the dictionary:

    >>> del conn2.root()['p2']
    >>> zodb.tests.util.commit()

    And pack the database, so that the no-longer referenced p2 is
    actuallt removed from the database.

    >>> zodb.tests.util.pack(db)

    Now if we access the dictionary in a new connection, it no longer
    has p2:
    
    >>> conn3 = db.open()
    >>> d = conn3.root()['d']
    >>> l = [(str(k), d[k], d.get(k)) for k in d]
    >>> l.sort()
    >>> l
    [('P(p1)', 1, 1), ('P(p3)', 3, 3)]

    It's worth nothing that that the versions of the dictionary in
    conn1 and conn2 still have p2, because p2 is still in the caches
    for those connections. 

    Always explicitly close databases: :)
    
    >>> db.close()

    """
    # XXX it is expensive trying to load dead objects from the database.
    #     It would be helpful if the data manager/connection cached these.

    
    def __init__(self):
        self.data = {}

    def __getstate__(self):
        state = Persistent.__getstate__(self)
        state['data'] = state['data'].items()
        return state

    def __setstate__(self, state):
        state['data'] = dict([
            (k, v) for (k, v) in state['data']
            if k() is not None
            ])
        Persistent.__setstate__(self, state)
        
    def __setitem__(self, key, value):
        self.data[WeakRef(key)] = value
        
    def __getitem__(self, key):
        return self.data[WeakRef(key)]
        
    def __delitem__(self, key):
        del self.data[WeakRef(key)]

    def get(self, key):
        return self.data.get(WeakRef(key))

    def __contains__(self, key):
        return WeakRef(key) in self.data
    
    def __iter__(self):
        for k in self.data:
            yield k()

    # XXX Someone else can fill out the rest of the methods, with tests. :)
    




More information about the Zodb-checkins mailing list