[Checkins] SVN: zc.beforestorage/branches/dev/ Initial working (I
think) version.
Jim Fulton
jim at zope.com
Mon Jan 21 14:23:44 EST 2008
Log message for revision 83070:
Initial working (I think) version.
Changed:
U zc.beforestorage/branches/dev/buildout.cfg
U zc.beforestorage/branches/dev/setup.py
A zc.beforestorage/branches/dev/src/zc/beforestorage/
A zc.beforestorage/branches/dev/src/zc/beforestorage/README.txt
A zc.beforestorage/branches/dev/src/zc/beforestorage/__init__.py
A zc.beforestorage/branches/dev/src/zc/beforestorage/tests.py
-=-
Modified: zc.beforestorage/branches/dev/buildout.cfg
===================================================================
--- zc.beforestorage/branches/dev/buildout.cfg 2008-01-21 18:24:07 UTC (rev 83069)
+++ zc.beforestorage/branches/dev/buildout.cfg 2008-01-21 19:23:43 UTC (rev 83070)
@@ -1,7 +1,16 @@
[buildout]
develop = .
-parts = test
+parts = test py
+versions = versions
+[versions]
+ZODB3 = 3.8.0c1
+
+[py]
+recipe = zc.recipe.egg
+eggs = zc.beforestorage
+interpreter = py
+
[test]
recipe = zc.recipe.testrunner
-eggs =
+eggs = zc.beforestorage
Modified: zc.beforestorage/branches/dev/setup.py
===================================================================
--- zc.beforestorage/branches/dev/setup.py 2008-01-21 18:24:07 UTC (rev 83069)
+++ zc.beforestorage/branches/dev/setup.py 2008-01-21 19:23:43 UTC (rev 83070)
@@ -4,7 +4,7 @@
"""
setup(
- name = '',
+ name = 'zc.beforestorage',
version = '0.1',
author = 'Jim Fulton',
author_email = 'jim at zope.com',
@@ -14,7 +14,7 @@
packages = find_packages('src'),
namespace_packages = ['zc'],
package_dir = {'': 'src'},
- install_requires = 'setuptools',
+ install_requires = ['ZODB3', 'setuptools'],
zip_safe = False,
entry_points=entry_points,
)
Added: zc.beforestorage/branches/dev/src/zc/beforestorage/README.txt
===================================================================
--- zc.beforestorage/branches/dev/src/zc/beforestorage/README.txt (rev 0)
+++ zc.beforestorage/branches/dev/src/zc/beforestorage/README.txt 2008-01-21 19:23:43 UTC (rev 83070)
@@ -0,0 +1,225 @@
+Before Storage
+==============
+
+ZODB storages typically store multiple object revisions to support
+features such as multi-version concurrency control and undo. In the
+case of the mod popular storage implementation, old revisions aren't
+discarded until a pack. This feature has often been exploited to
+perform time travel, allowing one to look at a database as it existed
+in at some point in time. In the past, this has been possible with
+file storage by specifying a time at which to open the file
+storage. This works fairly well, but is very slow for large databases
+because existing index files can't easily be used. Time travel is
+also supported for individual objects through the ZODB history
+mechanism.
+
+The introduction of multi-version concurrency control provided new
+opertunities for time travel. Using the storage loadBefore method,
+one can load transaction records written before a given time. ZODB
+3.9 will provide an option to the database open method for opening
+connections as of a point in time.
+
+Demo storage can be quite useful for testing, and especially staging
+applications. In a common configuration, they allow for storing
+changes to a base database without changing the underlying database.
+Zope functional testing frameworks leverage demo storages to easily
+roll-back database state after a test to a non-empty state before a
+test. A significant limitation of demo storages is that they can't be
+used with base storages that change. This means that they generaly
+can't be used with ZEO. It isn't enough to have a read-only
+connecttions, if the underlying database is still being changed by
+other clients.
+
+The "before" storage provides another way to leverage the loadBefore
+method to support time travel and a means to provide an unchanging
+view into a ZEO server. A before storage is a database adapter that
+provides a read-only view of an underlying storage as of a particular
+point in time.
+
+To see how this works, we'll create a file storage, and use a before
+storage to provide views on it.
+
+ >>> import ZODB.FileStorage
+ >>> fs = ZODB.FileStorage.FileStorage('Data.fs')
+ >>> from ZODB.DB import DB
+ >>> db = DB(fs)
+ >>> conn = db.open()
+ >>> root = conn.root()
+ >>> import persistent.mapping
+
+We'll record transaction identifiers, which we'll use to when opening
+the before storage.
+
+ >>> import transaction
+ >>> transactions = [root._p_serial]
+ >>> for i in range(1, 11):
+ ... root[i] = persistent.mapping.PersistentMapping()
+ ... transaction.get().note("trans %s" % i)
+ ... transaction.commit()
+ ... transactions.append(root._p_serial)
+
+We create a before storage by calling the Before constructer
+with an existing storage and a timestamp:
+
+ >>> import zc.beforestorage
+ >>> b5 = zc.beforestorage.Before(fs, transactions[5])
+ >>> db5 = DB(b5)
+ >>> conn5 = db5.open()
+ >>> root5 = conn5.root()
+ >>> len(root5)
+ 4
+
+here we see the database as it was before the 5th transaction was
+committed. If we try to access a later object, we'll get a
+POSKeyError:
+
+ >>> conn5.get(root[5]._p_oid)
+ Traceback (most recent call last):
+ ...
+ POSKeyError: 0x05
+
+Similarly, while we can access earlier object revisions, we can't
+access revisions at the before time or later:
+
+ >>> _ = b5.loadSerial(root._p_oid, transactions[2])
+
+ >>> b5.loadSerial(root._p_oid, transactions[5])
+ Traceback (most recent call last):
+ ...
+ POSKeyError: 0x00
+
+ >>> conn5.get(root[5]._p_oid)
+ Traceback (most recent call last):
+ ...
+ POSKeyError: 0x05
+
+Let's run through the storage methods:
+
+ >>> b5.getName()
+ 'Data.fs before 2008-01-21 18:22:50.000000'
+
+ >>> b5.getSize() == fs.getSize()
+ True
+
+ >>> for hd in b5.history(root._p_oid, size=3):
+ ... print hd['description']
+ trans 4
+ trans 3
+ trans 2
+
+ >>> b5.isReadOnly()
+ True
+
+ >>> transactions[4] <= b5.lastTransaction() < transactions[5]
+ True
+
+ >>> len(b5) == len(fs)
+ True
+
+ >>> p, s1, s2 = b5.loadBefore(root._p_oid, transactions[5])
+ >>> p == fs.loadSerial(root._p_oid, transactions[4])
+ True
+ >>> s1 == transactions[4]
+ True
+ >>> s2 is None
+ True
+
+ >>> p, s1, s2 = b5.loadBefore(root._p_oid, transactions[4])
+ >>> p == fs.loadSerial(root._p_oid, transactions[3])
+ True
+ >>> s1 == transactions[3]
+ True
+ >>> s2 == transactions[4]
+ True
+
+ >>> b5.new_oid()
+ Traceback (most recent call last):
+ ...
+ ReadOnlyError
+
+ >>> from ZODB.TimeStamp import TimeStamp
+ >>> b5.pack(TimeStamp(transactions[3]).timeTime(), lambda p: [])
+ Traceback (most recent call last):
+ ...
+ ReadOnlyError
+
+ >>> b5.registerDB(db5)
+
+ >>> b5.sortKey() == fs.sortKey()
+ True
+
+ >>> b5.tpc_begin(transaction.get())
+ Traceback (most recent call last):
+ ...
+ ReadOnlyError
+
+ >>> b5.store(root._p_oid, transactions[4], b5.load(root._p_oid)[0], '',
+ ... transaction.get())
+ ... # doctest: +ELLIPSIS
+ Traceback (most recent call last):
+ ...
+ StorageTransactionError: ...
+
+ >>> b5.tpc_vote(transaction.get())
+ ... # doctest: +ELLIPSIS
+ Traceback (most recent call last):
+ ...
+ StorageTransactionError: ...
+
+ >>> b5.tpc_finish(transaction)
+ ... # doctest: +ELLIPSIS
+ Traceback (most recent call last):
+ ...
+ StorageTransactionError: ...
+
+ >>> b5.tpc_abort(transaction)
+
+Before storages don't support undo:
+
+ >>> b5.supportsUndo
+ Traceback (most recent call last):
+ ...
+ AttributeError: Before instance has no attribute 'supportsUndo'
+
+(Don't even ask about versions. :)
+
+Closing a before storage closes the underlying storage:
+
+ >>> b5.close()
+ >>> fs.load(root._p_oid, '')
+ Traceback (most recent call last):
+ ...
+ ValueError: I/O operation on closed file
+
+If we ommit a timestamp when creating a before storage, the current
+time will be used:
+
+ >>> fs = ZODB.FileStorage.FileStorage('Data.fs')
+ >>> from ZODB.DB import DB
+ >>> db = DB(fs)
+ >>> conn = db.open()
+ >>> root = conn.root()
+
+ >>> bnow = zc.beforestorage.Before(fs)
+ >>> dbnow = DB(bnow)
+ >>> connnow = dbnow.open()
+ >>> rootnow = connnow.root()
+
+ >>> for i in range(1, 11):
+ ... root[i] = persistent.mapping.PersistentMapping()
+ ... transaction.get().note("trans %s" % i)
+ ... transaction.commit()
+ ... transactions.append(root._p_serial)
+
+ >>> len(rootnow)
+ 10
+
+The timestamp may be passed directory, or as an ISO time. For
+example:
+
+ >>> b5 = zc.beforestorage.Before(fs, '2008-01-21T18:22:50')
+ >>> db5 = DB(b5)
+ >>> conn5 = db5.open()
+ >>> root5 = conn5.root()
+ >>> len(root5)
+ 4
Property changes on: zc.beforestorage/branches/dev/src/zc/beforestorage/README.txt
___________________________________________________________________
Name: svn:eol-style
+ native
Added: zc.beforestorage/branches/dev/src/zc/beforestorage/__init__.py
===================================================================
--- zc.beforestorage/branches/dev/src/zc/beforestorage/__init__.py (rev 0)
+++ zc.beforestorage/branches/dev/src/zc/beforestorage/__init__.py 2008-01-21 19:23:43 UTC (rev 83070)
@@ -0,0 +1,133 @@
+##############################################################################
+#
+# Copyright (c) Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+import time
+
+import ZODB.POSException
+import ZODB.TimeStamp
+import ZODB.utils
+
+class Before:
+
+ def __init__(self, storage, before=None):
+ if before is None:
+ t = time.time()
+ g = time.gmtime(t)
+ before = repr(ZODB.TimeStamp.TimeStamp(*(g[:5] + (g[5]+(t%1), ))))
+ else:
+ assert isinstance(before, basestring)
+ if len(before) > 8:
+ if 'T' in before:
+ d, t = before.split('T')
+ else:
+ d, t = before, ''
+
+ d = map(int, d.split('-'))
+ if t:
+ t = t.split(':')
+ assert len(t) <= 3
+ d += map(int, t[:2]) + map(float, t[2:3])
+ before = repr(ZODB.TimeStamp.TimeStamp(*d))
+
+ self.storage = storage
+ self.before = before
+
+ def close(self):
+ self.storage.close()
+
+ def getName(self):
+ return "%s before %s" % (self.storage.getName(),
+ ZODB.TimeStamp.TimeStamp(self.before),
+ )
+
+ def getSize(self):
+ return self.storage.getSize()
+
+ def history(self, oid, version='', size=1):
+ assert version == ''
+
+ # This is awkward. We don't know how much history to ask for.
+ # We'll have to keep trying until we heve enough or until there isn't
+ # any more to chose from. :(
+
+ s = size
+ while 1:
+ base_history = self.storage.history(oid, version='', size=s)
+ result = [d for d in base_history
+ if d['tid'] < self.before
+ ]
+ if ((len(base_history) < s)
+ or
+ (len(result) >= size)
+ ):
+ if len(result) > size:
+ result = result[:size]
+ return result
+ s *= 2
+
+ def isReadOnly(self):
+ return True
+
+ def lastTransaction(self):
+ return ZODB.utils.p64(ZODB.utils.u64(self.before)-1)
+
+ def __len__(self):
+ return len(self.storage)
+
+ def load(self, oid, version=''):
+ assert version == ''
+ result = self.storage.loadBefore(oid, self.before)
+ if result:
+ return result[:2]
+ raise ZODB.POSException.POSKeyError(oid)
+
+ def loadBefore(self, oid, tid):
+ if self.before < tid:
+ tid = self.before
+ p, s1, s2 = self.storage.loadBefore(oid, tid)
+ if (s2 is not None) and (s2 >= self.before):
+ s2 = None
+ return p, s1, s2
+
+ def loadSerial(self, oid, serial):
+ if serial >= self.before:
+ raise ZODB.POSException.POSKeyError(oid)
+ return self.storage.loadSerial(oid, serial)
+
+ def new_oid(self):
+ raise ZODB.POSException.ReadOnlyError()
+
+ def pack(self, pack_time, referencesf):
+ raise ZODB.POSException.ReadOnlyError()
+
+ def registerDB(self, db):
+ pass
+
+ def sortKey(self):
+ return self.storage.sortKey()
+
+ def store(self, oid, serial, data, version, transaction):
+ raise ZODB.POSException.StorageTransactionError(self, transaction)
+
+ def tpc_abort(self, transaction):
+ pass
+
+ def tpc_begin(self, transaction):
+ raise ZODB.POSException.ReadOnlyError()
+
+ def tpc_finish(self, transaction, func = lambda: None):
+ raise ZODB.POSException.StorageTransactionError(self, transaction)
+
+ def tpc_vote(self, transaction):
+ raise ZODB.POSException.StorageTransactionError(self, transaction)
Property changes on: zc.beforestorage/branches/dev/src/zc/beforestorage/__init__.py
___________________________________________________________________
Name: svn:keywords
+ Id
Name: svn:eol-style
+ native
Added: zc.beforestorage/branches/dev/src/zc/beforestorage/tests.py
===================================================================
--- zc.beforestorage/branches/dev/src/zc/beforestorage/tests.py (rev 0)
+++ zc.beforestorage/branches/dev/src/zc/beforestorage/tests.py 2008-01-21 19:23:43 UTC (rev 83070)
@@ -0,0 +1,45 @@
+##############################################################################
+#
+# Copyright (c) Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+import time, unittest
+from zope.testing import doctest
+import zope.testing.setupstack
+
+
+
+def setUp(test):
+ zope.testing.setupstack.setUpDirectory(test)
+ now = [time.mktime((2008, 1, 21, 13, 22, 42, 0, 0, 0))]
+ def timetime():
+ now[0] += 1
+ return now[0]
+
+ old_timetime = time.time
+ zope.testing.setupstack.register(
+ test,
+ lambda : setattr(time, 'time', old_timetime)
+ )
+ time.time = timetime
+
+
+def test_suite():
+ return unittest.TestSuite((
+ doctest.DocFileSuite(
+ 'README.txt',
+ setUp=setUp, tearDown=zope.testing.setupstack.tearDown,
+ ),
+ ))
+
+if __name__ == '__main__':
+ unittest.main(defaultTest='test_suite')
+
Property changes on: zc.beforestorage/branches/dev/src/zc/beforestorage/tests.py
___________________________________________________________________
Name: svn:keywords
+ Id
Name: svn:eol-style
+ native
More information about the Checkins
mailing list