[Checkins] SVN: z3c.dobbin/trunk/ Added dictionary-support;
restructured documentation and cleaned up code.
Malthe Borch
mborch at gmail.com
Sat Jun 21 19:17:28 EDT 2008
Log message for revision 87642:
Added dictionary-support; restructured documentation and cleaned up code.
Changed:
U z3c.dobbin/trunk/CHANGES.txt
U z3c.dobbin/trunk/docs/DEVELOPER.txt
U z3c.dobbin/trunk/src/z3c/dobbin/README.txt
U z3c.dobbin/trunk/src/z3c/dobbin/bootstrap.py
U z3c.dobbin/trunk/src/z3c/dobbin/collections.py
U z3c.dobbin/trunk/src/z3c/dobbin/mapper.py
U z3c.dobbin/trunk/src/z3c/dobbin/relations.py
-=-
Modified: z3c.dobbin/trunk/CHANGES.txt
===================================================================
--- z3c.dobbin/trunk/CHANGES.txt 2008-06-21 22:09:19 UTC (rev 87641)
+++ z3c.dobbin/trunk/CHANGES.txt 2008-06-21 23:17:27 UTC (rev 87642)
@@ -4,8 +4,17 @@
0.3 dev
-------
+- Refactoring of table bootstrapping; internal tables now using a
+ naming convention less likely to clash with existing tables.
+ [malthe]
+
+- Added support for ``schema.Dict`` (including polymorphic dictionary
+ relation).
+ [malthe]
+
- Implemented polymorphic relations for a subset of the basic types
(int, str, unicode, tuple and list).
+ [malthe]
0.2.9
-----
Modified: z3c.dobbin/trunk/docs/DEVELOPER.txt
===================================================================
--- z3c.dobbin/trunk/docs/DEVELOPER.txt 2008-06-21 22:09:19 UTC (rev 87641)
+++ z3c.dobbin/trunk/docs/DEVELOPER.txt 2008-06-21 23:17:27 UTC (rev 87642)
@@ -1,7 +1,8 @@
-Developer information
-=====================
+Developer documentation
+=======================
-This section details the object persistence model.
+This section details the object persistence model and the relations
+machinery.
Introduction
------------
@@ -40,3 +41,10 @@
Essentially, all polymorphic relations are many-to-many from a
database perspective.
+Collections
+-----------
+
+Dictionaries are keyed by (unicode) string. Soup objects may be used
+as keys in which case a string representation of the UUID is
+used. Dictionaries are polymorphic such that any kind of value may be
+assigned for an entry.
Modified: z3c.dobbin/trunk/src/z3c/dobbin/README.txt
===================================================================
--- z3c.dobbin/trunk/src/z3c/dobbin/README.txt 2008-06-21 22:09:19 UTC (rev 87641)
+++ z3c.dobbin/trunk/src/z3c/dobbin/README.txt 2008-06-21 23:17:27 UTC (rev 87642)
@@ -1,13 +1,21 @@
-Developer documentation
-=======================
+Developer walk-through
+======================
-Dobbin creates ORM mappers based on class specification. Columns are
-infered from interface schema fields and attributes, and a class may
-be provided as the mapper metatype.
+This section demonstrates the main functionality of the package using
+the doctest format.
-Interface mapping
------------------
+Mapping
+-------
+Dobbin creates SQLAlchemy ORM mappers from Python classes based on
+class specification (class or interface):
+
+ * Columns are infered from interface schema fields and attributes
+ * Specification is kept as dotted name in a special column
+
+Interface specification
+-----------------------
+
An mapper adapter is provided.
>>> from z3c.dobbin.mapper import getMapper
@@ -110,6 +118,9 @@
>>> session.execute(metadata.tables[encode(ICompactDisc)].select()).fetchall()
[(3, 2005)]
+Concrete class specification
+----------------------------
+
Now we'll create a mapper based on a concrete class. We'll let the
class implement the interface that describes the attributes we want to
store, but also provides a custom method.
@@ -183,11 +194,18 @@
Relations
---------
-Most people have a favourite record.
+Relations are columns that act as references to other objects.
+As an example, let's create an object holds a reference to some
+favorite item. We use ``zope.schema.Object`` to declare this
+reference; relations are polymorphic and we needn't declare the schema
+of the referenced object in advance.
+
>>> class IFavorite(interface.Interface):
- ... item = schema.Object(title=u"Item", schema=IVinyl)
+ ... item = schema.Object(title=u"Item", schema=interface.Interface)
+ >>> __builtin__.IFavorite = IFavorite
+
Let's make our Diana Ross record a favorite.
>>> favorite = create(IFavorite)
@@ -196,12 +214,14 @@
<Vinyl Diana Ross and The Supremes: Taking Care of Business (@ 45 RPM)>
>>> session.save(favorite)
+
+We'll commit the transaction and lookup the object by its UUID.
+
+ >>> transaction.commit()
+
+ >>> from z3c.dobbin.soup import lookup
+ >>> favorite = lookup(favorite.uuid)
-Get back the object.
-
- >>> favorite = session.query(IFavorite.__mapper__).filter_by(
- ... spec=IFavorite.__mapper__.__spec__)[0]
-
When we retrieve the related items, it's automatically reconstructed
to match the specification to which it was associated.
@@ -264,57 +284,14 @@
This behavior should work well in a request-response type environment,
where the request will typically end with a commit.
-Polymorphic relations
----------------------
-
-We can create relations to instances as well as immutable objects
-(rocks).
-
-Integers, floats and unicode strings are straight-forward.
-
- >>> favorite.item = 42; transaction.commit()
- >>> favorite.item
- 42
-
- >>> favorite.item = 42.01; transaction.commit()
- >>> 42 < favorite.item <= 42.01
- True
-
- >>> favorite.item = u"My favorite number is 42."; transaction.commit()
- >>> favorite.item
- u'My favorite number is 42.'
-
-Normal strings need explicit coercing to ``str``.
-
- >>> favorite.item = "My favorite number is 42."; transaction.commit()
- >>> str(favorite.item)
- 'My favorite number is 42.'
-
-Or sequences of relations.
-
- >>> favorite.item = (u"green", u"blue", u"red"); transaction.commit()
- >>> favorite.item
- (u'green', u'blue', u'red')
-
-When we create relations to mutable objects, a hook is made into the
-transaction machinery to keep track of the pending state.
-
- >>> some_list = [u"green", u"blue", u"red"]; transaction.commit()
- >>> favorite.item = some_list
- >>> favorite.item
- [u'green', u'blue', u'red']
-
-Amorphic relations.
-
- >>> favorite.item = ((1, u"green"), (2, u"blue"), (3, u"red")); transaction.commit()
- >>> favorite.item
- ((1, u'green'), (2, u'blue'), (3, u'red'))
-
Collections
-----------
-Let's set up a record collection as a list.
+We can instrument properties that behave like collections by using the
+sequence and mapping schema fields.
+Let's set up a record collection as an ordered list.
+
>>> class ICollection(interface.Interface):
... records = schema.List(
... title=u"Records",
@@ -334,7 +311,6 @@
We can get our collection back.
- >>> from z3c.dobbin.soup import lookup
>>> collection = lookup(collection.uuid)
Let's verify that we've stored the Diana Ross record.
@@ -399,23 +375,142 @@
>>> empty_collection = create(ICollection)
>>> session.save(empty_collection)
+
+To demonstrate the mapping implementation, let's set up a catalog for
+our record collection. We'll index the records by their ASIN string.
-Let's index the collection by artist in a catalog.
+ >>> class ICatalog(interface.Interface):
+ ... index = schema.Dict(
+ ... title=u"Record index")
- >>> class ICatalog(interface.Interface):
- ... records_by_artist = schema.Dict(
- ... title=u"Records by artist",
+ >>> catalog = create(ICatalog)
+ >>> session.save(catalog)
+
+Add a record to the index.
+
+ >>> catalog.index[u"B00004WZ5Z"] = diana
+ >>> catalog.index[u"B00004WZ5Z"]
+ <Mapper (__builtin__.IVinyl) at ...>
+
+Verify state after commit.
+
+ >>> transaction.commit()
+ >>> catalog.index[u"B00004WZ5Z"]
+ <Mapper (__builtin__.IVinyl) at ...>
+
+Let's check that the standard dict methods are supported.
+
+ >>> catalog.index.values()
+ [<Mapper (__builtin__.IVinyl) at ...>]
+
+ >>> tuple(catalog.index.itervalues())
+ (<Mapper (__builtin__.IVinyl) at ...>,)
+
+ >>> catalog.index.setdefault(u"B00004WZ5Z", None)
+ <Mapper (__builtin__.IVinyl) at ...>
+
+ >>> catalog.index.pop(u"B00004WZ5Z")
+ <Mapper (__builtin__.IVinyl) at ...>
+
+ >>> len(catalog.index)
+ 0
+
+Concrete instances are supported.
+
+ >>> vinyl = Vinyl()
+ >>> vinyl.artist = diana.artist
+ >>> vinyl.title = diana.title
+ >>> vinyl.rpm = diana.rpm
+
+ >>> catalog.index[u"B00004WZ5Z"] = vinyl
+ >>> len(catalog.index)
+ 1
+
+ >>> catalog.index.popitem()
+ (u'B00004WZ5Z',
+ <Vinyl Diana Ross and The Supremes: Taking Care of Business (@ 45 RPM)>)
+
+ >>> catalog.index = {u"B00004WZ5Z": vinyl}
+ >>> len(catalog.index)
+ 1
+
+ >>> catalog.index.clear()
+ >>> len(catalog.index)
+ 0
+
+We may use a mapped object as index.
+
+ >>> catalog.index[diana] = diana
+ >>> catalog.index.keys()[0] == diana.uuid
+ True
+
+ >>> transaction.commit()
+
+ >>> catalog.index[diana]
+ <Mapper (__builtin__.IVinyl) at ...>
+
+ >>> class IDiscography(ICatalog):
+ ... records = schema.Dict(
+ ... title=u"Discographies by artist",
... value_type=schema.List())
- ...
- ... artist_biographies = schema.Dict(
- ... title=u"Artist biographies",
- ... value_type=schema.Text())
- >> catalog = create(ICatalog)
- >> session.add(catalog)
+Polymorphic relations
+---------------------
- >> session.records_by_artist[diana.artist] =
+We can create relations to instances as well as immutable objects
+(rocks).
+
+Integers, floats and unicode strings are straight-forward.
+
+ >>> favorite.item = 42; transaction.commit()
+ >>> favorite.item
+ 42
+
+ >>> favorite.item = 42.01; transaction.commit()
+ >>> 42 < favorite.item <= 42.01
+ True
+
+ >>> favorite.item = u"My favorite number is 42."; transaction.commit()
+ >>> favorite.item
+ u'My favorite number is 42.'
+
+Normal strings need explicit coercing to ``str``.
+ >>> favorite.item = "My favorite number is 42."; transaction.commit()
+ >>> str(favorite.item)
+ 'My favorite number is 42.'
+
+Or sequences of relations.
+
+ >>> favorite.item = (u"green", u"blue", u"red"); transaction.commit()
+ >>> favorite.item
+ (u'green', u'blue', u'red')
+
+Dictionaries.
+
+ >>> favorite.item = {u"green": 0x00FF00, u"blue": 0x0000FF, u"red": 0xFF0000}
+ >>> transaction.commit()
+ >>> favorite.item
+ {u'blue': 255, u'green': 65280, u'red': 16711680}
+
+ >>> favorite.item[u"black"] = 0x000000
+ >>> favorite.item
+ {u'blue': 255, u'green': 65280, u'black': 0, u'red': 16711680}
+
+When we create relations to mutable objects, a hook is made into the
+transaction machinery to keep track of the pending state.
+
+ >>> some_list = [u"green", u"blue", u"red"]; transaction.commit()
+ >>> favorite.item = some_list
+ >>> favorite.item
+ [u'green', u'blue', u'red']
+
+Amorphic relations.
+
+ >>> favorite.item = ((1, u"green"), (2, u"blue"), (3, u"red")); transaction.commit()
+ >>> favorite.item
+ ((1, u'green'), (2, u'blue'), (3, u'red'))
+
Security
--------
Modified: z3c.dobbin/trunk/src/z3c/dobbin/bootstrap.py
===================================================================
--- z3c.dobbin/trunk/src/z3c/dobbin/bootstrap.py 2008-06-21 22:09:19 UTC (rev 87641)
+++ z3c.dobbin/trunk/src/z3c/dobbin/bootstrap.py 2008-06-21 23:17:27 UTC (rev 87642)
@@ -13,8 +13,13 @@
setUp(metadata)
def setUp(metadata):
+ """Table setup.
+
+ This method sets up the tables that are necessary for the
+ operation of the persistence and relational framework.
+ """
+
soup_uuid = rdb.String(length=32)
- soup_fk = rdb.ForeignKey("dobbin:soup.uuid")
soup = rdb.Table(
'dobbin:soup',
@@ -24,6 +29,8 @@
rdb.Column('spec', rdb.String, index=True),
)
+ soup_fk = rdb.ForeignKey(soup.c.uuid)
+
int_relation = rdb.Table(
'dobbin:relation:int',
metadata,
@@ -32,18 +39,24 @@
rdb.Column('right', soup_uuid, soup_fk),
rdb.Column('order', rdb.Integer, nullable=False))
- catalog = rdb.Table(
- 'catalog',
+ str_relation = rdb.Table(
+ 'dobbin:relation:str',
metadata,
rdb.Column('id', rdb.Integer, primary_key=True, autoincrement=True),
rdb.Column('left', soup_uuid, soup_fk, index=True),
rdb.Column('right', soup_uuid, soup_fk),
- rdb.Column('name', rdb.String))
-
+ rdb.Column('key', rdb.Unicode, nullable=False))
+
+ # set up mappers
+ orm.mapper(Soup, soup)
orm.mapper(relations.OrderedRelation, int_relation)
- orm.mapper(Soup, soup)
-
+ orm.mapper(relations.KeyRelation, str_relation)
+
+ # create all tables
metadata.create_all()
class Soup(object):
- pass
+ """Soup class.
+
+ This stub is used as the mapper for the soup table.
+ """
Modified: z3c.dobbin/trunk/src/z3c/dobbin/collections.py
===================================================================
--- z3c.dobbin/trunk/src/z3c/dobbin/collections.py 2008-06-21 22:09:19 UTC (rev 87641)
+++ z3c.dobbin/trunk/src/z3c/dobbin/collections.py 2008-06-21 23:17:27 UTC (rev 87642)
@@ -5,6 +5,7 @@
import interfaces
import relations
import soup
+import types
class Tuple(object):
def __init__(self):
@@ -12,7 +13,7 @@
@property
def adapter(self):
- return self._sa_adapter
+ return orm.collections.collection_adapter(self)
@orm.collections.collection.appender
def _appender(self, item):
@@ -44,13 +45,18 @@
return converted
def __iter__(self):
- for relation in iter(self.data):
- obj = relation.target
- if interfaces.IBasicType.providedBy(obj):
- yield obj.value
- else:
- yield obj
+ return (self[i] for i in range(len(self.data)))
+ def __getitem__(self, index):
+ obj = self.data[index].target
+ if interfaces.IBasicType.providedBy(obj):
+ return obj.value
+ else:
+ return obj
+
+ def __setitem__(self, index, value):
+ return NotImplementedError("Object does not support item assignment.")
+
def __len__(self):
return len(self.data)
@@ -106,12 +112,126 @@
def extend(self, items):
map(self.append, items)
+ def count(self, value):
+ return NotImplementedError("Count-method not implemented.")
+
def __repr__(self):
return repr(list(self))
-
- def __getitem__(self, index):
- return self.data[index].target
-
+
def __setitem__(self, index, value):
return NotImplementedError("Setting items at an index is not implemented.")
+class Dict(dict):
+ __Security_checker__ = NamesChecker(
+ ('clear', 'copy', 'fromkeys', 'get', 'has_key', 'items', 'iteritems', 'iterkeys', 'itervalues', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values'))
+
+ @property
+ def adapter(self):
+ return orm.collections.collection_adapter(self)
+
+ @orm.collections.collection.appender
+ @orm.collections.collection.replaces(1)
+ def _appender(self, item):
+ dict.__setitem__(self, item.key, item)
+
+ @orm.collections.collection.iterator
+ def _iterator(self):
+ return dict.itervalues(self)
+
+ @orm.collections.collection.remover
+ def _remover(self, item):
+ dict.remove(item)
+
+ @orm.collections.collection.internally_instrumented
+ def __setitem__(self, key, item, _sa_initiator=None):
+ if not interfaces.IMapped.providedBy(item):
+ item = soup.persist(item)
+
+ # mapped objects may be used as key; internally, we'll use
+ # the UUID in this case, however.
+ if interfaces.IMapped.providedBy(key):
+ key = key.uuid
+
+ assert isinstance(key, types.StringTypes), \
+ "Only strings or mapped objects may be used as keys."
+
+ # set up relation
+ relation = relations.KeyRelation()
+ relation.target = item
+ relation.key = key
+
+ self.adapter.fire_append_event(relation, _sa_initiator)
+ dict.__setitem__(self, key, relation)
+
+ @orm.collections.collection.converter
+ def convert(self, d):
+ converted = []
+
+ for key, item in d.items():
+ if not interfaces.IMapped.providedBy(item):
+ item = soup.persist(item)
+
+ # set up relation
+ relation = relations.KeyRelation()
+ relation.target = item
+ relation.key = key
+
+ converted.append(relation)
+
+ return converted
+
+ def values(self):
+ return [self[key] for key in self]
+
+ def itervalues(self):
+ return (self[key] for key in self)
+
+ @orm.collections.collection.internally_instrumented
+ def pop(self, key, _sa_initiator=None):
+ relation = dict.pop(self, key)
+ obj = relation.target
+
+ self.adapter.fire_remove_event(relation, _sa_initiator)
+
+ if interfaces.IBasicType.providedBy(obj):
+ return obj.value
+ else:
+ return obj
+
+ @orm.collections.collection.internally_instrumented
+ def popitem(self, _sa_initiator=None):
+ key, relation = dict.popitem(self)
+ obj = relation.target
+
+ self.adapter.fire_remove_event(relation, _sa_initiator)
+
+ if interfaces.IBasicType.providedBy(obj):
+ return key, obj.value
+ else:
+ return key, obj
+
+ @orm.collections.collection.internally_instrumented
+ def clear(self, _sa_initiator=None):
+ for relation in dict.itervalues(self):
+ self.adapter.fire_remove_event(relation, _sa_initiator)
+
+ dict.clear(self)
+
+ def __getitem__(self, key):
+ # mapped objects may be used as key; internally, we'll use
+ # the UUID in this case, however.
+ if interfaces.IMapped.providedBy(key):
+ key = key.uuid
+
+ assert isinstance(key, types.StringTypes), \
+ "Only strings or mapped objects may be used as keys."
+
+ obj = dict.__getitem__(self, key).target
+ if interfaces.IBasicType.providedBy(obj):
+ return obj.value
+ else:
+ return obj
+
+ def __repr__(self):
+ return repr(dict(
+ (key, self[key]) for key in self))
Modified: z3c.dobbin/trunk/src/z3c/dobbin/mapper.py
===================================================================
--- z3c.dobbin/trunk/src/z3c/dobbin/mapper.py 2008-06-21 22:09:19 UTC (rev 87641)
+++ z3c.dobbin/trunk/src/z3c/dobbin/mapper.py 2008-06-21 23:17:27 UTC (rev 87642)
@@ -55,7 +55,7 @@
We're not checking type here, because we'll only be creating
relations to items that are joined with the soup.
"""
-
+
def __call__(self, field, column, metadata):
relation = relations.RelationProperty(field)
@@ -63,7 +63,7 @@
field.__name__: relation,
relation.name: orm.relation(
bootstrap.Soup,
- primaryjoin=(field.schema.__mapper__.uuid==column),
+ primaryjoin=bootstrap.Soup.c.uuid==column,
foreign_keys=[column],
enable_typechecks=False,
lazy=True)
@@ -73,65 +73,37 @@
"""A collection property."""
collection_class = None
+ relation_class = None
def __call__(self, field, column, metadata):
- relation_table = self.getRelationTable(metadata)
- soup_table = self.getSoupTable(metadata)
-
return {
field.__name__: orm.relation(
- relations.OrderedRelation,
- primaryjoin=soup_table.c.uuid==relation_table.c.left,
+ self.relation_class,
+ primaryjoin=self.getPrimaryJoinCondition(),
collection_class=self.collection_class,
enable_typechecks=False)
}
- def getRelationTable(self, metadata):
+ def getPrimaryJoinCondition(self):
return NotImplementedError("Must be implemented by subclass.")
-
- def getSoupTable(self, metadata):
- return metadata.tables['dobbin:soup']
-
+
class ListProperty(CollectionProperty):
collection_class = collections.OrderedList
+ relation_class = relations.OrderedRelation
- def getRelationTable(self, metadata):
- return metadata.tables['dobbin:relation:int']
-
+ def getPrimaryJoinCondition(self):
+ return bootstrap.Soup.c.uuid==relations.OrderedRelation.c.left
+
class TupleProperty(ListProperty):
collection_class = collections.Tuple
-class DictProperty(object):
- """A dict property.
+class DictProperty(CollectionProperty):
+ collection_class = collections.Dict
+ relation_class = relations.KeyRelation
- In SQLAlchemy, we need to model the following two defintion types:
-
- schema.Dict(
- value_type=schema.Object(
- schema=ISomeInterface)
- )
-
- schema.Dict(
- value_type=schema.Set(
- value_schema.Object(
- schema=ISomeInterface)
- )
- )
-
- Reference:
-
- http://blog.discorporate.us/2008/02/sqlalchemy-partitioned-collections-1
- http://www.sqlalchemy.org/docs/04/mappers.html#advdatamapping_relation_collections
-
- """
-
- def __call__(self, field, column, metadata):
- #
- #
- # TODO: Return column definition
-
- pass
-
+ def getPrimaryJoinCondition(self):
+ return bootstrap.Soup.c.uuid==relations.KeyRelation.c.left
+
fieldmap = {
schema.ASCII: StringTranslator(),
schema.ASCIILine: StringTranslator(),
@@ -140,7 +112,7 @@
schema.BytesLine: FieldTranslator(rdb.CLOB),
schema.Choice: StringTranslator(rdb.Unicode),
schema.Date: FieldTranslator(rdb.DATE),
- schema.Dict: (ObjectTranslator(), DictProperty()),
+ schema.Dict: (None, DictProperty()),
schema.DottedName: StringTranslator(),
schema.Float: FieldTranslator(rdb.Float),
schema.Id: StringTranslator(rdb.Unicode),
@@ -204,7 +176,6 @@
exclude.append(name)
# create joined table
- soup_table = table = metadata.tables['dobbin:soup']
properties = {}
first_table = None
@@ -254,7 +225,7 @@
del properties[name]
setattr(Mapper, name, prop)
-
+ soup_table = bootstrap.Soup.c.id.table
polymorphic = (
[Mapper], table.join(
soup_table, first_table.c.id==soup_table.c.id))
@@ -318,7 +289,7 @@
table = rdb.Table(
name,
metadata,
- rdb.Column('id', rdb.Integer, rdb.ForeignKey("dobbin:soup.id"), primary_key=True),
+ rdb.Column('id', rdb.Integer, rdb.ForeignKey(bootstrap.Soup.c.id), primary_key=True),
*columns,
**kw)
Modified: z3c.dobbin/trunk/src/z3c/dobbin/relations.py
===================================================================
--- z3c.dobbin/trunk/src/z3c/dobbin/relations.py 2008-06-21 22:09:19 UTC (rev 87641)
+++ z3c.dobbin/trunk/src/z3c/dobbin/relations.py 2008-06-21 23:17:27 UTC (rev 87642)
@@ -31,6 +31,9 @@
class OrderedRelation(Relation):
pass
+
+class KeyRelation(Relation):
+ pass
class RelationProperty(property):
More information about the Checkins
mailing list