[Checkins] SVN: grokapps/rdbexample/trunk/megrok.rdb/ A lot of
work. This will totally break rdbexample, but it was broken anyway
Martijn Faassen
faassen at infrae.com
Tue Jun 24 19:51:10 EDT 2008
Log message for revision 87734:
A lot of work. This will totally break rdbexample, but it was broken anyway
and now we actually have a test. What has happened:
* a doctest in README.txt
* use SQLAlchemy 0.5beta1; this has some declarative functionality we
need.
* use newest zope.sqlalchemy and z3c.saconfig which we'll use to set up the
database. (instead of collective.lead)
* use Grok 0.13, which has the Context and Container classes we need.
A few new directives are introduced.
A lot of work remains to be done, especially in the writing of more
tests. more rdb.key situations need to be explored, a lot of the directives
need to be firmed up (better default behavior, more validators, lots more
tests), but at least this is progress.
Changed:
_U grokapps/rdbexample/trunk/megrok.rdb/
A grokapps/rdbexample/trunk/megrok.rdb/buildout.cfg
U grokapps/rdbexample/trunk/megrok.rdb/setup.py
A grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/README.txt
U grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/__init__.py
U grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/components.py
U grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/db.py
U grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/directive.py
U grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/interfaces.py
U grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/meta.py
U grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/schema.py
A grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/tests.py
-=-
Property changes on: grokapps/rdbexample/trunk/megrok.rdb
___________________________________________________________________
Name: svn:ignore
+ bin
parts
.installed.cfg
develop-eggs
Name: svn:externals
+ z3c.saconfig svn://svn.zope.org/repos/main/z3c.saconfig/trunk
zope.sqlalchemy svn://svn.zope.org/repos/main/zope.sqlalchemy/trunk
Added: grokapps/rdbexample/trunk/megrok.rdb/buildout.cfg
===================================================================
--- grokapps/rdbexample/trunk/megrok.rdb/buildout.cfg (rev 0)
+++ grokapps/rdbexample/trunk/megrok.rdb/buildout.cfg 2008-06-24 23:51:10 UTC (rev 87734)
@@ -0,0 +1,56 @@
+[buildout]
+develop = . zope.sqlalchemy z3c.saconfig
+parts = app data zopectl test
+newest = false
+extends = http://grok.zope.org/releaseinfo/grok-0.13.cfg
+versions = versions
+
+[app]
+recipe = zc.zope3recipes>=0.5.3:application
+eggs = megrok.rdb
+site.zcml = <include package="rdbexample" />
+ <include package="zope.app.twisted" />
+
+ <configure i18n_domain="rdbexample">
+ <unauthenticatedPrincipal id="zope.anybody"
+ title="Unauthenticated User" />
+ <unauthenticatedGroup id="zope.Anybody"
+ title="Unauthenticated Users" />
+ <authenticatedGroup id="zope.Authenticated"
+ title="Authenticated Users" />
+ <everybodyGroup id="zope.Everybody"
+ title="All Users" />
+ <principal id="zope.manager"
+ title="Manager"
+ login="admin"
+ password_manager="Plain Text"
+ password="admin"
+ />
+
+ <!-- Replace the following directive if you don't want
+ public access -->
+ <grant permission="zope.View"
+ principal="zope.Anybody" />
+ <grant permission="zope.app.dublincore.view"
+ principal="zope.Anybody" />
+
+ <role id="zope.Manager" title="Site Manager" />
+ <role id="zope.Member" title="Site Member" />
+ <grantAll role="zope.Manager" />
+ <grant role="zope.Manager"
+ principal="zope.manager" />
+ </configure>
+
+[data]
+recipe = zc.recipe.filestorage
+
+# this section named so that the start/stop script is called bin/zopectl
+[zopectl]
+recipe = zc.zope3recipes:instance
+application = app
+zope.conf = ${data:zconfig}
+
+[test]
+recipe = zc.recipe.testrunner
+eggs = megrok.rdb
+defaults = ['--tests-pattern', '^f?tests$', '-v']
Modified: grokapps/rdbexample/trunk/megrok.rdb/setup.py
===================================================================
--- grokapps/rdbexample/trunk/megrok.rdb/setup.py 2008-06-24 23:46:08 UTC (rev 87733)
+++ grokapps/rdbexample/trunk/megrok.rdb/setup.py 2008-06-24 23:51:10 UTC (rev 87734)
@@ -24,9 +24,11 @@
zip_safe=True,
install_requires=[
'setuptools',
- 'SQLAlchemy',
- 'collective.lead', # needs elro-tcp branch for now
- ],
+ 'grok >= 0.13',
+ 'SQLAlchemy == 0.5beta1',
+ 'zope.sqlalchemy',
+ 'z3c.saconfig',
+ ],
entry_points="""
# -*- Entry points: -*-
""",
Added: grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/README.txt
===================================================================
--- grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/README.txt (rev 0)
+++ grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/README.txt 2008-06-24 23:51:10 UTC (rev 87734)
@@ -0,0 +1,206 @@
+==========
+megrok.rdb
+==========
+
+The ``megrok.rdb`` package adds powerful relational database support
+to Grok, based on the powerful SQLAlchemy_ library. It makes available
+a new ``megrok.rdb.Model`` and ``megrok.rdb.Container`` which behave
+much like ones in core Grok, but are instead backed by a relational
+database.
+
+.. _SQLAlchemy: http://www.sqlalchemy.org
+
+In this document we will show you how to use ``megrok.rdb``.
+
+``megrok.rdb`` uses SQLAlchemy's ORM system, in particular its
+declarative extension almost directly. ``megrok.rdb`` just supplies a
+few special base classes and directives to make things easier, and a few
+other conveniences that help with integration with Grok.
+
+We first import import the SQLAlchemy bits we'll need later::
+
+ >>> from sqlalchemy import Column, ForeignKey
+ >>> from sqlalchemy.types import Integer, String
+ >>> from sqlalchemy.orm import relation
+
+SQLAlchemy groups database schema information into a unit called
+``MetaData``. The schema can be reflected from the database schema, or
+can be created from a schema defined in Python. With ``megrok.rdb`` we
+typically do the latter, from within the content classes that they are
+mapped to using the ORM. We need to have some metadata to associate
+our content classes with.
+
+Let's set up the metadata object::
+
+ >>> from megrok import rdb
+ >>> metadata = rdb.MetaData()
+
+Now we'll set up a few content classes. We'll have a very simple
+structure where a (university) department has zero or more courses
+associated with it. First we'll define a container that can contain
+courses::
+
+ >>> class Courses(rdb.Container):
+ ... rdb.key('name')
+
+As you can see, we need to set up the attribute on which the key is
+based. We will use the ``name`` attribute, which we will define for
+the course later. Note that using the primary key attribute (such as
+``id`` in this example) is not a good idea if you expect the database
+to generate unique ids itself - the correct id will not be known yet
+before the object is commit to the database.
+
+Now we can set up the ``Department`` class. This has the ``courses``
+relation that links to its courses::
+
+ >>> class Department(rdb.Model):
+ ... rdb.metadata(metadata)
+ ...
+ ... id = Column('id', Integer, primary_key=True)
+ ... name = Column('name', String(50))
+ ...
+ ... courses = relation('Course',
+ ... backref='department',
+ ... collection_class=Courses)
+
+This is very similar to the way you'd use
+``sqlalchemy.ext.declarative``, but there are a few differences::
+
+* we inherit from ``rdb.Model`` to make this behave like a Grok model.
+
+* We don't need to use ``__tablename__`` to set up the table name. By
+ default the table name will be the class name, lowercased, but you
+ can override this by using the ``rdb.tablename`` directive.
+
+* we need to make explicit the metadata object that is used. We do
+ this in the tests, though in Grok applications it's enough to use
+ the ``rdb.metadata`` directive on a module-level to have all rdb
+ classes automatically associated with that metadata object.
+
+* we mark that the ``courses`` relation uses the ``Courses`` container
+ class we have defined before. This is a normal SQLAlchemy feature,
+ it's just we have to use it if we want to use Grok-style containers.
+
+We finish up our database definition by defining the ``Course``
+class::
+
+ >>> class Course(rdb.Model):
+ ... rdb.metadata(metadata)
+ ...
+ ... id = Column('id', Integer, primary_key=True)
+ ... department_id = Column('department_id', Integer,
+ ... ForeignKey('department.id'))
+ ... name = Column('name', String(50))
+
+We see here that ``Course`` links back to the department it is in,
+using a foreign key.
+
+We need to actually grok these objects to have them fully set
+up Normally grok takes care of this automatically, but in this case
+we'll need to do it manually.
+
+First we grok this package's grokkers::
+
+ >>> from grok.testing import grok
+ >>> grok('megrok.rdb.meta')
+
+Now we can grok the components::
+
+ >>> from grok.testing import grok_component
+ >>> grok_component('Courses', Courses)
+ True
+ >>> grok_component('Department', Department)
+ True
+ >>> grok_component('Course', Course)
+ True
+
+Once we have our metadata and object relational map defined, we need
+to have a database to actually put these in. While it is possible to
+set up a different database per Grok application, here we will use a
+single global database::
+
+ >>> TEST_DSN = 'sqlite:///:memory:'
+ >>> from z3c.saconfig import EngineFactory
+ >>> from z3c.saconfig.interfaces import IEngineFactory
+ >>> engine_factory = EngineFactory(TEST_DSN)
+
+We need to supply the engine factory as a utility. Grok can do this
+automatically for you using the module-level ``grok.global_utility``
+directive, like this::
+
+ grok.global_utility(engine_factory, provides=IEngineFactory, direct=True)
+
+In the tests we'll use the component architecture directly::
+
+ >>> from zope import component
+ >>> component.provideUtility(engine_factory, provides=IEngineFactory)
+
+Now that we've set up an engine, we can set up the SQLAlchemy session
+utility::
+
+ >>> from z3c.saconfig import GloballyScopedSession
+ >>> from z3c.saconfig.interfaces import IScopedSession
+ >>> scoped_session = GloballyScopedSession()
+
+With Grok, we'd register it like this::
+
+ grok.global_utility(scoped_session, provides=IScopedSession, direct=True)
+
+But again we'll just register it directly for the tests::
+
+ >>> component.provideUtility(scoped_session, provides=IScopedSession)
+
+We now need to create the tables we defined in our database::
+
+ >>> engine = engine_factory()
+ >>> metadata.create_all(engine)
+
+Now all that is out the way, we can use the ``rdb.Session`` object to make
+a connection to the database.
+
+ >>> session = rdb.Session()
+
+To make the next bit work with doctests, we need an utter hack::
+
+ >>> Courses.__module__ = 'foo'
+
+This is only because ``Courses`` is defined in a doctest. Because of
+this, the ``__module__`` attribute of of ``Courses`` will be set to
+``__builtin__``, and SQLAlchemy then concludes that ``Courses`` is
+really a builtin Python object and refuses to instrument it
+propertly. By changing ``__module__`` to something else, we avoid this
+problem.
+
+Let's now create a database structure. We have a department of philosophy::
+
+ >>> philosophy = Department(name="Philosophy")
+
+We need to manually add it to the database, as we haven't defined a
+particular ``departments`` container in our database::
+
+ >>> session.add(philosophy)
+
+The philosophy department has a number of courses::
+
+ >>> logic = Course(name="Logic")
+ >>> ethics = Course(name="Ethics")
+ >>> metaphysics = Course(name="Metaphysics")
+ >>> session.add_all([logic, ethics, metaphysics])
+
+We'll add them to the philosophy department's courses container. Since
+we want to leave it up to the database what the key will be, we will
+use the special ``set`` method that ``rdb.Container`` objects have to
+add the objects::
+
+ >>> philosophy.courses.set(logic)
+ >>> philosophy.courses.set(ethics)
+ >>> philosophy.courses.set(metaphysics)
+
+We can now verify that the courses are there::
+
+ >>> for key, value in sorted(philosophy.courses.items()):
+ ... print key, value.name, value.department.name
+ Ethics Ethics Philosophy
+ Logic Logic Philosophy
+ Metaphysics Metaphysics Philosophy
+
Modified: grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/__init__.py
===================================================================
--- grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/__init__.py 2008-06-24 23:46:08 UTC (rev 87733)
+++ grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/__init__.py 2008-06-24 23:51:10 UTC (rev 87734)
@@ -1,15 +1,7 @@
-import zope.component
-
from megrok.rdb.components import Model, Container
from megrok.rdb.schema import Fields
-from megrok.rdb.db import Database, session
+from megrok.rdb.directive import key, metadata, tablename
-from megrok.rdb.directive import key
+from sqlalchemy import MetaData
-import collective.lead.interfaces
-
-
-def query(class_):
- database = zope.component.getUtility(
- collective.lead.interfaces.IDatabase, name='megrok.rdb')
- return database.session.query(class_)
+from z3c.saconfig import Session
Modified: grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/components.py
===================================================================
--- grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/components.py 2008-06-24 23:46:08 UTC (rev 87733)
+++ grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/components.py 2008-06-24 23:51:10 UTC (rev 87734)
@@ -1,6 +1,4 @@
-from sqlalchemy.ext.declarative import DeclarativeMeta
from sqlalchemy.orm.collections import MappedCollection
-from sqlalchemy.schema import MetaData
from zope.interface import implements
@@ -8,15 +6,11 @@
from grok.interfaces import IContainer
from megrok.rdb import directive
+from z3c.saconfig import Session
-_lcl_metadata = MetaData()
-
class Model(Context):
- __metaclass__ = DeclarativeMeta
- metadata = _lcl_metadata
- _decl_class_registry = {}
-
def __init__(self, **kwargs):
+ # XXX can we use the __init__ that sqlalchemy.ext.declarative sets up?
for k in kwargs:
if not hasattr(type(self), k):
raise TypeError('%r is an invalid keyword argument for %s' %
@@ -28,13 +22,14 @@
if len(primary_keys) == 1:
return getattr(node, primary_keys[0])
else:
- raise RuntimeError("don't know how to do keying with composite primary keys")
+ raise RuntimeError(
+ "don't know how to do keying with composite primary keys")
class Container(MappedCollection):
implements(IContainer)
def __init__(self, *args, **kw):
- rdb_key = directive.key.get(self)
+ rdb_key = directive.key.bind().get(self)
if rdb_key:
keyfunc = lambda node:getattr(node, rdb_key)
elif hasattr(self, 'keyfunc'):
Modified: grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/db.py
===================================================================
--- grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/db.py 2008-06-24 23:46:08 UTC (rev 87733)
+++ grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/db.py 2008-06-24 23:51:10 UTC (rev 87734)
@@ -1,33 +1,25 @@
-import grok
+## import grok
-from zope import component
+## from zope import component
-from collective.lead import Database as DatabaseBase
-from collective.lead.interfaces import IDatabase
+## from megrok.rdb.components import Model
-from megrok.rdb.components import Model
-from megrok.rdb.interfaces import IDatabase as IRdbDatabase
+## class Database(grok.GlobalUtility, DatabaseBase):
+## grok.implements(IRdbDatabase)
+## grok.provides(IDatabase)
+## grok.name('megrok.rdb')
+## grok.baseclass()
+## @property
+## def _url(self):
+## # XXXX missing 'url' gets turned into an AttributeError for `_url`
+## # instead of `url`, which sucks.
+## return self.url
-class Database(grok.GlobalUtility, DatabaseBase):
- grok.implements(IRdbDatabase)
- grok.provides(IDatabase)
- grok.name('megrok.rdb')
- grok.baseclass()
+## def _setup_tables(self, metadata, tables):
+## self.metadata = metadata = Model.metadata
+## Model.metadata.create_all(self._engine)
+## self.setup(metadata)
- @property
- def _url(self):
- # XXXX missing 'url' gets turned into an AttributeError for `_url`
- # instead of `url`, which sucks.
- return self.url
-
- def _setup_tables(self, metadata, tables):
- self.metadata = metadata = Model.metadata
- Model.metadata.create_all(self._engine)
- self.setup(metadata)
-
- def setup(self, metadata):
- pass
-
-def session(name='megrok.rdb'):
- return component.getUtility(IDatabase, name).session
+## def setup(self, metadata):
+## pass
Modified: grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/directive.py
===================================================================
--- grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/directive.py 2008-06-24 23:46:08 UTC (rev 87733)
+++ grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/directive.py 2008-06-24 23:51:10 UTC (rev 87734)
@@ -1,7 +1,18 @@
-from martian import Directive, CLASS, ONCE
+from martian import Directive, CLASS, CLASS_OR_MODULE, ONCE
+# XXX add proper validation logic
+
class key(Directive):
scope = CLASS
store = ONCE
default = u''
+class metadata(Directive):
+ scope = CLASS_OR_MODULE
+ store = ONCE
+ default = None
+
+class tablename(Directive):
+ scope = CLASS
+ store = ONCE
+ default = u''
Modified: grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/interfaces.py
===================================================================
--- grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/interfaces.py 2008-06-24 23:46:08 UTC (rev 87733)
+++ grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/interfaces.py 2008-06-24 23:51:10 UTC (rev 87734)
@@ -1,8 +1,7 @@
-from zope.interface import Interface, Attribute
+from zope.interface import Interface
from zope.app.container.interfaces import IContainer as IContainerBase
-
class IContainer(IContainerBase):
def set(value):
"""Add a new value to the container without having to specify the key.
@@ -19,17 +18,3 @@
Defined by SQLAlchemy dictionary-based collections.
"""
-
-class IDatabase(Interface):
- # you have to implement this attribute to set up the connection URL
- url = Attribute(u"The connection URL of the database.")
-
- def setup(metadata):
- """Extra setup the database if required.
-
- Implement this method if you want to do extra setup for the database.
-
- The declarative base classes Model and Container already get set
- up automatically, but you may want to add extra tables and ORM mappers
- in this method.
- """
Modified: grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/meta.py
===================================================================
--- grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/meta.py 2008-06-24 23:46:08 UTC (rev 87733)
+++ grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/meta.py 2008-06-24 23:51:10 UTC (rev 87734)
@@ -1,14 +1,31 @@
import martian
from martian.error import GrokError
+from sqlalchemy.ext.declarative import instrument_declarative
-import megrok.rdb
-from megrok.rdb import directive
+from megrok import rdb
+def default_tablename(factory, module, **data):
+ return factory.__name__.lower()
+
+class ModelGrokker(martian.ClassGrokker):
+ martian.component(rdb.Model)
+ martian.directive(rdb.tablename, get_default=default_tablename)
+ martian.directive(rdb.metadata)
+
+ def execute(self, class_, tablename, metadata, **kw):
+ class_.__tablename__ = tablename
+ # we associate the _decl_registry with the metadata object
+ # to make sure it's unique per metadata. A bit of a hack..
+ if not hasattr(metadata, '_decl_registry'):
+ metadata._decl_registry = {}
+ instrument_declarative(class_, metadata._decl_registry, metadata)
+ return True
+
class ContainerGrokker(martian.ClassGrokker):
- component_class = megrok.rdb.Container
-
+ martian.component(rdb.Container)
+
def grok(self, name, factory, module_info, config, **kw):
- rdb_key = directive.key.get(factory)
+ rdb_key = rdb.key.bind().get(factory)
if rdb_key and hasattr(factory, 'keyfunc'):
raise GrokError(
"It is not allowed to specify a custom 'keyfunc' method "
Modified: grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/schema.py
===================================================================
--- grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/schema.py 2008-06-24 23:46:08 UTC (rev 87733)
+++ grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/schema.py 2008-06-24 23:51:10 UTC (rev 87734)
@@ -5,7 +5,6 @@
from zope.interface.interfaces import IInterface
from zope.component import getUtility
from sqlalchemy.types import String, Integer
-from collective.lead.interfaces import IDatabase
from megrok.rdb import Model
from zope.schema import Int
from zope.schema import Text
Added: grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/tests.py
===================================================================
--- grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/tests.py (rev 0)
+++ grokapps/rdbexample/trunk/megrok.rdb/src/megrok/rdb/tests.py 2008-06-24 23:51:10 UTC (rev 87734)
@@ -0,0 +1,21 @@
+import unittest
+import doctest
+from zope.testing import cleanup
+
+def tearDown(test):
+ cleanup.cleanUp()
+
+ # XXX clean up SQLAlchemy?
+
+def test_suite():
+ optionflags = doctest.NORMALIZE_WHITESPACE | doctest.ELLIPSIS
+ globs = {}
+
+ suite = unittest.TestSuite()
+
+ suite.addTest(doctest.DocFileSuite(
+ 'README.txt',
+ optionflags=optionflags,
+ tearDown=tearDown,
+ globs=globs))
+ return suite
More information about the Checkins
mailing list