[Checkins] SVN: martian/trunk/ Introduce a tutorial text and change
Martian around so it uses directives
Martijn Faassen
faassen at infrae.com
Fri Jun 6 10:42:18 EDT 2008
Log message for revision 87194:
Introduce a tutorial text and change Martian around so it uses directives
itself.
Changed:
U martian/trunk/CHANGES.txt
U martian/trunk/src/martian/README.txt
U martian/trunk/src/martian/__init__.py
U martian/trunk/src/martian/components.py
U martian/trunk/src/martian/core.py
U martian/trunk/src/martian/core.txt
U martian/trunk/src/martian/interfaces.py
A martian/trunk/src/martian/martiandirective.py
U martian/trunk/src/martian/tests/test_all.py
A martian/trunk/src/martian/tutorial.txt
-=-
Modified: martian/trunk/CHANGES.txt
===================================================================
--- martian/trunk/CHANGES.txt 2008-06-06 14:00:28 UTC (rev 87193)
+++ martian/trunk/CHANGES.txt 2008-06-06 14:42:13 UTC (rev 87194)
@@ -1,8 +1,8 @@
CHANGES
*******
-0.9.8 (unreleased)
-==================
+0.10 (unreleased)
+=================
Feature changes
---------------
@@ -12,6 +12,26 @@
* Moved ``FakeModule`` and ``fake_import`` into a ``martian.testing``
module so that they can be reused by external packages.
+* Introduce new tutorial text.
+
+* Introduce a ``GrokkerRegistry`` class that is a ``ModuleGrokker``
+ with a ``MetaMultiGrokker`` in it. This is the convenient thing to
+ instantiate to start working with Grok and is demonstrated in the
+ tutorial.
+
+* Introduced three new martian-specific directives:
+ ``martian.component``, ``martian.directive`` and
+ ``martian.priority``. These replace the ``component_class``,
+ ``directives`` and ``priority`` class-level attributes. This way
+ Grokkers look the same as what they grok. This breaks backwards
+ compatibility again, but it's an easy replace operation. Note that
+ ``martian.directive`` takes the directive itself as an argument, and
+ then optionally the same arguments as the ``bind`` method of
+ directives (``name``, ``default`` and ``get_default``). It may be
+ used multiple times.
+
+* For symmetry, add an ``execute`` method to ``InstanceGrokker``.
+
0.9.7 (2008-05-29)
==================
Modified: martian/trunk/src/martian/README.txt
===================================================================
--- martian/trunk/src/martian/README.txt 2008-06-06 14:00:28 UTC (rev 87193)
+++ martian/trunk/src/martian/README.txt 2008-06-06 14:42:13 UTC (rev 87194)
@@ -154,9 +154,9 @@
>>> import types
>>> from zope.interface import implements
- >>> from martian import InstanceGrokker
- >>> class FileTypeGrokker(InstanceGrokker):
- ... component_class = types.FunctionType
+ >>> import martian
+ >>> class FileTypeGrokker(martian.InstanceGrokker):
+ ... martian.component(types.FunctionType)
...
... def grok(self, name, obj, **kw):
... if not name.startswith('handle_'):
@@ -167,7 +167,7 @@
This ``InstanceGrokker`` allows us to grok instances of a particular
type (such as functions). We need to define the type of object we're
-looking for with the ``component_class`` attribute. In the ``grok``
+looking for with the ``martian.component`` directive. In the ``grok``
method, we first make sure we only grok functions that have a name
that starts with ``handle_``. Then we determine the used extension
from the name and register the funcion in the ``extension_handlers``
@@ -215,16 +215,15 @@
Grokking individual components is useful, but to make Martian really
useful we need to be able to grok whole modules or packages as well.
-Let's look at a special grokker that can grok a Python module::
+Let's look at a special grokker that can grok a Python module, the
+``ModuleGrokker``.
- >>> from martian import ModuleGrokker
-
The idea is that the ``ModuleGrokker`` groks any components in a
module that it recognizes. A ``ModuleGrokker`` does not work alone. It
needs to be supplied with one or more grokkers that can grok the
components to be founded in a module::
- >>> module_grokker = ModuleGrokker()
+ >>> module_grokker = martian.ModuleGrokker()
>>> module_grokker.register(filetype_grokker)
We now define a module that defines a few filetype handlers to be
@@ -309,8 +308,8 @@
``all_colors`` dictionary, with the names as the keys, and the color
object as the values. We can use ``InstanceGrokker`` to construct it::
- >>> class ColorGrokker(InstanceGrokker):
- ... component_class = color.Color
+ >>> class ColorGrokker(martian.InstanceGrokker):
+ ... martian.component(color.Color)
... def grok(self, name, obj, **kw):
... color.all_colors[name] = obj
... return True
@@ -337,7 +336,7 @@
... blue = Color(0, 0, 255)
... white = Color(255, 255, 255)
>>> colors = fake_import(colors)
- >>> colors_grokker = ModuleGrokker()
+ >>> colors_grokker = martian.ModuleGrokker()
>>> colors_grokker.register(color_grokker)
>>> colors_grokker.grok('colors', colors)
True
@@ -377,8 +376,8 @@
... all_sounds = {}
>>> sound = fake_import(sound)
- >>> class SoundGrokker(InstanceGrokker):
- ... component_class = sound.Sound
+ >>> class SoundGrokker(martian.InstanceGrokker):
+ ... martian.component(sound.Sound)
... def grok(self, name, obj, **kw):
... sound.all_sounds[name] = obj
... return True
@@ -419,7 +418,7 @@
Let's put our ``multi_grokker`` in a ``ModuleGrokker``. We can do
this by passing it explicitly to the ``ModuleGrokker`` factory::
- >>> module_grokker = ModuleGrokker(grokker=multi_grokker)
+ >>> module_grokker = martian.ModuleGrokker(grokker=multi_grokker)
We can now grok a module for both ``Color`` and ``Sound`` instances::
@@ -470,12 +469,9 @@
the latter case, we just have to declare what directives the grokker
may want to use on the class and the implement the ``execute`` method::
- >>> from martian import ClassGrokker
- >>> class AnimalGrokker(ClassGrokker):
- ... component_class = animal.Animal
- ... directives = [
- ... animal.name.bind()
- ... ]
+ >>> class AnimalGrokker(martian.ClassGrokker):
+ ... martian.component(animal.Animal)
+ ... martian.directive(animal.name)
... def execute(self, class_, name, **kw):
... animal.all_animals[name] = class_
... return True
@@ -500,9 +496,7 @@
default when binding the directive to the grokker:
>>> class AnimalGrokker(AnimalGrokker):
- ... directives = [
- ... animal.name.bind(default='generic animal')
- ... ]
+ ... martian.directive(animal.name, default='generic animal')
...
>>> class Generic(animal.Animal):
... pass
@@ -522,9 +516,7 @@
... return class_.__name__.lower()
...
>>> class AnimalGrokker(AnimalGrokker):
- ... directives = [
- ... animal.name.bind(get_default=default_animal_name)
- ... ]
+ ... martian.directive(animal.name, get_default=default_animal_name)
...
>>> class Mouse(animal.Animal):
... pass
@@ -549,12 +541,11 @@
>>> def default_zoological_name(class_, module, name, **data):
... return name
...
- >>> class ZooAnimalGrokker(ClassGrokker):
- ... component_class = animal.Animal
- ... directives = [
- ... animal.name.bind(get_default=default_animal_name),
- ... zoologicalname.bind(get_default=default_zoological_name)
- ... ]
+ >>> class ZooAnimalGrokker(martian.ClassGrokker):
+ ... martian.component(animal.Animal)
+ ... martian.directive(animal.name, get_default=default_animal_name)
+ ... martian.directive(zoologicalname, get_default=default_zoological_name)
+ ...
... def execute(self, class_, name, zoologicalname, **kw):
... print zoologicalname
... return True
@@ -570,6 +561,13 @@
hippopotamus
True
+If you pass a non-directive to ``martian.directive``, you get an error::
+
+ >>> class Test(martian.ClassGrokker):
+ ... martian.directive('foo')
+ Traceback (most recent call last):
+ GrokImportError: The 'directive' directive can only be called with a directive.
+
MethodGrokker
-------------
@@ -591,7 +589,7 @@
>>> circus_animals = {}
>>> from martian import MethodGrokker
>>> class CircusAnimalGrokker(MethodGrokker):
- ... component_class = CircusAnimal
+ ... martian.component(CircusAnimal)
... def execute(self, class_, method, **kw):
... circus_animals.setdefault(class_.__name__, []).append(method.__name__)
... return True
@@ -667,7 +665,7 @@
Now let's wrap it into a ``ModuleGrokker`` and grok the module::
- >>> grokker = ModuleGrokker(grokker=multi_grokker)
+ >>> grokker = martian.ModuleGrokker(grokker=multi_grokker)
>>> grokker.grok('animals', animals)
True
@@ -812,7 +810,7 @@
Let's construct a ``ModuleGrokker`` that can grok this module::
- >>> mix_grokker = ModuleGrokker(grokker=multi)
+ >>> mix_grokker = martian.ModuleGrokker(grokker=multi)
Note that this is actually equivalent to calling ``ModuleGrokker``
without arguments and then calling ``register`` for the individual
@@ -863,7 +861,7 @@
Let's construct a ``ModuleGrokker`` with this ``GlobalGrokker`` registered::
- >>> grokker = ModuleGrokker()
+ >>> grokker = martian.ModuleGrokker()
>>> grokker.register(AmountGrokker())
Now we grok and should pick up the right value::
@@ -889,16 +887,16 @@
Let's make a grokker for the old style class::
- >>> class MachineGrokker(ClassGrokker):
- ... component_class = oldstyle.Machine
+ >>> class MachineGrokker(martian.ClassGrokker):
+ ... martian.component(oldstyle.Machine)
... def grok(self, name, obj, **kw):
... oldstyle.all_machines[name] = obj
... return True
And another grokker for old style instances::
- >>> class MachineInstanceGrokker(InstanceGrokker):
- ... component_class = oldstyle.Machine
+ >>> class MachineInstanceGrokker(martian.InstanceGrokker):
+ ... martian.component(oldstyle.Machine)
... def grok(self, name, obj, **kw):
... oldstyle.all_machine_instances[name] = obj
... return True
@@ -930,8 +928,8 @@
>>> from martian.tests.testpackage import animal
>>> all_animals = {}
- >>> class AnimalGrokker(ClassGrokker):
- ... component_class = animal.Animal
+ >>> class AnimalGrokker(martian.ClassGrokker):
+ ... martian.component(animal.Animal)
... def grok(self, name, obj, **kw):
... all_animals[name] = obj
... return True
@@ -940,7 +938,7 @@
Let's register this grokker for a ModuleGrokker::
- >>> module_grokker = ModuleGrokker()
+ >>> module_grokker = martian.ModuleGrokker()
>>> module_grokker.register(AnimalGrokker())
Now let's grok the whole ``testpackage`` for animals::
@@ -965,14 +963,14 @@
... def __init__(self, nr):
... self.nr = nr
>>> all_numbers = {}
- >>> class NumberGrokker(InstanceGrokker):
- ... component_class = Number
+ >>> class NumberGrokker(martian.InstanceGrokker):
+ ... martian.component(Number)
... def grok(self, name, obj, multiplier, **kw):
... all_numbers[obj.nr] = obj.nr * multiplier
... return True
>>> def prepare(name, module, kw):
... kw['multiplier'] = 3
- >>> module_grokker = ModuleGrokker(prepare=prepare)
+ >>> module_grokker = martian.ModuleGrokker(prepare=prepare)
>>> module_grokker.register(NumberGrokker())
We have created a ``prepare`` function that does one thing: create a
@@ -997,7 +995,7 @@
>>> def finalize(name, module, kw):
... all_numbers['finalized'] = True
- >>> module_grokker = ModuleGrokker(prepare=prepare, finalize=finalize)
+ >>> module_grokker = martian.ModuleGrokker(prepare=prepare, finalize=finalize)
>>> module_grokker.register(NumberGrokker())
>>> all_numbers = {}
>>> module_grokker.grok('numbers', numbers)
@@ -1012,12 +1010,12 @@
it didn't. If they return something else (typically ``None`` as the
programmer forgot to), the system will raise an error::
- >>> class BrokenGrokker(InstanceGrokker):
- ... component_class = Number
+ >>> class BrokenGrokker(martian.InstanceGrokker):
+ ... martian.component(Number)
... def grok(self, name, obj, **kw):
... pass
- >>> module_grokker = ModuleGrokker()
+ >>> module_grokker = martian.ModuleGrokker()
>>> module_grokker.register(BrokenGrokker())
>>> module_grokker.grok('numbers', numbers)
Traceback (most recent call last):
@@ -1030,7 +1028,7 @@
>>> class MyGrokker(GlobalGrokker):
... def grok(self, name, module, **kw):
... return "Foo"
- >>> module_grokker = ModuleGrokker()
+ >>> module_grokker = martian.ModuleGrokker()
>>> module_grokker.register(MyGrokker())
>>> module_grokker.grok('numbers', numbers)
Traceback (most recent call last):
@@ -1046,7 +1044,7 @@
>>> from martian.core import MetaGrokker
>>> class ClassMetaGrokker(MetaGrokker):
- ... component_class = ClassGrokker
+ ... martian.component(martian.ClassGrokker)
>>> multi_grokker = MultiGrokker()
>>> multi_grokker.register(ClassMetaGrokker(multi_grokker))
@@ -1127,13 +1125,13 @@
... pass
>>> executed = []
>>> class somemodule(FakeModule):
- ... class TestGrokker(ClassGrokker):
- ... component_class = TestOnce
+ ... class TestGrokker(martian.ClassGrokker):
+ ... martian.component(TestOnce)
... def grok(self, name, obj, **kw):
... executed.append(name)
... return True
>>> somemodule = fake_import(somemodule)
- >>> module_grokker = ModuleGrokker(MetaMultiGrokker())
+ >>> module_grokker = martian.ModuleGrokker(MetaMultiGrokker())
Let's grok the module once::
@@ -1163,8 +1161,8 @@
... pass
>>> executed = []
>>> class somemodule(FakeModule):
- ... class TestGrokker(InstanceGrokker):
- ... component_class = TestInstanceOnce
+ ... class TestGrokker(martian.InstanceGrokker):
+ ... martian.component(TestInstanceOnce)
... def grok(self, name, obj, **kw):
... executed.append(name)
... return True
@@ -1230,7 +1228,7 @@
which names get grokked::
>>> order = []
- >>> class OrderGrokker(ClassGrokker):
+ >>> class OrderGrokker(martian.ClassGrokker):
... def grok(self, name, obj, **kw):
... order.append(name)
... return True
@@ -1239,10 +1237,10 @@
the ``BGrokker`` has a higher priority::
>>> class AGrokker(OrderGrokker):
- ... component_class = A
+ ... martian.component(A)
>>> class BGrokker(OrderGrokker):
- ... component_class = B
- ... priority = 10
+ ... martian.component(B)
+ ... martian.priority(10)
Let's register these grokkers::
@@ -1263,7 +1261,7 @@
We'll grok it::
- >>> module_grokker = ModuleGrokker(multi_grokker)
+ >>> module_grokker = martian.ModuleGrokker(multi_grokker)
>>> module_grokker.grok('mymodule', mymodule)
True
@@ -1277,7 +1275,7 @@
that has a higher priority than the default, but lower than B::
>>> class MyGlobalGrokker(GlobalGrokker):
- ... priority = 5
+ ... martian.priority(5)
... def grok(self, name, obj, **kw):
... order.append(name)
... return True
@@ -1323,7 +1321,7 @@
Clearly, it can't find the module-level variable yet:
- >>> module_grokker = ModuleGrokker()
+ >>> module_grokker = martian.ModuleGrokker()
>>> module_grokker.register(AnnotationsGrokker())
>>> import martian
>>> martian.grok_dotted_name('annotations', module_grokker)
Modified: martian/trunk/src/martian/__init__.py
===================================================================
--- martian/trunk/src/martian/__init__.py 2008-06-06 14:00:28 UTC (rev 87193)
+++ martian/trunk/src/martian/__init__.py 2008-06-06 14:42:13 UTC (rev 87194)
@@ -1,6 +1,6 @@
from martian.core import (
ModuleGrokker, MultiGrokker, MetaMultiGrokker, grok_dotted_name,
- grok_package, grok_module)
+ grok_package, grok_module, GrokkerRegistry)
from martian.components import GlobalGrokker, ClassGrokker, InstanceGrokker
from martian.components import MethodGrokker
from martian.util import scan_for_classes
@@ -10,3 +10,4 @@
from martian.directive import (
validateText, validateInterface, validateClass, validateInterfaceOrClass)
from martian.directive import baseclass
+from martiandirective import component, directive, priority
Modified: martian/trunk/src/martian/components.py
===================================================================
--- martian/trunk/src/martian/components.py 2008-06-06 14:00:28 UTC (rev 87193)
+++ martian/trunk/src/martian/components.py 2008-06-06 14:42:13 UTC (rev 87194)
@@ -17,14 +17,11 @@
from martian import util
from martian.error import GrokError
from martian.interfaces import IGrokker, IComponentGrokker
+from martiandirective import directive, component
-NOT_DEFINED = object()
-
class GrokkerBase(object):
implements(IGrokker)
-
- priority = 0
-
+
def grok(self, name, obj, **kw):
raise NotImplementedError
@@ -40,8 +37,6 @@
class ComponentGrokkerBase(GrokkerBase):
implements(IComponentGrokker)
- component_class = NOT_DEFINED
-
def grok(self, name, obj, **kw):
raise NotImplementedError
@@ -58,8 +53,8 @@
module = module_info.getModule()
# Populate the data dict with information from the directives:
- for directive in self.directives:
- kw[directive.name] = directive.get(class_, module, **kw)
+ for d in directive.bind().get(self.__class__):
+ kw[d.name] = d.get(class_, module, **kw)
return self.execute(class_, **kw)
def execute(self, class_, **data):
@@ -74,11 +69,13 @@
module = module_info.getModule()
# Populate the data dict with information from class or module
- for directive in self.directives:
- kw[directive.name] = directive.get(class_, module, **kw)
+ directives = directive.bind().get(self.__class__)
+ for d in directives:
+ kw[d.name] = d.get(class_, module, **kw)
# Ignore methods that are present on the component baseclass.
- basemethods = set(util.public_methods_from_class(self.component_class))
+ basemethods = set(util.public_methods_from_class(
+ component.bind().get(self.__class__)))
methods = set(util.public_methods_from_class(class_)) - basemethods
if not methods:
raise GrokError("%r does not define any public methods. "
@@ -91,11 +88,11 @@
# check each directive and potentially override the
# class-level value with a value from the method *locally*.
data = kw.copy()
- for bound_dir in self.directives:
- directive = bound_dir.directive
+ for bound_dir in directives:
+ d = bound_dir.directive
class_value = data[bound_dir.name]
- data[bound_dir.name] = directive.store.get(directive, method,
- default=class_value)
+ data[bound_dir.name] = d.store.get(d, method,
+ default=class_value)
results.append(self.execute(class_, method, **data))
return max(results)
@@ -107,4 +104,9 @@
class InstanceGrokker(ComponentGrokkerBase):
"""Grokker that groks instances in a module.
"""
- pass
+ def grok(self, name, class_, **kw):
+ return self.execute(class_, **kw)
+
+ def execute(self, class_, **kw):
+ raise NotImplementedError
+
Modified: martian/trunk/src/martian/core.py
===================================================================
--- martian/trunk/src/martian/core.py 2008-06-06 14:00:28 UTC (rev 87193)
+++ martian/trunk/src/martian/core.py 2008-06-06 14:42:13 UTC (rev 87194)
@@ -8,6 +8,7 @@
GlobalGrokker)
from martian.error import GrokError
+from martiandirective import component, priority
class MultiGrokkerBase(GrokkerBase):
implements(IMultiGrokker)
@@ -35,6 +36,11 @@
def grokkers(self, name, obj):
raise NotImplementedError
+def _grokker_sort_key((grokker, name, obj)):
+ """Helper function to calculate sort order of grokker.
+ """
+ return priority.bind().get(grokker)
+
class ModuleGrokker(MultiGrokkerBase):
def __init__(self, grokker=None, prepare=None, finalize=None):
@@ -59,7 +65,7 @@
# sort grokkers by priority
grokkers = sorted(self.grokkers(name, module),
- key=lambda (grokker, name, obj): grokker.priority,
+ key=_grokker_sort_key,
reverse=True)
for g, name, obj in grokkers:
@@ -104,7 +110,7 @@
self.clear()
def register(self, grokker):
- key = grokker.component_class
+ key = component.bind().get(grokker)
grokkers = self._grokkers.setdefault(key, [])
for g in grokkers:
if g.__class__ is grokker.__class__:
@@ -217,10 +223,14 @@
return True
class ClassMetaGrokker(MetaGrokker):
- component_class = ClassGrokker
+ component(ClassGrokker)
class InstanceMetaGrokker(MetaGrokker):
- component_class = InstanceGrokker
+ component(InstanceGrokker)
class GlobalMetaGrokker(MetaGrokker):
- component_class = GlobalGrokker
+ component(GlobalGrokker)
+
+class GrokkerRegistry(ModuleGrokker):
+ def __init__(self):
+ super(GrokkerRegistry, self).__init__(MetaMultiGrokker())
Modified: martian/trunk/src/martian/core.txt
===================================================================
--- martian/trunk/src/martian/core.txt 2008-06-06 14:00:28 UTC (rev 87193)
+++ martian/trunk/src/martian/core.txt 2008-06-06 14:42:13 UTC (rev 87194)
@@ -30,7 +30,7 @@
>>> import types
>>> class FunctionGrokker(martian.InstanceGrokker):
- ... component_class = types.FunctionType
+ ... martian.component(types.FunctionType)
... def grok(self, name, obj, **kw):
... print name, obj()
... return True
Modified: martian/trunk/src/martian/interfaces.py
===================================================================
--- martian/trunk/src/martian/interfaces.py 2008-06-06 14:00:28 UTC (rev 87193)
+++ martian/trunk/src/martian/interfaces.py 2008-06-06 14:42:13 UTC (rev 87194)
@@ -15,8 +15,12 @@
from zope.interface import Interface, Attribute
class IGrokker(Interface):
- priority = Attribute('Priority during module grokking.')
+ """A grokker that groks components.
+ Use the martian.priority directive to specify the priority
+ (within a module) with which to grok. The higher the priority,
+ the earlier the grokker will be executed.
+ """
def grok(name, obj, **kw):
"""Grok obj.
@@ -33,9 +37,8 @@
class IComponentGrokker(IGrokker):
"""A grokker that groks components in a module.
- Components may be instances or classes indicated by component_class.
+ Use the martian.component directive to specify the component to grok.
"""
- component_class = Attribute('Class of the component to match')
class IMultiGrokker(IComponentGrokker):
"""A grokker that is composed out of multiple grokkers.
Added: martian/trunk/src/martian/martiandirective.py
===================================================================
--- martian/trunk/src/martian/martiandirective.py (rev 0)
+++ martian/trunk/src/martian/martiandirective.py 2008-06-06 14:42:13 UTC (rev 87194)
@@ -0,0 +1,34 @@
+"""Martian-specific directives"""
+
+from martian.directive import (Directive, MultipleTimesDirective,
+ CLASS, ONCE, validateClass)
+from martian.error import GrokImportError
+
+class component(Directive):
+ scope = CLASS
+ store = ONCE
+ default = None
+ validate = validateClass
+
+class directive(MultipleTimesDirective):
+ scope = CLASS
+
+ def validate(self, directive, *args, **kw):
+ try:
+ if issubclass(directive, Directive):
+ return
+ except TypeError:
+ # directive is not a class, so error too
+ pass
+ raise GrokImportError("The '%s' directive can only be called with "
+ "a directive." % self.name)
+
+ def factory(self, directive, *args, **kw):
+ return directive.bind(*args, **kw)
+
+class priority(Directive):
+ scope = CLASS
+ store = ONCE
+ default = 0
+
+
Modified: martian/trunk/src/martian/tests/test_all.py
===================================================================
--- martian/trunk/src/martian/tests/test_all.py 2008-06-06 14:00:28 UTC (rev 87193)
+++ martian/trunk/src/martian/tests/test_all.py 2008-06-06 14:42:13 UTC (rev 87194)
@@ -9,6 +9,10 @@
def test_suite():
suite = unittest.TestSuite()
suite.addTests([
+ doctest.DocFileSuite('tutorial.txt',
+ package='martian',
+ globs=globs,
+ optionflags=optionflags),
doctest.DocFileSuite('README.txt',
package='martian',
globs=globs,
Copied: martian/trunk/src/martian/tutorial.txt (from rev 87170, martian/trunk/src/martian/README.txt)
===================================================================
--- martian/trunk/src/martian/tutorial.txt (rev 0)
+++ martian/trunk/src/martian/tutorial.txt 2008-06-06 14:42:13 UTC (rev 87194)
@@ -0,0 +1,411 @@
+Martian
+=======
+
+"There was so much to grok, so little to grok from." -- Stranger in a
+Strange Land, by Robert A. Heinlein
+
+Martian provides infrastructure for declarative configuration of
+Python code. Martian is especially useful for the construction of
+frameworks that need to provide a flexible plugin
+infrastructure. Martian doesn't actually provide any infrastructure
+for plugin registries and such. Many frameworks have their own, and if
+you need a generic one, you might want to consider
+``zope.component``. Martian just allows you to make the registration
+of such plugins less verbose.
+
+You can see Martian as doing something that you can also solve with
+metaclasses, with the following advantages:
+
+* the developer of the framework doesn't have to write a lot of ad-hoc
+ metaclasses anymore; instead we offer an infrastructure to make life
+ easier.
+
+* configuration doesn't need to happen at import time, but can happen at
+ program startup time. This also makes configuration more tractable for
+ a developer.
+
+* we don't bother the developer that *uses* the framework with the
+ surprising behavior that metaclasses sometimes bring. The classes
+ the user has to deal with are normal classes.
+
+Why is this package named ``martian``? In the novel "Stranger in a
+Strange Land", the verb *grok* is introduced:
+
+ Grok means to understand so thoroughly that the observer becomes a
+ part of the observed -- to merge, blend, intermarry, lose identity
+ in group experience.
+
+In the context of this package, "grokking" stands for the process of
+deducing declarative configuration actions from Python code. In the
+novel, grokking is originally a concept that comes from the planet
+Mars. Martians *grok*. Since this package helps you grok code, it's
+called Martian.
+
+The ``martian`` package is a spin-off from the `Grok project`_, in the
+context of which this codebase was first developed. While Grok uses
+it, the code is completely independent of Grok.
+
+.. _`Grok project`: http://grok.zope.org
+
+Motivation
+----------
+
+"Deducing declarative configuration actions from Python code" - that
+sounds very abstract. What does it actually mean? What is
+configuration? What is declarative configuration? In order to explain
+this, we'll first take a look at configuration.
+
+Larger frameworks often offer a lot of points where you can modify
+their behavior: ways to combine its own components with components you
+provide yourself to build a larger application. A framework offers
+points where it can be *configured* with plugin code. When you plug
+some code into a plugin point, it results in the updating of some
+registry somewhere with the new plugin. When the framework uses a
+plugin, it will first look it up in the registry. The action of
+registering some component into a registry can be called
+*configuration*.
+
+Let's look at an example framework that offers a plugin point. We
+introduce a very simple framework for plugging in different template
+languages, where each template language uses its own extension. You
+can then supply the framework with the template body and the template
+extension and some data, and render the template.
+
+Let's look at the framework::
+
+ >>> import string
+ >>> class templating(FakeModule):
+ ...
+ ... class InterpolationTemplate(object):
+ ... "Use %(foo)s for dictionary interpolation."
+ ... def __init__(self, text):
+ ... self.text = text
+ ... def render(self, **kw):
+ ... return self.text % kw
+ ...
+ ... class TemplateStringTemplate(object):
+ ... "PEP 292 string substitutions."
+ ... def __init__(self, text):
+ ... self.template = string.Template(text)
+ ... def render(self, **kw):
+ ... return self.template.substitute(**kw)
+ ...
+ ... # the registry, we plug in the two templating systems right away
+ ... extension_handlers = { '.txt': InterpolationTemplate,
+ ... '.tmpl': TemplateStringTemplate }
+ ...
+ ... def render(data, extension, **kw):
+ ... """Render the template at filepath with arguments.
+ ...
+ ... data - the data in the file
+ ... extension - the extension of the file
+ ... keyword arguments - variables to interpolate
+ ...
+ ... In a real framework you could pass in the file path instead of
+ ... data and extension, but we don't want to open files in our
+ ... example.
+ ...
+ ... Returns the rendered template
+ ... """
+ ... template = extension_handlers[extension](data)
+ ... return template.render(**kw)
+
+Since normally we cannot create modules in a doctest, we have emulated
+the ``templating`` Python module using the ``FakeModule``
+class. Whenever you see ``FakeModule`` subclasses, imagine you're
+looking at a module definition in a ``.py`` file. Now that we have
+defined a module ``templating``, we also need to be able to import
+it. To do so we can use a a fake import statement that lets us do
+this::
+
+ >>> templating = fake_import(templating)
+
+Now let's try the ``render`` function for the registered template
+types, to demonstrate that our framework works::
+
+ >>> templating.render('Hello %(name)s!', '.txt', name="world")
+ 'Hello world!'
+ >>> templating.render('Hello ${name}!', '.tmpl', name="universe")
+ 'Hello universe!'
+
+File extensions that we do not recognize cause a ``KeyError`` to be
+raised::
+
+ >>> templating.render('Hello', '.silly', name="test")
+ Traceback (most recent call last):
+ ...
+ KeyError: '.silly'
+
+We now want to plug into this filehandler framework and provide a
+handler for ``.silly`` files. Since we are writing a plugin, we cannot
+change the ``templating`` module directly. Let's write an extension
+module instead::
+
+ >>> class sillytemplating(FakeModule):
+ ... class SillyTemplate(object):
+ ... "Replace {key} with dictionary values."
+ ... def __init__(self, text):
+ ... self.text = text
+ ... def render(self, **kw):
+ ... text = self.text
+ ... for key, value in kw.items():
+ ... text = text.replace('{%s}' % key, value)
+ ... return text
+ ...
+ ... templating.extension_handlers['.silly'] = SillyTemplate
+ >>> sillytemplating = fake_import(sillytemplating)
+
+In the extension module, we manipulate the ``extension_handlers``
+dictionary of the ``templating`` module (in normal code we'd need to
+import it first), and plug in our own function. ``.silly`` handling
+works now::
+
+ >>> templating.render('Hello {name}!', '.silly', name="galaxy")
+ 'Hello galaxy!'
+
+Above we plug into our ``extension_handler`` registry using Python
+code. Using separate code to manually hook components into registries
+can get rather cumbersome - each time you write a plugin, you also
+need to remember you need to register it.
+
+Doing template registration in Python code also poses a maintenance
+risk. It is tempting to start doing fancy things in Python code such
+as conditional configuration, making the configuration state of a
+program hard to understand. Another problem is that doing
+configuration at import time can also lead to unwanted side effects
+during import, as well as ordering problems, where you want to import
+something that really needs configuration state in another module that
+is imported later. Finally, it can also make code harder to test, as
+configuration is loaded always when you import the module, even if in
+your test perhaps you don't want it to be.
+
+Martian provides a framework that allows configuration to be expressed
+in declarative Python code. Martian is based on the realization that
+what to configure where can often be deduced from the structure of
+Python code itself, especially when it can be annotated with
+additional declarations. The idea is to make it so easy to write and
+register a plugin so that even extensive configuration does not overly
+burden the developer.
+
+Configuration actions are executed during a separate phase ("grok
+time"), not at import time, which makes it easier to reason about and
+easier to test.
+
+Configuration the Martian Way
+-----------------------------
+
+Let's now transform the above ``templating`` module and the
+``sillytemplating`` module to use Martian. First we must recognize
+that every template language is configured to work for a particular
+extension. With Martian, we annotate the classes themselves with this
+configuration information. Annotations happen using *directives*,
+which look like function calls in the class body.
+
+Let's create an ``extension`` directive that can take a single string
+as an argument, the file extension to register the template class
+for::
+
+ >>> import martian
+ >>> class extension(martian.Directive):
+ ... scope = martian.CLASS
+ ... store = martian.ONCE
+ ... default = None
+
+We also need a way to easily recognize all template classes. The normal
+pattern for this in Martian is to use a base class, so let's define a
+``Template`` base class::
+
+ >>> class Template(object):
+ ... pass
+
+We now have enough infrastructure to allow us to change the code to use
+Martian style base class and annotations::
+
+ >>> class templating(FakeModule):
+ ...
+ ... class InterpolationTemplate(Template):
+ ... "Use %(foo)s for dictionary interpolation."
+ ... extension('.txt')
+ ... def __init__(self, text):
+ ... self.text = text
+ ... def render(self, **kw):
+ ... return self.text % kw
+ ...
+ ... class TemplateStringTemplate(Template):
+ ... "PEP 292 string substitutions."
+ ... extension('.tmpl')
+ ... def __init__(self, text):
+ ... self.template = string.Template(text)
+ ... def render(self, **kw):
+ ... return self.template.substitute(**kw)
+ ...
+ ... # the registry, empty to start with
+ ... extension_handlers = {}
+ ...
+ ... def render(data, extension, **kw):
+ ... # this hasn't changed
+ ... template = extension_handlers[extension](data)
+ ... return template.render(**kw)
+ >>> templating = fake_import(templating)
+
+As you can see, there have been very few changes:
+
+* we made the template classes inherit from ``Template``.
+
+* we use the ``extension`` directive in the template classes.
+
+* we stopped pre-filling the ``extension_handlers`` dictionary.
+
+So how do we fill the ``extension_handlers`` dictionary with the right
+template languages? Now we can use Martian. We define a *grokker* for
+``Template`` that registers the template classes in the
+``extension_handlers`` registry::
+
+ >>> class meta(FakeModule):
+ ... class TemplateGrokker(martian.ClassGrokker):
+ ... martian.component(Template)
+ ... martian.directive(extension)
+ ... def execute(self, class_, extension, **kw):
+ ... templating.extension_handlers[extension] = class_
+ ... return True
+ >>> meta = fake_import(meta)
+
+What does this do? A ``ClassGrokker`` has its ``execute`` method
+called for subclasses of what's indicated by the ``martian.component``
+directive. You can also declare what directives a ``ClassGrokker``
+expects on this component by using ``martian.directive()`` (the
+``directive`` directive!) one or more times.
+
+The ``execute`` method takes the class to be grokked as the first
+argument, and the values of the directives used will be passed in as
+additional parameters into the ``execute`` method. The framework can
+also pass along an arbitrary number of extra keyword arguments during
+the grokking process, so we need to declare ``**kw`` to make sure we
+can handle these.
+
+All our grokkers will be collected in a special Martian-specific
+registry::
+
+ >>> reg = martian.GrokkerRegistry()
+
+We will need to make sure the system is aware of the
+``TemplateGrokker`` defined in the ``meta`` module first, so let's
+register it first. We can do this by simply grokking the ``meta``
+module::
+
+ >>> reg.grok('meta', meta)
+ True
+
+Because ``TemplateGrokker`` is now registered, our registry now knows
+how to grok ``Template`` subclasses. Let's grok the ``templating``
+module::
+
+ >>> reg.grok('templating', templating)
+ True
+
+Let's try the ``render`` function of templating again, to demonstrate
+we have successfully grokked the template classes::
+
+ >>> templating.render('Hello %(name)s!', '.txt', name="world")
+ 'Hello world!'
+ >>> templating.render('Hello ${name}!', '.tmpl', name="universe")
+ 'Hello universe!'
+
+``.silly`` hasn't been registered yet::
+
+ >>> templating.render('Hello', '.silly', name="test")
+ Traceback (most recent call last):
+ ...
+ KeyError: '.silly'
+
+Let's now register ``.silly`` from an extension module::
+
+ >>> class sillytemplating(FakeModule):
+ ... class SillyTemplate(Template):
+ ... "Replace {key} with dictionary values."
+ ... extension('.silly')
+ ... def __init__(self, text):
+ ... self.text = text
+ ... def render(self, **kw):
+ ... text = self.text
+ ... for key, value in kw.items():
+ ... text = text.replace('{%s}' % key, value)
+ ... return text
+ >>> sillytemplating = fake_import(sillytemplating)
+
+As you can see, the developer that uses the framework has no need
+anymore to know about ``templating.extension_handlers``. Instead we can
+simply grok the module to have ``SillyTemplate`` be register appropriately::
+
+ >>> reg.grok('sillytemplating', sillytemplating)
+ True
+
+We can now use the ``.silly`` templating engine too::
+
+ >>> templating.render('Hello {name}!', '.silly', name="galaxy")
+ 'Hello galaxy!'
+
+Admittedly it is hard to demonstrate Martian well with a small example
+like this. In the end we have actually written more code than in the
+basic framework, after all. But even in this small example, the
+``templating`` and ``sillytemplating`` module have become more
+declarative in nature. The developer that uses the framework will not
+need to know anymore about things like
+``templating.extension_handlers`` or an API to register things
+there. Instead the developer can registering a new template system
+anywhere, as long as he subclasses from ``Template``, and as long as
+his code is grokked by the system.
+
+Finally note how Martian was used to define the ``TemplateGrokker`` as
+well. In this way Martian can use itself to extend itself.
+
+Grokking instances
+------------------
+
+Above we've seen how you can grok classes. Martian also supplies a way
+to grok instances. This is less common in typical frameworks, and has
+the drawback that no class-level directives can be used, but can still
+be useful.
+
+Let's imagine a case where we have a zoo framework with an ``Animal``
+class, and we want to track instances of it::
+
+ >>> class Animal(object):
+ ... def __init__(self, name):
+ ... self.name = name
+ >>> class zoo(FakeModule):
+ ... horse = Animal('horse')
+ ... chicken = Animal('chicken')
+ ... elephant = Animal('elephant')
+ ... lion = Animal('lion')
+ ... animals = {}
+ >>> zoo = fake_import(zoo)
+
+We define an ``InstanceGrokker`` subclass to grok ``Animal`` instances::
+
+ >>> class meta(FakeModule):
+ ... class AnimalGrokker(martian.InstanceGrokker):
+ ... martian.component(Animal)
+ ... def execute(self, instance, **kw):
+ ... zoo.animals[instance.name] = instance
+ ... return True
+ >>> meta = fake_import(meta)
+
+Let's create a new registry with the ``AnimalGrokker`` in it::
+
+ >>> reg = martian.GrokkerRegistry()
+ >>> reg.grok('meta', meta)
+ True
+
+We can now grok the ``zoo`` module::
+
+ >>> reg.grok('zoo', zoo)
+ True
+
+The animals will now be in the ``animals`` dictionary::
+
+ >>> sorted(zoo.animals.items())
+ [('chicken', <Animal object at ...>),
+ ('elephant', <Animal object at ...>),
+ ('horse', <Animal object at ...>),
+ ('lion', <Animal object at ...>)]
More information about the Checkins
mailing list