[Checkins] SVN: manuel/trunk/ initial import of manuel
Benji York
benji at zope.com
Fri Oct 17 17:31:24 EDT 2008
Log message for revision 92347:
initial import of manuel
Changed:
_U manuel/trunk/
A manuel/trunk/buildout.cfg
A manuel/trunk/setup.py
A manuel/trunk/src/
A manuel/trunk/src/manuel/
A manuel/trunk/src/manuel/README.txt
A manuel/trunk/src/manuel/__init__.py
A manuel/trunk/src/manuel/doctest.py
A manuel/trunk/src/manuel/footnote.py
A manuel/trunk/src/manuel/footnote.txt
A manuel/trunk/src/manuel/testing.py
A manuel/trunk/src/manuel/tests.py
-=-
Property changes on: manuel/trunk
___________________________________________________________________
Name: svn:ignore
+ develop-eggs
Python-trunk
dist
bin
parts
.installed.cfg
Added: manuel/trunk/buildout.cfg
===================================================================
--- manuel/trunk/buildout.cfg (rev 0)
+++ manuel/trunk/buildout.cfg 2008-10-17 21:31:23 UTC (rev 92347)
@@ -0,0 +1,15 @@
+[buildout]
+develop = .
+parts = test interpreter
+extensions = zc.buildoutsftp
+
+[test]
+recipe = zc.recipe.testrunner
+eggs = manuel
+defaults = '--tests-pattern tests --exit-with-status -1 --auto-color'.split()
+working-directory = .
+
+[interpreter]
+recipe = zc.recipe.egg
+eggs = manuel
+interpreter = py
Added: manuel/trunk/setup.py
===================================================================
--- manuel/trunk/setup.py (rev 0)
+++ manuel/trunk/setup.py 2008-10-17 21:31:23 UTC (rev 92347)
@@ -0,0 +1,21 @@
+import os
+
+from setuptools import setup, find_packages
+
+setup(
+ name='manuel',
+ version='0',
+ packages=find_packages('src'),
+ package_dir={'':'src'},
+ zip_safe=False,
+ author='Benji York',
+ author_email='benji at benjiyork.com',
+ description='Design test syntax to match the task at hand and/or make '
+ 'documentation testable.',
+ license='ZPL',
+ install_requires=[
+ 'setuptools',
+ 'zope.testing',
+ ],
+ include_package_data=True,
+ )
Property changes on: manuel/trunk/setup.py
___________________________________________________________________
Name: svn:keywords
+ Id
Name: svn:eol-style
+ native
Added: manuel/trunk/src/manuel/README.txt
===================================================================
--- manuel/trunk/src/manuel/README.txt (rev 0)
+++ manuel/trunk/src/manuel/README.txt 2008-10-17 21:31:23 UTC (rev 92347)
@@ -0,0 +1,427 @@
+======
+Manuel
+======
+
+Documentation and testing are important parts of software development. Often
+they can be combined such that you get tests that are well documented or
+documentation that is well tested. That's what Manuel is about.
+
+
+Overview
+========
+
+Manuel parses documents, evaluates their contents, then formats the result of
+the evaluation. All three phases are done in multiple passes. The passes
+continue until the data structure remains unchanged by a pass.
+
+The core functionality is accessed through an instance of a Manuel object. It
+is used to build up our handling of a document type. Each phase has a
+corresponding slot to which various implementations are attached.
+
+ >>> import manuel
+ >>> m = manuel.Manuel()
+ >>> m
+ <manuel.Manuel object at 0x...>
+
+
+Parsing
+-------
+
+Manuel operates on Documents. Each Document is created from a a string
+containing one or more lines.
+
+ >>> source = """\
+ ... This is our document, it has several lines.
+ ... one: 1, 2, 3
+ ... two: 4, 5, 7
+ ... three: 3, 5, 1
+ ... """
+ >>> document = manuel.Document(source)
+
+For example purposes we will create a type of test that consists of a sequence
+of numbers so lets create a NumbersTest object to represent the parsed list.
+
+ >>> class NumbersTest(object):
+ ... def __init__(self, description, numbers):
+ ... self.description = description
+ ... self.numbers = numbers
+
+The Document is divided into one or more regions. Each region is a distinct
+"chunk" of the document and will be acted uppon in later (post-parsing) phases.
+Initially the Document is made up of a single element, the source string.
+
+ >>> [region.source for region in document]
+ ['This is our document, it has several lines.\none: 1, 2, 3\ntwo: 4, 5, 7\nthree: 3, 5, 1\n']
+
+The Document offers a "find_regions" method to assist in locating the portions
+of the document a particular parser is interested in. Given a regular
+expression (either as a string, or compiled), it will return "region" objects
+that contain the matched source text, the line number (1 based) the region
+begins at, as well as the associated re.Match object.
+
+ >>> import re
+ >>> numbers_test_finder = re.compile(
+ ... r'^(?P<description>.*?): (?P<numbers>(\d+,?[ ]?)+)$', re.MULTILINE)
+ >>> regions = document.find_regions(numbers_test_finder)
+ >>> regions
+ [<manuel.Region object at 0x...>,
+ <manuel.Region object at 0x...>,
+ <manuel.Region object at 0x...>]
+ >>> regions[0].lineno
+ 2
+ >>> regions[0].source
+ 'one: 1, 2, 3\n'
+ >>> regions[0].start_match.group('description')
+ 'one'
+ >>> regions[0].start_match.group('numbers')
+ '1, 2, 3'
+
+If given two regular expressions find_regions will use the first to identify
+the begining of a region and the second to identify the end.
+
+ >>> region = document.find_regions(
+ ... re.compile('^one:.*$', re.MULTILINE),
+ ... re.compile('^three:.*$', re.MULTILINE),
+ ... )[0]
+ >>> region.lineno
+ 2
+ >>> print region.source
+ one: 1, 2, 3
+ two: 4, 5, 7
+ three: 3, 5, 1
+
+Also, instead of a single "match" attribute, the region will have start_match
+and end_match attributes.
+
+ >>> region.start_match
+ <_sre.SRE_Match object at 0x...>
+ >>> region.end_match
+ <_sre.SRE_Match object at 0x...>
+
+
+Regions must always consist of whole lines.
+
+ >>> document.find_regions('1, 2, 3')
+ Traceback (most recent call last):
+ ...
+ ValueError: Regions must start at the begining of a line.
+
+ >>> document.find_regions('three')
+ Traceback (most recent call last):
+ ...
+ ValueError: Regions must end at the ending of a line.
+
+ >>> document.find_regions(
+ ... re.compile('ne:.*$', re.MULTILINE),
+ ... re.compile('^one:.*$', re.MULTILINE),
+ ... )
+ Traceback (most recent call last):
+ ...
+ ValueError: Regions must start at the begining of a line.
+
+ >>> document.find_regions(
+ ... re.compile('^one:.*$', re.MULTILINE),
+ ... re.compile('^three:', re.MULTILINE),
+ ... )
+ Traceback (most recent call last):
+ ...
+ ValueError: Regions must end at the ending of a line.
+
+Now we can register a parser that will identify the regions we're interested in
+and create NumbersTest objects from the source text.
+
+ >>> @m.parser
+ ... def parse_numbers_test(document):
+ ... for region in document.find_regions(numbers_test_finder):
+ ... description = region.start_match.group('description')
+ ... numbers = map(
+ ... int, region.start_match.group('numbers').split(','))
+ ... test = NumbersTest(description, numbers)
+ ... document.replace_region(region, test)
+
+ >>> document.parse_with(m)
+ >>> [region.source for region in document]
+ ['This is our document, it has several lines.\n',
+ 'one: 1, 2, 3\n',
+ 'two: 4, 5, 7\n',
+ 'three: 3, 5, 1\n']
+ >>> [region.parsed for region in document]
+ [None,
+ <NumbersTest object at 0x...>,
+ <NumbersTest object at 0x...>,
+ <NumbersTest object at 0x...>]
+
+
+Evaluation
+----------
+
+After a document has been parsed the resulting tests are evaluated. Manuel
+provides another method to evaluate tests. Lets define a function to evaluate
+NumberTests. The function determines whether or not the numbers are in sorted
+order and records the result along with the description of the list of numbers.
+
+ >>> class NumbersResult(object):
+ ... def __init__(self, test, passed):
+ ... self.test = test
+ ... self.passed = passed
+
+ >>> @m.evaluater
+ ... def evaluate_numbers(document):
+ ... for region in document:
+ ... if not isinstance(region.parsed, NumbersTest):
+ ... continue
+ ... test = region.parsed
+ ... passed = sorted(test.numbers) == test.numbers
+ ... region.evaluated = NumbersResult(test, passed)
+
+ >>> document.evaluate_with(m)
+ >>> [region.evaluated for region in document]
+ [None,
+ <NumbersResult object at 0x...>,
+ <NumbersResult object at 0x...>,
+ <NumbersResult object at 0x...>]
+
+
+Formatting
+----------
+
+Once the evaluation phase is completed the results are formatted. You guessed
+it: manuel provides a method for formatting results. We'll build one to format
+a message about whether or not our lists of numbers are sorted properly. A
+formatting function returns None when it has no output, or a string otherwise.
+
+ >>> @m.formatter
+ ... def format(document):
+ ... for region in document:
+ ... if not isinstance(region.evaluated, NumbersResult):
+ ... continue
+ ... result = region.evaluated
+ ... if not result.passed:
+ ... region.formatted = (
+ ... "the numbers aren't in sorted order: "
+ ... + ', '.join(map(str, result.test.numbers)))
+
+Since our test case passed we don't get anything out of the report function.
+
+ >>> document.format_with(m)
+ >>> [region.formatted for region in document]
+ [None, None, None, "the numbers aren't in sorted order: 3, 5, 1"]
+
+
+We'll want to use this Manuel object later, so lets stash it away
+
+ >>> sorted_numbers_manuel = m
+
+
+Doctests
+========
+
+We can use manuel to run doctests. Let's create a simple doctest to
+demonstrate with.
+
+ >>> source = """This is my
+ ... doctest.
+ ...
+ ... >>> 1 + 1
+ ... 2
+ ... """
+ >>> document = manuel.Document(source)
+
+The manuel.doctest module has handlers for the various phases. First we'll
+look at parsing.
+
+ >>> import manuel.doctest
+ >>> m = manuel.doctest.Manuel()
+ >>> document.parse_with(m)
+ >>> for region in document:
+ ... print (region.lineno, region.parsed or region.source)
+ (1, 'This is my\ndoctest.\n\n')
+ (4, <zope.testing.doctest.Example instance at 0x...>)
+ (6, '\n')
+
+Now we can evaluate the examples.
+
+ >>> document.evaluate_with(m)
+ >>> for region in document:
+ ... print (region.lineno, region.evaluated or region.source)
+ (1, 'This is my\ndoctest.\n\n')
+ (4, <manuel.doctest.DocTestResult instance at 0x...>)
+ (6, '\n')
+
+And format the results.
+
+ >>> document.format_with(m)
+ >>> document.formatted()
+ ''
+
+Oh, we didn't have any failing tests, so we got no output. Let's try again
+with a failing test. This time we'll use the process function to simplify
+things.
+
+ >>> document = manuel.Document("""This is my
+ ... doctest.
+ ...
+ ... >>> 1 + 1
+ ... 42
+ ... """)
+
+ >>> document.process_with(m)
+ >>> print document.formatted()
+ File "<memory>", line 4, in <memory>
+ Failed example:
+ 1 + 1
+ Expected:
+ 42
+ Got:
+ 2
+
+
+Globals
+-------
+
+Even though each example is parsed into its own object, state is still shared
+between them.
+
+ >>> document = manuel.Document("""
+ ... >>> x = 1
+ ...
+ ... A little prose to separate the examples.
+ ...
+ ... >>> x
+ ... 1
+ ... """)
+ >>> document.process_with(m)
+ >>> print document.formatted()
+
+Imported modules are added to the global namespace as well.
+
+ >>> document = manuel.Document("""
+ ... >>> import string
+ ...
+ ... A little prose to separate the examples.
+ ...
+ ... >>> string.digits
+ ... '0123456789'
+ ...
+ ... """)
+ >>> document.process_with(m)
+ >>> print document.formatted()
+
+
+Combining Test Types
+====================
+
+Now that we have both doctests and the silly "sorted numbers" tests, lets
+create a single document that has both.
+
+ >>> document = manuel.Document("""
+ ... We can test Python...
+ ...
+ ... >>> 1 + 1
+ ... 42
+ ...
+ ... ...and lists of numbers.
+ ...
+ ... a very nice list: 3, 6, 2
+ ... """)
+
+Obviously both of those tests will fail, but first we have to configure Manuel
+to understand both test types. We'll start with a doctest configuration and add
+the number list testing on top.
+
+ >>> m = manuel.doctest.Manuel()
+
+Since we already have a Manuel instance configured for our "sorted numbers"
+tests, we can extend the built-in doctest configuration with it.
+
+ >>> m.extend(sorted_numbers_manuel)
+
+Now we can process our source that combines both types of tests and see what
+we get.
+
+ >>> document.process_with(m)
+
+The document was parsed and has a mixture of prose and parsed doctests and
+number tests.
+
+ >>> for region in document:
+ ... print (region.lineno, region.parsed or region.source)
+ (1, '\nWe can test Python...\n\n')
+ (4, <doctest.Example instance at 0x...>)
+ (6, '\n...and lists of numbers.\n\n')
+ (9, <NumbersTest object at 0x...>)
+
+We can look at the formatted output to see that each of the two tests failed.
+
+ >>> for region in document:
+ ... if region.formatted:
+ ... print '-'*70
+ ... print region.formatted,
+ ----------------------------------------------------------------------
+ File "<memory>", line 4, in <memory>
+ Failed example:
+ 1 + 1
+ Expected:
+ 42
+ Got:
+ 2
+ ----------------------------------------------------------------------
+ the numbers aren't in sorted order: 3, 6, 2
+
+
+Priorities
+==========
+
+Some functionality requires that code be called early or late in a phase. The
+"timing" keyword parameter allows either "early" or "late" to be specified.
+
+Early functions are run first (in arbitrary order), then functions with no
+specified timing, then the late functions are called (again in arbitrary
+order). This function also demonstrates the "copy" method of Region objects
+and the "insert_region_before" and "insert_region_after" methods of Documents.
+
+ >>> @m.parser(timing='late')
+ ... def cloner(document):
+ ... to_be_cloned = None
+ ... # find the region to clone
+ ... document_iter = iter(document)
+ ... for region in document_iter:
+ ... if region.parsed:
+ ... continue
+ ... if region.source.strip().endswith('my clone:'):
+ ... to_be_cloned = document_iter.next().copy()
+ ... break
+ ... # if we found the region to cloned, do so
+ ... if to_be_cloned:
+ ... # make a copy since we'll be mutating the document
+ ... for region in list(document):
+ ... if region.parsed:
+ ... continue
+ ... if 'clone before *here*' in region.source:
+ ... clone = to_be_cloned.copy()
+ ... clone.provenance = 'cloned to go before'
+ ... document.insert_region_before(region, clone)
+ ... if 'clone after *here*' in region.source:
+ ... clone = to_be_cloned.copy()
+ ... clone.provenance = 'cloned to go after'
+ ... document.insert_region_after(region, clone)
+
+ >>> source = """\
+ ... This is my clone:
+ ...
+ ... clone: 1, 2, 3
+ ...
+ ... I want some copies of my clone.
+ ...
+ ... For example, I'd like a clone before *here*.
+ ...
+ ... I'd also like a clone after *here*.
+ ... """
+ >>> document = manuel.Document(source)
+ >>> document.process_with(m)
+ >>> [(r.source, r.provenance) for r in document]
+ [('This is my clone:\n\n', None),
+ ('clone: 1, 2, 3\n', None),
+ ('clone: 1, 2, 3\n', 'cloned to go before'),
+ ("\nI want some copies of my clone.\n\nFor example, I'd like a clone before *here*.\n\nI'd also like a clone after *here*.\n", None),
+ ('clone: 1, 2, 3\n', 'cloned to go after')]
Property changes on: manuel/trunk/src/manuel/README.txt
___________________________________________________________________
Name: svn:eol-style
+ native
Added: manuel/trunk/src/manuel/__init__.py
===================================================================
--- manuel/trunk/src/manuel/__init__.py (rev 0)
+++ manuel/trunk/src/manuel/__init__.py 2008-10-17 21:31:23 UTC (rev 92347)
@@ -0,0 +1,289 @@
+import imp
+import re
+
+
+def absolute_import(name):
+ return imp.load_module(name, *imp.find_module(name))
+
+
+def newlineify(s):
+ if s[-1] != '\n':
+ s += '\n'
+ return s
+
+
+class Region(object):
+ """A portion of source found via regular expression."""
+
+ parsed = None
+ evaluated = None
+ formatted = None
+
+ def __init__(self, lineno, source, start_match=None, end_match=None,
+ provenance=None):
+ self.lineno = lineno
+ self.source = newlineify(source)
+ self.start_match = start_match
+ self.end_match = end_match
+ self.provenance = provenance
+
+ def copy(self):
+ """Private utility function to make a copy of this region.
+ """
+ copy = Region(self.lineno, self.source, provenance=self.provenance)
+ copy.parsed = self.parsed
+ copy.evaluated = self.evaluated
+ copy.formatted = self.formatted
+ return copy
+
+
+def find_line(region, index):
+ return region[:index].count('\n') + 1
+
+
+def check_region_start(region, match):
+ if match.start() != 0 \
+ and region.source[match.start()-1] != '\n':
+ raise ValueError(
+ 'Regions must start at the begining of a line.')
+
+
+def check_region_end(region, match):
+ if match.end() != len(region.source) \
+ and region.source[match.end()] != '\n':
+ raise ValueError(
+ 'Regions must end at the ending of a line.')
+
+
+def lines_to_string(lines):
+ return '\n'.join(lines) + '\n'
+
+
+def make_string_into_lines(s):
+ lines = newlineify(s).split('\n')
+ assert lines[-1] == ''
+ del lines[-1]
+ return lines
+
+
+def break_up_region(original, new, parsed):
+ assert original.parsed is None
+ lines = make_string_into_lines(original.source)
+
+ new_regions = []
+
+ # figure out if there are any lines before the given region
+ before_lines = lines[:new.lineno-original.lineno]
+ if before_lines:
+ new_regions.append(
+ Region(original.lineno, lines_to_string(before_lines)))
+
+ # put in the parsed
+ new.parsed = parsed
+ new_regions.append(new)
+
+ # figure out if there are any lines after the given region
+ assert new.source[-1] == '\n', 'all lines must end with a newline'
+ lines_in_new = new.source.count('\n')
+ after_lines = lines[len(before_lines)+lines_in_new:]
+ if after_lines:
+ first_line_after_new = new.lineno + lines_in_new
+ new_regions.append(
+ Region(first_line_after_new, lines_to_string(after_lines)))
+
+ assert original.source.count('\n') == \
+ sum(r.source.count('\n') for r in new_regions)
+ return new_regions
+
+
+class Document(object):
+
+ def __init__(self, source, location='<memory>'):
+ self.source = newlineify(source)
+ self.location = location
+
+ self.regions = [Region(lineno=1, source=source)]
+ self.shadow_regions = []
+
+ def find_regions(self, start, end=None):
+ def compile(regex):
+ if regex is not None and isinstance(regex, basestring):
+ regex = re.compile(regex)
+ return regex
+
+ start = compile(start)
+ end = compile(end)
+
+ results = []
+ for region in self.regions:
+ # can't parse things that have already been parsed
+ if region.parsed:
+ continue
+
+ for start_match in re.finditer(start, region.source):
+ first_lineno = region.lineno + find_line(
+ region.source, start_match.start()) - 1
+ check_region_start(region, start_match)
+
+ if end is None:
+ end_match = None
+ check_region_end(region, start_match)
+ text = start_match.group()
+ else:
+ end_match = end.search(region.source, start_match.end())
+ check_region_end(region, end_match)
+ text = region.source[start_match.start():end_match.end()]
+
+ if text[-1] != '\n':
+ text += '\n'
+
+ new_region = Region(first_lineno, text, start_match, end_match)
+ self.shadow_regions.append(new_region)
+ results.append(new_region)
+
+ return results
+
+ def split_region(self, region, lineno):
+ lineno -= region.lineno
+ assert lineno > 0
+ assert region in self.regions
+ assert region.parsed == region.evaluated == region.formatted == None
+ lines = make_string_into_lines(region.source)
+ source1 = lines_to_string(lines[:lineno])
+ source2 = lines_to_string(lines[lineno:])
+ region_index = self.regions.index(region)
+ del self.regions[region_index]
+ lines_in_source1 = source1.count('\n')
+ region1 = Region(region.lineno, source1)
+ region2 = Region(region.lineno+lines_in_source1, source2)
+ self.regions.insert(region_index, region2)
+ self.regions.insert(region_index, region1)
+ return region1, region2
+
+ # XXX this method needs a better name
+ def replace_region(self, to_be_replaced, parsed):
+ new_regions = []
+ old_regions = list(self.regions)
+ while old_regions:
+ region = old_regions.pop(0)
+ if region.lineno == to_be_replaced.lineno:
+ assert not region.parsed
+ new_regions.extend(break_up_region(
+ region, to_be_replaced, parsed))
+ break
+ elif region.lineno > to_be_replaced.lineno: # we "overshot"
+ assert not new_regions[-1].parsed
+ to_be_broken = new_regions[-1]
+ del new_regions[-1]
+ new_regions.extend(break_up_region(
+ to_be_broken, to_be_replaced, parsed))
+ new_regions.append(region)
+ break
+
+ new_regions.append(region)
+ else:
+ # we didn't make any replacements, so the parsed data must be for
+ # the very last region, which also must not have been parsed yet
+ assert not region.parsed
+ del new_regions[-1]
+ new_regions.extend(break_up_region(
+ region, to_be_replaced, parsed))
+
+ new_regions.extend(old_regions)
+ self.regions = new_regions
+
+ def insert_region(self, where, marker_region, new_region):
+ if new_region in self.regions:
+ raise ValueError(
+ 'Only regions not already in the document may be inserted.')
+ if new_region in self.shadow_regions:
+ raise ValueError(
+ 'Regions regurned by "find_regions" can not be directly '
+ 'inserted into a document. Use "replace_region" instead.')
+
+ for index, region in enumerate(self.regions):
+ if region is marker_region:
+ if where == 'after':
+ index += 1
+ self.regions.insert(index, new_region)
+ break
+
+ def remove_region(self, region):
+ self.regions.remove(region)
+
+ def insert_region_before(self, marker_region, new_region):
+ self.insert_region('before', marker_region, new_region)
+
+ def insert_region_after(self, marker_region, new_region):
+ self.insert_region('after', marker_region, new_region)
+
+
+ def do_with(self, things):
+ """Private helper for other do_* functions.
+ """
+ for timing, thing in sorted(things):
+ thing(self)
+
+ def parse_with(self, m):
+ self.do_with(m.parsers)
+
+ def evaluate_with(self, m):
+ self.do_with(m.evaluaters)
+
+ def format_with(self, m):
+ self.do_with(m.formatters)
+
+ def process_with(self, m):
+ """Run all phases of document processing using a Manuel instance.
+ """
+ self.parse_with(m)
+ self.evaluate_with(m)
+ self.format_with(m)
+
+ def formatted(self):
+ """Return a string of all non-boolean-false formatted regions.
+ """
+ return ''.join(region.formatted for region in self if region.formatted)
+
+ def __iter__(self):
+ """Iterate over all regions of the document.
+ """
+ return iter(self.regions)
+
+
+class Manuel(object):
+
+ def __init__(self):
+ self.parsers = []
+ self.evaluaters = []
+ self.formatters = []
+
+ def parser(self, func=None, timing=None):
+ return self.thinger(self.parsers, func, timing)
+
+ def evaluater(self, func=None, timing=None):
+ return self.thinger(self.evaluaters, func, timing)
+
+ def formatter(self, func=None, timing=None):
+ return self.thinger(self.formatters, func, timing)
+
+ def thinger(self, things, func, timing):
+ """Private helper for adding functions to a phase.
+ """
+ if func is None:
+ # the decorator is being called prior to being used as a decorator,
+ # return a callable that can be called to provide the function
+ # to be decorated
+ return lambda func: self.thinger(things, func, timing=timing)
+
+ assert timing in ('early', 'late', None)
+ if timing == None:
+ # arbitrarily chosen string that sorts between "early" and "late"
+ timing = 'k'
+
+ things.append((timing, func))
+
+ def extend(self, other):
+ self.parsers.extend(other.parsers)
+ self.evaluaters.extend(other.evaluaters)
+ self.formatters.extend(other.formatters)
Property changes on: manuel/trunk/src/manuel/__init__.py
___________________________________________________________________
Name: svn:keywords
+ Id
Name: svn:eol-style
+ native
Added: manuel/trunk/src/manuel/doctest.py
===================================================================
--- manuel/trunk/src/manuel/doctest.py (rev 0)
+++ manuel/trunk/src/manuel/doctest.py 2008-10-17 21:31:23 UTC (rev 92347)
@@ -0,0 +1,77 @@
+from zope.testing import doctest
+import StringIO
+#import UserDict
+import manuel
+import os.path
+
+
+class DocTestResult(StringIO.StringIO):
+ pass
+
+
+class SharedGlobs(dict):
+
+ def copy(self):
+ return self
+
+
+def Manuel(optionflags=0, checker=None):
+ m = manuel.Manuel()
+ m.runner = doctest.DocTestRunner(optionflags=optionflags, checker=checker)
+ m.debug_runner = doctest.DebugRunner(optionflags=optionflags)
+ m.globs = SharedGlobs()
+ m.debug = False
+
+ @m.parser
+ def parse(document):
+ for region in list(document):
+ if region.parsed:
+ continue
+ region_start = region.lineno
+ region_end = region.lineno + region.source.count('\n')
+ for chunk in doctest.DocTestParser().parse(region.source):
+ if isinstance(chunk, basestring):
+ continue
+ chunk_line_count = (chunk.source.count('\n')
+ + chunk.want.count('\n'))
+
+ split_line_1 = region_start + chunk.lineno
+ split_line_2 = split_line_1 + chunk_line_count
+
+ # if there is some source we need to trim off the front...
+ if split_line_1 > region.lineno:
+ _, region = document.split_region(region, split_line_1)
+
+ if split_line_2 <= region_end:
+ found, region = document.split_region(region, split_line_2)
+ document.replace_region(found, chunk)
+
+ assert region in document
+
+ @m.evaluater
+ def evaluate(document):
+ for region in document:
+ if not isinstance(region.parsed, doctest.Example):
+ continue
+ result = DocTestResult()
+ test_name = os.path.split(document.location)[1]
+ if m.debug:
+ runner = m.debug_runner
+ else:
+ runner = m.runner
+
+ runner.DIVIDER = '' # disable unwanted result formatting
+ runner.run(
+ doctest.DocTest([region.parsed], m.globs, test_name,
+ document.location, 0, None),
+ out=result.write, clear_globs=False)
+ region.evaluated = result
+
+ @m.formatter
+ def format(document):
+ for region in document:
+ if not isinstance(region.evaluated, DocTestResult):
+ continue
+ region.formatted = region.evaluated.getvalue().lstrip()
+
+ return m
Property changes on: manuel/trunk/src/manuel/doctest.py
___________________________________________________________________
Name: svn:keywords
+ Id
Name: svn:eol-style
+ native
Added: manuel/trunk/src/manuel/footnote.py
===================================================================
--- manuel/trunk/src/manuel/footnote.py (rev 0)
+++ manuel/trunk/src/manuel/footnote.py 2008-10-17 21:31:23 UTC (rev 92347)
@@ -0,0 +1,77 @@
+import re
+import manuel
+
+FOOTNOTE_REFERENCE_LINE_RE = re.compile(r'^.*\[([^\]]+)]_.*$', re.MULTILINE)
+FOOTNOTE_REFERENCE_RE = re.compile(r'\[([^\]]+)]_')
+FOOTNOTE_DEFINITION_RE = re.compile(
+ r'^\.\.\s*\[\s*([^\]]+)\s*\].*$', re.MULTILINE)
+END_OF_FOOTNOTE_RE = re.compile(r'^\S.*$', re.MULTILINE)
+
+
+class FootnoteReference(object):
+ def __init__(self, names):
+ self.names = names
+
+
+class FootnoteDefinition(object):
+ def __init__(self, name):
+ self.name = name
+
+
+def Manuel():
+ """Factory for Manuel objects that handle reST style footnotes.
+ """
+ m = manuel.Manuel()
+
+ @m.parser(timing='early')
+ def find_footnote_references(document):
+ # find the markers that show where footnotes have been defined.
+ footnote_names = []
+ for region in document.find_regions(FOOTNOTE_DEFINITION_RE):
+ name = region.start_match.group(1)
+ document.replace_region(region, FootnoteDefinition(name))
+ footnote_names.append(name)
+
+ # find the markers that show where footnotes have been referenced.
+ for region in document.find_regions(FOOTNOTE_REFERENCE_LINE_RE):
+ assert region.source.count('\n') == 1
+ names = FOOTNOTE_REFERENCE_RE.findall(region.source)
+ for name in names:
+ if name not in footnote_names:
+ raise RuntimeError('Unknown footnote: %r' % name)
+
+ assert names
+ document.replace_region(region, FootnoteReference(names))
+
+ @m.parser(timing='late')
+ def do_footnotes(document):
+ """Copy footnoted items into their appropriate position.
+ """
+ # first find all the regions that are in footnotes
+ footnotes = {}
+ name = None
+ for region in list(document):
+ if isinstance(region.parsed, FootnoteDefinition):
+ name = region.parsed.name
+ footnotes[name] = []
+ document.remove_region(region)
+ continue
+
+ if END_OF_FOOTNOTE_RE.search(region.source):
+ name = None
+
+ if name is not None:
+ footnotes[name].append(region)
+ document.remove_region(region)
+
+ # now make copies of the footnotes in the right places
+ for region in list(document):
+ if not isinstance(region.parsed, FootnoteReference):
+ continue
+ names = region.parsed.names
+ for name in names:
+ for footnoted in footnotes[name]:
+ document.insert_region_before(region, footnoted.copy())
+ document.remove_region(region)
+
+ return m
Property changes on: manuel/trunk/src/manuel/footnote.py
___________________________________________________________________
Name: svn:keywords
+ Id
Name: svn:eol-style
+ native
Added: manuel/trunk/src/manuel/footnote.txt
===================================================================
--- manuel/trunk/src/manuel/footnote.txt (rev 0)
+++ manuel/trunk/src/manuel/footnote.txt 2008-10-17 21:31:23 UTC (rev 92347)
@@ -0,0 +1,98 @@
+=========
+Footnotes
+=========
+
+The manuel.footnote module provides an implementation of reST footnote
+handling.
+
+ >>> import manuel.footnote
+ >>> m = manuel.footnote.Manuel()
+
+We'll try the footnotes out by combining them with doctests.
+
+ >>> import manuel.doctest
+ >>> m.extend(manuel.doctest.Manuel())
+ >>> import manuel
+ >>> document = manuel.Document("""\
+ ... Here we reference a footnote. [1]_
+ ...
+ ... >>> x
+ ... 42
+ ...
+ ... Here we reference another. [2]_
+ ...
+ ... >>> x
+ ... 100
+ ...
+ ... .. [1] This is a test footnote definition.
+ ...
+ ... >>> x = 42
+ ...
+ ... .. [2] This is another test footnote definition.
+ ...
+ ... >>> x = 100
+ ...
+ ... .. [3] This is a footnote that should never be executed.
+ ...
+ ... >>> raise RuntimeError('nooooo!')
+ ... """)
+
+ >>> document.process_with(m)
+
+Since all the examples in the doctest above are correct, we expect no errors.
+
+ >>> for region in document:
+ ... if region.formatted:
+ ... print '-'*70
+ ... print region.formatted,
+
+
+The order of examples in footnotes is preserved. If not, the document below
+will generate an error because "a" won't be defined when "b = a + 1" is
+evaluated.
+
+ >>> document = manuel.Document("""
+ ... Here we want some imports to be done. [foo]_
+ ...
+ ... >>> a + b
+ ... 3
+ ...
+ ... A little prose to separate the examples.
+ ...
+ ... .. [foo] Do something
+ ...
+ ... >>> a = 1
+ ...
+ ... >>> b = a + 1
+ ...
+ ... """)
+ >>> document.process_with(m)
+ >>> print document.formatted()
+
+It is possible to reference more than one footnote on a single line.
+
+ >>> document = manuel.Document("""
+ ... Here we want some imports to be done. [1]_ [2]_ [3]_
+ ...
+ ... >>> z
+ ... 105
+ ...
+ ... A little prose to separate the examples.
+ ...
+ ... .. [1] Do something
+ ...
+ ... >>> w = 3
+ ...
+ ... .. [2] Do something
+ ...
+ ... >>> x = 5
+ ...
+ ... .. [3] Do something
+ ...
+ ... >>> y = 7
+ ...
+ ... >>> z = w * x * y
+ ...
+ ... """)
+ >>> document.process_with(m)
+ >>> print document.formatted()
Property changes on: manuel/trunk/src/manuel/footnote.txt
___________________________________________________________________
Name: svn:eol-style
+ native
Added: manuel/trunk/src/manuel/testing.py
===================================================================
--- manuel/trunk/src/manuel/testing.py (rev 0)
+++ manuel/trunk/src/manuel/testing.py 2008-10-17 21:31:23 UTC (rev 92347)
@@ -0,0 +1,90 @@
+from zope.testing import doctest
+import manuel
+import unittest
+
+
+__all__ = ['TestSuite']
+
+class TestCase(unittest.TestCase):
+ def __init__(self, m, document, setUp=None, tearDown=None,
+ globs=None):
+ unittest.TestCase.__init__(self)
+ self.manuel = m
+ self.document = document
+ self.setUp_func = setUp
+ self.tearDown_func = tearDown
+ self.globs = globs
+
+ # we want to go ahead and do the parse phase so the countTestCases
+ # method can get a good idea of how many tests there are
+ document.parse_with(m)
+
+ def setUp(self):
+ if self.setUp_func is not None:
+ self.setUp_func(self.manuel)
+
+ def tearDown(self):
+ if self.tearDown_func is not None:
+ self.tearDown_func(self.manuel)
+
+ def runTest(self):
+ self.document.evaluate_with(self.manuel)
+ self.document.format_with(self.manuel)
+ results = [r.formatted for r in self.document if r.formatted]
+ if results:
+ DIVIDER = '-'*70 + '\n'
+ raise self.failureException(
+ '\n' + DIVIDER + DIVIDER.join(results))
+
+ def debug(self):
+ self.setUp()
+ self.manuel.debug = True
+ self.document.evaluate_with(self.manuel)
+ self.tearDown()
+
+ def countTestCases(self):
+ return len([r for r in self.document if r.parsed])
+
+ def id(self):
+ return self.document.location
+
+ def shortDescription(self):
+ return "Manuel Test: " + self.document.location
+
+ __str__ = __repr__ = shortDescription
+
+
+def TestSuite(m, *paths, **kws):
+ """A unittest suite that processes files with Manuel
+
+ The path to each document file is given as a string.
+
+ A number of options may be provided as keyword arguments:
+
+ `setUp`
+ A set-up function. This is called before running the tests in each file.
+ The setUp function will be passed a Manuel object. The setUp function
+ can access the test globals as the `globs` attribute of the instance
+ passed.
+
+ `tearDown`
+ A tear-down function. This is called after running the tests in each
+ file. The tearDown function will be passed a Manuel object. The
+ tearDown function can access the test globals as the `globs` attribute of
+ the instance passed.
+
+ `globs`
+ A dictionary containing initial global variables for the tests.
+ """
+
+ suite = unittest.TestSuite()
+
+ # inspect the stack frame to find the module that called this function
+ calling_module = doctest._normalize_module(None, depth=2)
+
+ for path in paths:
+ abs_path = doctest._module_relative_path(calling_module, path)
+ document = manuel.Document(open(abs_path).read(), location=abs_path)
+ suite.addTest(TestCase(m, document, **kws))
+
+ return suite
Property changes on: manuel/trunk/src/manuel/testing.py
___________________________________________________________________
Name: svn:keywords
+ Id
Name: svn:eol-style
+ native
Added: manuel/trunk/src/manuel/tests.py
===================================================================
--- manuel/trunk/src/manuel/tests.py (rev 0)
+++ manuel/trunk/src/manuel/tests.py 2008-10-17 21:31:23 UTC (rev 92347)
@@ -0,0 +1,30 @@
+from zope.testing import doctest
+from zope.testing import renormalizing
+import manuel
+import manuel.doctest
+import manuel.testing
+import re
+import unittest
+
+#doctest = manuel.absolute_import('doctest')
+
+def test_suite():
+ optionflags = doctest.NORMALIZE_WHITESPACE | doctest.ELLIPSIS
+ checker = renormalizing.RENormalizing([
+ (re.compile(r'<zope\.testing\.doctest\.'), '<doctest.'),
+ ])
+ suite = unittest.TestSuite()
+
+ tests = ['README.txt', 'footnote.txt']
+
+ # Run the tests once with just doctest.
+ README = doctest.DocFileSuite(*tests, **{
+ 'optionflags': optionflags,
+ 'checker': checker})
+ suite.addTest(README)
+
+ # Run them again with manuel.
+ m = manuel.doctest.Manuel(optionflags=optionflags, checker=checker)
+ suite.addTest(manuel.testing.TestSuite(m, *tests))
+
+ return suite
Property changes on: manuel/trunk/src/manuel/tests.py
___________________________________________________________________
Name: svn:keywords
+ Id
Name: svn:eol-style
+ native
More information about the Checkins
mailing list