[Checkins] SVN: z3c.recipe.filetemplate/trunk/ merge from branch (and I forgot how annoying this was in svn): svn merge -r 99568:99622 svn+ssh://svn.zope.org/repos/main/z3c.recipe.filetemplate/branches/gary-powertools
Gary Poster
gary at modernsongs.com
Thu Apr 30 14:56:10 EDT 2009
Log message for revision 99623:
merge from branch (and I forgot how annoying this was in svn): svn merge -r 99568:99622 svn+ssh://svn.zope.org/repos/main/z3c.recipe.filetemplate/branches/gary-powertools
Changed:
_U z3c.recipe.filetemplate/trunk/
U z3c.recipe.filetemplate/trunk/CHANGES.txt
A z3c.recipe.filetemplate/trunk/LICENSE.txt
U z3c.recipe.filetemplate/trunk/setup.py
U z3c.recipe.filetemplate/trunk/z3c/recipe/filetemplate/README.txt
U z3c.recipe.filetemplate/trunk/z3c/recipe/filetemplate/__init__.py
U z3c.recipe.filetemplate/trunk/z3c/recipe/filetemplate/tests.py
U z3c.recipe.filetemplate/trunk/z3c/recipe/filetemplate/tests.txt
-=-
Property changes on: z3c.recipe.filetemplate/trunk
___________________________________________________________________
Added: svn:externals
+ bootstrap svn://svn.zope.org/repos/main/zc.buildout/trunk/bootstrap
Modified: z3c.recipe.filetemplate/trunk/CHANGES.txt
===================================================================
--- z3c.recipe.filetemplate/trunk/CHANGES.txt 2009-04-30 18:50:14 UTC (rev 99622)
+++ z3c.recipe.filetemplate/trunk/CHANGES.txt 2009-04-30 18:56:10 UTC (rev 99623)
@@ -1,12 +1,37 @@
+=======
Changes
=======
-1.1 (unreleased)
-----------------
+2.0 (2009-04-30)
+================
-...
+--------
+Features
+--------
+- Store your template files in a separate directory structure, using the
+ ``source-directory`` option.
+
+- Specify multiple files automatically with globs.
+
+- Templates can reference other buildout sections using the usual syntax, e.g.
+ ${buildout:parts}
+
+- Share options with other sections using the typical ``extends`` option.
+
+- Create destination directories automatically.
+
+- Define option values for templates dynamically in Python with the
+ ``interpreted-options`` option.
+
+- Get paths for eggs by specifying ``eggs`` and ``extra-paths``, just like
+ zc.recipe.egg script recipe. These are available in template options in
+ colon-delimited, space-delimited, and quoted variants. You can also build
+ your own using the ``interpreted-options`` feature.
+
+- Templates are not processed if there are no changes to them or the buildout.
+
1.0 (2007-09-30)
-----------------
+================
Initial release.
Copied: z3c.recipe.filetemplate/trunk/LICENSE.txt (from rev 99622, z3c.recipe.filetemplate/branches/gary-powertools/LICENSE.txt)
===================================================================
--- z3c.recipe.filetemplate/trunk/LICENSE.txt (rev 0)
+++ z3c.recipe.filetemplate/trunk/LICENSE.txt 2009-04-30 18:56:10 UTC (rev 99623)
@@ -0,0 +1,54 @@
+Zope Public License (ZPL) Version 2.1
+-------------------------------------
+
+A copyright notice accompanies this license document that
+identifies the copyright holders.
+
+This license has been certified as open source. It has also
+been designated as GPL compatible by the Free Software
+Foundation (FSF).
+
+Redistribution and use in source and binary forms, with or
+without modification, are permitted provided that the
+following conditions are met:
+
+1. Redistributions in source code must retain the
+ accompanying copyright notice, this list of conditions,
+ and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the accompanying
+ copyright notice, this list of conditions, and the
+ following disclaimer in the documentation and/or other
+ materials provided with the distribution.
+
+3. Names of the copyright holders must not be used to
+ endorse or promote products derived from this software
+ without prior written permission from the copyright
+ holders.
+
+4. The right to distribute this software or to use it for
+ any purpose does not give you the right to use
+ Servicemarks (sm) or Trademarks (tm) of the copyright
+ holders. Use of them is covered by separate agreement
+ with the copyright holders.
+
+5. If any files are modified, you must cause the modified
+ files to carry prominent notices stating that you changed
+ the files and the date of any change.
+
+Disclaimer
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS''
+ AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT
+ NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
+ AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN
+ NO EVENT SHALL THE COPYRIGHT HOLDERS BE
+ LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
+ OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
+ DAMAGE.
\ No newline at end of file
Modified: z3c.recipe.filetemplate/trunk/setup.py
===================================================================
--- z3c.recipe.filetemplate/trunk/setup.py 2009-04-30 18:50:14 UTC (rev 99622)
+++ z3c.recipe.filetemplate/trunk/setup.py 2009-04-30 18:56:10 UTC (rev 99623)
@@ -1,3 +1,17 @@
+##############################################################################
+#
+# Copyright (c) 2007-2009 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
import os
from setuptools import setup, find_packages
@@ -5,12 +19,14 @@
return open(os.path.join(os.path.dirname(__file__), *rnames)).read()
setup(name='z3c.recipe.filetemplate',
- version = '1.1dev',
+ version = '2.0',
license='ZPL 2.1',
url='http://pypi.python.org/pypi/z3c.recipe.filetemplate',
description="zc.buildout recipe for creating files from file templates",
author='Philipp von Weitershausen',
author_email='philipp at weitershausen.de',
+ maintainer='Gary Poster',
+ maintainer_email='gary.poster at canonical.com',
long_description=(read('z3c', 'recipe', 'filetemplate', 'README.txt')
+ '\n\n' +
read('CHANGES.txt')),
@@ -27,6 +43,7 @@
namespace_packages=['z3c', 'z3c.recipe'],
install_requires=['setuptools',
'zc.buildout',
+ 'zc.recipe.egg',
],
zip_safe=True,
entry_points="""
Modified: z3c.recipe.filetemplate/trunk/z3c/recipe/filetemplate/README.txt
===================================================================
--- z3c.recipe.filetemplate/trunk/z3c/recipe/filetemplate/README.txt 2009-04-30 18:50:14 UTC (rev 99622)
+++ z3c.recipe.filetemplate/trunk/z3c/recipe/filetemplate/README.txt 2009-04-30 18:56:10 UTC (rev 99623)
@@ -1,14 +1,26 @@
+``z3c.recipe.filetemplate``
+***************************
+
+===========
+Basic Usage
+===========
+
With the ``z3c.recipe.filetemplate`` buildout recipe you can automate
the generation of text files from templates. Upon execution, the
-recipe will read a number of template files, perform a simple variable
+recipe will read a number of template files, perform variable
substitution and write the result to the corresponding output files.
+The recipe has several features, but it always takes template files with a
+``.in`` suffix, processes the template, and writes out the file to the desired
+location with the same file mode, and the same name but without the ``.in``
+suffix.
+
For example, consider this simple template for a text file:
- >>> write(sample_buildout, 'helloworld.txt.in',
- ... """
- ... Hello ${world}!
- ... """)
+ >>> write(sample_buildout, 'helloworld.txt.in',
+ ... """
+ ... Hello ${world}!
+ ... """)
Now let's create a buildout configuration so that we can substitute
the values in this file. All we have to do is define a part that uses
@@ -17,22 +29,395 @@
whitespace). Then we can add arbitrary parameters to the section.
Those will be used to fill the variables in the template:
- >>> write(sample_buildout, 'buildout.cfg',
- ... """
- ... [buildout]
- ... parts = file
- ...
- ... [file]
- ... recipe = z3c.recipe.filetemplate
- ... files = helloworld.txt
- ... world = Philipp
- ... """)
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = message
+ ...
+ ... [message]
+ ... recipe = z3c.recipe.filetemplate
+ ... files = helloworld.txt
+ ... world = Philipp
+ ... """)
After executing buildout, we can see that ``$world`` has indeed been
replaced by ``Philipp``:
- >>> print system(buildout)
- Installing file.
+ >>> print system(buildout)
+ Installing message.
- >>> cat(sample_buildout, 'helloworld.txt')
- Hello Philipp!
+ >>> cat(sample_buildout, 'helloworld.txt')
+ Hello Philipp!
+
+Note that the output file uses the same permission bits as found on the input
+file.
+
+ >>> import stat
+ >>> import os
+ >>> input = os.path.join(sample_buildout, 'helloworld.txt.in')
+ >>> output = input[:-3]
+ >>> os.chmod(input, 0755)
+ >>> stat.S_IMODE(os.stat(input).st_mode) == 0755
+ True
+ >>> stat.S_IMODE(os.stat(output).st_mode) == 0755
+ False
+ >>> print system(buildout)
+ Uninstalling message.
+ Installing message.
+ >>> stat.S_IMODE(os.stat(output).st_mode) == 0755
+ True
+
+Source Folders and Globs
+========================
+
+By default, the recipe looks for a ``.in`` file relative to the buildout root,
+and places it in the same folder relative to the buildout root. However, if
+you don't want to clutter up the destination folder, you can add a prefix to
+the source folder. Here is an example.
+
+First, we specify a ``source-directory`` in the buildout. You can specify
+``files`` as a filter if desired, but by default it will find any file (ending
+with ".in").
+
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = message
+ ...
+ ... [message]
+ ... recipe = z3c.recipe.filetemplate
+ ... source-directory = template
+ ... world = Philipp
+ ... """)
+
+Now we'll make a "template" directory, as listed in the buildout configuration
+above, and populate it for our example.
+
+ >>> mkdir(sample_buildout, 'template')
+ >>> mkdir(sample_buildout, 'template', 'etc')
+ >>> mkdir(sample_buildout, 'template', 'bin')
+ >>> write(sample_buildout, 'template', 'etc', 'helloworld.conf.in',
+ ... """
+ ... Hello ${world} from the etc dir!
+ ... """)
+ >>> write(sample_buildout, 'template', 'bin', 'helloworld.sh.in',
+ ... """
+ ... Hello ${world} from the bin dir!
+ ... """)
+ >>> os.chmod(
+ ... os.path.join(
+ ... sample_buildout, 'template', 'bin', 'helloworld.sh.in'),
+ ... 0711)
+
+Notice that, before running buildout, the ``helloworld.txt`` file is still
+around, we don't have an etc directory, and the bin directory doesn't have our
+``helloworld.sh``.
+
+ >>> ls(sample_buildout)
+ - .installed.cfg
+ d bin
+ - buildout.cfg
+ d develop-eggs
+ d eggs
+ - helloworld.txt
+ - helloworld.txt.in
+ d parts
+ d template
+ >>> ls(sample_buildout, 'bin')
+ - buildout
+
+Now we install. The old "helloworld.txt" is gone, and we now see etc. Note
+that, for the destination, intermediate folders are created if they do not
+exist.
+
+ >>> print system(buildout)
+ Uninstalling message.
+ Installing message.
+ >>> ls(sample_buildout)
+ - .installed.cfg
+ d bin
+ - buildout.cfg
+ d develop-eggs
+ d eggs
+ d etc
+ - helloworld.txt.in
+ d parts
+ d template
+
+The files exist and have the content we expect.
+
+ >>> ls(sample_buildout, 'bin')
+ - buildout
+ - helloworld.sh
+ >>> cat(sample_buildout, 'bin', 'helloworld.sh')
+ Hello Philipp from the bin dir!
+ >>> stat.S_IMODE(os.stat(os.path.join(
+ ... sample_buildout, 'bin', 'helloworld.sh')).st_mode) == 0711
+ True
+ >>> ls(sample_buildout, 'etc')
+ - helloworld.conf
+ >>> cat(sample_buildout, 'etc', 'helloworld.conf')
+ Hello Philipp from the etc dir!
+
+If you use the ``files`` option along with ``source-directory``, it becomes a
+filter. Every target file must match at least one of the names in ``files``.
+Therefore, if we only build .sh files, the etc directory will disappear.
+
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = message
+ ...
+ ... [message]
+ ... recipe = z3c.recipe.filetemplate
+ ... source-directory = template
+ ... files = *.sh
+ ... world = Philipp
+ ... """)
+
+ >>> print system(buildout)
+ Uninstalling message.
+ Installing message.
+ >>> ls(sample_buildout)
+ - .installed.cfg
+ d bin
+ - buildout.cfg
+ d develop-eggs
+ d eggs
+ - helloworld.txt.in
+ d parts
+ d template
+
+ >>> ls(sample_buildout, 'bin')
+ - buildout
+ - helloworld.sh
+
+Also note that, if you use a source directory and your ``files`` specify a
+directory, the directory must match precisely.
+
+==============
+Advanced Usage
+==============
+
+Substituting from Other Sections
+================================
+
+Substitutions can also come from other sections in the buildout, using the
+standard buildout syntax, but used in the template. Notice
+``${buildout:parts}`` in the template below.
+
+ >>> write(sample_buildout, 'helloworld.txt.in',
+ ... """
+ ... Hello ${world}. I used these parts: ${buildout:parts}.
+ ... """)
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = message
+ ...
+ ... [message]
+ ... recipe = z3c.recipe.filetemplate
+ ... files = helloworld.txt
+ ... world = Philipp
+ ... """)
+
+ >>> print system(buildout)
+ Uninstalling message.
+ Installing message.
+
+ >>> cat(sample_buildout, 'helloworld.txt')
+ Hello Philipp. I used these parts: message.
+
+Sharing variables
+=================
+
+The recipe allows extending one or more sections, to decrease repetition, using
+the ``extends`` option. For instance, consider the following buildout.
+
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = message
+ ...
+ ... [template_defaults]
+ ... mygreeting = Hi
+ ... myaudience = World
+ ...
+ ... [message]
+ ... recipe = z3c.recipe.filetemplate
+ ... files = helloworld.txt
+ ... extends = template_defaults
+ ...
+ ... myaudience = everybody
+ ... """)
+
+The "message" section now has values extended from the "template_defaults"
+section, and overwritten locally. A template of
+``${mygreeting}, ${myaudience}!``...
+
+ >>> write(sample_buildout, 'helloworld.txt.in',
+ ... """
+ ... ${mygreeting}, ${myaudience}!
+ ... """)
+
+...would thus result in ``Hi, everybody!``.
+
+ >>> print system(buildout)
+ Uninstalling message.
+ Installing message.
+
+ >>> cat(sample_buildout, 'helloworld.txt')
+ Hi, everybody!
+
+Specifying paths
+================
+
+You can specify eggs and extra-paths in the recipe. If you do, three
+predefined options will be available in the recipe's options for the template.
+If "paths" are the non-zip paths, and "all_paths" are all paths, then the
+options would be defined roughly as given here:
+
+ os-paths: (os.pathsep).join(paths)
+ string-paths: ', '.join(repr(p) for p in all_paths)
+ space-paths: ' '.join(paths)
+
+For instance, consider this example.
+
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = message
+ ...
+ ... [message]
+ ... recipe = z3c.recipe.filetemplate
+ ... files = helloworld.txt
+ ... eggs = demo<0.3
+ ...
+ ... find-links = %(server)s
+ ... index = %(server)s/index
+ ... """ % dict(server=link_server))
+
+
+ >>> write(sample_buildout, 'helloworld.txt.in',
+ ... """
+ ... Hello! Here are the paths for the ${eggs} eggs.
+ ... OS paths:
+ ... ${os-paths}
+ ... ---
+ ... String paths:
+ ... ${string-paths}
+ ... ---
+ ... Space paths:
+ ... ${space-paths}
+ ... """)
+
+ >>> print system(buildout)
+ Getting distribution for 'demo<0.3'.
+ Got demo 0.2.
+ Getting distribution for 'demoneeded'.
+ Got demoneeded 1.2c1.
+ Uninstalling message.
+ Installing message.
+
+ >>> cat(sample_buildout, 'helloworld.txt') # doctest:+ELLIPSIS
+ Hello! Here are the paths for the demo<0.3 eggs.
+ OS paths:
+ .../eggs/demo-0.2...egg:.../eggs/demoneeded-1.2c1...egg
+ ---
+ String paths:
+ '.../eggs/demo-0.2...egg', '.../eggs/demoneeded-1.2c1...egg'
+ ---
+ Space paths:
+ .../eggs/demo-0.2...egg .../eggs/demoneeded-1.2c1...egg
+
+You can specify extra-paths as well, which will go at the end of the egg paths.
+
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = message
+ ...
+ ... [message]
+ ... recipe = z3c.recipe.filetemplate
+ ... files = helloworld.txt
+ ... eggs = demo<0.3
+ ... extra-paths = ${buildout:directory}/foo
+ ...
+ ... find-links = %(server)s
+ ... index = %(server)s/index
+ ... """ % dict(server=link_server))
+
+ >>> print system(buildout)
+ Uninstalling message.
+ Installing message.
+
+ >>> cat(sample_buildout, 'helloworld.txt') # doctest:+ELLIPSIS
+ Hello! Here are the paths for the demo<0.3 eggs.
+ OS paths:
+ ...demo...:...demoneeded...:.../sample-buildout/foo
+ ---
+ String paths:
+ '...demo...', '...demoneeded...', '.../sample-buildout/foo'
+ ---
+ Space paths:
+ ...demo... ...demoneeded... .../sample-buildout/foo
+
+Defining options in Python
+==========================
+
+You can specify that certain variables should be interpreted as Python using
+``interpreted-options``. This takes zero or more lines. Each line should
+specify an option. It can define immediately (see ``duplicate-os-paths``,
+``foo-paths``, and ``silly-range`` in the example below) or point to an option
+to be interepreted, which can be useful if you want to define a
+multi-line expression (see ``first-interpreted-option`` and
+``message-reversed-is-egassem``).
+
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = message
+ ...
+ ... [message]
+ ... recipe = z3c.recipe.filetemplate
+ ... files = helloworld.txt
+ ... eggs = demo<0.3
+ ... interpreted-options = duplicate-os-paths=(os.pathsep).join(paths)
+ ... foo-paths='FOO'.join(all_paths)
+ ... silly-range = repr(range(5))
+ ... first-interpreted-option
+ ... message-reversed-is-egassem
+ ... first-interpreted-option =
+ ... options['interpreted-options'].split()[0].strip()
+ ... message-reversed-is-egassem=
+ ... ''.join(
+ ... reversed(
+ ... buildout['buildout']['parts']))
+ ... not-interpreted=hello world
+ ...
+ ... find-links = %(server)s
+ ... index = %(server)s/index
+ ... """ % dict(server=link_server))
+
+ >>> write(sample_buildout, 'helloworld.txt.in',
+ ... """
+ ... ${not-interpreted}!
+ ... duplicate-os-paths: ${duplicate-os-paths}
+ ... foo-paths: ${foo-paths}
+ ... silly-range: ${silly-range}
+ ... first-interpreted-option: ${first-interpreted-option}
+ ... message-reversed-is-egassem: ${message-reversed-is-egassem}
+ ... """)
+
+ >>> print system(buildout)
+ Uninstalling message.
+ Installing message.
+
+ >>> cat(sample_buildout, 'helloworld.txt') # doctest:+ELLIPSIS
+ hello world!
+ duplicate-os-paths: ...demo-0.2...egg:...demoneeded-1.2c1...egg
+ foo-paths: ...demo-0.2...eggFOO...demoneeded-1.2c1...egg
+ silly-range: [0, 1, 2, 3, 4]
+ first-interpreted-option: duplicate-os-paths=(os.pathsep).join(paths)
+ message-reversed-is-egassem: egassem
+
Modified: z3c.recipe.filetemplate/trunk/z3c/recipe/filetemplate/__init__.py
===================================================================
--- z3c.recipe.filetemplate/trunk/z3c/recipe/filetemplate/__init__.py 2009-04-30 18:50:14 UTC (rev 99622)
+++ z3c.recipe.filetemplate/trunk/z3c/recipe/filetemplate/__init__.py 2009-04-30 18:56:10 UTC (rev 99623)
@@ -1,46 +1,223 @@
+##############################################################################
+#
+# Copyright (c) 2007-2009 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+import fnmatch
+import logging
import os
+import re
+import stat
import string
-import logging
+import sys
+import traceback
+import zc.recipe.egg
import zc.buildout
+import zc.buildout.easy_install
+ABS_PATH_ERROR = ('%s is an absolute path. Paths must be '
+ 'relative to the buildout directory.')
+
class FileTemplate(object):
def __init__(self, buildout, name, options):
self.buildout = buildout
self.name = name
self.options = options
-
- def install(self, update=False):
+ self.logger=logging.getLogger(self.name)
+ # get defaults from extended sections
+ defaults = {}
+ extends = self.options.get('extends', '').split()
+ extends.reverse()
+ for section_name in extends:
+ defaults.update(self.buildout[section_name])
+ for key, value in defaults.items():
+ self.options.setdefault(key, value)
+ # set up paths for eggs, if given
+ if 'eggs' in self.options:
+ self.eggs = zc.recipe.egg.Scripts(buildout, name, options)
+ orig_distributions, ws = self.eggs.working_set()
+ # we want ws, eggs.extra_paths, eggs._relative_paths
+ all_paths = [
+ zc.buildout.easy_install.realpath(dist.location)
+ for dist in ws]
+ all_paths.sort()
+ all_paths.extend(
+ zc.buildout.easy_install.realpath(path)
+ for path in self.eggs.extra_paths)
+ else:
+ all_paths = []
+ paths = [path for path in all_paths if not path.endswith('.zip')]
+ self.options['os-paths'] = (os.pathsep).join(paths)
+ self.options['string-paths'] = ', '.join(repr(p) for p in all_paths)
+ self.options['space-paths'] = ' '.join(paths)
+ # get and check the files to be created
+ self.filenames = self.options.get('files', '*').split()
+ self.source_dir = self.options.get('source-directory', '').strip()
here = self.buildout['buildout']['directory']
- filenames = self.options['files'].split()
- logger = logging.getLogger(self.name)
-
- for filename in filenames:
+ self.destination_dir = here
+ if self.source_dir:
+ self.recursive = True
+ if os.path.isabs(self.source_dir):
+ self._user_error(ABS_PATH_ERROR, self.source_dir)
+ self.source_dir = zc.buildout.easy_install.realpath(
+ os.path.normpath(os.path.join(here, self.source_dir)))
+ if not self.source_dir.startswith(here):
+ self._user_error(
+ 'source-directory must be within the buildout directory')
+ else:
+ self.recursive = False
+ self.options['source-directory'] = ''
+ self.source_dir = self.buildout['buildout']['directory']
+ source_patterns = []
+ for filename in self.filenames:
if os.path.isabs(filename):
- msg = ('%s is an absolute path. File paths must be '
- 'relative to the buildout directory.' % filename)
- logger.error(msg)
- raise zc.buildout.UserError(msg)
+ self._user_error(ABS_PATH_ERROR, filename)
+ if not zc.buildout.easy_install.realpath(
+ os.path.normpath(os.path.join(self.source_dir, filename))
+ ).startswith(self.source_dir):
+ # path used ../ to get out of buildout dir
+ self._user_error(
+ 'source files must be within the buildout directory')
+ source_patterns.append('%s.in' % filename)
+ unmatched = set(source_patterns)
+ unexpected_dirs = []
+ self.actions = [] # each entry is tuple of
+ # (relative path, source last-modified-time, mode)
+ if self.recursive:
+ def visit(ignored, dirname, names):
+ relative_prefix = dirname[len(self.source_dir)+1:]
+ file_info = {}
+ for name in names:
+ val = os.path.join(relative_prefix, name)
+ source = os.path.join(self.source_dir, val)
+ statinfo = os.stat(source)
+ last_modified = statinfo.st_mtime
+ if stat.S_ISREG(statinfo.st_mode):
+ file_info[name] = (
+ val, last_modified, statinfo.st_mode)
+ found = set()
+ for orig_pattern in source_patterns:
+ parts = orig_pattern.split('/')
+ dir = os.path.sep.join(parts[:-1])
+ pattern = parts[-1]
+ if (dir and
+ relative_prefix != dir and
+ (dir != '.' or relative_prefix != '')):
+ # if a directory is specified, it must match
+ # precisely. We also support the '.' directory.
+ continue
+ matching = fnmatch.filter(file_info, pattern)
+ if matching:
+ unmatched.discard(orig_pattern)
+ found.update(matching)
+ for name in found:
+ self.actions.append(file_info[name])
+ os.path.walk(
+ self.source_dir, visit, None)
+ else:
+ for val in source_patterns:
+ source = zc.buildout.easy_install.realpath(
+ os.path.join(self.source_dir, val))
+ if os.path.exists(source):
+ unmatched.discard(val)
+ statinfo = os.stat(source)
+ last_modified = statinfo.st_mtime
+ if not stat.S_ISREG(statinfo.st_mode):
+ unexpected_dirs.append(source)
+ else:
+ self.actions.append(
+ (val, last_modified, statinfo.st_mode))
+ # This is supposed to be a flag so that when source files change, the
+ # recipe knows to reinstall.
+ self.options['_actions'] = repr(self.actions)
+ if unexpected_dirs:
+ self._user_error(
+ 'Expected file but found directory: %s',
+ ', '.join(unexpected_dirs))
+ if unmatched:
+ self._user_error(
+ 'No template found for these file names: %s',
+ ', '.join(unmatched))
+ # parse interpreted options
+ interpreted = self.options.get('interpreted-options')
+ if interpreted:
+ globs = {'__builtins__': __builtins__, 'os': os, 'sys': sys}
+ locs = {'name': name, 'options': options, 'buildout': buildout,
+ 'paths': paths, 'all_paths': all_paths}
+ for value in interpreted.split('\n'):
+ if value:
+ value = value.split('=', 1)
+ key = value[0].strip()
+ if len(value) == 1:
+ try:
+ expression = options[key]
+ except KeyError:
+ self._user_error(
+ 'Expression for key not found: %s', key)
+ else:
+ expression = value[1]
+ try:
+ evaluated = eval(expression, globs, locs)
+ except:
+ self._user_error(
+ 'Error when evaluating %r expression %r:\n%s',
+ key, expression, traceback.format_exc())
+ if not isinstance(evaluated, basestring):
+ self._user_error(
+ 'Result of evaluating Python expression must be a '
+ 'string. The result of %r expression %r was %r, '
+ 'a %s.',
+ key, expression, evaluated, type(evaluated))
+ options[key] = evaluated
+ def _user_error(self, msg, *args):
+ msg = msg % args
+ self.logger.error(msg)
+ raise zc.buildout.UserError(msg)
- absname = os.path.join(here, filename)
+ def install(self):
+ already_exists = [
+ rel_path for rel_path, last_mod, st_mode in self.actions
+ if os.path.exists(
+ os.path.join(self.destination_dir, rel_path[:-3]))
+ ]
+ if already_exists:
+ self._user_error(
+ 'Destinations already exist: %s. Please make sure that '
+ 'you really want to generate these automatically. Then '
+ 'move them away.', ', '.join(already_exists))
+ for rel_path, last_mod, st_mode in self.actions:
+ source = os.path.join(self.source_dir, rel_path)
+ dest = os.path.join(self.destination_dir, rel_path[:-3])
+ mode=stat.S_IMODE(st_mode)
+ template=open(source).read()
+ template=re.sub(r"\$\{([^:]+?)\}", r"${%s:\1}" % self.name,
+ template)
+ self._create_paths(os.path.dirname(dest))
+ # we process the file first so that it won't be created if there
+ # is a problem.
+ processed = self.options._sub(template, [])
+ result=open(dest, "wt")
+ result.write(processed)
+ result.close()
+ os.chmod(dest, mode)
+ self.options.created(rel_path[:-3])
+ return self.options.created()
- if not os.path.exists(absname + '.in'):
- msg = 'No template found at %s.in.' % filename
- logger.error(msg)
- raise zc.buildout.UserError(msg)
+ def _create_paths(self, path):
+ if not os.path.exists(path):
+ self._create_paths(os.path.dirname(path))
+ os.mkdir(path)
+ self.options.created(path)
- if not update and os.path.exists(absname):
- msg = ('File %s already exists. Please make sure that you '
- 'really want to have it generated automatically. Then '
- 'move it away.' % filename)
- logger.error(msg)
- raise zc.buildout.UserError(msg)
-
- templ = string.Template(open(absname + '.in').read())
- outfile = open(absname, 'w')
- outfile.write(templ.substitute(self.options))
- outfile.close()
- return filenames
-
def update(self):
- return self.install(update=True)
+ pass
Modified: z3c.recipe.filetemplate/trunk/z3c/recipe/filetemplate/tests.py
===================================================================
--- z3c.recipe.filetemplate/trunk/z3c/recipe/filetemplate/tests.py 2009-04-30 18:50:14 UTC (rev 99622)
+++ z3c.recipe.filetemplate/trunk/z3c/recipe/filetemplate/tests.py 2009-04-30 18:56:10 UTC (rev 99623)
@@ -1,8 +1,23 @@
+##############################################################################
+#
+# Copyright (c) 2007-2009 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
import zc.buildout.testing
+import zc.buildout.tests
from zope.testing import doctest
def setUp(test):
- zc.buildout.testing.buildoutSetUp(test)
+ zc.buildout.tests.easy_install_SetUp(test)
zc.buildout.testing.install_develop('z3c.recipe.filetemplate', test)
def test_suite():
Modified: z3c.recipe.filetemplate/trunk/z3c/recipe/filetemplate/tests.txt
===================================================================
--- z3c.recipe.filetemplate/trunk/z3c/recipe/filetemplate/tests.txt 2009-04-30 18:50:14 UTC (rev 99622)
+++ z3c.recipe.filetemplate/trunk/z3c/recipe/filetemplate/tests.txt 2009-04-30 18:56:10 UTC (rev 99623)
@@ -10,35 +10,35 @@
The recipe can subsitute the same set of variables on several files at
the same time:
- >>> write(sample_buildout, 'helloworld.txt.in',
- ... """
- ... Hello ${world}!
- ... """)
+ >>> write(sample_buildout, 'helloworld.txt.in',
+ ... """
+ ... Hello ${world}!
+ ... """)
+
+ >>> write(sample_buildout, 'goodbyeworld.txt.in',
+ ... """
+ ... Goodbye ${world}!
+ ... """)
- >>> write(sample_buildout, 'goodbyeworld.txt.in',
- ... """
- ... Goodbye ${world}!
- ... """)
-
File names are separated by any kind of whitespace:
- >>> write(sample_buildout, 'buildout.cfg',
- ... """
- ... [buildout]
- ... parts = multiple
- ...
- ... [multiple]
- ... recipe = z3c.recipe.filetemplate
- ... files = helloworld.txt
- ... goodbyeworld.txt
- ... world = Philipp
- ... """)
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = multiple
+ ...
+ ... [multiple]
+ ... recipe = z3c.recipe.filetemplate
+ ... files = helloworld.txt
+ ... goodbyeworld.txt
+ ... world = Philipp
+ ... """)
After executing buildout, we can see that ``$world`` has indeed been
replaced by ``Philipp``:
- >>> print system(buildout)
- Installing multiple.
+ >>> print system(buildout)
+ Installing multiple.
Absolute paths
--------------
@@ -46,113 +46,322 @@
The recipe only accepts relative file paths. For example, consider
this invalid buildout configuration:
- >>> write(sample_buildout, 'buildout.cfg',
- ... """
- ... [buildout]
- ... parts = evil
- ...
- ... [evil]
- ... recipe = z3c.recipe.filetemplate
- ... files = /etc/passwd.in
- ... root = me
- ... """)
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = evil
+ ...
+ ... [evil]
+ ... recipe = z3c.recipe.filetemplate
+ ... files = /etc/passwd.in
+ ... root = me
+ ... """)
+
+ >>> print system(buildout)
+ evil: /etc/passwd.in is an absolute path. Paths must be relative to the buildout directory.
+ While:
+ Installing.
+ Getting section evil.
+ Initializing part evil.
+ Error: /etc/passwd.in is an absolute path. Paths must be relative to the buildout directory.
- >>> print system(buildout)
- Uninstalling multiple.
- Installing evil.
- evil: /etc/passwd.in is an absolute path. File paths must be
- relative to the buildout directory.
- While:
- Installing evil.
- Error: /etc/passwd.in is an absolute path. File paths must be
- relative to the buildout directory.
-
-
Missing template
----------------
The recipe will also complain with an error if you specify a file name
for which no template can be found:
- >>> write(sample_buildout, 'buildout.cfg',
- ... """
- ... [buildout]
- ... parts = notthere
- ...
- ... [notthere]
- ... recipe = z3c.recipe.filetemplate
- ... files = doesntexist
- ... """)
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = notthere
+ ...
+ ... [notthere]
+ ... recipe = z3c.recipe.filetemplate
+ ... files = doesntexist
+ ... """)
+
+ >>> print system(buildout)
+ notthere: No template found for these file names: doesntexist.in
+ While:
+ Installing.
+ Getting section notthere.
+ Initializing part notthere.
+ Error: No template found for these file names: doesntexist.in
- >>> print system(buildout)
- Installing notthere.
- notthere: No template found at doesntexist.in.
- While:
- Installing notthere.
- Error: No template found at doesntexist.in.
-
-
Already existing file
---------------------
Another case where the recipe will complain is when you're trying to
replace a file that's already there:
- >>> write(sample_buildout, 'alreadyhere.txt',
- ... """
- ... I'm already here
- ... """)
+ >>> write(sample_buildout, 'alreadyhere.txt',
+ ... """
+ ... I'm already here
+ ... """)
+
+ >>> write(sample_buildout, 'alreadyhere.txt.in',
+ ... """
+ ... I'm the template that's supposed to replace the file above.
+ ... """)
+
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = alreadythere
+ ...
+ ... [alreadythere]
+ ... recipe = z3c.recipe.filetemplate
+ ... files = alreadyhere.txt
+ ... """)
- >>> write(sample_buildout, 'alreadyhere.txt.in',
- ... """
- ... I'm the template that's supposed to replace the file above.
- ... """)
-
- >>> write(sample_buildout, 'buildout.cfg',
- ... """
- ... [buildout]
- ... parts = alreadythere
- ...
- ... [alreadythere]
- ... recipe = z3c.recipe.filetemplate
- ... files = alreadyhere.txt
- ... """)
-
- >>> print system(buildout)
- Installing alreadythere.
- alreadythere: File alreadyhere.txt already exists. Please make sure
- that you really want to have it generated automatically.
- Then move it away.
- While:
+ >>> print system(buildout)
+ Uninstalling multiple.
Installing alreadythere.
- Error: File alreadyhere.txt already exists. Please make sure
- that you really want to have it generated automatically.
- Then move it away.
+ alreadythere: Destinations already exist: alreadyhere.txt.in. Please make
+ sure that you really want to generate these automatically.
+ Then move them away.
+ While:
+ Installing alreadythere.
+ Error: Destinations already exist: alreadyhere.txt.in. Please make sure
+ that you really want to generate these automatically. Then move
+ them away.
-
Missing variables
-----------------
The recipe will also fail to execute if a template refers to variables
that aren't defined in ``buildout.cfg``:
- >>> write(sample_buildout, 'missing.txt.in',
- ... """
- ... Hello ${world}!
- ... """)
+ >>> write(sample_buildout, 'missing.txt.in',
+ ... """
+ ... Hello ${world}!
+ ... """)
+
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = missing
+ ...
+ ... [missing]
+ ... recipe = z3c.recipe.filetemplate
+ ... files = missing.txt
+ ... """)
+
+ >>> print system(buildout)
+ Installing missing.
+ While:
+ Installing missing.
+ Error: Referenced option does not exist: missing world
- >>> write(sample_buildout, 'buildout.cfg',
- ... """
- ... [buildout]
- ... parts = missing
- ...
- ... [missing]
- ... recipe = z3c.recipe.filetemplate
- ... files = missing.txt
- ... """)
+No changes means just an update
+-------------------------------
- >>> print system(buildout)
- Installing missing.
- While:
- Installing missing.
- Error: Missing option: missing:world
+If there are no changes in the buildout section, or in the files it will build,
+the recipe will update, which is a no-op.
+
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = message
+ ...
+ ... [message]
+ ... recipe = z3c.recipe.filetemplate
+ ... files = helloworld.txt
+ ... world = Philipp
+ ... """)
+
+ >>> print system(buildout)
+ Installing message.
+ >>> print system(buildout)
+ Updating message.
+
+Changes in a source directory cause a re-install
+------------------------------------------------
+
+The recipe keeps track of what files were installed, and so adding and removing
+files causes a reinstall with the expected behavior.
+
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = message
+ ...
+ ... [message]
+ ... recipe = z3c.recipe.filetemplate
+ ... source-directory = template
+ ... world = Philipp
+ ... """)
+ >>> mkdir(sample_buildout, 'template')
+ >>> mkdir(sample_buildout, 'template', 'etc')
+ >>> write(sample_buildout, 'template', 'etc', 'helloworld.sh.in',
+ ... """
+ ... Hello ${world} from the .sh file!
+ ... """)
+ >>> print system(buildout)
+ Uninstalling message.
+ Installing message.
+ >>> ls(sample_buildout, 'etc')
+ - helloworld.sh
+ >>> write(sample_buildout, 'template', 'etc', 'helloworld.conf.in',
+ ... """
+ ... Hello ${world} from the etc dir!
+ ... """)
+ >>> print system(buildout)
+ Uninstalling message.
+ Installing message.
+ >>> ls(sample_buildout, 'etc')
+ - helloworld.conf
+ - helloworld.sh
+ >>> remove(sample_buildout, 'template', 'etc', 'helloworld.sh.in')
+ >>> print system(buildout)
+ Uninstalling message.
+ Installing message.
+ >>> ls(sample_buildout, 'etc')
+ - helloworld.conf
+
+The main README also has an example of modifying a file, which will also cause
+a reinstall.
+
+More than one file in ``files`` with ``source-directory``
+---------------------------------------------------------
+
+When using ``source-directory``, ``files`` is a glob-based filter. This is
+shown in the main README. What is not shown is that you can have more than
+one ``files`` filter. Here's an example.
+
+ >>> write(sample_buildout, 'template', 'etc', 'helloworld.sh.in',
+ ... """
+ ... Hello ${world} from the .sh file!
+ ... """)
+ >>> write(sample_buildout, 'template', 'etc', 'helloworld.cfg.in',
+ ... """
+ ... Hello ${world} from the .cfg file!
+ ... """)
+
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = message
+ ...
+ ... [message]
+ ... recipe = z3c.recipe.filetemplate
+ ... source-directory = template
+ ... files = *.conf
+ ... helloworld.cfg
+ ... world = Philipp
+ ... """)
+ >>> print system(buildout)
+ Uninstalling message.
+ Installing message.
+ >>> ls(sample_buildout, 'etc')
+ - helloworld.cfg
+ - helloworld.conf
+
+``files`` with a directory when using a ``source-directory``
+------------------------------------------------------------
+
+If you use a source directory and your ``files`` specify a directory, the
+directory must match precisely.
+
+ >>> mkdir(sample_buildout, 'template', 'etc', 'in')
+ >>> write(sample_buildout, 'template', 'etc', 'in', 'helloworld.cfg.in',
+ ... """
+ ... Hello ${world} from the inner .cfg file!
+ ... """)
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = message
+ ...
+ ... [message]
+ ... recipe = z3c.recipe.filetemplate
+ ... source-directory = template
+ ... files = *.sh
+ ... etc/helloworld.cfg
+ ... world = Philipp
+ ... """)
+ >>> print system(buildout)
+ Uninstalling message.
+ Installing message.
+ >>> ls(sample_buildout, 'etc')
+ - helloworld.cfg
+ - helloworld.sh
+
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = message
+ ...
+ ... [message]
+ ... recipe = z3c.recipe.filetemplate
+ ... source-directory = template
+ ... files = *.sh
+ ... etc/in/helloworld.cfg
+ ... world = Philipp
+ ... """)
+ >>> print system(buildout)
+ Uninstalling message.
+ Installing message.
+ >>> ls(sample_buildout, 'etc')
+ - helloworld.sh
+ d in
+ >>> ls(sample_buildout, 'etc', 'in')
+ - helloworld.cfg
+
+This works with the ./ directory also, so you can specify a file in the root
+of the source directory.
+
+At the start, we have the ``etc`` directory but not ``helloworld.cfg``.
+
+ >>> ls(sample_buildout)
+ - .installed.cfg
+ - alreadyhere.txt
+ - alreadyhere.txt.in
+ d bin
+ - buildout.cfg
+ d develop-eggs
+ d eggs
+ d etc
+ - goodbyeworld.txt.in
+ - helloworld.txt.in
+ - missing.txt.in
+ d parts
+ d template
+
+ >>> write(sample_buildout, 'template', 'helloworld.cfg.in',
+ ... """
+ ... Hello ${world} from the top-level .cfg file!
+ ... """)
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... parts = message
+ ...
+ ... [message]
+ ... recipe = z3c.recipe.filetemplate
+ ... source-directory = template
+ ... files = ./helloworld.cfg
+ ... world = Philipp
+ ... """)
+ >>> print system(buildout)
+ Uninstalling message.
+ Installing message.
+
+Now the reverse is true.
+
+ >>> ls(sample_buildout)
+ - .installed.cfg
+ - alreadyhere.txt
+ - alreadyhere.txt.in
+ d bin
+ - buildout.cfg
+ d develop-eggs
+ d eggs
+ - goodbyeworld.txt.in
+ - helloworld.cfg
+ - helloworld.txt.in
+ - missing.txt.in
+ d parts
+ d template
+
More information about the Checkins
mailing list