[Checkins] SVN: zc.buildout/branches/gary-betafix/ By default, Buildout and the bootstrap script now prefer final versions of Buildout, recipes, and extensions.
Gary Poster
gary.poster at canonical.com
Sun Aug 1 12:21:24 EDT 2010
Log message for revision 115338:
By default, Buildout and the bootstrap script now prefer final versions of Buildout, recipes, and extensions.
Changed:
U zc.buildout/branches/gary-betafix/CHANGES.txt
U zc.buildout/branches/gary-betafix/bootstrap/bootstrap.py
U zc.buildout/branches/gary-betafix/src/zc/buildout/bootstrap.txt
U zc.buildout/branches/gary-betafix/src/zc/buildout/buildout.py
U zc.buildout/branches/gary-betafix/src/zc/buildout/buildout.txt
U zc.buildout/branches/gary-betafix/src/zc/buildout/easy_install.py
U zc.buildout/branches/gary-betafix/src/zc/buildout/tests.py
U zc.buildout/branches/gary-betafix/src/zc/buildout/update.txt
-=-
Modified: zc.buildout/branches/gary-betafix/CHANGES.txt
===================================================================
--- zc.buildout/branches/gary-betafix/CHANGES.txt 2010-08-01 15:20:09 UTC (rev 115337)
+++ zc.buildout/branches/gary-betafix/CHANGES.txt 2010-08-01 16:21:22 UTC (rev 115338)
@@ -1,19 +1,52 @@
Change History
**************
+1.5.0b3 (unreleased)
+====================
+
+New features:
+
+- zc.buildout supports Python 2.7.
+
+- By default, Buildout and the bootstrap script now prefer final versions of
+ Buildout, recipes, and extensions. This can be changed by setting
+ ``prefer-final-build-system = false`` in your configuration's
+ [buildout] section, and by using the --accept-early-release flag when
+ calling bootstrap. This will hopefully allow beta releases to be more
+ easily and safely made in the future. Note that dependencies of your
+ software do not have this behavior: use the pre-existing switch
+ ``prefer-final = true`` to get this behavior.
+
+Bugs fixed:
+
+- You can now again use virtualenv with zc.buildout. The new features to let
+ buildout be used with a system Python are disabled in this configuration,
+ and the previous script generation behavior (1.4.3) is used, even if
+ the new function ``zc.buildout.easy_install.sitepackage_safe_scripts``
+ is used.
+
+1.5.0b2 (2010-04-29)
+====================
+
+This was a re-release of 1.4.3 in order to keep 1.5.0b1 release from hurting
+workflows that combined virtualenv with zc.buildout.
+
1.5.0b1 (2010-04-29)
====================
New Features:
-- zc.buildout supports Python 2.7.
-
- Added buildout:socket-timout option so that socket timeout can be configured
both from command line and from config files. (gotcha)
- Buildout can be safely used with a system Python (or any Python with code
- in site-packages), as long as you use the new z3c.recipe.scripts
- recipe to generate scripts and interpreters, rather than zc.recipe.egg.
+ in site-packages), as long as you use (1) A fresh checkout, (2) the
+ new bootstrap.py, and (3) recipes that use the new
+ ``zc.buildout.easy_install.sitepackage_safe_scripts`` function to generate
+ scripts and interpreters. Many recipes will need to be updated to use
+ this new function. The scripts and interpreters generated by
+ ``zc.recipe.egg`` will continue to use the older function, not safe
+ with system Pythons. Use the ``z3c.recipe.scripts`` as a replacement.
zc.recipe.egg is still a fully supported, and simpler, way of
generating scripts and interpreters if you are using a "clean" Python,
Modified: zc.buildout/branches/gary-betafix/bootstrap/bootstrap.py
===================================================================
--- zc.buildout/branches/gary-betafix/bootstrap/bootstrap.py 2010-08-01 15:20:09 UTC (rev 115337)
+++ zc.buildout/branches/gary-betafix/bootstrap/bootstrap.py 2010-08-01 16:21:22 UTC (rev 115338)
@@ -116,6 +116,17 @@
help=("Specify a directory for storing eggs. Defaults to "
"a temporary directory that is deleted when the "
"bootstrap script completes."))
+parser.add_option("--accept-early-release", dest='accept_early_release',
+ action="store_true", default=False,
+ help=("Normally, if you do not specify a --version, the "
+ "bootstrap script gets the newest *final* versions "
+ "of zc.buildout for you. If you use this flag, "
+ "bootstrap will get the newest releases even if they "
+ "are alphas or betas. Note that, if you do want to "
+ "use early buildout releases, you probably want "
+ "to also set ``prefer-final-build-system= false`` "
+ "in the [buildout] section of your configuration "
+ "file."))
parser.add_option("-c", None, action="store", dest="config_file",
help=("Specify the path to the buildout configuration "
"file to be used."))
@@ -177,24 +188,57 @@
if not has_broken_dash_S:
cmd.insert(1, '-S')
-if options.download_base:
- cmd.extend(['-f', quote(options.download_base)])
+find_links = options.download_base
+if not find_links:
+ find_links = os.environ.get('bootstrap-testing-find-links')
+if find_links:
+ cmd.extend(['-f', quote(find_links)])
-requirement = 'zc.buildout'
-if options.version:
- requirement = '=='.join((requirement, options.version))
-cmd.append(requirement)
-
if options.use_distribute:
setup_requirement = 'distribute'
else:
setup_requirement = 'setuptools'
ws = pkg_resources.working_set
+setup_requirement_path = ws.find(
+ pkg_resources.Requirement.parse(setup_requirement)).location
env = dict(
os.environ,
- PYTHONPATH=ws.find(
- pkg_resources.Requirement.parse(setup_requirement)).location)
+ PYTHONPATH=setup_requirement_path)
+requirement = 'zc.buildout'
+version = options.version
+if version is None and not options.accept_early_release:
+ # Figure out the most recent final version of zc.buildout.
+ import setuptools.package_index
+ _final_parts = '*final-', '*final'
+ def _final_version(parsed_version):
+ for part in parsed_version:
+ if (part[:1] == '*') and (part not in _final_parts):
+ return False
+ return True
+ index = setuptools.package_index.PackageIndex(
+ search_path=[setup_requirement_path])
+ if find_links:
+ index.add_find_links((find_links,))
+ req = pkg_resources.Requirement.parse(requirement)
+ if index.obtain(req) is not None:
+ best = []
+ bestv = None
+ for dist in index[req.project_name]:
+ distv = dist.parsed_version
+ if _final_version(distv):
+ if bestv is None or distv > bestv:
+ best = [dist]
+ bestv = distv
+ elif distv == bestv:
+ best.append(dist)
+ if best:
+ best.sort()
+ version = best[-1].version
+if version:
+ requirement = '=='.join((requirement, version))
+cmd.append(requirement)
+
if is_jython:
import subprocess
exitcode = subprocess.Popen(cmd, env=env).wait()
Modified: zc.buildout/branches/gary-betafix/src/zc/buildout/bootstrap.txt
===================================================================
--- zc.buildout/branches/gary-betafix/src/zc/buildout/bootstrap.txt 2010-08-01 15:20:09 UTC (rev 115337)
+++ zc.buildout/branches/gary-betafix/src/zc/buildout/bootstrap.txt 2010-08-01 16:21:22 UTC (rev 115338)
@@ -47,6 +47,87 @@
X...
d zc.buildout-...egg
+The buildout script it has generated is a new-style script, using a
+customized site.py.
+
+ >>> buildout_script = join(sample_buildout, 'bin', 'buildout')
+ >>> if sys.platform.startswith('win'):
+ ... buildout_script += '-script.py'
+ >>> print open(buildout_script).read() # doctest: +ELLIPSIS
+ #...
+ <BLANKLINE>
+ import sys
+ sys.path[0:0] = [
+ '/sample/parts/buildout',
+ ]
+ <BLANKLINE>
+ <BLANKLINE>
+ import os
+ path = sys.path[0]
+ if os.environ.get('PYTHONPATH'):
+ path = os.pathsep.join([path, os.environ['PYTHONPATH']])
+ os.environ['PYTHONPATH'] = path
+ import site # imports custom buildout-generated site.py
+ <BLANKLINE>
+ import zc.buildout.buildout
+ <BLANKLINE>
+ if __name__ == '__main__':
+ zc.buildout.buildout.main()
+ <BLANKLINE>
+
+The bootstrap process prefers final versions of zc.buildout, so it has
+selected the (generated-locally) 99.99 egg rather than the also-available
+100.0b1 egg. We can see that in the buildout script's site.py.
+
+ >>> buildout_site_py = join(
+ ... sample_buildout, 'parts', 'buildout', 'site.py')
+ >>> print open(buildout_site_py).read() # doctest: +ELLIPSIS
+ "...
+ buildout_paths = [
+ '/sample/eggs/setuptools-...egg',
+ '/sample/eggs/zc.buildout-99.99-pyN.N.egg'
+ ]
+ ...
+
+If you want to accept early releases of zc.buildout, you either need to
+specify an explicit version (using --version here and specifying the
+version in the buildout configuration file using the
+``buildout-version`` option or the ``versions`` option) or specify that you
+accept early releases.
+
+You accept early releases by using ``--accept-early-release`` on the
+bootstrap script and specifying ``prefer-final-build-system = false`` in the
+buildout configuration file. You must do both.
+
+Here's an example.
+
+ >>> write('buildout.cfg',
+ ... '''
+ ... [buildout]
+ ... parts =
+ ... prefer-final-build-system = false
+ ... ''')
+ >>> ignored = system(
+ ... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
+ ... 'bootstrap.py --accept-early-release')
+ >>> print open(buildout_site_py).read() # doctest: +ELLIPSIS
+ "...
+ buildout_paths = [
+ '/sample/eggs/setuptools-...egg',
+ '/sample/eggs/zc.buildout-100.0b1-pyN.N.egg'
+ ]
+ ...
+
+Notice we are now using zc.buildout 100.0b1, a non-final release.
+
+Now we'll go back to the default of preferring final versions.
+
+ >>> write('buildout.cfg',
+ ... '''
+ ... [buildout]
+ ... parts =
+ ... ''')
+
Now we will try the `--version` option, which lets you define a version for
`zc.buildout`. If not provided, bootstrap will look for the latest one.
@@ -71,11 +152,9 @@
<BLANKLINE>
X
-Let's make sure the generated `buildout` script uses it::
+Versions older than 1.5.0 put their egg dependencies in the ``buildout`` script.
+Let's make sure it was generated as we expect::
- >>> buildout_script = join(sample_buildout, 'bin', 'buildout')
- >>> if sys.platform.startswith('win'):
- ... buildout_script += '-script.py'
>>> print open(buildout_script).read() # doctest: +ELLIPSIS
#...
<BLANKLINE>
@@ -102,7 +181,7 @@
<BLANKLINE>
X
-Let's make sure the generated `buildout` script uses it::
+Let's make sure the generated ``buildout`` script uses it::
>>> print open(buildout_script).read() # doctest: +ELLIPSIS
#...
@@ -119,7 +198,7 @@
zc.buildout.buildout.main()
<BLANKLINE>
-`zc.buildout` now can also run with `Distribute` with the `--distribute`
+``zc.buildout`` now can also run with `Distribute` with the `--distribute`
option::
>>> print 'X'; print system(
@@ -131,23 +210,15 @@
Generated script '/sample/bin/buildout'...
X
-Let's make sure the generated `buildout` script uses it::
+Let's make sure the generated ``site.py`` uses it::
+ >>> print open(buildout_site_py).read() # doctest: +ELLIPSIS
+ "...
+ buildout_paths = [
+ '/sample/eggs/distribute-...egg',
+ '/sample/eggs/zc.buildout-99.99-pyN.N.egg'
+ ]
+ ...
- >>> print open(buildout_script).read() # doctest: +ELLIPSIS
- #...
- <BLANKLINE>
- import sys
- sys.path[0:0] = [
- '/sample/eggs/distribute-...egg',
- '/sample/eggs/zc.buildout-...egg',
- ]
- <BLANKLINE>
- import zc.buildout.buildout
- <BLANKLINE>
- if __name__ == '__main__':
- zc.buildout.buildout.main()
- <BLANKLINE>
-
Make sure both options can be used together::
>>> print 'X'; print system(
@@ -160,8 +231,8 @@
Generated script '/sample/bin/buildout'...
X
-Let's make sure the generated `buildout` script uses ``Distribute`` *and*
-``zc.buildout-1.2.1``::
+Let's make sure the old-style generated ``buildout`` script uses
+``Distribute`` *and* ``zc.buildout-1.2.1``::
>>> print open(buildout_script).read() # doctest: +ELLIPSIS
#...
@@ -258,5 +329,14 @@
--eggs=EGGS Specify a directory for storing eggs. Defaults to a
temporary directory that is deleted when the bootstrap
script completes.
+ --accept-early-release
+ Normally, if you do not specify a --version, the
+ bootstrap script gets the newest *final* versions of
+ zc.buildout for you. If you use this flag, bootstrap
+ will get the newest releases even if they are alphas
+ or betas. Note that, if you do want to use early
+ buildout releases, you probably want to also set
+ ``prefer-final-build-system= false`` in the [buildout]
+ section of your configuration file.
-c CONFIG_FILE Specify the path to the buildout configuration file to
be used.
Modified: zc.buildout/branches/gary-betafix/src/zc/buildout/buildout.py
===================================================================
--- zc.buildout/branches/gary-betafix/src/zc/buildout/buildout.py 2010-08-01 15:20:09 UTC (rev 115337)
+++ zc.buildout/branches/gary-betafix/src/zc/buildout/buildout.py 2010-08-01 16:21:22 UTC (rev 115338)
@@ -131,6 +131,7 @@
'offline': 'false',
'parts-directory': 'parts',
'prefer-final': 'false',
+ 'prefer-final-build-system': 'true',
'python': 'buildout',
'relative-paths': 'false',
'socket-timeout': '',
@@ -234,6 +235,8 @@
self._logger = logging.getLogger('zc.buildout')
self.offline = (buildout_section['offline'] == 'true')
self.newest = (buildout_section['newest'] == 'true')
+ self.prefer_final_build_system = (
+ buildout_section['prefer-final-build-system'] == 'true')
##################################################################
## WARNING!!!
@@ -267,42 +270,26 @@
self._setup_logging()
- offline = options['offline']
- if offline not in ('true', 'false'):
- self._error('Invalid value for offline option: %s', offline)
- self.offline = (offline == 'true')
-
- if self.offline:
- newest = options['newest'] = 'false'
- else:
- newest = options['newest']
- if newest not in ('true', 'false'):
- self._error('Invalid value for newest option: %s', newest)
- self.newest = (newest == 'true')
-
versions = options.get('versions')
if versions:
zc.buildout.easy_install.default_versions(dict(self[versions]))
- prefer_final = options['prefer-final']
- if prefer_final not in ('true', 'false'):
- self._error('Invalid value for prefer-final option: %s',
- prefer_final)
- zc.buildout.easy_install.prefer_final(prefer_final=='true')
- use_dependency_links = options['use-dependency-links']
- if use_dependency_links not in ('true', 'false'):
- self._error('Invalid value for use-dependency-links option: %s',
- use_dependency_links)
+ self.offline = options.get_bool('offline')
+ if self.offline:
+ options['newest'] = 'false'
+ self.newest = options.get_bool('newest')
+ zc.buildout.easy_install.prefer_final(
+ options.get_bool('prefer-final'))
+ self.prefer_final_build_system = options.get_bool(
+ 'prefer-final-build-system')
zc.buildout.easy_install.use_dependency_links(
- use_dependency_links == 'true')
-
- allow_picked_versions = options['allow-picked-versions']
- if allow_picked_versions not in ('true', 'false'):
- self._error('Invalid value for allow-picked-versions option: %s',
- allow_picked_versions)
+ options.get_bool('use-dependency-links'))
zc.buildout.easy_install.allow_picked_versions(
- allow_picked_versions == 'true')
+ options.get_bool('allow-picked-versions'))
+ zc.buildout.easy_install.install_from_cache(
+ options.get_bool('install-from-cache'))
+ zc.buildout.easy_install.always_unzip(options.get_bool('unzip'))
download_cache = options.get('download-cache')
if download_cache:
@@ -319,19 +306,6 @@
zc.buildout.easy_install.download_cache(download_cache)
- install_from_cache = options['install-from-cache']
- if install_from_cache not in ('true', 'false'):
- self._error('Invalid value for install-from-cache option: %s',
- install_from_cache)
- zc.buildout.easy_install.install_from_cache(
- install_from_cache=='true')
-
- always_unzip = options['unzip']
- if always_unzip not in ('true', 'false'):
- self._error('Invalid value for unzip option: %s',
- always_unzip)
- zc.buildout.easy_install.always_unzip(always_unzip=='true')
-
# "Use" each of the defaults so they aren't reported as unused options.
for name in _buildout_default_options:
options[name]
@@ -364,6 +338,7 @@
[options['develop-eggs-directory'],
options['eggs-directory']],
include_site_packages=_sys_executable_has_broken_dash_S,
+ prefer_final = self.prefer_final_build_system,
)
else:
ws = zc.buildout.easy_install.install(
@@ -375,6 +350,7 @@
newest=self.newest,
allow_hosts=self._allow_hosts,
include_site_packages=_sys_executable_has_broken_dash_S,
+ prefer_final = self.prefer_final_build_system,
)
# Now copy buildout and setuptools eggs, and record destination eggs:
@@ -410,6 +386,8 @@
else:
assert relative_paths == 'false'
relative_paths = ''
+ # Ideally the (possibly) new version of buildout would get a
+ # chance to write the script. Not sure how to do that.
zc.buildout.easy_install.sitepackage_safe_scripts(
options['bin-directory'], ws, options['executable'], partsdir,
reqs=['zc.buildout'], relative_paths=relative_paths,
@@ -427,7 +405,7 @@
# for eggs:
sys.path.insert(0, self['buildout']['develop-eggs-directory'])
- # Check for updates. This could cause the process to be rstarted
+ # Check for updates. This could cause the process to be restarted.
self._maybe_upgrade()
# load installed data
@@ -480,7 +458,7 @@
# compute new part recipe signatures
self._compute_part_signatures(install_parts)
- # uninstall parts that are no-longer used or who's configs
+ # uninstall parts that are no-longer used or whose configs
# have changed
for part in reversed(installed_parts):
if part in install_parts:
@@ -626,11 +604,11 @@
f.close()
def _uninstall_part(self, part, installed_part_options):
- # ununstall part
+ # uninstall part
__doing__ = 'Uninstalling %s.', part
self._logger.info(*__doing__)
- # run uinstall recipe
+ # run uninstall recipe
recipe, entry = _recipe(installed_part_options[part])
try:
uninstaller = _install_and_load(
@@ -859,7 +837,8 @@
index = options.get('index'),
path = [options['develop-eggs-directory']],
allow_hosts = self._allow_hosts,
- include_site_packages=_sys_executable_has_broken_dash_S
+ include_site_packages=_sys_executable_has_broken_dash_S,
+ prefer_final=self.prefer_final_build_system,
)
upgraded = []
@@ -913,6 +892,8 @@
# fast for Python to know to regenerate the .pyc/.pyo files.
shutil.rmtree(partsdir)
os.mkdir(partsdir)
+ # Ideally the new version of buildout would get a chance to write the
+ # script. Not sure how to do that.
zc.buildout.easy_install.sitepackage_safe_scripts(
options['bin-directory'], ws, sys.executable, partsdir,
reqs=['zc.buildout'],
@@ -957,7 +938,8 @@
links = self['buildout'].get('find-links', '').split(),
index = self['buildout'].get('index'),
newest=self.newest, allow_hosts=self._allow_hosts,
- include_site_packages=_sys_executable_has_broken_dash_S)
+ include_site_packages=_sys_executable_has_broken_dash_S,
+ prefer_final=self.prefer_final_build_system)
# Clear cache because extensions might now let us read pages we
# couldn't read before.
@@ -988,6 +970,7 @@
setup = os.path.abspath(setup)
fd, tsetup = tempfile.mkstemp()
+ exe = zc.buildout.easy_install._safe_arg(sys.executable)
try:
os.write(fd, zc.buildout.easy_install.runsetup_template % dict(
setuptools=pkg_resources_loc,
@@ -1001,14 +984,10 @@
for a in args:
arg_list.append(zc.buildout.easy_install._safe_arg(a))
- subprocess.Popen(
- [zc.buildout.easy_install._safe_arg(sys.executable)]
- + list(tsetup)
- + arg_list
- ).wait()
+ subprocess.Popen([exe] + list(tsetup) + arg_list).wait()
else:
- os.spawnl(os.P_WAIT, sys.executable, zc.buildout.easy_install._safe_arg (sys.executable), tsetup,
+ os.spawnl(os.P_WAIT, sys.executable, exe, tsetup,
*[zc.buildout.easy_install._safe_arg(a)
for a in args])
finally:
@@ -1075,7 +1054,8 @@
working_set=pkg_resources.working_set,
newest=buildout.newest,
allow_hosts=buildout._allow_hosts,
- include_site_packages=_sys_executable_has_broken_dash_S)
+ include_site_packages=_sys_executable_has_broken_dash_S,
+ prefer_final=buildout.prefer_final_build_system)
__doing__ = 'Loading %s recipe entry %s:%s.', group, spec, entry
return pkg_resources.load_entry_point(
@@ -1297,6 +1277,31 @@
self.name)
return self._created
+ def get_bool(self, name, default=None, on_error=None):
+ """Given a name, return a boolean value for that name.
+
+ ``default``, if given, should be 'true', 'false', or None. None
+ is the default, and means that there is no default for the
+ value: the call should raise a MissingOption error if the name
+ is not present.
+
+ ``on_error``, if given, should be a callable that takes the name and
+ the found value.
+ """
+ if default is None:
+ value = self[name]
+ else:
+ value = self.get(name, default=default)
+ if value not in ('true', 'false'):
+ if on_error is None:
+ raise zc.buildout.UserError(
+ 'Invalid value for %s option: %s' % (name, value))
+ else:
+ on_error(name, value)
+ else:
+ return value == 'true'
+
+
_spacey_nl = re.compile('[ \t\r\f\v]*\n[ \t\r\f\v\n]*'
'|'
'^[ \t\r\f\v]+'
Modified: zc.buildout/branches/gary-betafix/src/zc/buildout/buildout.txt
===================================================================
--- zc.buildout/branches/gary-betafix/src/zc/buildout/buildout.txt 2010-08-01 15:20:09 UTC (rev 115337)
+++ zc.buildout/branches/gary-betafix/src/zc/buildout/buildout.txt 2010-08-01 16:21:22 UTC (rev 115338)
@@ -765,6 +765,8 @@
DEFAULT_VALUE
prefer-final= false
DEFAULT_VALUE
+ prefer-final-build-system= true
+ DEFAULT_VALUE
python= buildout
DEFAULT_VALUE
relative-paths= false
@@ -2244,6 +2246,7 @@
parts =
parts-directory = /sample-buildout/parts
prefer-final = false
+ prefer-final-build-system = true
python = buildout
relative-paths = false
socket-timeout =
@@ -2484,26 +2487,35 @@
Preferring Final Releases
-------------------------
-Currently, when searching for new releases, the newest available
-release is used. This isn't usually ideal, as you may get a
-development release or alpha releases not ready to be widely used.
-You can request that final releases be preferred using the prefer
-final option in the buildout section::
+Currently, when searching for new releases of your project's
+dependencies, the newest available release is used. This isn't usually
+ideal, as you may get a development release or alpha releases not ready
+to be widely used. You can request that final releases be preferred
+using the ``prefer-final`` option in the buildout section::
[buildout]
...
prefer-final = true
-When the prefer-final option is set to true, then when searching for
+When the ``prefer-final`` option is set to true, then when searching for
new releases, final releases are preferred. If there are final
releases that satisfy distribution requirements, then those releases
-are used even if newer non-final releases are available. The buildout
-prefer-final option can be used to override this behavior.
+are used even if newer non-final releases are available.
-In buildout version 2, final releases will be preferred by default.
-You will then need to use a false value for prefer-final to get the
-newest releases.
+A separate option controls the behavior of the build system itself.
+When buildout looks for recipes, extensions, and for updates to itself,
+it does prefer final releases by default, as of the 1.5.0 release. The
+``prefer-final-build-system`` option will let you override this behavior.
+ [buildout]
+ ...
+ prefer-final-build-system = false
+
+In buildout version 2, all final releases will be preferred by
+default--that is ``prefer-final`` will also default to 'true'. You will
+then need to use a 'false' value for ``prefer-final`` to get the newest
+releases, like with ``prefer-final-build-system``.
+
Finding distributions
---------------------
Modified: zc.buildout/branches/gary-betafix/src/zc/buildout/easy_install.py
===================================================================
--- zc.buildout/branches/gary-betafix/src/zc/buildout/easy_install.py 2010-08-01 15:20:09 UTC (rev 115337)
+++ zc.buildout/branches/gary-betafix/src/zc/buildout/easy_install.py 2010-08-01 16:21:22 UTC (rev 115338)
@@ -322,7 +322,8 @@
use_dependency_links=None,
allow_hosts=('*',),
include_site_packages=None,
- allowed_eggs_from_site_packages=None
+ allowed_eggs_from_site_packages=None,
+ prefer_final=None,
):
self._dest = dest
self._allow_hosts = allow_hosts
@@ -336,6 +337,8 @@
if use_dependency_links is not None:
self._use_dependency_links = use_dependency_links
+ if prefer_final is not None:
+ self._prefer_final = prefer_final
self._links = links = list(_fix_file_links(links))
if self._download_cache and (self._download_cache not in links):
links.insert(0, self._download_cache)
@@ -1060,13 +1063,14 @@
executable=sys.executable, always_unzip=None,
path=None, working_set=None, newest=True, versions=None,
use_dependency_links=None, allow_hosts=('*',),
- include_site_packages=None, allowed_eggs_from_site_packages=None):
- installer = Installer(dest, links, index, executable, always_unzip, path,
- newest, versions, use_dependency_links,
- allow_hosts=allow_hosts,
- include_site_packages=include_site_packages,
- allowed_eggs_from_site_packages=
- allowed_eggs_from_site_packages)
+ include_site_packages=None, allowed_eggs_from_site_packages=None,
+ prefer_final=None):
+ installer = Installer(
+ dest, links, index, executable, always_unzip, path, newest,
+ versions, use_dependency_links, allow_hosts=allow_hosts,
+ include_site_packages=include_site_packages,
+ allowed_eggs_from_site_packages=allowed_eggs_from_site_packages,
+ prefer_final=prefer_final)
return installer.install(specs, working_set)
@@ -1075,11 +1079,11 @@
executable=sys.executable,
path=None, newest=True, versions=None, allow_hosts=('*',),
include_site_packages=None, allowed_eggs_from_site_packages=None):
- installer = Installer(dest, links, index, executable, True, path, newest,
- versions, allow_hosts=allow_hosts,
- include_site_packages=include_site_packages,
- allowed_eggs_from_site_packages=
- allowed_eggs_from_site_packages)
+ installer = Installer(
+ dest, links, index, executable, True, path, newest, versions,
+ allow_hosts=allow_hosts,
+ include_site_packages=include_site_packages,
+ allowed_eggs_from_site_packages=allowed_eggs_from_site_packages)
return installer.build(spec, build_ext)
@@ -1175,11 +1179,12 @@
[f() for f in undo]
def working_set(specs, executable, path, include_site_packages=None,
- allowed_eggs_from_site_packages=None):
+ allowed_eggs_from_site_packages=None, prefer_final=None):
return install(
specs, None, executable=executable, path=path,
include_site_packages=include_site_packages,
- allowed_eggs_from_site_packages=allowed_eggs_from_site_packages)
+ allowed_eggs_from_site_packages=allowed_eggs_from_site_packages,
+ prefer_final=prefer_final)
############################################################################
# Script generation functions
Modified: zc.buildout/branches/gary-betafix/src/zc/buildout/tests.py
===================================================================
--- zc.buildout/branches/gary-betafix/src/zc/buildout/tests.py 2010-08-01 15:20:09 UTC (rev 115337)
+++ zc.buildout/branches/gary-betafix/src/zc/buildout/tests.py 2010-08-01 16:21:22 UTC (rev 115338)
@@ -3126,6 +3126,105 @@
"""
+def buildout_prefer_final_build_system_option():
+ """
+The prefer-final-build-system buildout option can be used for overriding
+the default preference for final distributions for recipes, buildout
+extensions, and buildout itself.
+
+Set up. This creates sdists for demorecipe 1.0 and 1.1b1, and for
+demoextension 1.0 and 1.1b1.
+
+ >>> create_sample_recipe_sdists(sample_eggs)
+ >>> create_sample_extension_sdists(sample_eggs)
+
+The default is prefer-final-build-system = true:
+
+ >>> write('buildout.cfg',
+ ... '''
+ ... [buildout]
+ ... parts = demo
+ ... find-links = %(link_server)s
+ ... extensions = demoextension
+ ...
+ ... [demo]
+ ... recipe = demorecipe
+ ... ''' % globals())
+
+ >>> print system(buildout+' -v'), # doctest: +ELLIPSIS
+ Installing ...
+ Picked: demoextension = 1.0
+ ...
+ Picked: demorecipe = 1.0
+ ...
+
+Here we see that the final versions of demorecipe and demoextension were used.
+
+We get the same behavior if we explicitly state that
+prefer-final-build-system = true.
+
+ >>> write('buildout.cfg',
+ ... '''
+ ... [buildout]
+ ... parts = demo
+ ... find-links = %(link_server)s
+ ... extensions = demoextension
+ ... prefer-final-build-system = true
+ ...
+ ... [demo]
+ ... recipe = demorecipe
+ ... ''' % globals())
+
+ >>> print system(buildout+' -v'), # doctest: +ELLIPSIS
+ Installing ...
+ Picked: demoextension = 1.0
+ ...
+ Picked: demorecipe = 1.0
+ ...
+
+If we specify prefer-final-build-system = false, we'll get the newest
+distributions in the build system:
+
+ >>> write('buildout.cfg',
+ ... '''
+ ... [buildout]
+ ... parts = demo
+ ... find-links = %(link_server)s
+ ... extensions = demoextension
+ ... prefer-final-build-system = false
+ ...
+ ... [demo]
+ ... recipe = demorecipe
+ ... ''' % globals())
+
+ >>> print system(buildout+' -v'), # doctest: +ELLIPSIS
+ Installing ...
+ Picked: demoextension = 1.1b1
+ ...
+ Picked: demorecipe = 1.1b1
+ ...
+
+We get an error if we specify anything but true or false:
+
+ >>> write('buildout.cfg',
+ ... '''
+ ... [buildout]
+ ... parts = demo
+ ... find-links = %(link_server)s
+ ... extensions = demoextension
+ ... prefer-final-build-system = no
+ ...
+ ... [demo]
+ ... recipe = demorecipe
+ ... ''' % globals())
+
+ >>> print system(buildout+' -v'), # doctest: +ELLIPSIS
+ While:
+ Initializing.
+ Error: Invalid value for prefer-final-build-system option: no
+
+ """
+
def develop_with_modules():
"""
Distribution setup scripts can import modules in the distribution directory:
@@ -3492,6 +3591,68 @@
finally:
shutil.rmtree(tmp)
+def create_sample_extension_sdists(dest):
+ from zc.buildout.testing import write, mkdir
+ name = 'demoextension'
+ for version in ('1.0', '1.1b1'):
+ tmp = tempfile.mkdtemp()
+ try:
+ write(tmp, 'README.txt', '')
+ write(tmp, name + '.py',
+ "def ext(buildout):\n"
+ " pass\n"
+ "def unload(buildout):\n"
+ " pass\n"
+ % locals())
+ write(tmp, 'setup.py',
+ "from setuptools import setup\n"
+ "setup(\n"
+ " name = %(name)r,\n"
+ " py_modules = [%(name)r],\n"
+ " entry_points = {\n"
+ " 'zc.buildout.extension': "
+ "['ext = %(name)s:ext'],\n"
+ " 'zc.buildout.unloadextension': "
+ "['ext = %(name)s:unload'],\n"
+ " },\n"
+ " zip_safe=True, version=%(version)r,\n"
+ " author='bob', url='bob', author_email='bob')\n"
+ % locals())
+ zc.buildout.testing.sdist(tmp, dest)
+ finally:
+ shutil.rmtree(tmp)
+
+def create_sample_recipe_sdists(dest):
+ from zc.buildout.testing import write, mkdir
+ name = 'demorecipe'
+ for version in ('1.0', '1.1b1'):
+ tmp = tempfile.mkdtemp()
+ try:
+ write(tmp, 'README.txt', '')
+ write(tmp, name + '.py',
+ "import logging, os, zc.buildout\n"
+ "class Demorecipe:\n"
+ " def __init__(self, buildout, name, options):\n"
+ " self.name, self.options = name, options\n"
+ " def install(self):\n"
+ " return ()\n"
+ " def update(self):\n"
+ " pass\n"
+ % locals())
+ write(tmp, 'setup.py',
+ "from setuptools import setup\n"
+ "setup(\n"
+ " name = %(name)r,\n"
+ " py_modules = [%(name)r],\n"
+ " entry_points = {'zc.buildout': "
+ "['default = %(name)s:Demorecipe']},\n"
+ " zip_safe=True, version=%(version)r,\n"
+ " author='bob', url='bob', author_email='bob')\n"
+ % locals())
+ zc.buildout.testing.sdist(tmp, dest)
+ finally:
+ shutil.rmtree(tmp)
+
def _write_eggrecipedemoneeded(tmp, minor_version, suffix=''):
from zc.buildout.testing import write
write(tmp, 'README.txt', '')
@@ -3639,37 +3800,33 @@
egg_parse = re.compile('([0-9a-zA-Z_.]+)-([0-9a-zA-Z_.]+)-py(\d[.]\d).egg$'
).match
-def makeNewRelease(project, ws, dest):
+def makeNewRelease(project, ws, dest, version='99.99'):
dist = ws.find(pkg_resources.Requirement.parse(project))
eggname, oldver, pyver = egg_parse(
os.path.basename(dist.location)
).groups()
- dest = os.path.join(dest, "%s-99.99-py%s.egg" % (eggname, pyver))
+ dest = os.path.join(dest, "%s-%s-py%s.egg" % (eggname, version, pyver))
if os.path.isfile(dist.location):
shutil.copy(dist.location, dest)
zip = zipfile.ZipFile(dest, 'a')
zip.writestr(
'EGG-INFO/PKG-INFO',
zip.read('EGG-INFO/PKG-INFO').replace("Version: %s" % oldver,
- "Version: 99.99")
+ "Version: %s" % version)
)
zip.close()
else:
shutil.copytree(dist.location, dest)
info_path = os.path.join(dest, 'EGG-INFO', 'PKG-INFO')
info = open(info_path).read().replace("Version: %s" % oldver,
- "Version: 99.99")
+ "Version: %s" % version)
open(info_path, 'w').write(info)
-
-def updateSetup(test):
- zc.buildout.testing.buildoutSetUp(test)
- new_releases = test.globs['tmpdir']('new_releases')
- test.globs['new_releases'] = new_releases
+def getWorkingSetWithBuildoutEgg(test):
sample_buildout = test.globs['sample_buildout']
eggs = os.path.join(sample_buildout, 'eggs')
- # If the zc.buildout dist is a develo dist, convert it to a
+ # If the zc.buildout dist is a develop dist, convert it to a
# regular egg in the sample buildout
req = pkg_resources.Requirement.parse('zc.buildout')
dist = pkg_resources.working_set.find(req)
@@ -3699,9 +3856,16 @@
os.path.join(sample_buildout, 'bin'))
else:
ws = pkg_resources.working_set
+ return ws
+def updateSetup(test):
+ zc.buildout.testing.buildoutSetUp(test)
+ new_releases = test.globs['tmpdir']('new_releases')
+ test.globs['new_releases'] = new_releases
+ ws = getWorkingSetWithBuildoutEgg(test)
# now let's make the new releases
makeNewRelease('zc.buildout', ws, new_releases)
+ makeNewRelease('zc.buildout', ws, new_releases, '100.0b1')
os.mkdir(os.path.join(new_releases, 'zc.buildout'))
if zc.buildout.easy_install.is_distribute:
makeNewRelease('distribute', ws, new_releases)
@@ -3710,6 +3874,13 @@
makeNewRelease('setuptools', ws, new_releases)
os.mkdir(os.path.join(new_releases, 'setuptools'))
+def bootstrapSetup(test):
+ easy_install_SetUp(test)
+ sample_eggs = test.globs['sample_eggs']
+ ws = getWorkingSetWithBuildoutEgg(test)
+ makeNewRelease('zc.buildout', ws, sample_eggs)
+ makeNewRelease('zc.buildout', ws, sample_eggs, '100.0b1')
+ os.environ['bootstrap-testing-find-links'] = test.globs['link_server']
normalize_bang = (
re.compile(re.escape('#!'+
@@ -3952,12 +4123,13 @@
if os.path.exists(bootstrap_py):
test_suite.append(doctest.DocFileSuite(
'bootstrap.txt',
- setUp=easy_install_SetUp,
+ setUp=bootstrapSetup,
tearDown=zc.buildout.testing.buildoutTearDown,
checker=renormalizing.RENormalizing([
zc.buildout.testing.normalize_path,
zc.buildout.testing.normalize_endings,
zc.buildout.testing.normalize_script,
+ zc.buildout.testing.normalize_egg_py,
normalize_bang,
(re.compile('Downloading.*setuptools.*egg\n'), ''),
(re.compile('options:'), 'Options:'),
Modified: zc.buildout/branches/gary-betafix/src/zc/buildout/update.txt
===================================================================
--- zc.buildout/branches/gary-betafix/src/zc/buildout/update.txt 2010-08-01 15:20:09 UTC (rev 115337)
+++ zc.buildout/branches/gary-betafix/src/zc/buildout/update.txt 2010-08-01 16:21:22 UTC (rev 115338)
@@ -3,13 +3,14 @@
When a buildout is run, one of the first steps performed is to check
for updates to either zc.buildout or setuptools. To demonstrate this,
-we've creates some "new releases" of buildout and setuptools in a
+we've created some "new releases" of buildout and setuptools in a
new_releases folder:
>>> ls(new_releases)
d setuptools
- setuptools-99.99-py2.4.egg
d zc.buildout
+ - zc.buildout-100.0b1-pyN.N.egg
- zc.buildout-99.99-py2.4.egg
Let's update the sample buildout.cfg to look in this area:
@@ -78,6 +79,11 @@
zc.buildout 99.99
setuptools 99.99
+Notice that, even though we have a newer beta version of zc.buildout
+available, the final "99.99" was selected. If you want to get non-final
+versions, specify a specific version in your buildout's versions section,
+or use the ``prefer-final-build-system = false`` discussed below.
+
Our buildout script's site.py has been updated to use the new eggs:
>>> cat(sample_buildout, 'parts', 'buildout', 'site.py')
@@ -162,7 +168,7 @@
setuptools 0.6
We also won't upgrade if the buildout script being run isn't in the
-buildouts bin directory. To see this we'll create a new buildout
+buildout's bin directory. To see this we'll create a new buildout
directory:
>>> sample_buildout2 = tmpdir('sample_buildout2')
@@ -187,3 +193,33 @@
Not upgrading because not running a local buildout command.
>>> ls('bin')
+
+Notice that, as mentioned above, the ``prefer-final-build-system =
+false`` means that newer non-final versions of these dependencies are
+preferred.
+
+ >>> cd(sample_buildout)
+ >>> write(sample_buildout, 'buildout.cfg',
+ ... """
+ ... [buildout]
+ ... find-links = %(new_releases)s
+ ... index = %(new_releases)s
+ ... parts = show-versions
+ ... develop = showversions
+ ... prefer-final-build-system = false
+ ...
+ ... [show-versions]
+ ... recipe = showversions
+ ... """ % dict(new_releases=new_releases))
+
+ >>> print system(buildout),
+ Getting distribution for 'zc.buildout'.
+ Got zc.buildout 100.0b1.
+ Upgraded:
+ zc.buildout version 100.0b1,
+ setuptools version 99.99;
+ restarting.
+ Develop: '/sample-buildout/showversions'
+ Updating show-versions.
+ zc.buildout 100.0b1
+ setuptools 99.99
More information about the checkins
mailing list