[Checkins] SVN: zc.buildout/branches/gary-support-system-python/src/zc/buildout/ fix and test for edge case in include-site-packages code

[Gary Poster]

Log message for revision 101765:
  fix and test for edge case in include-site-packages code

Changed:
  U   zc.buildout/branches/gary-support-system-python/src/zc/buildout/easy_install.py
  U   zc.buildout/branches/gary-support-system-python/src/zc/buildout/tests.py

-=-
Modified: zc.buildout/branches/gary-support-system-python/src/zc/buildout/easy_install.py
===================================================================
--- zc.buildout/branches/gary-support-system-python/src/zc/buildout/easy_install.py	2009-07-09 14:43:15 UTC (rev 101764)
+++ zc.buildout/branches/gary-support-system-python/src/zc/buildout/easy_install.py	2009-07-09 14:43:18 UTC (rev 101765)
@@ -235,10 +235,10 @@
         path = (path and path[:] or [])
         if include_site_packages is not None:
             self._include_site_packages = include_site_packages
-        stdlib, site_packages = _get_system_packages(executable)
+        stdlib, self._site_packages = _get_system_packages(executable)
         if self._include_site_packages:
             path.extend(buildout_and_setuptools_path)
-            path.extend(site_packages)
+            path.extend(self._site_packages)
         # else we could try to still include the buildout_and_setuptools_path
         # if the elements are not in site_packages, but we're not bothering
         # with this optimization for now, in the name of code simplicity.
@@ -257,7 +257,19 @@
             self._versions = versions
 
     def _satisfied(self, req, source=None):
-        dists = [dist for dist in self._env[req.project_name] if dist in req]
+        # We get all distributions that match the given requirement.  If we
+        # are not supposed to include site-packages, we also filter those out.
+        # We need to do the filtering here even though we have exluded
+        # site packages from the _env's paths (see Installer.__init__) because
+        # an .egg-link, such as one for setuptools or zc.buildout installed
+        # by zc.buildout.buildout.Buildout.bootstrap, can indirectly include
+        # a path in our _site_packages.
+        dists = [dist for dist in self._env[req.project_name] if (
+                    dist in req and (
+                        self._include_site_packages or
+                        dist.location not in self._site_packages)
+                    )
+                ]
         if not dists:
             logger.debug('We have no distributions for %s that satisfies %r.',
                          req.project_name, str(req))
@@ -461,10 +473,19 @@
             # Nothing is available.
             return None
 
-        # Filter the available dists for the requirement and source flag
+        # Filter the available dists for the requirement and source flag. If we
+        # are not supposed to include site-packages, we also filter those out.
+        # We need to do the filtering here even though we have exluded site
+        # packages from the _index's paths (see Installer.__init__) because an
+        # .egg-link, such as one for setuptools or zc.buildout installed by
+        # zc.buildout.buildout.Buildout.bootstrap, can indirectly include a
+        # path in our _site_packages.
         dists = [dist for dist in index[requirement.project_name]
                  if ((dist in requirement)
                      and
+                     (self._include_site_packages or
+                      dist.location not in self._site_packages)
+                     and
                      ((not source) or
                       (dist.precedence == pkg_resources.SOURCE_DIST)
                       )

Modified: zc.buildout/branches/gary-support-system-python/src/zc/buildout/tests.py
===================================================================
--- zc.buildout/branches/gary-support-system-python/src/zc/buildout/tests.py	2009-07-09 14:43:15 UTC (rev 101764)
+++ zc.buildout/branches/gary-support-system-python/src/zc/buildout/tests.py	2009-07-09 14:43:18 UTC (rev 101765)
@@ -2376,6 +2376,53 @@
 
 That's a failure, as expected.
 
+Now we explore an important edge case.
+
+Some system Pythons include setuptools (and other Python packages) in their
+site-packages (or equivalent) using a .egg-info directory.  The pkg_resources
+module (from setuptools) considers a package installed using .egg-info to be a
+develop egg.
+
+zc.buildout.buildout.Buildout.bootstrap will make setuptools and zc.buildout
+available to the buildout via the eggs directory, for normal eggs; or the
+develop-eggs directory, for develop-eggs.
+
+If setuptools or zc.buildout is found in site-packages and considered by
+pkg_resources to be a develop egg, then the bootstrap code will use a .egg-link
+in the local develop-eggs, pointing to site-packages, in its entirety.  Because
+develop-eggs must always be available for searching for distributions, this
+indirectly brings site-packages back into the search path for distributions.
+
+Because of this, we have to take special care that we still exclude
+site-packages even in this case.  See the comments about site packages in the
+Installer._satisfied and Installer._obtain methods for the implementation
+(as of this writing).
+
+In this demonstration, we insert a link to the "other" distribution in our
+develop-eggs, which would bring the package back in, except for the special
+care we have taken to exclude it.
+
+    >>> zc.buildout.easy_install.clear_index_cache()
+    >>> rmdir(example_dest)
+    >>> example_dest = tmpdir('site-packages-example-install')
+    >>> mkdir(example_dest, 'develop-eggs')
+    >>> stdlib, site_packages = (
+    ...     zc.buildout.easy_install._get_system_packages(primed_executable))
+    >>> path_to_other = [p for p in site_packages if 'other' in p][0]
+    >>> write(example_dest, 'develop-eggs', 'other.egg-link', path_to_other)
+    >>> workingset = zc.buildout.easy_install.install(
+    ...     ['other'], example_dest, links=[],
+    ...     path=[join(example_dest, 'develop-eggs')],
+    ...     executable=primed_executable,
+    ...     index=None, include_site_packages=False)
+    Traceback (most recent call last):
+        ...
+    MissingDistribution: Couldn't find a distribution for 'other'.
+
+The MissingDistribution error shows that buildout correctly excluded the
+"site-packages" source even though it was indirectly included in the path
+via a .egg-link file.
+
     """
 
 def buildout_include_site_packages_option():
@@ -2395,6 +2442,7 @@
     ... '''
     ... [buildout]
     ... parts = eggs
+    ... find-links =
     ...
     ... [primed_python]
     ... executable = %(primed_executable)s
@@ -2418,6 +2466,7 @@
     ... '''
     ... [buildout]
     ... parts = eggs
+    ... find-links =
     ... include-site-packages = false
     ...
     ... [primed_python]