[Zope-dev] Re: ploneout - Or how using zc.buildout for a common
Zope2 project might look like
jim at zope.com
Thu Jan 25 18:37:32 EST 2007
On Jan 25, 2007, at 5:09 PM, Ian Bicking wrote:
> Whit pointed me to this thread.
Yeah, someone pointed me to it too. :)
> I won't reply to specifics, but maybe
> just describe what we're doing (and planning to do), and how
> differs from zc.buildout.
I'll avoid responding to general qualitative statements.
> I actually tried to do this once before with zc.buildout, but I didn't
> get far -- probably a result of lack of effort and lack of familiarity
> with the overall stack. But I also recognize lots of the questions
> about stuff like the zope.conf file and Data.fs that still seem
Certainly when you tried this, buildout was very young and we hadn't
written recipes to deal with these issues. We've made a lot of
progress since then.
> The thing that frustrated me with zc.buildout is that I
> knew how to do what I wanted to do, but I still felt like I was a long
> way from being able to do it. And little things, like unhelpful error
Yeah, buildout still needs to do a lot better with error messages,
although it has probably made some progress since you tried it.
> and frustratingly slow performance really killed my motivation.
That has improved quite a bit.
> And frankly I like easy_install. It's
> probably 10x faster than buildout.
I doubt that that is true now. Although that probably depends on
what you are doing. Early versions of buildout did a lot of things
inefficiently as I was still learning setuptools. Because of the way
that buildout caches index information, I expect that creating a
buildout from scratch that used a lot of eggs would be much faster
than using easy_install. One difference though is that buildout
checks for the most recent compatible versions of all of the eggs
it's using every time you run it, whereas, as I understand it, with
workingenv, you'd just run easy_install manually when you want a new
egg. You can bypass the checks by running in offline mode. Then
buildout runs very fast. Because of the ability to share eggs
accross buildouts, it is often possible to run a buidout using lots
of eggs in offline mode.
It has been suggested that there should be a mode for buildout that
only talks to the network when there isn't a local egg that satisfied
a requirement. This would make buildout work more like workingenv
when few if any eggs are actually needed.
> easy_install is what people use in
> documentation, and its conventions are the ones people know (why does
> buildout not use "Pkg==version", for instance?).
It does. When specifying eggs, you use standard setuptools
> As for the technical reasons they don't work together:
> * workingenv allows and leaves it to setuptools to maintain the
> installation database (basically easy-install.pth). This is not a
> good database, but eh. buildout doesn't really have a database, but
> instead just enforces what buildout.cfg indicates.
buildout uses the buildout configuration file to store what you
want. It uses .installed.cfg to capture what you have. These are
both databases of sorts.
> * workingenv relies on that database to give default versions and to
> setup the Python path. The fixup it does of installed scripts is
> minimal, just setting up sys.path enough to force its site.py to get
> called. buildout enumerates all the activated packages, and ignores
> easy-install.pth. This is basically what makes it
Yup. I wanted something far more static and predictable for scripts
generated by buildout.
> Plus buildout's desire to own everything and
> destroy everything it does not own ;)
I'm not aware that it destroys anything. Could you be more specific?
> * As a result buildout supports multiple things in the same buildout
> that have conflicting version requirements, but where the packages
> themselves don't realize this (but the deployer does). If the
> know their requirements then setuptools' native machinery allows
> to work fine.
Yes. I expect that usually, packages won't be very specific. The
buildout configuration file provides a place to be specific.
> * Some see bin/activate as a jail. Both workingenv and buildout are
> deliberately jail-like. Both Jim and I loathe the non-
> repeatability of
> system-wide installations (at least I think I can speak for him on
> one point ;). bin/activate lets you into that jail, and lets you work
> there. There is no way into a buildout.
I'm not familiar with bin/activate, but it sounds like an interpreter
script created with buildout.
> Frankly this weirds me out,
> and is a big part of my past frustration with it. Maybe that's
> I'm in the relatively uncommon situation that I actually know what's
> going on under the hood of Python imports and packaging, and so it
> bothers me that I can't debug things directly. Anyway, neither
> activation when using scripts generated in the environment. And
> bin/activate is really just something that sets PYTHONPATH and then
> other non-essential things like changing the prompt and $PATH -- I
> should probably document that more clearly.
sounds a lot like an buildout interpreter script.
> Neither can be entirely
> compatible with a system-wide Python installation, because Python's
> standard site.py f**ks up the environment really early in the process,
> and avoiding that isn't all that easy.
This reminds me of a place where buildout is looser than workenv.
buildout doesn;t try to disable anything in the system python. It
just augments it. I always use a clean python, so avoiding
customizations in the Python I use isn't a problem. If I wanted to
take advantage of something in a system Python, as I occasionally do,
I can do that with buildout.
Jim Fulton mailto:jim at zope.com Python Powered!
CTO (540) 361-1714 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org
More information about the Zope-Dev