[Checkins] SVN: grok/www/minitutorials/ buildout.html has been
added to minitutorials, this is generated from an adapted
Darryl Cousins
darryl at darrylcousins.net.nz
Fri Aug 3 23:56:13 EDT 2007
Log message for revision 78573:
buildout.html has been added to minitutorials, this is generated from an adapted
grok2html.py which uses the original tutorial at
svn.zope.org/zc.buildout/trunk/doc/tutorial.txt.
Author names have been added to the other 2 tutorials and credit also
acknowledged in minitutorials/index.html.
Changed:
A grok/www/minitutorials/buildout.html
U grok/www/minitutorials/index.html
U grok/www/minitutorials/macros.html
U grok/www/minitutorials/searching.html
-=-
Added: grok/www/minitutorials/buildout.html
===================================================================
--- grok/www/minitutorials/buildout.html (rev 0)
+++ grok/www/minitutorials/buildout.html 2007-08-04 03:56:13 UTC (rev 78573)
@@ -0,0 +1,1489 @@
+
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
+
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head>
+ <title>Introduction to zc.buildout</title><meta name="description" content="Grok - now even cavemen can use Zope3" />
+ <meta name="keywords" content="Grok, internet, zope, zope3, software, web apps, web applications, python" />
+ <style type="text/css"><!-- @import url(/resources/grok.css); --></style>
+</head>
+
+<body>
+<div class="header">
+
+ <a href="http://grok.zope.org">
+ <img src="/resources/grok-header.jpg" alt="GROK" /></a>
+ <ul id="navigation">
+ <li>
+ <a href="/index.html" class="" title="Home">Home</a></li>
+ <li>
+ <a href="/about.html" class="" title="About">About</a></li>
+ <li>
+ <a href="/tutorial.html" class=""
+ title="Tutorial">Tutorial</a></li>
+ <li>
+ <a href="/minitutorials/index.html" class=""
+ title="How Tos">How Tos</a></li>
+ </ul>
+</div>
+
+
+<div class="roundcont">
+
+ <div class="roundtop">
+ <img src="/resources/corner-topleft.jpg" alt="" width="45" height="45" class="corner" style="display: none" />
+ </div>
+
+ <div class="content">
+
+ <h1 class="title">Introduction to zc.buildout</h1>
+<!-- Definitions of interpreted text roles (classes) for S5/HTML data. -->
+<!-- This data file has been placed in the public domain. -->
+<!-- Colours
+======= -->
+<!-- Text Sizes
+========== -->
+<!-- Display in Slides (Presentation Mode) Only
+========================================== -->
+<!-- Display in Outline Mode Only
+============================ -->
+<!-- Display in Print Only
+===================== -->
+<!-- Incremental Display
+=================== -->
+<br /><br /><br /><p>Jim Fulton, Zope Corporation</p>
+<p>DZUG 2007</p>
+<div class="section">
+<h2><a id="what-is-zc-buildout" name="what-is-zc-buildout">What is zc.buildout?</a></h2>
+<ul>
+<li><p class="first">Course-grained python-based configuration-driven build tool</p>
+</li>
+<li><p class="first">Tool for working with eggs</p>
+</li>
+<li><p class="first">Repeatable</p>
+<p class="handout">It should be possible to check-in a buildout specification and
+reproduce the same software later by checking out the
+specification and rebuilding.</p>
+</li>
+<li><p class="first">Developer oriented</p>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="course-grained-building" name="course-grained-building">Course-grained building</a></h2>
+<ul>
+<li><p class="first">make and scons (and distutils) are fine grained</p>
+<ul>
+<li><p class="first">Focus on individual files</p>
+</li>
+<li><p class="first">Good when one file is computed from another</p>
+<p>.c -> .o -> .so</p>
+</li>
+<li><p class="first">rule-driven</p>
+</li>
+<li><p class="first">dependency and change driven</p>
+</li>
+</ul>
+</li>
+<li><p class="first">zc.buildout is course-grained</p>
+<ul class="simple">
+<li>Build large components of a system<ul>
+<li>applications</li>
+<li>configurations files</li>
+<li>databases</li>
+</ul>
+</li>
+<li>configuration driven</li>
+</ul>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="python-based" name="python-based">Python-based</a></h2>
+<ul>
+<li><p class="first">make is an awful scripting language</p>
+<ul class="simple">
+<li>uses shell</li>
+<li>non-portable</li>
+</ul>
+</li>
+<li><p class="first">Python is a good scripting language</p>
+<p class="handout">Fortunately, distutils addresses most of my building needs. If I
+had to write my own fine-grained build definition, I'd use scons.</p>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="working-with-eggs" name="working-with-eggs">Working with eggs</a></h2>
+<ul class="simple">
+<li>Eggs rock!</li>
+<li>easy_install<ul>
+<li>Easy!</li>
+<li>Installs into system Python</li>
+<li>Not much control</li>
+</ul>
+</li>
+<li>workingenv makes easy_install much more usable<ul>
+<li>Avoids installing into system Python</li>
+<li>Avoids conflicts with packages installed in site_packages</li>
+<li>Really nice for experimentation</li>
+<li>Easy!</li>
+<li>Not much control</li>
+</ul>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="zc-buildout-and-eggs" name="zc-buildout-and-eggs"><tt class="docutils literal"><span class="pre">zc.buildout</span></tt> and eggs</a></h2>
+<ul>
+<li><p class="first">Control</p>
+<ul>
+<li><p class="first">Configuration driven</p>
+<ul>
+<li><p class="first">easier to control versions used</p>
+</li>
+<li><p class="first">always look for most recent versions by default</p>
+<p class="handout">When upgrading a distribution, <tt class="docutils literal"><span class="pre">easy_install</span></tt> doesn't upgrade
+dependencies,</p>
+</li>
+<li><p class="first">support for custom build options</p>
+</li>
+</ul>
+</li>
+</ul>
+</li>
+<li><p class="first">Greater emphasis on develop eggs</p>
+<ul class="simple">
+<li>Automates install/uninstall</li>
+<li>preference to develop eggs</li>
+</ul>
+<p class="handout">I often switch between develop and non-develop eggs. I may be
+using a regular egg and realize I need to fix it. I checkout the
+egg's project into my buildout and tell buildout to treat it as a
+develop egg. It creates the egg link in develop eggs and will
+load the develop egg in preference to the non-develop egg.</p>
+<p class="handout">(<tt class="docutils literal"><span class="pre">easy_install</span></tt> gives preference to released eggs of the same
+version.)</p>
+<p class="handout">When I'm done making my change, I make a new egg release and tell
+buildout to stop using a develop egg.</p>
+</li>
+</ul>
+<p class="handout"><tt class="docutils literal"><span class="pre">zc.buildout</span></tt> is built on setuptools and <tt class="docutils literal"><span class="pre">easy_install</span></tt>.</p>
+</div>
+<div class="section">
+<h2><a id="zc-buildout-current-status" name="zc-buildout-current-status">zc.buildout current status</a></h2>
+<ul>
+<li><p class="first">Actively used for development and deployment</p>
+</li>
+<li><p class="first">Third-generation of ZC buildout tools</p>
+<p class="handout">Our earliest buildouts used make. These were difficult to
+maintain and reuse.</p>
+<p class="handout">Two years ago, we created a prototype Python-based buildout
+system.</p>
+<p class="handout"><tt class="docutils literal"><span class="pre">zc.buildout</span></tt> is a non-prototype system that reflects
+experience using the prototype.</p>
+</li>
+<li><p class="first">A number of "recipes" available</p>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="a-python-egg-primer" name="a-python-egg-primer">A Python Egg Primer</a></h2>
+<p>Eggs are simple!</p>
+<ul>
+<li><p class="first">directories to be added to path</p>
+<ul class="simple">
+<li>may be zipped</li>
+<li>"zero" installation</li>
+</ul>
+</li>
+<li><p class="first">Meta data</p>
+<ul class="simple">
+<li>dependencies</li>
+<li>entry points</li>
+</ul>
+</li>
+<li><p class="first">May be distributed as source distributions</p>
+<p class="handout"><tt class="docutils literal"><span class="pre">easy_install</span></tt> and <tt class="docutils literal"><span class="pre">zc.buildout</span></tt> can install source
+distributions as easily as installing eggs. I've found that
+source distributions are more convenient to distribute in a lot
+of ways.</p>
+</li>
+<li><p class="first">Automatic discovery through PyPI</p>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="egg-jargon" name="egg-jargon">Egg jargon</a></h2>
+<ul>
+<li><p class="first">Distribution</p>
+<p class="handout">"distribution" is the name distutils uses for something that can
+be distributed. There are several kinds of distributions that
+can be created by distutils, including source distributions,
+binary distributions, eggs, etc.</p>
+</li>
+<li><p class="first">source and binary distributions</p>
+<p class="handout">A source distribution contains the source for a project.</p>
+<p class="handout">A binary distributions contains a compiled version of a project,
+including .pyc files and built extension modules.</p>
+<p class="handout">Eggs are a type of binary distribution.</p>
+</li>
+<li><p class="first">Platform independent and platform dependent eggs</p>
+<p class="handout">Platform dependent eggs contain built extension modules and are
+thus tied to a specific operating system. In addition, they may
+depend on build options that aren't reflected in the egg name.</p>
+</li>
+<li><p class="first">develop egg links</p>
+<p class="handout">Develop egg links (aka develop eggs) are special files that allow
+a source directory to be treated as an egg. An egg links is a
+file containing the path of a source directory.</p>
+</li>
+<li><p class="first">requirements</p>
+<p class="handout">Requirements are strings that name distributions. This consist
+of a project name, optional version specifiers, and optional
+extras specifiers. Extras are names of features of a package
+that may have special dependencies.</p>
+</li>
+<li><p class="first">index and link servers</p>
+<p><tt class="docutils literal"><span class="pre">easy_install</span></tt> and <tt class="docutils literal"><span class="pre">zc.buildout</span></tt> will automatically download
+distributions from the Internet. When looking for distributions,
+they will look on zero or more links servers for links to
+distributions.</p>
+<p>They will also look on a single index server, typically (always)
+<a class="reference" href="http://www.python.org/pypi">http://www.python.org/pypi</a>. Index servers are required to provide
+a specific web interface.</p>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="entry-points" name="entry-points">Entry points</a></h2>
+<ul>
+<li><p class="first">Very similar to utilities</p>
+<ul class="simple">
+<li>Named entry point groups define entry point types</li>
+<li>Named entry points within groups provide named components of a
+given type.</li>
+</ul>
+</li>
+<li><p class="first">Allow automated script generation</p>
+<p>Wrapper script:</p>
+<ul>
+<li><p class="first">Sets up path</p>
+<p class="handout"><tt class="docutils literal"><span class="pre">easy_install</span></tt> and <tt class="docutils literal"><span class="pre">zc.buildout</span></tt> take very different
+approaches to this.</p>
+<p class="handout"><tt class="docutils literal"><span class="pre">easy_install</span></tt> generates scripts that call an API that loads
+eggs dynamically at run time.</p>
+<p class="handout"><tt class="docutils literal"><span class="pre">zc.buildout</span></tt> determines the needed eggs at build time and
+generates code in scripts to explicitly add the eggs to
+<tt class="docutils literal"><span class="pre">sys.path</span></tt>.</p>
+<p class="handout">The approach taken by <tt class="docutils literal"><span class="pre">zc,buildout</span></tt> is intended to make
+script execution deterministic and less susceptible to
+accidental upgrades.</p>
+</li>
+<li><p class="first">Imports entry point</p>
+</li>
+<li><p class="first">Calls entry point without arguments</p>
+<p class="handout">Buildout allows more control over script generation.
+Initialization code and entry point arguments can be
+specified.</p>
+</li>
+</ul>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="buildout-overview" name="buildout-overview">Buildout overview</a></h2>
+<ul>
+<li><p class="first">Configuration driven</p>
+<ul>
+<li><p class="first">ConfigParser +</p>
+<p class="handout">Buildout uses the raw ConfigParser format extended with
+a variable-substitution syntax that allows reference to
+variables by section and option:</p>
+<pre class="handout literal-block">
+${sectionname:optionname}
+</pre>
+</li>
+<li><p class="first">Allows full system to be defined with a single file</p>
+<p class="handout">Although it is possible and common to factor into multiple
+files.</p>
+</li>
+</ul>
+</li>
+<li><p class="first">Specify a set of "parts"</p>
+<ul class="simple">
+<li>recipe</li>
+<li>configuration data</li>
+</ul>
+<p class="handout">Each part is defined by a recipe, which is Python software for
+installing or uninstalling the part, and data used by the recipe.</p>
+</li>
+<li><p class="first">Install and uninstall</p>
+<p class="handout">If a part is removed from a specification, it is uninstalled.</p>
+<p class="handout">If a part's recipe or configuration changes, it is uninstalled
+and reinstalled.</p>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="buildout-overview-continued" name="buildout-overview-continued">Buildout overview (continued)</a></h2>
+<ul class="simple">
+<li>Recipes<ul>
+<li>Written in python</li>
+<li>Distributed as eggs</li>
+</ul>
+</li>
+<li>Egg support<ul>
+<li>Develop eggs</li>
+<li>Egg-support recipes</li>
+</ul>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="quick-intro" name="quick-intro">Quick intro</a></h2>
+<ul class="simple">
+<li>Most common case<ul>
+<li>Working on a package</li>
+<li>Want to run tests</li>
+<li>Want to generate distributions</li>
+</ul>
+</li>
+<li>buildout is source project</li>
+<li>Example: <tt class="docutils literal"><span class="pre">zope.event</span></tt></li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="zope-event-project-files" name="zope-event-project-files"><tt class="docutils literal"><span class="pre">zope.event</span></tt> project files</a></h2>
+<ul>
+<li><p class="first">source in <tt class="docutils literal"><span class="pre">src</span></tt> directory</p>
+<p class="handout">Placing source in a separate <tt class="docutils literal"><span class="pre">src</span></tt> directory is a common
+convention. It violates "shallow is better than nested". Smaller
+projects may benefit from putting sources in the root directory,</p>
+</li>
+<li><p class="first"><tt class="docutils literal"><span class="pre">setup.py</span></tt> for defining egg</p>
+<p class="handout">Assuming that the project will eventually produce an egg, we have a
+setup file for the project. As we'll see later, this can be very
+minimal to start.</p>
+</li>
+<li><p class="first"><tt class="docutils literal"><span class="pre">README.txt</span></tt></p>
+<p class="handout">It is conventional to put a README.txt in the root of the
+project. distutils used to complain if this wasn't available.</p>
+</li>
+<li><p class="first"><tt class="docutils literal"><span class="pre">bootstrap.py</span></tt> for bootstrapping buildout</p>
+<p class="handout">The bootstrap script makes it easy to install the buildout
+software. We'll see another way to do this later.</p>
+</li>
+<li><p class="first"><tt class="docutils literal"><span class="pre">buildout.cfg</span></tt> defines the buildout</p>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="zope-event-buildout-cfg" name="zope-event-buildout-cfg">zope.event buildout.cfg</a></h2>
+<pre class="literal-block">
+[buildout]
+parts = test
+develop = .
+
+[test]
+recipe = zc.recipe.testrunner
+eggs = zope.event
+</pre>
+<p class="handout">Let's go through this line by line.</p>
+<pre class="handout literal-block">
+[buildout]
+</pre>
+<p class="handout">defines the buildout section. It is the only required section in
+the configuration file. It is options in this section that may
+cause other sections to be used.</p>
+<pre class="handout literal-block">
+parts = test
+</pre>
+<p class="handout">Every buildout is required to specify a list of parts, although the
+parts list is allowed to be empty. The parts list specifies what
+to build. If any of the parts listed depend on other parts, then
+the other parts will be built too.</p>
+<pre class="handout literal-block">
+develop = .
+</pre>
+<p class="handout">The develop option is used to specify one or more directories from
+which to create develop eggs. Here we specify the current
+directory. Each of these directories must have a setup file.</p>
+<pre class="handout literal-block">
+[test]
+</pre>
+<p class="handout">The <tt class="docutils literal"><span class="pre">test</span></tt> section is used to define our test part.</p>
+<pre class="handout literal-block">
+recipe = zc.recipe.testrunner
+</pre>
+<p class="handout">Every part definition is required to specify a recipe. The recipe
+contains the Python code with the logic to install the part. A
+recipe specification is a distribution requirement. The requirement
+may be followed by an colon and a recipe name. Recipe eggs can
+contain multiple recipes and can also define an default recipe.</p>
+<p class="handout">The <tt class="docutils literal"><span class="pre">zc.recipe.testrunner</span></tt> egg defines a default recipe that
+creates a test runner using the <tt class="docutils literal"><span class="pre">zope.testing.testrunner</span></tt>
+framework.</p>
+<pre class="handout literal-block">
+eggs = zope.event
+</pre>
+<p class="handout">The zc.recipe.testrunnner recipe has an eggs option for specifying
+which eggs should be tested. The generated test script will load
+these eggs along with their dependencies.</p>
+<p class="handout">For more information on the <tt class="docutils literal"><span class="pre">zc.recipe.testrunner</span></tt> recipe, see
+<a class="reference" href="http://www.python.org/pypi/zc.recipe.testrunner">http://www.python.org/pypi/zc.recipe.testrunner</a>.</p>
+</div>
+<div class="section">
+<h2><a id="buildout-steps" name="buildout-steps">Buildout steps</a></h2>
+<ul>
+<li><p class="first">Bootstrap the buildout:</p>
+<pre class="literal-block">
+python bootstrap.py
+</pre>
+<p class="handout">This installs setuptools and zc.buildout locally in your
+buildout. This avoids changing your system Python.</p>
+</li>
+<li><p class="first">Run the buildout:</p>
+<pre class="literal-block">
+bin/buildout
+</pre>
+<p class="handout">This generates the test script, <tt class="docutils literal"><span class="pre">bin/test</span></tt>.</p>
+</li>
+<li><p class="first">Run the tests:</p>
+<pre class="literal-block">
+bin/test
+</pre>
+</li>
+<li><p class="first">Generate a distribution:</p>
+<pre class="literal-block">
+bin/buildout setup . sdist register upload
+bin/buildout setup . bdist_egg register upload
+</pre>
+<pre class="literal-block">
+bin/buildout setup . egg_info -rbdev sdist register upload
+</pre>
+<p class="handout">Buildout accepts a number of commands, one of which is
+<tt class="docutils literal"><span class="pre">setup</span></tt>. The <tt class="docutils literal"><span class="pre">setup</span></tt> command takes a directory name and runs
+the setup script found there. It arranges for setuptools to be
+imported before the script runs. This causes setuptools defined
+commands to work even for distributions that don't use
+setuptools.</p>
+<p class="handout">The sdist, register, upload, bdist_egg, and egg_info commands are
+setuptools and distutils defined commands.</p>
+<p class="handout">The sdist command causes a source distribution to be created.</p>
+<p class="handout">The register command causes a release to be registered with PyPI
+and the upload command uploads the generated distribution.
+You'll need to have an account on PyPI for this to work, but
+these commands will actually help you set an account up.</p>
+<p class="handout">The bdist_egg command generates an egg.</p>
+<p class="handout">The egg_info command allows control of egg meta-data. The -r
+option to the egg_info command causes the distribution to have a
+version number that includes the subversion revision number of
+the project. The -b option specified a revision tag. Here we
+specified a revision tag of "dev", which marks the release as a
+devlopment release. These are useful when making development
+releases.</p>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="exercise-1" name="exercise-1">Exercise 1</a></h2>
+<p class="handout">We won't have time to stop the lecture while you do the
+exercises. If you can play and listen at the same time, then feel
+free to work on them while I speak. Otherwise, I recommend doing
+them later in the week. Feel free to ask me questions if you run
+into problems.</p>
+<p>Try building out <tt class="docutils literal"><span class="pre">zope.event</span></tt>.</p>
+<ul class="simple">
+<li>Check out: svn://svn.zope.org/repos/main/zope.event/trunk</li>
+<li>Bootstrap</li>
+<li>Run the buildout</li>
+<li>Run the tests</li>
+<li>Look around the buildout to see how things are laid out.</li>
+<li>Look at the scripts in the bin directory.</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="buildout-layout" name="buildout-layout">buildout layout</a></h2>
+<ul>
+<li><p class="first"><tt class="docutils literal"><span class="pre">bin</span></tt> directory for generated scripts</p>
+</li>
+<li><p class="first"><tt class="docutils literal"><span class="pre">parts</span></tt> directory for generated part data</p>
+<p>Many parts don't use this.</p>
+</li>
+<li><p class="first"><tt class="docutils literal"><span class="pre">eggs</span></tt> directory for (most) installed eggs</p>
+<ul class="simple">
+<li>May be shared across buildouts.</li>
+</ul>
+</li>
+<li><p class="first"><tt class="docutils literal"><span class="pre">develop-eggs</span></tt> directory</p>
+<ul class="simple">
+<li>develop egg links</li>
+<li>custom eggs</li>
+</ul>
+</li>
+<li><p class="first"><tt class="docutils literal"><span class="pre">.installed.cfg</span></tt> records what has been installed</p>
+</li>
+</ul>
+<p class="handout">Some people find the buildout layout surprising, as it isn't
+similar to a Unix directory layout. The buildout layout was guided
+by "shallow is better than nested".</p>
+<p class="handout">If you prefer a different layout, you can specify a different
+layout using buildout options. You can set these options globally
+so that all of your buildouts have the same layout.</p>
+</div>
+<div class="section">
+<h2><a id="common-buildout-use-cases" name="common-buildout-use-cases">Common buildout use cases</a></h2>
+<ul>
+<li><p class="first">Working on a single package</p>
+<p class="handout">zope.event is an example of this use case.</p>
+</li>
+<li><p class="first">System assembly</p>
+</li>
+<li><p class="first">Try out new packages</p>
+<ul class="simple">
+<li>workingenv usually better</li>
+<li>buildout better when custom
+build options needed</li>
+</ul>
+</li>
+<li><p class="first">Installing egg-based scripts for personal use</p>
+<p><tt class="docutils literal"><span class="pre">~/bin</span></tt> directory is a buildout</p>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="creating-eggs" name="creating-eggs">Creating eggs</a></h2>
+<p>Three levels of egg development</p>
+<ul class="simple">
+<li>Develop eggs, a minimal starting point</li>
+<li>Adding data needed for distribution</li>
+<li>Polished distributions</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="a-minimal-develop-setup-py" name="a-minimal-develop-setup-py">A Minimal/Develop <tt class="docutils literal"><span class="pre">setup.py</span></tt></a></h2>
+<pre class="code-block Python literal-block">
+from setuptools import setup
+setup(
+ name='foo',
+ package_dir = {'':'src'},
+ )
+
+</pre>
+<p class="handout">If we're only going to use a package as a devlop egg, we just need
+to specify the project name, and, if there is a separate source
+directory, then we need to specify that location.</p>
+<p class="handout">We'd also need to specify entry points if we had any. We'll see an
+example of that later.</p>
+<p class="handout">See the setuptools and distutils documentation for more information.</p>
+</div>
+<div class="section">
+<h2><a id="distributable-setup-py" name="distributable-setup-py">Distributable <tt class="docutils literal"><span class="pre">setup.py</span></tt></a></h2>
+<pre class="code-block Python literal-block">
+from setuptools import setup, find_packages
+name='zope.event'
+setup(
+ name=name,
+ version='3.3.0',
+ url='http://www.python.org/pypi/'+name,
+ author='Zope Corporation and Contributors',
+ author_email='zope3-dev@zope.org',
+ package_dir = {'': 'src'},
+ packages=find_packages('src'),
+ namespace_packages=['zope',],
+ include_package_data = True,
+ install_requires=['setuptools'],
+ zip_safe = False,
+ )
+
+</pre>
+<p class="handout">If we want to be able to create a distribution, then we need to
+specify a lot more information.</p>
+<p class="handout">The options used are documented in either the distutils or
+setuptools documentation. Most of the options are fairly obvious.</p>
+<p class="handout">We have to specify the Python packages used. The <tt class="docutils literal"><span class="pre">find_packages</span></tt>
+function can figure this out for us, although it would often be
+easy to specify it ourselves. For example, we could have
+specified:</p>
+<pre class="handout literal-block">
+packages=['zope', 'zope.event'],
+</pre>
+<p class="handout">The zope package is a namespace package. This means that it exists
+solely as a container for other packages. It doesn't have any files
+or modules of it's own. It only contains an <span class="incremental">__init__</span> module
+with:</p>
+<pre class="handout literal-block">
+pkg_resources.declare_namespace(__name__)
+</pre>
+<p class="handout">or, perhaps:</p>
+<pre class="handout literal-block">
+# this is a namespace package
+try:
+ import pkg_resources
+ pkg_resources.declare_namespace(__name__)
+except ImportError:
+ import pkgutil
+ __path__ = pkgutil.extend_path(__path__, __name__)
+</pre>
+<p class="handout">Namespace packages have to be declared, as we've done here.</p>
+<p class="handout">We always want to include package data.</p>
+<p class="handout">Because the <span class="incremental">__init__</span> module uses setuptools, we declare it as a
+dependency, using <tt class="docutils literal"><span class="pre">install_requires</span></tt>.</p>
+<p class="handout">We always want to specify whether a package is zip safe. A zip
+safe package doesn't try to access the package as a directory. If
+in doubt, specify False. If you don't specify anything, setuptools
+will guess.</p>
+</div>
+<div class="section">
+<h2><a id="polished-setup-py-1-3" name="polished-setup-py-1-3">Polished <tt class="docutils literal"><span class="pre">setup.py</span></tt> (1/3)</a></h2>
+<pre class="code-block Python literal-block">
+import os
+from setuptools import setup, find_packages
+
+def read(*rnames):
+ return open(os.path.join(os.path.dirname(__file__), *rnames)).read()
+
+long_description=(
+ read('README.txt')
+ + '\n' +
+ 'Detailed Documentation\n'
+ '**********************\n'
+ + '\n' +
+ read('src', 'zope', 'event', 'README.txt')
+ + '\n' +
+ 'Download\n'
+ '**********************\n'
+ )
+
+open('documentation.txt', 'w').write(long_description)
+
+</pre>
+<p class="handout">In the polished version we flesh out the meta data a bit more.</p>
+<p>When I create distributions that I consider ready for broader use and
+upload to PyPI, I like to include the full documentation in the long
+description so PyPI serves it for me.</p>
+</div>
+<div class="section">
+<h2><a id="polished-setup-py-2-3" name="polished-setup-py-2-3">Polished <tt class="docutils literal"><span class="pre">setup.py</span></tt> (2/3)</a></h2>
+<pre class="code-block Python literal-block">
+name='zope.event'
+setup(
+ name=name,
+ version='3.3.0',
+ url='http://www.python.org/pypi/'+name,
+ license='ZPL 2.1',
+ description='Zope Event Publication',
+ author='Zope Corporation and Contributors',
+ author_email='zope3-dev@zope.org',
+ long_description=long_description,
+
+ packages=find_packages('src'),
+ package_dir = {'': 'src'},
+ namespace_packages=['zope',],
+ include_package_data = True,
+ install_requires=['setuptools'],
+ zip_safe = False,
+ )
+
+</pre>
+</div>
+<div class="section">
+<h2><a id="extras" name="extras">Extras</a></h2>
+<pre class="code-block Python literal-block">
+name = 'zope.component'
+setup(name=name,
+ ...
+ namespace_packages=['zope',],
+ install_requires=['zope.deprecation', 'zope.interface',
+ 'zope.deferredimport', 'zope.event',
+ 'setuptools', ],
+ extras_require = dict(
+ service = ['zope.exceptions'],
+ zcml = ['zope.configuration', 'zope.security', 'zope.proxy',
+ 'zope.i18nmessageid',
+ ],
+ test = ['zope.testing', 'ZODB3',
+ 'zope.configuration', 'zope.security', 'zope.proxy',
+ 'zope.i18nmessageid',
+ 'zope.location', # should be depenency of zope.security
+ ],
+ hook = ['zope.hookable'],
+ persistentregistry = ['ZODB3'],
+ ),
+ )
+
+</pre>
+<p class="handout">Extras provide a way to help manage dependencies.</p>
+<p class="handout">A common use of extras is to separate test dependencies from normal
+depenencies. A package may provide other optional features that
+cause other dependencies. For example, the zcml module in
+zope.component adds lots of depenencies that we don't want to
+impose on people that don't use it.</p>
+</div>
+<div class="section">
+<h2><a id="zc-recipe-egg" name="zc-recipe-egg"><tt class="docutils literal"><span class="pre">zc.recipe.egg</span></tt></a></h2>
+<p>Set of recipes for:</p>
+<ul class="simple">
+<li>installing eggs</li>
+<li>generating scripts</li>
+<li>custom egg compilation</li>
+<li>custom interpreters</li>
+</ul>
+<p>See: <a class="reference" href="http://www.python.org/pypi/zc.recipe.egg">http://www.python.org/pypi/zc.recipe.egg</a>.</p>
+</div>
+<div class="section">
+<h2><a id="installing-eggs" name="installing-eggs">Installing eggs</a></h2>
+<pre class="literal-block">
+[buildout]
+parts = some-eggs
+
+[some-eggs]
+recipe = zc.recipe.egg:eggs
+eggs = docutils
+ ZODB3 <=3.8
+ zope.event
+</pre>
+<p class="handout">The eggs option accepts one or more distribution requirements.
+Because requirements may contain spaces, each requirement must be
+on a separate line. We used the eggs option to specify the eggs we
+want.</p>
+<p class="handout">Any dependencies of the named eggs will also be installed.</p>
+</div>
+<div class="section">
+<h2><a id="installing-scripts" name="installing-scripts">Installing scripts</a></h2>
+<pre class="literal-block">
+[buildout]
+parts = rst2
+
+[rst2]
+recipe = zc.recipe.egg:scripts
+eggs = zc.rst2
+</pre>
+<p class="handout">If any of the of the named eggs have <tt class="docutils literal"><span class="pre">console_script</span></tt> entry
+points, then scripts will be generated for the entry points.</p>
+<p class="handout">If a distribution doesn't use setuptools, it may not declare it's
+entry points. In that case, you can specify entry points in the
+recipe data.</p>
+</div>
+<div class="section">
+<h2><a id="script-initialization" name="script-initialization">Script initialization</a></h2>
+<pre class="literal-block">
+[buildout]
+develop = codeblock
+parts = rst2
+find-links = http://sourceforge.net/project/showfiles.php?group_id=45693
+
+[rst2]
+recipe = zc.recipe.egg
+eggs = zc.rst2
+ codeblock
+initialization =
+ sys.argv[1:1] = (
+ 's5 '
+ '--stylesheet ${buildout:directory}/zope/docutils.css '
+ '--theme-url file://${buildout:directory}/zope'
+ ).split()
+scripts = rst2=s5
+</pre>
+<p class="handout">In this example, we omitted the recipe entry point entry name
+because the scripts recipe is the default recipe for the
+zc.recipe.egg egg.</p>
+<p class="handout">The initialization option lets us specify some Python code to be included.</p>
+<p class="handout">We can control which scripts get installed and what their names are
+with the scripts option. In this example, we've used the scripts
+option to request a script named <tt class="docutils literal"><span class="pre">s5</span></tt> from the <tt class="docutils literal"><span class="pre">rst2</span></tt> entry point.</p>
+</div>
+<div class="section">
+<h2><a id="custom-interpreters" name="custom-interpreters">Custom interpreters</a></h2>
+<p>The script recipe allows an interpreter script to be created.</p>
+<pre class="literal-block">
+[buildout]
+parts = mypy
+
+[mypy]
+recipe = zc.recipe.egg:script
+eggs = zope.component
+interpreter = py
+</pre>
+<p>This will cause a <tt class="docutils literal"><span class="pre">bin/py</span></tt> script to created.</p>
+<p class="handout">Custom interpreters can be used to get an interactive Python prompt
+with the specified eggs and and their dependencies on <tt class="docutils literal"><span class="pre">sys.path</span></tt>.</p>
+<p class="handout">You can also use custom interpreters to run scripts, just like you
+would with the usual Python interpreter. Just call the interpreter
+with the script path and arguments, if any.</p>
+</div>
+<div class="section">
+<h2><a id="exercise-2" name="exercise-2">Exercise 2</a></h2>
+<ul class="simple">
+<li>Add a part to the <tt class="docutils literal"><span class="pre">zope.event</span></tt> project to create a custom interpreter.</li>
+<li>Run the interpreter and verify that you can import zope.event.</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="custom-egg-building" name="custom-egg-building">Custom egg building</a></h2>
+<pre class="literal-block">
+[buildout]
+parts = spreadmodule
+
+[spreadtoolkit]
+recipe = zc.recipe.cmmi
+url = http://yum.zope.com/buildout/spread-src-3.17.1.tar.gz
+
+[spreadmodule]
+recipe = zc.recipe.egg:custom
+egg = SpreadModule ==1.4
+find-links = http://www.python.org/other/spread/
+include-dirs = ${spreadtoolkit:location}/include
+library-dirs = ${spreadtoolkit:location}/lib
+rpath = ${spreadtoolkit:location}/lib
+</pre>
+<p class="handout">Sometimes a distribution has extension modules that need to be
+compiled with special options, such as the location of include
+files and libraries, The custom recipe supports this. The
+resulting eggs are placed in the develop-eggs directory because the
+eggs are buildout specific.</p>
+<p class="handout">This example illustrates use of the zc.recipe.cmmi recipe with
+supports installation of software that uses configure and make.
+Here, we used the recipe to install the spread toolkit, which is
+installed in the parts directory.</p>
+</div>
+<div class="section">
+<h2><a id="part-dependencies" name="part-dependencies">Part dependencies</a></h2>
+<ul class="simple">
+<li>Parts can read configuration from other parts</li>
+<li>The parts read become dependencies of the reading parts<ul>
+<li>Dependencies are added to parts list, if necessary</li>
+<li>Dependencies are installed first</li>
+</ul>
+</li>
+</ul>
+<p class="handout">In the previous example, we used the spread toolkit location in the
+spreadmodule part definition. This reference was sufficient to make
+the spreadtoolkit part a dependency of the spreadmodule part and
+cause it to be installed first,</p>
+</div>
+<div class="section">
+<h2><a id="custom-develop-eggs" name="custom-develop-eggs">Custom develop eggs</a></h2>
+<pre class="literal-block">
+[buildout]
+parts = zodb
+
+[zodb]
+recipe = zc.recipe.egg:develop
+setup = zodb
+define = ZODB_64BIT_INTS
+</pre>
+<p class="handout">We can also specify custom build options for develop eggs. Here we
+used a develop egg just to make sure our custom build of ZODB took
+precedence over normal ZODB eggs in our shared eggs directory.</p>
+</div>
+<div class="section">
+<h2><a id="writing-recipes" name="writing-recipes">Writing recipes</a></h2>
+<ul>
+<li><p class="first">The recipe API</p>
+<ul>
+<li><p class="first">install</p>
+<ul>
+<li><p class="first">__init__</p>
+<p class="handout">The initializer is responsible for computing a part's
+options. After the initializer call, the options directory
+must reflect the full configuration of the part. In
+particular, if a recipe reads any data from other sections,
+it must be reflected in the options. The options data after
+the initializer is called is used to determine if a
+configuration has changed when deciding if a part has to
+be reinstalled. When a part is reinstalled, it is
+uninstalled and then installed.</p>
+</li>
+<li><p class="first">install</p>
+<p class="handout">The install method installs the part. It is used when a part
+is added to a buildout, or when a part is reinstalled.</p>
+<p class="handout">The install recipe must return a sequence of paths that that
+should be removed when the part is uninstalled. Most recipes
+just create files or directories and removing these is
+sufficient for uninstalling the part.</p>
+</li>
+<li><p class="first">update</p>
+<p class="handout">The update method is used when a part is already installed
+and it's configuration hasn't changed from previous
+buildouts. It can return None or a sequence of paths. If
+paths are returned, they are added to the set of installed
+paths.</p>
+</li>
+</ul>
+</li>
+<li><p class="first">uninstall</p>
+<p class="handout">Most recipes simply create files or directories and the
+build-in buildout uninstall support is sufficient. If a recipe
+does more than simply create files, then an uninstall recipe
+will likely be needed.</p>
+</li>
+</ul>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="install-recipes" name="install-recipes">Install Recipes</a></h2>
+<p><tt class="docutils literal"><span class="pre">mkdirrecipe</span></tt>.py:</p>
+<pre class="code-block Python literal-block">
+import logging, os, zc.buildout
+
+class Mkdir:
+
+ def __init__(self, buildout, name, options):
+ self.name, self.options = name, options
+ options['path'] = os.path.join(
+ buildout['buildout']['directory'],
+ options['path'],
+ )
+ if not os.path.isdir(os.path.dirname(options['path'])):
+ logging.getLogger(self.name).error(
+ 'Cannot create %s. %s is not a directory.',
+ options['path'], os.path.dirname(options['path']))
+ raise zc.buildout.UserError('Invalid Path')
+
+</pre>
+<ul class="handout simple">
+<li>The path option in our recipe is interpreted relative to the
+buildout. We reflect this by saving the adjusted path in the
+options.</li>
+<li>If there is a user error, we:<ul>
+<li>Log error details using the Python logger module.</li>
+<li>Raise a zc.buildout.UserErrpr exception.</li>
+</ul>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="mkdirrecipe-py-continued" name="mkdirrecipe-py-continued"><tt class="docutils literal"><span class="pre">mkdirrecipe</span></tt>.py continued</a></h2>
+<blockquote>
+<pre class="code-block Python literal-block">
+def install(self):
+ path = self.options['path']
+ logging.getLogger(self.name).info(
+ 'Creating directory %s', os.path.basename(path))
+ os.mkdir(path)
+ return path
+
+def update(self):
+ pass
+
+</pre>
+</blockquote>
+<p class="handout">A well-written recipe will log what it's doing.</p>
+<p class="handout">Often the update method is empty, as in this case.</p>
+</div>
+<div class="section">
+<h2><a id="uninstall-recipes" name="uninstall-recipes">Uninstall recipes</a></h2>
+<p><tt class="docutils literal"><span class="pre">servicerecipe.py</span></tt>:</p>
+<pre class="code-block Python literal-block">
+import os
+
+class Service:
+
+ def __init__(self, buildout, name, options):
+ self.options = options
+
+ def install(self):
+ os.system("chkconfig --add %s" % self.options['script'])
+ return ()
+
+ def update(self):
+ pass
+
+def uninstall_service(name, options):
+ os.system("chkconfig --del %s" % options['script'])
+
+</pre>
+<p class="handout">Uninstall recipes are callables that are passed the part name and
+the <strong>original options</strong>.</p>
+</div>
+<div class="section">
+<h2><a id="buildout-entry-points" name="buildout-entry-points">Buildout entry points</a></h2>
+<p><tt class="docutils literal"><span class="pre">setup.py</span></tt>:</p>
+<pre class="code-block Python literal-block">
+from setuptools import setup
+
+entry_points = """
+[zc.buildout]
+mkdir = mkdirrecipe:Mkdir
+service = servicerecipe:Service
+default = mkdirrecipe:Mkdir
+
+[zc.buildout.uninstall]
+service = servicerecipe:uninstall_service
+"""
+
+setup(name='recipes', entry_points=entry_points)
+
+</pre>
+</div>
+<div class="section">
+<h2><a id="exercise-3" name="exercise-3">Exercise 3</a></h2>
+<ul class="simple">
+<li>Write recipe that creates a file from source given in a
+configuration option.</li>
+<li>Try this out in a buildout, either by creating a new buildout, or by
+extending the <tt class="docutils literal"><span class="pre">zope.event</span></tt> buildout.</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="command-line-options" name="command-line-options">Command-line options</a></h2>
+<p>Buildout command-line:</p>
+<ul class="simple">
+<li>command-line options and option setting</li>
+<li>command and arguments</li>
+</ul>
+<pre class="literal-block">
+bin/buildout -U -c rpm.cfg install zrs
+</pre>
+<p class="handout">Option settings are of the form:</p>
+<pre class="handout literal-block">
+section:option=value
+</pre>
+<p class="handout">Any option you can set in the configuration file, you can set on
+the command-line. Option settings specified on the command line
+override settings read from configuration files.</p>
+<p class="handout">There are a few command-line options, like -c to specify a
+configuration file, or -U to disable reading user defaults.</p>
+<p class="handout">See the buildout documentation, or use the -h option to get a list
+of available options.</p>
+</div>
+<div class="section">
+<h2><a id="buildout-modes" name="buildout-modes">Buildout modes</a></h2>
+<ul class="simple">
+<li>newest<ul>
+<li>default mode always tries to get newest versions</li>
+<li>Turn off with -N or buildout newest option set to false.</li>
+</ul>
+</li>
+<li>offline<ul>
+<li>If enabled, then don't try to do network access</li>
+<li>Disabled by default</li>
+<li>If enabled with -o or buildout offline option set to false.</li>
+</ul>
+</li>
+</ul>
+<p class="handout">By default, buildout always tries to find the newest distributions
+that match requirements. Looking for new distributions can be very
+time consuming. Many people will want to specify the -N option to
+disable this. We'll see later how we can change this default
+behavior.</p>
+<p class="handout">If you aren't connected to a network, you'll want to use the
+offline mode, -o.</p>
+</div>
+<div class="section">
+<h2><a id="buildout-default-cfg" name="buildout-default-cfg"><tt class="docutils literal"><span class="pre">~/.buildout/default.cfg</span></tt></a></h2>
+<p>Provides default buildout settings (unless -U option is used):</p>
+<pre class="literal-block">
+[buildout]
+# Shared eggs directory:
+eggs-directory = /home/jim/.eggs
+# Newest mode off, reenable with -n
+newst = false
+
+[python24]
+executabe = /usr/local/python/2.4/bin/python
+
+[python25]
+executabe = /usr/local/python/2.5/bin/python
+</pre>
+<p class="handout">Unless the -U command-line option is used, user default settings
+are read before reading regular configuration files. The user
+defaults are read from the default.cfg file in the .buildout
+subdirectory of the directory specified in the HOME environment
+variable, if any.</p>
+<p class="handout">In this example:</p>
+<ul class="handout simple">
+<li>I set up a shared eggs directory.</li>
+<li>I changed the default mode to non-newest so that buildout doesn't
+look for new distributions if the distributions it has meet it's
+requirements. To get the newest distributions, I'll have to use
+the -n option.</li>
+<li>I've specified Python 2.4 and 2.5 sections that specify locations
+of Python interpreters. Sometimes, a buildout uses multiple
+versions of Python. Many recipes accept a python option that
+specifies the name of a section with an executable option
+specifying the location of a Python interpreter.</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="extending-configurations" name="extending-configurations">Extending configurations</a></h2>
+<p>The <tt class="docutils literal"><span class="pre">extends</span></tt> option allows one configuration file to extend
+another.</p>
+<p>For example:</p>
+<ul>
+<li><p class="first"><tt class="docutils literal"><span class="pre">base.cfg</span></tt> has common definitions and settings</p>
+</li>
+<li><p class="first"><tt class="docutils literal"><span class="pre">dev.cfg</span></tt> adds development-time options:</p>
+<pre class="literal-block">
+[buildout]
+extends = base.cfg
+
+...
+</pre>
+</li>
+<li><p class="first"><tt class="docutils literal"><span class="pre">rpm.cfg</span></tt> has options for generating an RPM packages from a
+buildout.</p>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="bootstrapping-from-existing-buildout" name="bootstrapping-from-existing-buildout">Bootstrapping from existing buildout</a></h2>
+<ul class="simple">
+<li>The buildout script has a <tt class="docutils literal"><span class="pre">bootstrap</span></tt> command</li>
+<li>Can use it to bootstrap any directory.</li>
+<li>Much faster than running <tt class="docutils literal"><span class="pre">bootstrap.py</span></tt> because it can use an already
+installed <tt class="docutils literal"><span class="pre">setuptools</span></tt> egg.</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="example-bin-directory" name="example-bin-directory">Example: ~/bin directory</a></h2>
+<pre class="literal-block">
+[buildout]
+parts = rst2 buildout24 buildout25
+bin-directory = .
+
+[rst2]
+recipe = zc.recipe.egg
+eggs = zc.rst2
+
+[buildout24]
+recipe = zc.recipe.egg
+eggs = zc.buildout
+scripts = buildout=buildout24
+python = python24
+
+[buildout25]
+recipe = zc.recipe.egg
+eggs = zc.buildout
+scripts = buildout=buildout25
+python = python25
+</pre>
+<p class="handout">Many people have a personal scripts directory.</p>
+<p class="handout">I've converted mine to a buildout using a buildout configuration
+like the one above.</p>
+<p class="handout">I've overridden the bin-directory location so that scripts are
+installed directly into the buildout directory.</p>
+<p class="handout">I've specified that I want the zc.rst2 distribution installed. The
+rst2 distribution has a generalized version of the restructured
+text processing scripts in a form that can be installed by buildout
+(or easy_install).</p>
+<p class="handout">I've specified that I want buildout scripts for Python 2.4 and
+2.5. (In my buildout, I also create one for Python 2.3.) These
+buildout scripts allow me to quickly bootstrap buildouts or to run
+setup files for a given version of python. For example, to
+bootstrap a buildout with Python 2.4, I'll run:</p>
+<pre class="handout literal-block">
+buildout24 bootstrap
+</pre>
+<p class="handout">in the directory containing the buildout. This can also be used to
+convert a directory to a buildout, creating a buildout.cfg file is
+it doesn't exist.</p>
+</div>
+<div class="section">
+<h2><a id="example-zc-sharing-1-2" name="example-zc-sharing-1-2">Example: zc.sharing (1/2)</a></h2>
+<pre class="literal-block">
+[buildout]
+develop = . zc.security
+parts = instance test
+find-links = http://download.zope.org/distribution/
+
+[instance]
+recipe = zc.recipe.zope3instance
+database = data
+user = jim:123
+eggs = zc.sharing
+zcml =
+ zc.resourcelibrary zc.resourcelibrary-meta
+ zc.sharing-overrides:configure.zcml zc.sharing-meta
+ zc.sharing:privs.zcml zc.sharing:zope.manager-admin.zcml
+ zc.security zc.table zope.app.securitypolicy-meta zope.app.twisted
+ zope.app.authentication
+</pre>
+<p class="handout">This is a small example of the "system assembly" use case. In this
+case, we define a Zope 3 instance, and a test script.</p>
+<p class="handout">You can largely ignore the details of the Zope 3 instance recipe.
+If you aren't a Zope user, you don't care. If you are a Zope user,
+you should be aware that much better recipes are in development.</p>
+<p class="handout">This project uses multiple source directories, the current
+directory and the zc.security directory, which is a subversion
+external to a project without its own distribution. We've listed
+both in the develop option.</p>
+<p class="handout">We've requested the instance and test parts. We'll get other parts
+installed due to dependencies of the instance part. In particular,
+we'll get a Zope 3 checkout because the instance recipe refers to
+the zope3 part. We'll get a database part because of the reference
+in the database option of the instance recipe.</p>
+<p class="handout">The buildout will look for distributions at
+<a class="reference" href="http://download.zope.org/distribution/">http://download.zope.org/distribution/</a>.</p>
+</div>
+<div class="section">
+<h2><a id="example-zc-sharing-2-2" name="example-zc-sharing-2-2">Example: zc.sharing (2/2)</a></h2>
+<pre class="literal-block">
+[zope3]
+recipe = zc.recipe.zope3checkout
+url = svn://svn.zope.org/repos/main/Zope3/branches/3.3
+
+[data]
+recipe = zc.recipe.filestorage
+
+[test]
+recipe = zc.recipe.testrunner
+defaults = ['--tests-pattern', 'f?tests$']
+eggs = zc.sharing
+ zc.security
+extra-paths = ${zope3:location}/src
+</pre>
+<p class="handout">Here we see the definition of the remaining parts.</p>
+<p class="handout">The test part has some options we haven't seen before.</p>
+<ul class="handout simple">
+<li>We've customized the way the testrunner finds tests by providing
+some testrunner default arguments.</li>
+<li>We've used the extra-paths option to tell the test runner to
+include the Zope 3 checkout source directory in sys.path. This
+won't be necessary when Zope 3 is available entirely as eggs.</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="source-vs-binary" name="source-vs-binary">Source vs Binary</a></h2>
+<ul>
+<li><p class="first">Binary distributions are Python version and often platform specific</p>
+</li>
+<li><p class="first">Platform-dependent distribution can reflect build-time setting not
+reflected in egg specification.</p>
+<ul class="simple">
+<li>Unicode size</li>
+<li>Library names and locations</li>
+</ul>
+</li>
+<li><p class="first">Source distributions are more flexible</p>
+</li>
+<li><p class="first">Binary eggs can go rotten when system libraries are upgraded</p>
+<p class="handout">Recently, I had to manually remove eggs from my shared eggs
+directory. I had installed an operating system upgrade that
+caused the names of open-ssl library files to change. Eggs build
+against the old libraries no-longer functioned.</p>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="rpm-experiments" name="rpm-experiments">RPM experiments</a></h2>
+<p>Initial work creating RPMs for deployment in our hosting environment:</p>
+<ul class="simple">
+<li>Separation of software and configuration</li>
+<li>Buildout used to create rpm containing software</li>
+<li>Later, the installed buildout is used to set up specific processes<ul>
+<li>Run as root in offline mode</li>
+<li>Uses network configuration server</li>
+</ul>
+</li>
+</ul>
+<p class="handout">Our philosophy is to separate software and configuration. We
+install software using RPMs. Later, we configure the use of the
+software using a centralized configuration database.</p>
+<p class="handout">I'll briefly present the RPM building process below. This is
+interesting, in part, because it illustrates some interesting issues.</p>
+</div>
+<div class="section">
+<h2><a id="zrs-spec-file-1-3" name="zrs-spec-file-1-3">ZRS spec file (1/3)</a></h2>
+<pre class="literal-block">
+%define python zpython
+%define svn_url svn+ssh://svn.zope.com/repos/main/ZRS-buildout/trunk
+requires: zpython
+Name: zrs15
+Version: 1.5.1
+Release: 1
+Summary: Zope Replication Service
+URL: http://www.zope.com/products/zope_replication_services.html
+
+Copyright: ZVSL
+Vendor: Zope Corporation
+Packager: Zope Corporation <sales@zope.com>
+Buildroot: /tmp/buildroot
+Prefix: /opt
+Group: Applications/Database
+AutoReqProv: no
+</pre>
+<p class="handout">Most of the options above are pretty run of the mill.</p>
+<p class="handout">We specify the Python that we're going to use as a dependency. We
+build our Python RPMs so we can control what's in them. System
+packagers tend to be too creative for us.</p>
+<p class="handout">Normally, RPM installs files in their run-time locations at build
+time. This is undesirable in a number of ways. I used the rpm
+build-root mechanism to allow files to be build in a temporary
+tree.</p>
+<p class="handout">Because the build location is different than the final install
+location, paths written by the buildout, such as egg paths in
+scripts are wrong. There are a couple of ways to deal with this:</p>
+<ul class="handout simple">
+<li>I could try to adjust the paths at build time,</li>
+<li>I could try to adjust the paths at install time.</li>
+</ul>
+<p class="handout">Adjusting the paths at build time means that the install locations
+can;'t be controlled at install time. It would also add complexity
+to all recipes that deal with paths. Adjusting the paths at
+install time simply requires rerunning some of the recipes to
+generate the paths.</p>
+<p class="handout">To reinforce the decision to allow paths to be specified at install
+time, we've made the RPM relocatable using the prefix option.</p>
+</div>
+<div class="section">
+<h2><a id="zrs-spec-file-2-3" name="zrs-spec-file-2-3">ZRS spec file (2/3)</a></h2>
+<pre class="literal-block">
+%description
+%{summary}
+
+%build
+rm -rf $RPM_BUILD_ROOT
+mkdir $RPM_BUILD_ROOT
+mkdir $RPM_BUILD_ROOT/opt
+mkdir $RPM_BUILD_ROOT/etc
+mkdir $RPM_BUILD_ROOT/etc/init.d
+touch $RPM_BUILD_ROOT/etc/init.d/%{name}
+svn export %{svn_url} $RPM_BUILD_ROOT/opt/%{name}
+cd $RPM_BUILD_ROOT/opt/%{name}
+%{python} bootstrap.py -Uc rpm.cfg
+bin/buildout -Uc rpm.cfg buildout:installed= \
+ bootstrap:recipe=zc.rebootstrap
+</pre>
+<p class="handout">I'm not an RPM expert and RPM experts would probably cringe to see
+my spec file. RPM specifies a number of build steps that I've
+collapsed into one.</p>
+<ul class="handout simple">
+<li>The first few lines set up build root.</li>
+<li>We export the buildout into the build root.</li>
+<li>We run the buildout<ul>
+<li>The -U option is used mainly to avoid using a shared eggs
+directory</li>
+<li>The -c option is used to specify an RPM-specific buildout file
+that installs just software, including recipe eggs that will be
+needed after installation for configuration.</li>
+<li>We suppress creation of an .installed.cfg file</li>
+<li>We specify a recipe for a special bootstrap part. The bootstrap
+part is a script that will adjust the paths in the buildout
+script after installation of the rpm.</li>
+</ul>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="zrs-spec-file-3-3" name="zrs-spec-file-3-3">ZRS spec file (3/3)</a></h2>
+<pre class="literal-block">
+%post
+cd $RPM_INSTALL_PREFIX/%{name}
+%{python} bin/bootstrap -Uc rpmpost.cfg
+bin/buildout -Uc rpmpost.cfg \
+ buildout:offline=true buildout:find-links= buildout:installed= \
+ mercury:name=%{name} mercury:recipe=buildoutmercury
+chmod -R -w .
+
+%preun
+cd $RPM_INSTALL_PREFIX/%{name}
+chmod -R +w .
+find . -name \*.pyc | xargs rm -f
+
+%files
+%attr(-, root, root) /opt/%{name}
+%attr(744, root, root) /etc/init.d/%{name}
+</pre>
+<p class="handout">We specify a post-installation script that:</p>
+<ul class="handout simple">
+<li>Re-bootstraps the buildout using the special bootstrap script
+installed in the RPM.</li>
+<li>Reruns the buildout:<ul>
+<li>Using a post-installation configuration that specified the
+parts who's paths need to be adjusted.</li>
+<li>In offline mode because we don't want any network access or new
+software installed that isn't in the RPM.</li>
+<li>Removing any find links. This is largely due to a specific
+detail of our configurations.</li>
+<li>Suppressing the creation of .installed.cfg</li>
+<li>Specifying information for installing a special script that
+reads our centralized configuration database to configure the
+application after the RPM is installed.</li>
+</ul>
+</li>
+</ul>
+<p class="handout">We have a pre-uninstall script that cleans up .pyc files.</p>
+<p class="handout">We specify the files to be installed. This is just the buildout
+directory and a configuration script.</p>
+</div>
+<div class="section">
+<h2><a id="repeatability" name="repeatability">Repeatability</a></h2>
+<p>We want to be able to check certtain configuration into svn that can
+be checked out and reproduced.</p>
+<ul>
+<li><p class="first">We let buildout tell is what versions it picked for distributions</p>
+<ul>
+<li><p class="first">Run with -v</p>
+</li>
+<li><p class="first">Look for outout lines of form:</p>
+<pre class="literal-block">
+Picked: foo = 1.2
+</pre>
+</li>
+</ul>
+</li>
+<li><p class="first">Include a versions section:</p>
+<pre class="literal-block">
+[buildout]
+...
+versions = myversions
+
+[myversions]
+foo = 1.2
+...
+</pre>
+</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="deployment-issues" name="deployment-issues">Deployment issues</a></h2>
+<ul class="simple">
+<li>Need a way to record the versions of eggs used.</li>
+<li>Need a way to generate distributable buildouts that contain all of the source
+distributions needed to build on a target machine (e.g. source
+RPMs).</li>
+<li>Need to be able to generate source distributions. We need a way of
+gathering the sources used by a buildout so they can be distributed
+with it.</li>
+</ul>
+</div>
+<div class="section">
+<h2><a id="pypi-availability" name="pypi-availability">PyPI availability</a></h2>
+<p>A fairly significant issue is the availability of PyPI. PyPI is
+sometimes not available for minutes or hours at a time. This can cause
+buildout to become unusable.</p>
+</div>
+<div class="section">
+<h2><a id="for-more-information" name="for-more-information">For more information</a></h2>
+<p>See <a class="reference" href="http://www.python.org/pypi/zc.buildout">http://www.python.org/pypi/zc.buildout</a></p>
+</div>
+
+ </div>
+
+ <div class="roundbottom">
+ <img src="/resources/corner-bottomleft.jpg" alt="" width="45" height="45" class="corner" style="display: none" />
+ </div>
+
+</div>
+
+<div class="footer">
+
+ <table><tr><td>
+ Grok cooks around the campfire of <br />
+ <a href="http://wiki.zope.org/zope3/FrontPage"><img src="/resources/zopelogo.gif" alt="Zope" style="padding: 0.5em;" /></a>
+ </td><td>
+ and roams free on the savannah of<br />
+ <a href="http://www.python.org"><img src="/resources/python-logo.gif" style="padding: 0.5em;" alt="Python" /></a>
+ </td></tr>
+ </table>
+
+ <p>Hosting provided by <a href="http://quintagroup.com/"><b>Quintagroup</b></a></p>
+</div>
+
+</body>
+</html>
Modified: grok/www/minitutorials/index.html
===================================================================
--- grok/www/minitutorials/index.html 2007-08-04 03:54:22 UTC (rev 78572)
+++ grok/www/minitutorials/index.html 2007-08-04 03:56:13 UTC (rev 78573)
@@ -47,13 +47,26 @@
for searching. In other words, you have to define the indexes you
want to use to search you objects before you perform the actual
search.</p>
+<p>Author: Sebastian Ware</p>
</li>
<li><p class="first"><a class="reference" href="/minitutorials/macros.html">Macros With Grok</a>:</p>
<p>Macros are a way to define a chunk of presentation in one template,
and share it in others. Changes to the macro are immediately
reflected in all templates, that use it.</p>
+<p>Author: Uli Fouquet</p>
</li>
</ul>
+<div class="section">
+<h2><a id="buildout" name="buildout">Buildout</a></h2>
+<p>Buildout is a python-based configuration-driven build tool for working with
+eggs.</p>
+<ul>
+<li><p class="first"><a class="reference" href="/minitutorials/buildout.html">Buildout Tutorial</a>:</p>
+<p>This is Jim Fulton's tutorial for using <a class="reference" href="http://www.python.org/pypi/zc.buildout">buildout</a> The original text document is
+available at <a class="reference" href="http://svn.zope.org/zc.buildout/trunk/doc/tutorial.txt">http://svn.zope.org/zc.buildout/trunk/doc/tutorial.txt</a>.</p>
+</li>
+</ul>
+</div>
</div>
Modified: grok/www/minitutorials/macros.html
===================================================================
--- grok/www/minitutorials/macros.html 2007-08-04 03:54:22 UTC (rev 78572)
+++ grok/www/minitutorials/macros.html 2007-08-04 03:56:13 UTC (rev 78573)
@@ -38,6 +38,14 @@
<div class="content">
<h1 class="title">Mini-Howto: Macros with Grok</h1>
+<table class="docinfo" frame="void" rules="none">
+<col class="docinfo-name" />
+<col class="docinfo-content" />
+<tbody valign="top">
+<tr><th class="docinfo-name">Author:</th>
+<td>Uli Fouquet</td></tr>
+</tbody>
+</table>
<p>Intended Audience:</p>
<blockquote>
<ul class="simple">
Modified: grok/www/minitutorials/searching.html
===================================================================
--- grok/www/minitutorials/searching.html 2007-08-04 03:54:22 UTC (rev 78572)
+++ grok/www/minitutorials/searching.html 2007-08-04 03:56:13 UTC (rev 78573)
@@ -38,6 +38,14 @@
<div class="content">
<h1 class="title">Newbie Search Tutorial</h1>
+<table class="docinfo" frame="void" rules="none">
+<col class="docinfo-name" />
+<col class="docinfo-content" />
+<tbody valign="top">
+<tr><th class="docinfo-name">Author:</th>
+<td>Sebastian Ware</td></tr>
+</tbody>
+</table>
<div class="section">
<h2><a id="introduction" name="introduction">Introduction</a></h2>
<p>Grok supports the vanilla indexing services available in Zope 3 straight out of
More information about the Checkins
mailing list