Manual: Remove tabs, indent consistently

This commit is contained in:
Eelco Dolstra 2012-02-28 15:38:47 +01:00
parent da26294fdb
commit 918fc5e6df
2 changed files with 280 additions and 269 deletions

View file

@ -2,213 +2,217 @@
xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xlink="http://www.w3.org/1999/xlink"
xml:id="chap-installation"> xml:id="chap-installation">
<title>Installation</title>
<para>
This chapter explains how to install Hydra on your own build farm server.
</para>
<section>
<title>Prerequisites</title>
<para>
To install and use Hydra you need to have installed the following dependencies:
<itemizedlist>
<listitem>Nix</listitem>
<listitem>either PostgreSQL or SQLite</listitem>
<listitem>many Perl packages, notably Catalyst, EmailSender,
and NixPerl (see the <link
xlink:href="https://svn.nixos.org/repos/nix/nixpkgs/trunk/pkgs/development/tools/misc/hydra/default.nix">Hydra
expression in Nixpkgs</link> for the complete
list).</listitem>
</itemizedlist>
At the moment, Hydra runs only on GNU/Linux
(<emphasis>i686-linux</emphasis> and
<emphasis>x86_64_linux</emphasis>).
</para>
<para>
For small projects, Hydra can be run on any reasonably modern
machine. For individual projects you can even run Hydra on a
laptop. However, the charm of a buildfarm server is usually that
it operates without disturbing the developer's working
environment and can serve releases over the internet. In
conjunction you should typically have your source code
administered in a version management system, such as
subversion. Therefore, you will probably want to install a
server that is connected to the internet. To scale up to large
and/or many projects, you will need at least a considerable
amount of diskspace to store builds. Since Hydra can schedule
multiple simultaneous build jobs, it can be useful to have a
multi-core machine, and/or attach multiple build machines in a
network to the central Hydra server.
</para>
<para>
Of course we think it is a good idea to use the <a
href="http://nixos.org/nixos">NixOS</a> GNU/Linux distribution
for your buildfarm server. But this is not a requirement. The
Nix software deployment system can be installed on any GNU/Linux
distribution in parallel to the regular package management
system. Thus, you can use Hydra on a Debian, Fedora, SuSE, or
Ubuntu system.
</para>
</section>
<section>
<title>Getting Nix</title>
<para>
If your server runs NixOS you are all set to continue with
installation of Hydra. Otherwise you first need to install Nix.
The latest stable version can be found one <link
xlink:href="http://nixos.org/nix/download.html">the Nix web
site</link>, along with a manual, which includes installation
instructions.
</para>
</section>
<section>
<title>Installation</title> <title>Installation</title>
<para> <para>
This chapter explains how to install Hydra on your own build farm server. Hydra can be installed using Nixpkgs:
<screen>
nix-env -Ai hydra -f /path/to/nixpkgs</screen>
This makes the tools available in your Nix user environment,
<literal>$HOME/.nix-profile</literal> by default.
</para> </para>
<section> <para>
<title>Prerequisites</title> Alternatively, the latest development snapshot can be installed
<para> by visiting the URL <link
To install and use Hydra you need to have installed the following dependencies: xlink:href="http://hydra.nixos.org/view/hydra/unstable"><literal>http://hydra.nixos.org/view/hydra/unstable</literal></link>
and use the one-click install available at one of the build
<itemizedlist> pages. You can also install Hydra through the channel by
<listitem>Nix</listitem> performing the following commands:
<listitem>either PostgreSQL or SQLite</listitem>
<listitem>many Perl packages, notably Catalyst,
EmailSender, and NixPerl (see the <link
xlink:href="https://svn.nixos.org/repos/nix/nixpkgs/trunk/pkgs/development/tools/misc/hydra/default.nix">Hydra
expression in Nixpkgs</link> for the complete
list).</listitem>
</itemizedlist>
At the moment, Hydra runs only on GNU/Linux
(<emphasis>i686-linux</emphasis> and
<emphasis>x86_64_linux</emphasis>).
</para>
<para>
For small projects, Hydra can be run on any reasonably
modern machine. For individual projects you can even run
Hydra on a laptop. However, the charm of a buildfarm server
is usually that it operates without disturbing the
developer's working environment and can serve releases over
the internet. In conjunction you should typically have your
source code administered in a version management system,
such as subversion. Therefore, you will probably want to
install a server that is connected to the internet. To scale
up to large and/or many projects, you will need at least a
considerable amount of diskspace to store builds. Since
Hydra can schedule multiple simultaneous build jobs, it can
be useful to have a multi-core machine, and/or attach
multiple build machines in a network to the central Hydra
server.
</para>
<para>
Of course we think it is a good idea to use the <a
href="http://nixos.org/nixos">NixOS</a> GNU/Linux
distribution for your buildfarm server. But this is not a
requirement. The Nix software deployment system can be
installed on any GNU/Linux distribution in parallel to the
regular package management system. Thus, you can use Hydra
on a Debian, Fedora, SuSE, or Ubuntu system.
</para>
</section>
<section>
<title>Getting Nix</title>
<para>
If your server runs NixOS you are all set to continue with
installation of Hydra. Otherwise you first need to install
Nix. The latest stable version can be found one <link
xlink:href="http://nixos.org/nix/download.html">the Nix web
site</link>, along with a manual, which includes installation
instructions.
</para>
</section>
<section>
<title>Installation</title>
<para>
Hydra can be installed using Nixpkgs:
<screen>
nix-env -Ai hydra -f /path/to/nixpkgs</screen>
This makes the tools available in your Nix user environment,
<literal>$HOME/.nix-profile</literal> by default.
</para>
<para>
Alternatively, the latest development snapshot can be
installed by visiting the URL
<link xlink:href="http://hydra.nixos.org/view/hydra/unstable"><literal>http://hydra.nixos.org/view/hydra/unstable</literal></link>
and use the one-click install available at one of the build pages. You can also
install Hydra through the channel by performing the following commands:
<screen> <screen>
nix-channel --add http://hydra.nixos.org/jobset/hydra/trunk/channel/latest nix-channel --add http://hydra.nixos.org/jobset/hydra/trunk/channel/latest
nix-channel --update nix-channel --update
nix-env -i hydra</screen> nix-env -i hydra</screen>
</para> </para>
<para> <para>
Command completion should reveal a number of command-line tools from Hydra: Command completion should reveal a number of command-line tools from Hydra:
<screen> <screen>
hydra-build hydra-evaluator hydra-server hydra-build hydra-evaluator hydra-server
hydra-eval-jobs hydra-queue-runner hydra-update-gc-roots</screen> hydra-eval-jobs hydra-queue-runner hydra-update-gc-roots</screen>
</para> </para>
</section> </section>
<section> <section>
<title>Creating the database</title> <title>Creating the database</title>
<para> <para>
Hydra stores its results in a database, which can be a Hydra stores its results in a database, which can be a
PostgreSQL or SQLite database. The latter is easier to PostgreSQL or SQLite database. The latter is easier to setup,
setup, but the former scales better. but the former scales better.
</para> </para>
<para>To setup a PostgreSQL <para>
database with <emphasis>hydra</emphasis> as database name To setup a PostgreSQL database with <emphasis>hydra</emphasis>
and user name, issue the following commands: as database name and user name, issue the following commands:
<screen>
<screen>
createdb hydra createdb hydra
echo "CREATE USER hydra WITH PASSWORD '&lt;your-password&gt;' ;" | psql hydra echo "CREATE USER hydra WITH PASSWORD '&lt;your-password&gt;' ;" | psql hydra
cat $prefix/share/hydra/sql/hydra-postgresql.sql | psql hydra cat $prefix/share/hydra/sql/hydra-postgresql.sql | psql hydra
echo "GRANT ALL ON DATABASE hydra TO hydra;" | psql hydra</screen> echo "GRANT ALL ON DATABASE hydra TO hydra;" | psql hydra</screen>
Note that <emphasis>$prefix</emphasis> is the location of
Hydra in the nix store.
</para>
<para> Note that <emphasis>$prefix</emphasis> is the location of Hydra
For SQLite, the following command is all it takes to in the nix store.
create the database: </para>
<screen> <para>
cat $prefix/share/hydra/sql/hydra-sqlite.sql | sqlite3 /path/to/hydra.sqlite For SQLite, the following command is all it takes to create the
</screen> database:
</para>
<para> <screen>
To add a user <emphasis>root</emphasis> with <emphasis>admin</emphasis> privileges, execute: cat $prefix/share/hydra/sql/hydra-sqlite.sql | sqlite3 /path/to/hydra.sqlite</screen>
<screen> </para>
<para>
To add a user <emphasis>root</emphasis> with
<emphasis>admin</emphasis> privileges, execute:
<screen>
echo "INSERT INTO Users(userName, emailAddress, password) VALUES ('root', 'some@email.adress.com', '$(echo -n foobar | sha1sum | cut -c1-40)');" | psql hydra echo "INSERT INTO Users(userName, emailAddress, password) VALUES ('root', 'some@email.adress.com', '$(echo -n foobar | sha1sum | cut -c1-40)');" | psql hydra
echo "INSERT INTO UserRoles(userName, role) values('root', 'admin');" | psql hydra echo "INSERT INTO UserRoles(userName, role) values('root', 'admin');" | psql hydra</screen>
</screen>
For SQLite the same commands can be used, with
<command>psql hydra</command> replaced by
<command>sqlite3 /path/to/hydra.sqlite</command>.
</para>
<para> For SQLite the same commands can be used, with <command>psql
Hydra uses an environment variable to know which database hydra</command> replaced by <command>sqlite3
should be used, and a variable which point to a location /path/to/hydra.sqlite</command>.
that holds some state. To set these variables for a </para>
PostgreSQL database, add the following to the
<filename>.profile</filename> of the user running the
Hydra services.
<screen> <para>
Hydra uses an environment variable to know which database should
be used, and a variable which point to a location that holds
some state. To set these variables for a PostgreSQL database,
add the following to the <filename>.profile</filename> of the
user running the Hydra services.
<screen>
export HYDRA_DBI="dbi:Pg:dbname=hydra;host=localhost;" export HYDRA_DBI="dbi:Pg:dbname=hydra;host=localhost;"
export HYDRA_DATA=/var/lib/hydra</screen> export HYDRA_DATA=/var/lib/hydra</screen>
Make sure that the <emphasis>HYDRA_DATA</emphasis> Make sure that the <emphasis>HYDRA_DATA</emphasis> directory
directory exists and is writable for the user which will exists and is writable for the user which will run the Hydra
run the Hydra services. For a SQLite database, the services. For a SQLite database, the
<varname>HYDRA_DBI</varname> should be set to something <varname>HYDRA_DBI</varname> should be set to something like
like <literal>dbi:SQLite:/path/to/hydra.sqlite</literal> <literal>dbi:SQLite:/path/to/hydra.sqlite</literal>
</para>
</section>
</para> <section>
</section> <title>Getting Started</title>
<section> <para>
<title>Getting Started</title> To start the Hydra web server, execute:
<screen>
<para>
To start the Hydra web server, execute:
<screen>
hydra-server</screen> hydra-server</screen>
When the server is started, you can browse to When the server is started, you can browse to
<ulink>http://localhost:3000/</ulink> to start configuring <ulink>http://localhost:3000/</ulink> to start configuring
your Hydra instance. your Hydra instance.
</para> </para>
<para> <para>
The <command>hydra-server</command> command launches the The <command>hydra-server</command> command launches the web
web server. There are two other processes that come into server. There are two other processes that come into play:
play:
<itemizedlist> <itemizedlist>
<listitem> <listitem>
The <emphasis>evaluator</emphasis> is responsible for The <emphasis>evaluator</emphasis> is responsible for
peridically evaluating job sets, checking out their peridically evaluating job sets, checking out their
dependencies off their version control systems (VCS), dependencies off their version control systems (VCS), and
and queueing new builds if the result of the evaluation queueing new builds if the result of the evaluation changed.
changed. It is launched by the It is launched by the <command>hydra-evaluator</command>
<command>hydra-evaluator</command> command. command.
</listitem> </listitem>
<listitem> <listitem>
The <emphasis>queue runner</emphasis> launches builds The <emphasis>queue runner</emphasis> launches builds (using
(using Nix) as they are queued by the evaluator, Nix) as they are queued by the evaluator, scheduling them
scheduling them onto the configured Nix hosts. It is onto the configured Nix hosts. It is launched using the
launched using the <command>hydra-queue-runner</command> command.
<command>hydra-queue-runner</command> command. </listitem>
</listitem> </itemizedlist>
</itemizedlist>
All three processes must be running for Hydra to be fully
functional, though it's possible to temporarily stop any one
of them for maintenance purposes, for instance.
</para>
</section>
All three processes must be running for Hydra to be fully
functional, though it's possible to temporarily stop any one of
them for maintenance purposes, for instance.
</para>
</section>
</chapter> </chapter>
<!--
Local Variables:
indent-tabs-mode: nil
ispell-local-dictionary: "american"
End:
-->

View file

@ -19,67 +19,67 @@
more in-depth testing than what developers could feasibly do manually: more in-depth testing than what developers could feasibly do manually:
<itemizedlist> <itemizedlist>
<listitem> <emphasis>Portability testing</emphasis>: The <listitem> <emphasis>Portability testing</emphasis>: The
software may need to be built and tested on many different software may need to be built and tested on many different
platforms. It is infeasible for each developer to do this platforms. It is infeasible for each developer to do this
before every commit. before every commit.
</listitem> </listitem>
<listitem> Likewise, many projects have very large test sets <listitem> Likewise, many projects have very large test sets
(e.g., regression tests in a compiler, or stress tests in a (e.g., regression tests in a compiler, or stress tests in a
DBMS) that can take hours or days to run to completion. DBMS) that can take hours or days to run to completion.
</listitem> </listitem>
<listitem> Many kinds of static and dynamic analyses can be <listitem> Many kinds of static and dynamic analyses can be
performed as part of the tests, such as code coverage runs and performed as part of the tests, such as code coverage runs and
static analyses. static analyses.
</listitem> </listitem>
<listitem> It may also be necessary to build many different <listitem> It may also be necessary to build many different
<emphasis>variants</emphasis> of the software. For instance, <emphasis>variants</emphasis> of the software. For instance,
it may be necessary to verify that the component builds with it may be necessary to verify that the component builds with
various versions of a compiler. various versions of a compiler.
</listitem> </listitem>
<listitem> Developers typically use incremental building to <listitem> Developers typically use incremental building to
test their changes (since a full build may take too long), but test their changes (since a full build may take too long), but
this is unreliable with many build management tools (such as this is unreliable with many build management tools (such as
Make), i.e., the result of the incremental build might differ Make), i.e., the result of the incremental build might differ
from a full build. from a full build.
</listitem> </listitem>
<listitem> It ensures that the software can be built from the <listitem> It ensures that the software can be built from the
sources under revision control. Users of version management sources under revision control. Users of version management
systems such as CVS and Subversion often forget to place systems such as CVS and Subversion often forget to place
source files under revision control. source files under revision control.
</listitem> </listitem>
<listitem> The machines on which the continuous integration <listitem> The machines on which the continuous integration
system runs ideally provides a clean, well-defined build system runs ideally provides a clean, well-defined build
environment. If this environment is administered through environment. If this environment is administered through
proper SCM techniques, then builds produced by the system can proper SCM techniques, then builds produced by the system can
be reproduced. In contrast, developer work environments are be reproduced. In contrast, developer work environments are
typically not under any kind of SCM control. typically not under any kind of SCM control.
</listitem> </listitem>
<listitem> In large projects, developers often work on a <listitem> In large projects, developers often work on a
particular component of the project, and do not build and test particular component of the project, and do not build and test
the composition of those components (again since this is the composition of those components (again since this is
likely to take too long). To prevent the phenomenon of ``big likely to take too long). To prevent the phenomenon of ``big
bang integration'', where components are only tested together bang integration'', where components are only tested together
near the end of the development process, it is important to near the end of the development process, it is important to
test components together as soon as possible (hence test components together as soon as possible (hence
<emphasis>continuous integration</emphasis>). <emphasis>continuous integration</emphasis>).
</listitem> </listitem>
<listitem> It allows software to be <listitem> It allows software to be
<emphasis>released</emphasis> by automatically creating <emphasis>released</emphasis> by automatically creating
packages that users can download and install. To do this packages that users can download and install. To do this
manually represents an often prohibitive amount of work, as manually represents an often prohibitive amount of work, as
one may want to produce releases for many different platforms: one may want to produce releases for many different platforms:
e.g., installers for Windows and Mac OS X, RPM or Debian e.g., installers for Windows and Mac OS X, RPM or Debian
packages for certain Linux distributions, and so on. packages for certain Linux distributions, and so on.
</listitem> </listitem>
</itemizedlist> </itemizedlist>
</para> </para>
@ -92,19 +92,19 @@
<itemizedlist> <itemizedlist>
<listitem>It obtains the latest version of the component's <listitem>It obtains the latest version of the component's
source code from the version management system. source code from the version management system.
</listitem> </listitem>
<listitem> It runs the component's build process (which <listitem> It runs the component's build process (which
presumably includes the execution of the component's test presumably includes the execution of the component's test
set). set).
</listitem> </listitem>
<listitem> It presents the results of the build (such as error <listitem> It presents the results of the build (such as error
logs and releases) to the developers, e.g., by producing a web logs and releases) to the developers, e.g., by producing a web
page. page.
</listitem> </listitem>
</itemizedlist> </itemizedlist>
@ -114,40 +114,40 @@
<itemizedlist> <itemizedlist>
<listitem> They do not manage the <emphasis>build <listitem> They do not manage the <emphasis>build
environment</emphasis>. The build environment consists of the environment</emphasis>. The build environment consists of the
dependencies necessary to perform a build action, e.g., dependencies necessary to perform a build action, e.g.,
compilers, libraries, etc. Setting up the environment is compilers, libraries, etc. Setting up the environment is
typically done manually, and without proper SCM control (so it typically done manually, and without proper SCM control (so it
may be hard to reproduce a build at a later time). Manual may be hard to reproduce a build at a later time). Manual
management of the environment scales poorly in the number of management of the environment scales poorly in the number of
configurations that must be supported. For instance, suppose configurations that must be supported. For instance, suppose
that we want to build a component that requires a certain that we want to build a component that requires a certain
compiler X. We then have to go to each machine and install X. compiler X. We then have to go to each machine and install X.
If we later need a newer version of X, the process must be If we later need a newer version of X, the process must be
repeated all over again. An ever worse problem occurs if repeated all over again. An ever worse problem occurs if
there are conflicting, mutually exclusive versions of the there are conflicting, mutually exclusive versions of the
dependencies. Thus, simply installing the latest version is dependencies. Thus, simply installing the latest version is
not an option. Of course, we can install these components in not an option. Of course, we can install these components in
different directories and manually pass the appropriate paths different directories and manually pass the appropriate paths
to the build processes of the various components. But this is to the build processes of the various components. But this is
a rather tiresome and error-prone process. a rather tiresome and error-prone process.
</listitem> </listitem>
<listitem> They do not easily support <emphasis>variability in software <listitem> They do not easily support <emphasis>variability in software
systems</emphasis>. A system may have a great deal of build-time systems</emphasis>. A system may have a great deal of build-time
variability: optional functionality, whether to build a debug or variability: optional functionality, whether to build a debug or
production version, different versions of dependencies, and so on. production version, different versions of dependencies, and so on.
(For instance, the Linux kernel now has over 2,600 build-time (For instance, the Linux kernel now has over 2,600 build-time
configuration switches.) It is therefore important that a continuous configuration switches.) It is therefore important that a continuous
integration tool can easily select and test different instances from integration tool can easily select and test different instances from
the configuration space of the system to reveal problems, such as the configuration space of the system to reveal problems, such as
erroneous interactions between features. In a continuous integration erroneous interactions between features. In a continuous integration
setting, it is also useful to test different combinations of versions setting, it is also useful to test different combinations of versions
of subsystems, e.g., the head revision of a component against stable of subsystems, e.g., the head revision of a component against stable
releases of its dependencies, and vice versa, as this can reveal releases of its dependencies, and vice versa, as this can reveal
various integration problems. various integration problems.
</listitem> </listitem>
</itemizedlist> </itemizedlist>
</para> </para>
@ -258,3 +258,10 @@
</section> </section>
</chapter> </chapter>
<!--
Local Variables:
indent-tabs-mode: nil
ispell-local-dictionary: "american"
End:
-->