268 lines
11 KiB
XML
268 lines
11 KiB
XML
<chapter xmlns="http://docbook.org/ns/docbook"
|
|
xmlns:xlink="http://www.w3.org/1999/xlink"
|
|
xml:id="chap-introduction">
|
|
|
|
<title>Introduction</title>
|
|
|
|
<section>
|
|
<title>About Hydra</title>
|
|
|
|
<para>
|
|
Hydra is a tool for continuous integration testing and software
|
|
release that uses a purely functional language to describe build jobs
|
|
and their dependencies. Continuous integration is a simple technique
|
|
to improve the quality of the software development process. An
|
|
automated system continuously or periodically checks out the source
|
|
code of a project, builds it, runs tests, and produces reports for the
|
|
developers. Thus, various errors that might accidentally be committed
|
|
into the code base are automatically caught. Such a system allows
|
|
more in-depth testing than what developers could feasibly do manually:
|
|
|
|
<itemizedlist>
|
|
<listitem> <emphasis>Portability testing</emphasis>: The
|
|
software may need to be built and tested on many different
|
|
platforms. It is infeasible for each developer to do this
|
|
before every commit.
|
|
</listitem>
|
|
|
|
<listitem> Likewise, many projects have very large test sets
|
|
(e.g., regression tests in a compiler, or stress tests in a
|
|
DBMS) that can take hours or days to run to completion.
|
|
</listitem>
|
|
|
|
<listitem> Many kinds of static and dynamic analyses can be
|
|
performed as part of the tests, such as code coverage runs and
|
|
static analyses.
|
|
</listitem>
|
|
|
|
<listitem> It may also be necessary to build many different
|
|
<emphasis>variants</emphasis> of the software. For instance,
|
|
it may be necessary to verify that the component builds with
|
|
various versions of a compiler.
|
|
</listitem>
|
|
|
|
<listitem> Developers typically use incremental building to
|
|
test their changes (since a full build may take too long), but
|
|
this is unreliable with many build management tools (such as
|
|
Make), i.e., the result of the incremental build might differ
|
|
from a full build.
|
|
</listitem>
|
|
|
|
<listitem> It ensures that the software can be built from the
|
|
sources under revision control. Users of version management
|
|
systems such as CVS and Subversion often forget to place
|
|
source files under revision control.
|
|
</listitem>
|
|
|
|
<listitem> The machines on which the continuous integration
|
|
system runs ideally provides a clean, well-defined build
|
|
environment. If this environment is administered through
|
|
proper SCM techniques, then builds produced by the system can
|
|
be reproduced. In contrast, developer work environments are
|
|
typically not under any kind of SCM control.
|
|
</listitem>
|
|
|
|
<listitem> In large projects, developers often work on a
|
|
particular component of the project, and do not build and test
|
|
the composition of those components (again since this is
|
|
likely to take too long). To prevent the phenomenon of ``big
|
|
bang integration'', where components are only tested together
|
|
near the end of the development process, it is important to
|
|
test components together as soon as possible (hence
|
|
<emphasis>continuous integration</emphasis>).
|
|
</listitem>
|
|
|
|
<listitem> It allows software to be
|
|
<emphasis>released</emphasis> by automatically creating
|
|
packages that users can download and install. To do this
|
|
manually represents an often prohibitive amount of work, as
|
|
one may want to produce releases for many different platforms:
|
|
e.g., installers for Windows and Mac OS X, RPM or Debian
|
|
packages for certain Linux distributions, and so on.
|
|
</listitem>
|
|
|
|
</itemizedlist>
|
|
</para>
|
|
|
|
<para>
|
|
In its simplest form, a continuous integration tool sits in a
|
|
loop building and releasing software components from a version
|
|
management system. For each component, it performs the
|
|
following tasks:
|
|
|
|
<itemizedlist>
|
|
|
|
<listitem>It obtains the latest version of the component's
|
|
source code from the version management system.
|
|
</listitem>
|
|
|
|
<listitem> It runs the component's build process (which
|
|
presumably includes the execution of the component's test
|
|
set).
|
|
</listitem>
|
|
|
|
<listitem> It presents the results of the build (such as error
|
|
logs and releases) to the developers, e.g., by producing a web
|
|
page.
|
|
</listitem>
|
|
|
|
</itemizedlist>
|
|
|
|
Examples of continuous integration tools include Jenkins,
|
|
CruiseControl Tinderbox, Sisyphus, Anthill and BuildBot. These
|
|
tools have various limitations.
|
|
|
|
<itemizedlist>
|
|
|
|
<listitem> They do not manage the <emphasis>build
|
|
environment</emphasis>. The build environment consists of the
|
|
dependencies necessary to perform a build action, e.g.,
|
|
compilers, libraries, etc. Setting up the environment is
|
|
typically done manually, and without proper SCM control (so it
|
|
may be hard to reproduce a build at a later time). Manual
|
|
management of the environment scales poorly in the number of
|
|
configurations that must be supported. For instance, suppose
|
|
that we want to build a component that requires a certain
|
|
compiler X. We then have to go to each machine and install X.
|
|
If we later need a newer version of X, the process must be
|
|
repeated all over again. An ever worse problem occurs if
|
|
there are conflicting, mutually exclusive versions of the
|
|
dependencies. Thus, simply installing the latest version is
|
|
not an option. Of course, we can install these components in
|
|
different directories and manually pass the appropriate paths
|
|
to the build processes of the various components. But this is
|
|
a rather tiresome and error-prone process.
|
|
</listitem>
|
|
|
|
<listitem> They do not easily support <emphasis>variability in software
|
|
systems</emphasis>. A system may have a great deal of build-time
|
|
variability: optional functionality, whether to build a debug or
|
|
production version, different versions of dependencies, and so on.
|
|
(For instance, the Linux kernel now has over 2,600 build-time
|
|
configuration switches.) It is therefore important that a continuous
|
|
integration tool can easily select and test different instances from
|
|
the configuration space of the system to reveal problems, such as
|
|
erroneous interactions between features. In a continuous integration
|
|
setting, it is also useful to test different combinations of versions
|
|
of subsystems, e.g., the head revision of a component against stable
|
|
releases of its dependencies, and vice versa, as this can reveal
|
|
various integration problems.
|
|
</listitem>
|
|
|
|
</itemizedlist>
|
|
</para>
|
|
|
|
<para>
|
|
<emphasis>Hydra</emphasis>, is a continuous integration tool
|
|
that solves these problems. It is built on top of the <link
|
|
xlink:href="http://nixos.org/nix/">Nix package manager</link>,
|
|
which has a purely functional language for describing package
|
|
build actions and their dependencies. This allows the build
|
|
environment for projects to be produced automatically and
|
|
deterministically, and variability in components to be expressed
|
|
naturally using functions; and as such is an ideal fit for a
|
|
continuous build system.
|
|
</para>
|
|
|
|
</section>
|
|
|
|
<section>
|
|
<title>About Us</title>
|
|
|
|
<para>
|
|
Hydra is the successor of the Nix Buildfarm, which was developed
|
|
in tandem with the Nix software deployment system. Nix was
|
|
originally developed at the Department of Information and
|
|
Computing Sciences, Utrecht University by the TraCE project
|
|
(2003-2008). The project was funded by the Software Engineering
|
|
Research Program Jacquard to improve the support for variability
|
|
in software systems. Funding for the development of Nix and
|
|
Hydra is now provided by the NIRICT LaQuSo Build Farm project.
|
|
</para>
|
|
</section>
|
|
|
|
<section>
|
|
<title>About this Manual</title>
|
|
|
|
<para>
|
|
This manual tells you how to install the Hydra buildfarm
|
|
software on your own server and how to operate that server using
|
|
its web interface.
|
|
</para>
|
|
</section>
|
|
|
|
|
|
<section>
|
|
<title>License</title>
|
|
|
|
<para>
|
|
Hydra is free software: you can redistribute it and/or
|
|
modify it under the terms of the GNU General Public License as
|
|
published by the Free Software Foundation, either version 3 of
|
|
the License, or (at your option) any later version.
|
|
</para>
|
|
|
|
<para>
|
|
Hydra is distributed in the hope that it will be useful,
|
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
<link xlink:href="http://www.gnu.org/licenses/">GNU General
|
|
Public License</link> for more details.
|
|
</para>
|
|
</section>
|
|
|
|
<section>
|
|
<title>Hydra at <literal>nixos.org</literal></title>
|
|
|
|
<para>
|
|
The <literal>nixos.org</literal> installation of Hydra runs at
|
|
<link
|
|
xlink:href="http://hydra.nixos.org/"><literal>http://hydra.nixos.org/</literal></link>.
|
|
|
|
That installation is used to build software components from the
|
|
<link xlink:href="http://nixos.org">Nix</link>,
|
|
<link xlink:href="http://nixos.org/nixos">NixOS</link>,
|
|
<link xlink:href="http://www.gnu.org/">GNU</link>,
|
|
<link xlink:href="http://strategoxt.org">Stratego/XT</link>,
|
|
and related projects.
|
|
</para>
|
|
|
|
<para>
|
|
If you are one of the developers on those projects, it is likely
|
|
that you will be using the NixOS Hydra server in some way. If
|
|
you need to administer automatic builds for your project, you
|
|
should pull the right strings to get an account on the
|
|
server. This manual will tell you how to set up new projects and
|
|
build jobs within those projects and write a release.nix file to
|
|
describe the build process of your project to Hydra. You can
|
|
skip the next chapter.
|
|
</para>
|
|
|
|
<para>
|
|
If your project does not yet have automatic builds within the
|
|
NixOS Hydra server, it may actually be eligible. We are in the
|
|
process of setting up a large buildfarm that should be able to
|
|
support open source and academic software projects. Get in
|
|
touch.
|
|
</para>
|
|
</section>
|
|
|
|
<section>
|
|
<title>Hydra on your own buildfarm</title>
|
|
|
|
<para>
|
|
If you need to run your own Hydra installation, <xref
|
|
linkend="chap-installation" /> explains how to download and
|
|
install the system on your own server.
|
|
</para>
|
|
</section>
|
|
|
|
</chapter>
|
|
|
|
<!--
|
|
Local Variables:
|
|
indent-tabs-mode: nil
|
|
ispell-local-dictionary: "american"
|
|
End:
|
|
-->
|