doc: Reintegrate the intro by Visser & Dolstra from `manual.html'.
The `manual.html' file had been deleted in r21718 ("Hydra/28: Rename
"scheduler" to "evaluator"".)
This commit is contained in:
Ludovic Courtès
parent
0a80c737d0
commit
b147e71dcd
|
|
@ -2,6 +2,259 @@
|
|||
xmlns:xlink="http://www.w3.org/1999/xlink"
|
||||
xml:id="chap-introduction">
|
||||
|
||||
<title>Introduction</title>
|
||||
<title>Introduction</title>
|
||||
|
||||
<section>
|
||||
<title>About Hydra</title>
|
||||
|
||||
<para>
|
||||
Hydra is a tool for continuous integration testing and software
|
||||
release that uses a purely functional language to describe build jobs
|
||||
and their dependencies. Continuous integration is a simple technique
|
||||
to improve the quality of the software development process. An
|
||||
automated system continuously or periodically checks out the source
|
||||
code of a project, builds it, runs tests, and produces reports for the
|
||||
developers. Thus, various errors that might accidentally be committed
|
||||
into the code base are automatically caught. Such a system allows
|
||||
more in-depth testing than what developers could feasibly do manually:
|
||||
|
||||
<itemizedlist>
|
||||
<listitem> <emphasis>Portability testing</emphasis>: The
|
||||
software may need to be built and tested on many different
|
||||
platforms. It is infeasible for each developer to do this
|
||||
before every commit.
|
||||
</listitem>
|
||||
|
||||
<listitem> Likewise, many projects have very large test sets
|
||||
(e.g., regression tests in a compiler, or stress tests in a
|
||||
DBMS) that can take hours or days to run to completion.
|
||||
</listitem>
|
||||
|
||||
<listitem> Many kinds of static and dynamic analyses can be
|
||||
performed as part of the tests, such as code coverage runs and
|
||||
static analyses.
|
||||
</listitem>
|
||||
|
||||
<listitem> It may also be necessary to build many different
|
||||
<emphasis>variants</emphasis> of the software. For instance,
|
||||
it may be necessary to verify that the component builds with
|
||||
various versions of a compiler.
|
||||
</listitem>
|
||||
|
||||
<listitem> Developers typically use incremental building to
|
||||
test their changes (since a full build may take too long), but
|
||||
this is unreliable with many build management tools (such as
|
||||
Make), i.e., the result of the incremental build might differ
|
||||
from a full build.
|
||||
</listitem>
|
||||
|
||||
<listitem> It ensures that the software can be built from the
|
||||
sources under revision control. Users of version management
|
||||
systems such as CVS and Subversion often forget to place
|
||||
source files under revision control.
|
||||
</listitem>
|
||||
|
||||
<listitem> The machines on which the continuous integration
|
||||
system runs ideally provides a clean, well-defined build
|
||||
environment. If this environment is administered through
|
||||
proper SCM techniques, then builds produced by the system can
|
||||
be reproduced. In contrast, developer work environments are
|
||||
typically not under any kind of SCM control.
|
||||
</listitem>
|
||||
|
||||
<listitem> In large projects, developers often work on a
|
||||
particular component of the project, and do not build and test
|
||||
the composition of those components (again since this is
|
||||
likely to take too long). To prevent the phenomenon of ``big
|
||||
bang integration'', where components are only tested together
|
||||
near the end of the development process, it is important to
|
||||
test components together as soon as possible (hence
|
||||
<emphasis>continuous integration</emphasis>).
|
||||
</listitem>
|
||||
|
||||
<listitem> It allows software to be
|
||||
<emphasis>released</emphasis> by automatically creating
|
||||
packages that users can download and install. To do this
|
||||
manually represents an often prohibitive amount of work, as
|
||||
one may want to produce releases for many different platforms:
|
||||
e.g., installers for Windows and Mac OS X, RPM or Debian
|
||||
packages for certain Linux distributions, and so on.
|
||||
</listitem>
|
||||
|
||||
</itemizedlist>
|
||||
</para>
|
||||
|
||||
<para>
|
||||
In its simplest form, a continuous integration tool sits in a
|
||||
loop building and releasing software components from a version
|
||||
management system. For each component, it performs the
|
||||
following tasks:
|
||||
|
||||
<itemizedlist>
|
||||
|
||||
<listitem>It obtains the latest version of the component's
|
||||
source code from the version management system.
|
||||
</listitem>
|
||||
|
||||
<listitem> It runs the component's build process (which
|
||||
presumably includes the execution of the component's test
|
||||
set).
|
||||
</listitem>
|
||||
|
||||
<listitem> It presents the results of the build (such as error
|
||||
logs and releases) to the developers, e.g., by producing a web
|
||||
page.
|
||||
</listitem>
|
||||
|
||||
</itemizedlist>
|
||||
|
||||
Examples of continuous integration tools include Jenkins,
|
||||
CruiseControl Tinderbox, Sisyphus, Anthill and BuildBot. These
|
||||
tools have various limitations.
|
||||
|
||||
<itemizedlist>
|
||||
|
||||
<listitem> They do not manage the <emphasis>build
|
||||
environment</emphasis>. The build environment consists of the
|
||||
dependencies necessary to perform a build action, e.g.,
|
||||
compilers, libraries, etc. Setting up the environment is
|
||||
typically done manually, and without proper SCM control (so it
|
||||
may be hard to reproduce a build at a later time). Manual
|
||||
management of the environment scales poorly in the number of
|
||||
configurations that must be supported. For instance, suppose
|
||||
that we want to build a component that requires a certain
|
||||
compiler X. We then have to go to each machine and install X.
|
||||
If we later need a newer version of X, the process must be
|
||||
repeated all over again. An ever worse problem occurs if
|
||||
there are conflicting, mutually exclusive versions of the
|
||||
dependencies. Thus, simply installing the latest version is
|
||||
not an option. Of course, we can install these components in
|
||||
different directories and manually pass the appropriate paths
|
||||
to the build processes of the various components. But this is
|
||||
a rather tiresome and error-prone process.
|
||||
</listitem>
|
||||
|
||||
<listitem> They do not easily support <emphasis>variability in software
|
||||
systems</emphasis>. A system may have a great deal of build-time
|
||||
variability: optional functionality, whether to build a debug or
|
||||
production version, different versions of dependencies, and so on.
|
||||
(For instance, the Linux kernel now has over 2,600 build-time
|
||||
configuration switches.) It is therefore important that a continuous
|
||||
integration tool can easily select and test different instances from
|
||||
the configuration space of the system to reveal problems, such as
|
||||
erroneous interactions between features. In a continuous integration
|
||||
setting, it is also useful to test different combinations of versions
|
||||
of subsystems, e.g., the head revision of a component against stable
|
||||
releases of its dependencies, and vice versa, as this can reveal
|
||||
various integration problems.
|
||||
</listitem>
|
||||
|
||||
</itemizedlist>
|
||||
</para>
|
||||
|
||||
<para>
|
||||
<emphasis>Hydra</emphasis>, is a continuous integration tool
|
||||
that solves these problems. It is built on top of the <link
|
||||
xlink:href="http://nixos.org/nix/">Nix package manager</link>,
|
||||
which has a purely functional language for describing package
|
||||
build actions and their dependencies. This allows the build
|
||||
environment for projects to be produced automatically and
|
||||
deterministically, and variability in components to be expressed
|
||||
naturally using functions; and as such is an ideal fit for a
|
||||
continuous build system.
|
||||
</para>
|
||||
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<title>About Us</title>
|
||||
|
||||
<para>
|
||||
Hydra is the successor of the Nix Buildfarm, which was developed
|
||||
in tandem with the Nix software deployment system. Nix was
|
||||
originally developed at the Department of Information and
|
||||
Computing Sciences, Utrecht University by the TraCE project
|
||||
(2003-2008). The project was funded by the Software Engineering
|
||||
Research Program Jacquard to improve the support for variability
|
||||
in software systems. Funding for the development of Nix and
|
||||
Hydra is now provided by the NIRICT LaQuSo Build Farm project.
|
||||
</para>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<title>About this Manual</title>
|
||||
|
||||
<para>
|
||||
This manual tells you how to install the Hydra buildfarm
|
||||
software on your own server and how to operate that server using
|
||||
its web interface.
|
||||
</para>
|
||||
</section>
|
||||
|
||||
|
||||
<section>
|
||||
<title>License</title>
|
||||
|
||||
<para>
|
||||
Hydra is free software: you can redistribute it and/or
|
||||
modify it under the terms of the GNU General Public License as
|
||||
published by the Free Software Foundation, either version 3 of
|
||||
the License, or (at your option) any later version.
|
||||
</para>
|
||||
|
||||
<para>
|
||||
Hydra is distributed in the hope that it will be useful,
|
||||
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
<link xlink:href="http://www.gnu.org/licenses/">GNU General
|
||||
Public License</link> for more details.
|
||||
</para>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<title>Hydra at <literal>nixos.org</literal></title>
|
||||
|
||||
<para>
|
||||
The <literal>nixos.org</literal> installation of Hydra runs at
|
||||
<link
|
||||
xlink:href="http://hydra.nixos.org/"><literal>http://hydra.nixos.org/</literal></link>.
|
||||
|
||||
That installation is used to build software components from the
|
||||
<link xlink:href="http://nixos.org">Nix</link>,
|
||||
<link xlink:href="http://nixos.org/nixos">NixOS</link>,
|
||||
<link xlink:href="http://www.gnu.org/">GNU</link>,
|
||||
<link xlink:href="http://strategoxt.org">Stratego/XT</link>,
|
||||
and related projects.
|
||||
</para>
|
||||
|
||||
<para>
|
||||
If you are one of the developers on those projects, it is likely
|
||||
that you will be using the NixOS Hydra server in some way. If
|
||||
you need to administer automatic builds for your project, you
|
||||
should pull the right strings to get an account on the
|
||||
server. This manual will tell you how to set up new projects and
|
||||
build jobs within those projects and write a release.nix file to
|
||||
describe the build process of your project to Hydra. You can
|
||||
skip the next chapter.
|
||||
</para>
|
||||
|
||||
<para>
|
||||
If your project does not yet have automatic builds within the
|
||||
NixOS Hydra server, it may actually be eligible. We are in the
|
||||
process of setting up a large buildfarm that should be able to
|
||||
support open source and academic software projects. Get in
|
||||
touch.
|
||||
</para>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<title>Hydra on your own buildfarm</title>
|
||||
|
||||
<para>
|
||||
If you need to run your own Hydra installation, <xref
|
||||
linkend="chap-installation" /> explains how to download and
|
||||
install the system on your own server.
|
||||
</para>
|
||||
</section>
|
||||
|
||||
</chapter>
|
||||
|
|
|
|||
|
|
@ -30,6 +30,17 @@
|
|||
</affiliation>
|
||||
<contrib>Author</contrib>
|
||||
</author>
|
||||
<author>
|
||||
<personname>
|
||||
<firstname>Eelco</firstname>
|
||||
<surname>Visser</surname>
|
||||
</personname>
|
||||
<affiliation>
|
||||
<orgname>Delft University of Technology</orgname>
|
||||
<orgdiv>Department of Software Technology</orgdiv>
|
||||
</affiliation>
|
||||
<contrib>Author</contrib>
|
||||
</author>
|
||||
</authorgroup>
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue