diff --git a/doc/manual/installation.xml b/doc/manual/installation.xml index 489f85e6..44280524 100644 --- a/doc/manual/installation.xml +++ b/doc/manual/installation.xml @@ -2,213 +2,217 @@ xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="chap-installation"> + Installation + + + This chapter explains how to install Hydra on your own build farm server. + + +
+ Prerequisites + + To install and use Hydra you need to have installed the following dependencies: + + + Nix + either PostgreSQL or SQLite + many Perl packages, notably Catalyst, EmailSender, + and NixPerl (see the Hydra + expression in Nixpkgs for the complete + list). + + + At the moment, Hydra runs only on GNU/Linux + (i686-linux and + x86_64_linux). + + + + For small projects, Hydra can be run on any reasonably modern + machine. For individual projects you can even run Hydra on a + laptop. However, the charm of a buildfarm server is usually that + it operates without disturbing the developer's working + environment and can serve releases over the internet. In + conjunction you should typically have your source code + administered in a version management system, such as + subversion. Therefore, you will probably want to install a + server that is connected to the internet. To scale up to large + and/or many projects, you will need at least a considerable + amount of diskspace to store builds. Since Hydra can schedule + multiple simultaneous build jobs, it can be useful to have a + multi-core machine, and/or attach multiple build machines in a + network to the central Hydra server. + + + + Of course we think it is a good idea to use the NixOS GNU/Linux distribution + for your buildfarm server. But this is not a requirement. The + Nix software deployment system can be installed on any GNU/Linux + distribution in parallel to the regular package management + system. Thus, you can use Hydra on a Debian, Fedora, SuSE, or + Ubuntu system. + + +
+ +
+ Getting Nix + + + If your server runs NixOS you are all set to continue with + installation of Hydra. Otherwise you first need to install Nix. + The latest stable version can be found one the Nix web + site, along with a manual, which includes installation + instructions. + +
+ +
Installation - This chapter explains how to install Hydra on your own build farm server. + Hydra can be installed using Nixpkgs: + + +nix-env -Ai hydra -f /path/to/nixpkgs + + This makes the tools available in your Nix user environment, + $HOME/.nix-profile by default. -
- Prerequisites - - To install and use Hydra you need to have installed the following dependencies: - - - Nix - either PostgreSQL or SQLite - many Perl packages, notably Catalyst, - EmailSender, and NixPerl (see the Hydra - expression in Nixpkgs for the complete - list). - - - At the moment, Hydra runs only on GNU/Linux - (i686-linux and - x86_64_linux). - - - - For small projects, Hydra can be run on any reasonably - modern machine. For individual projects you can even run - Hydra on a laptop. However, the charm of a buildfarm server - is usually that it operates without disturbing the - developer's working environment and can serve releases over - the internet. In conjunction you should typically have your - source code administered in a version management system, - such as subversion. Therefore, you will probably want to - install a server that is connected to the internet. To scale - up to large and/or many projects, you will need at least a - considerable amount of diskspace to store builds. Since - Hydra can schedule multiple simultaneous build jobs, it can - be useful to have a multi-core machine, and/or attach - multiple build machines in a network to the central Hydra - server. - - - - Of course we think it is a good idea to use the NixOS GNU/Linux - distribution for your buildfarm server. But this is not a - requirement. The Nix software deployment system can be - installed on any GNU/Linux distribution in parallel to the - regular package management system. Thus, you can use Hydra - on a Debian, Fedora, SuSE, or Ubuntu system. - - -
- -
- Getting Nix - - - If your server runs NixOS you are all set to continue with - installation of Hydra. Otherwise you first need to install - Nix. The latest stable version can be found one the Nix web - site, along with a manual, which includes installation - instructions. - -
- -
- Installation - - - Hydra can be installed using Nixpkgs: - - - nix-env -Ai hydra -f /path/to/nixpkgs - - This makes the tools available in your Nix user environment, - $HOME/.nix-profile by default. - - - - Alternatively, the latest development snapshot can be - installed by visiting the URL - http://hydra.nixos.org/view/hydra/unstable - and use the one-click install available at one of the build pages. You can also - install Hydra through the channel by performing the following commands: + + Alternatively, the latest development snapshot can be installed + by visiting the URL http://hydra.nixos.org/view/hydra/unstable + and use the one-click install available at one of the build + pages. You can also install Hydra through the channel by + performing the following commands: nix-channel --add http://hydra.nixos.org/jobset/hydra/trunk/channel/latest nix-channel --update nix-env -i hydra - + - - Command completion should reveal a number of command-line tools from Hydra: + + Command completion should reveal a number of command-line tools from Hydra: - + hydra-build hydra-evaluator hydra-server hydra-eval-jobs hydra-queue-runner hydra-update-gc-roots - -
+ +
-
- Creating the database - - Hydra stores its results in a database, which can be a - PostgreSQL or SQLite database. The latter is easier to - setup, but the former scales better. - +
+ Creating the database + + Hydra stores its results in a database, which can be a + PostgreSQL or SQLite database. The latter is easier to setup, + but the former scales better. + - To setup a PostgreSQL - database with hydra as database name - and user name, issue the following commands: - + + To setup a PostgreSQL database with hydra + as database name and user name, issue the following commands: + + createdb hydra echo "CREATE USER hydra WITH PASSWORD '<your-password>' ;" | psql hydra cat $prefix/share/hydra/sql/hydra-postgresql.sql | psql hydra echo "GRANT ALL ON DATABASE hydra TO hydra;" | psql hydra - Note that $prefix is the location of - Hydra in the nix store. - - - For SQLite, the following command is all it takes to - create the database: + Note that $prefix is the location of Hydra + in the nix store. + - - cat $prefix/share/hydra/sql/hydra-sqlite.sql | sqlite3 /path/to/hydra.sqlite - - + + For SQLite, the following command is all it takes to create the + database: - - To add a user root with admin privileges, execute: - + +cat $prefix/share/hydra/sql/hydra-sqlite.sql | sqlite3 /path/to/hydra.sqlite + + + + To add a user root with + admin privileges, execute: + echo "INSERT INTO Users(userName, emailAddress, password) VALUES ('root', 'some@email.adress.com', '$(echo -n foobar | sha1sum | cut -c1-40)');" | psql hydra -echo "INSERT INTO UserRoles(userName, role) values('root', 'admin');" | psql hydra - - For SQLite the same commands can be used, with - psql hydra replaced by - sqlite3 /path/to/hydra.sqlite. - +echo "INSERT INTO UserRoles(userName, role) values('root', 'admin');" | psql hydra - - Hydra uses an environment variable to know which database - should be used, and a variable which point to a location - that holds some state. To set these variables for a - PostgreSQL database, add the following to the - .profile of the user running the - Hydra services. + For SQLite the same commands can be used, with psql + hydra replaced by sqlite3 + /path/to/hydra.sqlite. + - + + Hydra uses an environment variable to know which database should + be used, and a variable which point to a location that holds + some state. To set these variables for a PostgreSQL database, + add the following to the .profile of the + user running the Hydra services. + + export HYDRA_DBI="dbi:Pg:dbname=hydra;host=localhost;" export HYDRA_DATA=/var/lib/hydra - Make sure that the HYDRA_DATA - directory exists and is writable for the user which will - run the Hydra services. For a SQLite database, the - HYDRA_DBI should be set to something - like dbi:SQLite:/path/to/hydra.sqlite + Make sure that the HYDRA_DATA directory + exists and is writable for the user which will run the Hydra + services. For a SQLite database, the + HYDRA_DBI should be set to something like + dbi:SQLite:/path/to/hydra.sqlite + +
- -
+
+ Getting Started -
- Getting Started - - - To start the Hydra web server, execute: - + + To start the Hydra web server, execute: + hydra-server - When the server is started, you can browse to - http://localhost:3000/ to start configuring - your Hydra instance. - + When the server is started, you can browse to + http://localhost:3000/ to start configuring + your Hydra instance. + - - The hydra-server command launches the - web server. There are two other processes that come into - play: + + The hydra-server command launches the web + server. There are two other processes that come into play: - - - The evaluator is responsible for - peridically evaluating job sets, checking out their - dependencies off their version control systems (VCS), - and queueing new builds if the result of the evaluation - changed. It is launched by the - hydra-evaluator command. - - - The queue runner launches builds - (using Nix) as they are queued by the evaluator, - scheduling them onto the configured Nix hosts. It is - launched using the - hydra-queue-runner command. - - - - All three processes must be running for Hydra to be fully - functional, though it's possible to temporarily stop any one - of them for maintenance purposes, for instance. - - -
+ + + The evaluator is responsible for + peridically evaluating job sets, checking out their + dependencies off their version control systems (VCS), and + queueing new builds if the result of the evaluation changed. + It is launched by the hydra-evaluator + command. + + + The queue runner launches builds (using + Nix) as they are queued by the evaluator, scheduling them + onto the configured Nix hosts. It is launched using the + hydra-queue-runner command. + + + All three processes must be running for Hydra to be fully + functional, though it's possible to temporarily stop any one of + them for maintenance purposes, for instance. + + +
+ + diff --git a/doc/manual/introduction.xml b/doc/manual/introduction.xml index e912d233..984fce10 100644 --- a/doc/manual/introduction.xml +++ b/doc/manual/introduction.xml @@ -19,67 +19,67 @@ more in-depth testing than what developers could feasibly do manually: - Portability testing: The - software may need to be built and tested on many different - platforms. It is infeasible for each developer to do this - before every commit. - + Portability testing: The + software may need to be built and tested on many different + platforms. It is infeasible for each developer to do this + before every commit. + - Likewise, many projects have very large test sets - (e.g., regression tests in a compiler, or stress tests in a - DBMS) that can take hours or days to run to completion. - + Likewise, many projects have very large test sets + (e.g., regression tests in a compiler, or stress tests in a + DBMS) that can take hours or days to run to completion. + - Many kinds of static and dynamic analyses can be - performed as part of the tests, such as code coverage runs and - static analyses. - + Many kinds of static and dynamic analyses can be + performed as part of the tests, such as code coverage runs and + static analyses. + - It may also be necessary to build many different - variants of the software. For instance, - it may be necessary to verify that the component builds with - various versions of a compiler. - + It may also be necessary to build many different + variants of the software. For instance, + it may be necessary to verify that the component builds with + various versions of a compiler. + - Developers typically use incremental building to - test their changes (since a full build may take too long), but - this is unreliable with many build management tools (such as - Make), i.e., the result of the incremental build might differ - from a full build. - + Developers typically use incremental building to + test their changes (since a full build may take too long), but + this is unreliable with many build management tools (such as + Make), i.e., the result of the incremental build might differ + from a full build. + - It ensures that the software can be built from the - sources under revision control. Users of version management - systems such as CVS and Subversion often forget to place - source files under revision control. - + It ensures that the software can be built from the + sources under revision control. Users of version management + systems such as CVS and Subversion often forget to place + source files under revision control. + - The machines on which the continuous integration - system runs ideally provides a clean, well-defined build - environment. If this environment is administered through - proper SCM techniques, then builds produced by the system can - be reproduced. In contrast, developer work environments are - typically not under any kind of SCM control. - + The machines on which the continuous integration + system runs ideally provides a clean, well-defined build + environment. If this environment is administered through + proper SCM techniques, then builds produced by the system can + be reproduced. In contrast, developer work environments are + typically not under any kind of SCM control. + - In large projects, developers often work on a - particular component of the project, and do not build and test - the composition of those components (again since this is - likely to take too long). To prevent the phenomenon of ``big - bang integration'', where components are only tested together - near the end of the development process, it is important to - test components together as soon as possible (hence - continuous integration). - + In large projects, developers often work on a + particular component of the project, and do not build and test + the composition of those components (again since this is + likely to take too long). To prevent the phenomenon of ``big + bang integration'', where components are only tested together + near the end of the development process, it is important to + test components together as soon as possible (hence + continuous integration). + - It allows software to be - released by automatically creating - packages that users can download and install. To do this - manually represents an often prohibitive amount of work, as - one may want to produce releases for many different platforms: - e.g., installers for Windows and Mac OS X, RPM or Debian - packages for certain Linux distributions, and so on. - + It allows software to be + released by automatically creating + packages that users can download and install. To do this + manually represents an often prohibitive amount of work, as + one may want to produce releases for many different platforms: + e.g., installers for Windows and Mac OS X, RPM or Debian + packages for certain Linux distributions, and so on. + @@ -92,19 +92,19 @@ - It obtains the latest version of the component's - source code from the version management system. - + It obtains the latest version of the component's + source code from the version management system. + - It runs the component's build process (which - presumably includes the execution of the component's test - set). - + It runs the component's build process (which + presumably includes the execution of the component's test + set). + - It presents the results of the build (such as error - logs and releases) to the developers, e.g., by producing a web - page. - + It presents the results of the build (such as error + logs and releases) to the developers, e.g., by producing a web + page. + @@ -114,40 +114,40 @@ - They do not manage the build - environment. The build environment consists of the - dependencies necessary to perform a build action, e.g., - compilers, libraries, etc. Setting up the environment is - typically done manually, and without proper SCM control (so it - may be hard to reproduce a build at a later time). Manual - management of the environment scales poorly in the number of - configurations that must be supported. For instance, suppose - that we want to build a component that requires a certain - compiler X. We then have to go to each machine and install X. - If we later need a newer version of X, the process must be - repeated all over again. An ever worse problem occurs if - there are conflicting, mutually exclusive versions of the - dependencies. Thus, simply installing the latest version is - not an option. Of course, we can install these components in - different directories and manually pass the appropriate paths - to the build processes of the various components. But this is - a rather tiresome and error-prone process. - + They do not manage the build + environment. The build environment consists of the + dependencies necessary to perform a build action, e.g., + compilers, libraries, etc. Setting up the environment is + typically done manually, and without proper SCM control (so it + may be hard to reproduce a build at a later time). Manual + management of the environment scales poorly in the number of + configurations that must be supported. For instance, suppose + that we want to build a component that requires a certain + compiler X. We then have to go to each machine and install X. + If we later need a newer version of X, the process must be + repeated all over again. An ever worse problem occurs if + there are conflicting, mutually exclusive versions of the + dependencies. Thus, simply installing the latest version is + not an option. Of course, we can install these components in + different directories and manually pass the appropriate paths + to the build processes of the various components. But this is + a rather tiresome and error-prone process. + - They do not easily support variability in software - systems. A system may have a great deal of build-time - variability: optional functionality, whether to build a debug or - production version, different versions of dependencies, and so on. - (For instance, the Linux kernel now has over 2,600 build-time - configuration switches.) It is therefore important that a continuous - integration tool can easily select and test different instances from - the configuration space of the system to reveal problems, such as - erroneous interactions between features. In a continuous integration - setting, it is also useful to test different combinations of versions - of subsystems, e.g., the head revision of a component against stable - releases of its dependencies, and vice versa, as this can reveal - various integration problems. - + They do not easily support variability in software + systems. A system may have a great deal of build-time + variability: optional functionality, whether to build a debug or + production version, different versions of dependencies, and so on. + (For instance, the Linux kernel now has over 2,600 build-time + configuration switches.) It is therefore important that a continuous + integration tool can easily select and test different instances from + the configuration space of the system to reveal problems, such as + erroneous interactions between features. In a continuous integration + setting, it is also useful to test different combinations of versions + of subsystems, e.g., the head revision of a component against stable + releases of its dependencies, and vice versa, as this can reveal + various integration problems. + @@ -258,3 +258,10 @@ + +