<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id='chap-distributed-builds'> <title>Setting Up Distributed Builds</title> <para>Nix supports distributed builds: a local Nix installation can forward Nix builds to other machines over the network. This allows multiple builds to be performed in parallel (thus improving performance) and allows Nix to perform multi-platform builds in a semi-transparent way. For instance, if you perform a build for a <literal>powerpc-darwin</literal> on an <literal>i686-linux</literal> machine, Nix can automatically forward the build to a <literal>powerpc-darwin</literal> machine, if available.</para> <para>You can enable distributed builds by setting the environment variable <envar>NIX_BUILD_HOOK</envar> to point to a program that Nix will call whenever it wants to build a derivation. The build hook (typically a shell or Perl script) can decline the build, in which Nix will perform it in the usual way if possible, or it can accept it, in which case it is responsible for somehow getting the inputs of the build to another machine, doing the build there, and getting the results back. The details of the build hook protocol are described in the documentation of the <link linkend="envar-build-hook"><envar>NIX_BUILD_HOOK</envar> variable</link>.</para> <example xml:id='ex-remote-systems'><title>Remote machine configuration: <filename>remote-systems.conf</filename></title> <programlisting> nix@mcflurry.labs.cs.uu.nl powerpc-darwin /home/nix/.ssh/id_quarterpounder_auto 2 nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2 nix@poochie.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2 kvm perf </programlisting> </example> <para>Nix ships with a build hook that should be suitable for most purposes. It uses <command>ssh</command> and <command>nix-copy-closure</command> to copy the build inputs and outputs and perform the remote build. To use it, you should set <envar>NIX_BUILD_HOOK</envar> to <filename><replaceable>prefix</replaceable>/libexec/nix/build-remote.pl</filename>. You should also define a list of available build machines and point the environment variable <envar>NIX_REMOTE_SYSTEMS</envar> to it. An example configuration is shown in <xref linkend='ex-remote-systems' />. Each line in the file specifies a machine, with the following bits of information: <orderedlist> <listitem><para>The name of the remote machine, with optionally the user under which the remote build should be performed. This is actually passed as an argument to <command>ssh</command>, so it can be an alias defined in your <filename>~/.ssh/config</filename>.</para></listitem> <listitem><para>A comma-separated list of Nix platform type identifiers, such as <literal>powerpc-darwin</literal>. It is possible for a machine to support multiple platform types, e.g., <literal>i686-linux,x86_64-linux</literal>.</para></listitem> <listitem><para>The SSH private key to be used to log in to the remote machine. Since builds should be non-interactive, this key should not have a passphrase!</para></listitem> <listitem><para>The maximum number of builds that <filename>build-remote.pl</filename> will execute in parallel on the machine. Typically this should be equal to the number of CPU cores. For instance, the machine <literal>itchy</literal> in the example will execute up to 8 builds in parallel.</para></listitem> <listitem><para>The “speed factor”, indicating the relative speed of the machine. If there are multiple machines of the right type, Nix will prefer the fastest, taking load into account.</para></listitem> <listitem><para>A comma-separated list of <emphasis>supported features</emphasis>. If a derivation has the <varname>requiredSystemFeatures</varname> attribute, then <filename>build-remote.pl</filename> will only perform the derivation on a machine that has the specified features. For instance, the attribute <programlisting> requiredSystemFeatures = [ "kvm" ]; </programlisting> will cause the build to be performed on a machine that has the <literal>kvm</literal> feature (i.e., <literal>scratchy</literal> in the example above).</para></listitem> <listitem><para>A comma-separated list of <emphasis>mandatory features</emphasis>. A machine will only be used to build a derivation if all of the machine’s mandatory features appear in the derivation’s <varname>requiredSystemFeatures</varname> attribute. Thus, in the example, the machine <literal>poochie</literal> will only do derivations that have <varname>requiredSystemFeatures</varname> set to <literal>["kvm" "perf"]</literal> or <literal>["perf"]</literal>.</para></listitem> </orderedlist> You should also set up the environment variable <envar>NIX_CURRENT_LOAD</envar> to point at a directory (e.g., <filename>/var/run/nix/current-load</filename>) that <filename>build-remote.pl</filename> uses to remember how many builds it is currently executing remotely. It doesn't look at the actual load on the remote machine, so if you have multiple instances of Nix running, they should use the same <envar>NIX_CURRENT_LOAD</envar> file. Maybe in the future <filename>build-remote.pl</filename> will look at the actual remote load.</para> </chapter>