forked from lix-project/lix
1539 lines
54 KiB
XML
1539 lines
54 KiB
XML
<chapter xmlns="http://docbook.org/ns/docbook"
|
||
xmlns:xlink="http://www.w3.org/1999/xlink"
|
||
xml:id='chap-writing-nix-expressions'>
|
||
|
||
<title>Writing Nix Expressions</title>
|
||
|
||
|
||
<para>This chapter shows you how to write Nix expressions, which are
|
||
the things that tell Nix how to build components. It starts with a
|
||
simple example (a Nix expression for GNU Hello), and then moves
|
||
on to a more in-depth look at the Nix expression language.</para>
|
||
|
||
|
||
<section><title>A simple Nix expression</title>
|
||
|
||
<para>This section shows how to add and test the <link
|
||
xlink:href='http://www.gnu.org/software/hello/hello.html'>GNU Hello
|
||
package</link> to the Nix Packages collection. Hello is a program
|
||
that prints out the text <quote>Hello, world!</quote>.</para>
|
||
|
||
<para>To add a component to the Nix Packages collection, you generally
|
||
need to do three things:
|
||
|
||
<orderedlist>
|
||
|
||
<listitem><para>Write a Nix expression for the component. This is a
|
||
file that describes all the inputs involved in building the
|
||
component, such as dependencies (other components required by the
|
||
component), sources, and so on.</para></listitem>
|
||
|
||
<listitem><para>Write a <emphasis>builder</emphasis>. This is a
|
||
shell script<footnote><para>In fact, it can be written in any
|
||
language, but typically it's a <command>bash</command> shell
|
||
script.</para></footnote> that actually builds the component from
|
||
the inputs.</para></listitem>
|
||
|
||
<listitem><para>Add the component to the file
|
||
<filename>pkgs/top-level/all-packages.nix</filename>. The Nix
|
||
expression written in the first step is a
|
||
<emphasis>function</emphasis>; it requires other components in order
|
||
to build it. In this step you put it all together, i.e., you call
|
||
the function with the right arguments to build the actual
|
||
component.</para></listitem>
|
||
|
||
</orderedlist>
|
||
|
||
</para>
|
||
|
||
|
||
<section><title>The Nix expression</title>
|
||
|
||
<example xml:id='ex-hello-nix'><title>Nix expression for GNU Hello
|
||
(<filename>default.nix</filename>)</title>
|
||
<programlisting>
|
||
{stdenv, fetchurl, perl}: <co xml:id='ex-hello-nix-co-1' />
|
||
|
||
stdenv.mkDerivation { <co xml:id='ex-hello-nix-co-2' />
|
||
name = "hello-2.1.1"; <co xml:id='ex-hello-nix-co-3' />
|
||
builder = ./builder.sh; <co xml:id='ex-hello-nix-co-4' />
|
||
src = fetchurl { <co xml:id='ex-hello-nix-co-5' />
|
||
url = ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz;
|
||
md5 = "70c9ccf9fac07f762c24f2df2290784d";
|
||
};
|
||
inherit perl; <co xml:id='ex-hello-nix-co-6' />
|
||
}</programlisting>
|
||
</example>
|
||
|
||
<para><xref linkend='ex-hello-nix' /> shows a Nix expression for GNU
|
||
Hello. It's actually already in the Nix Packages collection in
|
||
<filename>pkgs/applications/misc/hello/ex-1/default.nix</filename>.
|
||
It is customary to place each package in a separate directory and call
|
||
the single Nix expression in that directory
|
||
<filename>default.nix</filename>. The file has the following elements
|
||
(referenced from the figure by number):
|
||
|
||
<calloutlist>
|
||
|
||
<callout arearefs='ex-hello-nix-co-1'>
|
||
|
||
<para>This states that the expression is a
|
||
<emphasis>function</emphasis> that expects to be called with three
|
||
arguments: <varname>stdenv</varname>, <varname>fetchurl</varname>,
|
||
and <varname>perl</varname>. They are needed to build Hello, but
|
||
we don't know how to build them here; that's why they are function
|
||
arguments. <varname>stdenv</varname> is a component that is used
|
||
by almost all Nix Packages components; it provides a
|
||
<quote>standard</quote> environment consisting of the things you
|
||
would expect in a basic Unix environment: a C/C++ compiler (GCC,
|
||
to be precise), the Bash shell, fundamental Unix tools such as
|
||
<command>cp</command>, <command>grep</command>,
|
||
<command>tar</command>, etc. <varname>fetchurl</varname> is a
|
||
function that downloads files. <varname>perl</varname> is the
|
||
Perl interpreter.</para>
|
||
|
||
<para>Nix functions generally have the form <literal>{x, y, ...,
|
||
z}: e</literal> where <varname>x</varname>, <varname>y</varname>,
|
||
etc. are the names of the expected arguments, and where
|
||
<replaceable>e</replaceable> is the body of the function. So
|
||
here, the entire remainder of the file is the body of the
|
||
function; when given the required arguments, the body should
|
||
describe how to build an instance of the Hello component.</para>
|
||
|
||
</callout>
|
||
|
||
<callout arearefs='ex-hello-nix-co-2'>
|
||
|
||
<para>So we have to build a component. Building something from
|
||
other stuff is called a <emphasis>derivation</emphasis> in Nix (as
|
||
opposed to sources, which are built by humans instead of
|
||
computers). We perform a derivation by calling
|
||
<varname>stdenv.mkDerivation</varname>.
|
||
<varname>mkDerivation</varname> is a function provided by
|
||
<varname>stdenv</varname> that builds a component from a set of
|
||
<emphasis>attributes</emphasis>. An attribute set is just a list
|
||
of key/value pairs where each value is an arbitrary Nix
|
||
expression. They take the general form
|
||
<literal>{<replaceable>name1</replaceable> =
|
||
<replaceable>expr1</replaceable>; <replaceable>...</replaceable>
|
||
<replaceable>nameN</replaceable> =
|
||
<replaceable>exprN</replaceable>;}</literal>.</para>
|
||
|
||
</callout>
|
||
|
||
<callout arearefs='ex-hello-nix-co-3'>
|
||
|
||
<para>The attribute <varname>name</varname> specifies the symbolic
|
||
name and version of the component. Nix doesn't really care about
|
||
these things, but they are used by for instance <command>nix-env
|
||
-q</command> to show a <quote>human-readable</quote> name for
|
||
components. This attribute is required by
|
||
<varname>mkDerivation</varname>.</para>
|
||
|
||
</callout>
|
||
|
||
<callout arearefs='ex-hello-nix-co-4'>
|
||
|
||
<para>The attribute <varname>builder</varname> specifies the
|
||
builder. This attribute can sometimes be omitted, in which case
|
||
<varname>mkDerivation</varname> will fill in a default builder
|
||
(which does a <literal>configure; make; make install</literal>, in
|
||
essence). Hello is sufficiently simple that the default builder
|
||
would suffice, but in this case, we will show an actual builder
|
||
for educational purposes. The value
|
||
<command>./builder.sh</command> refers to the shell script shown
|
||
in <xref linkend='ex-hello-builder' />, discussed below.</para>
|
||
|
||
</callout>
|
||
|
||
<callout arearefs='ex-hello-nix-co-5'>
|
||
|
||
<para>The builder has to know what the sources of the component
|
||
are. Here, the attribute <varname>src</varname> is bound to the
|
||
result of a call to the <command>fetchurl</command> function.
|
||
Given a URL and an MD5 hash of the expected contents of the file
|
||
at that URL, this function builds a derivation that downloads the
|
||
file and checks its hash. So the sources are a dependency that
|
||
like all other dependencies is built before Hello itself is
|
||
built.</para>
|
||
|
||
<para>Instead of <varname>src</varname> any other name could have
|
||
been used, and in fact there can be any number of sources (bound
|
||
to different attributes). However, <varname>src</varname> is
|
||
customary, and it's also expected by the default builder (which we
|
||
don't use in this example).</para>
|
||
|
||
</callout>
|
||
|
||
<callout arearefs='ex-hello-nix-co-6'>
|
||
|
||
<para>Since the derivation requires Perl, we have to pass the
|
||
value of the <varname>perl</varname> function argument to the
|
||
builder. All attributes in the set are actually passed as
|
||
environment variables to the builder, so declaring an attribute
|
||
|
||
<programlisting>
|
||
perl = perl;</programlisting>
|
||
|
||
will do the trick: it binds an attribute <varname>perl</varname>
|
||
to the function argument which also happens to be called
|
||
<varname>perl</varname>. However, it looks a bit silly, so there
|
||
is a shorter syntax. The <literal>inherit</literal> keyword
|
||
causes the specified attributes to be bound to whatever variables
|
||
with the same name happen to be in scope.</para>
|
||
|
||
</callout>
|
||
|
||
</calloutlist>
|
||
|
||
</para>
|
||
|
||
</section>
|
||
|
||
|
||
<section><title>The builder</title>
|
||
|
||
<example xml:id='ex-hello-builder'><title>Build script for GNU Hello
|
||
(<filename>builder.sh</filename>)</title>
|
||
<programlisting>
|
||
source $stdenv/setup <co xml:id='ex-hello-builder-co-1' />
|
||
|
||
PATH=$perl/bin:$PATH <co xml:id='ex-hello-builder-co-2' />
|
||
|
||
tar xvfz $src <co xml:id='ex-hello-builder-co-3' />
|
||
cd hello-*
|
||
./configure --prefix=$out <co xml:id='ex-hello-builder-co-4' />
|
||
make <co xml:id='ex-hello-builder-co-5' />
|
||
make install</programlisting>
|
||
</example>
|
||
|
||
<para><xref linkend='ex-hello-builder' /> shows the builder referenced
|
||
from Hello's Nix expression (stored in
|
||
<filename>pkgs/applications/misc/hello/ex-1/builder.sh</filename>).
|
||
The builder can actually be made a lot shorter by using the
|
||
<emphasis>generic builder</emphasis> functions provided by
|
||
<varname>stdenv</varname>, but here we write out the build steps to
|
||
elucidate what a builder does. It performs the following
|
||
steps:</para>
|
||
|
||
<calloutlist>
|
||
|
||
<callout arearefs='ex-hello-builder-co-1'>
|
||
|
||
<para>When Nix runs a builder, it initially completely clears the
|
||
environment (except for the attributes declared in the
|
||
derivation). For instance, the <envar>PATH</envar> variable is
|
||
empty<footnote><para>Actually, it's initialised to
|
||
<filename>/path-not-set</filename> to prevent Bash from setting it
|
||
to a default value.</para></footnote>. This is done to prevent
|
||
undeclared inputs from being used in the build process. If for
|
||
example the <envar>PATH</envar> contained
|
||
<filename>/usr/bin</filename>, then you might accidentally use
|
||
<filename>/usr/bin/gcc</filename>.</para>
|
||
|
||
<para>So the first step is to set up the environment. This is
|
||
done by calling the <filename>setup</filename> script of the
|
||
standard environment. The environment variable
|
||
<envar>stdenv</envar> points to the location of the standard
|
||
environment being used. (It wasn't specified explicitly as an
|
||
attribute in <xref linkend='ex-hello-nix' />, but
|
||
<varname>mkDerivation</varname> adds it automatically.)</para>
|
||
|
||
</callout>
|
||
|
||
<callout arearefs='ex-hello-builder-co-2'>
|
||
|
||
<para>Since Hello needs Perl, we have to make sure that Perl is in
|
||
the <envar>PATH</envar>. The <envar>perl</envar> environment
|
||
variable points to the location of the Perl component (since it
|
||
was passed in as an attribute to the derivation), so
|
||
<filename><replaceable>$perl</replaceable>/bin</filename> is the
|
||
directory containing the Perl interpreter.</para>
|
||
|
||
</callout>
|
||
|
||
<callout arearefs='ex-hello-builder-co-3'>
|
||
|
||
<para>Now we have to unpack the sources. The
|
||
<varname>src</varname> attribute was bound to the result of
|
||
fetching the Hello source tarball from the network, so the
|
||
<envar>src</envar> environment variable points to the location in
|
||
the Nix store to which the tarball was downloaded. After
|
||
unpacking, we <command>cd</command> to the resulting source
|
||
directory.</para>
|
||
|
||
<para>The whole build is performed in a temporary directory
|
||
created in <varname>/tmp</varname>, by the way. This directory is
|
||
removed after the builder finishes, so there is no need to clean
|
||
up the sources afterwards. Also, the temporary directory is
|
||
always newly created, so you don't have to worry about files from
|
||
previous builds interfering with the current build.</para>
|
||
|
||
</callout>
|
||
|
||
<callout arearefs='ex-hello-builder-co-4'>
|
||
|
||
<para>GNU Hello is a typical Autoconf-based package, so we first
|
||
have to run its <filename>configure</filename> script. In Nix
|
||
every component is stored in a separate location in the Nix store,
|
||
for instance
|
||
<filename>/nix/store/9a54ba97fb71b65fda531012d0443ce2-hello-2.1.1</filename>.
|
||
Nix computes this path by cryptographically hashing all attributes
|
||
of the derivation. The path is passed to the builder through the
|
||
<envar>out</envar> environment variable. So here we give
|
||
<filename>configure</filename> the parameter
|
||
<literal>--prefix=$out</literal> to cause Hello to be installed in
|
||
the expected location.</para>
|
||
|
||
</callout>
|
||
|
||
<callout arearefs='ex-hello-builder-co-5'>
|
||
|
||
<para>Finally we build Hello (<literal>make</literal>) and install
|
||
it into the location specified by <envar>out</envar>
|
||
(<literal>make install</literal>).</para>
|
||
|
||
</callout>
|
||
|
||
</calloutlist>
|
||
|
||
<para>If you are wondering about the absence of error checking on the
|
||
result of various commands called in the builder: this is because the
|
||
shell script is evaluated with Bash's <option>-e</option> option,
|
||
which causes the script to be aborted if any command fails without an
|
||
error check.</para>
|
||
|
||
</section>
|
||
|
||
|
||
<section><title>Composition</title>
|
||
|
||
<example xml:id='ex-hello-composition'><title>Composing GNU Hello
|
||
(<filename>all-packages.nix</filename>)</title>
|
||
<programlisting>
|
||
...
|
||
|
||
rec { <co xml:id='ex-hello-composition-co-1' />
|
||
|
||
hello = (import ../applications/misc/hello/ex-1 <co xml:id='ex-hello-composition-co-2' />) { <co xml:id='ex-hello-composition-co-3' />
|
||
inherit fetchurl stdenv perl;
|
||
};
|
||
|
||
perl = (import ../development/interpreters/perl) { <co xml:id='ex-hello-composition-co-4' />
|
||
inherit fetchurl stdenv;
|
||
};
|
||
|
||
fetchurl = (import ../build-support/fetchurl) {
|
||
inherit stdenv; ...
|
||
};
|
||
|
||
stdenv = ...;
|
||
|
||
}
|
||
</programlisting>
|
||
</example>
|
||
|
||
<para>The Nix expression in <xref linkend='ex-hello-nix' /> is a
|
||
function; it is missing some arguments that have to be filled in
|
||
somewhere. In the Nix Packages collection this is done in the file
|
||
<filename>pkgs/top-level/all-packages.nix</filename>, where all
|
||
Nix expressions for components are imported and called with the
|
||
appropriate arguments. <xref linkend='ex-hello-composition' /> shows
|
||
some fragments of
|
||
<filename>all-packages.nix</filename>.</para>
|
||
|
||
<calloutlist>
|
||
|
||
<callout arearefs='ex-hello-composition-co-1'>
|
||
|
||
<para>This file defines a set of attributes, all of which are
|
||
concrete derivations (i.e., not functions). In fact, we define a
|
||
<emphasis>mutually recursive</emphasis> set of attributes. That
|
||
is, the attributes can refer to each other. This is precisely
|
||
what we want since we want to <quote>plug</quote> the
|
||
various components into each other.</para>
|
||
|
||
</callout>
|
||
|
||
<callout arearefs='ex-hello-composition-co-2'>
|
||
|
||
<para>Here we <emphasis>import</emphasis> the Nix expression for
|
||
GNU Hello. The import operation just loads and returns the
|
||
specified Nix expression. In fact, we could just have put the
|
||
contents of <xref linkend='ex-hello-nix' /> in
|
||
<filename>all-packages.nix</filename> at this point. That
|
||
would be completely equivalent, but it would make the file rather
|
||
bulky.</para>
|
||
|
||
<para>Note that we refer to
|
||
<filename>../applications/misc/hello/ex-1</filename>, not
|
||
<filename>../applications/misc/hello/ex-1/default.nix</filename>.
|
||
When you try to import a directory, Nix automatically appends
|
||
<filename>/default.nix</filename> to the file name.</para>
|
||
|
||
</callout>
|
||
|
||
<callout arearefs='ex-hello-composition-co-3'>
|
||
|
||
<para>This is where the actual composition takes place. Here we
|
||
<emphasis>call</emphasis> the function imported from
|
||
<filename>../applications/misc/hello/ex-1</filename> with an
|
||
attribute set containing the things that the function expects,
|
||
namely <varname>fetchurl</varname>, <varname>stdenv</varname>, and
|
||
<varname>perl</varname>. We use inherit again to use the
|
||
attributes defined in the surrounding scope (we could also have
|
||
written <literal>fetchurl = fetchurl;</literal>, etc.).</para>
|
||
|
||
<para>The result of this function call is an actual derivation
|
||
that can be built by Nix (since when we fill in the arguments of
|
||
the function, what we get is its body, which is the call to
|
||
<varname>stdenv.mkDerivation</varname> in <xref
|
||
linkend='ex-hello-nix' />).</para>
|
||
|
||
</callout>
|
||
|
||
<callout arearefs='ex-hello-composition-co-4'>
|
||
|
||
<para>Likewise, we have to instantiate Perl,
|
||
<varname>fetchurl</varname>, and the standard environment.</para>
|
||
|
||
</callout>
|
||
|
||
</calloutlist>
|
||
|
||
</section>
|
||
|
||
|
||
<section><title>Testing</title>
|
||
|
||
<para>You can now try to build Hello. Of course, you could do
|
||
<literal>nix-env -f pkgs/top-level/all-packages.nix -i hello</literal>,
|
||
but you may not want to install a possibly broken package just yet.
|
||
The best way to test the package is by using the command <command
|
||
linkend="sec-nix-build">nix-build</command>, which builds a Nix
|
||
expression and creates a symlink named <filename>result</filename> in
|
||
the current directory:
|
||
|
||
<screen>
|
||
$ nix-build pkgs/top-level/all-packages.nix -A hello
|
||
building path `/nix/store/632d2b22514d...-hello-2.1.1'
|
||
hello-2.1.1/
|
||
hello-2.1.1/intl/
|
||
hello-2.1.1/intl/ChangeLog
|
||
<replaceable>...</replaceable>
|
||
|
||
$ ls -l result
|
||
lrwxrwxrwx ... 2006-09-29 10:43 result -> /nix/store/632d2b22514d...-hello-2.1.1
|
||
|
||
$ ./result/bin/hello
|
||
Hello, world!</screen>
|
||
|
||
</para>
|
||
|
||
<para><command>nix-build</command> registers the
|
||
<filename>./result</filename> symlink as a garbage collection root, so
|
||
unless and until you delete the <filename>./result</filename> symlink,
|
||
the output of the build will be safely kept on your system. You can
|
||
use <command>nix-build</command>’s <option
|
||
linkend='opt-out-link'>-o</option> switch to give the symlink another
|
||
name.</para>
|
||
|
||
<para>Nix has a transactional semantics. Once a build finishes
|
||
successfully, Nix makes a note of this in its database: it registers
|
||
that the path denoted by <envar>out</envar> is now
|
||
<quote>valid</quote>. If you try to build the derivation again, Nix
|
||
will see that the path is already valid and finish immediately. If a
|
||
build fails, either because it returns a non-zero exit code, because
|
||
Nix or the builder are killed, or because the machine crashes, then
|
||
the output path will not be registered as valid. If you try to build
|
||
the derivation again, Nix will remove the output path if it exists
|
||
(e.g., because the builder died half-way through <literal>make
|
||
install</literal>) and try again. Note that there is no
|
||
<quote>negative caching</quote>: Nix doesn't remember that a build
|
||
failed, and so a failed build can always be repeated. This is because
|
||
Nix cannot distinguish between permanent failures (e.g., a compiler
|
||
error due to a syntax error in the source) and transient failures
|
||
(e.g., a disk full condition).</para>
|
||
|
||
<para>Nix also performs locking. If you run multiple Nix builds
|
||
simultaneously, and they try to build the same derivation, the first
|
||
Nix instance that gets there will perform the build, while the others
|
||
block (or perform other derivations if available) until the build
|
||
finishes:
|
||
|
||
<screen>
|
||
$ nix-build pkgs/top-level/all-packages.nix -A hello
|
||
waiting for lock on `/nix/store/0h5b7hp8d4hqfrw8igvx97x1xawrjnac-hello-2.1.1x'</screen>
|
||
|
||
So it is always safe to run multiple instances of Nix in parallel
|
||
(which isn’t the case with, say, <command>make</command>).</para>
|
||
|
||
<para>If you have a system with multiple CPUs, you may want to have
|
||
Nix build different derivations in parallel (insofar as possible).
|
||
Just pass the option <option linkend='opt-max-jobs'>-j
|
||
<replaceable>N</replaceable></option>, where
|
||
<replaceable>N</replaceable> is the maximum number of jobs to be run
|
||
in parallel, or set. Typically this should be the number of
|
||
CPUs.</para>
|
||
|
||
</section>
|
||
|
||
|
||
<section><title>The generic builder</title>
|
||
|
||
<para>Recall from <xref linkend='ex-hello-builder' /> that the builder
|
||
looked something like this:
|
||
|
||
<programlisting>
|
||
PATH=$perl/bin:$PATH
|
||
tar xvfz $src
|
||
cd hello-*
|
||
./configure --prefix=$out
|
||
make
|
||
make install</programlisting>
|
||
|
||
The builders for almost all Unix packages look like this — set up some
|
||
environment variables, unpack the sources, configure, build, and
|
||
install. For this reason the standard environment provides some Bash
|
||
functions that automate the build process. A builder using the
|
||
generic build facilities in shown in <xref linkend='ex-hello-builder2'
|
||
/>.</para>
|
||
|
||
<example xml:id='ex-hello-builder2'><title>Build script using the generic
|
||
build functions</title>
|
||
<programlisting>
|
||
buildInputs="$perl" <co xml:id='ex-hello-builder2-co-1' />
|
||
|
||
source $stdenv/setup <co xml:id='ex-hello-builder2-co-2' />
|
||
|
||
genericBuild <co xml:id='ex-hello-builder2-co-3' /></programlisting>
|
||
</example>
|
||
|
||
<calloutlist>
|
||
|
||
<callout arearefs='ex-hello-builder2-co-1'>
|
||
|
||
<para>The <envar>buildInputs</envar> variable tells
|
||
<filename>setup</filename> to use the indicated components as
|
||
<quote>inputs</quote>. This means that if a component provides a
|
||
<filename>bin</filename> subdirectory, it's added to
|
||
<envar>PATH</envar>; if it has a <filename>include</filename>
|
||
subdirectory, it's added to GCC's header search path; and so
|
||
on.</para>
|
||
|
||
</callout>
|
||
|
||
<callout arearefs='ex-hello-builder2-co-2'>
|
||
|
||
<para>The function <function>genericBuild</function> is defined in
|
||
the file <literal>$stdenv/setup</literal>.</para>
|
||
|
||
</callout>
|
||
|
||
<callout arearefs='ex-hello-builder2-co-3'>
|
||
|
||
<para>The final step calls the shell function
|
||
<function>genericBuild</function>, which performs the steps that
|
||
were done explicitly in <xref linkend='ex-hello-builder' />. The
|
||
generic builder is smart enough to figure out whether to unpack
|
||
the sources using <command>gzip</command>,
|
||
<command>bzip2</command>, etc. It can be customised in many ways;
|
||
see <xref linkend='sec-standard-environment' />.</para>
|
||
|
||
</callout>
|
||
|
||
</calloutlist>
|
||
|
||
<para>Discerning readers will note that the
|
||
<envar>buildInputs</envar> could just as well have been set in the Nix
|
||
expression, like this:
|
||
|
||
<programlisting>
|
||
buildInputs = [perl];</programlisting>
|
||
|
||
The <varname>perl</varname> attribute can then be removed, and the
|
||
builder becomes even shorter:
|
||
|
||
<programlisting>
|
||
source $stdenv/setup
|
||
genericBuild</programlisting>
|
||
|
||
In fact, <varname>mkDerivation</varname> provides a default builder
|
||
that looks exactly like that, so it is actually possible to omit the
|
||
builder for Hello entirely.</para>
|
||
|
||
</section>
|
||
|
||
|
||
</section>
|
||
|
||
|
||
|
||
<section><title>The Nix expression language</title>
|
||
|
||
<para>The Nix expression language is a pure, lazy, functional
|
||
language. Purity means that operations in the language don't have
|
||
side-effects (for instance, there is no variable assignment).
|
||
Laziness means that arguments to functions are evaluated only when
|
||
they are needed. Functional means that functions are
|
||
<quote>normal</quote> values that can be passed around and manipulated
|
||
in interesting ways. The language is not a full-featured, general
|
||
purpose language. It's main job is to describe components,
|
||
compositions of components, and the variability within
|
||
components.</para>
|
||
|
||
<para>This section presents the various features of the
|
||
language.</para>
|
||
|
||
|
||
<simplesect><title>Simple values</title>
|
||
|
||
<para>Nix has the following basic datatypes:
|
||
|
||
<itemizedlist>
|
||
|
||
<listitem><para><emphasis>Strings</emphasis>, enclosed between
|
||
double quotes, e.g., <literal>"foo bar"</literal>.</para></listitem>
|
||
|
||
<listitem><para><emphasis>Integers</emphasis>, e.g.,
|
||
<literal>123</literal>.</para></listitem>
|
||
|
||
<listitem><para><emphasis>URIs</emphasis> as defined in appendix B
|
||
of <link xlink:href='http://www.ietf.org/rfc/rfc2396.txt'>RFC
|
||
2396</link>, e.g.,
|
||
<literal>https://svn.cs.uu.nl:12443/dist/trace/trace-nix-trunk.tar.bz2</literal>.</para></listitem>
|
||
|
||
<listitem><para><emphasis>Paths</emphasis>, e.g.,
|
||
<filename>/bin/sh</filename> or <filename>./builder.sh</filename>.
|
||
A path must contain at least one slash to be recognised as such; for
|
||
instance, <filename>builder.sh</filename> is not a
|
||
path<footnote><para>It's parsed as an expression that selects the
|
||
attribute <varname>sh</varname> from the variable
|
||
<varname>builder</varname>.</para></footnote>. If the filename is
|
||
relative, i.e., if it does not begin with a slash, it is made
|
||
absolute at parse time relative to the directory of the Nix
|
||
expression that contained it. For instance, if a Nix expression in
|
||
<filename>/foo/bar/bla.nix</filename> refers to
|
||
<filename>../xyzzy/fnord.nix</filename>, the absolutised path is
|
||
<filename>/foo/xyzzy/fnord.nix</filename>.</para></listitem>
|
||
|
||
<listitem><para><emphasis>Booleans</emphasis> with values
|
||
<literal>true</literal> and
|
||
<literal>false</literal>.</para></listitem>
|
||
|
||
</itemizedlist>
|
||
|
||
</para>
|
||
|
||
</simplesect>
|
||
|
||
|
||
<simplesect><title>Lists</title>
|
||
|
||
<para>Lists are formed by enclosing a whitespace-separated list of
|
||
values between square bracktes. For example,
|
||
|
||
<programlisting>
|
||
[ 123 ./foo.nix "abc" (f {x=y;}) ]</programlisting>
|
||
|
||
defines a list of four elements, the last being the result of a call
|
||
to the function <varname>f</varname>. Note that function calls have
|
||
to be enclosed in parentheses. If they had been omitted, e.g.,
|
||
|
||
<programlisting>
|
||
[ 123 ./foo.nix "abc" f {x=y;} ]</programlisting>
|
||
|
||
the result would be a list of five elements, the fourth one being a
|
||
function and the fifth being an attribute set.</para>
|
||
|
||
</simplesect>
|
||
|
||
|
||
<simplesect><title>Attribute sets</title>
|
||
|
||
<para>Attribute sets are really the core of the language, since
|
||
ultimately it's all about creating derivations, which are really just
|
||
sets of attributes to be passed to build scripts.</para>
|
||
|
||
<para>Attribute sets are just a list of name/value pairs enclosed in
|
||
curly brackets, where each value is an arbitrary expression terminated
|
||
by a semicolon. For example:
|
||
|
||
<programlisting>
|
||
{ x = 123;
|
||
text = "Hello";
|
||
y = f { bla = 456; };
|
||
}</programlisting>
|
||
|
||
This defines an attribute set with attributes named
|
||
<varname>x</varname>, <varname>test</varname>, <varname>y</varname>.
|
||
The order of the attributes is irrelevant. An attribute name may only
|
||
occur once.</para>
|
||
|
||
<para>Attributes can be selected from an attribute set using the
|
||
<literal>.</literal> operator. For instance,
|
||
|
||
<programlisting>
|
||
{ a = "Foo"; b = "Bar"; }.a</programlisting>
|
||
|
||
evaluates to <literal>"Foo"</literal>.</para>
|
||
|
||
</simplesect>
|
||
|
||
|
||
<simplesect><title>Recursive attribute sets</title>
|
||
|
||
<para>Recursive attribute sets are just normal attribute sets, but the
|
||
attributes can refer to each other. For example,
|
||
|
||
<programlisting>
|
||
rec {
|
||
x = y;
|
||
y = 123;
|
||
}.x
|
||
</programlisting>
|
||
|
||
evaluates to <literal>123</literal>. Note that without
|
||
<literal>rec</literal> the binding <literal>x = y;</literal> would
|
||
refer to the variable <varname>y</varname> in the surrounding scope,
|
||
if one exists, and would be invalid if no such variable exists. That
|
||
is, in a normal (non-recursive) attribute set, attributes are not
|
||
added to the lexical scope; in a recursive set, they are.</para>
|
||
|
||
<para>Recursive attribute sets of course introduce the danger of
|
||
infinite recursion. For example,
|
||
|
||
<programlisting>
|
||
rec {
|
||
x = y;
|
||
y = x;
|
||
}.x</programlisting>
|
||
|
||
does not terminate<footnote><para>Actually, Nix detects infinite
|
||
recursion in this case and aborts (<quote>infinite recursion
|
||
encountered</quote>).</para></footnote>.</para>
|
||
|
||
</simplesect>
|
||
|
||
|
||
<simplesect><title>Let expressions</title>
|
||
|
||
<para>A <literal>let</literal> expression is a simple short-hand for a
|
||
<literal>rec</literal> expression followed by an attribute selection:
|
||
<literal>let { <replaceable>attrs</replaceable> }</literal> translates
|
||
to <literal>rec { <replaceable>attrs</replaceable>
|
||
}.body</literal>.</para>
|
||
|
||
<para>For instance,
|
||
|
||
<programlisting>
|
||
let {
|
||
x = "foo";
|
||
y = "bar";
|
||
body = x + y;
|
||
}</programlisting>
|
||
|
||
is equivalent to
|
||
|
||
<programlisting>
|
||
rec {
|
||
x = "foo";
|
||
y = "bar";
|
||
body = x + y;
|
||
}.body</programlisting>
|
||
|
||
and evaluates to <literal>"foobar"</literal>.
|
||
|
||
</para>
|
||
|
||
</simplesect>
|
||
|
||
|
||
<simplesect><title>Inheriting attributes</title>
|
||
|
||
<para>When defining an attribute set it is often convenient to copy
|
||
variables from the surrounding lexical scope (e.g., when you want to
|
||
propagate attributes). This can be shortened using the
|
||
<literal>inherit</literal> keyword. For instance,
|
||
|
||
<programlisting>
|
||
let {
|
||
x = 123;
|
||
body = {
|
||
inherit x;
|
||
y = 456;
|
||
};
|
||
}</programlisting>
|
||
|
||
evaluates to <literal>{x = 123; y = 456;}</literal>. (Note that this
|
||
works because <varname>x</varname> is added to the lexical scope by
|
||
the <literal>let</literal> construct.) It is also possible to inherit
|
||
attributes from another attribute set. For instance, in this fragment
|
||
from <filename>all-packages.nix</filename>,
|
||
|
||
<programlisting>
|
||
graphviz = (import ../tools/graphics/graphviz) {
|
||
inherit fetchurl stdenv libpng libjpeg expat x11 yacc;
|
||
inherit (xlibs) libXaw;
|
||
};
|
||
|
||
xlibs = {
|
||
libX11 = ...;
|
||
libXaw = ...;
|
||
...
|
||
}
|
||
|
||
libpng = ...;
|
||
libjpg = ...;
|
||
...</programlisting>
|
||
|
||
the attribute set used in the function call to the function defined in
|
||
<filename>../tools/graphics/graphviz</filename> inherits a number of
|
||
variables from the surrounding scope (<varname>fetchurl</varname>
|
||
... <varname>yacc</varname>), but also inherits
|
||
<varname>libXaw</varname> (the X Athena Widgets) from the
|
||
<varname>xlibs</varname> (X11 client-side libraries) attribute
|
||
set.</para>
|
||
|
||
</simplesect>
|
||
|
||
|
||
<simplesect xml:id="ss-functions"><title>Functions</title>
|
||
|
||
<para>Functions have the following form:
|
||
|
||
<programlisting>
|
||
{<replaceable>params</replaceable>}: <replaceable>body</replaceable></programlisting>
|
||
|
||
This defines a function that must be called with an attribute set
|
||
containing the attributes listed in <replaceable>params</replaceable>,
|
||
which is a comma-separated list of attribute names. Optionally, for
|
||
each parameter a <emphasis>default value</emphasis> may be specified
|
||
by writing <literal><replaceable>param</replaceable> ?
|
||
<replaceable>e</replaceable></literal>, where
|
||
<replaceable>e</replaceable> is an arbitrary expression. If a
|
||
parameter has a default, the corresponding attribute may be omitted in
|
||
function calls.</para>
|
||
|
||
<para>Note that functions do not have names. If you want to give them
|
||
a name, you can bind them to an attribute, e.g.,
|
||
|
||
<programlisting>
|
||
let {
|
||
concat = {x, y}: x + y;
|
||
body = concat {x = "foo"; y = "bar";};
|
||
}</programlisting>
|
||
|
||
</para>
|
||
|
||
<para>It is also possible to define a function that takes a single
|
||
argument and that does not need to be called with an attribute set as
|
||
argument. The syntax is
|
||
|
||
<programlisting>
|
||
<replaceable>var</replaceable>: <replaceable>body</replaceable></programlisting>
|
||
|
||
where <replaceable>var</replaceable> is the name of the argument. It
|
||
is not possible to define a default. Example:
|
||
|
||
<programlisting>
|
||
let {
|
||
negate = x: !x;
|
||
concat = x: y: x + y;
|
||
body = if negate true then concat "foo" "bar" else "";
|
||
}</programlisting>
|
||
|
||
Note that <function>concat</function> is a function that takes one
|
||
arguments and returns a function that takes another argument. This
|
||
allows partial parameterisation (i.e., only filling some of the
|
||
arguments of a function); e.g.,
|
||
|
||
<programlisting>
|
||
map (concat "foo") ["bar", "bla", "abc"]</programlisting>
|
||
|
||
evaluates to <literal>["foobar" "foobla" "fooabc"]</literal>.</para>
|
||
|
||
</simplesect>
|
||
|
||
|
||
<simplesect><title>Conditionals</title>
|
||
|
||
<para>Conditionals look like this:
|
||
|
||
<programlisting>
|
||
if <replaceable>e1</replaceable> then <replaceable>e2</replaceable> else <replaceable>e3</replaceable></programlisting>
|
||
|
||
where <replaceable>e1</replaceable> is an expression that should
|
||
evaluate to a boolean value (<literal>true</literal> or
|
||
<literal>false</literal>).</para>
|
||
|
||
</simplesect>
|
||
|
||
|
||
<simplesect><title>Assertions</title>
|
||
|
||
<para>Assertions are generally used to check that certain requirements
|
||
on or between features and dependencies hold. They look like this:
|
||
|
||
<programlisting>
|
||
assert <replaceable>e1</replaceable>; <replaceable>e2</replaceable></programlisting>
|
||
|
||
where <replaceable>e1</replaceable> is an expression that should
|
||
evaluate to a boolean value. If it evaluates to
|
||
<literal>true</literal>, <replaceable>e2</replaceable> is returned;
|
||
otherwise expression evaluation is aborted and a backtrace is printed.</para>
|
||
|
||
<example xml:id='ex-subversion-nix'><title>Nix expression for Subversion</title>
|
||
<programlisting>
|
||
{ localServer ? false
|
||
, httpServer ? false
|
||
, sslSupport ? false
|
||
, pythonBindings ? false
|
||
, javaSwigBindings ? false
|
||
, javahlBindings ? false
|
||
, stdenv, fetchurl
|
||
, openssl ? null, httpd ? null, db4 ? null, expat, swig ? null, j2sdk ? null
|
||
}:
|
||
|
||
assert localServer -> db4 != null; <co xml:id='ex-subversion-nix-co-1' />
|
||
assert httpServer -> httpd != null && httpd.expat == expat; <co xml:id='ex-subversion-nix-co-2' />
|
||
assert sslSupport -> openssl != null && (httpServer -> httpd.openssl == openssl); <co xml:id='ex-subversion-nix-co-3' />
|
||
assert pythonBindings -> swig != null && swig.pythonSupport;
|
||
assert javaSwigBindings -> swig != null && swig.javaSupport;
|
||
assert javahlBindings -> j2sdk != null;
|
||
|
||
stdenv.mkDerivation {
|
||
name = "subversion-1.1.1";
|
||
...
|
||
openssl = if sslSupport then openssl else null; <co xml:id='ex-subversion-nix-co-4' />
|
||
...
|
||
}</programlisting>
|
||
</example>
|
||
|
||
<para><xref linkend='ex-subversion-nix' /> show how assertions are
|
||
used in the Nix expression for Subversion.</para>
|
||
|
||
<calloutlist>
|
||
|
||
<callout arearefs='ex-subversion-nix-co-1'>
|
||
<para>This assertion states that if Subversion is to have support
|
||
for local repositories, then Berkeley DB is needed. So if the
|
||
Subversion function is called with the
|
||
<varname>localServer</varname> argument set to
|
||
<literal>true</literal> but the <varname>db4</varname> argument
|
||
set to <literal>null</literal>, then the evaluation fails.</para>
|
||
</callout>
|
||
|
||
<callout arearefs='ex-subversion-nix-co-2'>
|
||
<para>This is a more subtle condition: if Subversion is built with
|
||
Apache (<literal>httpServer</literal>) support, then the Expat
|
||
library (an XML library) used by Subversion should be same as the
|
||
one used by Apache. This is because in this configuration
|
||
Subversion code ends up being linked with Apache code, and if the
|
||
Expat libraries do not match, a build- or runtime link error or
|
||
incompatibility might occur.</para>
|
||
</callout>
|
||
|
||
<callout arearefs='ex-subversion-nix-co-2'>
|
||
<para>This assertion says that in order for Subversion to have SSL
|
||
support (so that it can access <literal>https</literal> URLs), an
|
||
OpenSSL library must be passed. Additionally, it says that
|
||
<emphasis>if</emphasis> Apache support is enabled, then Apache's
|
||
OpenSSL should match Subversion's. (Note that if Apache support
|
||
is not enabled, we don't care about Apache's OpenSSL.)</para>
|
||
</callout>
|
||
|
||
<callout arearefs='ex-subversion-nix-co-4'>
|
||
<para>The conditional here is not really related to assertions,
|
||
but is worth pointing out: it ensures that if SSL support is
|
||
disabled, then the Subversion derivation is not dependent on
|
||
OpenSSL, even if a non-<literal>null</literal> value was passed.
|
||
This prevents an unnecessary rebuild of Subversion if OpenSSL
|
||
changes.</para>
|
||
</callout>
|
||
|
||
</calloutlist>
|
||
|
||
</simplesect>
|
||
|
||
|
||
|
||
<simplesect><title>With expressions</title>
|
||
|
||
<para>A <emphasis>with</emphasis> expression,
|
||
|
||
<programlisting>
|
||
with <replaceable>e1</replaceable>; <replaceable>e2</replaceable></programlisting>
|
||
|
||
introduces the attribute set <replaceable>e1</replaceable> into the
|
||
lexical scope of the expression <replaceable>e2</replaceable>. For
|
||
instance,
|
||
|
||
<programlisting>
|
||
let {
|
||
as = {x = "foo"; y = "bar";};
|
||
|
||
body = with as; x + y;
|
||
}</programlisting>
|
||
|
||
evaluates to <literal>"foobar"</literal> since the
|
||
<literal>with</literal> adds the <varname>x</varname> and
|
||
<varname>y</varname> attributes of <varname>as</varname> to the
|
||
lexical scope in the expression <literal>x + y</literal>. The most
|
||
common use of <literal>with</literal> is in conjunction with the
|
||
<function>import</function> function. E.g.,
|
||
|
||
<programlisting>
|
||
with (import ./definitions.nix); ...</programlisting>
|
||
|
||
makes all attributes defined in the file
|
||
<filename>definitions.nix</filename> available as if they were defined
|
||
locally in a <literal>rec</literal>-expression.</para>
|
||
|
||
</simplesect>
|
||
|
||
|
||
<simplesect><title>Operators</title>
|
||
|
||
<para><xref linkend='table-operators' /> lists the operators in the
|
||
Nix expression language, in order of precedence (from strongest to
|
||
weakest binding).</para>
|
||
|
||
<table xml:id='table-operators'>
|
||
<title>Operators</title>
|
||
<tgroup cols='3'>
|
||
<thead>
|
||
<row>
|
||
<entry>Syntax</entry>
|
||
<entry>Associativity</entry>
|
||
<entry>Description</entry>
|
||
</row>
|
||
</thead>
|
||
<tbody>
|
||
<row>
|
||
<entry><replaceable>e</replaceable> ?
|
||
<replaceable>id</replaceable></entry>
|
||
<entry>none</entry>
|
||
<entry>Test whether attribute set <replaceable>e</replaceable>
|
||
contains an attribute named
|
||
<replaceable>id</replaceable>.</entry>
|
||
</row>
|
||
<row>
|
||
<entry><replaceable>e1</replaceable> ++ <replaceable>e2</replaceable></entry>
|
||
<entry>right</entry>
|
||
<entry>List concatenation.</entry>
|
||
</row>
|
||
<row>
|
||
<entry><replaceable>e1</replaceable> + <replaceable>e2</replaceable></entry>
|
||
<entry>left</entry>
|
||
<entry>String or path concatenation.</entry>
|
||
</row>
|
||
<row>
|
||
<entry>! <replaceable>e</replaceable></entry>
|
||
<entry>left</entry>
|
||
<entry>Boolean negation.</entry>
|
||
</row>
|
||
<row>
|
||
<entry><replaceable>e1</replaceable> //
|
||
<replaceable>e2</replaceable></entry>
|
||
<entry>right</entry>
|
||
<entry>Return an attribute set consisting of the attributes in
|
||
<replaceable>e1</replaceable> and
|
||
<replaceable>e2</replaceable> (with the latter taking
|
||
precedence over the former in case of equally named attributes).</entry>
|
||
</row>
|
||
<row>
|
||
<entry><replaceable>e1</replaceable> ==
|
||
<replaceable>e2</replaceable></entry>
|
||
<entry>none</entry>
|
||
<entry>Equality.</entry>
|
||
</row>
|
||
<row>
|
||
<entry><replaceable>e1</replaceable> !=
|
||
<replaceable>e2</replaceable></entry>
|
||
<entry>none</entry>
|
||
<entry>Inequality.</entry>
|
||
</row>
|
||
<row>
|
||
<entry><replaceable>e1</replaceable> &&
|
||
<replaceable>e2</replaceable></entry>
|
||
<entry>left</entry>
|
||
<entry>Logical AND.</entry>
|
||
</row>
|
||
<row>
|
||
<entry><replaceable>e1</replaceable> ||
|
||
<replaceable>e2</replaceable></entry>
|
||
<entry>left</entry>
|
||
<entry>Logical OR.</entry>
|
||
</row>
|
||
<row>
|
||
<entry><replaceable>e1</replaceable> ->
|
||
<replaceable>e2</replaceable></entry>
|
||
<entry>none</entry>
|
||
<entry>Logical implication (equivalent to
|
||
<literal>!<replaceable>e1</replaceable> ||
|
||
<replaceable>e2</replaceable></literal>).</entry>
|
||
</row>
|
||
</tbody>
|
||
</tgroup>
|
||
</table>
|
||
|
||
</simplesect>
|
||
|
||
|
||
<simplesect xml:id="ssec-derivation"><title>Derivations</title>
|
||
|
||
<para>The most important built-in function is
|
||
<function>derivation</function>, which is used to describe a
|
||
single derivation (a build action). It takes as input an attribute
|
||
set, the attributes of which specify the inputs of the build.</para>
|
||
|
||
<itemizedlist>
|
||
|
||
<listitem xml:id="attr-system"><para>There must be an attribute named
|
||
<varname>system</varname> whose value must be a string specifying a
|
||
Nix platform identifier, such as <literal>"i686-linux"</literal> or
|
||
<literal>"powerpc-darwin"</literal><footnote><para>To figure out
|
||
your platform identifier, look at the line <quote>Checking for the
|
||
canonical Nix system name</quote> in the output of Nix's
|
||
<filename>configure</filename> script.</para></footnote> The build
|
||
can only be performed on a machine and operating system matching the
|
||
platform identifier. (Nix can automatically forward builds for
|
||
other platforms by forwarding them to other machines; see <xref
|
||
linkend='sec-distributed-builds' />.)</para></listitem>
|
||
|
||
<listitem><para>There must be an attribute named
|
||
<varname>name</varname> whose value must be a string. This is used
|
||
as a symbolic name for the component by <command>nix-env</command>,
|
||
and it is appended to the hash in the output path of the
|
||
derivation.</para></listitem>
|
||
|
||
<listitem><para>There must be an attribute named
|
||
<varname>builder</varname> that identifies the program that is
|
||
executed to perform the build. It can be either a derivation or a
|
||
source (a local file reference, e.g.,
|
||
<filename>./builder.sh</filename>).</para></listitem>
|
||
|
||
<listitem><para>Every attribute is passed as an environment variable
|
||
to the builder. Attribute values are translated to environment
|
||
variables as follows:
|
||
|
||
<itemizedlist>
|
||
|
||
<listitem><para>Strings, URIs, and integers are just passed
|
||
verbatim.</para></listitem>
|
||
|
||
<listitem><para>A <emphasis>path</emphasis> (e.g.,
|
||
<filename>../foo/sources.tar</filename>) causes the referenced
|
||
file to be copied to the store; its location in the store is put
|
||
in the environment variable. The idea is that all sources
|
||
should reside in the Nix store, since all inputs to a derivation
|
||
should reside in the Nix store.</para></listitem>
|
||
|
||
<listitem><para>A <emphasis>derivation</emphasis> causes that
|
||
derivation to be built prior to the present derivation; the
|
||
output path is put in the environment
|
||
variable.</para></listitem>
|
||
|
||
<listitem><para>Lists of the previous types are also allowed.
|
||
They are simply concatenated, separated by
|
||
spaces.</para></listitem>
|
||
|
||
</itemizedlist>
|
||
|
||
</para></listitem>
|
||
|
||
<listitem><para>The optional argument <varname>args</varname>
|
||
specifies command-line arguments to be passed to the builder. It
|
||
should be a list.</para></listitem>
|
||
|
||
</itemizedlist>
|
||
|
||
<para>(Note that <function>mkDerivation</function> in the standard
|
||
environment is a wrapper around <function>derivation</function> that
|
||
adds a default value for <varname>system</varname> and always uses
|
||
Bash as the builder, to which the supplied builder is passed as a
|
||
command-line argument. See <xref linkend='sec-standard-environment'
|
||
/>.)</para>
|
||
|
||
<para>The builder is executed as follows:
|
||
|
||
<itemizedlist>
|
||
|
||
<listitem><para>A temporary directory is created under the directory
|
||
specified by <envar>TMPDIR</envar> (default
|
||
<filename>/tmp</filename>) where the build will take place. The
|
||
current directory is changed to this directory.</para></listitem>
|
||
|
||
<listitem><para>The environment is cleared and set to the derivation
|
||
attributes, as specified above.</para></listitem>
|
||
|
||
<listitem><para>In addition, the following variables are set:
|
||
|
||
<itemizedlist>
|
||
|
||
<listitem><para><envar>NIX_BUILD_TOP</envar> contains the path of
|
||
the temporary directory for this build.</para></listitem>
|
||
|
||
<listitem><para>Also, <envar>TMPDIR</envar>,
|
||
<envar>TEMPDIR</envar>, <envar>TMP</envar>, <envar>TEMP</envar>
|
||
are set to point to the temporary directory. This is to prevent
|
||
the builder from accidentally writing temporary files anywhere
|
||
else. Doing so might cause interference by other
|
||
processes.</para></listitem>
|
||
|
||
<listitem><para><envar>PATH</envar> is set to
|
||
<filename>/path-not-set</filename> to prevent shells from
|
||
initialising it to their built-in default value.</para></listitem>
|
||
|
||
<listitem><para><envar>HOME</envar> is set to
|
||
<filename>/homeless-shelter</filename> to prevent programs from
|
||
using <filename>/etc/passwd</filename> or the like to find the
|
||
user's home directory, which could cause impurity. Usually, when
|
||
<envar>HOME</envar> is set, it is used as the location of the home
|
||
directory, even if it points to a non-existent
|
||
path.</para></listitem>
|
||
|
||
<listitem><para><envar>NIX_STORE</envar> is set to the path of the
|
||
top-level Nix store directory (typically,
|
||
<filename>/nix/store</filename>).</para></listitem>
|
||
|
||
<listitem><para><envar>out</envar> is set to point to the output
|
||
path of the derivation, which is a subdirectory of the Nix store.
|
||
The output path is a concatenation of the cryptographic hash of
|
||
all build inputs, and the <varname>name</varname>
|
||
attribute.</para></listitem>
|
||
|
||
</itemizedlist>
|
||
|
||
</para></listitem>
|
||
|
||
<listitem><para>If the output path already exists, it is removed.
|
||
Also, locks are acquired to prevent multiple Nix instances from
|
||
performing the same build at the same time.</para></listitem>
|
||
|
||
<listitem><para>A log of the combined standard output and error is
|
||
written to <filename>/nix/var/log/nix</filename>.</para></listitem>
|
||
|
||
<listitem><para>The builder is executed with the arguments specified
|
||
by the attribute <varname>args</varname>. If it exits with exit
|
||
code 0, it is considered to have succeeded.</para></listitem>
|
||
|
||
<listitem><para>The temporary directory is removed (unless the
|
||
<option>-K</option> option was specified).</para></listitem>
|
||
|
||
<listitem><para>If the build was successful, Nix scans the output
|
||
for references to the paths of the inputs. These so-called
|
||
<emphasis>retained dependencies</emphasis> could be used when the
|
||
output of the derivation is used (e.g., when it's executed or used
|
||
as input to another derivation), so if we deploy the derivation, we
|
||
should copy the retained dependencies as well. The scan is
|
||
performed by looking for the hash parts of file names of the
|
||
inputs.</para></listitem>
|
||
|
||
<listitem><para>After the build, Nix sets the last-modified
|
||
timestamp on all files in the build result to 0 (00:00:00 1/1/1970
|
||
UTC), sets the group to the default group, and sets the mode of the
|
||
file to 0444 or 0555 (i.e., read-only, with execute permission
|
||
enabled if the file was originally executable). Note that possible
|
||
<literal>setuid</literal> and <literal>setgid</literal> bits are
|
||
cleared. Setuid and setgid programs are not currently supported by
|
||
Nix. This is because the Nix archives used in deployment have no
|
||
concept of ownership information, and because it makes the build
|
||
result dependent on the user performing the build.</para></listitem>
|
||
|
||
</itemizedlist>
|
||
|
||
</para>
|
||
|
||
</simplesect>
|
||
|
||
|
||
<simplesect><title>Other built-in functions</title>
|
||
|
||
<para>TODO</para>
|
||
|
||
</simplesect>
|
||
|
||
|
||
<simplesect><title>Comments</title>
|
||
|
||
<para>Comments can be single-line, started with a <literal>#</literal>
|
||
character, or inline/multi-line, enclosed within <literal>/*
|
||
... */</literal>.</para>
|
||
|
||
</simplesect>
|
||
|
||
|
||
</section>
|
||
|
||
|
||
|
||
<section xml:id='sec-standard-environment'><title>The standard environment</title>
|
||
|
||
<para>The standard build environment in the Nix Packages collection
|
||
provides a basic environment for building Unix packages. It consists
|
||
of the following components:
|
||
|
||
<itemizedlist>
|
||
|
||
<listitem><para>The GNU C Compiler, configured with C and C++
|
||
support. On Linux, the compiler has been patched to provide greater
|
||
<quote>purity</quote> assurance. For instance, the compiler doesn't
|
||
search in locations such as <filename>/usr/include</filename>. In
|
||
fact, attempts to add such directories through the
|
||
<option>-I</option> flag are filtered out. Likewise, the linker
|
||
(from GNU binutils) doesn't search in standard locations such as
|
||
<filename>/usr/lib</filename>. Programs built on Linux are linked
|
||
against a GNU C Library that likewise doesn't search in the default
|
||
system locations.</para></listitem>
|
||
|
||
<listitem><para>GNU coreutils (contains a few dozen standard Unix
|
||
commands).</para></listitem>
|
||
|
||
<listitem><para>GNU findutils (contains
|
||
<command>find</command>).</para></listitem>
|
||
|
||
<listitem><para>GNU diffutils (contains <command>diff</command>,
|
||
<command>cmp</command>).</para></listitem>
|
||
|
||
<listitem><para>GNU <command>sed</command>.</para></listitem>
|
||
|
||
<listitem><para>GNU <command>grep</command>.</para></listitem>
|
||
|
||
<listitem><para>GNU <command>awk</command>.</para></listitem>
|
||
|
||
<listitem><para>GNU <command>tar</command>.</para></listitem>
|
||
|
||
<listitem><para><command>gzip</command> and
|
||
<command>bzip2</command>.</para></listitem>
|
||
|
||
<listitem><para>GNU Make. It has been patched to provide
|
||
<quote>nested</quote> output that can be fed into the
|
||
<command>log2xml</command> command and <command>log2html</command>
|
||
stylesheet to create a structured, readable output of the build
|
||
steps performed by Make.</para></listitem>
|
||
|
||
<listitem><para>Bash. This is the shell used for all builders in
|
||
the Nix Packages collection. Not using <command>/bin/sh</command>
|
||
removes a large source of portability problems.</para></listitem>
|
||
|
||
<listitem><para>Patch.</para></listitem>
|
||
|
||
</itemizedlist>
|
||
|
||
</para>
|
||
|
||
<para>The standard environment is used by passing it as an input
|
||
called <envar>stdenv</envar> to the derivation, and then doing
|
||
|
||
<programlisting>
|
||
source $stdenv/setup</programlisting>
|
||
|
||
at the top of the builder.</para>
|
||
|
||
<para>Apart from adding the aforementioned commands to the
|
||
<envar>PATH</envar>, <filename>setup</filename> also does the
|
||
following:
|
||
|
||
<itemizedlist>
|
||
|
||
<listitem><para>All input components specified in the
|
||
<envar>buildInputs</envar> environment variable have their
|
||
<filename>/bin</filename> subdirectory added to <envar>PATH</envar>,
|
||
their <filename>/include</filename> subdirectory added to the C/C++
|
||
header file search path, and their <filename>/lib</filename>
|
||
subdirectory added to the linker search path. This can be extended.
|
||
For instance, when the <command>pkgconfig</command> component is
|
||
used, the subdirectory <filename>/lib/pkgconfig</filename> of each
|
||
input is added to the <envar>PKG_CONFIG_PATH</envar> environment
|
||
variable.</para></listitem>
|
||
|
||
<listitem><para>The environment variable
|
||
<envar>NIX_CFLAGS_STRIP</envar> is set so that the compiler strips
|
||
debug information from object files. This can be disabled by
|
||
setting <envar>NIX_STRIP_DEBUG</envar> to
|
||
<literal>0</literal>.</para></listitem>
|
||
|
||
</itemizedlist>
|
||
|
||
</para>
|
||
|
||
<para>The <filename>setup</filename> script also exports a function
|
||
called <function>genericBuild</function> that knows how to build
|
||
typical Autoconf-style components. It can be customised to perform
|
||
builds for any type of component. It is advisable to use
|
||
<function>genericBuild</function> since it provides facilities that
|
||
are almost always useful such as unpacking of sources, patching of
|
||
sources, nested logging, etc.</para>
|
||
|
||
<para>The definitive, up-to-date documentation of the generic builder
|
||
is the source itself, which resides in
|
||
<filename>pkgs/stdenv/generic/setup.sh</filename>.</para>
|
||
|
||
|
||
<section><title>Customising the generic builder</title>
|
||
|
||
<para>The operation of the generic builder can be modified in many
|
||
places by setting certain variables. These <emphasis>hook
|
||
variables</emphasis> are typically set to the name of some shell
|
||
function defined by you. For instance, to perform some additional
|
||
steps after <command>make install</command> you would set the
|
||
<varname>postInstall</varname> variable:
|
||
|
||
<programlisting>
|
||
postInstall=myPostInstall
|
||
|
||
myPostInstall() {
|
||
mkdir $out/share/extra
|
||
cp extrafiles/* $out/share/extra
|
||
}</programlisting>
|
||
|
||
</para>
|
||
|
||
<para>The generic builder has a number of <emphasis>phases</emphasis>,
|
||
each of which can be override in its entirety by setting the indicated
|
||
variable. The phases are:
|
||
|
||
<itemizedlist>
|
||
|
||
<listitem>
|
||
|
||
<para><function>unpackPhase</function> unpacks the source files
|
||
listed in the <envar>src</envar> environment variable to the
|
||
current directory. It supports <filename>tar</filename> files,
|
||
optionally compressed with <command>gzip</command> or
|
||
<command>bzip2</command>; Zip files (but note that the
|
||
<command>unzip</command> command is not a part of the standard
|
||
environment; you should add it as a build input yourself); and
|
||
unpacked source trees (i.e., directories; they are copied
|
||
verbatim). You can add support for other file types by setting
|
||
the <varname>findUnpacker</varname> hook. This hook should set
|
||
the variable <varname>unpackCmd</varname> to contain the command
|
||
to be executed to unpack the file.</para>
|
||
|
||
<para>After unpacking all source files,
|
||
<function>unpackPhase</function> changes the current directory to
|
||
the directory created by unpacking the sources. If there are
|
||
multiple source directories, you should set
|
||
<varname>sourceRoot</varname> to the name of the intended
|
||
directory.</para>
|
||
|
||
<para>It also calls the hook <varname>postUnpack</varname> after
|
||
unpacking.</para>
|
||
|
||
</listitem>
|
||
|
||
<listitem><para><function>patchPhase</function> calls the
|
||
<command>patch</command> command with the <option>-p1</option>
|
||
option for each patch file listed in the <envar>patches</envar>
|
||
variable.</para></listitem>
|
||
|
||
<listitem>
|
||
|
||
<para><function>configurePhase</function> runs the script called
|
||
<filename>configure</filename> in the current directory with a
|
||
<option>--prefix</option> set to the output path. You can add
|
||
additional flags through the <varname>configureFlags</varname>
|
||
variable. If <filename>configure</filename> does not exist,
|
||
nothing happens.</para>
|
||
|
||
<para>Before and after running <filename>configure</filename>, the
|
||
hooks <varname>preConfigure</varname> and
|
||
<varname>postConfigure</varname> are called, respectively.</para>
|
||
|
||
</listitem>
|
||
|
||
<listitem>
|
||
|
||
<para><function>buildPhase</function> calls
|
||
<command>make</command>. You can set flags for
|
||
<command>make</command> through the <varname>makeFlags</varname>
|
||
variable.</para>
|
||
|
||
<para>Before and after running <command>make</command>, the hooks
|
||
<varname>preBuild</varname> and <varname>postBuild</varname> are
|
||
called, respectively.</para>
|
||
|
||
</listitem>
|
||
|
||
<listitem><para><function>checkPhase</function> calls <command>make
|
||
check</command>, but only if the <varname>doCheck</varname> variable
|
||
is set to <literal>1</literal>. Additional flags can be set through
|
||
the <varname>checkFlags</varname> variable.</para></listitem>
|
||
|
||
<listitem>
|
||
|
||
<para><function>installPhase</function> calls <command>make
|
||
install</command>. Additional flags can be set through the
|
||
<varname>installFlags</varname> variable. It also strips any
|
||
static libraries in the output path of debug information unless
|
||
<varname>dontStrip</varname> is set to
|
||
<literal>1</literal>.</para>
|
||
|
||
<para>Before and after running <command>make install</command>,
|
||
the hooks <varname>preInstall</varname> and
|
||
<varname>postInstall</varname> are called, respectively.</para>
|
||
|
||
</listitem>
|
||
|
||
<listitem>
|
||
|
||
<para><function>distPhase</function> calls <command>make
|
||
dist</command>, but only if the <varname>doDist</varname> variable
|
||
is set to <literal>1</literal>. Additional flags can be set
|
||
through the <varname>distFlags</varname> variable. The resulting
|
||
tarball is copied to the <filename>/tarballs</filename>
|
||
subdirectory of the output path.</para>
|
||
|
||
<para>Before and after running <command>make dist</command>, the
|
||
hooks <varname>preDist</varname> and <varname>postDist</varname>
|
||
are called, respectively.</para>
|
||
|
||
</listitem>
|
||
|
||
</itemizedlist>
|
||
|
||
</para>
|
||
|
||
<para>You can change the order in which phases are executed, or add
|
||
new phases, by setting the <varname>phases</varname> variable. The
|
||
default is <literal>patchPhase configurePhase buildPhase checkPhase
|
||
installPhase distPhase</literal>.</para>
|
||
|
||
</section>
|
||
|
||
|
||
<section><title>Debugging failed builds</title>
|
||
|
||
<para>At the beginning of each phase, the set of all shell variables
|
||
is written to the file <filename>env-vars</filename> at the top-level
|
||
build directory. This is useful for debugging: it allows you to
|
||
recreate the environment in which a build was performed. For
|
||
instance, if a build fails, then assuming you used the
|
||
<option>-K</option> flag, you can go to the output directory and
|
||
<quote>switch</quote> to the environment of the builder:
|
||
|
||
<screen>
|
||
$ nix-build -K ./foo.nix
|
||
... fails, keeping build directory `/tmp/nix-1234-0'
|
||
|
||
$ cd /tmp/nix-1234-0
|
||
|
||
$ source env-vars
|
||
|
||
<lineannotation>(edit some files...)</lineannotation>
|
||
|
||
$ make
|
||
|
||
<lineannotation>(execution continues with the same GCC, make, etc.)</lineannotation></screen>
|
||
|
||
</para>
|
||
|
||
</section>
|
||
|
||
|
||
</section>
|
||
|
||
|
||
</chapter>
|