lix-project/lix
20
57
Fork 31
Code Issues 347 Code Review (Gerrit) Projects 3 Releases Wiki Activity

* Lots of manual stuff. Reference pages for most Nix commands.

Browse source 
* nix-pull now requires the full url to the manifest, i.e.,
  `/MANIFEST/' is no longer automatically appended.
* nix-prefetch-url works again.
This commit is contained in:
Eelco Dolstra 2004-02-10 16:14:47 +00:00
parent 6551b36790
commit 92e832348d
16 changed files with 678 additions and 24 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
doc/manual/Makefile.am
View file

@ -6,8 +6,12 @@ XSLTPROC = $(ENV) $(xsltproc) $(xmlflags) --catalogs \
--param section.label.includes.component.label 1 \
--param html.stylesheet \'style.css\'
man1_MANS = nix-env.1 nix-store.1 nix-instantiate.1 \
nix-collect-garbage.1 nix-push.1 nix-pull.1 \
nix-prefetch-url.1
SOURCES = manual.xml introduction.xml installation.xml overview.xml \
nix-env.xml nix-store.xml nix-instantiate.xml \
$(man1_MANS:.1=.xml) \
troubleshooting.xml bugs.xml opt-common.xml opt-common-syn.xml \
quick-start.xml style.css images
@ -18,8 +22,6 @@ manual.is-valid: $(SOURCES) version.xml
version.xml:
echo -n $(VERSION) > version.xml
man1_MANS = nix-env.1 nix-store.1 nix-instantiate.1
man $(MANS): $(SOURCES) manual.is-valid
$(XSLTPROC) $(docbookxsl)/manpages/docbook.xsl manual.xml

73
doc/manual/bugs.xml
View file

@ -9,6 +9,16 @@
</para>
</listitem>
<listitem>
<para>
Generations properly form a tree. E.g., if after switching to
generation 39, we perform an installation action, a generation
43 is created which is a descendant of 39, not 42. So a
rollback from 43 ought to go back to 39. This is not
currently implemented; generations form a linear sequence.
</para>
</listitem>
<listitem>
<para>
Unify the concepts of successors and substitutes into a
@ -43,12 +53,63 @@
</para>
</listitem>
<listitem>
<para>
The current garbage collector is a hack. It should be
integrated into <command>nix-store</command>. It should
delete derivations in an order determined by topologically
sorting derivations under the points-to relation. This
ensures that no store paths ever exist that point to
non-existant store paths.
</para>
</listitem>
<listitem>
<para>
There are race conditions between the garbage collector and
other Nix tools. For instance, when we run
<command>nix-env</command> to build and install a derivation
and run the garbage collector at the same time, the garbage
collector may kick in exactly between the build and
installation steps, i.e., before the newly built derivation
has become reachable from a root of the garbage collector.
</para>
<para>
One solution would be for these programs to properly register
temporary roots for the collector. Another would be to use
stop-the-world garbage collection: if any tool is running, the
garbage collector blocks, and vice versa. These solutions do
not solve the situation where multiple tools are involved,
e.g.,
<screen>
$ nix-store -r $(nix-instantiate foo.nix)</screen>
since even if <command>nix-instantiate</command> where to
register a temporary root, it would be released by the time
<command>nix-store</command> is started. A solution would be
to write the intermediate value to a file that is used as a
root to the collector, e.g.,
<screen>
$ nix-instantiate foo.nix > /nix/var/nix/roots/bla
$ nix-store -r $(cat /nix/var/nix/roots/bla)</screen>
</para>
</listitem>
<listitem>
<para>
For security, <command>nix-push</command> manifests should be
digitally signed, and <command>nix-pull</command> should
verify the signatures. The actual NAR archives in the cache
do not need to be signed, since the manifest contains
cryptographic hashes of these files (and
<filename>fetchurl.nix</filename> checks them).
</para>
</listitem>
</itemizedlist>
</appendix>
<!--
local variables:
sgml-parent-document: ("book.xml" "appendix")
end:
-->

20
doc/manual/manual.xml
View file

@ -12,6 +12,10 @@
<!ENTITY nix-env SYSTEM "nix-env.xml">
<!ENTITY nix-store SYSTEM "nix-store.xml">
<!ENTITY nix-instantiate SYSTEM "nix-instantiate.xml">
<!ENTITY nix-collect-garbage SYSTEM "nix-collect-garbage.xml">
<!ENTITY nix-push SYSTEM "nix-push.xml">
<!ENTITY nix-pull SYSTEM "nix-pull.xml">
<!ENTITY nix-prefetch-url SYSTEM "nix-prefetch-url.xml">
<!ENTITY troubleshooting SYSTEM "troubleshooting.xml">
<!ENTITY bugs SYSTEM "bugs.xml">
<!ENTITY version SYSTEM "version.xml">
@ -52,6 +56,22 @@
<title>nix-instantiate</title>
&nix-instantiate;
</sect1>
<sect1>
<title>nix-collect-garbage</title>
&nix-collect-garbage;
</sect1>
<sect1>
<title>nix-push</title>
&nix-push;
</sect1>
<sect1>
<title>nix-pull</title>
&nix-pull;
</sect1>
<sect1>
<title>nix-prefetch-url</title>
&nix-prefetch-url;
</sect1>
</appendix>
&troubleshooting;

75
doc/manual/nix-collect-garbage.xml Normal file
View file

@ -0,0 +1,75 @@
<refentry>
<refnamediv>
<refname>nix-collect-garbage</refname>
<refpurpose>determine the set of unreachable store paths</refpurpose>
</refnamediv>
<refsynopsisdiv>
<cmdsynopsis>
<command>nix-collect-garbage</command>
<arg><option>--invert</option></arg>
<arg><option>--no-successors</option></arg>
</cmdsynopsis>
</refsynopsisdiv>
<refsection>
<title>Description</title>
<para>
The command <command>nix-collect-garbage</command> determines
the paths in the Nix store that are garbage, that is, not
reachable from outside of the store. These paths can be safely
deleted without affecting the integrity of the system.
</para>
</refsection>
<refsection>
<title>Options</title>
<variablelist>
<varlistentry>
<term><option>--invert</option></term>
<listitem>
<para>
Causes the set of <emphasis>reachable</emphasis> paths to
be printed, rather than the unreachable paths. These are
the paths that may <emphasis>not</emphasis> be deleted.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>--no-successors</option></term>
<listitem>
<para>
Causes <command>nix-collect-garbage</command> not to
follow successor relations. By default, if a derivation
store expression is reachable, its successor (i.e., a
closure store expression) is also considered to be
reachable. This option is always safe, but garbage
collecting successors may cause undesirable rebuilds later
on.
</para>
</listitem>
</varlistentry>
</variablelist>
</refsection>
<refsection>
<title>Examples</title>
<para>
To delete all unreachable paths, do the following:
<screen>
$ nix-collect-garbage | xargs nix-store --delete</screen>
</para>
</refsection>
</refentry>

226
doc/manual/nix-env.xml
View file

@ -111,8 +111,9 @@
<listitem>
<para>
For the <option>--install</option>,
<option>--upgrade</option> and
<option>--uninstall</option> operations, this flag will
<option>--upgrade</option>, <option>--uninstall</option>,
<option>--switch-generation</option> and
<option>--rollback</option> operations, this flag will
cause <command>nix-env</command> to print what
<emphasis>would</emphasis> be done if this flag had not
been specified, without actually doing it.
@ -594,4 +595,225 @@ foo-1.2.3</screen>
</refsection>
<!--######################################################################-->
<refsection>
<title>Operation <option>--switch-profile</option></title>
<refsection>
<title>Synopsis</title>
<cmdsynopsis>
<command>nix-env</command>
<group choice='req'>
<arg choice='plain'><option>--switch-profile</option></arg>
<arg choice='plain'><option>-S</option></arg>
</group>
<arg choice='req'><replaceable>path</replaceable></arg>
</cmdsynopsis>
</refsection>
<refsection>
<title>Description</title>
<para>
This operation makes <replaceable>path</replaceable> the
current profile for the user. That is, the symlink
<filename>~/.nix-profile</filename> is made to point to
<replaceable>path</replaceable>.
</para>
</refsection>
<refsection>
<title>Examples</title>
<screen>
$ nix-env -S ~/my-profile</screen>
</refsection>
</refsection>
<!--######################################################################-->
<refsection>
<title>Operation <option>--list-generations</option></title>
<refsection>
<title>Synopsis</title>
<cmdsynopsis>
<command>nix-env</command>
<arg choice='req'><option>--list-generations</option></arg>
</cmdsynopsis>
</refsection>
<refsection>
<title>Description</title>
<para>
This operation print a list of all the currently existing
generations for the active profile. These may be switched to
using the <option>--switch-generation</option> operation. It
also prints the creation date of the generation, and indicates
the current generation.
</para>
</refsection>
<refsection>
<title>Examples</title>
<screen>
$ nix-env --list-generations
95 2004-02-06 11:48:24
96 2004-02-06 11:49:01
97 2004-02-06 16:22:45
98 2004-02-06 16:24:33 (current)</screen>
</refsection>
</refsection>
<!--######################################################################-->
<refsection>
<title>Operation <option>--switch-generation</option></title>
<refsection>
<title>Synopsis</title>
<cmdsynopsis>
<command>nix-env</command>
<group choice='req'>
<arg choice='plain'><option>--switch-generation</option></arg>
<arg choice='plain'><option>-G</option></arg>
</group>
<arg choice='req'><replaceable>generation</replaceable></arg>
</cmdsynopsis>
</refsection>
<refsection>
<title>Description</title>
<para>
This operation makes generation number
<replaceable>generation</replaceable> the current generation
of the active profile. That is, if the
<filename><replaceable>profile</replaceable></filename> is the
path to the active profile, then the symlink
<filename><replaceable>profile</replaceable></filename> is
made to point to
<filename><replaceable>profile</replaceable>-<replaceable>generation</replaceable>-link</filename>,
which is in turn a symlink to the actual user environment in
the Nix store.
</para>
<para>
Switching will fail if the specified generation does not
exist.
</para>
</refsection>
<refsection>
<title>Examples</title>
<screen>
$ nix-env -G 42
switching from generation 50 to 42</screen>
</refsection>
</refsection>
<!--######################################################################-->
<refsection>
<title>Operation <option>--rollback</option></title>
<refsection>
<title>Synopsis</title>
<cmdsynopsis>
<command>nix-env</command>
<arg choice='req'><option>--rollback</option></arg>
</cmdsynopsis>
</refsection>
<refsection>
<title>Description</title>
<para>
This operation switches to the <quote>previous</quote>
generation of the active profile, that is, the highest
numbered generation lower than the current generation, if it
exists. It is just a convenience wrapper around
<option>--list-generations</option> and
<option>--switch-generation</option>.
</para>
</refsection>
<refsection>
<title>Examples</title>
<screen>
$ nix-env --rollback
switching from generation 92 to 91
$ nix-env --rolback
error: no generation older than the current (91) exists</screen>
</refsection>
</refsection>
<!--######################################################################-->
<refsection>
<title>Operation <option>--import</option></title>
<refsection>
<title>Synopsis</title>
<cmdsynopsis>
<command>nix-env</command>
<group choice='req'>
<arg choice='plain'><option>--import</option></arg>
<arg choice='plain'><option>-I</option></arg>
</group>
<arg choice='req'><replaceable>path</replaceable></arg>
</cmdsynopsis>
</refsection>
<refsection>
<title>Description</title>
<para>
This operation makes <replaceable>path</replaceable> the
default active Nix expression for the user. That is, the
symlink <filename>~/.nix-userenv</filename> is made to point
to <replaceable>path</replaceable>.
</para>
</refsection>
<refsection>
<title>Examples</title>
<screen>
$ nix-env -I ~/nixpkgs-0.5/</screen>
</refsection>
</refsection>
</refentry>

46
doc/manual/nix-instantiate.xml
View file

@ -8,18 +8,58 @@
<cmdsynopsis>
<command>nix-instantiate</command>
&opt-common-syn;
<arg rep='repeat'><replaceable>files</replaceable></arg>
<arg choice='plain' rep='repeat'><replaceable>files</replaceable></arg>
</cmdsynopsis>
</refsynopsisdiv>
<refsect1>
<refsection>
<title>Description</title>
<para>
The command <command>nix-instantiate</command> generates
(low-level) store expressions from (high-level) Nix expressions.
It loads and evaluates the Nix expressions in each of
<replaceable>files</replaceable>. Each top-level expression
should evaluate to a derivation, a list of derivations, or a set
of derivations. The paths of the resulting store expressions
are printed on standard output.
</para>
</refsect1>
<para>
This command is generally used for testing Nix expression before
they are used with <command>nix-env</command>.
</para>
</refsection>
<refsection>
<title>Options</title>
<variablelist>
&opt-common;
</variablelist>
</refsection>
<refsection>
<title>Examples</title>
<screen>
$ nix-instantiate gcc.nix <lineannotation>(instantiate)</lineannotation>
/nix/store/468abdcb93aa22bb721142615b97698b-d-gcc-3.3.2.store
$ nix-store -r $(nix-instantiate gcc.nix) <lineannotation>(build)</lineannotation>
$ nix-store -r $(nix-instantiate gcc.nix) <lineannotation>(print output path)</lineannotation>
/nix/store/9afa718cddfdfe94b5b9303d0430ceb1-gcc-3.3.2
$ ls -l /nix/store/9afa718cddfdfe94b5b9303d0430ceb1-gcc-3.3.2
dr-xr-xr-x 2 eelco users 360 2003-12-01 16:12 bin
dr-xr-xr-x 3 eelco users 72 2003-12-01 16:12 include
...</screen>
</refsection>
</refentry>

54
doc/manual/nix-prefetch-url.xml Normal file
View file

@ -0,0 +1,54 @@
<refentry>
<refnamediv>
<refname>nix-prefetch-url</refname>
<refpurpose>copy a file from a URL into the store and print its MD5 hash</refpurpose>
</refnamediv>
<refsynopsisdiv>
<cmdsynopsis>
<command>nix-prefetch-url</command>
<arg choice='plain'><replaceable>url</replaceable></arg>
</cmdsynopsis>
</refsynopsisdiv>
<refsection>
<title>Description</title>
<para>
The command <command>nix-prefetch-url</command> downloads the
file referenced by the URL <replaceable>url</replaceable>,
prints its MD5 cryptographic hash code, and copies it into the
Nix store. The file name in the store is
<filename><replaceable>hash</replaceable>-<replaceable>basename</replaceable></filename>,
where <replaceable>basename</replaceable> is everything
following the final slash in <replaceable>url</replaceable>.
</para>
<para>
This command is just a convenience to Nix expression writers.
Often a Nix expressions fetch some source distribution from the
network using the <literal>fetchurl</literal> expression
contained in <literal>nixpkgs</literal>. However,
<literal>fetchurl</literal> requires an MD5 hash. If you don't
know the hash, you would have to download the file first, and
then <literal>fetchurl</literal> would download it again when
you build your Nix expression. Since
<literal>fetchurl</literal> uses the same name for the
downloaded file as <command>nix-prefetch-url</command>, the
redundant download can be avoided.
</para>
</refsection>
<refsection>
<title>Examples</title>
<screen>
$ nix-prefetch-url ftp://ftp.nluug.nl/pub/gnu/make/make-3.80.tar.bz2
...
file has hash 0bbd1df101bc0294d440471e50feca71
...</screen>
</refsection>
</refentry>

43
doc/manual/nix-pull.xml Normal file
View file

@ -0,0 +1,43 @@
<refentry>
<refnamediv>
<refname>nix-pull</refname>
<refpurpose>pull substitutes from a network cache</refpurpose>
</refnamediv>
<refsynopsisdiv>
<cmdsynopsis>
<command>nix-pull</command>
<arg choice='plain'><replaceable>url</replaceable></arg>
</cmdsynopsis>
</refsynopsisdiv>
<refsection>
<title>Description</title>
<para>
The command <command>nix-pull</command> obtains a list of
pre-built store paths from the URL
<replaceable>url</replaceable>, and for each of these store
paths, registers a substitute derivation that downloads and
unpacks it into the Nix store. This is used to speed up
installations: if you attempt to install something that has
already been built and stored into the network cache, Nix can
transparently re-use the pre-built store paths.
</para>
<para>
The file at <replaceable>url</replaceable> must be compatible
with the files created by <replaceable>nix-push</replaceable>.
</para>
</refsection>
<refsection>
<title>Examples</title>
<screen>
$ nix-pull http://catamaran.labs.cs.uu.nl/dist/nix/nixpkgs-0.5pre753/MANIFEST</screen>
</refsection>
</refentry>

138
doc/manual/nix-push.xml Normal file
View file

@ -0,0 +1,138 @@
<refentry>
<refnamediv>
<refname>nix-push</refname>
<refpurpose>push store paths onto a network cache</refpurpose>
</refnamediv>
<refsynopsisdiv>
<cmdsynopsis>
<command>nix-push</command>
<arg choice='plain'><replaceable>archives-put-url</replaceable></arg>
<arg choice='plain'><replaceable>archives-get-url</replaceable></arg>
<arg choice='plain'><replaceable>manifest-put-url</replaceable></arg>
<arg choice='plain' rep='repeat'><replaceable>paths</replaceable></arg>
</cmdsynopsis>
</refsynopsisdiv>
<refsection>
<title>Description</title>
<para>
The command <command>nix-push</command> builds a set of store
expressions (if necessary), and then packages and uploads all
store paths in the resulting closures to a server. A network
cache thus populated can subsequently be used to speed up
software deployment on other machines using the
<command>nix-pull</command> command.
</para>
<para>
<command>nix-push</command> performs the following actions.
<orderedlist>
<listitem>
<para>
The store expressions stored in
<replaceable>paths</replaceable> are realised (using
<literal>nix-store --realise</literal>).
</para>
</listitem>
<listitem>
<para>
All paths in the closure of the store expressions stored
in <replaceable>paths</replaceable> are determined (using
<literal>nix-store --query --requisites
--include-successors</literal>). It should be noted that
since the <option>--include-successors</option> flag is
used, if you specify a derivation store expression, you
get a combined source/binary distribution. If you only
want a binary distribution, you should specify the closure
store expression that result from realising these (see
below).
</para>
</listitem>
<listitem>
<para>
All store paths determined in the previous step are
packaged and compressed into a <command>bzip</command>ped
NAR archive (extension <filename>.nar.bz2</filename>).
</para>
</listitem>
<listitem>
<para>
A <emphasis>manifest</emphasis> is created that contains
information on the store paths, their eventual URLs in the
cache, and cryptographic hashes of the contents of the NAR
archives.
</para>
</listitem>
<listitem>
<para>
Each store path is uploaded to the remote directory
specified by <replaceable>archives-put-url</replaceable>.
HTTP PUT requests are used to do this. However, before a
file <varname>x</varname> is uploaded to
<literal><replaceable>archives-put-url</replaceable>/<varname>x</varname></literal>,
<command>nix-push</command> first determines whether this
upload is unnecessary by issuing a HTTP HEAD request on
<literal><replaceable>archives-get-url</replaceable>/<varname>x</varname></literal>.
This allows a cache to be shared between many partially
overlapping <command>nix-push</command> invocations.
(We use two URLs because the upload URL typically
refers to a CGI script, while the download URL just refers
to a file system directory on the server.)
</para>
</listitem>
<listitem>
<para>
The manifest is uploaded using an HTTP PUT request to
<replaceable>manifest-put-url</replaceable>. The
corresponding URL to download the manifest can then be
used by <command>nix-pull</command>.
</para>
</listitem>
</orderedlist>
</para>
</refsection>
<refsection>
<title>Examples</title>
<para>
To upload files there typically is some CGI script on the server
side. This script should be be protected with a password. The
following example uploads the store paths resulting from
building the Nix expressions in <filename>foo.nix</filename>,
passing appropriate authentication information:
<screen>
$ nix-push \
http://foo@bar:server.domain/cgi-bin/upload.pl/cache \
http://server.domain/cache \
http://foo@bar:server.domain/cgi-bin/upload.pl/MANIFEST \
$(nix-instantiate foo.nix)</screen>
This will push both sources and binaries (and any build-time
dependencies used in the build, such as compilers).
</para>
<para>
If we just want to push binaries, not sources and build-time
dependencies, we can do:
<screen>
$ nix-push <replaceable>urls</replaceable> $(nix-instantiate $(nix-store -r foo.nix))</screen>
</para>
</refsection>
</refentry>

5
doc/manual/overview.xml
View file

@ -173,9 +173,8 @@ $ nix-env -if pkgs/system/i686-linux.nix pan</screen>
</para>
<screen>
$ nix-pull
http://catamaran.labs.cs.uu.nl/dist/nix/nixpkgs-<replaceable>version</replaceable>/
obtaining list of Nix archives at http://catamaran.labs.cs.uu.nl/dist/nix/nixpkgs-<replaceable>version</replaceable>...
$ nix-pull http://catamaran.labs.cs.uu.nl/dist/nix/nixpkgs-<replaceable>version</replaceable>/MANIFEST
obtaining list of Nix archives at http://catamaran.labs.cs.uu.nl/dist/nix/nixpkgs-<replaceable>version</replaceable>/MANIFEST...
...</screen>
<para>

2
doc/manual/quick-start.xml
View file

@ -51,7 +51,7 @@ $ tar xfj nixpkgs-<replaceable>version</replaceable>.tar.bz2</screen>
network, rather than built from source.
<screen>
$ nix-pull http://<replaceable>...</replaceable>/nix/nixpkgs-<replaceable>version</replaceable>/</screen>
$ nix-pull http://<replaceable>...</replaceable>/nix/nixpkgs-<replaceable>version</replaceable>/MANIFEST</screen>
</para>

4
doc/manual/style.css
View file

@ -1,5 +1,5 @@
/* Copied from http://bakefile.sourceforge.net/ and covered by the GNU
GPL. */
/* Copied from http://bakefile.sourceforge.net/, which appears
licensed under the GNU GPL. */
/***************************************************************************

2
scripts/nix-prefetch-url.in
View file

@ -25,7 +25,7 @@ rename $out, $out2;
# Create a Nix expression.
my $nixexpr =
"(import @datadir@/nix/corepkgs/fetchurl) " .
"{url = $url; md5 = \"$hash\"; system = \"@system@\"}";
"{url = $url; md5 = \"$hash\"; system = \"@system@\";}";
print "expr: $nixexpr\n";

2
scripts/nix-push.in
View file

@ -169,5 +169,5 @@ foreach my $nararchive (@nararchives) {
# Upload the manifest.
print STDERR "uploading manifest...\n";
system("$curl --show-error --upload-file " .
"'$manifest' '$manifest_put_url/' > /dev/null") == 0 or
"'$manifest' '$manifest_put_url' > /dev/null") == 0 or
die "curl failed on $manifest: $?";

2
scripts/readmanifest.pm.in
View file

@ -11,7 +11,7 @@ sub processURL {
print "obtaining list of Nix archives at $url...\n";
system("curl --fail --silent --show-error " .
"'$url/MANIFEST' > '$manifest' 2> /dev/null") == 0
"'$url' > '$manifest' 2> /dev/null") == 0
or die "curl failed: $?";
open MANIFEST, "<$manifest";

2
src/nix-hash/help.txt
View file

@ -1,6 +1,6 @@
nix-hash [OPTIONS...] [FILES...]
`nix-hash computes and prints cryptographic hashes for the specified
`nix-hash' computes and prints cryptographic hashes for the specified
files.
--flat: compute hash of regular file contents, not metadata