diff --git a/doc/manual/Makefile.am b/doc/manual/Makefile.am index 60853df05..f5a93cf30 100644 --- a/doc/manual/Makefile.am +++ b/doc/manual/Makefile.am @@ -20,6 +20,7 @@ MANUAL_SRCS = manual.xml introduction.xml installation.xml \ $(man1_MANS:.1=.xml) \ troubleshooting.xml bugs.xml opt-common.xml opt-common-syn.xml \ env-common.xml quick-start.xml nix-lang-ref.xml glossary.xml \ + conf-file.xml \ style.css images manual.is-valid: $(MANUAL_SRCS) version.txt diff --git a/doc/manual/bugs.xml b/doc/manual/bugs.xml index 8a56a28c5..9f8faae96 100644 --- a/doc/manual/bugs.xml +++ b/doc/manual/bugs.xml @@ -2,36 +2,25 @@ - - - The man-pages generated from the DocBook documentation are ugly. - - +The man-pages generated from the DocBook documentation +are ugly. - - - Generations properly form a tree. E.g., if after switching to - generation 39, we perform an installation action, a generation - 43 is created which is a descendant of 39, not 42. So a - rollback from 43 ought to go back to 39. This is not - currently implemented; generations form a linear sequence. - - +Generations properly form a tree. E.g., if after +switching to generation 39, we perform an installation action, a +generation 43 is created which is a descendant of 39, not 42. So a +rollback from 43 ought to go back to 39. This is not currently +implemented; generations form a linear sequence. - - - Build management. In principle it is already - possible to do build management using Nix (by writing builders that - perform appropriate build steps), but the Nix expression language is - not yet powerful enough to make this pleasant (?). The language should - be extended with features from the Maak build manager. - Another interesting idea is to write a make - implementation that uses Nix as a back-end to support legacy - build files. - - +Build management. In principle it +is already possible to do build management using Nix (by writing +builders that perform appropriate build steps), but the Nix expression +language is not yet powerful enough to make this pleasant (?). The +language should be extended with features from the Maak build manager. +Another interesting idea is to write a make +implementation that uses Nix as a back-end to support legacy +build files. For security, nix-push manifests should be digitally signed, and nix-pull should @@ -44,6 +33,14 @@ them). nix-env --delete-generations to remove non-current generations older than a certain age. +There should be a flexible way to change the user +environment builder. Currently, you have to replace +prefix/share/nix/corepkgs/buildenv/builder.pl, +which is hard-coded into nix-env. Also, the +default builder should be more powerful. For instance, there should +be some way to specify priorities to resolve +collisions. + diff --git a/doc/manual/conf-file.xml b/doc/manual/conf-file.xml new file mode 100644 index 000000000..13fac5335 --- /dev/null +++ b/doc/manual/conf-file.xml @@ -0,0 +1,82 @@ +Nix configuration file + +A number of persistent settings of Nix are stored in the file +prefix/etc/nix/nix.conf. +This file is a list of name = +value pairs, one per line. +Comments start with a # character. An example +configuration file is shown in . + +Nix expression for GNU Hello + + +gc-keep-outputs = true # Nice for developers +gc-keep-derivations = true # Idem +env-keep-derivations = false + + + +The following variables are currently available: + + + + gc-keep-outputs + + If true, the garbage collector + will keep the outputs of non-garbage derivations. If + false (default), outputs will be deleted unless + they are GC roots themselves (or reachable from other roots). + + In general, outputs must be registered as roots separately. + However, even if the output of a derivation is registered as a + root, the collector will still delete store paths that are used + only at build time (e.g., the C compiler, or source tarballs + downloaded from the network). To prevent it from doing so, set + this option to true. + + + + gc-keep-derivations + + If true (default), the garbage + collector will keep the derivations from which non-garbage store + paths were built. If false, they will be + deleted unless explicitly registered as a root (or reachable from + other roots). + + Keeping derivation around is useful for querying and + traceability (e.g., it allows you to ask with what dependencies or + options a store path was built), so by default this option is on. + Turn it off to safe a bit of disk space (or a lot if + gc-keep-outputs is also turned on). + + + + env-keep-derivations + + If false (default), derivations + are not stored in Nix user environments. That is, the derivation + any build-time-only dependencies may be garbage-collected. + + If true, when you add a Nix derivation to + a user environment, the path of the derivation is stored in the + user environment. Thus, the derivation will not be + garbage-collected until the user environment generation is deleted + (nix-env --delete-generations). To prevent + build-time-only dependencies from being collected, you should also + turn on gc-keep-outputs. + + The difference between this option and + gc-keep-derivations is that this one is + “sticky”: it applies to any user environment created while this + option was enabled, while gc-keep-derivations + only applies at the moment the garbage collector is + run. + + + + + + + + diff --git a/doc/manual/glossary.xml b/doc/manual/glossary.xml index c3dc36fb3..cee6dac25 100644 --- a/doc/manual/glossary.xml +++ b/doc/manual/glossary.xml @@ -79,17 +79,47 @@ -closure +reference + + A store path P is said to have a + reference to a store path Q if the store object + at P contains the path Q + somewhere. This implies than an execution involving + P potentially needs Q to be + present. The references of a store path are + the set of store paths to which it has a reference. + + + + +closure The closure of a store path is the set of store paths that are directly or indirectly “reachable” from that store - path. For instance, if the store object at path - P contains a reference to path - Q, then Q is in the closure of - P. For correct deployment it is necessary to - deploy whole closures, since otherwise at runtime files could be - missing. The command nix-store -qR prints out - closures of store paths. + path; that is, it’s the closure of the path under the references relation. For instance, + if the store object at path P contains a + reference to path Q, then Q is + in the closure of P. For correct deployment it + is necessary to deploy whole closures, since otherwise at runtime + files could be missing. The command nix-store + -qR prints out closures of store paths. + + + + +output path + + A store path produced by a derivation. + + + + +deriver + + The deriver of an output path is the store + derivation that built it. diff --git a/doc/manual/manual.xml b/doc/manual/manual.xml index b71798e80..f1dd0430a 100644 --- a/doc/manual/manual.xml +++ b/doc/manual/manual.xml @@ -35,6 +35,7 @@ Command Reference + nix-env diff --git a/doc/manual/nix-push.xml b/doc/manual/nix-push.xml index c9c4b40c6..f683784ad 100644 --- a/doc/manual/nix-push.xml +++ b/doc/manual/nix-push.xml @@ -10,14 +10,14 @@ nix-push - archives-put-url - archives-get-url - manifest-put-url + archivesPutURL + archivesGetURL + manifestPutURL - archives-dir - manifest-file + archivesDir + manifestFile paths @@ -59,20 +59,20 @@ machines using the nix-pull command. archives. Each store path is uploaded to the remote directory - specified by archives-put-url. HTTP PUT + specified by archivesPutURL. HTTP PUT requests are used to do this. However, before a file x is uploaded to - archives-put-url/x, + archivesPutURL/x, nix-push first determines whether this upload is unnecessary by issuing a HTTP HEAD request on - archives-get-url/x. + archivesGetURL/x. This allows a cache to be shared between many partially overlapping nix-push invocations. (We use two URLs because the upload URL typically refers to a CGI script, while the download URL just refers to a file system directory on the server.) The manifest is uploaded using an HTTP PUT request - to manifest-put-url. The corresponding + to manifestPutURL. The corresponding URL to download the manifest can then be used by nix-pull. diff --git a/doc/manual/nix-store.xml b/doc/manual/nix-store.xml index 0fec253bd..614d30b6a 100644 --- a/doc/manual/nix-store.xml +++ b/doc/manual/nix-store.xml @@ -168,7 +168,8 @@ produced by nix-instantiate: -$ nix-store -r $(nix-instantiate ./foo.nix) +$ nix-store -r $(nix-instantiate ./test.nix) +/nix/store/31axcgrlbfsxzmfff1gyj1bf62hvkby2-aterm-2.3.1 This is essentially what nix-build does. @@ -213,7 +214,9 @@ the Nix store not reachable via file system references from a set of This operation prints on standard output the set - of roots used by the garbage collector. + of roots used by the garbage collector. What constitutes a root + is described in . @@ -247,35 +250,11 @@ the Nix store not reachable via file system references from a set of - - - -Roots - -The roots of the garbage collector are all store paths to which -there are symlinks in the directory -prefix/nix/var/nix/gcroots. -For instance, the following command makes the path -/nix/store/d718ef...-foo a root of the collector: - - -$ ln -s /nix/store/d718ef...-foo /nix/var/nix/gcroots/bar - -That is, after this command, the garbage collector will not remove -/nix/store/d718ef...-foo or any of its -dependencies. - -Subdirectories of -prefix/nix/var/nix/gcroots -are also searched for symlinks. Symlinks to non-store paths are -followed and searched for roots, but TODO. - - - - -Configuration file - -TODO +The behaviour of the collector is influenced by the gc-keep-outputs +and gc-keep-derivations +variables in the Nix configuration file. @@ -309,18 +288,24 @@ $ nix-store --gc - - + - + + + + + + + name + - - + + - args + paths @@ -329,9 +314,14 @@ $ nix-store --gc Description The operation displays various bits of -information about store paths. The queries are described below. At +information about the store paths . The queries are described below. At most one query can be specified. The default query is -. +. + +The paths paths may also be symlinks +from outside of the Nix store, to the Nix store. In that case, the +query is applied to the target of the symlink. + @@ -365,47 +355,21 @@ most one query can be specified. The default query is - / + - Prints out the output paths - of the store expressions indicated by the identifiers - args. In the case of a derivation - expression, these are the paths that will be produced when the - derivation is realised. In the case of a closure expression, - these are the paths that were produced the derivation expression - of which the closure expression is a successor. + Prints out the output paths of the store + derivations paths. These are the paths + that will be produced when the derivation is + built. / - Prints out the requisite paths of the store - expressions indicated by the identifiers - args. The requisite paths of a Nix - expression are the paths that need to be present in the system to - be able to realise the expression. That is, they form the - closure of the expression in the file system - (i.e., no path in the set of requisite paths points to anything - outside the set of requisite paths). - - The notion of requisite paths is very useful when one wants - to distribute store expressions. Since they form a closure, they - are the only paths one needs to distribute to another system to be - able to realise the expression on the other system. - - This query is generally used to implement various kinds of - deployment. A source deployment is obtained - by distributing the requisite paths of a derivation expression. A - binary deployment is obtained by distributing - the requisite paths of a closure expression. A cache - deployment is obtained by distributing the requisite - paths of a derivation expression and specifying the option - . This will include not just - the paths of a source and binary deployment, but also all - expressions and paths of subterms of the source. This is useful - if one wants to realise on the target system a Nix expression that - is similar but not quite the same as the one being distributed, - since any common subterms will be reused. + Prints out the closure of the store path + paths. This query has one option: @@ -420,17 +384,104 @@ most one query can be specified. The default query is + This query can be used to implement various kinds of + deployment. A source deployment is obtained + by distributing the closure of a store derivation. A + binary deployment is obtained by distributing + the closure of an output path. A cache + deployment (combined source/binary deployment, + including binaries of build-time-only dependencies) is obtained by + distributing the closure of a store derivation and specifying the + option . + + + + Prints the set of references of the store paths + paths, that is, their immediate + dependencies. (For all dependencies, use + .) + + + + + + Prints the set of referers of + the store paths paths, that is, the + store paths currently existing in the Nix store that refer to one + of paths. Note that contrary to the + references, the set of referers is not constant; it can change as + store paths are added or removed. + + + + + + Prints the closure of the set of store paths + paths under the referers relation; that + is, all store paths that directly or indirectly refer to one of + paths. These are all the path currently + in the Nix store that are dependent on + paths. + + + + + + Prints the deriver of the store paths + paths. If the path has no deriver + (e.g., if it is a source file), or if the deriver is not known + (e.g., in the case of a binary-only deployment), the string + unknown-deriver is printed. + + - Prints a graph of the closure of the store - expressions identified by args in the - format of the dot tool of AT&T's GraphViz - package. + Prints the references graph of the store paths + paths in the format of the + dot tool of AT&T's Graphviz package. This can + be used to visualise dependency graphs. To obtain a build-time + dependency graph, apply this to a store derivation. To obtain a + runtime dependency graph, apply it to an output + path. + + + + + + Prints the references graph of the store paths + paths as a nested ASCII tree. + References are ordered by descending closure size; this tends to + flatten the tree, making it more readable. The query only + recurses into a store path when it is first encountered; this + prevents a blowup of the tree representation of the + graph. + + + + name + + Prints the value of the attribute + name (i.e., environment variable) of + the store derivations paths. It is an + error for a derivation to not have the specified + attribute. + + + + + + Prints the SHA-256 hash of the contents of the + store path paths. Since the hash is + stored in the Nix database, this is a fast + operation. @@ -439,6 +490,81 @@ most one query can be specified. The default query is +Examples + +Print the closure (runtime dependencies) of the +svn program in the current user environment: + + +$ nix-store -qR $(which svn) +/nix/store/4mbglq5ldqld8sj57273aljwkfvj22mc-subversion-1.1.4 +/nix/store/8lz9yc6zgmc0vlqmn2ipcpkjlmbi51vv-glibc-2.3.4 +... + + + +Print the build-time dependencies of svn: + + +$ nix-store -qR $(nix-store -qd $(which svn)) +/nix/store/02iizgn86m42q905rddvg4ja975bk2i4-grep-2.5.1.tar.bz2.drv +/nix/store/07a2bzxmzwz5hp58nf03pahrv2ygwgs3-gcc-wrapper.sh +/nix/store/0ma7c9wsbaxahwwl04gbw3fcd806ski4-glibc-2.3.4.drv +... lots of other paths ... + +The difference with the previous example is that we ask the closure of +the derivation (), not the closure of the output +path that contains svn. + +Show the build-time dependencies as a tree: + + +$ nix-store -q --tree $(nix-store -qd $(which svn)) +/nix/store/7i5082kfb6yjbqdbiwdhhza0am2xvh6c-subversion-1.1.4.drv ++---/nix/store/d8afh10z72n8l1cr5w42366abiblgn54-builder.sh ++---/nix/store/fmzxmpjx2lh849ph0l36snfj9zdibw67-bash-3.0.drv +| +---/nix/store/570hmhmx3v57605cqg9yfvvyh0nnb8k8-bash +| +---/nix/store/p3srsbd8dx44v2pg6nbnszab5mcwx03v-builder.sh +... + + + +Show all paths that depend on the same OpenSSL library as +svn: + + +$ nix-store -q --referers $(nix-store -q --binding openssl $(nix-store -qd $(which svn))) +/nix/store/23ny9l9wixx21632y2wi4p585qhva1q8-sylpheed-1.0.0 +/nix/store/4mbglq5ldqld8sj57273aljwkfvj22mc-subversion-1.1.4 +/nix/store/dpmvp969yhdqs7lm2r1a3gng7pyq6vy4-subversion-1.1.3 +/nix/store/l51240xqsgg8a7yrbqdx1rfzyv6l26fx-lynx-2.8.5 + + + +Show all paths that directly or indirectly depend on the Glibc +(C library) used by svn: + + +$ nix-store -q --referers-closure $(ldd $(which svn) | grep /libc.so | awk '{print $3}') +/nix/store/034a6h4vpz9kds5r6kzb9lhh81mscw43-libgnomeprintui-2.8.2 +/nix/store/05l3yi0d45prm7a82pcrknxdh6nzmxza-gawk-3.1.4 +... + +Note that ldd is a command that prints out the +dynamic libraries used by an ELF executable. + +Make a picture of the runtime dependency graph of the current +user environment: + + +$ nix-store -q --graph ~/.nix-profile | dot -Tps > graph.ps +$ gv graph.ps + + + + + + diff --git a/doc/manual/opt-common.xml b/doc/manual/opt-common.xml index fb35c1a74..467959f19 100644 --- a/doc/manual/opt-common.xml +++ b/doc/manual/opt-common.xml @@ -35,33 +35,33 @@ 0 - Errors only: only print messages + “Errors only”: only print messages explaining why the Nix invocation failed. 1 - Informational: print + “Informational”: print useful messages about what Nix is doing. This is the default. 2 - Talkative: print more informational + “Talkative”: print more informational messages. 3 - Chatty: print even more + “Chatty”: print even more informational messages. 4 - Debug: print debug + “Debug”: print debug information. 5 - Vomit: print vast amounts of debug + “Vomit”: print vast amounts of debug information. @@ -120,19 +120,21 @@ - Whenever Nix attempts to realise a derivation for which a - closure is already known, but this closure cannot be realised, fall - back on normalising the derivation. + Whenever Nix attempts to build a derivation for which + substitutes are known for each output path, but realising the output + paths through the substitutes fails, fall back on building the + derivation. The most common scenario in which this is useful is when we have registered substitutes in order to perform binary distribution from, say, a network repository. If the repository is down, the realisation of the derivation will fail. When this option is - specified, Nix will build the derivation instead. Thus, binary - installation falls back on a source installation. This option is - not the default since it is generally not desirable for a transient - failure in obtaining the substitutes to lead to a full build from - source (with the related consumption of resources). + specified, Nix will build the derivation instead. Thus, + installation from binaries falls back on nstallation from source. + This option is not the default since it is generally not desirable + for a transient failure in obtaining the substitutes to lead to a + full build from source (with the related consumption of + resources). diff --git a/doc/manual/package-management.xml b/doc/manual/package-management.xml index 0b0e57504..b827ae75b 100644 --- a/doc/manual/package-management.xml +++ b/doc/manual/package-management.xml @@ -369,30 +369,39 @@ $ nix-env --delete-generations 10 11 14 garbage collector as follows: -$ nix-collect-garbage +$ nix-store --gc -You can alo first view what files would be deleted: +If you are feeling uncertain, you can also first view what files would +be deleted: -$ nix-collect-garbage --print-dead +$ nix-store --gc --print-dead Likewise, the option will show the paths that won’t be deleted. + Garbage collector roots -TODO +The roots of the garbage collector are all store paths to which +there are symlinks in the directory +prefix/nix/var/nix/gcroots. +For instance, the following command makes the path +/nix/store/d718ef...-foo a root of the collector: -The garbage collector uses as roots all store expressions -mentioned in all files with extension .gcroot in -the directory -prefix/var/nix/gcroots/, -or in any file or directory symlinked to from that directory. E.g., -by default, -prefix/var/nix/gcroots/ -contains a symlink to -prefix/var/nix/profiles/, -so all generations of all profiles are also roots of the collector. + +$ ln -s /nix/store/d718ef...-foo /nix/var/nix/gcroots/bar + +That is, after this command, the garbage collector will not remove +/nix/store/d718ef...-foo or any of its +dependencies. + +Subdirectories of +prefix/nix/var/nix/gcroots +are also searched for symlinks. Symlinks to non-store paths are +followed and searched for roots, but symlinks to non-store paths +inside the paths reached in that way are not +followed to prevent infinite recursion. diff --git a/doc/manual/quick-start.xml b/doc/manual/quick-start.xml index e12b28da2..a6433ae42 100644 --- a/doc/manual/quick-start.xml +++ b/doc/manual/quick-start.xml @@ -88,8 +88,8 @@ $ nix-channel --update $ nix-env -u '*' The latter command will upgrade each installed component for which -there is a newer version (as determined by comparing -the version numbers). +there is a “newer” version (as determined by comparing the version +numbers). If you're unhappy with the result of a nix-env action (e.g., an upgraded component turned @@ -106,12 +106,12 @@ actually delete them: $ nix-env --delete-generations old -$ nix-collect-garbage +$ nix-store --gc -The first command deletes old generations of your -profile (making rollbacks impossible, but also making the components -in those old generations available for garbage collection), while the -second command actually deletes them. +The first command deletes old “generations” of your profile (making +rollbacks impossible, but also making the components in those old +generations available for garbage collection), while the second +command actually deletes them. diff --git a/doc/manual/troubleshooting.xml b/doc/manual/troubleshooting.xml index cd0bafe3d..4179eb841 100644 --- a/doc/manual/troubleshooting.xml +++ b/doc/manual/troubleshooting.xml @@ -1,31 +1,71 @@ - - Troubleshooting +Troubleshooting - This section provides solutions for some common problems. +This section provides solutions for some common problems. - Berkeley DB: <quote>Cannot allocate memory</quote> - Symptom: Nix operations (in particular the - nix-store operations , - , and - — the latter being called by nix-channel - --update) failing: +Berkeley DB: <quote>Cannot allocate memory</quote> + +Symptom: Nix operations (in particular the +nix-store operations , +, and — +the latter being called by nix-channel --update) +failing: $ nix-store --verify error: Db::del: Cannot allocate memory - Possible solution: make sure that no Nix processes are running, - then do: + + +Possible solution: make sure that no Nix processes are running, +then do: $ cd /nix/var/nix/db $ rm __db.00* - + + + + + +Collisions in <command>nix-env</command> + +Symptom: when installing or upgrading, you get an error message such as + + +$ nix-env -i docbook-xml +... +adding /nix/store/s5hyxgm62gk2...-docbook-xml-4.2 +collission between `/nix/store/s5hyxgm62gk2...-docbook-xml-4.2/xml/dtd/docbook/calstblx.dtd' + and `/nix/store/06h377hr4b33...-docbook-xml-4.3/xml/dtd/docbook/calstblx.dtd' + at /nix/store/...-builder.pl line 62. + + + +The cause is that two installed packages in the user environment +have overlapping filenames (e.g., +xml/dtd/docbook/calstblx.dtd. This usually +happens when you accidentally try to install two versions of the same +package. For instance, in the example above, the Nix Packages +collection contains two versions of docbook-xml, so +nix-env -i will try to install both. The default +user environment builder has no way to way to resolve such conflicts, +so it just gives up. + +Solution: remove one of the offending packages from the user +environment (if already installed) using nix-env +-u, or specify exactly which version should be installed +(e.g., nix-env -i docbook-xml-4.2). + +Alternatively, you can modify the user environment builder +script (in +prefix/share/nix/corepkgs/buildenv/builder.pl) +to implement some conflict resolution policy. E.g., the script could +be modified to rename conflicting file names, or to pick one over the +other. + + + - - - -