diff --git a/doc/manual/build-farm.xml b/doc/manual/build-farm.xml index 4745d205c..c86e19cdf 100644 --- a/doc/manual/build-farm.xml +++ b/doc/manual/build-farm.xml @@ -65,7 +65,10 @@ will call whenever it wants to build a derivation. The build hook will perform it in the usual way if possible, or it can accept it, in which case it is responsible for somehow getting the inputs of the build to another machine, doing the build there, and getting the -results back. +results back. The details of the build hook protocol are described in +the documentation of the NIX_BUILD_HOOK +variable. Remote machine configuration: <filename>remote-systems.conf</filename> diff --git a/doc/manual/env-common.xml b/doc/manual/env-common.xml index cf84dd054..ec468eba1 100644 --- a/doc/manual/env-common.xml +++ b/doc/manual/env-common.xml @@ -118,6 +118,158 @@ $ mount -o bind /mnt/otherdisk/nix /nix +NIX_BUILD_HOOK + + + + Specifies the location of the build hook, + which is a program (typically some script) that Nix will call + whenever it wants to build a derivation. This is used to implement + distributed builds (see ). The protocol by which the calling Nix process and the build + hook communicate is as follows. + + The build hook is called with the following command-line + arguments: + + + + A boolean value 0 or + 1 specifying whether Nix can locally execute + more builds, as per the option. + The purpose of this argument is to allow the hook to not have to + maintain bookkeeping for the local machine. + + The Nix platform identifier for the local machine + (e.g., i686-linux). + + The Nix platform identifier for the derivation, + i.e., its system + attribute. + + The store path of the derivation. + + + + + + On the basis of this information, and whatever persistent + state the build hook keeps about other machines and their current + load, it has to decide what to do with the build. It should print + out on file descriptor 3 one of the following responses (terminated + by a newline, "\n"): + + + + decline + + The build hook is not willing or able to perform + the build; the calling Nix process should do the build itself, + if possible. + + + + postpone + + The build hook cannot perform the build now, but + can do so in the future (e.g., because all available build slots + on remote machines are in use). The calling Nix process should + postpone this build until at least one currently running build + has terminated. + + + + accept + + The build hook has accepted the + build. + + + + + + + + If the build hook accepts the build, it is possible that it is + no longer necessary to do the build because some other process has + performed the build in the meantime. To prevent races, the hook + must read from file descriptor 4 a single line that tells it whether + to continue: + + + + cancel + + The build has already been done, so the hook + should exit. + + + + okay + + The hook should proceed with the build. At this + point, the calling Nix process has acquired locks on the output + path, so no other Nix process will perform the + build. + + + + + + + + If the hook has been told to proceed, Nix will store in the + hook’s current directory a number of text files that contain + information about the derivation: + + + + inputs + + The set of store paths that are inputs to the + build process (one per line). These have to be copied + to the remote machine (in addition to the + store derivation itself). + + + + outputs + + The set of store paths that are outputs of the + derivation (one per line). These have to be copied + from the remote machine if the build + succeeds. + + + + references + + The reference graph of the inputs, in the format + accepted by the command nix-store + --register-validity. It is necessary to run + this command on the remote machine after copying the inputs to + inform Nix on the remote machine that the inputs are valid + paths. + + + + + + + + The hook should copy the inputs to the remote machine, + register the validity of the inputs, perform the remote build, and + copy the outputs back to the local machine. An exit code other than + 0 indicates that the hook has failed. + + + + + + + diff --git a/doc/manual/nix-store.xml b/doc/manual/nix-store.xml index ac803ce5f..d9ef17e26 100644 --- a/doc/manual/nix-store.xml +++ b/doc/manual/nix-store.xml @@ -262,321 +262,225 @@ $ nix-store --gc - - Operation <option>--query</option> +Operation <option>--query</option> - - Synopsis - - nix-store - - - - - - - - - - - - - - - - - args - - +Synopsis - - Description + + nix-store + + + + + + + + + + + + + + + + + args + + + + + +Description - - The operation displays various bits - of information about store expressions or store paths. The - queries are described below. At most one query can be - specified. The default query is . - +The operation displays various bits of +information about store paths. The queries are described below. At +most one query can be specified. The default query is +. - + - - Common query options +Common query options - + - - / - - - For those queries that take a Nix store expression, this - option causes those expressions to be normalised first. - - - + / + + For each argument to the query that is a store + derivation, apply the query to the output path of the derivation + instead. + + - - / - - - For those queries that take a Nix store expression, this - option causes those expressions to be realised first. - This is just a short-cut for the common idiom - - -nix-store --realise /nix/store/bla.store -x=`nix-store --query --normalise /nix/store/bla.store` -(do something with the path $x - - which using this flag can be written as - - -x=`nix-store --query --normalise --force-realise /nix/store/bla.store` -(do something with the path $x - - + / + + Realise each argument to the query first (see + nix-store + --realise). - + + + - + - - Queries +Queries - + - - / - - - Prints out the output paths of the - store expressions indicated by the identifiers - args. In the case of a - derivation expression, these are the paths that will be - produced when the derivation is realised. In the case - of a closure expression, these are the paths that were - produced the derivation expression of which the closure - expression is a successor. - - - + / - - / - - - Prints out the requisite paths of the store expressions - indicated by the identifiers - args. The requisite paths of - a Nix expression are the paths that need to be present - in the system to be able to realise the expression. - That is, they form the closure of - the expression in the file system (i.e., no path in the - set of requisite paths points to anything outside the - set of requisite paths). - + Prints out the output paths + of the store expressions indicated by the identifiers + args. In the case of a derivation + expression, these are the paths that will be produced when the + derivation is realised. In the case of a closure expression, + these are the paths that were produced the derivation expression + of which the closure expression is a successor. + + - - The notion of requisite paths is very useful when one - wants to distribute store expressions. Since they form a - closure, they are the only paths one needs to distribute - to another system to be able to realise the expression - on the other system. - + / - - This query is generally used to implement various kinds - of deployment. A source deployment - is obtained by distributing the requisite paths of a - derivation expression. A binary - deployment is obtained by distributing the - requisite paths of a closure expression. A - cache deployment is obtained by - distributing the requisite paths of a derivation - expression and specifying the option - . This will - include not just the paths of a source and binary - deployment, but also all expressions and paths of - subterms of the source. This is useful if one wants to - realise on the target system a Nix expression that is - similar but not quite the same as the one being - distributed, since any common subterms will be reused. - + Prints out the requisite paths of the store + expressions indicated by the identifiers + args. The requisite paths of a Nix + expression are the paths that need to be present in the system to + be able to realise the expression. That is, they form the + closure of the expression in the file system + (i.e., no path in the set of requisite paths points to anything + outside the set of requisite paths). - - This query has a number of options: - + The notion of requisite paths is very useful when one wants + to distribute store expressions. Since they form a closure, they + are the only paths one needs to distribute to another system to be + able to realise the expression on the other system. - + This query is generally used to implement various kinds of + deployment. A source deployment is obtained + by distributing the requisite paths of a derivation expression. A + binary deployment is obtained by distributing + the requisite paths of a closure expression. A cache + deployment is obtained by distributing the requisite + paths of a derivation expression and specifying the option + . This will include not just + the paths of a source and binary deployment, but also all + expressions and paths of subterms of the source. This is useful + if one wants to realise on the target system a Nix expression that + is similar but not quite the same as the one being distributed, + since any common subterms will be reused. - - - - - Excludes the paths of store expressions. This - causes the closure property to be lost, that is, - the resulting set of paths is not enough to ensure - realisibility. - - - + This query has one option: - - - - - Also include the requisites of successors (normal forms). - Only the requisites of known - successors are included, i.e., the normal forms of - derivation expressions that have never been normalised will - not be included. - + - - Note that not just the successor of a derivation expression - will be included, but also the successors of all input - expressions of that derivation expression. I.e., all - normal forms of subterms involved in the normalisation of - the top-level term are included. - - - + + + Also include the output path of store + derivations, and their closures. + + - + - - + - - - - - For each store expression stored at paths - args, prints its - predecessors. A derivation - expression p is a predecessor of a - store expression q iff - q is a successor of - p. - - - + - - - - - Prints a graph of the closure of the store expressions - identified by args in the - format of the dot tool of AT&T's - GraphViz package. - - - - + + + Prints a graph of the closure of the store + expressions identified by args in the + format of the dot tool of AT&T's GraphViz + package. - + - + + + + + + - + - - Operation <option>--successor</option> +Operation <option>--register-validity</option> - - Synopsis - - nix-store - - srcpath sucpath - - +Synopsis - - Description + + nix-store + + + + +Description - - The operation registers that the - closure expression in sucpath is a - successor of the derivation expression in - srcpath. This is used to implement - binary deployment. - +TODO - + - + - + - - Operation <option>--substitute</option> +Operation <option>--substitute</option> - - Synopsis - - nix-store - - srcpath subpath - - +Synopsis - - Description + + nix-store + + srcpath subpath + + + +Description - - The operation registers that the - store path srcpath can be built by - realising the derivation expression in - subpath. This is used to implement - binary deployment. - +The operation registers that the +store path srcpath can be built by +realising the derivation expression in +subpath. This is used to implement binary +deployment. - + - + - + - - Operation <option>--verify</option> +Operation <option>--verify</option> - - Synopsis - - nix-store - - - + + Synopsis + + nix-store + + + - - Description +Description - - The operation verifies the internal - consistency of the Nix database, and the consistency between - the Nix database and the Nix store. Any inconsistencies - encountered are automatically repaired. Inconsistencies are - generally the result of the Nix store or database being - modified by non-Nix tools, or of bugs in Nix itself. - +The operation verifies the internal +consistency of the Nix database, and the consistency between the Nix +database and the Nix store. Any inconsistencies encountered are +automatically repaired. Inconsistencies are generally the result of +the Nix store or database being modified by non-Nix tools, or of bugs +in Nix itself. - + - + diff --git a/doc/manual/opt-common.xml b/doc/manual/opt-common.xml index a8f252128..fb35c1a74 100644 --- a/doc/manual/opt-common.xml +++ b/doc/manual/opt-common.xml @@ -84,7 +84,7 @@ - / + / Sets the maximum number of build jobs that Nix will perform in parallel to the specified number. The default is 1. A diff --git a/doc/manual/writing-nix-expressions.xml b/doc/manual/writing-nix-expressions.xml index 50cbc4249..1b177c7da 100644 --- a/doc/manual/writing-nix-expressions.xml +++ b/doc/manual/writing-nix-expressions.xml @@ -1112,7 +1112,7 @@ set, the attributes of which specify the inputs of the build. - There must be an attribute named + There must be an attribute named system whose value must be a string specifying a Nix platform identifier, such as "i686-linux" or "powerpc-darwin"To figure out