diff --git a/NEWS b/NEWS deleted file mode 100644 index 09ec14b4e..000000000 --- a/NEWS +++ /dev/null @@ -1,327 +0,0 @@ -Version 0.9 - -NOTE: this version of Nix uses Berkeley DB 4.3 instead of 4.2. The -database is upgraded automatically, but you should be careful not to -use old versions of Nix that still use Berkeley DB 4.2. In -particular, if you use a Nix installed through Nix, you should run - - nix-store --clear-substitutes - -first. - -* Unpacking of patch sequences is much faster now since we no longer - do redundant unpacking and repacking of intermediate paths. - -* Nix now uses Berkeley DB 4.3. - -* The `derivation' primitive is lazier. Attributes of dependent - derivations can mutually refer to each other (as long as there are - no data dependencies on the `outPath' and `drvPath' attributes - computed by `derivation'). - - For example, the expression `derivation attrs' now evaluates to - (essentially) - - attrs // { - type = "derivation"; - outPath = derivation! attrs; - drvPath = derivation! attrs; - } - - where `derivation!' is a primop that does the actual derivation - instantiation (i.e., it does what `derivation' used to do). The - advantage is that it allows commands such as `nix-env -qa' and - `nix-env -i' to be much faster since they no longer need to - instantiate all derivations, just the `name' attribute. - - Also, it allows derivations to cyclically reference each other, for - example, - - webServer = derivation { - ... - hostName = "svn.cs.uu.nl"; - services = [svnService]; - }; - - svnService = derivation { - ... - hostName = webServer.hostName; - }; - - Previously, this would yield a black hole (infinite recursion). - -* `nix-build' now defaults to using `./default.nix' if no Nix - expression is specified. - -* `nix-instantiate', when applied to a Nix expression that evaluates - to a function, will call the function automatically if all its - arguments have defaults. - -* Nix now uses libtool to build dynamic libraries. This reduces the - size of executables. - -* A new list concatenation operator `++'. For example, `[1 2 3] ++ [4 - 5 6]' evaluates to `[1 2 3 4 5 6]'. - -* Some currently undocumented primops to support low-level build - management using Nix (i.e., using Nix as a Make replacement). See - the commit message for r3578 and r3580. - -* Various bug fixes and performance improvements. - - -Version 0.8 (April 11, 2005) - -NOTE: the hashing scheme in Nix 0.8 changed (as detailed below). As a -result, `nix-pull' manifests and channels built for Nix 0.7 and below -will now work anymore. However, the Nix expression language has not -changed, so you can still build from source. Also, existing user -environments continue to work. Nix 0.8 will automatically upgrade the -database schema of previous installations when it is first run. - -If you get the error message - - you have an old-style manifest `/nix/var/nix/manifests/[...]'; - please delete it - -you should delete previously downloaded manifests: - - $ rm /nix/var/nix/manifests/* - -If `nix-channel' gives the error message - - manifest `http://catamaran.labs.cs.uu.nl/dist/nix/channels/[channel]/MANIFEST' - is too old (i.e., for Nix <= 0.7) - -then you should unsubscribe from the offending channel (`nix-channel ---remove URL'; leave out `/MANIFEST'), and subscribe to the same URL, -with `channels' replaced by `channels-v3' (e.g., -http://catamaran.labs.cs.uu.nl/dist/nix/channels-v3/nixpkgs-unstable). - -Nix 0.8 has the following improvements: - -* The cryptographic hashes used in store paths are now 160 bits long, - but encoded in base-32 so that they are still only 32 characters - long (e.g., /nix/store/csw87wag8bqlqk7ipllbwypb14xainap-atk-1.9.0). - (This is actually a 160 bit truncation of a SHA-256 hash.) - -* Big cleanups and simplifications of the basic store semantics. The - notion of "closure store expressions" is gone (and so is the notion - of "successors"); the file system references of a store path are now - just stored in the database. - - For instance, given any store path, you can query its closure: - - $ nix-store -qR $(which firefox) - ... lots of paths ... - - Also, Nix now remembers for each store path the derivation that - built it (the "deriver"): - - $ nix-store -qR $(which firefox) - /nix/store/4b0jx7vq80l9aqcnkszxhymsf1ffa5jd-firefox-1.0.1.drv - - So to see the build-time dependencies, you can do - - $ nix-store -qR $(nix-store -qd $(which firefox)) - - or, in a nicer format: - - $ nix-store -q --tree $(nix-store -qd $(which firefox)) - - File system references are also stored in reverse. For instance, - you can query all paths that directly or indirectly use a certain - Glibc: - - $ nix-store -q --referers-closure \ - /nix/store/8lz9yc6zgmc0vlqmn2ipcpkjlmbi51vv-glibc-2.3.4 - -* The concept of fixed-output derivations has been formalised. - Previously, functions such as `fetchurl' in Nixpkgs used a hack - (namely, explicitly specifying a store path hash) to prevent changes - to, say, the URL of the file from propagating upwards through the - dependency graph, causing rebuilds of everything. This can now be - done cleanly by specifying the `outputHash' and `outputHashAlgo' - attributes. Nix itself checks that the content of the output has - the specified hash. (This is important for maintaining certain - invariants necessary for future work on secure shared stores.) - -* One-click installation :-) It is now possible to install any - top-level component in Nixpkgs directly, through the web - see, - e.g., http://catamaran.labs.cs.uu.nl/dist/nixpkgs-0.8/. All you - have to do is associate `/nix/bin/nix-install-package' with the MIME - type `application/nix-package' (or the extension `.nixpkg'), and - clicking on a package link will cause it to be installed, with all - appropriate dependencies. If you just want to install some specific - application, this is easier than subscribing to a channel. - -* `nix-store -r PATHS' now builds all the derivations PATHS in - parallel. Previously it did them sequentially (though exploiting - possible parallelism between subderivations). This is nice for - build farms. - -* `nix-channel' has new operations `--list' and `--remove'. - -* New ways of installing components into user environments: - - - Copy from another user environment: - - $ nix-env -i --from-profile .../other-profile firefox - - - Install a store derivation directly (bypassing the Nix expression - language entirely): - - $ nix-env -i /nix/store/z58v41v21xd3...-aterm-2.3.1.drv - - (This is used to implement `nix-install-package', which is - therefore immune to evolution in the Nix expression language.) - - - Install an already built store path directly: - - $ nix-env -i /nix/store/hsyj5pbn0d9i...-aterm-2.3.1 - - - Install the result of a Nix expression specified as a command-line - argument: - - $ nix-env -f .../i686-linux.nix -i -E 'x: x.firefoxWrapper' - - The difference with the normal installation mode is that `-E' does - not use the `name' attributes of derivations. Therefore, this can - be used to disambiguate multiple derivations with the same name. - -* A hash of the contents of a store path is now stored in the database - after a succesful build. This allows you to check whether store - paths have been tampered with: `nix-store --verify --check-contents'. - -* Implemented a concurrent garbage collector. It is now always safe - to run the garbage collector, even if other Nix operations are - happening simultaneously. - - However, there can still be GC races if you use `nix-instantiate' - and `nix-store -r' directly to build things. To prevent races, use - the `--add-root' flag of those commands. - -* The garbage collector now finally deletes paths in the right order - (i.e., topologically sorted under the `references' relation), thus - making it safe to interrupt the collector without risking a store - that violates the closure invariant. - -* Likewise, the substitute mechanism now downloads files in the right - order, thus preserving the closure invariant at all times. - -* The result of `nix-build' is now registered as a root of the garbage - collector. If the `./result' link is deleted, the GC root - disappears automatically. - -* The behaviour of the garbage collector can be changed globally by - setting options in `/nix/etc/nix/nix.conf'. - - - `gc-keep-derivations' specifies whether deriver links should be - followed when searching for live paths. - - - `gc-keep-outputs' specifies whether outputs of derivations should - be followed when searching for live paths. - - - `env-keep-derivations' specifies whether user environments should - store the paths of derivations when they are added (thus keeping - the derivations alive). - -* New `nix-env' query flags `--drv-path' and `--out-path'. - -* `fetchurl' allows SHA-1 and SHA-256 in addition to MD5. Just - specify the attribute `sha1' or `sha256' instead of `md5'. - -* Manual updates. - - -Version 0.7 (January 12, 2005) - -* Binary patching. When upgrading components using pre-built binaries - (through nix-pull / nix-channel), Nix can automatically download and - apply binary patches to already installed components instead of full - downloads. Patching is "smart": if there is a *sequence* of patches - to an installed component, Nix will use it. Patches are currently - generated automatically between Nixpkgs (pre-)releases. - -* Simplifications to the substitute mechanism. - -* Nix-pull now stores downloaded manifests in /nix/var/nix/manifests. - -* Metadata on files in the Nix store is canonicalised after builds: - the last-modified timestamp is set to 0 (00:00:00 1/1/1970), the - mode is set to 0444 or 0555 (readable and possibly executable by - all; setuid/setgid bits are dropped), and the group is set to the - default. This ensures that the result of a build and an - installation through a substitute is the same; and that timestamp - dependencies are revealed. - - -Version 0.6 (November 14, 2004) - -Major changes include the following: - -* Rewrite of the normalisation engine. - - * Multiple builds can now be performed in parallel (option `-j'). - - * Distributed builds. Nix can now call a shell script to forward - builds to Nix installations on remote machines, which may or may - not be of the same platform type. - - * Option `--fallback' allows recovery from broken substitutes. - - * Option `--keep-going' causes building of other (unaffected) - derivations to continue if one failed. - -* Improvements to the garbage collector (i.e., it should actually work - now). - -* Setuid Nix installations allow a Nix store to be shared among - multiple users. - -* Substitute registration is much faster now. - -* A utility `nix-build' to build a Nix expression and create a symlink - to the result int the current directory; useful for testing Nix - derivations. - -* Manual updates. - -* `nix-env' changes: - - * Derivations for other platforms are filtered out (which can be - overriden using `--system-filter'). - - * `--install' by default now uninstall previous derivations with the - same name. - - * `--upgrade' allows upgrading to a specific version. - - * New operation `--delete-generations' to remove profile - generations (necessary for effective garbage collection). - - * Nicer output (sorted, columnised). - -* More sensible verbosity levels all around (builder output is now - shown always, unless `-Q' is given). - -* Nix expression language changes: - - * New language construct: `with E1; E2' brings all attributes - defined in the attribute set E1 in scope in E2. - - * Added a `map' function. - - * Various new operators (e.g., string concatenation). - -* Expression evaluation is much faster. - -* An Emacs mode for editing Nix expressions (with syntax highlighting - and indentation) has been added. - -* Many bug fixes. - - -Version 0.5 and earlier - -Please refer to the Subversion commit log messages. diff --git a/doc/manual/Makefile.am b/doc/manual/Makefile.am index 81367fd14..2cc376612 100644 --- a/doc/manual/Makefile.am +++ b/doc/manual/Makefile.am @@ -37,7 +37,24 @@ manual.html: $(MANUAL_SRCS) manual.is-valid images $(XSLTPROC) --nonet --xinclude --output manual.html \ $(docbookxsl)/html/docbook.xsl manual.xml -all-local: manual.html + +NEWS_OPTS = \ + --stringparam generate.toc "article nop" \ + --stringparam section.autolabel.max.depth 0 \ + --stringparam header.rule 0 + +NEWS.html: release-notes.xml + $(XSLTPROC) --nonet --xinclude --output $@ $(NEWS_OPTS) \ + $(docbookxsl)/html/docbook.xsl release-notes.xml + +NEWS.txt: release-notes.xml + $(XSLTPROC) --nonet --xinclude quote-literals.xsl release-notes.xml | \ + $(XSLTPROC) --nonet --output $@.tmp.html $(NEWS_OPTS) \ + $(docbookxsl)/html/docbook.xsl - + w3m -dump $@.tmp.html > $@ + + +all-local: manual.html NEWS.html NEWS.txt install-data-local: manual.html $(INSTALL) -d $(DESTDIR)$(datadir)/nix/manual diff --git a/doc/manual/introduction.xml b/doc/manual/introduction.xml index 9f94b2d53..fb6e79fcb 100644 --- a/doc/manual/introduction.xml +++ b/doc/manual/introduction.xml @@ -130,8 +130,8 @@ collection. It also discusses some advanced topics, such as setting up a Nix-based build farm, and doing service deployment using Nix. -Some background information on Nix can be found in three -papers. The ICSE 2004 paper Some background information on Nix can be found in a +number of papers. The ICSE 2004 paper Imposing a Memory Management Discipline on Software Deployment discusses the hashing mechanism used to @@ -144,7 +144,10 @@ Deployment gives a more general discussion of Nix from a system-administration perspective. The CBSE 2005 paper Efficient Upgrading in a Purely Functional Component Deployment Model - is about transparent patch deployment in -Nix. + is about transparent patch deployment in Nix. +Finally, the SCM-12 paper +Service Configuration Management shows how services (e.g., web +servers) can be deployed and managed through Nix. diff --git a/doc/manual/quote-literals.xsl b/doc/manual/quote-literals.xsl new file mode 100644 index 000000000..acc7c8a0b --- /dev/null +++ b/doc/manual/quote-literals.xsl @@ -0,0 +1,46 @@ + + + + + + + `' + + + + + + + + + + + +
+ + + +
+ + + + + + + + + + + + + + + diff --git a/doc/manual/release-notes.xml b/doc/manual/release-notes.xml new file mode 100644 index 000000000..669fcc26f --- /dev/null +++ b/doc/manual/release-notes.xml @@ -0,0 +1,505 @@ + + + +
Nix Release Notes + + +
Release 0.9 + +NOTE: this version of Nix uses Berkeley DB 4.3 instead of 4.2. +The database is upgraded automatically, but you should be careful not +to use old versions of Nix that still use Berkeley DB 4.2. In +particular, if you use a Nix installed through Nix, you should run + + +$ nix-store --clear-substitutes + +first. + + + + + Unpacking of patch sequences is much faster now + since we no longer do redundant unpacking and repacking of + intermediate paths. + + Nix now uses Berkeley DB 4.3. + + The derivation primitive is + lazier. Attributes of dependent derivations can mutually refer to + each other (as long as there are no data dependencies on the + outPath and drvPath attributes + computed by derivation). + + For example, the expression derivation + attrs now evaluates to (essentially) + + +attrs // { + type = "derivation"; + outPath = derivation! attrs; + drvPath = derivation! attrs; +} + + where derivation! is a primop that does the + actual derivation instantiation (i.e., it does what + derivation used to do). The advantage is that + it allows commands such as nix-env -qa and + nix-env -i to be much faster since they no longer + need to instantiate all derivations, just the + name attribute. + + Also, it allows derivations to cyclically reference each + other, for example, + + +webServer = derivation { + ... + hostName = "svn.cs.uu.nl"; + services = [svnService]; +}; + +svnService = derivation { + ... + hostName = webServer.hostName; +}; + + Previously, this would yield a black hole (infinite recursion). + + + + nix-build now defaults to using + ./default.nix if no Nix expression is + specified. + + nix-instantiate, when applied to + a Nix expression that evaluates to a function, will call the + function automatically if all its arguments have + defaults. + + Nix now uses libtool to build dynamic libraries. + This reduces the size of executables. + + A new list concatenation operator + ++. For example, [1 2 3] ++ [4 5 + 6] evaluates to [1 2 3 4 5 + 6]. + + Some currently undocumented primops to support + low-level build management using Nix (i.e., using Nix as a Make + replacement). See the commit messages for r3578 + and r3580. + + Various bug fixes and performance + improvements. + + + +
+ + + +
Release 0.8 (April 11, 2005) + +NOTE: the hashing scheme in Nix 0.8 changed (as detailed below). +As a result, nix-pull manifests and channels built +for Nix 0.7 and below will now work anymore. However, the Nix +expression language has not changed, so you can still build from +source. Also, existing user environments continue to work. Nix 0.8 +will automatically upgrade the database schema of previous +installations when it is first run. + +If you get the error message + + +you have an old-style manifest `/nix/var/nix/manifests/[...]'; please +delete it + +you should delete previously downloaded manifests: + + +$ rm /nix/var/nix/manifests/* + +If nix-channel gives the error message + + +manifest `http://catamaran.labs.cs.uu.nl/dist/nix/channels/[channel]/MANIFEST' +is too old (i.e., for Nix <= 0.7) + +then you should unsubscribe from the offending channel +(nix-channel --remove +URL; leave out +/MANIFEST), and subscribe to the same URL, with +channels replaced by channels-v3 +(e.g., +http://catamaran.labs.cs.uu.nl/dist/nix/channels-v3/nixpkgs-unstable). + +Nix 0.8 has the following improvements: + + + + The cryptographic hashes used in store paths are now + 160 bits long, but encoded in base-32 so that they are still only 32 + characters long (e.g., + /nix/store/csw87wag8bqlqk7ipllbwypb14xainap-atk-1.9.0). (This is + actually a 160 bit truncation of a SHA-256 hash.) + + Big cleanups and simplifications of the basic store + semantics. The notion of "closure store expressions" is gone (and + so is the notion of "successors"); the file system references of a + store path are now just stored in the database. + + For instance, given any store path, you can query its closure: + + +$ nix-store -qR $(which firefox) +... lots of paths ... + + Also, Nix now remembers for each store path the derivation that + built it (the "deriver"): + + +$ nix-store -qR $(which firefox) +/nix/store/4b0jx7vq80l9aqcnkszxhymsf1ffa5jd-firefox-1.0.1.drv + + So to see the build-time dependencies, you can do + + +$ nix-store -qR $(nix-store -qd $(which firefox)) + + or, in a nicer format: + + +$ nix-store -q --tree $(nix-store -qd $(which firefox)) + + + + File system references are also stored in reverse. For + instance, you can query all paths that directly or indirectly use a + certain Glibc: + + +$ nix-store -q --referers-closure \ + /nix/store/8lz9yc6zgmc0vlqmn2ipcpkjlmbi51vv-glibc-2.3.4 + + + + + + The concept of fixed-output derivations has been + formalised. Previously, functions such as + fetchurl in Nixpkgs used a hack (namely, + explicitly specifying a store path hash) to prevent changes to, say, + the URL of the file from propagating upwards through the dependency + graph, causing rebuilds of everything. This can now be done cleanly + by specifying the outputHash and + outputHashAlgo attributes. Nix itself checks + that the content of the output has the specified hash. (This is + important for maintaining certain invariants necessary for future + work on secure shared stores.) + + One-click installation :-) It is now possible to + install any top-level component in Nixpkgs directly, through the web + - see, e.g., http://catamaran.labs.cs.uu.nl/dist/nixpkgs-0.8/. All + you have to do is associate + /nix/bin/nix-install-package with the MIME type + application/nix-package (or the extension + .nixpkg), and clicking on a package link will + cause it to be installed, with all appropriate dependencies. If you + just want to install some specific application, this is easier than + subscribing to a channel. + + nix-store -r + PATHS now builds all the + derivations PATHS in parallel. Previously it did them sequentially + (though exploiting possible parallelism between subderivations). + This is nice for build farms. + + nix-channel has new operations + and + . + + New ways of installing components into user + environments: + + + + Copy from another user environment: + + +$ nix-env -i --from-profile .../other-profile firefox + + + + Install a store derivation directly (bypassing the + Nix expression language entirely): + + +$ nix-env -i /nix/store/z58v41v21xd3...-aterm-2.3.1.drv + + (This is used to implement nix-install-package, + which is therefore immune to evolution in the Nix expression + language.) + + Install an already built store path directly: + + +$ nix-env -i /nix/store/hsyj5pbn0d9i...-aterm-2.3.1 + + + + Install the result of a Nix expression specified + as a command-line argument: + + +$ nix-env -f .../i686-linux.nix -i -E 'x: x.firefoxWrapper' + + The difference with the normal installation mode is that + does not use the name + attributes of derivations. Therefore, this can be used to + disambiguate multiple derivations with the same + name. + + + + A hash of the contents of a store path is now stored + in the database after a succesful build. This allows you to check + whether store paths have been tampered with: nix-store + --verify --check-contents. + + + + Implemented a concurrent garbage collector. It is now + always safe to run the garbage collector, even if other Nix + operations are happening simultaneously. + + However, there can still be GC races if you use + nix-instantiate and nix-store + --realise directly to build things. To prevent races, + use the flag of those commands. + + + + The garbage collector now finally deletes paths in + the right order (i.e., topologically sorted under the + references relation), thus making it safe to + interrupt the collector without risking a store that violates the + closure invariant. + + Likewise, the substitute mechanism now downloads + files in the right order, thus preserving the closure invariant at + all times. + + The result of nix-build is now + registered as a root of the garbage collector. If the + ./result link is deleted, the GC root + disappears automatically. + + + + The behaviour of the garbage collector can be changed + globally by setting options in + /nix/etc/nix/nix.conf. + + + + gc-keep-derivations specifies + whether deriver links should be followed when searching for live + paths. + + gc-keep-outputs specifies + whether outputs of derivations should be followed when searching + for live paths. + + env-keep-derivations + specifies whether user environments should store the paths of + derivations when they are added (thus keeping the derivations + alive). + + + + + + New nix-env query flags + and + . + + fetchurl allows SHA-1 and SHA-256 + in addition to MD5. Just specify the attribute + sha1 or sha256 instead of + md5. + + Manual updates. + + + + + +
+ + + +
Release 0.7 (January 12, 2005) + + + + Binary patching. When upgrading components using + pre-built binaries (through nix-pull / nix-channel), Nix can + automatically download and apply binary patches to already installed + components instead of full downloads. Patching is "smart": if there + is a *sequence* of patches to an installed component, Nix will use + it. Patches are currently generated automatically between Nixpkgs + (pre-)releases. + + Simplifications to the substitute + mechanism. + + Nix-pull now stores downloaded manifests in + /nix/var/nix/manifests. + + Metadata on files in the Nix store is canonicalised + after builds: the last-modified timestamp is set to 0 (00:00:00 + 1/1/1970), the mode is set to 0444 or 0555 (readable and possibly + executable by all; setuid/setgid bits are dropped), and the group is + set to the default. This ensures that the result of a build and an + installation through a substitute is the same; and that timestamp + dependencies are revealed. + + + +
+ + + +
Release 0.6 (November 14, 2004) + + + + + Rewrite of the normalisation engine. + + + + Multiple builds can now be performed in parallel + (option ). + + Distributed builds. Nix can now call a shell + script to forward builds to Nix installations on remote + machines, which may or may not be of the same platform + type. + + Option allows + recovery from broken substitutes. + + Option causes + building of other (unaffected) derivations to continue if one + failed. + + + + + + + + Improvements to the garbage collector (i.e., it + should actually work now). + + Setuid Nix installations allow a Nix store to be + shared among multiple users. + + Substitute registration is much faster + now. + + A utility nix-build to build a + Nix expression and create a symlink to the result int the current + directory; useful for testing Nix derivations. + + Manual updates. + + + + nix-env changes: + + + + Derivations for other platforms are filtered out + (which can be overriden using + ). + + by default now + uninstall previous derivations with the same + name. + + allows upgrading to a + specific version. + + New operation + to remove profile + generations (necessary for effective garbage + collection). + + Nicer output (sorted, + columnised). + + + + + + + + More sensible verbosity levels all around (builder + output is now shown always, unless is + given). + + + + Nix expression language changes: + + + + New language construct: with + E1; + E2 brings all attributes + defined in the attribute set E1 in + scope in E2. + + Added a map + function. + + Various new operators (e.g., string + concatenation). + + + + + + + + Expression evaluation is much + faster. + + An Emacs mode for editing Nix expressions (with + syntax highlighting and indentation) has been + added. + + Many bug fixes. + + + +
+ + + +
Release 0.5 and earlier + +Please refer to the Subversion commit log messages. + +
+ + + +
diff --git a/doc/manual/writing-nix-expressions.xml b/doc/manual/writing-nix-expressions.xml index d9c78dbec..222092f63 100644 --- a/doc/manual/writing-nix-expressions.xml +++ b/doc/manual/writing-nix-expressions.xml @@ -1046,6 +1046,11 @@ weakest binding). contains an attribute named id. + + e1 ++ e2 + right + List concatenation. + e1 + e2 left