Commit graph

693 commits

Author SHA1 Message Date
Eelco Dolstra ef2d4a2da9 * Print a more useful stack trace when an error occurs deep in the
derivation dependency graph.
2006-03-24 14:02:44 +00:00
Eelco Dolstra b69e469328 * In `nix-env', look for derivations inside attribute sets that have
the `recurseForDerivations' attribute set to `true'.
2006-03-23 16:43:07 +00:00
Eelco Dolstra 49ce8b57dd * Hm. 2006-03-23 16:37:49 +00:00
Eelco Dolstra fdea084c36 * Allow `make check' to work in directories that have symlink
components.
2006-03-10 22:27:26 +00:00
Eelco Dolstra 37d1b1cafd * `nix-env -qa --description' shows human-readable descriptions of
packages (provided that they have a `meta.description' attribute).
  E.g.,

  $ ./src/nix-env/nix-env -qa --description gcc
  gcc-4.0.2   GNU Compiler Collection, 4.0.x (cross-compiler for sparc-linux)
  gcc-4.0.2   GNU Compiler Collection, 4.0.x (cross-compiler for mips-linux)
  gcc-4.0.2   GNU Compiler Collection, 4.0.x (cross-compiler for arm-linux)
  gcc-4.0.2   GNU Compiler Collection, 4.0.x
2006-03-10 16:20:42 +00:00
Eelco Dolstra 2b3b6c9b34 * In theory, this should reduce the number of ATermMap
re-allocations.
2006-03-10 16:14:13 +00:00
Eelco Dolstra 4ada6db1fc * `nix-env -q' now accepts arguments that allow specific derivations
to be queried, e.g., `nix-env -qa firefox'.  This does require the
  argument '*' to be passed if one wants information about all
  derivations, so the old `nix-env -qa' now is `nix-env -qa "*"'.
2006-03-10 10:24:46 +00:00
Eelco Dolstra 18c321308d * Ugh, printHash() was very inefficient because it used
ostringstreams.  Around 11% of execution time was spent here (now
  it's 0.5%).
2006-03-09 17:07:25 +00:00
Eelco Dolstra b90c00e63f * Regression: semantics of the result of getDerivation() changed. 2006-03-09 15:10:01 +00:00
Eelco Dolstra 922697c8b2 * Big speedup (factor > 2.5) in all nix-env operations that do actual
instantiation, e.g. "nix-env -i" and "nix-env -qas" (but not
  "nix-env -qa").  It turns out that many redundant calls to
  addToStore(path) were made, which reads and hashes the entire path.
  For instance, the bash bootstrap binary in Nixpkgs would be read and
  hashed many times.  As a result nix-env would spend around 92% of
  its time in the function sha256_block (according to callgrind).
  Some simple memoization fixes this.
2006-03-09 15:09:18 +00:00
Eelco Dolstra 6dca5c9099 * When obtaining derivations from Nix expressions, ignore all
expressions that cause an assertion failure (like `assert system ==
  "i686-linux"').  This allows all-packages.nix in Nixpkgs to be used
  on all platforms, even if some Nix expressions don't work on all
  platforms.

  Not sure if this is a good idea; it's a bit hacky.  In particular,
  due to laziness some derivations might appear in `nix-env -qa' but
  disappear in `nix-env -qas' or `nix-env -i'.

  Commit 5000!
2006-03-08 16:03:58 +00:00
Eelco Dolstra 9088dee9e2 * Some refactoring of the exception handling code so that we can catch
Nix expression assertion failures.
2006-03-08 14:11:19 +00:00
Eelco Dolstra fa72ae1e9c * GCC 4.1 compatibility. 2006-03-06 14:40:10 +00:00
Eelco Dolstra c8bfb11b34 * `nix-env (-i|-u) --dry-run' now shows exactly which missing paths
will be built or substituted.
2006-03-06 11:21:15 +00:00
Eelco Dolstra 7ba1fd2029 * Regularise help text a bit. 2006-03-06 11:04:39 +00:00
Eelco Dolstra 70dee0f8ca * Flags --to-base32' and --to-base16' to convert between hex and
base-32 hashes.
2006-03-01 18:05:47 +00:00
Eelco Dolstra e136532800 * Ouch, parseHash32 was completely broken. All digits >= 4 were
parsed as 4.

  For a moment I worried that printHash32 was broken, and that would
  have been really, *really* bad ;-)
2006-03-01 17:59:08 +00:00
Eelco Dolstra 74166f2f44 * db.hh shouldn't depend on the Berkeley DB headers. 2006-03-01 17:44:28 +00:00
Eelco Dolstra d822bf32e4 * Close the database before the destructor runs. 2006-03-01 16:36:35 +00:00
Eelco Dolstra b6780b9e10 * Uninitialised variable. Fixes the --delete test. 2006-03-01 14:39:42 +00:00
Eelco Dolstra b602d2dfdf * Wrong delete. Thanks valgrind. 2006-03-01 14:39:10 +00:00
Eelco Dolstra 458820df6c * Generate valid HTML. 2006-03-01 13:24:47 +00:00
Eelco Dolstra 6b8bb8d74a * Remove dead code. 2006-03-01 12:51:39 +00:00
Eelco Dolstra 46f0cb0869 * In "nix-env -i", print a warning if there are multiple derivations
with the same name *and* version number, and pick the first one
  (this means that the order in which channels appear in
  ~/.nix-channels matters).  E.g.:

    $ nix-env ii aterm
    warning: there are multiple derivations named `aterm-2.4.2'; using the first one
    installing `aterm-2.4.2'
2006-02-17 18:11:45 +00:00
Eelco Dolstra 7a3e715980 * Fix for NIX-31: "nix-env -i foo" installing all derivations named
foo.  Now it will only install the one with the highest version
  number.
2006-02-17 17:47:54 +00:00
Eelco Dolstra 889ef564fd * Fix the infamous NIX-17: nix-env -i prints misleading messages
("installing `foo'" followed by "uninstalling `foo'").
2006-02-17 16:26:28 +00:00
Eelco Dolstra 345a95afe9 * Allow the size of the GC reserved file to be specified in nix.conf
through the new `gc-reserved-space' option.
2006-02-16 13:58:10 +00:00
Eelco Dolstra 651ab439cf * A simple hack to fix NIX-18: the garbage collector cannot run when
the disk is full (because to delete something from the Nix store, we
  need a Berkeley DB transaction, which takes up disk space).  Under
  normal operation, we make sure that there exists a file
  /nix/var/nix/db/reserved of 1 MB.  When running the garbage
  collector, we delete that file before we open the Berkeley DB
  environment.
2006-02-16 13:19:15 +00:00
Eelco Dolstra d6f586d0ea * Optional switch "--with-openssl=<PATH>" to use OpenSSL's
implementations of MD5, SHA-1 and SHA-256.  The main benefit is that
  we get assembler-optimised implementations of MD5 and SHA-1 (though
  not SHA-256 (at least on x86), unfortunately).  OpenSSL's SHA-1
  implementation on Intel is twice as fast as ours.
2006-02-13 19:52:43 +00:00
Eelco Dolstra e8475bbd5b * Use a union. 2006-02-13 18:00:08 +00:00
Eelco Dolstra 2d2e28d02c * Override YYMALLOC and YYFREE so that we can call AT[un]protectMemory
on the Bison parse stack.  Otherwise, a garbage collect during
  parsing could lead to a crash.
2006-02-13 13:09:23 +00:00
Eelco Dolstra 982399bb14 * Enable the --attr in nix-build as well (and add -A as an alias).
Example:

    $ nix-build ./all-packages.nix -A xlibs.libX11

  So finally it's easy to perform a test build of a Nix expression!
2006-02-10 17:37:35 +00:00
Eelco Dolstra c6120352b3 * In nix-instantiate, allow us to specify a "path" to the
derivation(s) we're interested, e.g.,

    $ nix-instantiate ./all-packages.nix --attr xlibs.libX11

  List elements can also be selected:

    $ nix-instantiate ./build-for-release.nix --attr 0.subversion

  This allows a non-ambiguous specification of a derivation.  Of
  course, this should also be added to nix-env and nix-build.
2006-02-10 17:25:59 +00:00
Eelco Dolstra 9e51abc7dc * Make --parse-only work when *not* reading from stdin. 2006-02-10 15:28:47 +00:00
Eelco Dolstra f848a45739 * Cleanup: use the code shared with nix-env. 2006-02-10 15:14:57 +00:00
Eelco Dolstra 4eb637c799 * When querying all derivations, filter out syntactically equal derivations. 2006-02-08 16:15:07 +00:00
Eelco Dolstra 8688e83194 * When evaluating, automatically call functions with default arguments. 2006-02-08 15:22:30 +00:00
Eelco Dolstra 287d0ef41c * Oops, fix breakage. 2006-02-08 14:32:33 +00:00
Eelco Dolstra 39f50db731 * Refactoring: move derivation evaluation to libexpr. 2006-02-08 13:21:16 +00:00
Eelco Dolstra 81de538e46 * Use setsid instead of setpgrp in child processes. This not only
creates a new process group but also a new session.  New sessions
  have no controlling tty, so child processes like ssh cannot open
  /dev/tty (which is bad).
2006-02-03 14:20:59 +00:00
Eelco Dolstra b90daaaf6c * When killing a build hook, kill the entire process group (as
intended).  This ensures that any ssh child processes to remote
  machines are also killed, and thus the Nix process on the remote
  machine also exits.  Without this, the remote Nix process will
  continue until it exists or until its stdout buffer gets full and it
  locks up.  (Partially fixes NIX-35.)
2006-02-02 16:27:31 +00:00
Eelco Dolstra 6e2eaaec96 * Print a better error message when writing the patched file (e.g.,
"No space left on device" instead of "Success").  Reported by Karina
  Olmos.
2006-02-01 17:28:13 +00:00
Eelco Dolstra 8291f6d968 * bsdiff updated to 4.3. This makes Nix depend on libbz2. 2006-02-01 16:48:49 +00:00
Eelco Dolstra 9e4ffc43a2 * The "S" bit should be based on the output path, not the derivation path. 2006-02-01 16:47:51 +00:00
Eelco Dolstra db2275cb99 * Oops, the "I" bit in "nix-env -qas" was broken. Reported by Nicolae Vintila. 2006-01-26 23:18:26 +00:00
Eelco Dolstra 1029716a8a * Don't show cycles, they're not very useful. 2006-01-19 15:35:34 +00:00
Eelco Dolstra 5b527901ae * dirOf: return "/", not "", for paths in the root directory. Fixes NIX-26. 2006-01-09 14:52:46 +00:00
Eelco Dolstra 04be39734f * Resolve all symlink components in the location of the temporary
build directory (TMPDIR, i.e., /tmp).  Fixes NIX-26.
2006-01-08 17:16:03 +00:00
Eelco Dolstra 0f8d3c871b * More GCC 2.95 compatibility. 2005-12-25 11:29:12 +00:00
Eelco Dolstra 52d20ef124 * Hack around a GCC 2.95 bug. 2005-12-25 02:02:29 +00:00
Eelco Dolstra 1440419b45 * GCC 2.95 compatibility. 2005-12-24 23:32:59 +00:00
Eelco Dolstra 1d2460ee5d * Documentation fixes. 2005-12-24 23:22:05 +00:00
Eelco Dolstra f96d2dea26 * Added a flag --ignore-liveness' to nix-store --delete'. It
deletes a path even if it is reachable from a root.  However, it
  won't delete a path that still has referrers (since that would
  violate store invariants).

  Don't try this at home.  It's a useful hack for recovering from
  certain situations in a somewhat clean way (e.g., holes in closures
  due to disk corruption).
2005-12-23 21:36:44 +00:00
Eelco Dolstra 4b9e7f59ca * Revived the old "nix-store --delete" operation that deletes the
specified paths from the Nix store.  However, this operation is
  safe: it refuses to delete anything that the garbage collector
  wouldn't delete.
2005-12-23 21:08:42 +00:00
Eelco Dolstra 530b27df1e * `nix-store --gc' prints out the number of bytes freed on stdout
(even when it is interrupted by a signal).
2005-12-15 21:11:39 +00:00
Eelco Dolstra b1eed6b586 * Split the database upgrade into multiple transactions to prevent
Berkeley DB from running out of locks.
2005-12-15 16:53:21 +00:00
Eelco Dolstra ab5c6bb3a3 * Change referer' to referrer' throughout. In particular, the
nix-store query options `--referer' and `--referer-closure' have
  been changed to `--referrer' and `--referrer-closure' (but the old
  ones are still accepted for compatibility).
2005-12-13 21:04:48 +00:00
Eelco Dolstra d87549c1c7 * Automatically delete the old referers table. 2005-12-12 19:14:38 +00:00
Eelco Dolstra 8463f27d8c * Fix NIX-23: quadratic complexity in maintaining the referers
mapping.  The referer table is replaced by a referrer table (note
  spelling fix) that stores each referrer separately.  That is,
  instead of having

    referer[P] = {Q_1, Q_2, Q_3, ...}

  we store

    referer[(P, Q_1)] = ""
    referer[(P, Q_2)] = ""
    referer[(P, Q_3)] = ""
    ...

  To find the referrers of P, we enumerate over the keys with a value
  lexicographically greater than P.  This requires the referrer table
  to be stored as a B-Tree rather than a hash table.

  (The tuples (P, Q) are stored as P + null-byte + Q.)

  Old Nix databases are upgraded automatically to the new schema.
2005-12-12 18:24:42 +00:00
Eelco Dolstra a33b561a6b * Use Berkeley DB 4.4's process registry feature to recover from
crashed Nix instances, and toss out our own recovery code.
2005-12-09 22:55:07 +00:00
Eelco Dolstra 760264bffe * Require Berkeley DB 4.4.
* Checkpoint after an upgrade.
2005-12-06 15:00:04 +00:00
Eelco Dolstra b7f008fc35 * Did something useful while waiting at IAD: reference scanning is now
much faster.
2005-11-16 08:27:06 +00:00
Eelco Dolstra 9311ab76a5 * Install signal handlers for SIGTERM and SIGHUP. This ensures that
Nix is properly shut down when it receives those signals.  In
  particular this ensures that killing the garbage collector doesn't
  cause a subsequent database recovery.
2005-11-04 15:34:09 +00:00
Eelco Dolstra 5bf939885a * Memoise checkVarDefs since internally produced terms (i.e., not the
result of parsing) can have very heavy sharing, causing exponential
  complexity if we naively recurse into them.  ATerms are graphs, not
  trees!
2005-11-04 15:17:05 +00:00
Eelco Dolstra 221c79013f * Turn off build hooks in nix-push because of an impurity (NIX-21). 2005-10-29 18:17:45 +00:00
Rob Vermaas f0856fd905 * Repair the referers table from the references table. 2005-10-29 15:44:02 +00:00
Eelco Dolstra 92d599c6a7 * Prevent uids from being used for more than one build
simultaneously.  We do this using exclusive locks on uid files in
  /nix/var/nix/userpool, e.g., /nix/var/nix/userpool/123 for uid 123.
2005-10-20 16:58:34 +00:00
Eelco Dolstra e932c40f8e * Oops. Fixed-output derivations were broken. 2005-10-19 14:27:44 +00:00
Eelco Dolstra 13b089c890 * Also kill all processes of the build user after the build. This is
critical to prevent certain kinds of 0wnage.
2005-10-17 17:43:21 +00:00
Eelco Dolstra f1b3a418fa * Before starting a build under some uid, kill all current processes
running under that uid.
2005-10-17 17:35:37 +00:00
Eelco Dolstra 439823ae80 * Check that the build result is owned by the build user, and that
nobody else has write permission to the build result.  This catches
  most hack attempts.
2005-10-17 16:59:25 +00:00
Eelco Dolstra 7ef574e5d0 * Don't use FIFOs to make Nix create the output path on behalf of the
builder.  Instead, require that the Nix store has sticky permission
  (S_ISVTX); everyone can created files in the Nix store, but they
  cannot delete, rename or modify files created by others.
2005-10-17 16:52:29 +00:00
Eelco Dolstra 32282abcea * Beginning of secure multi-user Nix stores. If Nix is started as
root (or setuid root), then builds will be performed under one of
  the users listed in the `build-users' configuration variables.  This
  is to make it impossible to influence build results externally,
  allowing locally built derivations to be shared safely between
  users (see ASE-2005 paper).

  To do: only one builder should be active per build user.
2005-10-17 15:33:24 +00:00
Eelco Dolstra dfbf520ec2 * Swap the system and version comparion columns. 2005-10-06 15:51:59 +00:00
Eelco Dolstra cec2be64f3 * Only colorise if we are attached to a terminal. 2005-10-06 15:01:46 +00:00
Eelco Dolstra b87b9c0d1f * New query option: --compare-versions' or -c' to compare installed
versions to available versions, or vice versa.

  For example, the following compares installed versions to available
  versions:

    $ nix-env -qc
    autoconf-2.59            = 2.59
    automake-1.9.4           < 1.9.6
    f-spot-0.0.10            - ?
    firefox-1.0.4            < 1.0.7
    ...

  I.e., there are newer versions available (in the current default Nix
  expression) for Automake and Firefox, but not for Autoconf, and
  F-Spot is missing altogether.

  Conversely, the available versions can be compared to the installed
  versions:

    $ nix-env -qac
    autoconf-2.59                  = 2.59
    automake-1.9.6                 > 1.9.4
    bash-3.0                       - ?
    firefox-1.0.7                  > 1.0.4
    ...

  Note that bash is available but no version of it is installed.

  If multiple versions are available for comparison, then the highest
  is used.  E.g., if Subversion 1.2.0 is installed, and Subversion
  1.1.4 and 1.2.3 are available, then `nix-env -qc' will print `<
  1.2.3', not `> 1.1.4'.

  If higher versions are available, the version column is printed in
  red (using ANSI escape codes).
2005-10-06 14:44:54 +00:00
Eelco Dolstra 0e38578433 * log2xml -> nix-log2xml. 2005-10-05 09:42:13 +00:00
Eelco Dolstra d47e03fccd * Install the XSL stylesheets for log to html conversion. 2005-10-05 09:37:58 +00:00
Eelco Dolstra ebfe57166d * Lets not go wild with templates. 2005-09-22 17:23:43 +00:00
Eelco Dolstra 4578a490ce * Parse multi-valued options. 2005-09-22 15:43:22 +00:00
Eelco Dolstra 8d06842a76 * Configuration options for trusted local builds. 2005-09-21 12:19:39 +00:00
Eelco Dolstra ef9e2c8e73 * Typo. 2005-09-20 16:14:00 +00:00
Eelco Dolstra ed1db42915 * List concatenation must be right-associative for efficiency. 2005-09-14 11:41:59 +00:00
Eelco Dolstra cf2bb91ec8 * Missing #include. 2005-09-13 13:17:01 +00:00
Eelco Dolstra 2bcd65ecf6 * `nix-env -e' corrupts memory due to incorrect use of iterators.
Reported by Rob Vermaas.
2005-09-01 18:14:04 +00:00
Eelco Dolstra e1a6fb7870 * `dependencyClosure' now allows a search path, e.g.,
dependencyClosure { ... searchPath = [ ../foo ../bar ]; ... }

* Primop `dirOf' to return the directory part of a path (e.g., dirOf
  /a/b/c == /a/b).

* Primop `relativise' (according to Webster that's a real word!) that
  given paths A and B returns a string representing path B relative
  path to A; e.g., relativise /a/b/c a/b/x/y => "../x/y".
2005-08-14 14:00:39 +00:00
Eelco Dolstra 08c53923db * A primitive operation `dependencyClosure' to do automatic dependency
determination (e.g., finding the header files dependencies of a C
  file) in Nix low-level builds automatically.

  For instance, in the function `compileC' in make/lib/default.nix, we
  find the header file dependencies of C file `main' as follows:

    localIncludes =
      dependencyClosure {
        scanner = file:
          import (findIncludes {
            inherit file;
          });
        startSet = [main];
      };

  The function works by "growing" the set of dependencies, starting
  with the set `startSet', and calling the function `scanner' for each
  file to get its dependencies (which should yield a list of strings
  representing relative paths).  For instance, when `scanner' is
  called on a file `foo.c' that includes the line

    #include "../bar/fnord.h"

  then `scanner' should yield ["../bar/fnord.h"].  This list of
  dependencies is absolutised relative to the including file and added
  to the set of dependencies.  The process continues until no more
  dependencies are found (hence its a closure).

  `dependencyClosure' yields a list that contains in alternation a
  dependency, and its relative path to the directory of the start
  file, e.g.,

    [ /bla/bla/foo.c
      "foo.c"
      /bla/bar/fnord.h
      "../bar/fnord.h"
    ]

  These relative paths are necessary for the builder that compiles
  foo.c to reconstruct the relative directory structure expected by
  foo.c.

  The advantage of `dependencyClosure' over the old approach (using
  the impure `__currentTime') is that it's completely pure, and more
  efficient because it only rescans for dependencies (i.e., by
  building the derivations yielded by `scanner') if sources have
  actually changed.  The old approach rescanned every time.
2005-08-14 12:38:47 +00:00
Eelco Dolstra 0399365675 * nix-hash: option `--truncate' to truncate the hash to 160 bits. Hmm,
kind of ad hoc ;-)
2005-08-14 10:09:56 +00:00
Eelco Dolstra c680f835c9 * Escape ASCII characters < 32 to Unicode FFFD (REPLACEMENT CHARACTER)
so that we don't produce un-wellformed XML.
2005-08-01 13:39:19 +00:00
Eelco Dolstra f450c8ea2f * Oops. XSL stylesheet to mark errors. 2005-08-01 13:24:04 +00:00
Eelco Dolstra 800a6ff845 * Mark error lines in red, and expand subtrees containing errors
automatically.
2005-08-01 13:23:43 +00:00
Eelco Dolstra 991a130b1e * Added a list concatenation operator:
[1 2 3] ++ [4 5 6] => [1 2 3 4 5 6]
2005-07-25 15:05:34 +00:00
Eelco Dolstra 4c20a08293 * Build dynamic libraries. 2005-07-22 14:52:45 +00:00
Eelco Dolstra 0f827cc607 * Prevent repeated wrapping of closed terms
(closed(closed(closed(...)))) since this reduces performance by
  producing bigger terms and killing caching (which incidentally also
  prevents useful infinite recursion detection).
2005-07-19 11:48:05 +00:00
Eelco Dolstra 2135e7c041 * Wat cleanups. 2005-07-16 23:19:20 +00:00
Eelco Dolstra 6f82a78de7 * Define paths using regexps, as is done in the Flex definition. 2005-07-16 21:38:15 +00:00
Eelco Dolstra 6f91f02f75 * Make the rejects a bit more compact.
* Add lexical restrictions for keywords.
2005-07-16 20:43:58 +00:00
Eelco Dolstra 9590009a74 * Fix ambiguity. 2005-07-16 15:41:27 +00:00
Eelco Dolstra f797cb5855 * Revive and update the SDF grammar for Nix expressions. 2005-07-16 14:07:35 +00:00
Eelco Dolstra 22d3587f3b * In nix-instantiate, at top-level, call functions that have arguments
with default values automatically.  I.e., e -> e {}.

  This feature makes convenience expressions such as
  pkgs/system/i686-linux.nix in Nixpkgs obsolete, since we can just do

  $ nix-instantiate ./pkgs/system/all-packages.nix

  since all-packages.nix takes a single argument (system) that has a
  default value (__thisSystem).
2005-07-12 16:06:25 +00:00