Commit graph

645 commits

Author SHA1 Message Date
Eelco Dolstra 0f8d3c871b * More GCC 2.95 compatibility. 2005-12-25 11:29:12 +00:00
Eelco Dolstra 52d20ef124 * Hack around a GCC 2.95 bug. 2005-12-25 02:02:29 +00:00
Eelco Dolstra 1440419b45 * GCC 2.95 compatibility. 2005-12-24 23:32:59 +00:00
Eelco Dolstra 1d2460ee5d * Documentation fixes. 2005-12-24 23:22:05 +00:00
Eelco Dolstra f96d2dea26 * Added a flag --ignore-liveness' to nix-store --delete'. It
deletes a path even if it is reachable from a root.  However, it
  won't delete a path that still has referrers (since that would
  violate store invariants).

  Don't try this at home.  It's a useful hack for recovering from
  certain situations in a somewhat clean way (e.g., holes in closures
  due to disk corruption).
2005-12-23 21:36:44 +00:00
Eelco Dolstra 4b9e7f59ca * Revived the old "nix-store --delete" operation that deletes the
specified paths from the Nix store.  However, this operation is
  safe: it refuses to delete anything that the garbage collector
  wouldn't delete.
2005-12-23 21:08:42 +00:00
Eelco Dolstra 530b27df1e * `nix-store --gc' prints out the number of bytes freed on stdout
(even when it is interrupted by a signal).
2005-12-15 21:11:39 +00:00
Eelco Dolstra b1eed6b586 * Split the database upgrade into multiple transactions to prevent
Berkeley DB from running out of locks.
2005-12-15 16:53:21 +00:00
Eelco Dolstra ab5c6bb3a3 * Change referer' to referrer' throughout. In particular, the
nix-store query options `--referer' and `--referer-closure' have
  been changed to `--referrer' and `--referrer-closure' (but the old
  ones are still accepted for compatibility).
2005-12-13 21:04:48 +00:00
Eelco Dolstra d87549c1c7 * Automatically delete the old referers table. 2005-12-12 19:14:38 +00:00
Eelco Dolstra 8463f27d8c * Fix NIX-23: quadratic complexity in maintaining the referers
mapping.  The referer table is replaced by a referrer table (note
  spelling fix) that stores each referrer separately.  That is,
  instead of having

    referer[P] = {Q_1, Q_2, Q_3, ...}

  we store

    referer[(P, Q_1)] = ""
    referer[(P, Q_2)] = ""
    referer[(P, Q_3)] = ""
    ...

  To find the referrers of P, we enumerate over the keys with a value
  lexicographically greater than P.  This requires the referrer table
  to be stored as a B-Tree rather than a hash table.

  (The tuples (P, Q) are stored as P + null-byte + Q.)

  Old Nix databases are upgraded automatically to the new schema.
2005-12-12 18:24:42 +00:00
Eelco Dolstra a33b561a6b * Use Berkeley DB 4.4's process registry feature to recover from
crashed Nix instances, and toss out our own recovery code.
2005-12-09 22:55:07 +00:00
Eelco Dolstra 760264bffe * Require Berkeley DB 4.4.
* Checkpoint after an upgrade.
2005-12-06 15:00:04 +00:00
Eelco Dolstra b7f008fc35 * Did something useful while waiting at IAD: reference scanning is now
much faster.
2005-11-16 08:27:06 +00:00
Eelco Dolstra 9311ab76a5 * Install signal handlers for SIGTERM and SIGHUP. This ensures that
Nix is properly shut down when it receives those signals.  In
  particular this ensures that killing the garbage collector doesn't
  cause a subsequent database recovery.
2005-11-04 15:34:09 +00:00
Eelco Dolstra 5bf939885a * Memoise checkVarDefs since internally produced terms (i.e., not the
result of parsing) can have very heavy sharing, causing exponential
  complexity if we naively recurse into them.  ATerms are graphs, not
  trees!
2005-11-04 15:17:05 +00:00
Eelco Dolstra 221c79013f * Turn off build hooks in nix-push because of an impurity (NIX-21). 2005-10-29 18:17:45 +00:00
Rob Vermaas f0856fd905 * Repair the referers table from the references table. 2005-10-29 15:44:02 +00:00
Eelco Dolstra 92d599c6a7 * Prevent uids from being used for more than one build
simultaneously.  We do this using exclusive locks on uid files in
  /nix/var/nix/userpool, e.g., /nix/var/nix/userpool/123 for uid 123.
2005-10-20 16:58:34 +00:00
Eelco Dolstra e932c40f8e * Oops. Fixed-output derivations were broken. 2005-10-19 14:27:44 +00:00
Eelco Dolstra 13b089c890 * Also kill all processes of the build user after the build. This is
critical to prevent certain kinds of 0wnage.
2005-10-17 17:43:21 +00:00
Eelco Dolstra f1b3a418fa * Before starting a build under some uid, kill all current processes
running under that uid.
2005-10-17 17:35:37 +00:00
Eelco Dolstra 439823ae80 * Check that the build result is owned by the build user, and that
nobody else has write permission to the build result.  This catches
  most hack attempts.
2005-10-17 16:59:25 +00:00
Eelco Dolstra 7ef574e5d0 * Don't use FIFOs to make Nix create the output path on behalf of the
builder.  Instead, require that the Nix store has sticky permission
  (S_ISVTX); everyone can created files in the Nix store, but they
  cannot delete, rename or modify files created by others.
2005-10-17 16:52:29 +00:00
Eelco Dolstra 32282abcea * Beginning of secure multi-user Nix stores. If Nix is started as
root (or setuid root), then builds will be performed under one of
  the users listed in the `build-users' configuration variables.  This
  is to make it impossible to influence build results externally,
  allowing locally built derivations to be shared safely between
  users (see ASE-2005 paper).

  To do: only one builder should be active per build user.
2005-10-17 15:33:24 +00:00
Eelco Dolstra dfbf520ec2 * Swap the system and version comparion columns. 2005-10-06 15:51:59 +00:00
Eelco Dolstra cec2be64f3 * Only colorise if we are attached to a terminal. 2005-10-06 15:01:46 +00:00
Eelco Dolstra b87b9c0d1f * New query option: --compare-versions' or -c' to compare installed
versions to available versions, or vice versa.

  For example, the following compares installed versions to available
  versions:

    $ nix-env -qc
    autoconf-2.59            = 2.59
    automake-1.9.4           < 1.9.6
    f-spot-0.0.10            - ?
    firefox-1.0.4            < 1.0.7
    ...

  I.e., there are newer versions available (in the current default Nix
  expression) for Automake and Firefox, but not for Autoconf, and
  F-Spot is missing altogether.

  Conversely, the available versions can be compared to the installed
  versions:

    $ nix-env -qac
    autoconf-2.59                  = 2.59
    automake-1.9.6                 > 1.9.4
    bash-3.0                       - ?
    firefox-1.0.7                  > 1.0.4
    ...

  Note that bash is available but no version of it is installed.

  If multiple versions are available for comparison, then the highest
  is used.  E.g., if Subversion 1.2.0 is installed, and Subversion
  1.1.4 and 1.2.3 are available, then `nix-env -qc' will print `<
  1.2.3', not `> 1.1.4'.

  If higher versions are available, the version column is printed in
  red (using ANSI escape codes).
2005-10-06 14:44:54 +00:00
Eelco Dolstra 0e38578433 * log2xml -> nix-log2xml. 2005-10-05 09:42:13 +00:00
Eelco Dolstra d47e03fccd * Install the XSL stylesheets for log to html conversion. 2005-10-05 09:37:58 +00:00
Eelco Dolstra ebfe57166d * Lets not go wild with templates. 2005-09-22 17:23:43 +00:00
Eelco Dolstra 4578a490ce * Parse multi-valued options. 2005-09-22 15:43:22 +00:00
Eelco Dolstra 8d06842a76 * Configuration options for trusted local builds. 2005-09-21 12:19:39 +00:00
Eelco Dolstra ef9e2c8e73 * Typo. 2005-09-20 16:14:00 +00:00
Eelco Dolstra ed1db42915 * List concatenation must be right-associative for efficiency. 2005-09-14 11:41:59 +00:00
Eelco Dolstra cf2bb91ec8 * Missing #include. 2005-09-13 13:17:01 +00:00
Eelco Dolstra 2bcd65ecf6 * `nix-env -e' corrupts memory due to incorrect use of iterators.
Reported by Rob Vermaas.
2005-09-01 18:14:04 +00:00
Eelco Dolstra e1a6fb7870 * `dependencyClosure' now allows a search path, e.g.,
dependencyClosure { ... searchPath = [ ../foo ../bar ]; ... }

* Primop `dirOf' to return the directory part of a path (e.g., dirOf
  /a/b/c == /a/b).

* Primop `relativise' (according to Webster that's a real word!) that
  given paths A and B returns a string representing path B relative
  path to A; e.g., relativise /a/b/c a/b/x/y => "../x/y".
2005-08-14 14:00:39 +00:00
Eelco Dolstra 08c53923db * A primitive operation `dependencyClosure' to do automatic dependency
determination (e.g., finding the header files dependencies of a C
  file) in Nix low-level builds automatically.

  For instance, in the function `compileC' in make/lib/default.nix, we
  find the header file dependencies of C file `main' as follows:

    localIncludes =
      dependencyClosure {
        scanner = file:
          import (findIncludes {
            inherit file;
          });
        startSet = [main];
      };

  The function works by "growing" the set of dependencies, starting
  with the set `startSet', and calling the function `scanner' for each
  file to get its dependencies (which should yield a list of strings
  representing relative paths).  For instance, when `scanner' is
  called on a file `foo.c' that includes the line

    #include "../bar/fnord.h"

  then `scanner' should yield ["../bar/fnord.h"].  This list of
  dependencies is absolutised relative to the including file and added
  to the set of dependencies.  The process continues until no more
  dependencies are found (hence its a closure).

  `dependencyClosure' yields a list that contains in alternation a
  dependency, and its relative path to the directory of the start
  file, e.g.,

    [ /bla/bla/foo.c
      "foo.c"
      /bla/bar/fnord.h
      "../bar/fnord.h"
    ]

  These relative paths are necessary for the builder that compiles
  foo.c to reconstruct the relative directory structure expected by
  foo.c.

  The advantage of `dependencyClosure' over the old approach (using
  the impure `__currentTime') is that it's completely pure, and more
  efficient because it only rescans for dependencies (i.e., by
  building the derivations yielded by `scanner') if sources have
  actually changed.  The old approach rescanned every time.
2005-08-14 12:38:47 +00:00
Eelco Dolstra 0399365675 * nix-hash: option `--truncate' to truncate the hash to 160 bits. Hmm,
kind of ad hoc ;-)
2005-08-14 10:09:56 +00:00
Eelco Dolstra c680f835c9 * Escape ASCII characters < 32 to Unicode FFFD (REPLACEMENT CHARACTER)
so that we don't produce un-wellformed XML.
2005-08-01 13:39:19 +00:00
Eelco Dolstra f450c8ea2f * Oops. XSL stylesheet to mark errors. 2005-08-01 13:24:04 +00:00
Eelco Dolstra 800a6ff845 * Mark error lines in red, and expand subtrees containing errors
automatically.
2005-08-01 13:23:43 +00:00
Eelco Dolstra 991a130b1e * Added a list concatenation operator:
[1 2 3] ++ [4 5 6] => [1 2 3 4 5 6]
2005-07-25 15:05:34 +00:00
Eelco Dolstra 4c20a08293 * Build dynamic libraries. 2005-07-22 14:52:45 +00:00
Eelco Dolstra 0f827cc607 * Prevent repeated wrapping of closed terms
(closed(closed(closed(...)))) since this reduces performance by
  producing bigger terms and killing caching (which incidentally also
  prevents useful infinite recursion detection).
2005-07-19 11:48:05 +00:00
Eelco Dolstra 2135e7c041 * Wat cleanups. 2005-07-16 23:19:20 +00:00
Eelco Dolstra 6f82a78de7 * Define paths using regexps, as is done in the Flex definition. 2005-07-16 21:38:15 +00:00
Eelco Dolstra 6f91f02f75 * Make the rejects a bit more compact.
* Add lexical restrictions for keywords.
2005-07-16 20:43:58 +00:00
Eelco Dolstra 9590009a74 * Fix ambiguity. 2005-07-16 15:41:27 +00:00
Eelco Dolstra f797cb5855 * Revive and update the SDF grammar for Nix expressions. 2005-07-16 14:07:35 +00:00
Eelco Dolstra 22d3587f3b * In nix-instantiate, at top-level, call functions that have arguments
with default values automatically.  I.e., e -> e {}.

  This feature makes convenience expressions such as
  pkgs/system/i686-linux.nix in Nixpkgs obsolete, since we can just do

  $ nix-instantiate ./pkgs/system/all-packages.nix

  since all-packages.nix takes a single argument (system) that has a
  default value (__thisSystem).
2005-07-12 16:06:25 +00:00
Eelco Dolstra 040140dd1c * Added a primop `removeAttrs' to remove attributes from a set, e.g.,
`removeAttrs attrs ["x", "y"]' returns the set `attrs' with the
  attributes named `x' and `y' removed.  It is not an error for the
  named attributes to be missing from the input set.
2005-05-18 17:19:21 +00:00
Eelco Dolstra 109cde6706 * Ignore (with a warning) invalid garbage collector roots. 2005-05-10 14:56:10 +00:00
Eelco Dolstra 456f3251d2 * Typo. 2005-05-10 14:21:46 +00:00
Eelco Dolstra 9ec7e58aa4 * Handle store path arguments in `nix-env -i' correctly again. 2005-05-09 17:55:35 +00:00
Eelco Dolstra 8f57634c14 * Automatically upgrade the Berkeley DB environment if necessary. 2005-05-09 15:25:47 +00:00
Eelco Dolstra edd145d2fb * Lazily compute the derivation and output paths of derivations. This
makes most query and installation operations much faster (e.g.,
  `nix-env -qa' on the current Nixpkgs is about 10 times faster).
2005-05-08 10:32:09 +00:00
Eelco Dolstra 426593162e * ATermMap needs an assignment operator, otherwise we are screwed. 2005-05-08 10:28:19 +00:00
Eelco Dolstra 77557a6f06 Commit 3000!
* Make the `derivation' primitive much more lazy.  The expression
  `derivation attrs' now evaluates to (essentially)

    attrs // {
      type = "derivation";
      outPath = derivation! attrs;
      drvPath = derivation! attrs;
    }

  where `derivation!' is a primop that does the actual derivation
  instantiation (i.e., it does what `derivation' used to do).  The
  advantage is that it allows commands such as `nix-env -qa' and
  `nix-env -i' to be much faster since they no longer need to
  instantiate all derivations, just the `name' attribute.  (However,
  `nix-env' doesn't yet take advantage of this since it still always
  evaluates the `outPath' and `drvPath' attributes).

  Also, this allows derivations to cyclically reference each other,
  for example,

    webServer = derivation {
      ...
      hostName = "svn.cs.uu.nl";
      services = [svnService];
    };

    svnService = derivation {
      ...
      hostName = webServer.hostName;
    };

  Previously, this would yield a black hole (infinite recursion).
2005-05-07 21:48:49 +00:00
Eelco Dolstra 6057b51835 * Don't try to register GC roots in read-only mode. 2005-05-07 21:33:31 +00:00
Eelco Dolstra 52a2f41320 * Include some required header files. 2005-05-04 16:33:20 +00:00
Eelco Dolstra 26fd28432d * FreeBSD 4.x doesn't have stdint.h, use inttypes.h instead (which is
also part of ISO C).
2005-05-04 16:32:54 +00:00
Eelco Dolstra 5dea0622d1 * Idem (constness fix).
* `compare' in GCC 2.95 is broken.
2005-05-04 16:31:49 +00:00
Eelco Dolstra 4a266e35d4 * GCC 2.95 compatibility fix in constness; strangely, I think this
should not have worked at all.
2005-05-04 16:31:24 +00:00
Eelco Dolstra d7b3cdbd91 * GCC 2.95 compatibility. Prevents internal compiler error in member
template friends.
2005-05-04 16:30:35 +00:00
Eelco Dolstra ae6d9033a1 * The eof() state isn't guaranteed to be set non-lazily. GCC 2.95
compatibility fix.
2005-05-04 16:29:44 +00:00
Eelco Dolstra 02f2da0142 * Merging from nix-make branch:
- Add __currentTime primitive (dangerous!).
  - Allow imports of derivations.
2005-05-02 14:44:58 +00:00
Eelco Dolstra f913283570 * Remove redundant message. 2005-04-13 09:20:27 +00:00
Eelco Dolstra f3660b1c8c * Garbage collector fix: allow deletion of paths that have invalid
(but substitutable) referers.
2005-04-12 10:51:00 +00:00
Eelco Dolstra cab7816b56 * Slightly nicer message. 2005-04-11 08:07:41 +00:00
Eelco Dolstra c9c58dba55 * Primop `__currentSystem' to return the current platform identifier. 2005-04-10 17:38:19 +00:00
Eelco Dolstra c702dfca3f * nix-store: --substitute' -> --register-substitutes'. 2005-04-08 13:48:41 +00:00
Eelco Dolstra 4271385a73 * Make `nix-store --query --tree' work on non-derivations (i.e., on
any store path).
2005-04-08 12:57:16 +00:00
Eelco Dolstra 10c429c757 * If store paths are specified as sources in Nix expressions, don't
copy them, but use them directly.
2005-04-07 14:35:01 +00:00
Eelco Dolstra f9848d4f31 * Support base-32 hash representations. 2005-04-07 14:33:32 +00:00
Eelco Dolstra c815aff21b * `nix-store --add-fixed' to preload the outputs of fixed-output
derivations.  This is mostly to simplify the implementation of
  nix-prefetch-{url, svn}, which now work properly in setuid
  installations.

* Enforce valid store names in `nix-store --add / --add-fixed'.
2005-04-07 14:01:51 +00:00
Eelco Dolstra 6f788880b6 * Re-enable dot graph generation. 2005-03-26 22:06:57 +00:00
Eelco Dolstra 298dd487bb * When finding live paths, the deriver need not be valid. 2005-03-25 14:31:12 +00:00
Eelco Dolstra 7eaf038763 * `nix-store --verify': repair bad referer mappings. 2005-03-25 14:21:49 +00:00
Eelco Dolstra c6178f0b03 * Create missing log and temproots directories automatically (reported
by Rob).
2005-03-24 17:46:38 +00:00
Eelco Dolstra cff6bc06df * Fix endianness bug. 2005-03-23 19:18:22 +00:00
Eelco Dolstra 0df9f08078 * Export the references graph to the build hook. 2005-03-23 13:16:36 +00:00
Eelco Dolstra 3f236f01ae * `nix-store --register-validity': allow a path to refer to a path
listed later in the list of new valid paths.
2005-03-23 13:07:28 +00:00
Eelco Dolstra a04c62e0c4 * Canonicalise path meta-data in `nix-store --register-validity'. 2005-03-23 12:06:57 +00:00
Eelco Dolstra f20f081560 * nix-store: --isvalid' -> --check-validity', `--validpath' ->
`--register-validity'.
* `nix-store --register-validity': read arguments from stdin, and
  allow the references and deriver to be set.
2005-03-23 11:25:20 +00:00
Eelco Dolstra c757d16c8c * Bug in clearSubstitutes(). 2005-03-14 18:54:40 +00:00
Eelco Dolstra 08df443618 * Check for duplicate attributes and formal parameters in Nix
expressions.
2005-03-10 11:33:46 +00:00
Eelco Dolstra 4bbdcfbb45 * Don't use fork() in copyPath(), but a string buffer. 2005-03-03 13:58:02 +00:00
Eelco Dolstra 86cb3cc554 * Increase Berkeley DB limits a bit more.
* Maintain the cleanup invariant in clearSubstitutes().
2005-03-03 13:10:44 +00:00
Eelco Dolstra 07b4399fb6 * `nix-store -q --hash' to quickly query the hash of the contents of a
store path (which is stored in the database).
2005-03-02 15:57:06 +00:00
Eelco Dolstra bfaf83a0fd * When multiple derivations are specified in `nix-store -r', don't
continue building when one fails unless `--keep-going' is
  specified.
* When `--keep-going' is specified, print out the set of failing
  derivations at the end (otherwise it can be hard to find out which
  failed).
2005-02-23 11:19:27 +00:00
Eelco Dolstra 3a2c3f0cf2 * Support for fixed-output hashes over directory trees (i.e., over the
NAR dump of the path).
2005-02-22 21:14:41 +00:00
Eelco Dolstra eda2c3c253 * Compatibility hack so that Nixpkgs can continue to do hash checking
in `fetchurl' in Nix <= 0.7, but doesn't in Nix 0.8.
2005-02-22 15:23:24 +00:00
Eelco Dolstra 3c1630131e * Subtle bug in the builder: if a subgoal that is instantiated
multiple times is also a top-level goal, then the second and later
  instantiations would never be created because there would be a
  stable pointer to the first one that would keep it alive in the
  WeakGoalMap.
* Some tracing code for debugging this kind of problem.
2005-02-18 09:50:20 +00:00
Eelco Dolstra e0181f56be * `nix-store -q --tree' shows a tree representing the dependency graph
of the given derivation.  Useful for getting a quick overview of how
  something was built.  E.g., to find out how the `baffle' program in
  your user environment was built, you can do

    $ nix-store -q --tree $(nix-store -qd $(which baffle))

  Tree nesting depth is minimised (?) by topologically sorting paths
  under the relation A < B iff A \in closure(B).
2005-02-17 15:57:46 +00:00
Eelco Dolstra 74ab0695b5 * Compatibility hack with older user environments. 2005-02-17 15:48:50 +00:00
Eelco Dolstra 8a3a96dd5b * Switch to the calling user context for some more operations in a
setuid installation.
2005-02-17 13:55:18 +00:00
Eelco Dolstra 88273f9574 * Put build logs in $prefix/var/nix/log/drvs/. 2005-02-17 13:54:45 +00:00
Eelco Dolstra 202d5bbda5 * Compatibility with older GCCs. 2005-02-15 12:05:47 +00:00