Commit graph

161 commits

Author SHA1 Message Date
Eelco Dolstra 816f9c0f6f * Use std::tr1::unordered_set instead of std::set for the symbol
table.  This gives a 10% speed increase on `nix-instantiate
  /etc/nixos/nixos -A system --readonly-mode'.
2010-04-13 14:34:11 +00:00
Eelco Dolstra 7d47498b5e * Evaluate lets directly (i.e. without desugaring to `rec { attrs...;
<let-body> = e; }.<let-body>).  This prevents the unnecessary
  allocation of an attribute set.
2010-04-13 13:42:25 +00:00
Eelco Dolstra ac1e8f40d4 * Use a symbol table to represent identifiers and attribute names
efficiently.  The symbol table ensures that there is only one copy
  of each symbol, thus allowing symbols to be compared efficiently
  using a pointer equality test.
2010-04-13 12:25:42 +00:00
Eelco Dolstra 10e8b1fd15 * Finished the ATerm-less parser. 2010-04-12 23:33:23 +00:00
Eelco Dolstra d4f0b0fc6c * Indented strings. 2010-04-12 22:03:27 +00:00
Eelco Dolstra a60317f20f * More missing constructs. 2010-04-12 21:21:24 +00:00
Eelco Dolstra 4d6ad5be17 * Don't use ATerms for the abstract syntax trees anymore. Not
finished yet.
2010-04-12 18:30:11 +00:00
Eelco Dolstra db90b88e65 * Hack to support builderDefs expressions. 2010-04-12 09:50:20 +00:00
Eelco Dolstra 4e49002576 * Doh. 2010-04-12 09:45:00 +00:00
Eelco Dolstra f3dc7ab877 * Keep more statistics about stack space usage.
* Reduce stack space usage.
2010-04-09 12:00:49 +00:00
Eelco Dolstra b7b3dd55f9 * Remove a lot of dead code. 2010-04-08 11:41:19 +00:00
Eelco Dolstra 7e048eddf5 * Fix blackholing. If evaluation fails due to an assertion failure,
then the blackhole has to be removed to ensure that repeated
  evaluation of the same value gives an assertion failure again rather
  than an "infinite recursion" error.
2010-04-08 11:25:14 +00:00
Eelco Dolstra af2a372bb0 * Update autoCallFunction() and findAlongAttrPath(). 2010-04-07 15:47:06 +00:00
Eelco Dolstra fc92244ba8 * Implemented the primops necessary for generating the NixOS manual. 2010-04-07 13:55:46 +00:00
Eelco Dolstra a353aef0b1 * In eval(), don't use the target value `v' as a temporary.
Overwriting `v' breaks when the expression evaluation to an
  assertion failure or throw.
2010-04-06 14:15:29 +00:00
Eelco Dolstra c172274e17 * Quick hack to make coerceToString work more or less correctly on
nested lists.  `nix-instantiate' can now evaluate the NixOS system
  derivation attribute correctly (in 2.1s on my laptop vs. 6.2s for
  the trunk).
2010-04-01 14:35:03 +00:00
Eelco Dolstra 7b851915bf * Improve sharing. 2010-04-01 12:04:57 +00:00
Eelco Dolstra 95cc417d76 * Functions are incomparable. 2010-04-01 10:55:36 +00:00
Eelco Dolstra 71f026292b * Make `derivation' lazy again for performance. It also turns out
that there are some places in Nixpkgs (php_configurable /
  composableDerivation, it seems) that call `derivation' with
  incorrect arguments (namely, the `name' attribute missing) but get
  away with it because of laziness.
2010-04-01 09:55:57 +00:00
Eelco Dolstra dc31305b38 * Fixed the trace primop and path comparison.
* Removed exprToString and stringToExpr because there is no ATerm
  representation to work on anymore (and exposing the internals of the
  evaluator like this is not a good idea anyway).
2010-03-31 20:09:20 +00:00
Eelco Dolstra 979f163615 * Handle string contexts. `nix-instantiate' can now correctly compute
the `firefoxWrapper' attribute in Nixpkgs, and it's about 3 times
  faster than the trunk :-)
2010-03-31 19:52:29 +00:00
Eelco Dolstra 55e207b2dc * Cache parse trees to prevent repeated parsing of imported Nix
expressions.
2010-03-31 16:14:32 +00:00
Eelco Dolstra 3d94be61ea * Implemented derivations. 2010-03-31 15:38:03 +00:00
Eelco Dolstra 13c2adc897 * Implemented `rec { inherit ...; }'. 2010-03-31 11:05:39 +00:00
Eelco Dolstra 4c53ca2692 * Compare nulls. 2010-03-31 09:54:12 +00:00
Eelco Dolstra 7f19e03c65 * More primops. 2010-03-30 22:39:48 +00:00
Eelco Dolstra 47df476daa * More operators / primops. 2010-03-30 18:05:54 +00:00
Eelco Dolstra c9170be2bd * More primops. 2010-03-30 15:18:20 +00:00
Eelco Dolstra c3aa615a5f * More primops. 2010-03-30 14:39:27 +00:00
Eelco Dolstra 5b72d8a749 * Implemented `map'. 2010-03-30 13:47:59 +00:00
Eelco Dolstra d78a05ab40 * Make `import' work. 2010-03-30 09:22:33 +00:00
Eelco Dolstra 31428c3a06 * Started integrating the new evaluator. 2010-03-29 14:37:56 +00:00
Eelco Dolstra 8a10360c91 * Simplify @-patterns: only {attrs}@name' or name@{attrs}' are now
allowed.  So `name1@name2', `{attrs1}@{attrs2}' and so on are now no
  longer legal.  This is no big loss because they were not useful
  anyway.

  This also changes the output of builtins.toXML for @-patterns
  slightly.
2010-03-25 12:19:41 +00:00
Eelco Dolstra 50d11b90ca * Allow unsafe (unspecified) comparisons between attrsets unless
NIX_NO_UNSAFE_EQ is set, for now.
2009-05-12 11:06:24 +00:00
Eelco Dolstra c34e6d71bc * Disallow equality tests between attribute sets. This was always
broken, but now the evaluator checks for it to prevent Nix
  expressions from relying on undefined behaviour.  Equality tests are
  implemented using a shallow pointer equality test between ATerms.
  However, because attribute sets are lazy and contain position
  information, this can give false positives.  For instance,
  previously

    let y = {x = 1;}; in y == y

  evaluated to true, while the equivalent expression

    {x = 1;} == {x = 1;}

  evaluated to false.  So disallow these tests for now.  (Eventually
  we may want to implement deep equality tests for attribute sets,
  like lib.eqStrict.)
  
* Idem: disallow comparisons between functions.

* Implemented deep comparisons of lists.  This had the same problem as
  attribute sets - the elements in the list weren't evaluated.  For
  instance,

    ["xy"] == [("x" + "y")]

  evaluated to false.  Now it works properly.
2009-05-11 15:50:14 +00:00
Eelco Dolstra 99dc3e613a * Require that __overrides is defined as a non-recursive attribute
(which means it can only be defined via "inherit"), otherwise we get
  scoping bugs, since __overrides can't be recursive (or at least, it
  would be hard).
2008-08-26 14:05:59 +00:00
Michael Raskin b7ff182b6e Fixing an obvious typo in override code. I do not know whether it works correctly after the change, but at least it ca nbe compiled now. 2008-08-14 22:01:43 +00:00
Eelco Dolstra ca07f3e370 * Another experimental feature: a way to truly override attributes in
a rec.  This will be very useful to allow end-user customisation of
  all-packages.nix, for instance globally overriding GCC or some other
  dependency.  The // operator doesn't cut it: you could replace the
  "gcc" attribute, but all other attributes would continue to
  reference the original value due to the substitution semantics of
  rec.

  The syntax is a bit hacky but this is to allow backwards
  compatibility.
2008-08-14 16:59:37 +00:00
Eelco Dolstra 9279174dde * Added an experimental feature suggested by Andres: ellipses ("...")
in attribute set pattern matches.  This allows defining a function
  that takes *at least* the listed attributes, while ignoring
  additional attributes.  For instance,

    {stdenv, fetchurl, fuse, ...}:
    
    stdenv.mkDerivation {
      ...
    };
    
  defines a function that requires an attribute set that contains the 
  specified attributes but ignores others.  The main advantage is that
  we can then write in all-packages.nix

    aefs = import ../bla/aefs pkgs;

  instead of

    aefs = import ../bla/aefs {
      inherit stdenv fetchurl fuse;
    };

  This saves a lot of typing (not to mention not having to update
  all-packages.nix with purely mechanical changes).  It saves as much
  typing as the "args: with args;" style, but has the advantage that
  the function arguments are properly declared (not implicit in what
  the body of the "with" uses).
2008-08-14 14:00:44 +00:00
Eelco Dolstra 1b962fc720 * @-patterns as in Haskell. For instance, in a function definition
f = args @ {x, y, z}: ...;

  `args' refers to the argument as a whole, which is further
  pattern-matched against the attribute set pattern {x, y, z}.
2008-08-14 12:53:29 +00:00
Eelco Dolstra efe4b690ae * Refactoring: combine functions that take an attribute set and
functions that take a single argument (plain lambdas) into one AST
  node (Function) that contains a Pattern node describing the
  arguments.  Current patterns are single lazy arguments (VarPat) and
  matching against an attribute set (AttrsPat).

  This refactoring allows other kinds of patterns to be added easily,
  such as Haskell-style @-patterns, or list pattern matching.
2008-08-14 10:04:22 +00:00
Eelco Dolstra 5664b6d7ba * Removed the "valid values" feature. Nobody uses it anyway. 2008-08-11 13:36:40 +00:00
Eelco Dolstra fc691e1cbd * Print a better error message when a non-derivation attribute set is
coerced to a string.
2008-07-24 14:52:25 +00:00
Eelco Dolstra bddc83a148 * New builtin function "isFunction". You're not supposed to use it
;-)
* Channels: fix channels that are plain lists of derivations (like
  strategoxt-unstable) instead  of functions (like nixpkgs-unstable).
  This fixes the error message "error: the left-hand side of the
  function call is neither a function nor a primop (built-in
  operation) but a list".
2007-05-16 16:17:04 +00:00
Eelco Dolstra 5f2492eaec * New primop "throw <string>" to throw an error. This is like abort,
only thrown errors are caught by the top-level derivation evaluation
  in nix-env -qa / -i.
2007-04-16 15:03:19 +00:00
Eelco Dolstra 044b6482c1 * Greatly reduced the amount of stack space used by the Nix expression
evaluator.  This was important because the NixOS expressions started
  to hit 2 MB default stack size on Linux.

  GCC is really dumb about stack space: it just adds up all the local
  variables and temporaries of every scope into one huge stack frame.
  This is really bad for deeply recursive functions.  For instance,
  every `throw Error(format("error message"))' causes a format object
  of a few hundred bytes to be allocated on the stack.  As a result,
  every recursive call to evalExpr2() consumed 4680 bytes.  By
  splitting evalExpr2() and by moving the exception-throwing code out
  of the main functions, evalExpr2() now only consumes 40 bytes.
  Similar for evalExpr().
2007-02-27 19:10:45 +00:00
Eelco Dolstra adce01a8d0 * When NIX_SHOW_STATS=1, show the amount of stack space consumed by
the Nix expression evaluator.
2007-02-27 17:28:51 +00:00
Eelco Dolstra 05879db628 * Memoize strict evaluation. 2007-01-13 15:41:54 +00:00
Eelco Dolstra 792878af91 * Make printing an expression as XML interruptible. 2007-01-13 14:48:41 +00:00
Eelco Dolstra 11158028be * Cleanup. 2007-01-13 14:21:49 +00:00
Eelco Dolstra a824d58b56 * Merge addToStore and addToStoreFixed.
* addToStore now adds unconditionally, it doesn't use readOnlyMode.
  Read-only operation is up to the caller (who can call
  computeStorePathForPath).
2006-12-01 20:51:18 +00:00
Eelco Dolstra e2ef5e07fd * Refactoring. There is now an abstract interface class StoreAPI
containing functions that operate on the Nix store.  One
  implementation is LocalStore, which operates on the Nix store
  directly.  The next step, to enable secure multi-user Nix, is to
  create a different implementation RemoteStore that talks to a
  privileged daemon process that uses LocalStore to perform the actual
  operations.
2006-11-30 17:43:04 +00:00
Eelco Dolstra 4bd5cdb90b * Print out the offending path. 2006-10-17 14:01:28 +00:00
Eelco Dolstra 58ff6939f4 * An awful backwards compatibility hack. 2006-10-17 12:58:42 +00:00
Eelco Dolstra 7de5fe2fc2 * Do the path check on the normal form. 2006-10-17 10:57:25 +00:00
Eelco Dolstra d7efd76394 * Big cleanup of the semantics of paths, strings, contexts, string
concatenation and string coercion.  This was a big mess (see
  e.g. NIX-67).  Contexts are now folded into strings, so that they
  don't cause evaluation errors when they're not expected.  The
  semantics of paths has been clarified (see nixexpr-ast.def).
  toString() and coerceToString() have been merged.

  Semantic change: paths are now copied to the store when they're in a
  concatenation (and in most other situations - that's the
  formalisation of the meaning of a path).  So

    "foo " + ./bla

  evaluates to "foo /nix/store/hash...-bla", not "foo
  /path/to/current-dir/bla".  This prevents accidental impurities, and
  is more consistent with the treatment of derivation outputs, e.g.,
  `"foo " + bla' where `bla' is a derivation.  (Here `bla' would be
  replaced by the output path of `bla'.)
2006-10-16 15:55:34 +00:00
Eelco Dolstra 7d4567f2cc * Removed URIs from the evaluator (NIX-66). They are now just another
kind of notation for strings.
2006-10-11 21:59:33 +00:00
Eelco Dolstra 0c4c5c2020 * Quick hack to fix NIX-67: evaluation result differing if the Nix
expression resides in the store.
2006-10-10 21:23:35 +00:00
Eelco Dolstra d20c3011a0 * toFile: added an additional argument to specify the store path
suffix, e.g., `builtins.toFile "builder.sh" "..."'.
* toFile: handle references to other files correctly.
2006-10-03 14:55:54 +00:00
Eelco Dolstra e347033f71 * The result of a concatenation with a derivation on the left-hand
side should be a path, I guess.
* Handle paths that are in the store but not direct children of the
  store directory.
* Ugh, hack to prevent double context wrapping.
2006-09-24 21:39:57 +00:00
Eelco Dolstra 2ab4bc44c7 * Builtin function `add' to add integers.
* Put common test functions in tests/lang/lib.nix.
2006-09-22 15:29:21 +00:00
Eelco Dolstra 75068e7d75 * Use a proper namespace.
* Optimise header file usage a bit.
* Compile the parser as C++.
2006-09-04 21:06:23 +00:00
Eelco Dolstra f93f7b75be * Okay, that's a bit harder than expected. 2006-08-30 13:10:04 +00:00
Eelco Dolstra 3151bdea55 * Uninitialised variable. 2006-08-30 12:00:27 +00:00
Eelco Dolstra 2132d9ddeb * Fix the ~ operator. 2006-08-29 15:29:38 +00:00
Eelco Dolstra 1fca76870b * Removed processBinding, instead we now apply toString to all
derivation attributes to flatten them into strings.  This is
  possible since string can nowadays be wrapped in contexts that
  describe the derivations/sources referenced by the evaluation of the
  string.
2006-08-28 13:31:06 +00:00
Eelco Dolstra da25d80152 * Strict evaluation and XML printing of lists. 2006-08-24 14:03:39 +00:00
Eelco Dolstra 943ab38a0d * Refactoring: move strictEval to libexpr. 2006-08-24 13:39:22 +00:00
Eelco Dolstra 38f18aa6d4 * New primop: abort "error message". 2006-08-23 15:46:00 +00:00
Eelco Dolstra 1854f84e83 * Fix a few warnings. 2006-08-04 17:07:13 +00:00
Eelco Dolstra 4661282fde * `nix-instantiate ... --arg NAME VALUE': allow arguments to be passed
to functions from the command line.
* nix-build: started removing backticks.
2006-07-28 16:03:28 +00:00
Eelco Dolstra ca2238cf81 * Refactoring: get the selection path stuff out of getDerivations()
and put it into a separate function findAlongAttrPath().
2006-07-26 15:05:15 +00:00
Eelco Dolstra 7a3a5d1608 * When there is a domain check, we have to evaluate the argument.
Can't be lazy!
2006-07-24 16:49:28 +00:00
Eelco Dolstra f4c5531d92 * New language feature: domain checks, which check whether a function
argument has a valid value, i.e., is in a certain domain.  E.g.,

    { foo : [true false]
    , bar : ["a" "b" "c"]
    }: ...

  This previously could be done using assertions, but domain checks
  will allow the buildfarm to automatically extract the configuration
  space from functions.
2006-07-24 16:35:34 +00:00
Eelco Dolstra 57751fdb55 * Refactoring to support domain checks. 2006-07-24 15:16:03 +00:00
Eelco Dolstra 4f3725b167 * Better error messages (especially wrt types). 2006-07-19 15:36:15 +00:00
Eelco Dolstra 5cabd47394 * Allow function argument default values to refer to other arguments
of the function.  Implements NIX-45.
2006-05-08 12:52:47 +00:00
Eelco Dolstra 310e605995 * Show evaluation stats when NIX_SHOW_STATS=1. 2006-05-08 10:00:37 +00:00
Eelco Dolstra 0832956089 * Use the new ATermMap. 2006-05-04 12:21:08 +00:00
Eelco Dolstra d300b4383d * Optimise null-ary term builders. Also declare all term builder
functions as pure, which might improve performance a bit.
2006-05-02 21:58:46 +00:00
Eelco Dolstra 68174bdc7d * Use a linked list of substitutions. This reduces the amount of
copying.
2006-05-02 21:39:02 +00:00
Eelco Dolstra dc719e6ba5 * Some preliminaries towards NIX-45. 2006-05-02 13:39:55 +00:00
Eelco Dolstra 0064599a27 * String interpolation. Expressions like
"--with-freetype2-library=" + freetype + "/lib"

  can now be written as

    "--with-freetype2-library=${freetype}/lib"

  An arbitrary expression can be enclosed within ${...}, not just
  identifiers.

* Escaping in string literals: \n, \r, \t interpreted as in C, any
  other character following \ is interpreted as-is.
  
* Newlines are now allowed in string literals.
2006-05-01 14:01:47 +00:00
Eelco Dolstra 6cecad2be0 * Allow string concatenations involving derivations, e.g.,
configureFlags = "--with-freetype2-library="
      + freetype + "/lib";
2006-05-01 09:56:56 +00:00
Eelco Dolstra 2b3b6c9b34 * In theory, this should reduce the number of ATermMap
re-allocations.
2006-03-10 16:14:13 +00:00
Eelco Dolstra 6dca5c9099 * When obtaining derivations from Nix expressions, ignore all
expressions that cause an assertion failure (like `assert system ==
  "i686-linux"').  This allows all-packages.nix in Nixpkgs to be used
  on all platforms, even if some Nix expressions don't work on all
  platforms.

  Not sure if this is a good idea; it's a bit hacky.  In particular,
  due to laziness some derivations might appear in `nix-env -qa' but
  disappear in `nix-env -qas' or `nix-env -i'.

  Commit 5000!
2006-03-08 16:03:58 +00:00
Eelco Dolstra 9088dee9e2 * Some refactoring of the exception handling code so that we can catch
Nix expression assertion failures.
2006-03-08 14:11:19 +00:00
Eelco Dolstra 991a130b1e * Added a list concatenation operator:
[1 2 3] ++ [4 5 6] => [1 2 3 4 5 6]
2005-07-25 15:05:34 +00:00
Eelco Dolstra cb7ccb528b * string2ATerm -> overloaded toATerm. 2004-11-03 18:12:03 +00:00
Eelco Dolstra a69534fc21 * Drop ATmake / ATMatcher also in handling store expressions. 2004-10-29 11:22:49 +00:00
Eelco Dolstra 5fe9222b36 * Don't use ATmake / ATmatch anymore, nor the ATMatcher class.
Instead we generate data bindings (build and match functions) for
  the constructors specified in `constructors.def'.  In particular
  this removes the conversions between AFuns and strings, and Nix
  expression evaluation now seems 3 to 4 times faster.
2004-10-26 22:54:26 +00:00
Eelco Dolstra 033d7c6593 * Doh! 2004-10-26 17:04:55 +00:00
Eelco Dolstra 9fa07b376d * String/path concatenation operator (`+'). 2004-10-26 17:01:35 +00:00
Eelco Dolstra 37d7abd694 * New language feature: with expressions.
The expression `with E1; E2' evaluates to E2 with all bindings in
  the attribute set E1 substituted.  E.g.,

    with {x = 123;}; x

  evaluates to 123.  That is, the attribute set E1 is in scope in E2.

  This is particularly useful when importing files containing lots
  definitions.  E.g., instead of

    let {
      inherit (import ./foo.nix) a b c d e f;

      body = ... a ... f ...;
    }

  we can now say

    with import ./foo.nix;

    ... a ... f ...

  I.e., we don't have to say what variables should be brought into scope.
2004-10-25 16:54:56 +00:00
Eelco Dolstra d8989b1fb4 * Every real language has a `map' function. 2004-08-04 11:27:53 +00:00
Eelco Dolstra bbfdd64741 * Allow primops with more that 1 arguments. 2004-08-04 10:59:20 +00:00
Eelco Dolstra 59b94ee18a * When something goes wrong in the evaluation of a Nix expression,
print a nice backtrace of the stack, rather than vomiting a gigantic
  (and useless) aterm on the screen.  Example:

    error: while evaluating file `.../pkgs/system/test.nix':
    while evaluating attribute `subversion' at `.../pkgs/system/all-packages-generic.nix', line 533:
    while evaluating function at `.../pkgs/applications/version-management/subversion/default.nix', line 1:
    assertion failed at `.../pkgs/applications/version-management/subversion/default.nix', line 13

  Since the Nix expression language is lazy, the trace may be
  misleading.  The purpose is to provide a hint as to the location of
  the problem.
2004-04-05 22:27:41 +00:00
Eelco Dolstra c4ac2a164a * The recent change in nixpkgs of calling `stdenv.mkDerivation'
instead of `derivation' triggered a huge slowdown in the Nix
  expression evaluator.  Total execution time of `nix-env -qa' went up
  by a factor of 60 or so.

  This scalability problem was caused by expressions such as

    (x: y: ... x ...) a b

  where `a' is a large term (say, the one in
  `all-packages-generic.nix').  Then the first beta-reduction would
  produce

    (y: ... a ...) b

  by substituting `a' for `x'.  The second beta-reduction would then
  substitute `b' for `y' into the body `... a ...', which is a large
  term due to `a', and thus causes a large traversal to be performed
  by substitute() in the second reduction.  This is however entirely
  redundant, since `a' cannot contain free variables (since we never
  substitute below a weak head normal form).

  The solution is to wrap substituted terms into a `Closed'
  constructor, i.e.,

    subst(subs, Var(x)) = Closed(e) iff subs[x] = e

  have substitution not descent into closed terms,

    subst(subs, Closed(x)) = Closed(x)

  and otherwise ignore them for evaluation,

    eval(Closed(x)) = eval(x).

* Fix a typo that caused incorrect substitutions to be performed in
  simple lambdas, e.g., `(x: x: x) a' would reduce to `(x: a)'.
2004-03-30 15:05:35 +00:00
Eelco Dolstra ac4d39f9db * Added an operator `?' to test for attribute existence, e.g.,
`attrs ? x' yields true iff `attrs' has an attribute named `x'.
2004-03-28 21:15:01 +00:00
Eelco Dolstra f958bcdf1f * Added an operator `~' to select paths within a derivation. E.g.,
{stdenv, bash}: derivation {
      builder = bash ~ /bin/sh;
      args = ["-e" "-x" ./builder.sh];
      ...
    }

  Here the attribute `builder' will evaluate to, e.g.,
  `/nix/store/1234abcd...-bash-2.0.1/bin/sh'.
2004-03-28 20:58:28 +00:00