Commit graph

797 commits

Author SHA1 Message Date
Shea Levy a8fb575c98 Share code between scopedImport and import
In addition to reducing duplication, this fixes both import from
derivation and import of derivation for scopedImport
2014-06-12 12:52:39 +02:00
Eelco Dolstra ee7fe64c0a == operator: Ignore string context
There really is no case I can think of where taking the context into
account is useful. Mostly it's just very inconvenient.
2014-06-10 14:02:56 +02:00
Eelco Dolstra becc2b0167 Sort nixPath attributes 2014-05-29 19:02:14 +02:00
Eelco Dolstra 54a34119f3 Use std::unordered_set 2014-05-26 17:53:17 +02:00
Eelco Dolstra d8c061e044 Remove ExprBuiltin
It's slower than ExprVar since it doesn't compute a static
displacement. Since we're not using the throw primop in the
implementation of <...> anymore, it's also not really needed.
2014-05-26 17:14:28 +02:00
Eelco Dolstra 62a6eeb1f3 Make the Nix search path declarative
Nix search path lookups like <nixpkgs> are now desugared to ‘findFile
nixPath <nixpkgs>’, where ‘findFile’ is a new primop. Thus you can
override the search path simply by saying

  let
    nixPath = [ { prefix = "nixpkgs"; path = "/my-nixpkgs"; } ];
  in ... <nixpkgs> ...

In conjunction with ‘scopedImport’ (commit
c273c15cb1), the Nix search path can be
propagated across imports, e.g.

  let

    overrides = {
      nixPath = [ ... ] ++ builtins.nixPath;
      import = fn: scopedImport overrides fn;
      scopedImport = attrs: fn: scopedImport (overrides // attrs) fn;
      builtins = builtins // overrides;
    };

  in scopedImport overrides ./nixos
2014-05-26 17:02:22 +02:00
Eelco Dolstra 39d72640c2 Ensure that -I flags get included in nixPath
Also fixes #261.
2014-05-26 16:52:31 +02:00
Eelco Dolstra a8edf185a9 Add constant ‘nixPath’
It contains the Nix expression search path as a list of { prefix, path
} sets, e.g.

  [ { path = "/nix/var/nix/profiles/per-user/root/channels/nixos"; prefix = ""; }
    { path = "/etc/nixos/configuration.nix"; prefix = "nixos-config"; }
    { path = "/home/eelco/Dev/nix/inst/share/nix/corepkgs"; prefix = "nix"; }
  ]
2014-05-26 14:55:47 +02:00
Eelco Dolstra c273c15cb1 Add primop ‘scopedImport’
‘scopedImport’ works like ‘import’, except that it takes a set of
attributes to be added to the lexical scope of the expression,
essentially extending or overriding the builtin variables.  For
instance, the expression

  scopedImport { x = 1; } ./foo.nix

where foo.nix contains ‘x’, will evaluate to 1.

This has a few applications:

* It allows getting rid of function argument specifications in package
  expressions. For instance, a package expression like:

    { stdenv, fetchurl, libfoo }:

    stdenv.mkDerivation { ... buildInputs = [ libfoo ]; }

  can now we written as just

    stdenv.mkDerivation { ... buildInputs = [ libfoo ]; }

  and imported in all-packages.nix as:

    bar = scopedImport pkgs ./bar.nix;

  So whereas we once had dependencies listed in three places
  (buildInputs, the function, and the call site), they now only need
  to appear in one place.

* It allows overriding builtin functions. For instance, to trace all
  calls to ‘map’:

  let
    overrides = {
      map = f: xs: builtins.trace "map called!" (map f xs);

      # Ensure that our override gets propagated by calls to
      # import/scopedImport.
      import = fn: scopedImport overrides fn;

      scopedImport = attrs: fn: scopedImport (overrides // attrs) fn;

      # Also update ‘builtins’.
      builtins = builtins // overrides;
    };
  in scopedImport overrides ./bla.nix

* Similarly, it allows extending the set of builtin functions. For
  instance, during Nixpkgs/NixOS evaluation, the Nixpkgs library
  functions could be added to the default scope.

There is a downside: calls to scopedImport are not memoized, unlike
import. So importing a file multiple times leads to multiple parsings
/ evaluations. It would be possible to construct the AST only once,
but that would require careful handling of variables/environments.
2014-05-26 14:26:29 +02:00
Eelco Dolstra f0fdbd0897 Shut up some signedness warnings 2014-05-26 12:34:15 +02:00
Shea Levy eac5841970 Provide a more useful error message when a dynamic attr lookup fails 2014-05-15 17:56:24 +02:00
Eelco Dolstra dfa2f77d2e If a .drv cannot be parsed, show its path
Otherwise you just get ‘expected string `Derive(['’ which isn't very helpful.
2014-04-08 19:24:29 +02:00
Eelco Dolstra 8e5fbf4d73 Show position info in attribute selection errors 2014-04-04 22:52:14 +02:00
Eelco Dolstra 4c5faad994 Show position info in Boolean operations 2014-04-04 22:43:52 +02:00
Eelco Dolstra bd9b1d97b4 Show position info in string concatenation / addition errors 2014-04-04 22:19:33 +02:00
Eelco Dolstra a5fe730940 forceString: Show position info 2014-04-04 21:14:11 +02:00
Eelco Dolstra 27b44b8cf7 forceAttrs: Show position info 2014-04-04 19:11:40 +02:00
Eelco Dolstra 96b695ccab forceList: Show position info 2014-04-04 19:05:36 +02:00
Eelco Dolstra b62d36963c forceInt: Show position info 2014-04-04 18:59:29 +02:00
Eelco Dolstra c28de6d96e Pass position information to primop calls
For example:

  error: `tail' called on an empty list, at
    /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/ex-2/default.nix:13:7
2014-04-04 18:59:29 +02:00
Eelco Dolstra 8b31ffd10d Remove unnecessary quotes around file names 2014-04-04 18:59:29 +02:00
Eelco Dolstra b72c8d2e5b Include position info in function application
This allows error messages like:

  error: the anonymous function at `/etc/nixos/configuration.nix:1:1'
    called without required argument `foo', at
    `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/lib/modules.nix:77:59'
2014-04-04 18:59:29 +02:00
Eelco Dolstra ac6ceea764 Fix potential segfault
The newEnv variable was accessed (via the dynamicEnv) pointer after it
had gone out of scope.

Fixes #234.
2014-04-01 17:04:38 +02:00
Eelco Dolstra 1c2550a2ae boost::shared_ptr -> std::shared_ptr 2014-03-30 00:49:23 +01:00
Eelco Dolstra 9becaa041f Drop pointless #include 2014-03-29 22:20:33 +01:00
Shea Levy 049a379ec6 The expr of AttrNames/DynamicAttrDefs is always an ExprConcatStrings 2014-03-10 10:14:50 +01:00
Shea Levy 908e9ce259 If a dynamic attribute name evaluates to null, remove it from the set 2014-03-10 10:14:50 +01:00
Eelco Dolstra 2caab81660 Revert "Make ifs and asserts tail-recursive"
This reverts commit 273322c773.
2014-03-05 16:18:13 +01:00
Eelco Dolstra c9f6232304 Correctly detect infinite recursion in function application
If we're evaluating some application ‘v = f x’, we can't store ‘f’
temporarily in ‘v’, because if ‘f x’ refers to ‘v’, it will get ‘f’
rather than an infinite recursion error.

Unfortunately, this breaks the tail call optimisation introduced in
c897bac549.

Fixes #217.
2014-02-27 21:47:59 +01:00
Eelco Dolstra f14ef84a51 Warn about missing -I paths
Fixes #121.  Note that we don't warn about missing $NIX_PATH entries
because it's intended that some may be missing (cf. the default
$NIX_PATH on NixOS, which includes paths like /etc/nixos/nixpkgs for
backward compatibility).
2014-02-26 15:21:56 +01:00
Shea Levy a7e70518b8 lexer-tab.o and parser-tab.o require each other's headers 2014-02-18 17:31:30 +01:00
Eelco Dolstra 844d83352c More "make dist" fixes 2014-02-01 15:18:48 +01:00
Eelco Dolstra 6ef32bddc1 Fix "make dist" 2014-02-01 14:38:12 +01:00
Eelco Dolstra 0c6d62cf27 Remove Automakefiles 2014-02-01 13:54:38 +01:00
Eelco Dolstra 16e7d69209 Update Makefile variable names 2014-02-01 13:54:38 +01:00
Eelco Dolstra e0234dfddc Rename Makefile -> local.mk 2014-01-30 12:11:06 +01:00
Eelco Dolstra 94f9c14d52 Fix some clang warnings 2014-01-21 18:29:55 +01:00
Eelco Dolstra 68cde53c47 Fix building against Bison 3.0.2 2014-01-21 15:34:04 +01:00
Eelco Dolstra 81628a6ccc Merge branch 'master' into make
Conflicts:
	src/libexpr/eval.cc
2014-01-21 15:30:01 +01:00
Shea Levy f5e5793cd2 Bare dynamic attrs: Match interpolation semantics
Signed-off-by: Shea Levy <shea@shealevy.com>
2014-01-14 14:00:15 +01:00
Shea Levy f9913f4422 Allow "bare" dynamic attrs
Now, in addition to a."${b}".c, you can write a.${b}.c (applicable
wherever dynamic attributes are valid).

Signed-off-by: Shea Levy <shea@shealevy.com>
2014-01-14 14:00:15 +01:00
Eelco Dolstra b4c684e0f9 Update Makefiles 2014-01-09 16:53:47 +01:00
Shea Levy cd49fe4f9b Don't use any syntactic sugar for dynamic attrs
This doesn't change any functionality but moves some behavior out of the
parser and into the evaluator in order to simplify the code.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-12-31 23:56:26 +00:00
Shea Levy 6f3a51809a Fold dynamic binds handling into addAttr
Since addAttr has to iterate through the AttrPath we pass it, it makes
more sense to just iterate through the AttrNames in addAttr instead. As
an added bonus, this allows attrsets where two dynamic attribute paths
have the same static leading part (see added test case for an example
that failed previously).

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-12-31 17:57:10 -05:00
Shea Levy 18fefacf7d Dynamic attrs
This adds new syntax for attribute names:

* attrs."${name}" => getAttr name attrs
* attrs ? "${name}" => isAttrs attrs && hasAttr attrs name
* attrs."${name}" or def => if attrs ? "${name}" then attrs."${name}" else def
* { "${name}" = value; } => listToAttrs [{ inherit name value; }]

Of course, it's a bit more complicated than that. The attribute chains
can be arbitrarily long and contain combinations of static and dynamic
parts (e.g. attrs."${foo}".bar."${baz}" or qux), which is relatively
straightforward for the getAttrs/hasAttrs cases but is more complex for
the listToAttrs case due to rules about duplicate attribute definitions.

For attribute sets with dynamic attribute names, duplicate static
attributes are detected at parse time while duplicate dynamic attributes
are detected when the attribute set is forced. So, for example, { a =
null; a.b = null; "${"c"}" = true; } will be a parse-time error, while
{ a = {}; "${"a"}".b = null; c = true; } will be an eval-time error
(technically that case could theoretically be detected at parse time,
but the general case would require full evaluation). Moreover, duplicate
dynamic attributes are not allowed even in cases where they would be
with static attributes ({ a.b.d = true; a.b.c = false; } is legal, but {
a."${"b"}".d = true; a."${"b"}".c = false; } is not). This restriction
might be relaxed in the future in cases where the static variant would
not be an error, but it is not obvious that that is desirable.

Finally, recursive attribute sets with dynamic attributes have the
static attributes in scope but not the dynamic ones. So rec { a = true;
"${"b"}" = a; } is equivalent to { a = true; b = true; } but rec {
"${"a"}" = true; b = a; } would be an error or use a from the
surrounding scope if it exists.

Note that the getAttr, getAttr or default, and hasAttr are all
implemented purely in the parser as syntactic sugar, while attribute
sets with dynamic attribute names required changes to the AST to be
implemented cleanly.

This is an alternative solution to and closes #167

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-12-31 20:59:49 +00:00
Shea Levy 136f2f7046 Add the ExprBuiltin Expr type to the AST
Certain desugaring schemes may require the parser to use some builtin
function to do some of the work (e.g. currently `throw` is used to
lazily cause an error if a `<>`-style path is not in the search path)
Unfortunately, these names are not reserved keywords, so an expression
that uses such a syntactic sugar will not see the expected behavior
(see tests/lang/eval-okay-redefine-builtin.nix for an example).

This adds the ExprBuiltin AST type, which when evaluated uses the value
from the rootmost variable scope (which of course is initialized
internally and can't shadow any of the builtins).

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-12-31 17:45:47 +00:00
Shea Levy e36229d27f Bump language version for new storePath feature
This will allow e.g. channel expressions to use builtins.storePath IFF
it is safe to do so without knowing if the path is valid yet.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-12-10 15:07:04 +01:00
Shea Levy 22d665019a builtins.storePath: Try to substitute the path if it is not yet valid
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-12-10 15:07:04 +01:00
Eelco Dolstra 7ce0e05ad8 Rename Makefile.new -> Makefile 2013-11-25 15:25:13 +00:00
Eelco Dolstra 2bd0fcc966 Use libnix as a prefix for all Nix libraries
In particular "libutil" was always a problem because it collides with
Glibc's libutil.  Even if we install into $(libdir)/nix, the linker
sometimes got confused (e.g. if a program links against libstore but
not libutil, then ld would report undefined symbols in libstore
because it was looking at Glibc's libutil).
2013-11-23 23:53:41 +00:00
Eelco Dolstra 5a1114ecdb Drop the dependency on libgc in libmain
Instead, libexpr now depends on libgc.  This means commands like
nix-store that don't do any evaluation no longer require libgc.
2013-11-23 20:19:36 +00:00
Eelco Dolstra 06a8ac96e7 Initialise Boehm GC only once 2013-11-23 20:15:57 +00:00
Eelco Dolstra 90dfb37f14 Allow (dynamic) libraries to depend on other libraries 2013-11-23 20:11:02 +00:00
Eelco Dolstra 611868a909 Implement basic ‘make install’ 2013-11-23 17:04:27 +00:00
Eelco Dolstra 754c05ed6c Rename $(here) to $(d) for brevity, and remove trailing slash 2013-11-22 16:45:52 +00:00
Eelco Dolstra 1474ecfe42 Generate the parser and the lexer 2013-11-22 16:22:31 +01:00
Eelco Dolstra b8e9efc476 New non-recursive, plain Make-based build system 2013-11-22 15:54:18 +01:00
Eelco Dolstra 30b986908e Check meta values and warn about bad ones 2013-11-19 14:29:39 +01:00
Eelco Dolstra 0f24400d90 Generalise meta attributes 2013-11-19 14:09:14 +01:00
Eelco Dolstra 990126cde0 Shorter error message 2013-11-19 14:09:14 +01:00
Eelco Dolstra 5fea98111b Refactor JSON output 2013-11-19 00:33:06 +01:00
Eelco Dolstra 77c13cdf56 Add a toJSON primop 2013-11-19 00:04:11 +01:00
Eelco Dolstra 285df765b9 Add a primop unsafeGetAttrPos to return the position of an attribute 2013-11-18 22:22:35 +01:00
Eelco Dolstra fc33fd86b7 Add a symbol __curPos that expands to the current source location
I.e. an attribute set { file = <string>; line = <int>; column = <int>; }.
2013-11-18 20:16:02 +01:00
Eelco Dolstra 90b5e69284 Support quoted attribute names in -A
This is requires if you have attribute names with dots in them.  So
you can now say:

  $ nix-instantiate '<nixos>' -A 'config.systemd.units."postgresql.service".text' --eval-only

Fixes #151.
2013-11-18 10:21:12 +00:00
Eelco Dolstra 89e6781cc5 Make function calls show up in stack traces again
Note that adding --show-trace prevents functions calls from being
tail-recursive, so an expression that evaluates without --show-trace
may fail with a stack overflow if --show-trace is given.
2013-11-12 12:51:59 +01:00
Eelco Dolstra c897bac549 Make function calls tail-recursive 2013-11-12 11:32:23 +00:00
Eelco Dolstra 273322c773 Make ifs and asserts tail-recursive
The local Value object prevented g++ from making a tail call.  Not
clear why.  In any case, not using a temporary makes g++ do the tail
call.
2013-11-12 11:32:23 +00:00
Eelco Dolstra 4badd7ed17 Get rid of an intermediary on the stack 2013-11-12 11:32:23 +00:00
Eelco Dolstra 8d6418d46e Fix building without a garbage collector
http://hydra.nixos.org/build/6695350
2013-10-28 22:51:12 +01:00
Eelco Dolstra dec2f19502 Fix a segfault in genericClosure
It kept temporary data in STL containers that were not scanned by
Boehm GC, so Nix programs using genericClosure could randomly crash if
the garbage collector kicked in at a bad time.

Also make it a bit more efficient by copying points to values rather
than values.
2013-10-28 18:52:26 +01:00
Eelco Dolstra ea6bf0c21f Slightly optimize listToAttrs 2013-10-28 07:34:44 +01:00
Eelco Dolstra 5bc41d78ff Rename "attribute sets" to "sets"
We don't have any other kind of sets so calling them attribute sets is
unnecessarily verbose.
2013-10-24 16:41:04 +02:00
Eelco Dolstra 69befd33a9 Remove unnecessary call to forceStringNoCtx 2013-10-24 03:08:34 +02:00
Eelco Dolstra 05d02f798f Add a typeOf primop
We already have some primops for determining the type of a value, such
as isString, but they're incomplete: for instance, there is no isPath.
Rather than adding more isBla functions, the generic typeOf function
returns a string representing the type of the argument (e.g. "int").
2013-10-24 02:49:13 +02:00
Eelco Dolstra 543d8a5942 Don't require NIX_SHOW_STATS for NIX_COUNT_CALLS 2013-10-24 02:20:54 +02:00
Eelco Dolstra fe95650487 Memoize evalFile() lookups under both the original and resolved name
Previously we only used the resolved name, causing repeated resolution
(e.g. /dir to /dir/default.nix).
2013-10-23 11:19:01 +00:00
Eelco Dolstra 792fd51f41 Fold two stack trace messages in derivations
Combined with the previous changes, stack traces involving derivations
are now much less verbose, since something like

  while evaluating the builtin function `getAttr':
  while evaluating the builtin function `derivationStrict':
  while instantiating the derivation named `gtk+-2.24.20' at `/home/eelco/Dev/nixpkgs/pkgs/development/libraries/gtk+/2.x.nix:11:3':
  while evaluating the derivation attribute `propagatedNativeBuildInputs' at `/home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/default.nix:78:17':
  while evaluating the attribute `outPath' at `/nix/store/212ngf4ph63mp6p1np2bapkfikpakfv7-nix-1.6/share/nix/corepkgs/derivation.nix:18:9':
  ...

now reads

  while evaluating the attribute `propagatedNativeBuildInputs' of the derivation `gtk+-2.24.20' at `/home/eelco/Dev/nixpkgs/pkgs/development/libraries/gtk+/2.x.nix:11:3':
  ...
2013-10-17 11:57:25 +02:00
Eelco Dolstra f440558acc Don't show <nix/derivation.nix> in stack traces
Messages like

  while evaluating the attribute `outPath' at `/nix/store/212ngf4ph63mp6p1np2bapkfikpakfv7-nix-1.6/share/nix/corepkgs/derivation.nix:18:9':

are redundant, because Nix already shows that it's evaluating a derivation:

  while instantiating the derivation named `firefox-24.0' at `/home/eelco/Dev/nixpkgs/pkgs/applications/networking/browsers/firefox/default.nix:131:5':
  while evaluating the derivation attribute `nativeBuildInputs' at `/home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/default.nix:76:17':
2013-10-17 11:47:38 +02:00
Eelco Dolstra b8034e5581 Ensure proper type checking/coercion of "${expr}"
Now we only rewrite "${expr}" to expr if expr is a string literal.
2013-10-17 00:57:24 +02:00
Eelco Dolstra d6a7aa8f48 Revert the behaviour of antiquoted paths to pre-Nix 1.6
Commit 159e621d1a accidentally changed
the behaviour of antiquoted paths, e.g.

  "${/foo}/bar"

used to evaluate to "/nix/store/<hash>-foo/bar" (where /foo gets
copied to the store), but in Nix 1.6 it evaluates to "/foo/bar".  This
is inconsistent, since

  " ${/foo}/bar"

evaluates to " /nix/store/<hash>-foo/bar".  So revert to the old
behaviour.
2013-10-17 00:39:59 +02:00
Eelco Dolstra 7bdb85453d printStats(): Print the size of the symbol table in bytes 2013-10-08 15:37:08 +02:00
Eelco Dolstra 9deb822180 Deduplicate filenames in Pos
This saves ~4 MiB of RAM for NixOS system instantiation, and ~18 MiB
for "nix-env -qa".
2013-10-08 15:36:10 +02:00
Eelco Dolstra b1e3b1a4ac Treat undefined variable errors consistently
Previously, a undefined variable inside a "with" caused an EvalError
(which can be caught), while outside, it caused a ParseError (which
cannot be caught).  Now both cause an UndefinedVarError (which cannot
be caught).
2013-10-08 14:45:36 +02:00
Eelco Dolstra 6b47de580f Show the exact position of undefined variables
In particular, undefined variable errors in a "with" previously didn't
show *any* position information, so this should help a lot in those
cases.
2013-10-08 14:40:51 +02:00
Eelco Dolstra a5e0f64db3 Remove some unused functions 2013-10-08 12:30:23 +00:00
Eelco Dolstra 221a2daf34 Merge VarRef into ExprVar 2013-10-08 14:24:53 +02:00
Eelco Dolstra 176c666f36 Don't show calls to primops in stack traces
Since they don't have location information, they just give you crap
like:

  while evaluating the builtin function `getAttr':
  while evaluating the builtin function `derivationStrict':
  ...
2013-10-07 18:02:32 +02:00
Eelco Dolstra c945f015de Fix segfault in nix-repl / hydra-eval-jobs
If a "with" attribute set fails to evaluate, we have to make sure its
Env record remains unchanged.  Otherwise, repeated evaluation gives a
segfault:

  nix-repl> :a with 0; { a = x; b = x; }
  Added 2 variables.

  nix-repl> a
  error: value is an integer while an attribute set was expected

  nix-repl> b
  Segmentation fault
2013-10-02 15:24:45 +02:00
Eelco Dolstra 88c07341a6 nix-env: Use wildcard match by default
That is, you don't need to pass '*' anymore, so

  nix-env -qa

is equivalent to

  nix-env -qa '*'
2013-09-03 16:35:49 +02:00
Eelco Dolstra 07a08bddf0 nix-env: Load files in ~/.nix-defexpr on demand
So if you do "nix-env -qa -A nixos", then other channels won't be
parsed/evaluated at all.
2013-09-03 15:45:32 +02:00
Eelco Dolstra ef4f5ba85e Work on Values instead of Exprs
This prevents some duplicate evaluation in nix-env and
nix-instantiate.

Also, when traversing ~/.nix-defexpr, only read regular files with the
extension .nix.  Previously it was reading files like
.../channels/binary-caches/<name>.  The only reason this didn't cause
problems is pure luck (namely, <name> shadows an actual Nix
expression, the binary-caches files happen to be syntactically valid
Nix expressions, and we iterate over the directory contents in just
the right order).
2013-09-03 13:17:51 +00:00
Eelco Dolstra 6f809194d7 Get rid of the parse tree cache
Since we already cache files in normal form (fileEvalCache), caching
parse trees is redundant.

Note that getting rid of this cache doesn't actually save much memory
at the moment, because parse trees are currently not freed / GC'ed.
2013-09-03 13:01:42 +02:00
Eelco Dolstra 57d18df7d0 Add some support code for nix-repl 2013-09-02 18:34:04 +02:00
Eelco Dolstra 92077b4547 Get rid of a signedness warning 2013-09-02 16:39:17 +02:00
Eelco Dolstra 33972629d7 Fix whitespace 2013-09-02 16:29:15 +02:00
Shea Levy afc6c1bad6 Simplify inherited attribute handling
This reduces the difference between inherited and non-inherited
attribute handling to the choice of which env to use (in recs and lets)
by setting the AttrDef::e to a new ExprVar in the parser rather than
carrying a separate AttrDef::v VarRef member.

As an added bonus, this allows inherited attributes that inherit from a
with to delay forcing evaluation of the with's attributes.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-08-26 11:31:56 +02:00
Ivan Kozik 34bb806f74 Fix typos, especially those that end up in the Nix manual 2013-08-26 11:15:22 +02:00
Eelco Dolstra d308aeaf53 Store Nix integers as longs
So on 64-bit systems, integers are now 64-bit.

Fixes #158.
2013-08-19 12:35:03 +02:00
Eelco Dolstra 46222bbc43 Typo 2013-08-14 22:32:49 +02:00
Eelco Dolstra fd7d979c79 Remove obsolete reference to ATerms 2013-08-06 14:15:11 +02:00
Eelco Dolstra 46ffcce0c3 In the profiler output, show function names (if available) 2013-08-02 18:29:23 +00:00
Eelco Dolstra 8e74c0bfd1 Let the ordering operators also work on strings
E.g. ‘"foo" < "bar"’ now works.
2013-08-02 18:53:02 +02:00
Eelco Dolstra 3d77b28eac Add comparison operators ‘<’, ‘<=’, ‘>’ and ‘>=’ 2013-08-02 18:39:40 +02:00
Eelco Dolstra 47701677e8 Add integer ‘-’, ‘*’ and ‘/’ operators 2013-08-02 16:03:02 +00:00
Eelco Dolstra 5d147e125c Add a unary integer negation operator
This allows saying "-1" instead of "builtins.sub 0 1".
2013-08-02 15:43:31 +00:00
Eelco Dolstra 159e621d1a Overload the ‘+’ operator to support integer addition 2013-08-02 15:21:17 +00:00
Eelco Dolstra 0a470fc345 Make Env smaller
Commit 20866a7031 added a ‘withAttrs’
field to Env, which is annoying because it makes every Env structure
bigger and we allocate millions of them.  E.g. NixOS evaluation took
18 MiB more.  So this commit squeezes ‘withAttrs’ into values[0].
Probably should use a union...
2013-07-31 12:44:21 +02:00
Eelco Dolstra 8ae6d55db1 Don't use NULL 2013-07-31 12:11:14 +02:00
Shea Levy e068f49f7d Avoid thunks when a fromWith var can be looked up without evaluation
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-31 11:48:39 +02:00
Shea Levy 20866a7031 Delay evaulation of with attrs until a variable lookup needs them
Evaluation of attribute sets is strict in the attribute names, which
means immediate evaluation of `with` attribute sets rules out some
potentially interesting use cases (e.g. where the attribute names of one
set depend in some way on another but we want to bring those names into
scope for some values in the second set).

The major example of this is overridable self-referential package sets
(e.g. all-packages.nix). With immediate `with` evaluation, the only
options for such sets are to either make them non-recursive and
explicitly use the name of the overridden set in non-overridden one
every time you want to reference another package, or make the set
recursive and use the `__overrides` hack. As shown in the test case that
comes with this commit, though, delayed `with` evaluation allows a nicer
third alternative.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-31 11:48:39 +02:00
Eelco Dolstra 18a48d80a0 Show function names in error messages
Functions in Nix are anonymous, but if they're assigned to a
variable/attribute, we can use the variable/attribute name in error
messages, e.g.

while evaluating `concatMapStrings' at `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/strings.nix:18:25':
...
2013-05-16 19:08:02 +02:00
Eelco Dolstra 1b3a03f161 Show which function argument was unexpected
Fixes #116.
2013-05-16 17:56:14 +02:00
Eelco Dolstra 229567293c Shut up a compiler warning 2013-05-16 17:48:19 +02:00
Eelco Dolstra 804709706c Fix building against Bison 2.6 2013-03-14 18:31:08 +01:00
Eelco Dolstra c56bc3d81c Make sure that thunks are restored properly if an exception occurs
Fixes Hydra bug #67.
2013-03-14 17:21:13 +01:00
Eelco Dolstra bdd4646338 Revert "Prevent config.h from being clobbered"
This reverts commit 28bba8c44f.
2013-03-08 01:24:59 +01:00
Eelco Dolstra 28bba8c44f Prevent config.h from being clobbered 2013-03-07 23:55:55 +01:00
Eelco Dolstra 5f18cd2e84 Make "${./path} ..." evaluate to a string, not a path
Wacky string coercion semantics caused expressions like

  exec = "${./my-script} params...";

to evaluate to a path (‘/path/my-script params’), because
anti-quotations are desuged to string concatenation:

  exec = ./my-script + " params...";

By constrast, adding a space at the start would yield a string as
expected:

  exec = " ${./my-script} params...";

Now the first example also evaluates to a string.
2013-02-08 20:04:04 +01:00
Eelco Dolstra 52172607cf Rename "hash" to "hashString" and handle SHA-1 2013-02-08 19:36:23 +01:00
Marc Weber 01a5ea9914 experimental/hash
adding primop function calculating hash of a string

Signed-off-by: Marc Weber <marco-oweber@gmx.de>
2013-02-08 19:26:34 +01:00
Stuart Pernsteiner a9045c727f fix use-after-free bug in mkString(Value&, Symbol&) 2012-12-13 11:13:20 +01:00
Eelco Dolstra 3100b29fc5 Tiny optimisation in the filter primop 2012-12-04 17:22:20 +01:00
Eelco Dolstra 8eed07cda4 nix-env -q --out-path: Support multiple outputs
We now print all output paths of a package, e.g.

  openssl-1.0.0i  bin=/nix/store/gq2mvh0wb9l90djvsagln3aqywqmr6vl-openssl-1.0.0i-bin;man=/nix/store/7zwf5r5hsdarl3n86dasvb4chm2xzw9n-openssl-1.0.0i-man;/nix/store/cj7xvk7fjp9q887359j75pw3pzjfmqf1-openssl-1.0.0i

or (in XML mode)

  <item attrPath="openssl" name="openssl-1.0.0i" system="x86_64-linux">
    <output name="bin" path="/nix/store/gq2mvh0wb9l90djvsagln3aqywqmr6vl-openssl-1.0.0i-bin" />
    <output name="man" path="/nix/store/7zwf5r5hsdarl3n86dasvb4chm2xzw9n-openssl-1.0.0i-man" />
    <output name="out" path="/nix/store/cj7xvk7fjp9q887359j75pw3pzjfmqf1-openssl-1.0.0i" />
  </item>
2012-11-28 13:49:44 +01:00
Eelco Dolstra 6c98e6a5de Optionally ignore null-valued derivation attributes
This allows adding attributes like

  attr = if stdenv.system == "bla" then something else null;

without changing the resulting derivation on non-<bla> platforms.

We once considered adding a special "ignore" value for this purpose,
but using null seems more elegant.
2012-11-27 15:01:32 +01:00
Eelco Dolstra 8b8ee53bc7 Add builtin constants ‘langVersion’ and ‘nixVersion’
The integer constant ‘langVersion’ denotes the current language
version.  It gets increased every time a language feature is
added/changed/removed.  It's currently 1.

The string constant ‘nixVersion’ contains the current Nix version,
e.g. "1.2pre2980_9de6bc5".
2012-11-27 13:29:55 +01:00
Eelco Dolstra 408a7bfac1 nix-instantiate: Fix read-only evaluation 2012-11-26 17:41:10 +01:00
Eelco Dolstra 17dc306aa3 Revert "prim_toPath: Actually make the string a path"
This reverts commit 2980d1fba9.  It
causes a regression in NixOS evaluation:

string `/nix/store/ya3s5gmj3b28170fpbjhgsk8wzymkpa1-pommed-1.39/etc/pommed.conf' cannot refer to other paths
2012-11-15 23:04:34 +01:00
Eelco Dolstra 88164325fa Fix a segfault when auto-calling a "a@{...}" function
Since the called function can return its argument attribute set
(e.g. "a"), the latter should not be allocated on the stack.

Reported by Shea.
2012-11-09 15:09:31 +01:00
Shea Levy 2980d1fba9 prim_toPath: Actually make the string a path 2012-10-23 16:39:35 +02:00
Eelco Dolstra 70f75be199 getDerivation(): Don't always quietly ignore assertion failure
Ignoring assertion failures makes some sense for nix-env -qa, but not
for nix-instantiate/nix-build or hydra-eval-jobs.
2012-10-04 15:22:25 -04:00
Eelco Dolstra 0a7084567f Add a ‘--repair’ flag to nix-instantiate
This allows repairing corrupted derivations and other source files.
2012-10-03 15:09:18 -04:00
Eelco Dolstra 95c74eae26 Allow dashes in identifiers
In Nixpkgs, the attribute in all-packages.nix corresponding to a
package is usually equal to the package name.  However, this doesn't
work if the package contains a dash, which is fairly common.  The
convention is to replace the dash with an underscore (e.g. "dbus-lib"
becomes "dbus_glib"), but that's annoying.  So now dashes are valid in
variable / attribute names, allowing you to write:

  dbus-glib = callPackage ../development/libraries/dbus-glib { };

and

  buildInputs = [ dbus-glib ];

Since we don't have a negation or subtraction operation in Nix, this
is unambiguous.
2012-09-27 15:49:20 -04:00
Eelco Dolstra 76e88871b2 Templatise tokenizeString() 2012-09-19 15:43:23 -04:00
Eelco Dolstra e94806d030 Merge branch 'master' into no-manifests 2012-08-27 11:09:07 -04:00
Eelco Dolstra 767101824a Avoid concatenating lists of one string 2012-08-13 15:10:29 -04:00
Eelco Dolstra e5c589d271 Don't allocate empty lists
This saves about 4 MB when evaluating a NixOS system configuration.
2012-08-13 15:02:09 -04:00
Eelco Dolstra 3e89ef597c Optimise concatenating a list to an empty list
More precisely, in concatLists, if all lists except one are empty,
then just return the non-empty list.  This reduces the number of list
element allocations by 32% when evaluating a NixOS system
configuration.
2012-08-13 14:58:54 -04:00
Eelco Dolstra 9c2d63084b Add a primop ‘elemAt’ to get an element from a list 2012-08-13 13:46:42 -04:00
Eelco Dolstra 198d0338be Add a primop ‘concatLists’
This can serve as a generic efficient list builder.  For instance, the
function ‘catAttrs’ in Nixpkgs can be rewritten from

  attr: l: fold (s: l: if hasAttr attr s then [(getAttr attr s)] ++ l else l) [] l

to

  attr: l: builtins.concatLists (map (s: if hasAttr attr s then [(getAttr attr s)] else []) l)

Statistics before:

  time elapsed: 1.08683
  size of a value: 24
  environments allocated: 1384376 (35809568 bytes)
  list elements: 6946783 (55574264 bytes)
  list concatenations: 37434
  values allocated: 1760440 (42250560 bytes)
  attribute sets allocated: 392040
  right-biased unions: 186334
  values copied in right-biased unions: 591137
  symbols in symbol table: 18273
  number of thunks: 1297673
  number of thunks avoided: 1380759
  number of attr lookups: 430802
  number of primop calls: 628912
  number of function calls: 1333544

Statistics after (including new catAttrs):

  time elapsed: 0.959854
  size of a value: 24
  environments allocated: 1010198 (26829296 bytes)
  list elements: 1984878 (15879024 bytes)
  list concatenations: 30488
  values allocated: 1589760 (38154240 bytes)
  attribute sets allocated: 392040
  right-biased unions: 186334
  values copied in right-biased unions: 591137
  symbols in symbol table: 18274
  number of thunks: 1040925
  number of thunks avoided: 1038428
  number of attr lookups: 438419
  number of primop calls: 474844
  number of function calls: 959366
2012-08-13 01:53:10 -04:00
Eelco Dolstra b9e5b908ed Provide an efficient implementation of ‘elem’
The one in Nixpkgs is O(n^2), this one is O(n).  Big reduction in the
number of list allocations.

Statistics before (on a NixOS system config):

  time elapsed: 1.17982
  size of a value: 24
  environments allocated: 1543334 (39624560 bytes)
  list elements: 9612638 (76901104 bytes)
  list concatenations: 37434
  values allocated: 1854933 (44518392 bytes)
  attribute sets allocated: 392040
  right-biased unions: 186334
  values copied in right-biased unions: 591137
  symbols in symbol table: 18272
  number of thunks: 1392467
  number of thunks avoided: 1507311
  number of attr lookups: 430801
  number of primop calls: 691600
  number of function calls: 1492502

Statistics after:

  time elapsed: 1.08683
  size of a value: 24
  environments allocated: 1384376 (35809568 bytes)
  list elements: 6946783 (55574264 bytes)
  list concatenations: 37434
  values allocated: 1760440 (42250560 bytes)
  attribute sets allocated: 392040
  right-biased unions: 186334
  values copied in right-biased unions: 591137
  symbols in symbol table: 18273
  number of thunks: 1297673
  number of thunks avoided: 1380759
  number of attr lookups: 430802
  number of primop calls: 628912
  number of function calls: 1333544
2012-08-13 01:05:35 -04:00
Eelco Dolstra 4ccd48ce24 Add a "filter" primop
Evaluation of a NixOS configuration spends quite a lot of time in the
"filter" function in Nixpkgs.  As implemented in Nixpkgs, this is a
O(n^2) operation, so it's a good candidate for providing a more
efficient (i.e. primop) implementation.  Using it gives a ~10% speed
increase and a significant reduction in the number of evaluations.

Statistics before (on a NixOS system config):

  time elapsed: 1.3258
  size of a value: 24
  environments allocated: 1980939 (50127080 bytes)
  list elements: 14679308 (117434464 bytes)
  list concatenations: 50828
  values allocated: 2098938 (50374512 bytes)
  attribute sets allocated: 392040
  right-biased unions: 186334
  values copied in right-biased unions: 591137
  symbols in symbol table: 18271
  number of thunks: 1645752
  number of thunks avoided: 1921196
  number of attr lookups: 430798
  number of primop calls: 838807
  number of function calls: 1930107

Statistics after:

  time elapsed: 1.17982
  size of a value: 24
  environments allocated: 1543334 (39624560 bytes)
  list elements: 9612638 (76901104 bytes)
  list concatenations: 37434
  values allocated: 1854933 (44518392 bytes)
  attribute sets allocated: 392040
  right-biased unions: 186334
  values copied in right-biased unions: 591137
  symbols in symbol table: 18272
  number of thunks: 1392467
  number of thunks avoided: 1507311
  number of attr lookups: 430801
  number of primop calls: 691600
  number of function calls: 1492502
2012-08-13 00:28:08 -04:00
Eelco Dolstra 62f72eb9e1 Add some more evaluations stats 2012-08-12 23:41:48 -04:00
Eelco Dolstra e82767910c Add some basic profiling support to the evaluator
Setting the environment variable NIX_COUNT_CALLS to 1 enables some
basic profiling in the evaluator.  It will count calls to functions
and primops as well as evaluations of attributes.

For example, to see where evaluation of a NixOS configuration spends
its time:

$ NIX_SHOW_STATS=1 NIX_COUNT_CALLS=1 ./src/nix-instantiate/nix-instantiate '<nixos>' -A system --readonly-mode
...
calls to 39 primops:
    239532 head
    233962 tail
    191252 hasAttr
...
calls to 1595 functions:
    224157 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/lists.nix:17:19'
    221767 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/lists.nix:17:14'
    221767 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/lists.nix:17:10'
...
evaluations of 7088 attributes:
    167377 undefined position
    132459 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/attrsets.nix:119:41'
     47322 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/attrsets.nix:13:21'
...
2012-08-12 23:29:28 -04:00
Eelco Dolstra 97421eb5ec Refactor settings processing
Put all Nix configuration flags in a Settings object.
2012-07-30 19:55:41 -04:00
Eelco Dolstra 8c79100839 Merge branch 'master' into no-manifests 2012-07-26 15:14:33 -04:00
Shea Levy f5954e2d94 prim_import: When importing .drvs, allocate the intermediate attrset on the heap just in case it escapes the stack frame. 2012-07-25 09:30:17 -04:00
Shea Levy b1112bbef1 import: If the path is a valid .drv file, parse it and generate a derivation attrset.
The generated attrset has drvPath and outPath with the right string context, type 'derivation', outputName with
the right name, all with a list of outputs, and an attribute for each output.

I see three uses for this (though certainly there may be more):

* Using derivations generated by something besides nix-instantiate (e.g. guix)

* Allowing packages provided by channels to be used in nix expressions. If a channel installed a valid deriver
  for each package it provides into the store, then those could be imported and used as dependencies or installed
  in environment.systemPackages, for example.

* Enable hydra to be consistent in how it treats inputs that are outputs of another build. Right now, if an
  input is passed as an argument to the job, it is passed as a derivation, but if it is accessed via NIX_PATH
  (i.e. through the <> syntax), then it is a path that can be imported. This is problematic because the build
  being depended upon may have been built with non-obvious arguments passed to its jobset file. With this
  feature, hydra can just set the name of that input to the path to its drv file in NIX_PATH
2012-07-25 09:30:17 -04:00
Eelco Dolstra b7fd2c2822 Use "#pragma once" to prevent repeated header file inclusion 2012-07-18 14:59:03 -04:00
Eelco Dolstra fe241ece29 Merge branch 'master' into no-manifests 2012-07-18 10:47:59 -04:00
Eelco Dolstra 1217204c81 Remove dead code 2012-07-17 14:07:52 -04:00
Eelco Dolstra a7a43adb79 builtins.storePath: resolve symlinks
Needed for Charon/Hydra interaction.
2012-07-12 18:25:01 -04:00
Eelco Dolstra 6586414bc7 nix-env: Determine which paths have substitutes in parallel 2012-07-11 10:14:06 -04:00
Eelco Dolstra 98a423b75a prim_import(): prefetch substitute info in parallel using queryMissing() 2012-07-09 09:59:34 -04:00
Eelco Dolstra 1aba0bf0fa nix-store -r: do substitutions in parallel
I.e. when multiple non-derivation arguments are passed to ‘nix-store
-r’ to be substituted, do them in parallel.
2012-06-27 16:58:15 -04:00
Eelco Dolstra 00c98a6bef Use Bison 2.5 2012-04-13 14:28:26 +02:00
Eelco Dolstra 605b16cd7b Fix compilation on FreeBSD
http://hydra.nixos.org/build/2213576

Not sure why compilation doesn't fail on other platforms...
2012-03-05 22:04:40 +01:00
Eelco Dolstra e9fc91df45 Fix error message
This fixes the error message

  error: file `' was not found in the Nix search path (add it using $NIX_PATH or -I)
2012-02-09 18:56:48 +01:00
Eelco Dolstra f491ae97d4 * Inline some functions and get rid of the indirection through
EvalState::eval().  This gives a 12% speedup on ‘nix-instantiate
  /etc/nixos/nixos/ -A system --readonly-mode’ (from 1.01s to 0.89s).
2012-02-04 13:50:25 +00:00
Eelco Dolstra 2bda12ef3b * Print elapsed evaluation time. 2012-02-04 13:27:11 +00:00
Eelco Dolstra baa0501cc1 * Fix importing a derivation. This gave a segfault. 2012-01-26 13:13:00 +00:00
Eelco Dolstra 330df4b4db * Allow comparisons between derivations by comparing the outPath
attributes.
2012-01-19 23:08:47 +00:00
Eelco Dolstra 18047d4625 * Add some debug output to print the derivation name once it's known.
This makes it easier to pinpoint the source of a crash.
2012-01-19 23:07:42 +00:00
Eelco Dolstra 9fe24c5a0d * Don't create thunks for simple constants (integers, strings, paths)
and allocate them only once.
* Move Value and related functions into value.hh.
2012-01-07 17:26:33 +00:00
Eelco Dolstra b52966e821 * Remove dead code. 2012-01-04 21:47:48 +00:00
Eelco Dolstra 35f2a6ba82 * Don't use dynamic_cast, it's very slow. "nix-instantiate
/etc/nixos/nixos -A system" spent about 10% of its time in
  dynamic_cast.
2012-01-04 21:24:11 +00:00
Eelco Dolstra 71f3c46cf6 * Drop the inefficient "Path" suffix in output attribute names. 2012-01-03 15:27:18 +00:00
Eelco Dolstra 921111d197 * Move the implementation of the ‘derivation’ primop into a separate
file.
2012-01-03 14:01:47 +00:00
Eelco Dolstra 5679041132 * The ‘foo.drvPath’ feature was already broken in read-only mode.
Since it's rarely used and fixing it is too much work right now,
  just document it.
2011-12-21 18:19:05 +00:00
Eelco Dolstra b19a0f63db * Simplify the context handling logic. 2011-12-21 15:33:30 +00:00
Eelco Dolstra 194d21f9f6 * Sync with the trunk. 2011-12-16 23:33:01 +00:00
Eelco Dolstra 49f59dceca * Move parseHash16or32 into libutil, and use in nix-hash. 2011-12-02 11:47:06 +00:00
Eelco Dolstra b12b21825c * Allow '<nixexpr>' syntax to be used in nix-instantiate, nix-build
and nix-env, e.g.,

  $ nix-env -f '<nixpkgs>' -i patchelf

  or

  $ nix-build '<nixos/tests>' -A login.test
2011-12-01 16:41:43 +00:00
Shea Levy 3c3107da86 There's no need to mess with drvPath at all 2011-11-06 07:18:19 +00:00
Shea Levy 2ab29be70c Fix faulty reversion of my changes to unsafeDiscardOutputDependency 2011-11-06 07:03:14 +00:00
Shea Levy ca0d47a70c Respect all outputs passed to the derivation, not just the last one 2011-11-06 06:54:05 +00:00
Shea Levy 24b65937e1 Remove the unused sCurrentOutput symbol 2011-11-06 06:28:34 +00:00
Shea Levy 3522730316 Embed output name into the context of the *OutPath attributes and extract it for input derivations
Multiple outputs test passes!
2011-11-06 06:28:30 +00:00
Shea Levy af2e53fd48 Include all outputs of derivations in the closure of explicitly-passed derivation paths
This required adding a queryOutputDerivationNames function in the store API
2011-11-06 06:28:20 +00:00
Shea Levy 981edeab7b The 'insert output between = signs' approach was not helpful 2011-11-06 06:28:14 +00:00
Shea Levy 2721e9f56f Merge from trunk 2011-11-06 00:13:09 +00:00
Eelco Dolstra a12095d3be * In printValueAsXML, handle the case where a "type" attribute is not
a string.  This happens in the NixOS option system.
* Remove a bogus comparison of a unsigned integer with -1.
2011-10-27 19:06:23 +00:00
Shea Levy e81c09edbf Remove the current output metadata from the string for unsaveDiscardOutputDependency 2011-09-16 11:31:00 +00:00
Shea Levy bf50d6ad32 Add information about which output is active to drvPath's context
This will break things that depend on being able to just strip away an equals sign, so those have to be updated next
2011-09-16 11:30:52 +00:00
Shea Levy ffa038f66d Add an sCurrentOutput member to EvalState 2011-09-16 11:30:44 +00:00
Shea Levy f3e410d4bf Add a currentOutput attribute to derivations keep track of which output is active 2011-09-16 11:30:03 +00:00
Shea Levy c172d16b00 First attempt at the output-as-derivation semantics
For each output, this adds a corresponding attribute to the derivation that is
the same as the derivation except for outPath, which is set to the path specific
to that output. Additionally, an "all" attribute is added that is a list of all
of the output derivations. This has to be done outside of derivationStrict as
each output is itself a derivation that contains itself (and all other outputs)
as an attribute. The derivation itself is equivalent to the first output in the
outputs list (or "out" if that list isn't set).
2011-09-14 05:59:17 +00:00
Eelco Dolstra 93227ff65c * Eliminate all uses of the global variable ‘store’ from libstore.
This should also fix:

    nix-instantiate: ./../boost/shared_ptr.hpp:254: T* boost::shared_ptr<T>::operator->() const [with T = nix::StoreAPI]: Assertion `px != 0' failed.

  which was caused by hashDerivationModulo() calling the ‘store’
  object (during store upgrades) before openStore() assigned it.
2011-08-31 21:11:50 +00:00
Eelco Dolstra 2d663b502d * Cache the result of file evaluation (i.e, memoize evalFile()). This
prevents files from being evaluated and stored as values multiple
  times.  For instance, evaluation of the ‘system’ attribute in NixOS
  causes ‘nixpkgs/pkgs/lib/lists.nix’ to be evaluated 2019 times.

  Caching gives a modest speedup and a decent memory footprint
  reduction (e.g., from 1.44s to 1.28s, and from 81 MiB to 59 MiB with
  GC_INITIAL_HEAP_SIZE=100000 on my system).
2011-08-06 19:45:43 +00:00
Eelco Dolstra 9d091ee99a * Handle the case where the search path element is a regular file. 2011-08-06 18:45:28 +00:00
Eelco Dolstra 00a724ebc6 * Remove a debug statement. 2011-08-06 18:24:43 +00:00
Eelco Dolstra 07340b8be7 * Add the Nix corepkgs to the end of the search path. This makes it
possible for other Nix expressions to use corepkgs (mostly useful
  for the buildenv function).
2011-08-06 18:23:38 +00:00
Eelco Dolstra c7101dac0b * Allow redirections in search path entries. E.g. if you have a
directory

    /home/eelco/src/stdenv-updates

  that you want to use as the directory for import such as

    with (import <nixpkgs> { });

  then you can say

    $ nix-build -I nixpkgs=/home/eelco/src/stdenv-updates
2011-08-06 17:48:57 +00:00
Eelco Dolstra 1ecc97b6bd * Add a Nix expression search path feature. Paths between angle
brackets, e.g.

    import <nixpkgs/pkgs/lib>

  are resolved by looking them up relative to the elements listed in
  the search path.  This allows us to get rid of hacks like

    import "${builtins.getEnv "NIXPKGS_ALL"}/pkgs/lib"

  The search path can be specified through the ‘-I’ command-line flag
  and through the colon-separated ‘NIX_PATH’ environment variable,
  e.g.,

    $ nix-build -I /etc/nixos ...

  If a file is not found in the search path, an error message is
  lazily thrown.
2011-08-06 16:05:24 +00:00
Eelco Dolstra 54945a2950 * Refactoring: move parseExprFromFile() and parseExprFromString() into
the EvalState class.
2011-08-06 13:02:55 +00:00
Eelco Dolstra c8606664ab * Don't allow derivations with fixed and non-fixed outputs. 2011-07-20 18:26:00 +00:00
Eelco Dolstra b2027f70d9 * Fix a huuuuge security hole in the Nix daemon. It didn't check that
derivations added to the store by clients have "correct" output
  paths (meaning that the output paths are computed by hashing the
  derivation according to a certain algorithm).  This means that a
  malicious user could craft a special .drv file to build *any*
  desired path in the store with any desired contents (so long as the
  path doesn't already exist).  Then the attacker just needs to wait
  for a victim to come along and install the compromised path.

  For instance, if Alice (the attacker) knows that the latest Firefox
  derivation in Nixpkgs produces the path

    /nix/store/1a5nyfd4ajxbyy97r1fslhgrv70gj8a7-firefox-5.0.1

  then (provided this path doesn't already exist) she can craft a .drv
  file that creates that path (i.e., has it as one of its outputs),
  add it to the store using "nix-store --add", and build it with
  "nix-store -r".  So the fake .drv could write a Trojan to the
  Firefox path.  Then, if user Bob (the victim) comes along and does

    $ nix-env -i firefox
    $ firefox

  he executes the Trojan injected by Alice.

  The fix is to have the Nix daemon verify that derivation outputs are
  correct (in addValidPath()).  This required some refactoring to move
  the hash computation code to libstore.
2011-07-20 18:10:47 +00:00
Eelco Dolstra d329c3ea9d * Support multiple outputs. A derivation can declare multiple outputs
by setting the ‘outputs’ attribute.  For example:

    stdenv.mkDerivation {
      name = "aterm-2.5";

      src = ...;

      outputs = [ "out" "tools" "dev" ];

      configureFlags = "--bindir=$(tools)/bin --includedir=$(dev)/include";
    }

  This derivation creates three outputs, named like this:

    /nix/store/gcnqgllbh01p3d448q8q6pzn2nc2gpyl-aterm-2.5
    /nix/store/gjf1sgirwfnrlr0bdxyrwzpw2r304j02-aterm-2.5-tools
    /nix/store/hp6108bqfgxvza25nnxfs7kj88xi2vdx-aterm-2.5-dev

  That is, the symbolic name of the output is suffixed to the store
  path (except for the ‘out’ output).  Each path is passed to the
  builder through the corresponding environment variable, e.g.,
  ${tools}.

  The main reason for multiple outputs is to allow parts of a package
  to be distributed and garbage-collected separately.  For instance,
  most packages depend on Glibc for its libraries, but don't need its
  header files.  If these are separated into different store paths,
  then a package that depends on the Glibc libraries only causes the
  libraries and not the headers to be downloaded.

  The main problem with multiple outputs is that if one output exists
  while the others have been garbage-collected (or never downloaded in
  the first place), and we want to rebuild the other outputs, then
  this isn't possible because we can't clobber a valid output (it
  might be in active use).  This currently gives an error message
  like:

    error: derivation `/nix/store/1s9zw4c8qydpjyrayxamx2z7zzp5pcgh-aterm-2.5.drv' is blocked by its output paths

  There are two solutions: 1) Do the build in a chroot.  Then we don't
  need to overwrite the existing path.  2) Use hash rewriting (see the
  ASE-2005 paper).  Scary but it should work.

  This is not finished yet.  There is not yet an easy way to refer to
  non-default outputs in Nix expressions.  Also, mutually recursive
  outputs aren't detected yet and cause the garbage collector to
  crash.
2011-07-18 23:31:03 +00:00
Eelco Dolstra d5d4dcd4c9 * Allow attribute names to be strings. Based on the
allow-arbitrary-strinsg-in-names patch by Marc Weber.
2011-07-13 15:53:24 +00:00
Eelco Dolstra 0a623a10c7 * Allow a default value in attribute selection by writing
x.y.z or default

  (as originally proposed in
  https://mail.cs.uu.nl/pipermail/nix-dev/2009-September/002989.html).

  For instance, an expression like

    stdenv.lib.attrByPath ["features" "ckSched"] false args

  can now be written as

    args.features.ckSched or false
2011-07-13 12:19:57 +00:00
Eelco Dolstra 2b9e29b1c8 * Change the right-hand side of the ‘.’ operator from an attribute to
an attribute path.  This is a refactoring to support default values.
2011-07-06 12:28:57 +00:00
Eelco Dolstra 5637037802 * In the ‘?’ operator, allow attribute paths. For instance, you can
write ‘attrs ? a.b’ to test whether ‘attrs’ has an attribute ‘a’
  containing an attribute ‘b’.  This is more convenient than ‘attrs ?
  a && attrs.a ? b’.

  Slight change in the semantics: it's no longer an error if the
  left-hand side of ‘?’ is not an attribute set.  In that case it just
  returns false.  So, ‘null ? foo’ no longer throws an error.
2011-07-06 10:58:17 +00:00
Eelco Dolstra 538b7caab0 * Don't allocate a big initial GC address space on machines with
little RAM.  Even if the memory isn't actually used, it can cause
  problems with the overcommit heuristics in the kernel.  So use a VM
  space of 25% of RAM, up to 384 MB.
2011-02-10 14:31:04 +00:00
Eelco Dolstra 5a6b039802 * Don't call GC_expand_hp unless we're actually using the garbage
collector.
2011-02-09 22:59:50 +00:00
Ludovic Courtès f1462c208b Use $BDW_GC_LIBS instead of a custom variable. 2011-02-09 21:23:13 +00:00
Eelco Dolstra 9db190eb31 * builtins.substring: if "start" is beyond the end of the string,
return the empty string.
2011-01-14 12:47:10 +00:00
Eelco Dolstra 812fae424e * DrvInfo contains pointers to expressions, so DrvInfos should be
traced by the garbage collector.  Otherwise "nix-env -u" can crash
  randomly.
2010-11-25 13:47:34 +00:00
Eelco Dolstra 0c4828ea05 * new(UseGC) is inexplicably slower than GC_MALLOC, so prefer the
latter.
2010-10-28 12:50:01 +00:00
Eelco Dolstra e11e6fb1c6 * Handle out of memory condition. 2010-10-28 12:29:40 +00:00
Eelco Dolstra 43535499f3 * When allocating an attribute set, reserve enough space for all
elements.  This prevents the vector from having to resize itself.
2010-10-24 20:09:37 +00:00
Eelco Dolstra e0b7fb8f27 * Keep attribute sets in sorted order to speed up attribute lookups.
* Simplify the representation of attributes in the AST.
* Change the behaviour of listToAttrs() in case of duplicate names.
2010-10-24 19:52:33 +00:00
Eelco Dolstra 2dc6d50941 * Don't create thunks for variable lookups (if possible). This
significantly reduces the number of values allocated (e.g. from 8.7m
  to 4.9m for the Bittorrent test).
2010-10-24 14:20:02 +00:00
Eelco Dolstra 0b305c534f * Store attribute sets as a vector instead of a map (i.e. a red-black
tree).  This saves a lot of memory.  The vector should be sorted so
  that names can be looked up using binary search, but this is not the
  case yet.  (Surprisingly, looking up attributes using linear search
  doesn't have a big impact on performance.)

  Memory consumption for

    $ nix-instantiate /etc/nixos/nixos/tests -A bittorrent.test --readonly-mode

  on x86_64-linux with GC enabled is now 185 MiB (compared to 946
  MiB on the trunk).
2010-10-24 00:41:29 +00:00
Eelco Dolstra a247d20604 * Fix compiling without Boehm.
* Fix the stats.
2010-10-23 22:58:24 +00:00
Eelco Dolstra b2ba62170c * Optimise string constants by putting them in the symbol table. 2010-10-23 21:11:59 +00:00
Eelco Dolstra 8ac06726b9 * Make Value smaller by not storing redundant PrimOp info.
* Clear pointers in Values after overwriting them to make sure that no
  objects are kept alive unnecessarily.
2010-10-23 20:07:47 +00:00
Eelco Dolstra 3f66cfb96b * Remove allocValues(). 2010-10-23 18:18:07 +00:00
Eelco Dolstra 4dee289550 * In environments, store pointers to values rather than values. This
improves GC effectiveness a bit more (because a live value doesn't
  keep other values in the environment plus the parent environments
  alive), and removes the need for copy nodes.
2010-10-22 15:51:52 +00:00
Eelco Dolstra 41c45a9b31 * Store Value nodes outside of attribute sets. I.e., Attr now stores
a pointer to a Value, rather than the Value directly.  This improves
  the effectiveness of garbage collection a lot: if the Value is
  stored inside the set directly, then any live pointer to the Value
  causes all other attributes in the set to be live as well.
2010-10-22 14:47:42 +00:00
Eelco Dolstra 64c3325b0b * Make building against the Boehm GC a configure option. 2010-10-22 13:39:15 +00:00
Eelco Dolstra 76feaf016a * Keep some more stats. 2010-10-20 15:48:00 +00:00
Eelco Dolstra e879a0371b * Use the Boehm garbage collector to reclaim unused memory in the Nix
expression evaluator.
2010-10-20 11:38:30 +00:00
Eelco Dolstra 705868a8a9 * Make sure that config.h is included before the system headers,
because it defines _FILE_OFFSET_BITS.  Without this, on 
  OpenSolaris the system headers define it to be 32, and then 
  the 32-bit stat() ends up being called with a 64-bit "struct 
  stat", or vice versa.

  This also ensures that we get 64-bit file sizes everywhere.

* Remove the redundant call to stat() in parseExprFromFile().
  The file cannot be a symlink because that's the exit condition
  of the loop before.
2010-10-04 17:55:38 +00:00
Eelco Dolstra 95f4f2cf61 * If std::tr1::unordered_set is unavailable, use std::set. 2010-10-04 16:16:19 +00:00
Eelco Dolstra 4aa9245083 * Hack needed for GCC 4.3.2 on OpenSolaris. 2010-10-04 10:51:16 +00:00
Eelco Dolstra df50916e46 * Oops - "null" was displayed as "true". 2010-08-27 12:10:56 +00:00
Eelco Dolstra 6d6200f37a * Optimisation in the // operator: if one of the sets is empty, return
the other set.
2010-08-02 16:31:05 +00:00
Eelco Dolstra 7af6a2fd71 * intersectAttrs: optimise for the case where the second set is larger
than the first set.  (That's usually the case with callPackage.)
2010-08-02 11:54:44 +00:00
Eelco Dolstra f16fe2af8d * builtins.toXML: propagate the string context. This is a regression
from the old ATerm-based evaluator.
2010-06-10 10:29:50 +00:00
Eelco Dolstra 1ab67cf437 2010-06-02 09:43:04 +00:00
Eelco Dolstra 89865da76d * Turn build errors during evaluation into EvalErrors. 2010-06-01 11:19:32 +00:00
Eelco Dolstra 93cd5a4a13 * The << operator on values should be const. 2010-05-18 10:36:37 +00:00
Eelco Dolstra b2235d81d1 * Restore the __overrides feature that was lost somewhere in the
fast-eval branch.
2010-05-15 08:10:12 +00:00
Eelco Dolstra bd25ac2260 * Print attributes in sorted order. 2010-05-12 12:15:49 +00:00
Eelco Dolstra 81a4b4e49b * Implemented tryEval, the last missing primop in the fast-eval
branch.  Also added a test for tryEval.
2010-05-12 11:23:44 +00:00
Eelco Dolstra 1a8eb6e3ec 2010-05-07 15:26:33 +00:00
Eelco Dolstra 83dfa89870 * Sync with the trunk. 2010-05-07 14:46:47 +00:00
Eelco Dolstra 01e58adce0 * Store position info for inherited attributes. 2010-05-07 12:43:57 +00:00
Eelco Dolstra 83d7b89660 * Updated addErrorContext. 2010-05-07 12:33:14 +00:00
Eelco Dolstra e2d5e40f4f * Keep track of the source positions of attributes. 2010-05-07 12:11:05 +00:00
Eelco Dolstra 84ce7ac76f * Store attribute positions in the AST and report duplicate attribute
errors with position info.
* For all positions, use the position of the first character of the
  first token, rather than the last character of the first token plus
  one.
2010-05-06 16:46:48 +00:00
Eelco Dolstra 2be6118f4c * Don't need the test program anymore. 2010-04-23 09:09:32 +00:00
Eelco Dolstra 0bc468f195 * Simplify the implementation of `with'. This gives a 7% speedup in
evaluating the NixOS system configuration.
2010-04-22 15:08:09 +00:00
Eelco Dolstra ebade9ff8b * Check for duplicate attribute names / function arguments. `make
check' now succeeds :-)
* An attribute set such as `{ foo = { enable = true; };
  foo.port = 23; }' now parses.  It was previously rejected, but I'm
  too lazy to implement the check.  (The only reason to reject it is
  that the reverse, `{ foo.port = 23; foo = { enable = true; }; }', is
  rejected, which is kind of ugly.)
2010-04-22 11:02:24 +00:00
Eelco Dolstra 2d7636529f * String equality tests should take the context into account. All the
evaluation test cases now succeed.
2010-04-22 09:54:11 +00:00
Eelco Dolstra 6f0f16497a * Fix the interpretation of ''\<character> in indented strings. 2010-04-21 16:18:27 +00:00
Eelco Dolstra 0777448ca6 * Fixed builtins.genericClosure. 2010-04-21 15:57:11 +00:00
Eelco Dolstra fe2d869e04 * Store user environment manifests as a Nix expression in
$out/manifest.nix rather than as an ATerm.

  (Hm, I thought I committed this two days ago...)
2010-04-21 15:08:58 +00:00
Eelco Dolstra f3b8833a48 * Drop the dependency on the ATerm library. 2010-04-19 14:51:58 +00:00
Eelco Dolstra efc7a579e8 * Don't use the ATerm library for parsing/printing .drv files. 2010-04-19 13:46:58 +00:00