Commit graph

344 commits

Author SHA1 Message Date
Eelco Dolstra cb4320c1a0 Cleanup 2015-07-23 23:14:07 +02:00
Eelco Dolstra c8bb2371eb Optimize empty sets
This reduces the number of Bindings allocations by about 10%.
2015-07-23 23:11:08 +02:00
Eelco Dolstra 19eddecc0f Merge branch 'attr-set-hh' of https://github.com/nbp/nix
Conflicts:
	src/libexpr/eval.cc
2015-07-23 22:16:01 +02:00
Eelco Dolstra b83801f8b3 Optimize small lists
The value pointers of lists with 1 or 2 elements are now stored in the
list value itself. In particular, this makes the "concatMap (x: if
cond then [(f x)] else [])" idiom cheaper.
2015-07-23 22:05:09 +02:00
Eelco Dolstra 6bd2c7bb38 OCD: foreach -> C++11 ranged for 2015-07-17 20:13:56 +02:00
Eelco Dolstra f39979c6d3 Make printValue() interruptible
Fixes #572.
2015-07-17 11:33:39 +02:00
Nicolas B. Pierron db21cfa688 Move attribute set data structures into their own header file.
This modification moves Attr and Bindings structures into their own header
file which is dedicated to the attribute set representation. The goal of to
isolate pieces of code which are related to the attribute set
representation. Thus future modifications of the attribute set
representation will only have to modify these files, and not every other
file across the evaluator.
2015-07-14 19:23:17 +02:00
Eelco Dolstra 65f17cd330 Support URLs in $NIX_PATH
This didn't work (despite claims in the manual), because the colon in
"http://" was parsed as a element separator. So handle "://"
specially.
2015-06-17 16:20:11 +02:00
Eelco Dolstra 920f5fd4dd Fix import-from-derivation in restricted eval mode
This relaxes restricted mode to allow access to anything in the
store. In the future, it would be better to allow access to only paths
that have been constructed in the current evaluation (so a hard-coded
/nix/store/blabla in a Nix expression would still be
rejected). However, note that reading /nix/store itself is still
rejected, so you can't use this so get access to things you don't know
about.
2015-05-22 12:18:23 +02:00
Eelco Dolstra 035aeb9547 Fix using restricted mode with chroots 2015-04-16 18:46:17 +02:00
Eelco Dolstra 7ea6ecf855 addToStore(): Take explicit name argument 2015-03-25 17:06:12 +01:00
Eelco Dolstra b005e63ccf Disable scanning for interior pointers
This may remove the "Repeated allocation of very large block"
warnings.
2015-03-19 20:10:08 +01:00
Eelco Dolstra 726f7f7fc9 Fix Boehm API violation
We were calling GC_INIT() after doing an allocation (in the baseEnv
construction), which is not allowed.
2015-03-19 20:02:37 +01:00
Eelco Dolstra da6b704b19 Check return values from malloc/strdup 2015-03-19 14:11:35 +01:00
Eelco Dolstra fa47279440 Print some Boehm GC stats 2015-03-18 16:24:54 +01:00
Eelco Dolstra 7a84143910 valueSize(): Take into account list/bindings/env size 2015-03-18 14:41:28 +01:00
Eelco Dolstra 17c71334e1 forceValueDeep: Add to error prefix 2015-03-06 15:10:12 +01:00
Eelco Dolstra 5badc8f975 Improve error message 2015-03-06 14:24:08 +01:00
Eelco Dolstra 15d2d3c34e Add restricted evaluation mode
If ‘--option restrict-eval true’ is given, the evaluator will throw an
exception if an attempt is made to access any file outside of the Nix
search path. This is primarily intended for Hydra, where we don't want
people doing ‘builtins.readFile ~/.ssh/id_dsa’ or stuff like that.
2015-02-23 15:54:31 +01:00
Shea Levy 4646e94610 ExprConcatStrings: canonicalize concatenated paths 2015-02-19 08:39:25 -05:00
Eelco Dolstra 8aedaf111e Remove canary stuff 2014-12-12 10:59:50 +01:00
Shea Levy 608110804c Make all ExternalValueBase functions const 2014-12-02 10:27:10 -05:00
Shea Levy 320659b0cd Allow external code using libnixexpr to add types
Code that links to libnixexpr (e.g. plugins loaded with importNative, or
nix-exec) may want to provide custom value types and operations on
values of those types. For example, nix-exec is currently using sets
where a custom IO value type would be more appropriate. This commit
provides a generic hook for such types in the form of tExternal and the
ExternalBase virtual class, which contains all functions necessary for
libnixexpr's type-polymorphic functions (e.g. `showType`) to be
implemented.
2014-12-02 10:27:04 -05:00
Eelco Dolstra 4e340a983f forceString(): Accept pos argument 2014-11-25 10:23:36 +01:00
Shea Levy 997defa166 Add functors (callable attribute sets).
With this, attribute sets with a `__functor` attribute can be applied
just like normal functions. This can be used to attach arbitrary
metadata to a function without callers needing to treat it specially.
2014-11-15 16:12:05 -05:00
Eelco Dolstra 3a9b4a1467 Fix more warnings 2014-10-31 08:49:15 +01:00
Eelco Dolstra 6bb4c0b712 mkList: Scrub better
Clearing v.app.right was not enough, because the length field of a
list only takes 32 bits, so the most significant 32 bits of v.app.left
(a.k.a. v.thunk.env) would remain. This could cause Boehm GC to
interpret it as a valid pointer.

This change reduces maximum RSS for evaluating the ‘tested’ job in
nixos/release-small.nix from 1.33 GiB to 0.80 GiB, and runtime by
about 8%.
2014-10-09 13:08:53 +02:00
Eelco Dolstra b6809608cc Get rid of some unnecessary ExprConcatStrings nodes in dynamic attrs
This gives a ~18% speedup in NixOS evaluation (after converting
most calls to hasAttr/getAttr to dynamic attrs).
2014-10-05 01:04:58 +02:00
Eelco Dolstra 1418806969 Show total allocations 2014-10-05 00:39:28 +02:00
Eelco Dolstra a17c23426e printValue(): Don't print <CYCLE> for repeated values 2014-10-01 15:54:40 +02:00
Eelco Dolstra 831fc8ea21 Make forceValueDeep work on values with cycles 2014-09-22 16:05:00 +02:00
Eelco Dolstra 1e0a799bef Rename strictForceValue -> forceValueDeep 2014-09-22 16:05:00 +02:00
Eelco Dolstra 022618c794 Handle cycles when printing a value
So this no longer crashes with a stack overflow:

  nix-instantiate -E --eval 'let as = { x = as; }; in as'

Instead it prints:

  { x = { x = <CYCLE>; }; }
2014-09-22 16:05:00 +02:00
Eelco Dolstra eff120d1b9 Add a function ‘valueSize’
It returns the size of value, including all other values and
environments reachable from it. It is intended for debugging memory
consumption issues.
2014-09-22 16:05:00 +02:00
Eelco Dolstra 93e4f01ee3 Inline Bindings::find() 2014-09-19 16:56:13 +02:00
Eelco Dolstra 5b58991a71 Store Attrs inside Bindings
This prevents a double allocation per attribute set.
2014-09-19 16:49:41 +02:00
Eelco Dolstra 6e5b02bee4 Add some instrumentation for debugging GC leaks 2014-09-17 15:19:07 +02:00
Eelco Dolstra 9472b4157d Fix boost::too_many_args error
Fixes #333.
2014-09-02 22:53:01 +02:00
Eelco Dolstra 11849a320e Use proper quotes everywhere 2014-08-20 18:03:48 +02:00
Eelco Dolstra 5bed74d1b0 Fix warning about non-existant -I directories 2014-08-13 02:57:59 +02:00
Eelco Dolstra ee7fe64c0a == operator: Ignore string context
There really is no case I can think of where taking the context into
account is useful. Mostly it's just very inconvenient.
2014-06-10 14:02:56 +02:00
Eelco Dolstra d8c061e044 Remove ExprBuiltin
It's slower than ExprVar since it doesn't compute a static
displacement. Since we're not using the throw primop in the
implementation of <...> anymore, it's also not really needed.
2014-05-26 17:14:28 +02:00
Eelco Dolstra 39d72640c2 Ensure that -I flags get included in nixPath
Also fixes #261.
2014-05-26 16:52:31 +02:00
Shea Levy eac5841970 Provide a more useful error message when a dynamic attr lookup fails 2014-05-15 17:56:24 +02:00
Eelco Dolstra 8e5fbf4d73 Show position info in attribute selection errors 2014-04-04 22:52:14 +02:00
Eelco Dolstra 4c5faad994 Show position info in Boolean operations 2014-04-04 22:43:52 +02:00
Eelco Dolstra bd9b1d97b4 Show position info in string concatenation / addition errors 2014-04-04 22:19:33 +02:00
Eelco Dolstra a5fe730940 forceString: Show position info 2014-04-04 21:14:11 +02:00
Eelco Dolstra 27b44b8cf7 forceAttrs: Show position info 2014-04-04 19:11:40 +02:00
Eelco Dolstra 96b695ccab forceList: Show position info 2014-04-04 19:05:36 +02:00
Eelco Dolstra b62d36963c forceInt: Show position info 2014-04-04 18:59:29 +02:00
Eelco Dolstra c28de6d96e Pass position information to primop calls
For example:

  error: `tail' called on an empty list, at
    /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/ex-2/default.nix:13:7
2014-04-04 18:59:29 +02:00
Eelco Dolstra b72c8d2e5b Include position info in function application
This allows error messages like:

  error: the anonymous function at `/etc/nixos/configuration.nix:1:1'
    called without required argument `foo', at
    `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/lib/modules.nix:77:59'
2014-04-04 18:59:29 +02:00
Eelco Dolstra 1c2550a2ae boost::shared_ptr -> std::shared_ptr 2014-03-30 00:49:23 +01:00
Shea Levy 049a379ec6 The expr of AttrNames/DynamicAttrDefs is always an ExprConcatStrings 2014-03-10 10:14:50 +01:00
Shea Levy 908e9ce259 If a dynamic attribute name evaluates to null, remove it from the set 2014-03-10 10:14:50 +01:00
Eelco Dolstra 2caab81660 Revert "Make ifs and asserts tail-recursive"
This reverts commit 273322c773.
2014-03-05 16:18:13 +01:00
Eelco Dolstra c9f6232304 Correctly detect infinite recursion in function application
If we're evaluating some application ‘v = f x’, we can't store ‘f’
temporarily in ‘v’, because if ‘f x’ refers to ‘v’, it will get ‘f’
rather than an infinite recursion error.

Unfortunately, this breaks the tail call optimisation introduced in
c897bac549.

Fixes #217.
2014-02-27 21:47:59 +01:00
Eelco Dolstra 81628a6ccc Merge branch 'master' into make
Conflicts:
	src/libexpr/eval.cc
2014-01-21 15:30:01 +01:00
Shea Levy cd49fe4f9b Don't use any syntactic sugar for dynamic attrs
This doesn't change any functionality but moves some behavior out of the
parser and into the evaluator in order to simplify the code.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-12-31 23:56:26 +00:00
Shea Levy 18fefacf7d Dynamic attrs
This adds new syntax for attribute names:

* attrs."${name}" => getAttr name attrs
* attrs ? "${name}" => isAttrs attrs && hasAttr attrs name
* attrs."${name}" or def => if attrs ? "${name}" then attrs."${name}" else def
* { "${name}" = value; } => listToAttrs [{ inherit name value; }]

Of course, it's a bit more complicated than that. The attribute chains
can be arbitrarily long and contain combinations of static and dynamic
parts (e.g. attrs."${foo}".bar."${baz}" or qux), which is relatively
straightforward for the getAttrs/hasAttrs cases but is more complex for
the listToAttrs case due to rules about duplicate attribute definitions.

For attribute sets with dynamic attribute names, duplicate static
attributes are detected at parse time while duplicate dynamic attributes
are detected when the attribute set is forced. So, for example, { a =
null; a.b = null; "${"c"}" = true; } will be a parse-time error, while
{ a = {}; "${"a"}".b = null; c = true; } will be an eval-time error
(technically that case could theoretically be detected at parse time,
but the general case would require full evaluation). Moreover, duplicate
dynamic attributes are not allowed even in cases where they would be
with static attributes ({ a.b.d = true; a.b.c = false; } is legal, but {
a."${"b"}".d = true; a."${"b"}".c = false; } is not). This restriction
might be relaxed in the future in cases where the static variant would
not be an error, but it is not obvious that that is desirable.

Finally, recursive attribute sets with dynamic attributes have the
static attributes in scope but not the dynamic ones. So rec { a = true;
"${"b"}" = a; } is equivalent to { a = true; b = true; } but rec {
"${"a"}" = true; b = a; } would be an error or use a from the
surrounding scope if it exists.

Note that the getAttr, getAttr or default, and hasAttr are all
implemented purely in the parser as syntactic sugar, while attribute
sets with dynamic attribute names required changes to the AST to be
implemented cleanly.

This is an alternative solution to and closes #167

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-12-31 20:59:49 +00:00
Shea Levy 136f2f7046 Add the ExprBuiltin Expr type to the AST
Certain desugaring schemes may require the parser to use some builtin
function to do some of the work (e.g. currently `throw` is used to
lazily cause an error if a `<>`-style path is not in the search path)
Unfortunately, these names are not reserved keywords, so an expression
that uses such a syntactic sugar will not see the expected behavior
(see tests/lang/eval-okay-redefine-builtin.nix for an example).

This adds the ExprBuiltin AST type, which when evaluated uses the value
from the rootmost variable scope (which of course is initialized
internally and can't shadow any of the builtins).

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-12-31 17:45:47 +00:00
Eelco Dolstra 5a1114ecdb Drop the dependency on libgc in libmain
Instead, libexpr now depends on libgc.  This means commands like
nix-store that don't do any evaluation no longer require libgc.
2013-11-23 20:19:36 +00:00
Eelco Dolstra 06a8ac96e7 Initialise Boehm GC only once 2013-11-23 20:15:57 +00:00
Eelco Dolstra 77c13cdf56 Add a toJSON primop 2013-11-19 00:04:11 +01:00
Eelco Dolstra 285df765b9 Add a primop unsafeGetAttrPos to return the position of an attribute 2013-11-18 22:22:35 +01:00
Eelco Dolstra fc33fd86b7 Add a symbol __curPos that expands to the current source location
I.e. an attribute set { file = <string>; line = <int>; column = <int>; }.
2013-11-18 20:16:02 +01:00
Eelco Dolstra 89e6781cc5 Make function calls show up in stack traces again
Note that adding --show-trace prevents functions calls from being
tail-recursive, so an expression that evaluates without --show-trace
may fail with a stack overflow if --show-trace is given.
2013-11-12 12:51:59 +01:00
Eelco Dolstra c897bac549 Make function calls tail-recursive 2013-11-12 11:32:23 +00:00
Eelco Dolstra 273322c773 Make ifs and asserts tail-recursive
The local Value object prevented g++ from making a tail call.  Not
clear why.  In any case, not using a temporary makes g++ do the tail
call.
2013-11-12 11:32:23 +00:00
Eelco Dolstra 4badd7ed17 Get rid of an intermediary on the stack 2013-11-12 11:32:23 +00:00
Eelco Dolstra ea6bf0c21f Slightly optimize listToAttrs 2013-10-28 07:34:44 +01:00
Eelco Dolstra 5bc41d78ff Rename "attribute sets" to "sets"
We don't have any other kind of sets so calling them attribute sets is
unnecessarily verbose.
2013-10-24 16:41:04 +02:00
Eelco Dolstra 69befd33a9 Remove unnecessary call to forceStringNoCtx 2013-10-24 03:08:34 +02:00
Eelco Dolstra 543d8a5942 Don't require NIX_SHOW_STATS for NIX_COUNT_CALLS 2013-10-24 02:20:54 +02:00
Eelco Dolstra fe95650487 Memoize evalFile() lookups under both the original and resolved name
Previously we only used the resolved name, causing repeated resolution
(e.g. /dir to /dir/default.nix).
2013-10-23 11:19:01 +00:00
Eelco Dolstra f440558acc Don't show <nix/derivation.nix> in stack traces
Messages like

  while evaluating the attribute `outPath' at `/nix/store/212ngf4ph63mp6p1np2bapkfikpakfv7-nix-1.6/share/nix/corepkgs/derivation.nix:18:9':

are redundant, because Nix already shows that it's evaluating a derivation:

  while instantiating the derivation named `firefox-24.0' at `/home/eelco/Dev/nixpkgs/pkgs/applications/networking/browsers/firefox/default.nix:131:5':
  while evaluating the derivation attribute `nativeBuildInputs' at `/home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/default.nix:76:17':
2013-10-17 11:47:38 +02:00
Eelco Dolstra d6a7aa8f48 Revert the behaviour of antiquoted paths to pre-Nix 1.6
Commit 159e621d1a accidentally changed
the behaviour of antiquoted paths, e.g.

  "${/foo}/bar"

used to evaluate to "/nix/store/<hash>-foo/bar" (where /foo gets
copied to the store), but in Nix 1.6 it evaluates to "/foo/bar".  This
is inconsistent, since

  " ${/foo}/bar"

evaluates to " /nix/store/<hash>-foo/bar".  So revert to the old
behaviour.
2013-10-17 00:39:59 +02:00
Eelco Dolstra 7bdb85453d printStats(): Print the size of the symbol table in bytes 2013-10-08 15:37:08 +02:00
Eelco Dolstra 9deb822180 Deduplicate filenames in Pos
This saves ~4 MiB of RAM for NixOS system instantiation, and ~18 MiB
for "nix-env -qa".
2013-10-08 15:36:10 +02:00
Eelco Dolstra b1e3b1a4ac Treat undefined variable errors consistently
Previously, a undefined variable inside a "with" caused an EvalError
(which can be caught), while outside, it caused a ParseError (which
cannot be caught).  Now both cause an UndefinedVarError (which cannot
be caught).
2013-10-08 14:45:36 +02:00
Eelco Dolstra 6b47de580f Show the exact position of undefined variables
In particular, undefined variable errors in a "with" previously didn't
show *any* position information, so this should help a lot in those
cases.
2013-10-08 14:40:51 +02:00
Eelco Dolstra a5e0f64db3 Remove some unused functions 2013-10-08 12:30:23 +00:00
Eelco Dolstra 221a2daf34 Merge VarRef into ExprVar 2013-10-08 14:24:53 +02:00
Eelco Dolstra 176c666f36 Don't show calls to primops in stack traces
Since they don't have location information, they just give you crap
like:

  while evaluating the builtin function `getAttr':
  while evaluating the builtin function `derivationStrict':
  ...
2013-10-07 18:02:32 +02:00
Eelco Dolstra c945f015de Fix segfault in nix-repl / hydra-eval-jobs
If a "with" attribute set fails to evaluate, we have to make sure its
Env record remains unchanged.  Otherwise, repeated evaluation gives a
segfault:

  nix-repl> :a with 0; { a = x; b = x; }
  Added 2 variables.

  nix-repl> a
  error: value is an integer while an attribute set was expected

  nix-repl> b
  Segmentation fault
2013-10-02 15:24:45 +02:00
Eelco Dolstra 07a08bddf0 nix-env: Load files in ~/.nix-defexpr on demand
So if you do "nix-env -qa -A nixos", then other channels won't be
parsed/evaluated at all.
2013-09-03 15:45:32 +02:00
Eelco Dolstra 6f809194d7 Get rid of the parse tree cache
Since we already cache files in normal form (fileEvalCache), caching
parse trees is redundant.

Note that getting rid of this cache doesn't actually save much memory
at the moment, because parse trees are currently not freed / GC'ed.
2013-09-03 13:01:42 +02:00
Eelco Dolstra 57d18df7d0 Add some support code for nix-repl 2013-09-02 18:34:04 +02:00
Eelco Dolstra 33972629d7 Fix whitespace 2013-09-02 16:29:15 +02:00
Shea Levy afc6c1bad6 Simplify inherited attribute handling
This reduces the difference between inherited and non-inherited
attribute handling to the choice of which env to use (in recs and lets)
by setting the AttrDef::e to a new ExprVar in the parser rather than
carrying a separate AttrDef::v VarRef member.

As an added bonus, this allows inherited attributes that inherit from a
with to delay forcing evaluation of the with's attributes.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-08-26 11:31:56 +02:00
Ivan Kozik 34bb806f74 Fix typos, especially those that end up in the Nix manual 2013-08-26 11:15:22 +02:00
Eelco Dolstra d308aeaf53 Store Nix integers as longs
So on 64-bit systems, integers are now 64-bit.

Fixes #158.
2013-08-19 12:35:03 +02:00
Eelco Dolstra 46ffcce0c3 In the profiler output, show function names (if available) 2013-08-02 18:29:23 +00:00
Eelco Dolstra 159e621d1a Overload the ‘+’ operator to support integer addition 2013-08-02 15:21:17 +00:00
Eelco Dolstra 0a470fc345 Make Env smaller
Commit 20866a7031 added a ‘withAttrs’
field to Env, which is annoying because it makes every Env structure
bigger and we allocate millions of them.  E.g. NixOS evaluation took
18 MiB more.  So this commit squeezes ‘withAttrs’ into values[0].
Probably should use a union...
2013-07-31 12:44:21 +02:00
Eelco Dolstra 8ae6d55db1 Don't use NULL 2013-07-31 12:11:14 +02:00
Shea Levy e068f49f7d Avoid thunks when a fromWith var can be looked up without evaluation
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-31 11:48:39 +02:00
Shea Levy 20866a7031 Delay evaulation of with attrs until a variable lookup needs them
Evaluation of attribute sets is strict in the attribute names, which
means immediate evaluation of `with` attribute sets rules out some
potentially interesting use cases (e.g. where the attribute names of one
set depend in some way on another but we want to bring those names into
scope for some values in the second set).

The major example of this is overridable self-referential package sets
(e.g. all-packages.nix). With immediate `with` evaluation, the only
options for such sets are to either make them non-recursive and
explicitly use the name of the overridden set in non-overridden one
every time you want to reference another package, or make the set
recursive and use the `__overrides` hack. As shown in the test case that
comes with this commit, though, delayed `with` evaluation allows a nicer
third alternative.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-31 11:48:39 +02:00
Eelco Dolstra 18a48d80a0 Show function names in error messages
Functions in Nix are anonymous, but if they're assigned to a
variable/attribute, we can use the variable/attribute name in error
messages, e.g.

while evaluating `concatMapStrings' at `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/strings.nix:18:25':
...
2013-05-16 19:08:02 +02:00