Commit graph

772 commits

Author SHA1 Message Date
Ben Burdette 4610e02d04 remove debug code 2021-12-27 18:12:46 -07:00
Ben Burdette 9760fa8661 add DebugTrace for the current error 2021-12-27 17:35:27 -07:00
Ben Burdette 6801a423fc :d env 2021-12-27 16:28:45 -07:00
Ben Burdette ff82ba98b4 don't add builtins to extras, initEnv() in regular repl 2021-12-27 14:06:04 -07:00
Ben Burdette d0d5890445 don't add underscore names to extras 2021-12-27 13:47:35 -07:00
Ben Burdette e5eebda194 DebugTrace 2021-12-23 13:36:39 -07:00
Ben Burdette deb1fd66e8 makeDebugTraceStacker 2021-12-23 09:08:41 -07:00
Ben Burdette 1bda6a01e1 indenting 2021-12-23 08:14:17 -07:00
Ben Burdette bc20e54e00 stack traces basically working 2021-12-22 19:40:08 -07:00
Ben Burdette b4a59a5eec DebugStackTracker class in one place 2021-12-22 15:38:49 -07:00
pennae 09b245690a bulk-allocate Value instances in the evaluator
calling GC_malloc for each value is significantly more expensive than
allocating a bunch of values at once with GC_malloc_many. "a bunch" here
is a GC block size, ie 16KiB or less.

this gives a 1.5% performance boost when evaluating our nixos system.

tested with

nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'

 # on master

  Time (mean ± σ):      3.335 s ±  0.007 s    [User: 2.774 s, System: 0.293 s]
  Range (min … max):    3.315 s …  3.347 s    50 runs

 # with this change

  Time (mean ± σ):      3.288 s ±  0.006 s    [User: 2.728 s, System: 0.292 s]
  Range (min … max):    3.274 s …  3.307 s    50 runs
2021-12-20 23:01:28 +01:00
Eelco Dolstra cc6406cc59 Merge branch 'better-interpolation-error-location' of https://github.com/greedy/nix 2021-12-13 19:20:48 +01:00
Ben Burdette e82aec4efc fix merge issues 2021-11-30 14:15:02 -07:00
Eelco Dolstra 6f46434f32
Merge pull request #5648 from edolstra/list-iter
Support range-based for loop over list values
2021-11-25 17:13:25 +01:00
Ben Burdette 64c4ba8f66 Merge branch 'master' into debug-merge 2021-11-25 08:53:59 -07:00
Eelco Dolstra b6c8e57056 Support range-based for loop over list values 2021-11-25 16:31:39 +01:00
Ben Burdette 69e26c5c4b more cleanup 2021-11-25 08:23:07 -07:00
regnat c47027f3a1 Fix the error when accessing a forbidden path in pure eval
If we’re in pure eval mode, then tell that in the error message rather
than (wrongly) speaking about restricted mode.

Fix https://github.com/NixOS/nix/issues/5611
2021-11-25 14:48:01 +01:00
Tom Bereknyei 4318ba2ec5 add real path to allowedPaths 2021-11-20 00:25:36 -05:00
Eelco Dolstra d7bae52b9d Call functors with both arguments at once
This is not really useful on its own, but it does recover the
'infinite recursion' error message for '{ __functor = x: x; } 1', and
is more efficient in conjunction with #3718.

Fixes #5515.
2021-11-16 22:34:17 +01:00
Eelco Dolstra e41cf8511f Don't hang when calling an attrset
Fixes #5565.
2021-11-16 17:44:19 +01:00
Ben Burdette 7e2a3db4eb cleanup 2021-11-09 13:14:49 -07:00
Ben Burdette 885f819922 remove dead code 2021-11-09 11:20:14 -07:00
Eelco Dolstra 40925337a9 Remove maxPrimOpArity 2021-11-04 15:04:07 +01:00
Eelco Dolstra acd6bddec7 Fix derivation primop 2021-11-04 15:04:00 +01:00
Eelco Dolstra bcf4780006 Add level / displacement types 2021-11-04 15:03:45 +01:00
Eelco Dolstra 81e7c40264 Optimize primop calls
We now parse function applications as a vector of arguments rather
than as a chain of binary applications, e.g. 'substring 1 2 "foo"' is
parsed as

  ExprCall { .fun = <substring>, .args = [ <1>, <2>, <"foo"> ] }

rather than

  ExprApp (ExprApp (ExprApp <substring> <1>) <2>) <"foo">

This allows primops to be called immediately (if enough arguments are
supplied) without having to allocate intermediate tPrimOpApp values.

On

  $ nix-instantiate --dry-run '<nixpkgs/nixos/release-combined.nix>' -A nixos.tests.simple.x86_64-linux

this gives a substantial performance improvement:

  user CPU time:      median =      0.9209  mean =      0.9218  stddev =      0.0073  min =      0.9086  max =      0.9340  [rejected, p=0.00000, Δ=-0.21433±0.00677]
  elapsed time:       median =      1.0585  mean =      1.0584  stddev =      0.0024  min =      1.0523  max =      1.0623  [rejected, p=0.00000, Δ=-0.20594±0.00236]

because it reduces the number of tPrimOpApp allocations from 551990 to
42534 (i.e. only small minority of primop calls are partially
applied) which in turn reduces time spent in the garbage collector.
2021-11-04 15:03:40 +01:00
Eelco Dolstra ab35cbd675 StaticEnv: Use std::vector instead of std::map 2021-11-04 15:03:34 +01:00
regnat af99941279 Make experimental-features a proper type
Rather than having them plain strings scattered through the whole
codebase, create an enum containing all the known experimental features.

This means that
- Nix can now `warn` when an unkwown experimental feature is passed
  (making it much nicer to spot typos and spot deprecated features)
- It’s now easy to remove a feature altogether (once the feature isn’t
  experimental anymore or is dropped) by just removing the field for the
  enum and letting the compiler point us to all the now invalid usages
  of it.
2021-10-26 07:02:31 +02:00
Ben Burdette fb8377547b more code cleanup 2021-10-22 14:49:58 -06:00
Ben Burdette 71da988d47 more debug removal 2021-10-22 14:34:50 -06:00
Ben Burdette e54f17eb46 remove more debug code 2021-10-22 14:27:04 -06:00
Ben Burdette cbc2f0fe31 remove dead code 2021-10-22 14:02:47 -06:00
Ben Burdette 427fb8d158 comment out debugs 2021-10-11 16:48:10 -06:00
Ben Burdette 98eb13691a print staticenv bindings 2021-10-11 16:32:43 -06:00
Ben Burdette 2ee1fa4afd add nullable Expr argument 2021-10-11 14:42:29 -06:00
Eelco Dolstra d39692e6b3 Make builtins.{path,filterSource} work with chroot stores 2021-10-07 14:22:39 +02:00
Eelco Dolstra 972405edf5 Allow access to path copied to the store
Fixes https://github.com/NixOS/nix/pull/5163#issuecomment-931733912.
2021-10-07 12:15:22 +02:00
Eelco Dolstra cfaad7168e Refactoring: Add allowPath() method 2021-10-07 12:11:00 +02:00
Andreas Rammhold cae41eebff libexpr: remove matchAttrs boolean from ExprLambda
The boolean is only used to determine if the formals are set to a
non-null pointer in all our cases. We can get rid of that allocation and
instead just compare the pointer value with NULL. Saving up to
sizeof(bool) + platform specific alignment per ExprLambda instace.
Probably not a lot of memory but perhaps a few kilobyte with nixpkgs?

This also gets rid of a potential issue with dereferencing formals based on
the value of the boolean that didn't have to be aligned with the formals
pointer but was in all our cases.
2021-10-06 17:24:06 +02:00
Ben Burdette aad27143c6 storing staticenv bindings 2021-10-02 13:47:36 -06:00
Maximilian Bosch 2b02ce0e48
libexpr: throw a more helpful eval-error if a builtin is not available due to a missing feature-flag
I found it somewhat confusing to have an error like

    error: attribute 'getFlake' missing

if the required experimental-feature (`flakes`) is not enabled. Instead,
I'd expect Nix to throw an error just like it's the case when using e.g. `nix
flake` without `flakes` being enabled.

With this change, the error looks like this:

    $ nix-instantiate -E 'builtins.getFlake "nixpkgs"'
    error: Cannot call 'builtins.getFlake' because experimental Nix feature 'flakes' is disabled. You can enable it via '--extra-experimental-features flakes'.

           at «string»:1:1:

                1| builtins.getFlake "nixpkgs"
                 | ^

I didn't use `settings.requireExperimentalFeature` here on purpose
because this doesn't contain a position. Also, it doesn't seem as if we
need to catch the error and check for the missing feature here since
this already happens at evaluation time.
2021-09-29 11:57:15 +02:00
Geoff Reedy 9d67332e4b Better eval error locations for interpolation and +
Previously, type or coercion errors for string interpolation, path
interpolation, and plus expressions were always reported at the
beginning of the outer expression. This leads to confusing evaluation
error messages making it hard to accurately diagnose and then fix the
error.

For example, errors were reported as follows.

```
cannot coerce an integer to a string
1| let foo = 7; in "bar" + foo
 |                 ^

cannot add a string to an integer
1| let foo = "bar"; in 4 + foo
 |                     ^

cannot coerce an integer to a string
1| let foo = 7; in "x${foo}"
 |                  ^
```

This commit changes the ExprConcatStrings expression vector to store a
sequence of expressions *and* their expansion locations so that error
locations can be reported accurately. For interpolation, the error is
reported at the beginning of the entire `${foo}`, not at the beginning
of `foo` because I thought this was slightly clearer. The previous
errors are now reported as:

```
cannot coerce an integer to a string
1| let foo = 7; in "bar" + foo
 |                         ^

cannot add a string to an integer
1| let foo = "bar"; in 4 + foo
 |                         ^

cannot coerce an integer to a string
1| let foo = 7; in "x${foo}"
 |                   ^
```

The error is reported at this kind of precise location even for
multi-line indented strings.

This probably helps with at least some of the cases mentioned in #561
2021-09-22 20:57:34 -06:00
Ben Burdette c07edb1932 staticenv should be With 2021-09-22 18:14:57 -06:00
Ben Burdette cd8c232b55 add cout debugging 2021-09-15 16:16:53 -06:00
Ben Burdette 21071bfdeb shared_ptr for StaticEnv 2021-09-14 10:49:22 -06:00
Ben Burdette 176911102c printEnvPosChain 2021-09-13 11:57:25 -06:00
Eelco Dolstra 49a932fb18 nix --help: Display help using lowdown instead of man
Fixes #4476.
Fixes #5231.
2021-09-13 14:45:21 +02:00
Eelco Dolstra 7ee639f9db
Merge pull request #5066 from Radvendii/master
add antiquotations to paths
2021-09-01 12:55:04 +02:00
Taeer Bar-Yam 9da8f5e25d path antiquotations: canonizePath -> canonicalizePath 2021-08-31 08:02:04 -04:00
Robert Hensing f10465774f Force all Pos* to be non-null
This fixes a class of crashes and introduces ptr<T> to make the
code robust against this failure mode going forward.

Thanks regnat for the idea of a ref<T> without overhead!

Closes #4895
Closes #4893
Closes #5127
Closes #5113
2021-08-29 18:11:58 +02:00
Ben Burdette 310c689d31 remove more explicit valmap code 2021-08-25 13:18:27 -06:00
Ben Burdette d8a977a22e adding all the value names from env.values[0] 2021-08-25 11:19:09 -06:00
Ben Burdette bd3b5329f9 print env bindings 2021-08-24 16:32:54 -06:00
Ben Burdette 4b5f9b35f0 env to bindings 2021-08-18 21:25:26 -06:00
Ben Burdette 2272021536 more error fixes 2021-08-18 20:02:23 -06:00
Ben Burdette e82cf13b1e switch to fakeenvs 2021-08-18 17:53:10 -06:00
Ben Burdette b6eb38016b moving towards env in exceptions 2021-08-17 14:39:50 -06:00
Ben Burdette 030271184f trying env args; but unecessary? 2021-08-09 14:30:47 -06:00
Ben Burdette 89264d20e6 move valmap to hh; add to env 2021-08-06 11:09:27 -06:00
Taeer Bar-Yam 8f9429dcab add antiquotations to paths 2021-08-06 06:46:05 -04:00
Eelco Dolstra c000cec27f
Merge pull request #5048 from tweag/flox-eval-store
--eval-store and faster closure copying
2021-07-27 12:20:32 +02:00
Pamplemousse c2d7c0cdb9 libexpr: Remove unused code
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
2021-07-26 09:14:34 -07:00
Eelco Dolstra bef40c2949 Add --eval-store option 2021-07-22 09:59:51 +02:00
Pamplemousse c1c5dd7449 Avoid global counters
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
2021-07-21 16:49:52 -07:00
Eelco Dolstra 24bc935462 Merge branch 'disable_gc' of https://github.com/Pamplemousse/nix 2021-07-05 13:53:52 +02:00
Domen Kožar f9d72855ae
Merge pull request #4967 from Pamplemousse/specific_errors
Prefer to throw specific errors
2021-07-03 10:34:56 +02:00
Pamplemousse 4a7a8b87cd Prefer to throw specific errors
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
2021-07-01 11:09:31 -07:00
Pamplemousse 5be17a4b96 Allow to compile after ./configure --enable-gc=no
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
2021-06-30 09:45:21 -07:00
Robert Hensing bec83a6f95 BoehmGCStackAllocator: ignore stack protection page
This fixes a crash that looks like:

```
Thread 1 "nix-build" received signal SIGSEGV, Segmentation fault.
0x00007ffff7ad22a0 in GC_push_all_eager () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
(gdb) bt
0  0x00007ffff7ad22a0 in GC_push_all_eager () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
1  0x00007ffff7adeefb in GC_push_all_stacks () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
2  0x00007ffff7ad5ac7 in GC_mark_some () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
3  0x00007ffff7ad77bd in GC_stopped_mark () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
4  0x00007ffff7adbe3a in GC_try_to_collect_inner.part.0 () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
5  0x00007ffff7adc2a2 in GC_collect_or_expand () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
6  0x00007ffff7adc4f8 in GC_allocobj () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
7  0x00007ffff7adc88f in GC_generic_malloc_inner () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
8  0x00007ffff7ae1a04 in GC_generic_malloc_many () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
9  0x00007ffff7ae1c72 in GC_malloc_kind () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
10 0x00007ffff7e003d6 in nix::EvalState::allocValue() () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
11 0x00007ffff7e04b9c in nix::EvalState::callPrimOp(nix::Value&, nix::Value&, nix::Value&, nix::Pos const&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
12 0x00007ffff7e0a773 in nix::EvalState::callFunction(nix::Value&, nix::Value&, nix::Value&, nix::Pos const&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
13 0x00007ffff7e0a91d in nix::ExprApp::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
14 0x00007ffff7e0a8f8 in nix::ExprApp::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
15 0x00007ffff7e0e0e8 in nix::ExprOpNEq::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
16 0x00007ffff7e0d708 in nix::ExprOpOr::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
17 0x00007ffff7e0d695 in nix::ExprOpOr::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
18 0x00007ffff7e0d695 in nix::ExprOpOr::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
19 0x00007ffff7e0d695 in nix::ExprOpOr::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
20 0x00007ffff7e0d695 in nix::ExprOpOr::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
21 0x00007ffff7e09e19 in nix::ExprOpNot::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
22 0x00007ffff7e0a792 in nix::EvalState::callFunction(nix::Value&, nix::Value&, nix::Value&, nix::Pos const&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
23 0x00007ffff7e8cba0 in nix::addPath(nix::EvalState&, nix::Pos const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Value*, nix::FileIngestionMethod, std::optional<nix::Hash>, nix::Value&)::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
24 0x00007ffff752e6f9 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
25 0x00007ffff752e8e2 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
26 0x00007ffff752e8e2 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
27 0x00007ffff752e8e2 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
28 0x00007ffff752e8e2 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
29 0x00007ffff752e8e2 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
30 0x00007ffff757f8c0 in void boost::context::detail::fiber_entry<boost::context::detail::fiber_record<boost::context::fiber, nix::VirtualStackAllocator, boost::coroutines2::detail::pull_coroutine<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::control_block::control_block<nix::VirtualStackAllocator, nix::sinkToSource(std::function<void (nix::Sink&)>, std::function<void ()>)::SinkToSource::read(char*, unsigned long)::{lambda(boost::coroutines2::detail::push_coroutine<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >&)#1}>(boost::context::preallocated, nix::VirtualStackAllocator&&, nix::sinkToSource(std::function<void (nix::Sink&)>, std::function<void ()>)::SinkToSource::read(char*, unsigned long)::{lambda(boost::coroutines2::detail::push_coroutine<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >&)#1}&&)::{lambda(boost::context::fiber&&)#1}> >(boost::context::detail::transfer_t) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
31 0x00007ffff6f331ef in make_fcontext () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libboost_context.so.1.69.0
32 0x0000000000000000 in ?? ()
```
2021-06-24 12:20:39 +02:00
Ben Burdette 129dd760e6 mapEnvBindings 2021-06-11 18:55:15 -06:00
Ben Burdette d22de1dd0c remove dead code 2021-06-09 15:38:08 -06:00
Ben Burdette 93ca9381da formatting; string arg for throwTypeError 2021-06-08 18:37:28 -06:00
Ben Burdette ebf530d31e line endings 2021-06-08 18:17:58 -06:00
Ben Burdette ff2e72054f another throwTypeError form 2021-06-08 14:44:41 -06:00
Ben Burdette 99304334ca showType(fun) 2021-05-14 18:09:30 -06:00
Ben Burdette 644567cf7e clean up w LocalNoInline macro 2021-05-14 13:40:00 -06:00
Ben Burdette 17af7dc326 throwAssertionError, throwUndefinedError -> valmap-ized 2021-05-14 11:29:26 -06:00
Ben Burdette d041dd874e throwEvalError form 4 2021-05-14 11:15:24 -06:00
Ben Burdette ed74eaa07f throwEvalError form 3 2021-05-14 11:09:18 -06:00
Ben Burdette 989a4181a8 throwEvalError form 2 2021-05-14 11:06:20 -06:00
Ben Burdette ab19d1685d throwEvalError; mapBindings 2021-05-13 16:00:48 -06:00
Ben Burdette 459bccc750 plain env pointer 2021-05-12 11:33:31 -06:00
Ben Burdette 0c2265da85 unique_ptr for valmap 2021-05-12 09:43:58 -06:00
Ben Burdette e7847ad7a1 map1/2 for stack usage 2021-05-11 15:38:49 -06:00
Ben Burdette a8fef9a6b1 throwTypeError with debugger/env 2021-05-10 18:36:57 -06:00
Ben Burdette 2dd61411af debugger on autoCallFunction error 2021-05-03 14:37:33 -06:00
Ben Burdette 57c2dd5d85 fixes 2021-04-28 09:55:08 -06:00
Ben Burdette b466d51952 Merge branch 'master' into debugger-merge 2021-04-27 12:31:01 -06:00
Eelco Dolstra 293220bed5
Merge pull request #4440 from Ma27/misc-pos-fixes
Miscellaneous improvements for positioning in eval-errors
2021-04-23 11:10:59 +02:00
Maximilian Bosch 7c76964daa
libexpr: misc improvements for proper error position
When working on some more complex Nix code, there are sometimes rather
unhelpful or misleading error messages, especially if coerce-errors are
thrown.

This patch is a first steps towards improving that. I'm happy to file
more changes after that, but I'd like to gather some feedback first.

To summarize, this patch does the following things:

* Attrsets (a.k.a. `Bindings` in `libexpr`) now have a `Pos`. This is
  helpful e.g. to identify which attribute-set in `listToAttrs` is
  invalid.

* The `Value`-struct has a new method named `determinePos` which tries
  to guess the position of a value and falls back to a default if that's
  not possible.

  This can be used to provide better messages if a coercion fails.

* The new `determinePos`-API is used by `builtins.concatMap` now. With
  that change, Nix shows the exact position in the error where a wrong
  value was returned by the lambda.

  To make sure it's still obvious that `concatMap` is the problem,
  another stack-frame was added.

* The changes described above can be added to every other `primop`, but
  first I'd like to get some feedback about the overall approach.
2021-04-13 23:12:38 +02:00
Graham Christensen d04969ffa5 EvalSettings::getDefaultNixPath: respect {restrict,pure}Eval
Otherwise Nix may look to invalid locations for channels.
2021-04-12 12:37:15 -04:00
Eelco Dolstra 665d4ec2da nix repl :doc: Don't return docs for partially applied primops
This gives misleading results for Nixpkgs functions like lib.toUpper.

Fixes #4596.
2021-03-03 17:52:57 +01:00
Eelco Dolstra 574eb2be81
Tweak error message 2021-02-22 15:24:14 +01:00
Eelco Dolstra 8d4268d190 Improve error formatting
Changes:

* The divider lines are gone. These were in practice a bit confusing,
  in particular with --show-trace or --keep-going, since then there
  were multiple lines, suggesting a start/end which wasn't the case.

* Instead, multi-line error messages are now indented to align with
  the prefix (e.g. "error: ").

* The 'description' field is gone since we weren't really using it.

* 'hint' is renamed to 'msg' since it really wasn't a hint.

* The error is now printed *before* the location info.

* The 'name' field is no longer printed since most of the time it
  wasn't very useful since it was just the name of the exception (like
  EvalError). Ideally in the future this would be a unique, easily
  googleable error ID (like rustc).

* "trace:" is now just "…". This assumes error contexts start with
  something like "while doing X".

Example before:

  error: --- AssertionError ---------------------------------------------------------------------------------------- nix
  at: (7:7) in file: /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix

       6|
       7|   x = assert false; 1;
        |       ^
       8|

  assertion 'false' failed
  ----------------------------------------------------- show-trace -----------------------------------------------------
  trace: while evaluating the attribute 'x' of the derivation 'hello-2.10'
  at: (192:11) in file: /home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/make-derivation.nix

     191|         // (lib.optionalAttrs (!(attrs ? name) && attrs ? pname && attrs ? version)) {
     192|           name = "${attrs.pname}-${attrs.version}";
        |           ^
     193|         } // (lib.optionalAttrs (stdenv.hostPlatform != stdenv.buildPlatform && !dontAddHostSuffix && (attrs ? name || (attrs ? pname && attrs ? version)))) {

Example after:

  error: assertion 'false' failed

         at: (7:7) in file: /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix

              6|
              7|   x = assert false; 1;
               |       ^
              8|

         … while evaluating the attribute 'x' of the derivation 'hello-2.10'

         at: (192:11) in file: /home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/make-derivation.nix

            191|         // (lib.optionalAttrs (!(attrs ? name) && attrs ? pname && attrs ? version)) {
            192|           name = "${attrs.pname}-${attrs.version}";
               |           ^
            193|         } // (lib.optionalAttrs (stdenv.hostPlatform != stdenv.buildPlatform && !dontAddHostSuffix && (attrs ? name || (attrs ? pname && attrs ? version)))) {
2021-01-21 11:02:09 +01:00
Domen Kožar 00f99fdfe6
Merge pull request #4240 from bburdette/2259-error-message
2259 error message - "auto-call" error
2021-01-15 18:26:21 +00:00
Eelco Dolstra 75efa42134 Move <nix/fetchurl.nix> into the nix binary
This makes the statically linked nix binary just work, without needing
any additional files.
2020-12-22 14:43:20 +01:00
Silvan Mosberger b70d22baca
Replace Value type setters with mk* functions
Move clearValue inside Value

mkInt instead of setInt

mkBool instead of setBool

mkString instead of setString

mkPath instead of setPath

mkNull instead of setNull

mkAttrs instead of setAttrs

mkList instead of setList*

mkThunk instead of setThunk

mkApp instead of setApp

mkLambda instead of setLambda

mkBlackhole instead of setBlackhole

mkPrimOp instead of setPrimOp

mkPrimOpApp instead of setPrimOpApp

mkExternal instead of setExternal

mkFloat instead of setFloat

Add note that the static mk* function should be removed eventually
2020-12-18 21:48:22 +01:00
Silvan Mosberger 12e65078ef
Rename Value::normalType() -> Value::type() 2020-12-17 14:45:45 +01:00
Silvan Mosberger d67e02919c
Rename ValueType -> InternalType, NormalType -> ValueType
And Value::type to Value::internalType, such that type() can be used in
the next commit to get the new ValueType
2020-12-17 14:45:22 +01:00
Silvan Mosberger 730b152b19
Make Value::type private
This is an implementation detail and shouldn't be used. Use normalType()
and the various is<Type> functions instead
2020-12-12 03:31:52 +01:00
Silvan Mosberger bf98903967
Add ValueType checking functions for types that have the same NormalType 2020-12-12 03:31:50 +01:00
Silvan Mosberger 22ead43a0b
Use Value::normalType on all forced values instead of Value::type 2020-12-12 03:31:48 +01:00
Silvan Mosberger 9f056f7afd
Introduce Value type setters and make use of them 2020-12-12 03:31:48 +01:00
Silvan Mosberger fa307875e9
Introduce NormalType for the normal type of a Value
This will be useful to abstract over the ValueType implementation
details

Make use of it already to replace the showType(ValueType) function
2020-12-12 03:31:46 +01:00
Christian Höppner c4c3c15c19 Fix default nix-path
The default nix-path values for nixpkgs and root channels were
incorrect.
2020-11-12 15:46:08 +00:00
Ben Burdette b327de9c2d change message 2020-11-11 11:09:59 -07:00
Ben Burdette 8895fa70a4 pare down the error message 2020-11-11 11:05:21 -07:00
Ben Burdette 3edfe6090e missing argument error 2020-11-11 09:29:32 -07:00
Ben Burdette d8ef423a18 error message formatting 2020-11-09 19:16:50 -07:00
Ben Burdette 9f2b25ce55 remove unused ftn; reformat line breaks 2020-11-09 17:17:47 -07:00
Ben Burdette 6c2933a8d7 add position 2020-11-09 17:04:52 -07:00
Ben Burdette 107c91f5fe auto-call error 2020-11-09 16:48:35 -07:00
Eelco Dolstra 387f824cab
Merge pull request #4206 from hercules-ci/fix-coroutine-gc
Fix memory corruption caused by GC-invisible coroutine stacks
2020-11-05 10:18:31 +01:00
Eelco Dolstra 5e6eabe155 Fix error message 'assertion failed at' 2020-11-03 14:45:24 +01:00
Robert Hensing b43c13a916 BoehmGCStackAllocator: increase stack size to 8MB
The default stack size was not based on the normal stack size and
was too small.
2020-10-30 23:18:26 +01:00
Robert Hensing c4d903ddb0 Fix memory corruption caused by GC-invisible coroutine stacks
Crucially this introduces BoehmGCStackAllocator, but it also
adds a bunch of wiring to avoid making libutil depend on bdw-gc.

Part of the solutions for #4178, #4200
2020-10-30 21:21:59 +01:00
Eelco Dolstra 731edf0d9b isTrivial(): Support trivial lists 2020-10-26 20:37:11 +01:00
Eelco Dolstra 85c8be6286 Remove static variable name clashes
This was useful for an experiment with building Nix as a single
compilation unit. It's not very useful otherwise but also doesn't
hurt...
2020-10-06 13:49:20 +02:00
Eelco Dolstra e8e1d420f3 Don't include <regex> in header files
This reduces compilation time by ~15 seconds (CPU time).

Issue #4045.
2020-09-21 18:22:45 +02:00
Eelco Dolstra 0066ef6c59 Fix doc generation 2020-09-16 16:56:28 +02:00
regnat 250f8a4bba Escape ${ in strings when printing Nix expressions
Otherwise the result of the printing can't be parsed back correctly by
Nix (because the unescaped `${` will be parsed as the begining of an
anti-quotation).

Fix #3989
2020-09-14 17:19:25 +02:00
Silvan Mosberger 8dbd57a6a5
Fix auto argument passing for more auto arguments than formals
The change in 626200713b didn't account
for when the number of auto arguments is bigger than the number of
formal arguments. This causes the following:

  $ nix-instantiate --eval -E '{ ... }@args: args.foo' --argstr foo foo
  nix-instantiate: src/libexpr/attr-set.hh:55: void nix::Bindings::push_back(const nix::Attr&): Assertion `size_ < capacity_' failed.
  Aborted (core dumped)
2020-09-11 20:41:51 +02:00
Eelco Dolstra 84f5cabbea Merge remote-tracking branch 'origin/master' into markdown 2020-08-31 14:24:26 +02:00
Griffin Smith 626200713b Pass all args when auto-calling a function with an ellipsis
The command line options --arg and --argstr that are used by a bunch of
CLI commands to pass arguments to top-level functions in files go
through the same code-path as auto-calling top-level functions with
their default arguments - this, however, was only passing the arguments
that were *explicitly* mentioned in the formals of the function - in the
case of an as-pattern with an ellipsis (eg args @ { ... }) extra passed
arguments would get omitted. This fixes that to instead pass *all*
specified auto args in the case that our function has an ellipsis.

Fixes #598
2020-08-27 12:38:25 -04:00
Eelco Dolstra f53b5f1058
Add getDoc() function 2020-08-25 13:31:11 +02:00
Eelco Dolstra d9a8619762
Don't barf if corepkgs is in the store but not a valid path
This can happen when using a dummy store (or indeed any non-local store).
2020-08-24 19:15:17 +02:00
Eelco Dolstra 33b1679d75
Allow primops to have Markdown documentation 2020-08-24 13:16:02 +02:00
Eelco Dolstra 25ecfffdc3
Remove PrimOp constructor 2020-08-20 12:34:04 +02:00
Eelco Dolstra e5662ba652 Add a flag to start the REPL on evaluation errors
This allows interactively inspecting the state of the evaluator at the
point of failure.

Example:

  $ nix eval path:///home/eelco/Dev/nix/flake2#modules.hello-closure._final --start-repl-on-eval-errors
  error: --- TypeError -------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
  at: (20:53) in file: /nix/store/4264z41dxfdiqr95svmpnxxxwhfplhy0-source/flake.nix

      19|
      20|           _final = builtins.foldl' (xs: mod: xs // (mod._module.config { config = _final; })) _defaults _allModules;
        |                                                     ^
      21|         };

  attempt to call something which is not a function but a set

  Starting REPL to allow you to inspect the current state of the evaluator.

  The following extra variables are in scope: arg, fun

  Welcome to Nix version 2.4. Type :? for help.

  nix-repl> fun
  error: --- EvalError -------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
  at: (150:28) in file: /nix/store/4264z41dxfdiqr95svmpnxxxwhfplhy0-source/flake.nix

     149|
     150|           tarballClosure = (module {
        |                            ^
     151|             extends = [ self.modules.derivation ];

  attribute 'derivation' missing

  nix-repl> :t fun
  a set

  nix-repl> builtins.attrNames fun
  [ "tarballClosure" ]

  nix-repl>
2020-08-05 21:26:17 +02:00
John Ericson 92ad550e96 Merge remote-tracking branch 'obsidian/misc-ca' into derivation-primop-floating-output 2020-08-05 14:51:41 +00:00
Eelco Dolstra 088dcea0e8 Typo 2020-08-05 15:41:51 +02:00
John Ericson 9423f64ee2 Parse CA derivations using new output variants
We no longer need `ParsedDerivation` because everything libstore needs
to know about is in the `BasicDerivation` proper.
2020-07-22 23:59:25 +00:00
Eelco Dolstra 832e111494 Merge remote-tracking branch 'origin/master' into flakes 2020-07-14 13:56:18 +02:00
Eelco Dolstra c0dd05131e toStorePath(): Return a StorePath and the suffix 2020-07-13 16:25:48 +02:00
Eelco Dolstra 54712aaf8a Merge remote-tracking branch 'origin/master' into flakes 2020-07-06 16:40:10 +02:00
Ben Burdette bf2788e4c1 move showTrace to new loggerSettings 2020-07-02 09:04:31 -06:00
Ben Burdette 70bcb39d3f double addtrace for 'called from' 2020-06-30 15:44:19 -06:00
Eelco Dolstra 50f13b06fb EvalCache: Store string contexts 2020-06-29 19:08:37 +02:00
Eelco Dolstra adf2fbbdc2 Merge remote-tracking branch 'origin/master' into flakes 2020-06-26 08:46:46 +02:00
Ben Burdette 023912def3 convenience form of addTrace 2020-06-24 13:46:25 -06:00
Ben Burdette 1d43a6e123 use plain errPos instead of nixCode; fix tests 2020-06-23 15:30:13 -06:00
Ben Burdette 54e8f550c9 addErrorTrace 2020-06-19 13:44:08 -06:00
Eelco Dolstra 2a61bbf77f Some backports from the flakes branch 2020-06-18 14:03:00 +02:00
Eelco Dolstra 377345e26f Remove unneeded #include 2020-06-18 13:47:05 +02:00
Eelco Dolstra 7083d33efe Make constant primops lazy
(cherry picked from commit aa0e2a2e70)
2020-06-18 13:42:47 +02:00
Eelco Dolstra 1524752c17 Merge remote-tracking branch 'origin/master' into flakes 2020-06-17 10:26:52 +02:00
Eelco Dolstra 1fb762d11f Get rid of explicit ErrorInfo constructors 2020-06-15 14:06:58 +02:00
Eelco Dolstra fd64e4fb96 Disambiguate BaseError(Args) constructor
This means that 'throw Error({ ... ErrorInfo ... })' now works.
2020-06-15 13:50:33 +02:00
Eelco Dolstra c20591ddc3 Merge remote-tracking branch 'origin/master' into flakes 2020-06-03 16:15:22 +02:00
Ben Burdette 734283d636 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-05-29 09:51:37 -06:00
Carlo Nucera 6d73c10041 Merge remote-tracking branch 'origin/master' into enum-FileIngestionMethod 2020-05-26 11:14:08 -04:00
Ben Burdette 2a19bf8619 move pos to the first arg, to indicate its not used in a fmt template 2020-05-12 11:27:37 -06:00
Ben Burdette ec870b9c85 new pos format for more errors 2020-05-12 10:52:26 -06:00
Ben Burdette 59b1f5c701 Merge branch 'master' into errors-phase-2 2020-05-11 14:35:30 -06:00
Ben Burdette 55eb717148 add pos to errorinfo, remove from hints 2020-05-08 18:18:28 -06:00
Eelco Dolstra b4e23dcd9e nix search: Search legacyPackages recursively 2020-04-27 16:29:26 +02:00
Eelco Dolstra ef4d3fc111 Merge remote-tracking branch 'origin/master' into flakes 2020-04-23 15:16:18 +02:00
Ben Burdette e4fb9a3849 remove 'format' from Error constructor calls 2020-04-21 17:07:07 -06:00
Domen Kožar 25ed842725
Merge pull request #3502 from NixOS/more-pos
pass Pos to forceValue to improve infinite recursion error
2020-04-18 14:05:21 +02:00
Eelco Dolstra aaa109565e Use a more space/time-efficient representation for the eval cache 2020-04-17 23:04:21 +02:00
Eelco Dolstra f89349f07e Merge remote-tracking branch 'origin/master' into flakes 2020-04-16 18:33:10 +02:00
Eelco Dolstra b3e5eea4a9 Add function to allocate a Value in traceable memory 2020-04-16 17:30:05 +02:00
Domen Kožar b865b5b40c
pass Pos to forceValue to improve infinite recursion error 2020-04-16 12:32:07 +02:00
Eelco Dolstra d103c79144
Merge remote-tracking branch 'origin/master' into flakes 2020-04-09 15:54:21 +02:00
Domen Kožar a693a9fa4b
Attach pos to if expression errors 2020-04-09 09:45:15 +02:00
Nikola Knezevic f5095594e7 datatransfer.{cc,hh} -> filetransfer.{cc,hh} 2020-04-08 22:26:57 +02:00
Nikola Knezevic 741e9012d3 Rename src/lib/download.* to src/lib/datatransfer.* 2020-04-08 22:26:57 +02:00
John Ericson 225e62a56a Replace some bool recursive with a new FileIngestionMethod enum 2020-03-29 15:16:20 -04:00
Eelco Dolstra d1165d8791
Require shallow clones to be requested explicitly
If you do a fetchTree on a Git repository, whether the result contains
a revCount attribute should not depend on whether that repository
happens to be a shallow clone or not. That would complicate caching a
lot and would be semantically messy. So applying fetchTree/fetchGit to
a shallow repository is now an error unless you pass the attribute
'shallow = true'. If 'shallow = true', we don't return revCount, even
if the repository is not actually shallow.

Note that Nix itself is not doing shallow clones at the moment. But it
could do so as an optimisation if the user specifies 'shallow = true'.

Issue #2988.
2020-03-17 22:35:29 +01:00
Eelco Dolstra 6fadb3fc03 Merge remote-tracking branch 'origin/master' into flakes 2020-01-21 21:18:52 +01:00
Eelco Dolstra c3181e21e7 Tweak error message 2020-01-13 21:52:03 +01:00
Daiderd Jordan 307bcb9a8e
libexpr: show expression in assertion errors
Includes the expression of the condition in the assertion message if
the assertion failed, making assertions much easier to debug. eg.

    error: assertion (withPython -> (python2Packages != null)) failed at pkgs/tools/security/nmap/default.nix:11:1
2020-01-11 15:45:41 +01:00
Eelco Dolstra 1dc29df1d3 Merge remote-tracking branch 'origin/master' into flakes 2020-01-06 12:43:07 +01:00
Eelco Dolstra cb90e382b5 Hide FunctionCallTrace constructor/destructor
This prevents them from being inlined. On gcc 9, this reduces the
stack size needed for

  nix-instantiate '<nixpkgs>' -A texlive.combined.scheme-full --dry-run

from 12.9 MiB to 4.8 MiB.
2020-01-05 16:21:34 +01:00
Eelco Dolstra ecb3a1afa2 Merge remote-tracking branch 'origin/master' into flakes 2019-12-11 14:53:30 +01:00
Eelco Dolstra bbe97dff8b Make the Store API more type-safe
Most functions now take a StorePath argument rather than a Path (which
is just an alias for std::string). The StorePath constructor ensures
that the path is syntactically correct (i.e. it looks like
<store-dir>/<base32-hash>-<name>). Similarly, functions like
buildPaths() now take a StorePathWithOutputs, rather than abusing Path
by adding a '!<outputs>' suffix.

Note that the StorePath type is implemented in Rust. This involves
some hackery to allow Rust values to be used directly in C++, via a
helper type whose destructor calls the Rust type's drop()
function. The main issue is the dynamic nature of C++ move semantics:
after we have moved a Rust value, we should not call the drop function
on the original value. So when we move a value, we set the original
value to bitwise zero, and the destructor only calls drop() if the
value is not bitwise zero. This should be sufficient for most types.

Also lots of minor cleanups to the C++ API to make it more modern
(e.g. using std::optional and std::string_view in some places).
2019-12-10 22:06:05 +01:00
Eelco Dolstra 98ef11677c EvalState::callFunction(): Make FunctionCallTrace use less stack space
The FunctionCallTrace object consumes a few hundred bytes of stack
space, even when tracing is disabled. This was causing stack overflows:

  $ nix-instantiate '<nixpkgs> -A texlive.combined.scheme-full --dry-run
  error: stack overflow (possible infinite recursion)

This is with the default stack size of 8 MiB.

Putting the object on the heap reduces stack usage to < 5 MiB.
2019-12-10 13:32:30 +01:00
Eelco Dolstra 0678e4d56a Move #include
(cherry picked from commit 8beedd4486)
2019-12-05 20:30:29 +01:00
Eelco Dolstra c3c23a52ee Merge remote-tracking branch 'origin/master' into flakes 2019-12-04 00:31:09 +01:00
Eelco Dolstra 2d6f1ddbb5
Remove builtins.valueSize
Fixes #3246.
2019-11-28 13:52:42 +01:00
puck cd55f91ad2 Ensure enough space in attrset bindings when using both __overrides and dynamic attributes 2019-11-25 12:37:14 +00:00
Eelco Dolstra d12d69ea1a
Turn NIX_PATH into a config setting
This allows it to be set in nix.conf.
2019-11-22 23:07:35 +01:00
Eelco Dolstra ec9dd9a5ae
Provide a default value for NIX_PATH 2019-11-22 22:08:51 +01:00
Eelco Dolstra ba87b08f85
getEnv(): Return std::optional
This allows distinguishing between an empty value and no value.
2019-11-22 16:18:13 +01:00
Eelco Dolstra 8beedd4486 Move #include 2019-11-20 13:04:39 +01:00
Eelco Dolstra 5ee23c35b9
Merge pull request #3219 from Ericson2314/semicolons
Fix extra semicolons warnings
2019-11-11 12:13:51 +01:00
John Ericson 4c34054673 Remove unneeded semicolons 2019-11-10 11:24:47 -05:00
Eelco Dolstra b81d9d26f5
Merge remote-tracking branch 'origin/master' into flakes 2019-11-04 22:29:31 +01:00
Eelco Dolstra e012384fe9
Merge branch 'tojson-tostring-fix' of https://github.com/mayflower/nix 2019-10-27 12:18:35 +01:00
Robin Gloster e583df5280
builtins.toJSON: fix __toString usage 2019-10-27 10:15:51 +01:00
Eelco Dolstra e99bb91217
Merge remote-tracking branch 'origin/master' into flakes 2019-10-10 12:54:37 +02:00
Eelco Dolstra 99b73fb507
OCD performance fix: {find,count}+insert => insert 2019-10-09 16:06:29 +02:00
Eelco Dolstra f97d3753a1
Require flake.nix to be an attrset (not a non-trivial thunk) 2019-09-09 17:34:38 +02:00
Eelco Dolstra c87840ae14
Don't allow arbitrary computations in flake attributes
E.g. you can write 'edition = 201909' but not 'edition = 201909 + 0'.

Fixes #3075.
2019-09-09 16:34:44 +02:00
Eelco Dolstra 89468410d5 Extract flake dependencies from the 'outputs' arguments
That is, instead of

  inputs = [ "nixpkgs" ];

  outputs = inputs: ... inputs.nixpkgs ...;

you can write

  outputs = { nixpkgs }: ... inputs.nixpkgs ...;
2019-08-30 11:22:34 +02:00
Eelco Dolstra ebc4dae517 Merge remote-tracking branch 'origin/master' into flakes 2019-08-29 16:11:38 +02:00
Graham Christensen ee9c988a1b
Track function start and ends for flame graphs
With this patch, and this file I called `log.py`:

    #!/usr/bin/env nix-shell
    #!nix-shell -i python3 -p python3 --pure

    import sys
    from pprint import pprint

    stack = []
    timestack = []

    for line in open(sys.argv[1]):
        components = line.strip().split(" ", 2)
        if components[0] != "function-trace":
            continue

        direction = components[1]
        components = components[2].rsplit(" ", 2)

        loc = components[0]
        _at = components[1]
        time = int(components[2])

        if direction == "entered":
            stack.append(loc)
            timestack.append(time)
        elif direction == "exited":
            dur = time - timestack.pop()
            vst = ";".join(stack)
            print(f"{vst} {dur}")
            stack.pop()

and:

    nix-instantiate --trace-function-calls -vvvv ../nixpkgs/pkgs/top-level/release.nix -A unstable > log.matthewbauer 2>&1
    ./log.py ./log.matthewbauer > log.matthewbauer.folded
    flamegraph.pl --title matthewbauer-post-pr log.matthewbauer.folded > log.matthewbauer.folded.svg

I can make flame graphs like: http://gsc.io/log.matthewbauer.folded.svg

---

Includes test cases around function call failures and tryEval. Uses
RAII so the finish is always called at the end of the function.
2019-08-14 16:09:35 -04:00
Eelco Dolstra 54aff8430c
Move flake-related stuff to src/libexpr/flake 2019-06-05 16:51:54 +02:00
Eelco Dolstra 66f1d7ee95 Fetch the flake registry from the NixOS/flake-registry repo 2019-05-22 22:56:46 +02:00
Eelco Dolstra 2bc55aba1e
Merge remote-tracking branch 'origin/master' into flakes 2019-05-08 14:30:27 +02:00
Eelco Dolstra 3c171851a8 Make the URL/path of the global flake registry configurable 2019-05-07 22:49:13 +02:00
Eelco Dolstra bb6e6923f2 Add environment variable NIX_SHOW_SYMBOLS for dumping the symbol table 2019-04-11 23:04:13 +02:00
Eelco Dolstra b5565a7081 Merge remote-tracking branch 'origin/master' into flakes 2019-03-26 11:44:14 +01:00
Linus Heckemann 2aa89daab3 eval: improve type description for primops and applied primops
This can make type errors a little easier to understand.
2019-03-21 15:31:46 +01:00
Eelco Dolstra 91a6a47b0e
Improve flake references 2019-02-12 18:23:11 +01:00
Eelco Dolstra 7a5cf31060
Initial flake support 2019-02-11 12:00:13 +01:00
Eelco Dolstra aa0e2a2e70
Make constant primops lazy 2019-02-11 11:52:07 +01:00
Will Dietz 21ea00d3ec EvalState::resetFileCache: clear parse cache as well as eval cache
Fixes #2546.

(at least the basic reproduction I've been testing)
2018-12-31 10:18:28 -06:00
Eelco Dolstra 91405986f4
Convert NIX_COUNT_CALLS to JSON too 2018-09-05 21:57:54 +02:00
Eelco Dolstra 0a2545f95c
Log stats to stderr
We shouldn't pollute stdout.
2018-09-05 21:35:58 +02:00
Michael Bishop 4b034f390c remove the old text format output 2018-09-02 18:25:23 -03:00
Michael Bishop 2fd1008c70 add JSON to NIX_SHOW_STATS 2018-09-01 20:05:06 -03:00
Michael Bishop c29e5fbb13 improve the stats when profiling 2018-09-01 17:11:56 -03:00
Eelco Dolstra bc65e02d96
Merge pull request #2326 from aszlig/fix-symlink-leak
Fix symlink leak in restricted eval mode
2018-08-03 17:01:34 +02:00
aszlig 43e28a1b75
Fix symlink leak in restricted eval mode
In EvalState::checkSourcePath, the path is checked against the list of
allowed paths first and later it's checked again *after* resolving
symlinks.

The resolving of the symlinks is done via canonPath, which also strips
out "../" and "./". However after the canonicalisation the error message
pointing out that the path is not allowed prints the symlink target in
the error message.

Even if we'd suppress the message, symlink targets could still be leaked
if the symlink target doesn't exist (in this case the error is thrown in
canonPath).

So instead, we now do canonPath() without symlink resolving first before
even checking against the list of allowed paths and then later do the
symlink resolving and checking the allowed paths again.

The first call to canonPath() should get rid of all the "../" and "./",
so in theory the only way to leak a symlink if the attacker is able to
put a symlink in one of the paths allowed by restricted evaluation mode.

For the latter I don't think this is part of the threat model, because
if the attacker can write to that path, the attack vector is even
larger.

Signed-off-by: aszlig <aszlig@nix.build>
2018-08-03 06:46:43 +02:00
Eelco Dolstra 45bcf5416a
Merge branch 'prim_mapAttr-fix' of https://github.com/volth/nix 2018-07-31 20:05:07 +02:00
Eelco Dolstra 875cd9da2b
Remove unused function printStats2()
Closes #2282.
2018-07-11 20:29:18 +02:00
volth 627e28ba33 prim_mapAttrs: `f' must be evaluated lazily to avoid infinite recursion 2018-07-06 21:52:54 +00:00
Eelco Dolstra c905d8b0a8
GC_malloc -> GC_MALLOC
This makes it possible to build with -DGC_DEBUG.
2018-06-12 17:49:55 +02:00
Eelco Dolstra 455d1f01d0
Don't scan for roots in dynamic libraries
This reduces the risk of object liveness misdetection. For example,
Glibc has an internal variable "mp_" that often points to a Boehm
object, keeping it alive unnecessarily. Since we don't store any
actual roots in global variables, we can just disable data segment
scanning.

With this, the max RSS doing 100 evaluations of
nixos.tests.firefox.x86_64-linux.drvPath went from 718 MiB to 455 MiB.
2018-06-12 17:49:55 +02:00
Eelco Dolstra 30964103dc
Add temporary stats 2018-06-12 17:49:55 +02:00
Eelco Dolstra 24c6806994
Cache parse trees
This prevents EvalState::resetFileCache() from parsing everything all
over again.
2018-06-12 17:49:55 +02:00
Eelco Dolstra 6ad0a2f749
Remove duplicate definition of allocBytes() 2018-06-12 17:49:51 +02:00
Eelco Dolstra 5d4a9d5677
Fix static assertion failure on 32-bit systems 2018-05-30 17:47:30 +02:00
Eelco Dolstra 1672bcd230
Move evaluator-specific settings out of libstore 2018-05-30 13:29:50 +02:00
Will Dietz 93ae90de0f eval.cc: add message to static_assert, message can be omitted w/c++17 2018-05-26 14:04:41 -05:00
Eelco Dolstra 4bb8741b98
Make Env self-describing
If the Env denotes a 'with', then values[0] may be an Expr* cast to a
Value*. For code that generically traverses Values/Envs, it's useful
to know this.
2018-05-22 16:02:32 +02:00
Eelco Dolstra 9fd7cf98db
Memoise checkSourcePath()
This prevents hydra-eval-jobs from statting the same files over and
over again.
2018-05-22 13:02:14 +02:00
Eelco Dolstra a91c4ca01f
In restricted eval mode, allow access to the closure of store paths
E.g. this makes

  nix eval --restrict-eval -I /nix/store/foo '(builtins.readFile "/nix/store/foo/symlink/bla")'

(where /nix/store/foo/symlink is a symlink to another path in the
closure of /nix/store/foo) succeed.

This fixes a regression in Hydra compared to Nix 1.x (where there were
no restrictions at all on access to the Nix store).
2018-05-09 15:45:05 +02:00
Eelco Dolstra 53ec5ac69f
Fix some random -Wconversion warnings 2018-05-02 13:56:34 +02:00
Tuomas Tynkkynen af86132e1a libexpr: Make unsafeGetAttrPos not crash on noPos
Currently e.g. `builtins.unsafeGetAttrPos "abort" builtins` will
eventually segfault because pos->file is an unset Symbol.

Found by afl-fuzz.
2018-04-03 15:54:42 +03:00
Shea Levy e2088febf3
concatLists: Don't pass NULL pointers to memcpy.
This is UB, even if the size is 0. See #1976.

Fixes #1976.
2018-03-14 23:44:02 -04:00
Shea Levy 14ca85688c
Actually fix nixDataDir in non-canonical path 2018-02-28 06:19:40 -05:00
Eelco Dolstra 179b896acb
Merge branch 'data-dir-non-canon' of https://github.com/shlevy/nix 2018-02-22 14:20:43 +01:00
Shea Levy ddbcd01c83
Fix restricted mode when installing in non-canonical data dir 2018-02-22 07:18:14 -05:00
Tuomas Tynkkynen 7e0360504d libexpr: Optimize prim_derivationStrict by using more symbol comparisons 2018-02-17 16:54:21 +02:00
Tuomas Tynkkynen 0845cdf944 libexpr: Rely on Boehm returning zeroed memory in EvalState::allocEnv()
Boehm guarantees that memory returned by GC_malloc() is zeroed, so take
advantage of that.
2018-02-17 16:54:21 +02:00
Eelco Dolstra 7828dca9e8
Merge branch 'register-constant' of https://github.com/shlevy/nix 2018-02-13 12:24:48 +01:00
Frederik Rietdijk 60eca58533 Nix stats: flatten statistics
Flattens the list of statistics as suggested in
https://github.com/NixOS/ofborg/issues/67. This makes it easier to work
with.
2018-02-11 14:37:50 +01:00
Shea Levy 081f14a169
Allow using RegisterPrimop to define constants.
This enables plugins to add new constants, as well as new primops.
2018-02-08 14:35:50 -05:00
Eelco Dolstra abe6be578b
Merge pull request #1816 from shlevy/add-path
Add path primop.
2018-02-07 13:32:35 +01:00
Shea Levy 69d82e5c58
Add path primop.
builtins.path allows specifying the name of a path (which makes paths
with store-illegal names now addable), allows adding paths with flat
instead of recursive hashes, allows specifying a filter (so is a
generalization of filterSource), and allows specifying an expected
hash (enabling safe path adding in pure mode).
2018-02-06 16:48:08 -05:00
Eelco Dolstra f24e726ba5
checkURI(): Check file URIs against allowedPaths
This makes e.g. 'fetchGit ./.' work (assuming that ./. is an allowed
path).
2018-02-06 14:35:33 +01:00
Eelco Dolstra 89a2a11d9f
Don't use [[noreturn]] 2018-01-19 15:00:38 +01:00
Eelco Dolstra d4dcffd643
Add pure evaluation mode
In this mode, the following restrictions apply:

* The builtins currentTime, currentSystem and storePath throw an
  error.

* $NIX_PATH and -I are ignored.

* fetchGit and fetchMercurial require a revision hash.

* fetchurl and fetchTarball require a sha256 attribute.

* No file system access is allowed outside of the paths returned by
  fetch{Git,Mercurial,url,Tarball}. Thus 'nix build -f ./foo.nix' is
  not allowed.

Thus, the evaluation result is completely reproducible from the
command line arguments. E.g.

  nix build --pure-eval '(
    let
      nix = fetchGit { url = https://github.com/NixOS/nixpkgs.git; rev = "9c927de4b179a6dd210dd88d34bda8af4b575680"; };
      nixpkgs = fetchGit { url = https://github.com/NixOS/nixpkgs.git; ref = "release-17.09"; rev = "66b4de79e3841530e6d9c6baf98702aa1f7124e4"; };
    in (import (nix + "/release.nix") { inherit nix nixpkgs; }).build.x86_64-linux
  )'

The goal is to enable completely reproducible and traceable
evaluation. For example, a NixOS configuration could be fully
described by a single Git commit hash. 'nixos-rebuild' would do
something like

  nix build --pure-eval '(
    (import (fetchGit { url = file:///my-nixos-config; rev = "..."; })).system
  ')

where the Git repository /my-nixos-config would use further fetchGit
calls or Git externals to fetch Nixpkgs and whatever other
dependencies it has. Either way, the commit hash would uniquely
identify the NixOS configuration and allow it to reproduced.
2018-01-16 19:23:18 +01:00
Eelco Dolstra 74f75c8558
import, builtins.readFile: Handle diverted stores
Fixes #1791
2018-01-12 17:31:08 +01:00
Eelco Dolstra 812e027e1d
Add option allowed-uris
This allows network access in restricted eval mode.
2017-10-30 12:41:49 +01:00
Eelco Dolstra c1ae18941a
nix repl :t: Indicate strings that have a context 2017-10-29 17:11:22 +01:00
Jörg Thalheim 2fd8f8bb99 Replace Unicode quotes in user-facing strings by ASCII
Relevant RFC: NixOS/rfcs#4

$ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-07-30 12:32:45 +01:00