Commit graph

11560 commits

Author SHA1 Message Date
Eelco Dolstra be64fb9b51 DerivationGoal::loadDerivation(): Don't use derivationFromPath()
This causes a recursive call to ensurePath(), which is not a good
idea.
2022-01-07 13:22:34 +01:00
pennae 26a8b220eb avoid ostream sentries per json string character
we don't have to create an ostream sentry object for every character of a JSON
string we write. format a bunch of characters and flush them to the stream all
at once instead.

this doesn't affect small numbers of string characters, but larger numbers of
total JSON string characters written gain a lot. at 1MB of total string written
we gain almost 30%, at 16MB it's almost a factor of 3x. large numbers of JSON
string characters do occur naturally in a nixos system evaluation to generate
documentation (though this is now somewhat mitigated by caching the largest part
of nixos option docs).

benchmarked with

  hyperfine 'nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) {e})"' --warmup 1 -L e 1,4,256,4096,65536

before:

  Benchmark 1: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 1)"
    Time (mean ± σ):      12.5 ms ±   0.2 ms    [User: 9.2 ms, System: 4.0 ms]
    Range (min … max):    11.9 ms …  13.1 ms    223 runs

  Benchmark 2: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 4)"
    Time (mean ± σ):      12.5 ms ±   0.2 ms    [User: 9.3 ms, System: 3.8 ms]
    Range (min … max):    11.9 ms …  13.2 ms    220 runs

  Benchmark 3: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 256)"
    Time (mean ± σ):      13.2 ms ±   0.3 ms    [User: 9.8 ms, System: 4.0 ms]
    Range (min … max):    12.6 ms …  14.3 ms    205 runs

  Benchmark 4: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 4096)"
    Time (mean ± σ):      24.0 ms ±   0.4 ms    [User: 19.4 ms, System: 5.2 ms]
    Range (min … max):    22.7 ms …  25.8 ms    119 runs

  Benchmark 5: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 65536)"
    Time (mean ± σ):     196.0 ms ±   3.7 ms    [User: 171.2 ms, System: 25.8 ms]
    Range (min … max):   190.6 ms … 201.5 ms    14 runs

after:

  Benchmark 1: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 1)"
    Time (mean ± σ):      12.4 ms ±   0.3 ms    [User: 9.1 ms, System: 4.0 ms]
    Range (min … max):    11.7 ms …  13.3 ms    204 runs

  Benchmark 2: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 4)"
    Time (mean ± σ):      12.4 ms ±   0.2 ms    [User: 9.2 ms, System: 3.9 ms]
    Range (min … max):    11.8 ms …  13.0 ms    214 runs

  Benchmark 3: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 256)"
    Time (mean ± σ):      12.6 ms ±   0.2 ms    [User: 9.5 ms, System: 3.8 ms]
    Range (min … max):    12.1 ms …  13.3 ms    209 runs

  Benchmark 4: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 4096)"
    Time (mean ± σ):      15.9 ms ±   0.2 ms    [User: 11.4 ms, System: 5.1 ms]
    Range (min … max):    15.2 ms …  16.4 ms    171 runs

  Benchmark 5: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 65536)"
    Time (mean ± σ):      69.0 ms ±   0.9 ms    [User: 44.3 ms, System: 25.3 ms]
    Range (min … max):    67.2 ms …  70.9 ms    42 runs
2022-01-07 06:53:47 +01:00
Farid Zakaria e36fdbbfd9 Check for --meta before emitting it for --json
Check that the meta flag is present when emitting JSON query information
for `nix-env`.

fixes #5882
2022-01-06 14:40:02 -08:00
Eelco Dolstra f71d84672b Merge branch 'install-nlohmann-json-headers' of https://github.com/hercules-ci/nix 2022-01-06 22:34:03 +01:00
Eelco Dolstra 5c7d3b351f
Merge pull request #5878 from fzakaria/faridzakaria/json-outputs
Add outputs to JSON query
2022-01-06 22:33:11 +01:00
Eelco Dolstra 3ec02c6743
Merge pull request #5546 from NixOS/allow-paths-in-flake-local-options
allow paths in flake local options
2022-01-06 21:55:20 +01:00
Farid Zakaria 65257614ea Add outputs to JSON query
Emit output information when printing JSON information
and `--out-paths` is requested.

fixes #5877
2022-01-06 09:13:18 -08:00
Domen Kožar bdc577936f
Merge pull request #5749 from GavinRay97/patch-1
Better diagnostics if no valid signature found
2022-01-06 08:39:58 +01:00
Ben Burdette 84aeb74377 revert value-add 2022-01-05 14:25:45 -07:00
Ben Burdette bf8a065be0 add colors; remove headings 2022-01-05 12:28:31 -07:00
Ben Burdette a4d8a799b7 tidy up debugtraces 2022-01-05 12:21:18 -07:00
Eelco Dolstra 6d7844695b
Merge pull request #5864 from edolstra/attr-sets-cleanup
Attrset improvements
2022-01-05 10:11:02 +01:00
Eelco Dolstra 2b4c944823 Remove EvalState::mkAttrs() 2022-01-04 20:29:17 +01:00
Eelco Dolstra 17daec0b83 Move empty attrset optimisation 2022-01-04 19:23:11 +01:00
Eelco Dolstra ca5baf2392 Turn mkString(Symbol) into a method 2022-01-04 19:09:40 +01:00
Eelco Dolstra ed93aec3c3 Remove non-method mkPath() 2022-01-04 18:45:16 +01:00
Eelco Dolstra 263a8d293c Remove non-method mk<X> functions 2022-01-04 18:40:39 +01:00
Eelco Dolstra cc08364315 Remove non-method mkString() 2022-01-04 18:24:42 +01:00
Eelco Dolstra 6d9a6d2cc3 Ensure that attrsets are sorted
Previously you had to remember to call value->attrs->sort() after
populating value->attrs. Now there is a BindingsBuilder helper that
wraps Bindings and ensures that sort() is called before you can use
it.
2022-01-04 18:00:33 +01:00
Eelco Dolstra 8f214a3771
Merge pull request #5862 from SuperSandro2000/patch-1
Add iana-etc for /etc/protocols to default packages
2022-01-04 15:19:35 +01:00
Sandro 9dd45dfc3d
Add iana-etc for /etc/protocols to default packages 2022-01-04 12:49:02 +01:00
Eelco Dolstra 1ffacad8a5
Merge pull request #5830 from pennae/zipAttrsWith
add zipAttrsWith primop
2022-01-04 11:16:50 +01:00
Ben Burdette 1b6b33d43d filter out underscore names 2022-01-03 18:29:43 -07:00
Ben Burdette c669108981 merge cleanup 2022-01-03 18:13:16 -07:00
Ben Burdette a47de1ac37 Merge branch 'master' into debug-exploratory-PR 2022-01-03 16:08:28 -07:00
pennae 00c993f48b add zipAttrsWith primop
nixpkgs can save a good bit of eval memory with this primop. zipAttrsWith is
used quite a bit around nixpkgs (eg in the form of recursiveUpdate), but the
most costly application for this primop is in the module system. it improves
the implementation of zipAttrsWith from nixpkgs by not checking an attribute
multiple times if it occurs more than once in the input list, allocates less
values and set elements, and just avoids many a temporary object in general.

nixpkgs has a more generic version of this operation, zipAttrsWithNames, but
this version is only used once so isn't suitable for being the base of a new
primop. if it were to be used more we should add a second primop instead.
2022-01-03 21:05:53 +01:00
Eelco Dolstra 96d08fcd66
Merge pull request #5839 from tweag/balsoft/yet-another-follows-bugfix
flake.cc: computeLocks: Only verify overrides when they could change
2022-01-03 20:51:58 +01:00
Eelco Dolstra 70dfcbbb37
Merge pull request #5840 from tweag/balsoft/nix-repl-show-trace
nix repl: fix --show-trace and add the ability to set trace display
2022-01-03 20:41:59 +01:00
Eelco Dolstra d7b286fc77
Merge pull request #5844 from Kha/fix-ifd-chroot-once-more
Fix IFD with chroot store
2022-01-03 20:41:29 +01:00
Eelco Dolstra 1c099dee67
Merge pull request #5852 from autc04/docker-tmp-permissions
docker.nix: set correct permissions on /tmp (fix #5851)
2022-01-03 14:35:31 +01:00
Wolfgang Thaller d9aabb7acb docker.nix: set correct permissions on /tmp 2022-01-01 16:47:05 +01:00
Sebastian Ullrich d0c8e9254e Fix IFD with chroot store 2021-12-29 19:00:02 +01:00
Alexander Bantyev a26351da02
Add ability to toggle show-trace from within the repl 2021-12-29 02:54:36 +03:00
Eelco Dolstra ed3bc63666
Merge pull request #5841 from lilyinstarlight/fix-fromtoml-attrs
Sort attrs from tables in fromTOML
2021-12-29 00:03:10 +01:00
Lily Foster 92c7d33ee3 Sort attrs from tables in fromTOML
This was dropped in 10a8b5d for the migration from cpptoml to toml11 but
seems to be necessary for the attrsets to work correctly.

Fixes #5833
2021-12-28 10:11:00 -05:00
Alexander Bantyev 2dc29e0d93
Add a test that nix repl --show-trace actually shows the trace 2021-12-28 15:54:46 +03:00
Alexander Bantyev 581f774284
BaseError::calcWhat: take loggerSettings.showTrace into account
Text representation for errors should include the trace if
--show-trace is passed.
2021-12-28 15:53:21 +03:00
Alexander Bantyev 2664a216e5
flake.cc: computeLocks: Only verify overrides when they could change
When we check for disappeared overrides, we can get "false positives"
for follows and overrides which are defined in the dependencies of the
flake we are locking, since they are not parsed by
parseFlakeInputs. However, at that point we already know that the
overrides couldn't have possible been changed if the input itself
hasn't changed (since we check that oldLock->originalRef == *input.ref
for the input's parent). So, to prevent this, only perform this check
when it was possible that the flake changed (e.g. the flake we're
locking, or a new input, or the input has changed and mustRefetch ==
true).
2021-12-28 15:30:06 +03:00
Ben Burdette 5954cbf3e9 more cleanup 2021-12-27 18:29:55 -07:00
Ben Burdette 4610e02d04 remove debug code 2021-12-27 18:12:46 -07:00
Ben Burdette 9760fa8661 add DebugTrace for the current error 2021-12-27 17:35:27 -07:00
Ben Burdette 6801a423fc :d env 2021-12-27 16:28:45 -07:00
Ben Burdette 2a66c120e6 by refernce for addVarToScope 2021-12-27 14:48:34 -07:00
Ben Burdette ff82ba98b4 don't add builtins to extras, initEnv() in regular repl 2021-12-27 14:06:04 -07:00
Ben Burdette d0d5890445 don't add underscore names to extras 2021-12-27 13:47:35 -07:00
Eelco Dolstra 0e90b13ab1
Merge pull request #5835 from yorickvP/fast-repl-load
Fix accidental O(n^2 * log n) performance in NixRepl::addAttrsToScope
2021-12-27 16:56:25 +01:00
Yorick a4ab0a74d9
Fix accidental O(n^2 * log n) performance in NixRepl::addAttrsToScope
Only sort once, after adding all of the attrs first. This reduces my
`nix repl '<nixpkgs>'` loading time from 1.07s to 103ms.

Fixes #5823
2021-12-27 13:18:55 +01:00
Ben Burdette e5eebda194 DebugTrace 2021-12-23 13:36:39 -07:00
tomberek f404189368
Merge branch 'master' into flake_search 2021-12-23 15:22:52 -05:00
Ben Burdette deb1fd66e8 makeDebugTraceStacker 2021-12-23 09:08:41 -07:00