Commit graph

12299 commits

Author SHA1 Message Date
pennae 8e2eaaaf69 make Finally more local
no need for function<> with c++17 deduction. this saves allocations and virtual
calls, but has the same semantics otherwise. not going through function has the
side effect of giving compilers more insight into the cleanup code, so we need a
few local warning disables.
2022-03-09 00:16:50 +01:00
pennae 47baa9d43c make Pos smaller
reduces peak hep memory use on eval of our test system from 264.4MB to 242.3MB,
possibly also a slight performance boost.

theoretically memory use could be cut down by another eight bytes per Pos on
average by turning it into a tuple containing an index into a global base
position table with row and column offsets, but that doesn't seem worth the
effort at this point.
2022-03-08 23:30:18 +01:00
pennae c96460f352 force-inline a few much-used functions
these functions are called a whole lot, and they're all comparatively small.
always inlining them gives ~0.7% performance boost on eval.

before:

  Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      6.935 s ±  0.052 s    [User: 5.852 s, System: 0.853 s]
    Range (min … max):    6.808 s …  7.026 s    20 runs

  Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     329.8 ms ±   2.7 ms    [User: 299.0 ms, System: 30.8 ms]
    Range (min … max):   326.6 ms … 336.5 ms    20 runs

  Benchmark 3: nix flakes eval --raw --impure --file expr.nix
    Time (mean ± σ):      2.655 s ±  0.038 s    [User: 2.364 s, System: 0.220 s]
    Range (min … max):    2.574 s …  2.737 s    20 runs

after:

  Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      6.912 s ±  0.036 s    [User: 5.823 s, System: 0.856 s]
    Range (min … max):    6.849 s …  6.980 s    20 runs

  Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     325.1 ms ±   2.5 ms    [User: 293.2 ms, System: 31.8 ms]
    Range (min … max):   322.2 ms … 332.8 ms    20 runs

  Benchmark 3: nix flakes eval --raw --impure --file expr.nix
    Time (mean ± σ):      2.636 s ±  0.024 s    [User: 2.352 s, System: 0.226 s]
    Range (min … max):    2.574 s …  2.681 s    20 runs
2022-03-08 23:30:18 +01:00
pennae 60ed4e908a cache singleton Envs just like Values
vast majority of envs is this size.

before:

  Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      6.946 s ±  0.041 s    [User: 5.875 s, System: 0.835 s]
    Range (min … max):    6.834 s …  7.005 s    20 runs

  Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     330.3 ms ±   2.5 ms    [User: 299.2 ms, System: 30.9 ms]
    Range (min … max):   327.5 ms … 337.7 ms    20 runs

  Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.671 s ±  0.035 s    [User: 2.370 s, System: 0.232 s]
    Range (min … max):    2.597 s …  2.749 s    20 runs

after:

  Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      6.935 s ±  0.052 s    [User: 5.852 s, System: 0.853 s]
    Range (min … max):    6.808 s …  7.026 s    20 runs

  Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     329.8 ms ±   2.7 ms    [User: 299.0 ms, System: 30.8 ms]
    Range (min … max):   326.6 ms … 336.5 ms    20 runs

  Benchmark 3: nix flakes eval --raw --impure --file expr.nix
    Time (mean ± σ):      2.655 s ±  0.038 s    [User: 2.364 s, System: 0.220 s]
    Range (min … max):    2.574 s …  2.737 s    20 runs
2022-03-08 23:30:18 +01:00
pennae 4b2b0d3a55 remove GC_PTR_STORE_AND_DIRTY
turns out it's only necessary for MANUAL_VDB, which nix doesn't use. omitting
them gives a slight performance improvement on eval.

before:

  Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      6.988 s ±  0.061 s    [User: 5.935 s, System: 0.845 s]
    Range (min … max):    6.865 s …  7.075 s    20 runs

  Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     332.6 ms ±   3.9 ms    [User: 299.6 ms, System: 32.9 ms]
    Range (min … max):   328.1 ms … 339.1 ms    20 runs

  Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.681 s ±  0.049 s    [User: 2.382 s, System: 0.228 s]
    Range (min … max):    2.607 s …  2.776 s    20 runs

after:

  Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      6.946 s ±  0.041 s    [User: 5.875 s, System: 0.835 s]
    Range (min … max):    6.834 s …  7.005 s    20 runs

  Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     330.3 ms ±   2.5 ms    [User: 299.2 ms, System: 30.9 ms]
    Range (min … max):   327.5 ms … 337.7 ms    20 runs

  Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.671 s ±  0.035 s    [User: 2.370 s, System: 0.232 s]
    Range (min … max):    2.597 s …  2.749 s    20 runs
2022-03-08 23:30:18 +01:00
Eelco Dolstra a4604f1928 Add Store::buildPathsWithResults()
This function is like buildPaths(), except that it returns a vector of
BuildResults containing the exact statuses and output paths of each
derivation / substitution. This is convenient for functions like
Installable::build(), because they then don't need to do another
series of calls to get the outputs of CA derivations. It's also a
precondition to impure derivations, where we *can't* query the output
of those derivations since they're not stored in the Nix database.

Note that PathSubstitutionGoal can now also return a BuildStatus.
2022-03-08 19:56:34 +01:00
regnat f6078e474d Also display some suggestions for invalid formal arguments
```console
$ nix eval --expr '({ foo ? 1 }: foo) { fob = 2; }'
error: anonymous function at (string):1:2 called with unexpected argument 'fob'

       at «string»:1:1:

            1| ({ foo ? 1 }: foo) { fob = 2; }
             | ^
       Did you mean foo?
```

Not that because Nix will first check for _missing_ arguments before
checking for extra arguments, `({ foo }: foo) { fob = 1; }` will
complain about the missing `foo` argument (rather than extra `fob`) and
so won’t display a suggestion.
2022-03-08 16:40:22 +01:00
regnat 33b7514035 Try and make the darwin build happy 2022-03-08 16:07:17 +01:00
regnat 0c6e46e349 Add some suggestions to the evaluator
Make the evaluator show some suggestions when trying to access an
invalid field from an attrset.

```console
$ nix eval --expr '{ foo = 1; }.foa'
error: attribute 'foa' missing

       at «string»:1:1:

            1| { foo = 1; }.foa
             | ^
       Did you mean foo?
```
2022-03-08 06:21:45 +01:00
Eelco Dolstra 92b8d4d886 Bump version 2022-03-07 20:12:35 +01:00
Eelco Dolstra ad7c99ef20 Move rl-next.md to rl-2.7.md 2022-03-07 20:10:18 +01:00
Eelco Dolstra 504e3b2a8f Style 2022-03-07 20:07:43 +01:00
Eelco Dolstra fa614fac7f Merge branch 'GuillaumeDesforges/issue6192' of https://github.com/GuillaumeDesforges/nix 2022-03-07 19:58:06 +01:00
Eelco Dolstra 0123b9aec8 Tweak release notes 2022-03-07 19:56:10 +01:00
Eelco Dolstra 30ddd37873 Merge branch 'cli-suggestions' of https://github.com/thufschmitt/nix 2022-03-07 19:47:45 +01:00
Guillaume Desforges 436c6e900f Explicit error in flake init/new when not in store 2022-03-07 17:57:52 +01:00
regnat dcf4f77fac Merge or-suggestions.hh into suggestions.hh
No real need for keeping a separate header for such a simple class.

This requires changing a bit `OrSuggestions<T>::operator*` to not throw
an `Error` to prevent a cyclic dependency. But since this error is only
thrown on programmer error, we can replace the whole method by a direct
call to `std::get` which will raise its own assertion if needs be.
2022-03-07 17:49:02 +01:00
Eelco Dolstra 60a68a4fee
Merge pull request #6173 from hercules-ci/fix-mkString-for-empty-string-view
Fix `mkString` for empty `string_view`
2022-03-07 17:10:41 +01:00
Robert Hensing da260f579d dupStringWithLen -> makeImmutableString
Refactor the `size == 0` logic into a new helper function that
replaces dupStringWithLen.
The name had to change, because unlike a `dup`-function, it does
not always allocate a new string.
2022-03-07 16:09:12 +01:00
Robert Hensing bbf55383e7 Value::mkPath: Avoid potential crash from null string_view 2022-03-07 16:09:12 +01:00
Robert Hensing 1b978596b5 Value::mkString: Avoid crash from null string_view 2022-03-07 16:09:12 +01:00
Naïm Favier da7d8daa77
Add shell completion for --override-flake
Requires moving the MixEvalArgs class from libexpr to libcmd because
that's where completeFlakeRef is.
2022-03-07 12:01:54 +01:00
Naïm Favier 55c6906701
Perform tilde expansion when completing flake fragments
Allows completing `nix build ~/flake#<Tab>`.
We can implement expansion for `~user` later if needed.
Not using wordexp(3) since that expands way too much.
2022-03-07 12:01:54 +01:00
Naïm Favier 5461ff532d
Make completeDir follow symlinks
Allows completing `nix why-depends /run/cur<Tab>` to /run/current-system
2022-03-07 12:01:54 +01:00
Naïm Favier a6d7cd4183
Ensure the completion marker is not processed beyond completion
I was surprised to see an error mentioning ___COMPLETE___ when trying to
complete a flag argument that had no completer implemented
2022-03-07 12:01:54 +01:00
Naïm Favier 5f06a91bf7
Fix completion of nested attributes in completeInstallable
Without this, completing `nix eval -f file.nix foo.<Tab>` suggests `bar`
instead of `foo.bar`, which messes up the command
2022-03-07 12:01:54 +01:00
Naïm Favier 7ddcb39206
Add shell completion for --override-input 2022-03-07 12:01:54 +01:00
Naïm Favier 7f5cf87d56
Accept and discard fragments in getFlakeRefForCompletion
Otherwise trying to complete `nix build foo#bar --update-input <Tab>`
fails with "unexpected fragment"
2022-03-07 12:01:54 +01:00
Eelco Dolstra c28e2b1b29 Tweak release notes 2022-03-07 11:30:40 +01:00
regnat 313bbc07a8 Implement operator<< for Suggestions
That way there’s no need to explicitely convert it to a string when
printing it
2022-03-07 10:09:10 +01:00
regnat fd45d85b41 Move OrSuggestions to its own header
Prevents a recursive inclusion
2022-03-07 10:09:10 +01:00
regnat b44cebd1fd Add a release-notes entry for the cli suggestions 2022-03-07 10:09:10 +01:00
regnat 91635206c0 Add some end-to-end tests for the suggestions 2022-03-07 10:09:10 +01:00
regnat 98e361ad4c Also display suggestions for the commands using the eval cache
Make `nix build .#nix-armv8l-linux` work for example
2022-03-07 10:09:10 +01:00
regnat 2405bbbb5e Add some tests for the suggestions 2022-03-07 10:09:10 +01:00
regnat c0792b1546 Implement a suggestions mechanism
Each `Error` class now includes a set of suggestions, and these are printed by
the top-level handler.
2022-03-07 10:09:09 +01:00
Théophane Hufschmitt a2ace54fe4
Merge pull request #6029 from Ma27/nix-log-ssh-ng
ssh-ng: also store build logs to make them accessible by `nix log`
2022-03-07 09:51:40 +01:00
Théophane Hufschmitt 860016bcbf
Explain why the log tests are disabled for CA derivations 2022-03-07 09:15:34 +01:00
Théophane Hufschmitt 2051c0f409
Merge pull request #6208 from sielicki/fix-url-github-oauth
Point to new github oauth docs url
2022-03-07 08:54:50 +01:00
Nicholas Sielicki 314852a10e
Point to new github oauth docs url
Previous URL was 404'ing.
2022-03-06 17:01:14 -06:00
Maximilian Bosch 697d1dac01
tests: grep for string in nix log for remote-builds 2022-03-04 16:58:27 +01:00
Eelco Dolstra b09baf690b
Merge pull request #6188 from obsidiansystems/store-gc-subclass
Factor out a `GcStore` interface
2022-03-03 20:53:10 +01:00
John Ericson 6636202356 Factor out a GcStore interface
Starts progress on #5729.

The idea is that we should not have these default methods throwing
"unimplemented". This is a small step in that direction.

I kept `addTempRoot` because it is a no-op, rather than failure. Also,
as a practical matter, it is called all over the place, while doing
other tasks, so the downcasting would be annoying.

Maybe in the future I could move the "real" `addTempRoot` to `GcStore`,
and the existing usecases use a `tryAddTempRoot` wrapper to downcast or
do nothing, but I wasn't sure whether that was a good idea so with a
bias to less churn I didn't do it yet.
2022-03-03 19:01:25 +00:00
regnat 417aaf4ff7 Correctly hijack the file:// uri scheme with _NIX_FORCE_HTTP
Setting the `_NIX_FORCE_HTTP` environment variable is supposed to force `file://` store urls to use the `HttpBinaryCacheStore` implementation rather than the `LocalBinaryCacheStore` one (very useful for testing).
However because of a name mismatch, the `LocalBinaryCacheStore` was still registering the `file` scheme when this variable was set, meaning that the actual store implementation picked up on `file://` uris was dependent on the registration order of the stores (itself dependent on the link order of the object files).

Fix this by making the `LocalBinaryCacheStore` gracefully not register the `file` uri scheme when the variable is set.
2022-03-03 17:47:50 +01:00
pennae 2799fe4cdb enable LTO in optimized builds
gives 2-5% performance improvement across a board of tests.
LTO is broken when using clang; some libs link fine while others crash
the linker with a segfault in the llvm linker plugin. 🙁
2022-03-03 17:47:49 +01:00
Eelco Dolstra 391f4fcabe
Merge pull request #6201 from edolstra/print-value
printValue(): Don't show repeated values
2022-03-03 14:35:16 +01:00
Eelco Dolstra e9c04c3351 Be more aggressive in hiding repeated values
We now memoize on Bindings / list element vectors rather than Values,
so that e.g. two Values that point to the same Bindings will be
printed only once.
2022-03-03 13:33:34 +01:00
Eelco Dolstra ecff9d969a printValue(): Don't show repeated values
Fixes #6157.
2022-03-03 13:18:23 +01:00
Eelco Dolstra 6097790863 Fix segfault in headerCallback()
https://hydra.nixos.org/build/168594664
2022-03-03 11:11:16 +01:00
Eelco Dolstra 885d709393
Merge pull request #6198 from edolstra/coerce-store-path
Add EvalState::coerceToStorePath() helper
2022-03-03 10:02:51 +01:00