Commit graph

15933 commits

Author SHA1 Message Date
Eelco Dolstra 14508ade28 Typo 2023-12-21 16:25:55 +01:00
Eelco Dolstra d77a39a314 Fix indent 2023-12-21 16:22:34 +01:00
Eelco Dolstra 9d9d9ff0de Merge remote-tracking branch 'origin/master' into profile-names-instead-of-index 2023-12-21 16:21:26 +01:00
Robert Hensing 8e64cd59b0
Merge pull request #9645 from tweag/undeprecate-isNull
Undeprecate isNull
2023-12-20 18:15:49 +01:00
Silvan Mosberger ea454d8687 Undeprecate isNull
There's no good reason to deprecate it:
- For consistency reasons it should continue to exist, such that all
primitive types have a corresponding `builtins.is*` primop.
- There's no implementation cost to continuing to have this function
- It costs users time to try to migrate away from it, e.g.
  https://github.com/NixOS/nixpkgs/pull/219747 and https://github.com/NixOS/nixpkgs/pull/275548
- Using it can give easier-to-read code like `all isNull list`

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-20 17:55:19 +01:00
Eelco Dolstra 75e10e42f3
Merge pull request #9631 from cole-h/fixup-check-warnings
Fix warnings when running checks
2023-12-20 17:47:30 +01:00
Eelco Dolstra 16a77767cd
Merge pull request #9635 from 9999years/document-makefile-variables
Document more `Makefile` variables
2023-12-20 17:47:07 +01:00
Théophane Hufschmitt e3cdad4cf8
Merge pull request #9646 from obsidiansystems/fix-cross-build
Move down fallback targets in `Makefile`
2023-12-20 13:44:40 +01:00
John Ericson 26d60b837c Move down fallback targets in Makefile
This ensures `lib.mk` still defines `default` as the first target. This
fixes some builds.
2023-12-20 03:23:49 -05:00
pennae 2b0e95e7aa use singleton expr to generate black hole errors
this also reduces forceValue code size and removes the need for
hideInDiagnostics. coopting thunk forcing like this has the additional
benefit of clarifying how these errors can happen in the first place.
2023-12-19 19:32:16 +01:00
pennae f9db4de0f3 force-inline forceValue
forceValue is extremely hot. interestingly adding likeliness annotations
to the branches does not seem to make a difference.

before:

  Time (mean ± σ):      4.224 s ±  0.005 s    [User: 3.711 s, System: 0.512 s]
  Range (min … max):    4.218 s …  4.234 s    10 runs

after:

  Time (mean ± σ):      4.140 s ±  0.009 s    [User: 3.647 s, System: 0.492 s]
  Range (min … max):    4.130 s …  4.152 s    10 runs
2023-12-19 19:32:16 +01:00
pennae 69ed4aee61 remove lazy-pos forceValue
almost all uses of this are interactive, except for deepSeq. deepSeq is
going to be expensive and rare enough to not care much about, and
Value::determinePos should usually be cheap enough to not be too much of
a burden in any case.
2023-12-19 19:32:16 +01:00
pennae f9aee2f2c4 don't malloc/memset posix accessor buffer
it's relatively small and fits on the stack nicely, and we don't need it
initialized either.
2023-12-19 19:32:16 +01:00
pennae b78e77b34c use custom location type in the parser
~1% parser speedup from not using TLS indirections, less on system eval.
this could have also gone in flex yyextra data, but that's significantly
slower for some reason (albeit still faster than thread locals).

before:

  Time (mean ± σ):      4.231 s ±  0.004 s    [User: 3.725 s, System: 0.504 s]
  Range (min … max):    4.226 s …  4.240 s    10 runs

after:

  Time (mean ± σ):      4.224 s ±  0.005 s    [User: 3.711 s, System: 0.512 s]
  Range (min … max):    4.218 s …  4.234 s    10 runs
2023-12-19 19:32:16 +01:00
pennae 2e0321912a use aligned flex tables
~2% speedup on parsing without eval, less (but still significant) on
system eval. having flex generate faster parsers leads to very strange
misparses. maybe re2c is worth investigating.

before:

  Time (mean ± σ):      4.260 s ±  0.003 s    [User: 3.754 s, System: 0.505 s]
  Range (min … max):    4.257 s …  4.266 s    10 runs

after:

  Time (mean ± σ):      4.231 s ±  0.004 s    [User: 3.725 s, System: 0.504 s]
  Range (min … max):    4.226 s …  4.240 s    10 runs
2023-12-19 19:32:16 +01:00
pennae cc4038d541 use std::tie() for macro-generated operators
as written the comparisons generate copies, even though it looks as
though they shouldn't.

before:

  Time (mean ± σ):      4.396 s ±  0.002 s    [User: 3.894 s, System: 0.501 s]
  Range (min … max):    4.393 s …  4.399 s    10 runs

after:

  Time (mean ± σ):      4.260 s ±  0.003 s    [User: 3.754 s, System: 0.505 s]
  Range (min … max):    4.257 s …  4.266 s    10 runs
2023-12-19 19:32:16 +01:00
pennae 74c134914c compare string values with strcmp
string_view()ification calls strlen() first, which we don't need here.
2023-12-19 19:32:16 +01:00
pennae 78353deb02 encode black holes as tApp values
checking for isBlackhole in the forceValue hot path is rather more
expensive than necessary, and with a little bit of trickery we can move
such handling into the isApp case. small performance benefit, but under
some circumstances we've seen 2% improvement as well.

〉 nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'

before:

  Time (mean ± σ):      4.429 s ±  0.002 s    [User: 3.929 s, System: 0.500 s]
  Range (min … max):    4.427 s …  4.433 s    10 runs

after:

  Time (mean ± σ):      4.396 s ±  0.002 s    [User: 3.894 s, System: 0.501 s]
  Range (min … max):    4.393 s …  4.399 s    10 runs
2023-12-19 19:32:16 +01:00
pennae 0218e4e6c3 memset less in addToStoreFromDump
resizing a std::string clears the newly added bytes, which is not
necessary here and comes with a ~1.4% slowdown on our test nixos config.

〉 nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'

before:

  Time (mean ± σ):      4.486 s ±  0.003 s    [User: 3.978 s, System: 0.507 s]
  Range (min … max):    4.482 s …  4.492 s    10 runs

after:

  Time (mean ± σ):      4.429 s ±  0.002 s    [User: 3.929 s, System: 0.500 s]
  Range (min … max):    4.427 s …  4.433 s    10 runs
2023-12-19 19:32:16 +01:00
Eelco Dolstra 7cfd6c0efe
Merge pull request #9325 from NixOS/accessor-add-to-store
Content addressing and adding to store cleanup
2023-12-19 15:10:31 +01:00
Théophane Hufschmitt 83a32969ad
Merge pull request #9633 from cole-h/fix-sysconfdir
package: don't set sysconfdir in devShells
2023-12-19 10:14:48 +01:00
Théophane Hufschmitt 94a03a810e
Merge pull request #9637 from 9999years/fix-logging-test-on-macos
Fix `logging.sh` test on macOS
2023-12-19 10:13:08 +01:00
Théophane Hufschmitt b077d545ad
Merge pull request #9638 from NixOS/dependabot/github_actions/zeebe-io/backport-action-2.3.0
build(deps): bump zeebe-io/backport-action from 2.2.0 to 2.3.0
2023-12-19 10:11:37 +01:00
Théophane Hufschmitt e85fd92816
Merge pull request #9639 from awakesecurity/mz/darwin-UID-override
installer: allow overriding of NIX_FIRST_BUILD_ID on darwin
2023-12-19 10:11:24 +01:00
Théophane Hufschmitt c6d7013583
Merge pull request #9641 from amarshall/darwin-sandbox-stderr
Allow access to /dev/stderr in Darwin sandbox
2023-12-19 10:09:38 +01:00
Andrew Marshall 7526b7ded6 Allow access to /dev/stderr in Darwin sandbox
We allow /dev/stdout, so why not this? Since it is process-local,
anyway, should not be possible to escape sandbox using it.
2023-12-18 19:33:20 -05:00
Mel Zuser fa4bbe53e8 installer: allow overriding of NIX_FIRST_BUILD_ID on darwin
because there are often already users in the 300 range and it's painful
to work around.

revives #6466
2023-12-18 15:02:26 -08:00
dependabot[bot] 23fb19cb18
build(deps): bump zeebe-io/backport-action from 2.2.0 to 2.3.0
Bumps [zeebe-io/backport-action](https://github.com/zeebe-io/backport-action) from 2.2.0 to 2.3.0.
- [Release notes](https://github.com/zeebe-io/backport-action/releases)
- [Commits](https://github.com/zeebe-io/backport-action/compare/v2.2.0...v2.3.0)

---
updated-dependencies:
- dependency-name: zeebe-io/backport-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-18 22:36:31 +00:00
Rebecca Turner 0cee56db1a
Fix logging.sh test on macOS
On macOS in the `nix develop` shell, `make
tests/functional/logging.sh.test` errors:

    ++(logging.sh:18) mktemp
    +(logging.sh:18) builder=/var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.StuabKUhMh
    +(logging.sh:19) echo -e '#!/bin/sh\nmkdir $out'
    +++(logging.sh:22) mktemp -d
    ++(logging.sh:22) nix-build -E 'with import ./config.nix; mkDerivation { name = "fnord"; builder = /var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.StuabKUhMh; }' --out-link /var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.oaKcy0NXqC/result
    error:
           … while calling the 'derivationStrict' builtin
             at <nix/derivation-internal.nix>:9:12:
                8|
                9|   strict = derivationStrict drvAttrs;
                 |            ^
               10|

           … while evaluating derivation 'fnord'
             whose name attribute is located at «string»:1:42

           … while evaluating attribute 'args' of derivation 'fnord'
             at /Users/wiggles/nix/tests/functional/config.nix:23:7:
               22|       builder = shell;
               23|       args = ["-e" args.builder or (builtins.toFile "builder-${args.name}.sh" ''
                 |       ^
               24|         if [ -e "$NIX_ATTRS_SH_FILE" ]; then source $NIX_ATTRS_SH_FILE; fi;

           error: path '/var' is a symlink
    +(logging.sh:22) outp=
    ++(logging.sh:22) onError
    ++(/Users/wiggles/nix/tests/functional/common/vars-and-functions.sh:237) set +x
    logging.sh: test failed at:
      main in logging.sh:22

This is because `mktemp` returns a path like
`/var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.qDY24l6bIM`,
where `/var` is a symlink to `/private/var`.

Then, we attempt to use that path as a `builder`, which errors because
symlinks are impure or whatever.

Anyways, we can fix this by using `realpath "$(mktemp)"` instead of
`mktemp` directly.

NB: This error doesn't seem to happen when I run the tests through `nix
flake check`. I'm not sure if Nix does something to `TMP` in that case.
2023-12-18 14:04:25 -08:00
Rebecca Turner 6f4930382b
Document more Makefile variables 2023-12-18 13:34:08 -08:00
Cole Helbling ba0087316a package: don't set sysconfdir in devShells 2023-12-18 12:59:58 -08:00
Cole Helbling 123ef6a996 Fix warnings when running checks
`nix flake check` had these warnings:

    trace: warning: Module argument `nodes.client.config` is deprecated. Use `nodes.client` instead.
    trace: warning: Module argument `nodes.client.config` is deprecated. Use `nodes.client` instead.
    trace: warning: The option `services.openssh.permitRootLogin' defined in `/nix/store/3m3hfpmbjdf4w39qfjami7ljhvhczay1-source/tests/nixos/nix-copy.nix' has been renamed to `services.openssh.settings.PermitRootLogin'.
    trace: warning: Module argument `nodes.http_dns.config` is deprecated. Use `nodes.http_dns` instead.
    trace: warning: Module argument `nodes.github.config` is deprecated. Use `nodes.github` instead.
    trace: warning: Module argument `nodes.sourcehut.config` is deprecated. Use `nodes.sourcehut` instead.
2023-12-18 10:20:52 -08:00
Robert Hensing 5d5b25f2e3
Merge pull request #9611 from obsidiansystems/fix-cross-configure
Split `--disable-tests`, fix cross builds
2023-12-18 17:25:19 +01:00
John Ericson 7feabf7d44 Split --disable-tests, fix cross builds
It might seem obnoxious to have yet more configure flags, but I found
controlling both the unit and functional tests with one flag was quite
confusing because they are so different:

- unit tests depending on building, functional tests don't (e.g. when
  we test already-built Nix)

- unit tests can be installed, functional tests cannot

- unit tests neeed extra libraries (GTest, RapidCheck), functional
  tests need extra executables (jq).

- unit tests are run by `make check`, functional tests are run by `make
  installcheck`

Really on a technical level, they seem wholly independent. Only on a
human level ("they are both are tests") do they have anything in common.

I had messed up the logic in cross builds because of this. Now I
split the flag in two (and cleaned up a few other inconsistencies), and
the logic fixed itself.

Co-Authored-By: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:47:50 -05:00
tomberek b1c559eabc
docs: add link to project board to PRs (#9630)
* docs: add link to project board to PRs

* Update .github/PULL_REQUEST_TEMPLATE.md

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>

* fix wording

* add note on the process

---------

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-18 15:45:57 +00:00
John Ericson ed26b186fb Remove now-redundant text-hashing store methods
`addTextToStore` and `computeStorePathFromDump` are now redundant.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:44:10 -05:00
John Ericson dfc876531f Organize content addressing, use SourceAccessor with Store::addToStore
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:41:54 -05:00
Valentin Gagarin d19a667528
CODEOWNERS: unsubscribe fricklerhandwerk (#9614) 2023-12-18 13:54:40 +01:00
Théophane Hufschmitt ebfbc5a6a5
Merge pull request #9628 from fricklerhandwerk/add-links
add cross-reference
2023-12-18 13:07:39 +01:00
Robert Hensing cea8354436
Merge pull request #9619 from 9999years/remove-blank-lines-in-errors
Remove some blank lines from stack traces
2023-12-18 12:50:50 +01:00
Valentin Gagarin 4f95800854 add cross-reference 2023-12-18 11:41:52 +01:00
David Arnold c05d4fadd5
fix: valid branch name 2023-12-16 23:14:33 +01:00
Rebecca Turner 7f5ed330e4
Document Makefile variables in hacking.md (#9620) 2023-12-16 12:05:31 +00:00
Rebecca Turner 5cb98095ba
Remove some blank lines from stack traces
This keeps hint messages, source location information, and source code
snippets grouped together, while making stack traces shorter (so that
more stack frames can be viewed on the same terminal).

Before:

    error:
           … while evaluating the attribute 'body'

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:3:

                3|
                4|   body = x "x";
                 |   ^
                5| }

           … from call site

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:10:

                3|
                4|   body = x "x";
                 |          ^
                5| }

           … while calling 'x'

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:7:

                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |       ^
                3|

           error: assertion '(arg == "y")' failed

           at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:12:

                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |            ^
                3|

After:

    error:
           … while evaluating the attribute 'body'
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:3:
                3|
                4|   body = x "x";
                 |   ^
                5| }

           … from call site
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:10:
                3|
                4|   body = x "x";
                 |          ^
                5| }

           … while calling 'x'
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:7:
                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |       ^
                3|

           error: assertion '(arg == "y")' failed
           at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:12:
                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |            ^
                3|
2023-12-15 23:57:26 -08:00
John Ericson 419a64f9e0
Merge pull request #9613 from jlesquembre/jl/clang-stdenv
Fix clang devshell
2023-12-15 09:08:41 -05:00
José Luis Lafuente 66d37b7338
Move clang dev deps to the nix devshell override 2023-12-15 12:41:38 +01:00
John Ericson 071dbbee33
Merge pull request #4093 from matthewbauer/eval-system
Add eval-system option
2023-12-14 20:41:29 -05:00
John Ericson 228e995cde Add release not for eval-system 2023-12-14 19:55:50 -05:00
John Ericson 70f50cbb2a Functional Test for builtins.storeDir and builtins.currentSystem
These were under-tested. This tests the status quo and especially
previous commit of this PR better.
2023-12-14 19:47:10 -05:00
Matthew Bauer bcbdb09ccf Add eval-system option
`eval-system` option overrides just the value of `builtins.currentSystem`.
This is more useful than overriding `system` since you can build these
derivations on remote builders which can work on the given system.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-14 19:04:00 -05:00