Commit graph

15920 commits

Author SHA1 Message Date
pennae b78e77b34c use custom location type in the parser
~1% parser speedup from not using TLS indirections, less on system eval.
this could have also gone in flex yyextra data, but that's significantly
slower for some reason (albeit still faster than thread locals).

before:

  Time (mean ± σ):      4.231 s ±  0.004 s    [User: 3.725 s, System: 0.504 s]
  Range (min … max):    4.226 s …  4.240 s    10 runs

after:

  Time (mean ± σ):      4.224 s ±  0.005 s    [User: 3.711 s, System: 0.512 s]
  Range (min … max):    4.218 s …  4.234 s    10 runs
2023-12-19 19:32:16 +01:00
pennae 2e0321912a use aligned flex tables
~2% speedup on parsing without eval, less (but still significant) on
system eval. having flex generate faster parsers leads to very strange
misparses. maybe re2c is worth investigating.

before:

  Time (mean ± σ):      4.260 s ±  0.003 s    [User: 3.754 s, System: 0.505 s]
  Range (min … max):    4.257 s …  4.266 s    10 runs

after:

  Time (mean ± σ):      4.231 s ±  0.004 s    [User: 3.725 s, System: 0.504 s]
  Range (min … max):    4.226 s …  4.240 s    10 runs
2023-12-19 19:32:16 +01:00
pennae cc4038d541 use std::tie() for macro-generated operators
as written the comparisons generate copies, even though it looks as
though they shouldn't.

before:

  Time (mean ± σ):      4.396 s ±  0.002 s    [User: 3.894 s, System: 0.501 s]
  Range (min … max):    4.393 s …  4.399 s    10 runs

after:

  Time (mean ± σ):      4.260 s ±  0.003 s    [User: 3.754 s, System: 0.505 s]
  Range (min … max):    4.257 s …  4.266 s    10 runs
2023-12-19 19:32:16 +01:00
pennae 74c134914c compare string values with strcmp
string_view()ification calls strlen() first, which we don't need here.
2023-12-19 19:32:16 +01:00
pennae 78353deb02 encode black holes as tApp values
checking for isBlackhole in the forceValue hot path is rather more
expensive than necessary, and with a little bit of trickery we can move
such handling into the isApp case. small performance benefit, but under
some circumstances we've seen 2% improvement as well.

〉 nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'

before:

  Time (mean ± σ):      4.429 s ±  0.002 s    [User: 3.929 s, System: 0.500 s]
  Range (min … max):    4.427 s …  4.433 s    10 runs

after:

  Time (mean ± σ):      4.396 s ±  0.002 s    [User: 3.894 s, System: 0.501 s]
  Range (min … max):    4.393 s …  4.399 s    10 runs
2023-12-19 19:32:16 +01:00
pennae 0218e4e6c3 memset less in addToStoreFromDump
resizing a std::string clears the newly added bytes, which is not
necessary here and comes with a ~1.4% slowdown on our test nixos config.

〉 nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'

before:

  Time (mean ± σ):      4.486 s ±  0.003 s    [User: 3.978 s, System: 0.507 s]
  Range (min … max):    4.482 s …  4.492 s    10 runs

after:

  Time (mean ± σ):      4.429 s ±  0.002 s    [User: 3.929 s, System: 0.500 s]
  Range (min … max):    4.427 s …  4.433 s    10 runs
2023-12-19 19:32:16 +01:00
Eelco Dolstra 7cfd6c0efe
Merge pull request #9325 from NixOS/accessor-add-to-store
Content addressing and adding to store cleanup
2023-12-19 15:10:31 +01:00
Théophane Hufschmitt 83a32969ad
Merge pull request #9633 from cole-h/fix-sysconfdir
package: don't set sysconfdir in devShells
2023-12-19 10:14:48 +01:00
Théophane Hufschmitt 94a03a810e
Merge pull request #9637 from 9999years/fix-logging-test-on-macos
Fix `logging.sh` test on macOS
2023-12-19 10:13:08 +01:00
Théophane Hufschmitt b077d545ad
Merge pull request #9638 from NixOS/dependabot/github_actions/zeebe-io/backport-action-2.3.0
build(deps): bump zeebe-io/backport-action from 2.2.0 to 2.3.0
2023-12-19 10:11:37 +01:00
Théophane Hufschmitt e85fd92816
Merge pull request #9639 from awakesecurity/mz/darwin-UID-override
installer: allow overriding of NIX_FIRST_BUILD_ID on darwin
2023-12-19 10:11:24 +01:00
Théophane Hufschmitt c6d7013583
Merge pull request #9641 from amarshall/darwin-sandbox-stderr
Allow access to /dev/stderr in Darwin sandbox
2023-12-19 10:09:38 +01:00
Andrew Marshall 7526b7ded6 Allow access to /dev/stderr in Darwin sandbox
We allow /dev/stdout, so why not this? Since it is process-local,
anyway, should not be possible to escape sandbox using it.
2023-12-18 19:33:20 -05:00
Mel Zuser fa4bbe53e8 installer: allow overriding of NIX_FIRST_BUILD_ID on darwin
because there are often already users in the 300 range and it's painful
to work around.

revives #6466
2023-12-18 15:02:26 -08:00
dependabot[bot] 23fb19cb18
build(deps): bump zeebe-io/backport-action from 2.2.0 to 2.3.0
Bumps [zeebe-io/backport-action](https://github.com/zeebe-io/backport-action) from 2.2.0 to 2.3.0.
- [Release notes](https://github.com/zeebe-io/backport-action/releases)
- [Commits](https://github.com/zeebe-io/backport-action/compare/v2.2.0...v2.3.0)

---
updated-dependencies:
- dependency-name: zeebe-io/backport-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-18 22:36:31 +00:00
Rebecca Turner 0cee56db1a
Fix logging.sh test on macOS
On macOS in the `nix develop` shell, `make
tests/functional/logging.sh.test` errors:

    ++(logging.sh:18) mktemp
    +(logging.sh:18) builder=/var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.StuabKUhMh
    +(logging.sh:19) echo -e '#!/bin/sh\nmkdir $out'
    +++(logging.sh:22) mktemp -d
    ++(logging.sh:22) nix-build -E 'with import ./config.nix; mkDerivation { name = "fnord"; builder = /var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.StuabKUhMh; }' --out-link /var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.oaKcy0NXqC/result
    error:
           … while calling the 'derivationStrict' builtin
             at <nix/derivation-internal.nix>:9:12:
                8|
                9|   strict = derivationStrict drvAttrs;
                 |            ^
               10|

           … while evaluating derivation 'fnord'
             whose name attribute is located at «string»:1:42

           … while evaluating attribute 'args' of derivation 'fnord'
             at /Users/wiggles/nix/tests/functional/config.nix:23:7:
               22|       builder = shell;
               23|       args = ["-e" args.builder or (builtins.toFile "builder-${args.name}.sh" ''
                 |       ^
               24|         if [ -e "$NIX_ATTRS_SH_FILE" ]; then source $NIX_ATTRS_SH_FILE; fi;

           error: path '/var' is a symlink
    +(logging.sh:22) outp=
    ++(logging.sh:22) onError
    ++(/Users/wiggles/nix/tests/functional/common/vars-and-functions.sh:237) set +x
    logging.sh: test failed at:
      main in logging.sh:22

This is because `mktemp` returns a path like
`/var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.qDY24l6bIM`,
where `/var` is a symlink to `/private/var`.

Then, we attempt to use that path as a `builder`, which errors because
symlinks are impure or whatever.

Anyways, we can fix this by using `realpath "$(mktemp)"` instead of
`mktemp` directly.

NB: This error doesn't seem to happen when I run the tests through `nix
flake check`. I'm not sure if Nix does something to `TMP` in that case.
2023-12-18 14:04:25 -08:00
Rebecca Turner 6f4930382b
Document more Makefile variables 2023-12-18 13:34:08 -08:00
Cole Helbling ba0087316a package: don't set sysconfdir in devShells 2023-12-18 12:59:58 -08:00
Cole Helbling 123ef6a996 Fix warnings when running checks
`nix flake check` had these warnings:

    trace: warning: Module argument `nodes.client.config` is deprecated. Use `nodes.client` instead.
    trace: warning: Module argument `nodes.client.config` is deprecated. Use `nodes.client` instead.
    trace: warning: The option `services.openssh.permitRootLogin' defined in `/nix/store/3m3hfpmbjdf4w39qfjami7ljhvhczay1-source/tests/nixos/nix-copy.nix' has been renamed to `services.openssh.settings.PermitRootLogin'.
    trace: warning: Module argument `nodes.http_dns.config` is deprecated. Use `nodes.http_dns` instead.
    trace: warning: Module argument `nodes.github.config` is deprecated. Use `nodes.github` instead.
    trace: warning: Module argument `nodes.sourcehut.config` is deprecated. Use `nodes.sourcehut` instead.
2023-12-18 10:20:52 -08:00
Robert Hensing 5d5b25f2e3
Merge pull request #9611 from obsidiansystems/fix-cross-configure
Split `--disable-tests`, fix cross builds
2023-12-18 17:25:19 +01:00
John Ericson 7feabf7d44 Split --disable-tests, fix cross builds
It might seem obnoxious to have yet more configure flags, but I found
controlling both the unit and functional tests with one flag was quite
confusing because they are so different:

- unit tests depending on building, functional tests don't (e.g. when
  we test already-built Nix)

- unit tests can be installed, functional tests cannot

- unit tests neeed extra libraries (GTest, RapidCheck), functional
  tests need extra executables (jq).

- unit tests are run by `make check`, functional tests are run by `make
  installcheck`

Really on a technical level, they seem wholly independent. Only on a
human level ("they are both are tests") do they have anything in common.

I had messed up the logic in cross builds because of this. Now I
split the flag in two (and cleaned up a few other inconsistencies), and
the logic fixed itself.

Co-Authored-By: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:47:50 -05:00
tomberek b1c559eabc
docs: add link to project board to PRs (#9630)
* docs: add link to project board to PRs

* Update .github/PULL_REQUEST_TEMPLATE.md

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>

* fix wording

* add note on the process

---------

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-18 15:45:57 +00:00
John Ericson ed26b186fb Remove now-redundant text-hashing store methods
`addTextToStore` and `computeStorePathFromDump` are now redundant.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:44:10 -05:00
John Ericson dfc876531f Organize content addressing, use SourceAccessor with Store::addToStore
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:41:54 -05:00
Valentin Gagarin d19a667528
CODEOWNERS: unsubscribe fricklerhandwerk (#9614) 2023-12-18 13:54:40 +01:00
Théophane Hufschmitt ebfbc5a6a5
Merge pull request #9628 from fricklerhandwerk/add-links
add cross-reference
2023-12-18 13:07:39 +01:00
Robert Hensing cea8354436
Merge pull request #9619 from 9999years/remove-blank-lines-in-errors
Remove some blank lines from stack traces
2023-12-18 12:50:50 +01:00
Valentin Gagarin 4f95800854 add cross-reference 2023-12-18 11:41:52 +01:00
David Arnold c05d4fadd5
fix: valid branch name 2023-12-16 23:14:33 +01:00
Rebecca Turner 7f5ed330e4
Document Makefile variables in hacking.md (#9620) 2023-12-16 12:05:31 +00:00
Rebecca Turner 5cb98095ba
Remove some blank lines from stack traces
This keeps hint messages, source location information, and source code
snippets grouped together, while making stack traces shorter (so that
more stack frames can be viewed on the same terminal).

Before:

    error:
           … while evaluating the attribute 'body'

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:3:

                3|
                4|   body = x "x";
                 |   ^
                5| }

           … from call site

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:10:

                3|
                4|   body = x "x";
                 |          ^
                5| }

           … while calling 'x'

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:7:

                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |       ^
                3|

           error: assertion '(arg == "y")' failed

           at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:12:

                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |            ^
                3|

After:

    error:
           … while evaluating the attribute 'body'
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:3:
                3|
                4|   body = x "x";
                 |   ^
                5| }

           … from call site
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:10:
                3|
                4|   body = x "x";
                 |          ^
                5| }

           … while calling 'x'
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:7:
                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |       ^
                3|

           error: assertion '(arg == "y")' failed
           at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:12:
                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |            ^
                3|
2023-12-15 23:57:26 -08:00
John Ericson 419a64f9e0
Merge pull request #9613 from jlesquembre/jl/clang-stdenv
Fix clang devshell
2023-12-15 09:08:41 -05:00
José Luis Lafuente 66d37b7338
Move clang dev deps to the nix devshell override 2023-12-15 12:41:38 +01:00
John Ericson 071dbbee33
Merge pull request #4093 from matthewbauer/eval-system
Add eval-system option
2023-12-14 20:41:29 -05:00
John Ericson 228e995cde Add release not for eval-system 2023-12-14 19:55:50 -05:00
John Ericson 70f50cbb2a Functional Test for builtins.storeDir and builtins.currentSystem
These were under-tested. This tests the status quo and especially
previous commit of this PR better.
2023-12-14 19:47:10 -05:00
Matthew Bauer bcbdb09ccf Add eval-system option
`eval-system` option overrides just the value of `builtins.currentSystem`.
This is more useful than overriding `system` since you can build these
derivations on remote builders which can work on the given system.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-14 19:04:00 -05:00
José Luis Lafuente 8d39c0c196
Fix clang devshell
Issue introduced in https://github.com/NixOS/nix/pull/9535
2023-12-14 23:14:59 +01:00
Eelco Dolstra fc0accf3dc
Merge pull request #9567 from obsidiansystems/fix-sys-xattr-h
Fix `sys/xattr.h` check
2023-12-14 18:24:24 +01:00
John Ericson e13fc0bbdb Fix sys/xattr.h check
I wrote the `configure.ac` wrong, and so we just got no builds
supporting ACLs.

Also, it needs to be more precise because Darwin puts other stuff in
that same header, evidently.
2023-12-14 10:03:48 -05:00
Eelco Dolstra 790cf13c26
Merge pull request #9608 from NixOS/default-lowdown
Remove custom lowdown
2023-12-14 13:18:32 +01:00
John Ericson e8ebcfd665
Merge pull request #9607 from 9999years/macos-boost
Use `--with-boost` on macOS to fix local builds
2023-12-13 23:15:44 -05:00
Rebecca Turner 6ed803737c
Use --with-boost on macOS
`configureFlags` only included `--with-boost` on Linux, which makes
local builds as outlined in `doc/manual/src/contributing/hacking.md`
fail when performed on macOS.
2023-12-13 14:02:52 -08:00
Ramses 1e3d811840
worker protocol: serialise cgroup stats in BuildResult (#9598)
By doing so, they get reported when building through the daemon via either `unix://` or `ssh-ng://`.
2023-12-13 16:37:17 -05:00
John Ericson 7b29b44d8e Remove custom lowdown
This was last upgraded in 788008385e, but
the version in Nixpkgs is a now a lot newer. I think the custom was
added to get ahead of Nixpkgs before, and so now that we are in fact
behind, it is no longer needed.
2023-12-13 16:25:18 -05:00
John Ericson e6515bd47b
Merge pull request #9535 from tfc/package-nix
Simplify flake.nix by creating generic callPackage-able package.nix
2023-12-13 16:22:03 -05:00
John Ericson 2d24875fe4 package.nix: Avoid ${..} for conditional strings
Using `+` is Nixpkgs standard ideom for this, and helps avoid needless
rebuilds somewhat.
2023-12-13 20:49:31 +00:00
John Ericson ed93e51f4b
Merge pull request #9604 from obsidiansystems/test-substitute-on-destination
Test `nix copy --substitute-on-destination`
2023-12-13 15:47:56 -05:00
John Ericson 28f2f3136d Delete stray install_name_tool call 2023-12-13 20:47:36 +00:00
John Ericson bf5804d46a flake.nix: Delete uneeded attrs0 binding 2023-12-13 20:41:41 +00:00