Commit graph

15274 commits

Author SHA1 Message Date
Rebecca Turner cf89bfc499 Format Nix code with nixfmt
Change-Id: I61efeb666ff7481c05fcb247168290e86a250151
2024-03-29 17:23:08 -07:00
Rebecca Turner e7316a8e99 Add treefmt pre-commit hook
This lets us use different formatters for different filetypes.

Change-Id: Ib52383dd5097c8919a65e299aca2b5a55412223c
2024-03-29 17:23:06 -07:00
Rebecca Turner 7b3ef56094 Add pre-commit checks
The big ones here are `trim-trailing-whitespace` and `end-of-file-fixer`
(which makes sure that every file ends with exactly one newline
character).

Change-Id: Idca73b640883188f068f9903e013cf0d82aa1123
2024-03-29 17:22:56 -07:00
eldritch horrors 5a54b0a20c meson: install systemd files
Change-Id: Idacf602fd379c82a051f00df2293cb02c8b286d4
2024-03-29 20:10:33 +00:00
eldritch horrors e28dc26084 meson: install shell files
Change-Id: I7c30690e5763d095cf7444333f7b687509051c5f
2024-03-29 20:10:33 +00:00
eldritch horrors 1da1f501fc meson: fix state-dir default value
the autoconf build system defaults to /nix/var, not /nix/var/nix. the
latter is only used in libstore, so we'll move the extra segment there.

Change-Id: Idfbc988ee302355982abdcd51d6d7b5d5d661c0d
2024-03-29 19:14:23 +00:00
Winter Cute 6646b80396 meson: add missing explicit dependency on nlohmann_json
Without this, the Meson setup won't bail out if nlohmann_json is
missing, leading to subpar DX (and maybe worse, but I'm not entirely
sure).

Change-Id: I5913111060226b540dcf003257c99a08e84da0de
2024-03-29 14:16:58 -04:00
Rebecca Turner 9d97d1cb68 Merge "Set MAKEFLAGS=-j and GTEST_BRIEF in .envrc" into main 2024-03-29 17:09:13 +00:00
Rebecca Turner 877750b7c5 Merge "Move DebugChar into its own file" into main 2024-03-29 16:20:14 +00:00
eldritch horrors 6e5db5e4a2 meson: install missing/generated headers
one headers (args/root.hh) was simply missing, and the generated headers
were not installed. not all of them *should* be installed either, only a
select few (and sadly this needs a custom target for each one, it seems)

Change-Id: I37b25517895d0e5e521abc1202fa65624de57ed1
2024-03-29 02:45:48 +00:00
eldritch horrors 69bfd21e20 meson: install pkg-config files for libraries
Change-Id: I14b9d81d09f188eacfb9c68bcfb84751c18e3779
2024-03-29 02:45:48 +00:00
eldritch horrors 86b954a7af meson: increase functional test timeout
sometimes these fail with timeouts on loaded machines. let's up the
timeouts until we can pull the tests apart to more reasonable sizes

Change-Id: I2dfff2183cc1f3ff5e6107f43748ac046fe00d05
2024-03-29 02:19:36 +00:00
Rebecca Turner f9d5079c69 Set MAKEFLAGS=-j and GTEST_BRIEF in .envrc
- Enable parallel builds by default (and allow using environment
  variables to override `make` variables)
  - Hopefully we can get rid of this once we have Meson
- Set `GTEST_BRIEF=1`
  - This only shows failed tests, instead of listing every test on its
    own line.

```
$ GTEST_BRIEF=1 make check
[==========] 328 tests from 15 test suites ran. (37 ms total)
[  PASSED  ] 328 tests.
```

Change-Id: Id8103a8f24a9681be2be87e1b4df6fd5fdd7e4fd
2024-03-28 18:17:28 -07:00
Rebecca Turner 236bc046ba Merge "Remove HintFmt::operator%" into main 2024-03-29 01:13:45 +00:00
raito 55350bd68d Merge "feat: unprivileged read-only open of SQLite DB" into main 2024-03-29 00:49:17 +00:00
Rebecca Turner 5ec2efb686 Move DebugChar into its own file
Change-Id: Ia40549e5d0b78ece8dd0722c3a5a032b9915f24b
2024-03-28 15:54:12 -07:00
Rebecca Turner 62332c1250 Merge "Move shell_words into its own file" into main 2024-03-28 22:49:00 +00:00
Qyriad 81e50fef70 Merge "meson: implement functional tests" into main 2024-03-28 20:38:05 +00:00
jade 61d394e344 Merge "Build with traps on signed overflow" into main 2024-03-28 20:27:32 +00:00
jade ae065a992d Merge "progress-bar.cc: fix signed overflow" into main 2024-03-28 15:21:11 +00:00
jade 14207e1cf8 Build with traps on signed overflow
This is UB, we should not be doing it, and we can cheaply turn it into
crashes reliably. We would much rather have crashes than the program
doing something silly.

Benchmarks, but i wonder if they are nonsense because they get identical
times across compilers?!

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `result-clang/bin/nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix` | 375.5 ± 24.0 | 353.8 | 408.8 | 1.00 |
| `result-gcc/bin/nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix` | 407.9 ± 26.0 | 385.1 | 449.5 | 1.09 ± 0.10 |
| `result-clangsan/bin/nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix` | 382.2 ± 26.6 | 354.9 | 419.0 | 1.02 ± 0.10 |
| `result-gccsan/bin/nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix` | 408.6 ± 24.6 | 384.5 | 441.9 | 1.09 ± 0.10 |

| Command | Mean [s] | Min [s] | Max [s] | Relative |
|:---|---:|---:|---:|---:|
| `result-clang/bin/nix search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello` | 17.199 ± 0.167 | 16.930 | 17.499 | 1.01 ± 0.01 |
| `result-gcc/bin/nix search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello` | 17.409 ± 0.126 | 17.242 | 17.633 | 1.02 ± 0.01 |
| `result-clangsan/bin/nix search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello` | 17.080 ± 0.137 | 16.879 | 17.350 | 1.00 |
| `result-gccsan/bin/nix search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello` | 17.396 ± 0.160 | 17.131 | 17.660 | 1.02 ± 0.01 |

| Command | Mean [s] | Min [s] | Max [s] | Relative |
|:---|---:|---:|---:|---:|
| `result-clang/bin/nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 6.267 ± 0.069 | 6.197 | 6.415 | 1.02 ± 0.01 |
| `result-gcc/bin/nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 6.232 ± 0.045 | 6.180 | 6.311 | 1.01 ± 0.01 |
| `result-clangsan/bin/nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 6.162 ± 0.020 | 6.133 | 6.196 | 1.00 |
| `result-gccsan/bin/nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 6.229 ± 0.031 | 6.199 | 6.289 | 1.01 ± 0.01 |

| Command | Mean [s] | Min [s] | Max [s] | Relative |
|:---|---:|---:|---:|---:|
| `GC_INITIAL_HEAP_SIZE=10g result-clang/bin/nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 4.683 ± 0.044 | 4.630 | 4.761 | 1.00 |
| `GC_INITIAL_HEAP_SIZE=10g result-gcc/bin/nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 4.750 ± 0.041 | 4.680 | 4.812 | 1.01 ± 0.01 |
| `GC_INITIAL_HEAP_SIZE=10g result-clangsan/bin/nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 4.703 ± 0.040 | 4.640 | 4.760 | 1.00 ± 0.01 |
| `GC_INITIAL_HEAP_SIZE=10g result-gccsan/bin/nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 4.766 ± 0.037 | 4.727 | 4.844 | 1.02 ± 0.01 |

Change-Id: I616ca3eab670317587d47b41870d8ac963c019ae
2024-03-27 23:54:04 -07:00
jade ffbad9b762 progress-bar.cc: fix signed overflow
this was caused by the use of std::chrono::duration::max() which gets
multiplied by some ratio to calculate nanoseconds to wait. then, it
explodes because that is a signed integer overflow. this was definitely
a bug.

error below:

/nix/store/fdiknsmnnczx6brsbppyljcs9hqckawk-gcc-12.3.0/include/c++/12.3.0/bits/chrono.h:225:38: runtime error: signed integer overflow: 9223372036854775807 * 1000000 cannot be represented in type 'long'
    #0 0x736d376b2b69 in std::chrono::duration<long, std::ratio<1l, 1000000000l>> std::chrono::__duration_cast_impl<std::chrono:
:duration<long, std::ratio<1l, 1000000000l>>, std::ratio<1000000l, 1l>, long, false, true>::__cast<long, std::ratio<1l, 1000l>>(
std::chrono::duration<long, std::ratio<1l, 1000l>> const&) /nix/store/fdiknsmnnczx6brsbppyljcs9hqckawk-gcc-12.3.0/include/c++/12
.3.0/bits/chrono.h:225:38
    #1 0x736d376b2b69 in std::enable_if<__is_duration<std::chrono::duration<long, std::ratio<1l, 1000000000l>>>::value, std::chr
ono::duration<long, std::ratio<1l, 1000000000l>>>::type std::chrono::duration_cast<std::chrono::duration<long, std::ratio<1l, 10
00000000l>>, long, std::ratio<1l, 1000l>>(std::chrono::duration<long, std::ratio<1l, 1000l>> const&) /nix/store/fdiknsmnnczx6brs
bppyljcs9hqckawk-gcc-12.3.0/include/c++/12.3.0/bits/chrono.h:270:9
    #2 0x736d376b2b69 in std::enable_if<__is_duration<std::chrono::duration<long, std::ratio<1l, 1000000000l>>>::value, std::chr
ono::duration<long, std::ratio<1l, 1000000000l>>>::type std::chrono::ceil<std::chrono::duration<long, std::ratio<1l, 1000000000l
>>, long, std::ratio<1l, 1000l>>(std::chrono::duration<long, std::ratio<1l, 1000l>> const&) /nix/store/fdiknsmnnczx6brsbppyljcs9
hqckawk-gcc-12.3.0/include/c++/12.3.0/bits/chrono.h:386:14
    #3 0x736d376b2b69 in std::cv_status std::condition_variable::wait_for<long, std::ratio<1l, 1000l>>(std::unique_lock<std::mut
ex>&, std::chrono::duration<long, std::ratio<1l, 1000l>> const&) /nix/store/fdiknsmnnczx6brsbppyljcs9hqckawk-gcc-12.3.0/include/
c++/12.3.0/condition_variable:164:6
    #4 0x736d376b1ee9 in std::cv_status nix::Sync<nix::ProgressBar::State, std::mutex>::Lock::wait_for<long, std::ratio<1l, 1000
l>>(std::condition_variable&, std::chrono::duration<long, std::ratio<1l, 1000l>> const&) /home/jade/lix/lix/src/libutil/sync.hh:
65:23
    #5 0x736d376b1ee9 in nix::ProgressBar::ProgressBar(bool)::'lambda'()::operator()() const /home/jade/lix/lix/src/libmain/prog
ress-bar.cc:99:27
    #6 0x736d36de25c2 in execute_native_thread_routine (/nix/store/a3zlvnswi1p8cg7i9w4lpnvaankc7dxx-gcc-12.3.0-lib/lib/libstdc++
.so.6+0xe05c2)
    #7 0x736d36b6b0e3 in start_thread (/nix/store/1zy01hjzwvvia6h9dq5xar88v77fgh9x-glibc-2.38-44/lib/libc.so.6+0x8b0e3) (BuildId
: 287831bffdbdde0ec25dbd021d12bdfc0ab9f5ff)
    #8 0x736d36bed5e3 in __clone (/nix/store/1zy01hjzwvvia6h9dq5xar88v77fgh9x-glibc-2.38-44/lib/libc.so.6+0x10d5e3) (BuildId: 28
7831bffdbdde0ec25dbd021d12bdfc0ab9f5ff)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /nix/store/fdiknsmnnczx6brsbppyljcs9hqckawk-gcc-12.3.0/include/c++/12.3.
0/bits/chrono.h:225:38 in

Change-Id: Ia0303242cdfd5d49385ae9e99718d709625a4633
2024-03-27 22:56:04 -07:00
Winter Cute 80405d0626 Stop vendoring toml11
We don't apply any patches to it, and vendoring it locks users into
bugs (it hasn't been updated since its introduction in late 2021).

Closes #164

Change-Id: Ied071c841fc30b0dfb575151afd1e7f66970fdb9
2024-03-27 21:04:00 -04:00
Qyriad 038daad218 meson: implement functional tests
Functional tests can be run with
`meson test -C build --suite installcheck`.

Notably, functional tests must be run *after* running `meson install`
(Lix's derivation runs the installcheck suite in installCheckPhase so it
does this correctly), due to some quirks between Meson and the testing
system.

As far as I can tell the functional tests are meant to be run after
installing anyway, but unfortunately I can't transparently make
`meson test --suite installcheck` depend on the install targets.

The script that runs the functional tests, meson/run-test.py, checks
that `meson install` has happened and fails fast with a (hopefully)
helpful error message if any of the functional tests are run before
installing.

TODO: this change needs reflection in developer documentation

Change-Id: I8dcb5fdfc0b6cb17580973d24ad930abd57018f6
2024-03-27 18:37:50 -06:00
jade edba570664 HOT SALE: 15% off your build times!
This was achieved by running maintainers/buildtime_report.sh on the
build directory of a meson build, then asking "why the heck is json
eating our build times", and strategically moving the json using bits
out of widely included headers.

It turns out that putting literally any metrics whatsoever into the
build had immediate and predictable results.

Results are 1382.5s frontend time -> 1175.4s frontend time, back end
time approximately invariant.

Related: #159

Change-Id: I7edea95c8536203325c8bb4dae5f32d727a21b2d
2024-03-27 03:52:57 +00:00
jade 412a9c9f67 Enable clang build timing analysis
I didn't enable this by default for clang due to making the build time
10% worse or so. Unfortunate, but tbh devs for whom 10% of build time is
not *that* bad should probably simply enable this.

Change-Id: I8d1e5b6f3f76c649a4e2f115f534f7f97cee46e6
2024-03-27 03:52:57 +00:00
jade 50c6feeb77 Add release notes system for dev facing release notes
We keep changing dev stuff and we probably should keep the news up to
date?

Change-Id: I819da6a29f1c56c8ab8d758c159a9c96164cb04e
2024-03-27 03:52:57 +00:00
eldritch horrors 0436f4cfa6 flake: always build release notes in devshell
Change-Id: I0e02567fe8f102a8a8f1558aa094eefacdac9393
2024-03-27 03:09:14 +00:00
eldritch horrors 279e30e7ef build: replace changelog-d with local script
hacking changelog-d to support not just github but also forgejo and
gerrit is a lot more complicated than it's worth, even moreso since
the entire thing can just as well be done with ~60 lines of python.
this new script is also much cheaper to instantiate (being python),
so having it enabled in all shells is far less of a hassle.

we've also adjusted existing release notes that referenced a gerrit
cl to auto-link to the cl in question, making the diff a bit bigger

closes #176

Change-Id: I8ba7dd0070aad9ba4474401731215fcf5d9d2130
2024-03-27 03:09:14 +00:00
eldritch horrors 8fd02df90d manual: fix release notes
fix key spelling errors, type errors, things-should-not-be-comments errors

Change-Id: I3ce12873aa78002bca686bd88404771895b05d30
2024-03-27 03:09:14 +00:00
Rebecca Turner aee3d639b5 Move shell_words into its own file
Change-Id: I34c0ebfb6dcea49bf632d8880e04075335a132bf
2024-03-26 16:44:04 -07:00
Rebecca Turner 8e63eca912 Remove HintFmt::operator%
Change-Id: Ibcf1a7848b4b18ec9b0807628ff229079ae7a0fe
2024-03-26 15:40:05 -07:00
jade da22dbc333 Merge "Issue importer: do not notify" into main 2024-03-26 22:04:48 +00:00
Ilya K fa3088a878 Merge "envrc: improve" into main 2024-03-26 20:49:40 +00:00
jade a5254186fd Merge "libstore/filetransfer: use Lix UA and unnix error message" into main 2024-03-26 20:46:27 +00:00
Qyriad 207d24da4e Merge "meson: implement unit tests" into main 2024-03-26 20:06:09 +00:00
Ilya K 312cee142a envrc: improve
- add shellcheck hint
- load .envrc.local if one exists
- use a variable to allow choosing the shell variant

Change-Id: Iea34e5a800f5d463e5792020c5c293b8b3071ca5
2024-03-26 19:40:43 +03:00
raito 80b66b5065 libstore/filetransfer: use Lix UA and unnix error message
Once this commit lands, we are even more visible in analytics FWIW.

Change-Id: Id7e0c162315d0f191edbea9cb5fb82ce363704b9
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-26 16:06:27 +00:00
jade 531b8d0ab8 Merge "libmain: version printer uses Lix instead of Nix" into main 2024-03-26 16:06:15 +00:00
Ilya K 7170e9f947 envrc: add
direnv good.

Change-Id: I81bd0086b847c6ee642a201fbc991a1f63fc7d7a
2024-03-26 15:19:37 +03:00
Ilya K a69f6e185a build-remote: fix format string shenanigans
HintFmt(string) invokes the HintFmt("%s", literal) constructor,
which is not what we want here. Add a constructor with a proper name
and call that.

Next step: rename all the other ones to HintFmt::literal(string).

Fixes: #178

Change-Id: If52d2eb8864ceb8663e05992e9d1fffef573d6b8
2024-03-26 07:58:24 +00:00
Lunaphied 73624f7d9c Merge "Merge pull request #8817 from iFreilicht/flake-update-lock-overhaul" into main 2024-03-26 00:45:49 +00:00
Qyriad e1ffe56793 meson: implement unit tests
Unit tests can be run with `meson test -C build --suite check`.
`--suite check` is optional, as right now that's the only test suite,
but when functional tests are added those will be in a separate suite.

Change-Id: I7f22f1cde4b489b3cdb5f9a36a544f0c409fcc1f
2024-03-26 00:43:33 +01:00
Théophane Hufschmitt 86881226b0 Merge pull request #8817 from iFreilicht/flake-update-lock-overhaul
Overhaul `nix flake update` and `nix flake lock` UX

(cherry picked from commit 12a0ae73dbb37becefa5a442eb4532ff0de9ce65)
Change-Id: Iff3b4f4235ebb1948ec612036b39ab29e4ca22b2
2024-03-25 17:36:24 -06:00
eldritch horrors c3a5f937d3 flake.nix: linearize meson builds
parallel meson builds need too much ram. linearize them for now, and
hopefully we can remove the make build system and this hack soonish.

Change-Id: I42c092db8b0c63680e77da2263cdfe9e7f6575be
2024-03-25 21:48:55 +00:00
Qyriad 787c4397f1 Merge "issue template: use nix --version instead of nix-env --version" into main 2024-03-25 21:47:53 +00:00
Qyriad 1da1aa5045 issue template: use nix --version instead of nix-env --version
`nix --version` doesn't require `nix-command` experimental feature to
run and we could all do with less nix-env

Change-Id: I90748d591c574d96eda46591e9f9ce828311da29
2024-03-25 12:59:30 -06:00
Eelco Dolstra aa7653608d Minor cleanup in libexpr/flake/flake.cc
(cherry picked from commit 05316d401fa509557c71140e17bb19814412fcb8)
Change-Id: I6ba0b55709f5fe21beb4e9f3bf72ee28715d15f3
2024-03-25 15:30:36 +00:00
Eelco Dolstra b525d0f20c Input: Replace markFileChanged() by putFile()
Committing a lock file using markFileChanged() required the input to
be writable by the caller in the local filesystem (using the path
returned by getSourcePath()). putFile() abstracts over this.

(cherry picked from commit 95d657c8b3ae4282e24628ba7426edb90c8f3942)
Change-Id: Ie081c5d9eb4e923b229191c5e23ece85145557ff
2024-03-25 15:30:36 +00:00
John Ericson 3d065192c0 Overhaul completions, redo #6693 (#8131)
As I complained in
https://github.com/NixOS/nix/pull/6784#issuecomment-1421777030 (a
comment on the wrong PR, sorry again!), #6693 introduced a second
completions mechanism to fix a bug. Having two completion mechanisms
isn't so nice.

As @thufschmitt also pointed out, it was a bummer to go from `FlakeRef`
to `std::string` when collecting flake refs. Now it is `FlakeRefs`
again.

The underlying issue that sought to work around was that completion of
arguments not at the end can still benefit from the information from
latter arguments.

To fix this better, we rip out that change and simply defer all
completion processing until after all the (regular, already-complete)
arguments have been passed.

In addition, I noticed the original completion logic used some global
variables. I do not like global variables, because even if they save
lines of code, they also obfuscate the architecture of the code.

I got rid of them  moved them to a new `RootArgs` class, which now has
`parseCmdline` instead of `Args`. The idea is that we have many argument
parsers from subcommands and what-not, but only one root args that owns
the other per actual parsing invocation. The state that was global is
now part of the root args instead.

This did, admittedly, add a bunch of new code. And I do feel bad about
that. So I went and added a lot of API docs to try to at least make the
current state of things clear to the next person.

--

This is needed for RFC 134 (tracking issue #7868). It was very hard to
modularize `Installable` parsing when there were two completion
arguments. I wouldn't go as far as to say it is *easy* now, but at least
it is less hard (and the completions test finally passed).

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Change-Id: If18cd5be78da4a70635e3fdcac6326dbfeea71a5
(cherry picked from commit 67eb37c1d0de28160cd25376e51d1ec1b1c8305b)
2024-03-25 15:30:36 +00:00