Commit graph

16009 commits

Author SHA1 Message Date
Qyriad 2cd1ef2201 Merge "libfetchers: fix URL logging" into main 2024-05-27 14:20:15 +00:00
Qyriad ec5039653d Merge "Revert "bump the extra --version info from info to notice, -vv -> -v"" into main 2024-05-26 21:51:05 +00:00
eldritch horrors ed6b3165ea worker: process timeouts first, and exclusively
check goals for timeouts first, and their activity fds only if no
timeout has occurred. checking for timeouts *after* activity sets
us up for assertion failures by running multiple build completion
notifiers, the first of which will kill/reap the the goal process
and consuming the Pid instance. when the second notifier attempts
to do the same it will core dump with an assertion failure in Pid
and take down not only the single goal, but the entire daemon and
all goals it was building. luckily this is rare in practice since
it requires a build to both finish and time out at the same time.

writing a test for this is not feasible due to how much it relies
on scheduling to actually trigger the underlying bug, but on idle
machines it can usually be triggered by running multiple sleeping
builds with timeout set to the sleep duration and `--keep-going`:

    nix-build --timeout 10 --builders '' --keep-going -E '
      with import <nixpkgs> {};
      builtins.genList
        (i: runCommand "foo-${toString i}" {} "sleep 10")
        100
    '

Change-Id: I394d36b2e5ffb909cf8a19977d569bbdb71cb67b
2024-05-26 20:57:09 +00:00
Qyriad 742c62a6eb docstrings: NixRepl::getDerivationPath: exceptions directly thrown
getDerivationPath() directly throws nix::Error for invalid derivations

Change-Id: I81ead950060b789794fa683b61c6349fece1690d
2024-05-26 17:19:26 +00:00
Qyriad 9f16a20f3d Revert "bump the extra --version info from info to notice, -vv -> -v"
This reverts commit d003dcd7f4.

d0390b5cf¹ bumped the default verbosity of nix3 commands, making this
change obsolete.

[1]: d0390b5cf2

Change-Id: I1ddc36ea4514cd8d2855ff334accf6e98c210879
2024-05-26 11:15:10 -06:00
Maximilian Bosch 5986a720d4 Merge "git putFile: support flake maximalists" into main 2024-05-26 10:16:40 +00:00
Qyriad ebd00b2d0b libfetchers: fix URL logging
8c06b7b43¹ made libfetchers log the URL being fetched just before the
actual fetch, particularly in case something freezes. This used the base
URL, to not include query parameters, as the Nixpkgs lib tests assume
that stderr logs will be equal across shallow and non-shallow git
fetches (and shallow fetches have the ?shallow=1 query parameter).

8c06b7b43 assumed that the `base` field of ParsedURL would be populated,
as the comment simply says "URL without query/fragment"... but
apparently it is not populated when the URL being fetched is *already*
fetched, which caused libfetchers to log things like

fetching gitlab input ''

which is. silly. but you know, busted lix be busted.

Anyway, with this commit we just remove the query params before printing
instead, which seems to do the right thing

[1]: 8c06b7b431

Change-Id: I9b9988992029aa6abef786f20b66e68c2ebb97d4
2024-05-25 19:37:38 -06:00
Artemis Tosini 53e2b0740c
libstore: remove duplicate builder variable
The `builder` local variable and duplicate `args.push_back` are no
longer required since the Darwin sandbox stopped using `sandbox-exec`.
The `drv->isBuiltin` check is not required either, as args are not
accessed when the builder is builtin.

Change-Id: I80b939bbd6f727b01793809921810ff09b579d54
2024-05-25 22:14:55 +00:00
jade dd53bce476 Merge "libstore: parse the buildMode instead of unchecked cast" into main 2024-05-25 17:42:09 +00:00
alois31 ddfe379a6b Merge "libstore/build: always enable seccomp filtering and no-new-privileges" into main 2024-05-25 04:21:53 +00:00
jade 2a7a824d83 libstore: parse the buildMode instead of unchecked cast
Change-Id: Icf6af7935e8f139bef36b40ad475e973aa48855c
2024-05-24 20:45:05 -06:00
Qyriad 5b4b216fac Merge "nix3-upgrade-nix: fix when there are differing pnames" into main 2024-05-25 02:20:24 +00:00
Qyriad 1c0f3c540e Merge changes I3ab84cc5,Iba34ad42 into main
* changes:
  nix3: always use the same verbosity default (info)
  libfetchers: log fetches by URL just before they happen
2024-05-25 01:52:44 +00:00
Qyriad f3f68fcfac nix3-upgrade-nix: fix when there are differing pnames
Change-Id: I19c7e24a4d46137127e76b7bb133e0184d73d1b6
2024-05-25 01:50:32 +00:00
Qyriad 076c19e0d1 change "evaluating file" logs to debug
I can't imagine wanting this unless you are debugging something (in
which case it's very useful)

Change-Id: I90c6f182c18486e9f6b15a59379bbb8e88fb8e7f
2024-05-24 23:03:08 +00:00
alois31 f047e4357b libstore/build: always enable seccomp filtering and no-new-privileges
Seccomp filtering and the no-new-privileges functionality improve the security
of the sandbox, and have been enabled by default for a long time. In
lix-project/lix#265 it was decided that they
should be enabled unconditionally. Accordingly, remove the allow-new-privileges
(which had weird behavior anyway) and filter-syscall settings, and force the
security features on. Syscall filtering can still be enabled at build time to
support building on architectures libseccomp doesn't support.

Change-Id: Iedbfa18d720ae557dee07a24f69b2520f30119cb
2024-05-24 21:19:29 +00:00
Qyriad d0390b5cf2 nix3: always use the same verbosity default (info)
Change-Id: I3ab84cc583e3e8b1c05a8ae1a7a087204f513d03
2024-05-24 15:15:42 -06:00
Qyriad 8c06b7b431 libfetchers: log fetches by URL just before they happen
Addresses but does not close #305, as we still need an indicator for
frozen fetches.

Change-Id: Iba34ad42dc1c8772f7da249b90fe794b041bbf73
2024-05-24 15:15:42 -06:00
jade 19ea351642 Merge "clang-tidy: work with angle brackets and external projects" into main 2024-05-24 19:11:26 +00:00
Qyriad adfc22c3e3 Merge "make CTRL+Z work in the REPL" into main 2024-05-24 17:37:33 +00:00
Qyriad 933f1f48a2 Merge "justfile: remove --quiet from just test" into main 2024-05-24 17:37:25 +00:00
Qyriad 65da3e7199 make CTRL+Z work in the REPL
Editline just wasn't being built with --enable-sigstop lol

Change-Id: I35a78f74ea100d97f26b2b41990deb373fd9cd9a
2024-05-24 03:10:12 +00:00
Qyriad 2b397c6629 Merge "add editorconfig for markdown" into main 2024-05-24 03:08:25 +00:00
jade 745b5d3d4f clang-tidy: work with angle brackets and external projects
Also fix the readme

Change-Id: I422dff5536bf01d43983621aa01035bd77ac0252
2024-05-24 02:22:58 +00:00
Qyriad e1d2fb4a65 add editorconfig for markdown
Change-Id: I493fc37fde425fc5c5c24f9b077bdc235271233c
2024-05-24 02:22:44 +00:00
Qyriad bb6d43b63b justfile: remove --quiet from just test
`meson test` refuses to let `--verbose` (which shows the entire
invocation and stdio) override `--quiet`, but if neither are specified
in the justfile then you can use either `just test -q` or `just test -v`

Change-Id: I449e13084ce64666b7ee2ab4280818782fb8185a
2024-05-24 02:22:25 +00:00
Pierre Bourdon 6922d67eb3 package.nix: add meta.mainProgram
nix-repl> lib.getExe (builtins.getFlake ".").packages.x86_64-linux.nix
"/nix/store/ajps2zn5hlap0l2abvnfdaphg8k0789r-lix-2.90.0pre20240524_dirty/bin/nix"

Fixes #316.

Change-Id: Iec3125dc2dc99d100beb6357f7d1555456924ddc
2024-05-24 02:15:45 +00:00
jade d1fa446454 [emergency revert]: make pname nix once more
Embarrassingly, I submitted a CL overriding submit requirements since
I thought it was spurious failures. However, the CI failure was in fact
real, and I have hopefully learned my lesson. The CI failure is that:

```
vm-test-run-nix-upgrade-nix> machine # installing 'nix-2.18.1'
vm-test-run-nix-upgrade-nix> machine # building '/nix/store/2b6fdf7wvahd00bg2ff0393bhd597a0h-user-environment.drv'...
vm-test-run-nix-upgrade-nix> machine # error: Unable to build profile. There is a conflict for the following files:
vm-test-run-nix-upgrade-nix> machine #
vm-test-run-nix-upgrade-nix> machine #          /nix/store/dn6mhhr92bh3ad0n4pd1538ww88khjii-nix-2.18.1/lib/libboost_context.so
vm-test-run-nix-upgrade-nix> machine #          /nix/store/w4vffn9iq0znk8bcg5i2giij90xy6db6-lix-2.90.0pre20240523_c97e171/lib/libboost_context.so
vm-test-run-nix-upgrade-nix> machine # error: builder for '/nix/store/2b6fdf7wvahd00bg2ff0393bhd597a0h-user-environment.drv' failed with exit code 1
vm-test-run-nix-upgrade-nix> machine # error: program '/nix/store/w4vffn9iq0znk8bcg5i2giij90xy6db6-lix-2.90.0pre20240523_c97e171/bin/nix-env' failed with exit code 100
```

This is definitely caused by the pname not being the same, so we had
better revert that part of the change until we know we won't regress
anything by doing this.

Fixes: https://gerrit.lix.systems/c/lix/+/1152/5
Change-Id: I0e9d573987f2819c106fb7cea87410fa75152274
2024-05-24 01:33:42 +00:00
Pierre Bourdon 6ade981476 Merge "nixos/tests: enable remoteBuilds tests against Nix 2.18" into main 2024-05-24 00:55:25 +00:00
Qyriad 00bf2b105d Merge changes I462a8cf0,I3b0bcea3,I2acd56e7,Ifc149764,I9e2ef170 into main
* changes:
  docs: linkify nix3-build mention in nix-build.md
  build: make internal-api-docs PHONY
  cleanup lookupFileArg
  add docstring to lookupFileArg
  add libcmd test for lookupFileArg
2024-05-23 23:28:42 +00:00
jade 9530b7f2b2 Merge "packaging: rename nixexpr -> lixexpr and so on" into main 2024-05-23 23:19:28 +00:00
jade c97e17144e packaging: rename nixexpr -> lixexpr and so on
This breaks downstreams linking to us on purpose to make sure that if
someone is linking to Lix they're doing it on purpose and crucially not
mixing up Nix and Lix versions in compatibility code.

We still need to fix the internal includes to follow the same schema so
we can drop the single-level include system entirely. However, this
requires a little more effort.

This adds pkg-config for libfetchers and config.h.

Migration path:
expr.hh      -> lix/libexpr/expr.hh
nix/config.h -> lix/config.h

To apply this migration automatically, remove all `<nix/>` from
includes, so: `#include <nix/expr.hh>` -> `#include <expr.hh>`. Then,
the correct paths will be resolved from the tangled mess, and the
clang-tidy automated fix will work.

Then run the following for out of tree projects:

```
lix_root=$HOME/lix
(cd $lix_root/clang-tidy && nix develop -c 'meson setup build && ninja -C build')
run-clang-tidy -checks='-*,lix-fixincludes' -load=$lix_root/clang-tidy/build/liblix-clang-tidy.so -p build/ -fix src
```

Related: lix-project/nix-eval-jobs#5
Fixes: lix-project/lix#279
Change-Id: I7498e903afa6850a731ef8ce77a70da6b2b46966
2024-05-23 16:45:23 -06:00
Qyriad a0172dc81b docs: linkify nix3-build mention in nix-build.md
Change-Id: I462a8cf0da42b5045ce84b48dc1841ecdccbb89e
2024-05-23 14:11:10 -06:00
Qyriad a8b2fc6d41 build: make internal-api-docs PHONY
Since we're skipping Meson's dependency tracking, for the
internal-api-docs custom target, we should just consider it a phony
target and build it on every request.

Change-Id: I3b0bcea30ee9a4830023ccc5bededf995e96cccc
2024-05-23 14:11:10 -06:00
Qyriad a0dcfbb084 cleanup lookupFileArg
Change-Id: I2acd56e7a542b12138f43c95af78fdd50e944619
2024-05-23 14:11:10 -06:00
Qyriad 3a597f1d0b add docstring to lookupFileArg
Change-Id: Ifc149764f5a15725d3d630677c6da29def4b0f3e
2024-05-23 14:11:10 -06:00
Qyriad 0565f97e78 add libcmd test for lookupFileArg
Change-Id: I9e2ef170ffe916f902daec8b5630d29434c5d5f2
2024-05-23 14:11:10 -06:00
Pierre Bourdon 6260563bed
nixos/tests: enable remoteBuilds tests against Nix 2.18
Fixes #321.

Change-Id: I60812aec9f9b68ab742413835c581d3b53432b9b
2024-05-23 21:16:40 +02:00
Qyriad 677cf75473 repl: log errors writing to history file
These errors are now logged and explicitly ignored, rather than
implicitly ignored.

Change-Id: Ia26015466a17f2b11952df5317a4d150d79dc184
2024-05-23 18:10:23 +00:00
Pierre Bourdon 79121e0c44 Merge "repl: do not crash when tab-completing import errors" into main 2024-05-23 10:48:38 +00:00
eldritch horrors 829521b91a Merge "doc: sort change-authors.yml" into main 2024-05-23 02:35:53 +00:00
eldritch horrors f281727e70 Merge "libstore: remove unused copyPath function" into main 2024-05-23 02:31:50 +00:00
eldritch horrors 9a75150d19 doc: sort change-authors.yml
this should make it easier to spot future instances of entries being
duplicated by accident. also add a pre-commit check to remain sorted

Change-Id: I500caf862e93480b38c9d51144273bb2dcab1af0
2024-05-23 02:09:05 +00:00
Pierre Bourdon d8bc3bfb6d
repl: do not crash when tab-completing import errors
File not found while importing causes a SysError, not an EvalError,
which is not currently caught by the tab-completion handler. Ignoring
all SysErrors might seem "dangerous" but this is the tab-completion
handler, any exception being bubbled up from there causes unexpected
behavior (causes the whole repl to exit).

Fixes #340.

Change-Id: I643048a47935e77f582decc539d9e51bdb96c890
2024-05-23 02:52:54 +02:00
jade d05e0b9f1f fix: enlarge envSize by several times to not be close to nixpkgs size
nixpkgs has 23000 attributes, and our previous limit would be hit if you
have more than one nixpkgs in the environment, for example, because
`repl-overlays` will load the new stuff from the environment on top of
the existing environment.

This is not really testable since if we did write such a test, it would
just be testing this constant tbh...

Fixes: lix-project/lix#337
Change-Id: I49197bfb4db55b082f914f0d70e84f5f5f110954
2024-05-22 17:08:37 -07:00
Qyriad 06c1375e52 Merge "libfetchers: fallback to memory SQLite if fs IO fails" into main 2024-05-22 21:17:46 +00:00
jade dcc7ea5498 release notes: add a bunch of them
Also fix typos introduced by the commits I read.

I have run the addDrvOutputDependencies release note past Ericson since
I was confused by what the heck it was doing, and he was saying it was
reasonable.

Change-Id: Id015353b00938682f7faae7de43df7f991a5237e
2024-05-22 21:13:56 +02:00
alois31 0bf4c2971f Merge "doc: fix repl-interrupt release note entry" into main 2024-05-22 18:59:54 +00:00
eldritch horrors 47523944c5 libstore: remove unused copyPath function
Change-Id: Ibfb0848d8b337bd38947a745c240838865cea401
2024-05-22 18:26:01 +02:00
Qyriad 6881476232 libfetchers: fallback to memory SQLite if fs IO fails
nix::fetchers::CacheImpl uses $XDG_CACHE_HOME, or its default based on
$HOME, to store its SQLite database. If the current process can't write
to that directory for whatever reason, though, any eval-time fetching
would fail just initializing the cache.

With this change, IO errors initializing the fetcher cache are logged
but ignored, and nix::fetchers::CacheImpl falls back to an in-memory¹
database instead.

Notably, this will fix any uses eval fetching while Lix itself is being
run in a derivation builder (such as during tests), as the derivation
builder does not set $XDG_CACHE_HOME, and sets $HOME to the non-existent
directory /homeless-shelter.

Before:

$ env -u XDG_CACHE_HOME HOME=/homeless-shelter nix -Lv eval --impure -E 'fetchTarball "https://git.lix.systems/lix-project/lix/archive/main.tar.gz"'
error:
       … while calling the 'fetchTarball' builtin
         at «string»:1:1:
            1| fetchTarball "https://git.lix.systems/lix-project/lix/archive/main.tar.gz"
             | ^

       error: creating directory '/homeless-shelter': Permission denied

After:

$ env -u XDG_CACHE_HOME HOME=/homeless-shelter nix -Lv eval --impure -E 'fetchTarball "https://git.lix.systems/lix-project/lix/archive/main.tar.gz"'
warning: ignoring error initializing Lix fetcher cache: error: creating directory '/homeless-shelter': Permission denied
"/nix/store/s9lxdnn0awp37n560bg4fgr497ah4hvw-source"

¹: https://www.sqlite.org/inmemorydb.html

Change-Id: I15c38c9baaf215fc6e192b8a4c70b9692a69bc22
2024-05-21 09:30:25 -06:00