* changes:
Change error messages about 'invalid paths' to 'path does not exist'.
Add a clearer error message for InvalidPathError during evaluation
Harmonise the Store::queryPathInfoUncached interface
This:
- Consistently returns `nullptr` for a non-existent
store path, instead of a mix of `nullptr` and
throwing exceptions.
- If a store returns "bad" store paths in response
to a request (e.g. incorrect hash or name), don't
cache this result. This removes some duplication
of code at the cache-access layer of queryPathInfo()
checking this, and allows us to provide more
specific errors.
Part of #270.
Change-Id: I86612c6499b1a37ab872c712c2304d6a3ff19edb
This commit constitutes the branch-off for 2.91. The parent of this
commit will be the branch point for release-2.90.
Change-Id: I7f047545df29a9cff93346137c865dcbf1415488
For now we just need to put the release notes in the final spot. We will
have to fix the date on both 2.90 and 2.91 branches, but such as it is.
Release created with releng/create_release.xsh
Closes: lix-project/lix#318
Change-Id: I38e79b40e7f632c8a286f2f09865a84dc93eca90
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh
Change-Id: I92296a1746b54a081004fe2bb23e9e37fd33b3e5
* changes:
releng: add sha256 for the manual tarball
releng: fix upload of multiarch images to forgejo
releng: fix git checking
releng: fix logging inside interactive xonsh
releng: support multiple systems
version: update to 2.90.0-rc1
Forgejo appears to immediately delete registry content that is
overwritten. This means that we are forced to delete our previous
workaround of making a temporary tag and use a new, more absurd
workaround of making an entire temporary image that we basically only
need to create to get its hash.
However, on the plus side, the new workaround doesn't create garbage
tags to begin with, which means that we don't have to deal with GitHub
not implementing the standardized tag delete endpoint and instead
only implementing a proprietary one.
Upstream-Bug: https://github.com/containers/skopeo/issues/2354
Change-Id: I220e7ce9a17fd230c38882f12c009a166dcc9336
I don't know when this broke, it seems like it happened since the 24.05
upgrade, so xonsh 0.15.
What happened is that xonsh was trying to intercept log output, which
explodes if you have the logger survive past one command input. This is,
however, impossible to avoid if you are trying to use logging when you
import releng from inside xonsh for interactive use!
The error below is because the memory handler backing the stdout/stderr
of the one command that's just been run was closed after the command
completed.
Change-Id: I2be642aebf93da9818d08ff8b97c2e72ba5ac581
--- Logging error ---
Traceback (most recent call last):
File "/nix/store/7hnr99nxrd2aw6lghybqdmkckq60j6l9-python3-3.11.9/lib/python3.11/logging/__init__.py", line 1113, in emit
stream.write(msg + self.terminator)
File "/nix/store/34951j60xcsw6zj4v8lsaf491acv0by3-python3-3.11.9-env/lib/python3.11/site-packages/xonsh/base_shell.py", line 183, in write
self.mem.write(s)
ValueError: I/O operation on closed file.
Call stack:
File "/nix/store/xgdp1p1gv8ni1awnkzyqasnn6gz5wlvx-xonsh-0.15.1/bin/xonsh", line 8, in <module>
sys.exit(main())
File "/nix/store/34951j60xcsw6zj4v8lsaf491acv0by3-python3-3.11.9-env/lib/python3.11/site-packages/xonsh/main.py", line 470, in main
sys.exit(main_xonsh(args))
File "/nix/store/34951j60xcsw6zj4v8lsaf491acv0by3-python3-3.11.9-env/lib/python3.11/site-packages/xonsh/main.py", line 514, in main_xonsh
shell.shell.cmdloop()
File "/nix/store/34951j60xcsw6zj4v8lsaf491acv0by3-python3-3.11.9-env/lib/python3.11/site-packages/xonsh/ptk_shell/shell.py", line 406, in cmd
loop
line = self.singleline(auto_suggest=auto_suggest)
File "/nix/store/34951j60xcsw6zj4v8lsaf491acv0by3-python3-3.11.9-env/lib/python3.11/site-packages/xonsh/ptk_shell/shell.py", line 374, in sin
gleline
line = self.prompter.prompt(**prompt_args)
File "/nix/store/34951j60xcsw6zj4v8lsaf491acv0by3-python3-3.11.9-env/lib/python3.11/site-packages/prompt_toolkit/shortcuts/prompt.py", line 1
026, in prompt
return self.app.run(
File "/nix/store/34951j60xcsw6zj4v8lsaf491acv0by3-python3-3.11.9-env/lib/python3.11/site-packages/prompt_toolkit/application/application.py",
line 1002, in run
return asyncio.run(coro)
File "/nix/store/7hnr99nxrd2aw6lghybqdmkckq60j6l9-python3-3.11.9/lib/python3.11/asyncio/runners.py", line 189, in run
with Runner(debug=debug) as runner:
File "/nix/store/7hnr99nxrd2aw6lghybqdmkckq60j6l9-python3-3.11.9/lib/python3.11/asyncio/runners.py", line 59, in __enter__
self._lazy_init()
File "/nix/store/7hnr99nxrd2aw6lghybqdmkckq60j6l9-python3-3.11.9/lib/python3.11/asyncio/runners.py", line 137, in _lazy_init
self._loop = events.new_event_loop()
File "/nix/store/7hnr99nxrd2aw6lghybqdmkckq60j6l9-python3-3.11.9/lib/python3.11/asyncio/events.py", line 810, in new_event_loop
return get_event_loop_policy().new_event_loop()
File "/nix/store/7hnr99nxrd2aw6lghybqdmkckq60j6l9-python3-3.11.9/lib/python3.11/asyncio/events.py", line 699, in new_event_loop
return self._loop_factory()
File "/nix/store/7hnr99nxrd2aw6lghybqdmkckq60j6l9-python3-3.11.9/lib/python3.11/asyncio/unix_events.py", line 64, in __init__
super().__init__(selector)
File "/nix/store/7hnr99nxrd2aw6lghybqdmkckq60j6l9-python3-3.11.9/lib/python3.11/asyncio/selector_events.py", line 54, in __init__
logger.debug('Using selector: %s', selector.__class__.__name__)
Message: 'Using selector: %s'
Arguments: ('EpollSelector',)
Change-Id: I90959809129aaf96aad4577599031688599ed85e
This is motivated by flakes being bad and all the stuff that calls
things by "system" being utterly unable to cope with cross compilation.
So if we go shove it in package.nix it is suddenly usable from cross
contexts.
Usage:
```
nix build -L .#nix-riscv64-linux.binaryTarball
```
Change-Id: I702ebf2ac5bd9d1c57662f968b000073134df336
It builds. I have not tested the binaries since I don't have hardware,
but I would be rather surprised if it were broken, given that nix *runs*
on this platform.
Change-Id: I0b474ffcd4a431bf117a303d0b65fa6532113f48
There were two bugs I found:
1. If the build isn't already done in the store, nix-store --realise
does not know how to build it. You have to just give it the
derivation and I guess it will realise all outputs, which is fine.
2. cp without -T will not overwrite an existing manual directory,
creating a path manual/manual.
Change-Id: Ibebfd136a266da5330944a985e636ebb776f1909
This seems to have been caused by having the wrong PID. I don't know why
it worked before in the sandbox, but the code was definitely wrong
before, so let's just fix it.
Change-Id: I556580bdf614c716566310e975a36daa6d6c9a91
This was originally going to be just the testsuite but I kinda just
documented all of them.
I am tired of us not documenting these. This is a starting point to
producing an actually good index. I would like to enforce it in a
pre-commit hook eventually that we document all environment variables
used in Lix itself, even if it is terse dev facing docs.
This is full of a bunch of TODOs caused by auditing code. They should
probably be done at some point.
Change-Id: I7c0d3b257e19bae23d47d1efbd7361d203bccb0e
It's in the security section, and it was totally outdated anyway.
I took the opportunity to write down the stuff we already believed.
Change-Id: I73e62ae85a82dad13ef846e31f377c3efce13cb0
Followup to https://gerrit.lix.systems/c/lix/+/1417 to ensure that this
parser will never take something that doesn't look like a version.
It turns out this problem is less alarming than initially thought
because it only applies to the testsuite in a non-default mode.
Change-Id: I26aba24aaf0215f2b782966314b94784db766266
We should not let these regress in CI by having broken dependencies or
similar. Still need to fix the evaluation error checking in
buildbot-nix, but this is a useful step regardless.
Fixes: lix-project/lix#383
Change-Id: I3883184165440e66256c989117f2ab2e54c3aafd
-- message from cl/1418 --
The boehmgc changes are bundled into this commit because doing otherwise
would require an annoying dance of "adding compatibility for < 8.2.6 and
>= 8.2.6" then updating the pin then removing the (now unneeded)
compatibility. It doesn't seem worth the trouble to me given the low
complexity of said changes.
Rebased coroutine-sp-fallback.diff patch taken from https://github.com/NixOS/nixpkgs/pull/317227
-- jade resubmit changes --
This is a resubmission of https://gerrit.lix.systems/c/lix/+/1418, which
was reverted in https://gerrit.lix.systems/c/lix/+/1432 for breaking CI
evaluation without being detected.
I have run `nix flake check -Lv` on this one before submission and it
passes on my machine and crucially without eval errors, so the CI result
should be accurate.
It seems like someone renamed forbiddenDependenciesRegex to
forbiddenDependenciesRegexes in nixpkgs and also changed the type
incompatibly. That's pretty silly, but at least it's just an eval error.
Also, `xonsh` regressed the availability of `xonsh-unwrapped`, but it
was fixed by us in https://github.com/NixOS/nixpkgs/pull/317636, which
is now in our channel, so we update nixpkgs compared to the original
iteration of this to simply get that.
We originally had a regression related to some reorganization of the
nixpkgs lib test suite in which there was broken parameter passing.
This, too, we got quickfixed in nixpkgs, so we don't need any changes
for it: https://github.com/NixOS/nixpkgs/pull/317772
Related: https://gerrit.lix.systems/c/lix/+/1428
Fixes: lix-project/lix#385
Change-Id: I26d41ea826fec900ebcad0f82a727feb6bcd28f3
The libcmd unit test creates files (more specifically, the fetcher cache) in
its home directory. In the single-user sandbox, this leads to the creation of
/homeless-shelter, since this is the default HOME and the root is writable.
Unfortunately, this conflicts with the assumption of the functional tests that
this directory does not exist. Use a different home directory to prevent these
test failures, and thus restore the ability to build inside the single-user
sandbox.
Fixes: lix-project/lix#365
Change-Id: I4df8c53d043234b95a7c0ac45fc5ee89e8d46aff
* changes:
releng: add prod environment, ready for release
releng: automatically figure out if we should tag latest for docker
releng: support multiarch docker images
manual: rewrite the docker guide now that we have images
Rewrite docker to be sensible and smaller
Implement docker upload in the releng tools
I am *reasonably* confident that this releng infrastructure can actually
build a Lix 2.90 and release it successfully. Let's make it possible to
do, and add some cute colours to the confirmation message.
Change-Id: I85e498b6fb49ffc5e75c0a72c5e45fb1f69030d3
For example, when releasing from release-2.90, if `main` has a 2.91 tag
ancestor, we know that 2.91 was released, so we should *not* tag latest.
Change-Id: Ia56b17a2ee03bbec74b7c271c742858c690d450d
If we don't want to have separate registry tags by architecture (EWWWW),
we need to be able to build multiarch docker images. This is pretty
simple, and just requires making a manifest pointing to each of the
component images.
I was *going* to just do this API prodding with manifest-tool, but it
doesn't support putting metadata on the outer manifest, which is
actually kind of a problem because it then doesn't render the metadata
on github. So I guess we get a simple little containers API
implementation that is 90% auth code.
Change-Id: I8bdd118d4cbc13b23224f2fb174b232432686bea
I have checked the image can build things and inspected `diff -ru`
compared to the old image. As far as I can tell it is more or less
the same besides the later git change.
Layers are now 65MB or less, and we aren't against the maxLayers limit
for the broken automatic layering to do anything but shove one store
path in a layer (which is good behaviour, actually).
This uses nix2container which streams images, so the build time is much
shorter.
I have also taken the opportunity to, in addition to fixing the 400MB
single layer (terrible, and what motivated this in the first place),
delete about 200MB of closure size inflicted by git vs gitMinimal
causing both perl and python to get into closure.
People mostly use this thing for CI, so I don't really think you need
advanced git operations, and large git can be added at the user side if
really motivated.
With love for whichever container developer somewhat ironically assumed
that one would not run skopeo in a minimal container that doesn't have a
/var/tmp.
Fixes: lix-project/lix#378
Change-Id: Icc3aa20e64446276716fbbb87535fd5b50628010
The boehmgc changes are bundled into this commit because doing otherwise
would require an annoying dance of "adding compatibility for < 8.2.6 and
>= 8.2.6" then updating the pin then removing the (now unneeded)
compatibility. It doesn't seem worth the trouble to me given the low
complexity of said changes.
Rebased coroutine-sp-fallback.diff patch taken from https://github.com/NixOS/nixpkgs/pull/317227
Change-Id: I8c590e9fe25c0f566d0cfeacb96d8cf50abf12e8
4b128008c5d9fde881ce1b0a25e60ae0415a14d5 in nixpkgs introduced a default
hashedPasswordFile for root in NixOS tests, which takes precedence over
the password option set in the nix-copy test.
Change-Id: Iffaebec5992e50614b854033f0d14312c8d275b5
Since ad8a4b380e, the version printer returns "nix (Lix, like Nix) 2.x",
hence the `daemonVersion` was being set to the string "like".
Using `compareVersions` with a letter compares them lexicographically:
builtins.compareVersions "like" "2.12pre20230103" // => -1
builtins.compareVersions "like" "2.16.0" // => -1
This caused that `isDaemonNewer` always returned 1, falsy in Bash terms.
Therefore, the test suite skipped those tests where they use it.
Fixes lix-project/lix#324
Change-Id: If6682515bf0bf8b8add641af9a4e98b50a9acb51
This can release x86_64-linux binaries to staging, with ephemeral keys.
I think it's good enough to review at least at this point, so we don't
keep adding more stuff to it to make it harder to review.
Change-Id: Ie95e8f35d1252f5d014e819566f170b30eda152e
We realized that there's really no good place to put these dev facing
bulletins, and the user-facing release notes aren't really the worst
place to put them, I guess, and we do kind of hope that it converts
users to devs.
Change-Id: Id9387b2964fe291cb5a3f74ad6344157f19b540c
clangd seems to break if GCC is using precompiled headers for C++'s
standard library, so this sets -Denable-pch-std=${stdenv.cc.isClang}
Fixes#374.
Change-Id: Ic4be41ebe7576ebcb9c208275596f953c2003109
They are enabled by default, and Meson will also prints whether or not
they're enabled at the bottom at the end of configuration.
Change-Id: I48db238510bf9e74340b86f243f4bbe360794281
Fixes a compiler error that looks like:
error: could not convert '[...]' from 'future<void>' to 'future<nix::FileTransferResult>'
Change-Id: I4aeadfeba0dadfdf133f25e6abce90ede7a86ca6
In most real world cases, the Link header is set on the redirect, not on
the final file. This regressed in Lix earlier and while new unit tests
were added to cover it, this integration test should probably have also
caught it.
Change-Id: I2a9d8d952fff36f2c22cfd751451c2b523f7045c
This reverts commit d0390b5cf2.
Other parts of the codebase will need to be adjusted in response to a
default verbosity change. Let's just push this to after 2.90.
Fixes#362.
Fixes#367.
Change-Id: I04648473579146851bda41d764adc1ef954c355d
Here's my guide so far:
$ rg '((?!(recursive).*) Nix
(?!(daemon|store|expression|Rocks!|Packages|language|derivation|archive|account|user|sandbox|flake).*))'
-g '!doc/' --pcre2
All items from this query have been tackled. For the documentation side:
that's for lix-project/lix#162.
Additionally, all remaining references to github.com/NixOS/nix which
were not relevant were also replaced.
Fixes: lix-project/lix#148.
Fixes: lix-project/lix#162.
Change-Id: Ib3451fae5cb8ab8cd9ac9e4e4551284ee6794545
Signed-off-by: Raito Bezarius <raito@lix.systems>
This causes libstore, libexpr, libfetchers, and libutil to be linked
with -Wl,--whole-archive to executables, when building statically.
libstore for the store backends, libexpr for the primops, libfetchers
for the fetcher backends I assume(?), and libutil for the nix::logger
initializer (which notably shows in pre-main constructors when HOME is
not owned by the user. cursed.).
This workaround should be removed when #359 is fixed.
Fixes#306.
Change-Id: Ie9ef0154e09a6ed97920ee8ab23810ca5e2de84c
Fixes: lix-project/lix#157
Fixes: lix-project/lix#221
Previously the entire escaped store URI was included. This would cause
build failures if a very long or deeply nested path was being used in
the store.
Now, we use the first 48 characters of the URL (escaped), then 16 bytes
of hash of the entire URL. This should never collide and limits the
length of the file name to a bit over 64, which is fine.
Change-Id: Ic1ba690a94e83749567c2c29460b8d1bcf2ac413
This is because a dynamic_cast<nix::RootArgs *> of a (n-e-j) MyArgs
returns nullptr even though MyArgs has virtual nix::RootArgs as a
parent.
class MyArgs : virtual public nix::MixEvalArgs,
virtual public nix::MixCommonArgs,
virtual nix::RootArgs { ... };
So this should work right?? But it does not. We found out that it's
caused by -fvisibility=hidden in n-e-j, but honestly this code was bad
anyway.
The trivial solution is to simply stop relying on RTTI working properly
here, which is probably better OO architecture anyway. However, I am not
100% confident *this* is sound, since we have this horrible hierarchy:
Args (defines getRoot)
/ | \
RootArgs MixCommonArgs MixEvalArgs
(overrides)
I am not confident that this is guaranteed to resolve from Args always
in the case of this override.
Assertion failed: (res), function getRoot, file src/libutil/args.cc, line 67.
6MyArgsProcess 60503 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = hit program assert
frame #4: 0x0000000100b1a41c liblixutil.dylib`nix::Args::processArgs(std::__1::list<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, bool) [inlined] nix::Args::getRoot(this=0x00000001000d0688) at args.cc:67:5 [opt]
64 std::cout << typeid(*p).name();
65
66 auto * res = dynamic_cast<RootArgs *>(p);
-> 67 assert(res);
68 return *res;
69 }
70
Target 0: (nix-eval-jobs) stopped.
(lldb) p this
(MyArgs *) 0x00000001000d0688
(lldb) p *this
(nix::Args) {
longFlags = size=180 { ... }
shortFlags = size=4 { ... }
expectedArgs = size=1 { ... }
processedArgs = size=0 {}
hiddenCategories = size=1 {
[0] = "Options to override configuration settings"
}
parent = nullptr
}
We also found that if we did this:
class [[gnu::visibility("default")]] RootArgs : virtual public Args
it would work properly (???!). This is of course, very strange, because
objdump -Ct output on liblixexpr.dylib is identical both with and
without it.
Possibly related: https://www.qt.io/blog/quality-assurance/one-way-dynamic_cast-across-library-boundaries-can-fail-and-how-to-fix-it
Fixes: lix-project/nix-eval-jobs#2
Change-Id: I6b9ed968ed56420a9c4d2dffd18999d78c2761bd
It seems like someone implemented precompiled headers a long time ago
and then it never got ported to meson or maybe didn't work at all.
This is, however, blessedly easy to simply implement. I went looking for
`#define` that could affect the result of precompiling the headers, and
as far as I can tell we aren't doing any of that, so this should truly
just be free build time savings.
Previous state:
Compilation (551 times):
Parsing (frontend): 1302.1 s
Codegen & opts (backend): 956.3 s
New state:
**** Time summary:
Compilation (567 times):
Parsing (frontend): 1123.0 s
Codegen & opts (backend): 1078.1 s
I wonder if the "regression" in codegen time is just doing the PCH
operation a few times, because meson does it per-target.
Change-Id: I664366b8069bab4851308b3a7571bea97ac64022
Before:
$ nix flake lock --override-input nixpkgs gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent
fetching git input 'git+file:///home/linus/projects/lix'
fetching gitlab input 'gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent'
error: [json.exception.type_error.302] type must be string, but is null
After:
$ outputs/out/bin/nix flake lock --override-input nixpkgs gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent
fetching git input 'git+file:///home/linus/projects/lix'
fetching gitlab input 'gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent'
error:
… while updating the lock file of flake 'git+file:///home/linus/projects/lix?ref=refs/heads/fix-gitlab-nonexistent&rev=915f16a619a36237a099b9aa9afed6d14ff613b4'
… while updating the flake input 'nixpkgs'
… while fetching the input 'gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent'
error: No commits returned by GitLab API -- does the ref really exist?
Change-Id: Id9bc79d98348500e152ed519bb3ac79a3d15c38d
This reverts commit 285bc67318.
Reason for revert: lix-project/lix#364
For some reason this broke `main` even though the change we are reverting passed CI! Mysterious, haunted, etc. Needs more debugging, let's turn it off for now.
Change-Id: Ica4819d61cd35b83eb52985bfcb657e858f025a9
If our shellHook is being run from a nested nix-shell (see 7a12bc200¹),
then (I think) it is run from a bash function due to the nesting, then
`return` is correct. If its `eval`'d though, then there isn't really a
correct way to early exit. So we can just unconditionally be executed in
a function.
Basically, we have IIFE at home.
[1]: 7a12bc2007
Change-Id: Iacad25cbbf66cde2911604e6061e56ad6212af7e
We reviewed this code a while ago, and we neglected to get a comment in
saying why it's Like This at the time. Let's fix that, since it is code
that looks very absurd at first glance.
Change-Id: Ib67b49605ef9ef1c84ecda1db16be74fc9105398
* changes:
util.hh: Delete remaining file and clean up headers
util.hh: Move nativeSystem to local-derivation-goal.cc
util.hh: Move stuff to types.hh
util.cc: Delete remaining file
util.{hh,cc}: Move ignoreException to error.{hh,cc}
util.{hh,cc}: Split out namespaces.{hh,cc}
util.{hh,cc}: Split out users.{hh,cc}
util.{hh,cc}: Split out strings.{hh,cc}
util.{hh,cc}: Split out unix-domain-socket.{hh,cc}
util.{hh,cc}: Split out child.{hh,cc}
util.{hh,cc}: Split out current-process.{hh,cc}
util.{hh,cc}: Split out processes.{hh,cc}
util.{hh,cc}: Split out file-descriptor.{hh,cc}
util.{hh,cc}: Split out file-system.{hh,cc}
util.{hh,cc}: Split out terminal.{hh,cc}
util.{hh,cc}: Split out environment-variables.{hh,cc}
Folding by default would prevent things like "Ctrl+F for nix-env" from
working trivially, but the user should be able to fold if they want to.
Change-Id: I5273272289f0f24e1f040c691580acfe33f66bd4
while refactoring the curl wrapper we inadvertently broken the immutable
flake protocol, because the immutable flake protocol accumulates headers
across the entire redirect chain instead of using only the headers given
in the final response of the chain. this is a problem because Some Known
Providers Of Flake Infrastructure set rel=immutable link headers only in
the penultimate entry of the redirect chain, and curl does not regard it
as worth returning to us via its response header enumeration mechanisms.
fixes lix-project/lix#358
Change-Id: I645c3932b465cde848bd6a3565925a1e3cbcdda0
Documents some of the weirdness of __curPos and the or keyword.
This does not fit well into any existing section for either of
them, though the use of or as a quasi-operator is mentioned in
the section on operators.
Addresses lix-project/lix#353
Change-Id: I7c906c8368843dca6944e8b22573b6d201cd9a76
grepQuietInvert is a typo introduced by c11836126b.
The test functional-timeout was failing silently because Bash considered
the command-not-found error as truthy.
Change-Id: Ic13829d02ec55d6ecd63a0f4d34ec0d32379609f
If a nested nix-shell is run inside a nix-shell, then the outer shell's
shellHook will be passed through and run again, unless the nested shell
defines its own.
With lix's hook, this can be annoying: forgetting to exit its nix-shell,
cd'ing to another repository & entering a nested nix-shell will happily
install lix's pre-commit hook in it.
This change makes lix's hook return early in such cases.
Change-Id: I91cb6eb6668f3a8eace36ecbdb01eb367861d77b
According to doc/manual/src/contributing/hacking.md, all development
shells should also be available in classic nix using the -A argument
to nix-shell, e.g.
nix-shell -A native-clangStdenvPackages
This was not actually the case; flake-compat generates attrsets like
nix-shell -A devShells.x86_64-linux.native-clangStdenvPackages
instead. These are unwieldy to use, so rather than changing the docs,
this changes the shell.nix file to result in the documented behavior.
Change-Id: I0920ccbdfddacf371f0aeaae7e290db65cf76ee7
check goals for timeouts first, and their activity fds only if no
timeout has occurred. checking for timeouts *after* activity sets
us up for assertion failures by running multiple build completion
notifiers, the first of which will kill/reap the the goal process
and consuming the Pid instance. when the second notifier attempts
to do the same it will core dump with an assertion failure in Pid
and take down not only the single goal, but the entire daemon and
all goals it was building. luckily this is rare in practice since
it requires a build to both finish and time out at the same time.
writing a test for this is not feasible due to how much it relies
on scheduling to actually trigger the underlying bug, but on idle
machines it can usually be triggered by running multiple sleeping
builds with timeout set to the sleep duration and `--keep-going`:
nix-build --timeout 10 --builders '' --keep-going -E '
with import <nixpkgs> {};
builtins.genList
(i: runCommand "foo-${toString i}" {} "sleep 10")
100
'
Change-Id: I394d36b2e5ffb909cf8a19977d569bbdb71cb67b
8c06b7b43¹ made libfetchers log the URL being fetched just before the
actual fetch, particularly in case something freezes. This used the base
URL, to not include query parameters, as the Nixpkgs lib tests assume
that stderr logs will be equal across shallow and non-shallow git
fetches (and shallow fetches have the ?shallow=1 query parameter).
8c06b7b43 assumed that the `base` field of ParsedURL would be populated,
as the comment simply says "URL without query/fragment"... but
apparently it is not populated when the URL being fetched is *already*
fetched, which caused libfetchers to log things like
fetching gitlab input ''
which is. silly. but you know, busted lix be busted.
Anyway, with this commit we just remove the query params before printing
instead, which seems to do the right thing
[1]: 8c06b7b431
Change-Id: I9b9988992029aa6abef786f20b66e68c2ebb97d4
The `builder` local variable and duplicate `args.push_back` are no
longer required since the Darwin sandbox stopped using `sandbox-exec`.
The `drv->isBuiltin` check is not required either, as args are not
accessed when the builder is builtin.
Change-Id: I80b939bbd6f727b01793809921810ff09b579d54
Seccomp filtering and the no-new-privileges functionality improve the security
of the sandbox, and have been enabled by default for a long time. In
lix-project/lix#265 it was decided that they
should be enabled unconditionally. Accordingly, remove the allow-new-privileges
(which had weird behavior anyway) and filter-syscall settings, and force the
security features on. Syscall filtering can still be enabled at build time to
support building on architectures libseccomp doesn't support.
Change-Id: Iedbfa18d720ae557dee07a24f69b2520f30119cb
`meson test` refuses to let `--verbose` (which shows the entire
invocation and stdio) override `--quiet`, but if neither are specified
in the justfile then you can use either `just test -q` or `just test -v`
Change-Id: I449e13084ce64666b7ee2ab4280818782fb8185a
Embarrassingly, I submitted a CL overriding submit requirements since
I thought it was spurious failures. However, the CI failure was in fact
real, and I have hopefully learned my lesson. The CI failure is that:
```
vm-test-run-nix-upgrade-nix> machine # installing 'nix-2.18.1'
vm-test-run-nix-upgrade-nix> machine # building '/nix/store/2b6fdf7wvahd00bg2ff0393bhd597a0h-user-environment.drv'...
vm-test-run-nix-upgrade-nix> machine # error: Unable to build profile. There is a conflict for the following files:
vm-test-run-nix-upgrade-nix> machine #
vm-test-run-nix-upgrade-nix> machine # /nix/store/dn6mhhr92bh3ad0n4pd1538ww88khjii-nix-2.18.1/lib/libboost_context.so
vm-test-run-nix-upgrade-nix> machine # /nix/store/w4vffn9iq0znk8bcg5i2giij90xy6db6-lix-2.90.0pre20240523_c97e171/lib/libboost_context.so
vm-test-run-nix-upgrade-nix> machine # error: builder for '/nix/store/2b6fdf7wvahd00bg2ff0393bhd597a0h-user-environment.drv' failed with exit code 1
vm-test-run-nix-upgrade-nix> machine # error: program '/nix/store/w4vffn9iq0znk8bcg5i2giij90xy6db6-lix-2.90.0pre20240523_c97e171/bin/nix-env' failed with exit code 100
```
This is definitely caused by the pname not being the same, so we had
better revert that part of the change until we know we won't regress
anything by doing this.
Fixes: https://gerrit.lix.systems/c/lix/+/1152/5
Change-Id: I0e9d573987f2819c106fb7cea87410fa75152274
* changes:
docs: linkify nix3-build mention in nix-build.md
build: make internal-api-docs PHONY
cleanup lookupFileArg
add docstring to lookupFileArg
add libcmd test for lookupFileArg
This breaks downstreams linking to us on purpose to make sure that if
someone is linking to Lix they're doing it on purpose and crucially not
mixing up Nix and Lix versions in compatibility code.
We still need to fix the internal includes to follow the same schema so
we can drop the single-level include system entirely. However, this
requires a little more effort.
This adds pkg-config for libfetchers and config.h.
Migration path:
expr.hh -> lix/libexpr/expr.hh
nix/config.h -> lix/config.h
To apply this migration automatically, remove all `<nix/>` from
includes, so: `#include <nix/expr.hh>` -> `#include <expr.hh>`. Then,
the correct paths will be resolved from the tangled mess, and the
clang-tidy automated fix will work.
Then run the following for out of tree projects:
```
lix_root=$HOME/lix
(cd $lix_root/clang-tidy && nix develop -c 'meson setup build && ninja -C build')
run-clang-tidy -checks='-*,lix-fixincludes' -load=$lix_root/clang-tidy/build/liblix-clang-tidy.so -p build/ -fix src
```
Related: lix-project/nix-eval-jobs#5
Fixes: lix-project/lix#279
Change-Id: I7498e903afa6850a731ef8ce77a70da6b2b46966
Since we're skipping Meson's dependency tracking, for the
internal-api-docs custom target, we should just consider it a phony
target and build it on every request.
Change-Id: I3b0bcea30ee9a4830023ccc5bededf995e96cccc
this should make it easier to spot future instances of entries being
duplicated by accident. also add a pre-commit check to remain sorted
Change-Id: I500caf862e93480b38c9d51144273bb2dcab1af0
File not found while importing causes a SysError, not an EvalError,
which is not currently caught by the tab-completion handler. Ignoring
all SysErrors might seem "dangerous" but this is the tab-completion
handler, any exception being bubbled up from there causes unexpected
behavior (causes the whole repl to exit).
Fixes#340.
Change-Id: I643048a47935e77f582decc539d9e51bdb96c890
nixpkgs has 23000 attributes, and our previous limit would be hit if you
have more than one nixpkgs in the environment, for example, because
`repl-overlays` will load the new stuff from the environment on top of
the existing environment.
This is not really testable since if we did write such a test, it would
just be testing this constant tbh...
Fixes: lix-project/lix#337
Change-Id: I49197bfb4db55b082f914f0d70e84f5f5f110954
Also fix typos introduced by the commits I read.
I have run the addDrvOutputDependencies release note past Ericson since
I was confused by what the heck it was doing, and he was saying it was
reasonable.
Change-Id: Id015353b00938682f7faae7de43df7f991a5237e
nix::fetchers::CacheImpl uses $XDG_CACHE_HOME, or its default based on
$HOME, to store its SQLite database. If the current process can't write
to that directory for whatever reason, though, any eval-time fetching
would fail just initializing the cache.
With this change, IO errors initializing the fetcher cache are logged
but ignored, and nix::fetchers::CacheImpl falls back to an in-memory¹
database instead.
Notably, this will fix any uses eval fetching while Lix itself is being
run in a derivation builder (such as during tests), as the derivation
builder does not set $XDG_CACHE_HOME, and sets $HOME to the non-existent
directory /homeless-shelter.
Before:
$ env -u XDG_CACHE_HOME HOME=/homeless-shelter nix -Lv eval --impure -E 'fetchTarball "https://git.lix.systems/lix-project/lix/archive/main.tar.gz"'
error:
… while calling the 'fetchTarball' builtin
at «string»:1:1:
1| fetchTarball "https://git.lix.systems/lix-project/lix/archive/main.tar.gz"
| ^
error: creating directory '/homeless-shelter': Permission denied
After:
$ env -u XDG_CACHE_HOME HOME=/homeless-shelter nix -Lv eval --impure -E 'fetchTarball "https://git.lix.systems/lix-project/lix/archive/main.tar.gz"'
warning: ignoring error initializing Lix fetcher cache: error: creating directory '/homeless-shelter': Permission denied
"/nix/store/s9lxdnn0awp37n560bg4fgr497ah4hvw-source"
¹: https://www.sqlite.org/inmemorydb.html
Change-Id: I15c38c9baaf215fc6e192b8a4c70b9692a69bc22
This turns errors like:
error: flake output attribute 'hydraJobs' is not a derivation or path
into errors like:
error: expected flake output attribute 'hydraJobs' to be a derivation or
path but found a set: { binaryTarball = «thunk»; build = «thunk»; etc> }
This change affects all InstallableFlake commands.
Change-Id: I899757af418b6f98201006ec6ee13a448c07077c
Passing the commit message as an argument causes update failures on repositories with
lots of flake inputs. In some cases, the commit message is over 250,000 bytes.
Upstream PR: https://github.com/NixOS/nix/pull/10686
(cherry picked from commit 8b5e8f4fba5728f2b3e90fcd1ab15df77e3ea0e8)
Change-Id: I2c196a21cc9bedc24d57a828a0c5b9467e072f76
Move the identical static `chmod_` functions in libstore to
libutil. the function is called `chmodPath` instead of `chmod`
as otherwise it will shadow the standard library chmod in the nix
namespace, which is somewhat confusing.
Change-Id: I7b5ce379c6c602e3d3a1bbc49dbb70b1ae8f7bad
2bbe3efd1¹ added the -Wdeprecated-copy warning, and fixed the instances
of it which GCC warned about, in HintFmt and ref<T>. However, when
building with Clang, there is an additional deprecated-copy warning in
BaseError. This commit explicitly defaults the copy assignment operator
for BaseError and silences this warning.
1: 2bbe3efd16
Change-Id: I50aa4a7ab1a7aae5d7b31f765994abd3db06379d
Fate has something different in store for the release process,
backporting process and the general maintainer documentation.
See lix-project/lix#260.
Change-Id: I626686ff4059aee22a3ab1664b52581b2dbf6ed7
Signed-off-by: Raito Bezarius <raito@lix.systems>
having the serializer write into `*conn` is not legal because we are
in a sinkToSource that will be drained by the remote we're connected
to. writing into `*conn` directly can break the framing protocol. it
is unlikely this code was ever run: to protocol it caters to is from
2016(!) and thoroughly untested in-tree, and since it's been present
since nix 2.17 and the 1.18 protocol broken here is nix 2.0 we might
safely assume that daemons older than nix 2.1 are no longer used now
see also #325 (though that wants <2.3 gone, this is sadly only <2.1)
Change-Id: I9d674c18f6d802f61c5d85dfd9608587b73e70a5
On several occasions I've found myself confused when trying to delete
a store path, because I am told it's still alive, but
nix-store --query --roots doesn't show anything. Let's save future
users this confusion by mentioning that a path might be alive due to
having referrers, not just roots.
(cherry picked from commit 979a019014569eee7d0071605f6ff500b544f6ac)
Upstream-PR: https://github.com/NixOS/nix/pull/10733
Change-Id: I54ae839a85f3de3393493fba27fd40d7d3af0516
These commands outputs data that may not end with a newline. This
causes problems when the progress bar redraws, as that completely
wipes the last line of output. As nix key generate-secret outputs
a single line of text with no output, it shows up entirely blank,
making it look like nothing happened.
Fixes: lix-project/lix#320
Change-Id: I5ac706d71d839b6dfa760b60a351414cd96297cf
Fixes#183, #110, #116.
The default flake-registry option becomes 'vendored', and refers
to a vendored flake-registry.json file in the install path.
Vendored copy of the flake-registry is from github:NixOS/flake-registry
at commit 9c69f7bd2363e71fe5cd7f608113290c7614dcdd.
Change-Id: I752b81c85ebeaab4e582ac01c239d69d65580f37
This should have been in there originally, which is our mistake,
considering that debugging CI failures is basically impossible without
it.
Change-Id: I4ab8799e6e0abca1984ed9801fe10c58200861a3
We don't need bear anymore, since we don't have any more bad build
systems that lack compile commands generation inside Lix.
Change-Id: I7809ddfd993180468f846e8cd862bdd54d5b31ec
Both of these still needs their own actual documentation, but they are
at least now mentioned that they exist and what they're enabled by.
Change-Id: I235b9e8e627e04ed06611423c8e67a8eca233120
Example: /nix/store/dr53sp25hyfsnzjpm8mh3r3y36vrw3ng-neovim-0.9.5^out
This is nonsensical since selecting outputs can only be done for a
buildable derivation, not for a realised store path. The build worker
side of things ends up crashing with an assertion when trying to handle
such malformed paths.
Change-Id: Ia3587c71fe3da5bea45d4e506e1be4dd62291ddf
This builtin was always a problem and nixpkgs uses it in exactly one
place, to give up if the Nix version is absurdly old. It has no other
use cases, and doesn't work in a multi-implementation world anyway.
Change-Id: I03c36e118591029e2ef14b091fe14a311c66a08a
There are a few places in nixpkgs lib where `**Foo**:` is used as a heading instead of the usual markdown `# Foo` ones. I think this is intentional with how it gets rendered in the manual, e.g. [`lib.lists.sortOn`][1].
[1]: https://nixos.org/manual/nixpkgs/stable/#function-library-lib.lists.sortOn
`nix-doc` prints this as
```
*Laws**:
```nix
sortOn f == sort (p: q: f p < f q)
```
```
chomping off the first asterisk as part of `cleanup_single_line` that's meant to deal with `/** \n * \n * \n */` style doc comments. This also means the usage in lix ends up funny-looking with a trailing asterisk as if there's a footnote to pay attention to (which is how I first noticed it, heh)
The fix:
When cleaning up a single line and removing a prefix comment character,
ensure it's followed by whitespace (or the last character of the line).
Upstream-PR: https://github.com/lf-/nix-doc/pull/26
Change-Id: If2870c53a632f6bbbcca98a4bfbd72f5bef37879
Surely if you have unreleased changes you want them on a page right?
`officialRelease` means "this is a *release version*", which is a
reasonable case to not want it, but we are not that here.
I understand wanting to be able to turn it off for deps reasons or
something, but other than that, uhh, seems better to just turn it on
always; it is basically free compute-wise to the point we run it on
pre-commit.
Part two of fixing lix#297.
Fixes: lix-project/lix#297
Change-Id: I0f8dd1ae42458df371aef529c456e47a7ac04ae0
This allows us to have links to peoples' GitHub and Forgejo profiles.
I used YAML because I don't want to introduce a dependency on having a
working Nix evaluator to be able to build release notes, and we already
have a YAML parser in this script.
Change-Id: Idf2813f79e0407460c796cba6c383496465e152d
This does not add missing release notes, and it doesn't do anything
about the profiles feature we would really like to have so we can have
consistent credit.
Change-Id: I72a6f7acfcff85f380be17dac76501a6f4693776
This was a combination of two problems: the python didn't throw an error
because apparently glob on a nonexistent directory doesn't crash, and
secondarily, bash ignores bad exit codes without `set -e` if they are
not in the final/only command.
Change-Id: I812bde7a4daee5c77ffe9d7c73a25fd14969f548
Use the correct directory for the rl-next build, so that the release notes
actually get built and the page doesn't end up empty. I don't know why the
exception didn't cause a build failure before.
Fixes: lix-project/lix#297
Change-Id: Ic72b9bb4c0d2d1f633f2af90cce4a3a2796d7f9b
Basically I'd expect the same behavior as with `nix-build`, i.e.
with `--keep-going` the hash-mismatch error of each failing
fixed-output derivation is shown.
The approach is derived from `Store::buildPaths` (`entry-point.cc`):
instead of throwing the first build-result, check if there are any build
errors and if so, display all of them and throw after that.
Unfortunately, the BuildResult struct doesn't have an `ErrorInfo`
(there's a FIXME for that at least), so I have to construct my own here.
This is a rather cheap bugfix and I decided against touching too many
parts of libstore for that (also I don't know if that's in line with the
ongoing refactoring work).
Closes lix-project/lix#302
Change-Id: I378ab984fa271e6808c6897c45e0f070eb4c6fac
Now, we can credit folks for their work.
The credit generator is very basic, we probably want a database of
profiles and link to their preferred page or something.
Change-Id: Ida81905750371e5e125d0ce7e554d0526265cf8e
Co-Authored-By: Jade Lovelace <lix@jade.fyi>
Signed-off-by: Raito Bezarius <raito@lix.systems>
This is not like, perfect, since it is a manual operation, but we can
automate it in the future. rclone is used, since it seems like awscli is
not (obviously at least?) able to sync directories such that old things
are deleted, and rclone does this thing properly.
Fixes: lix-project/meta#2
Change-Id: Ia6a46d861342a6d29b22f981ba4e35e79f79e60e
Otherwise, it will be thrown again during exit when the repl is terminated by
end-of-input after the last command was interrupted.
Change-Id: I8456c47bc36cfb0892efdad5420f318f7e6526d5
The interrupt-blocking code was originally introduced 20 years ago so that
trying to log an error message does not result in an interrupt exception being
thrown and then going unhandled (c8d3882cdc).
However, the logging code does not check for interrupts any more
(054be50257), so this reasoning is no longer
applicable. Delete this code so that later interrupts are unblocked again, for
example in the next line entered into the repl.
Closes: lix-project/lix#296
Change-Id: I48253f5f4272e75001148c13046e709ef5427fbd
Very basic behavior test to ensure that gzip data gets internally
decompressed by the file transfer pipeline.
Change a std::string_view return value in the test harness to
std::string. I wouldn't call myself a C++ beginner and I still managed
to shoot myself in the foot like three times with the lifetime
managements there (e.g. [&] { return an_std_string; } ends up with a
dangling string_view!).
Change-Id: I1360750d4181ce1ca2a3aa4dc0e97e131351c469
it's no longer used. it really shouldn't have existed this long since it
was just a mashup of both std::promise and std::packaged_task in a shape
that makes composition unnecessarily difficult. all but a single case of
Callback pattern calls were fully synchronous anyway, and even this sole
outlier was by far not important enough to justify the extra complexity.
Change-Id: I208aec4572bf2501cdbd0f331f27d505fca3a62f
also add a few more tests for exception propagation behavior. using
packaged_tasks and futures (which only allow a single call to a few
of their methods) introduces error paths that weren't there before.
Change-Id: I42ca5236f156fefec17df972f6e9be45989cf805
this is the *only* real user of file transfer download completion
callbacks, and a pretty spurious user at that (seeing how nothing
here is even turned on by default and indeed a dependency of path
substitution which *isn't* async, and concurrency-limited). it'll
be a real pain to keep this around, and realistically it would be
a lot better to overhaul substitution in general to be *actually*
async. that requires a proper async framework footing though, and
we don't have anything of the sort, but it's also blocking *that*
Change-Id: I1bf671f217c654a67377087607bf608728cbfc83
The fix for the Darwin vulnerability in ecdbc3b207
also broke setting `__sandboxProfile` when `sandbox=relaxed` or
`sandbox=false`. This cppnix change fixes `sandbox=relaxed` and
adds a suitable test.
Co-Authored-By: Artemis Tosini <lix@artem.ist>
Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
Change-Id: I40190f44f3e1d61846df1c7b89677c20a1488522
In commit 946fc12e4e, the progress bar in the
repl was disabled again because it was observed to erase incremental output
from attrset evaluations from the terminal. Let's try adding the progress bar
again, this time showing up only when a build is initiated, which does not have
incremental output that could be destroyed to begin with. While this does mean
that we won't have a progress bar for eval-time fetching or IFD, it's still
better than nothing.
Change-Id: If4eb1035cd0c876f5b4ff1e2434b9baf99f150ac
Now instead of a derivation overridden from Lix, we use a mkShell
derivation parameterized on an already called package.nix. This also
lets callPackage take care of the buildPackages distinction for the
devShell.
Change-Id: I5ddfec40d83fa6136032da7606fe6d3d5014ef42
* changes:
flake: fix devShell on i686-linux by disabling ClangBuildAnalyzer on it
flake: fix eval of checks & devshell on i686-linux
flake: move the pre-commit definition to its own file
ClangBuildAnalyzer doesn't build on i686-linux due to
`long long int`/`size_t` conversion errors, so let's just exclude it
from the devshell on that platform
Change-Id: If1077a7b3860db4381999c8e304f6d4b2bc96a05
Because of an objc quirk[1], calling curl_global_init for the first time
after fork() will always result in a crash.
Up until now the solution has been to set
OBJC_DISABLE_INITIALIZE_FORK_SAFETY for every nix process to ignore
that error.
This is less than ideal because we were setting it in package.nix,
which meant that running nix tests locally would fail because
that variable was not set.
Instead of working around that error we address it at the core -
by calling curl_global_init inside initLibStore, which should mean
curl will already have been initialized by the time we try to do so in
a forked process.
[1] 01edf1705f/runtime/objc-initialize.mm (L614-L636)
Change-Id: Icf26010a8be655127cc130efb9c77b603a6660d0
only two users of this function exist. only one used it in a way that
even bears resemblance to asynchronicity, and even that one didn't do
it right. fully async and parallel computation would have only worked
if any getEdgesAsync never calls the continuation it receives itself,
only from more derived callbacks running on other threads. calling it
directly would cause the decoupling promise to be awaited immediately
*on the original thread*, completely negating all nice async effects.
Change-Id: I0aa640950cf327533a32dee410105efdabb448df
this seems to be an oversight, considering that regular substitutions
are concurrency-limited. while not particularly necessary at present,
once we've removed the `Callback` based interfaces it will be needed.
Change-Id: Ide2d08169fcc24752cbd07a1d33fb8482f7034f5
When /nix/var (or, more precisely, NIX_STATE_DIR) does not exist at all,
Lix falls back to creating an adhoc chroot store in XDG_DATA_HOME.
b247ef72d[1] changed the way Store classes are initialized, and in the
migration, a `params2` was accidentally changed to `params`. This commit
restores the correct behavior, and in lieu of a single *character* fix,
this commit also changes the variable name to something more reasonable.
Fixes#274.
[1]: b247ef72dc
n.b., this code might deserve some more looking at anyway. this fallback
store creation throws away *all* Store params passed to
openFromNonUri() in favor of an entirely new set which only contains
the `root` param, which may or may not be the correct behavior
Change-Id: Ibea559b88a50e6d6e75a1f87d9d7816cabb2a8f3
As per our bootstrap governance discussions, here's a very simple
proposal which links as much as possible to our wiki.
Change-Id: I88b1c43f933ff7e529151b1e933fad40283383c4
Signed-off-by: Raito Bezarius <raito@lix.systems>
returning 0 from the callback for errors signals successful transfer if
the source returned no data even though the exception we've just caught
clearly disagrees. while this is not all that important (since the only
viable cause of such errors will be dataCallback, and the sole instance
of it being used already takes care of exceptions) we can just do this.
Change-Id: I2bb150eff447121d82e8e3aa4e00057c40523ac6
this will be necessary if we want download() to return a source instead
of consuming a sink, which will in turn be needed to remove coroutines.
Change-Id: I34ec241e9bbc5d32fbcd243b244e29c3757533aa
Some tests were failing on darwin,
if the auto-allocate-uids featrure was enabled.
This was because AAU on darwin works by setuid-ing as a non-existent
user, so the tests that were relying on `whoami` were failing.
In the case of trusted-users we fall back to printing the user id,
which is already handled gracefully in the daemon code - i.e. when
a user does not exist or for some other reason looking up their
username is not possible, the daemon falls back to searching for their
uid inside the trusted-users list.
When whoami is used to print the username for other purpose,
we default to printing nixbld.
Change-Id: Ib61615677565098cb5fbf5e26a946ef427c58caf
This doesn't comprehensively fix everything outdated in the manual, or
make the manual greatly better, but it does note down where at least
jade noticed it was wrong, and it does fix all the instances of
referencing Nix to conform to the style guide to the best of our
ability.
A lot of things have been commented out for being wrong, and there are
three types of FIXME introduced:
- FIXME(Lix): generically Lix needs to fix it
- FIXME(Qyriad): re lix-project/lix#215
- FIXME(meson): docs got outdated by meson changes and need rewriting
I did fix a bunch of it that I could, but there could certainly be
mistakes and this is definitely just an incremental improvement.
Fixes: lix-project/lix#266
Change-Id: I5993c4603d7f026a887089fce77db08394362135
sizeof(long) is 4 bytes on i686 GCC.
With ~32 GiB of memory and a page size of 4096, there are 7988420 pages.
(7988420 * 4096) is bigger than INT32_MAX folks.
This has gone unnoticed for 9 years, and only came up thanks to
94ea517db[1] adding integer overflow sensitization checks, which caused
this broken code to emit an illegal instruction, crashing Lix the
instant the buildsystem ran Lix to generate the docs files.
[1]: 94ea517dbe729765b69638190f4bea3f6a632b40
Change-Id: I50bb9ea072aac11b449d79e5d55525887a6e5a99
not doing this will cause transfers that had their readers disappear to
linger. with lingering transfers the curl thread can't shut down, which
will cause nix itself to not shut down until the transfer finishes some
other way (most likely network timeouts). also add a new test for this.
Change-Id: Id2401b3ac85731c824db05918d4079125be25b57
mdbook has the unfortunate habit of creating stub files for chapters it
can't find on disk. turn off this helpful feature as it masks errors in
the summary file, and fix a recently introduced instance of this error.
Change-Id: I10d86aac0489c9c494bd5c8a50047415f4d4b18d
If unprivileged userns are *believed* to be disabled (such as with
"kernel.unprivileged_userns_clone = 0"), Lix would previously *give up*
on trying to use a user namespace before actually trying it, even if, in
cases such as unprivileged_userns_clone, it would actually be allowed
since Nix has CAP_SYS_ADMIN when running as daemon.
(see, e.g. 25d4709a4f)
We changed it to actually try it first, and then diagnose possible
causes, and also to be more loud about the whole thing, using warnings
instead of debugs. These warnings will only print on the first build run
by the daemon, which is, tbh, eh, shrug.
This is what led to us realizing that no-userns was a poorly exercised
condition.
Change-Id: I8e4f21afc89c574020dc7e89a560cc740ce6573a
This was found when `logrotate.conf` failed to build in a NixOS system
with:
/nix/store/26zdl4pyw5qazppj8if5lm8bjzxlc07l-coreutils-9.3/bin/id: cannot find name for group ID 30000
This was surprising because it seemed to mean that /etc/group was busted
in the sandbox. Indeed it was:
root❌0:
nixbld:!💯
nogroup❌65534:
We diagnosed this to sandboxUid() being called before
usingUserNamespace() was called, in setting up /etc/group inside the
sandbox. This code desperately needs refactoring.
We also moved the /etc/group code to be with the /etc/passwd code, but
honestly this code is all spaghetti'd all over the place and needs some
more serious tidying than we did here.
We also moved some checks to be earlier to improve locality with where
the things they are checking come from.
Change-Id: Ie29798771f3593c46ec313a32960fa955054aceb
This has the following downsides:
* you cannot build Lix against nixos-unstable.
* this will immediately break as soon as libseccomp will hit
nixos-23.11 (given that people will probably use the package.nix via
our overlay or override nixpkgs via `follows`).
Hence, removing the assert again and add a better FIXME comment.
Change-Id: I284e10cf08e1873fef70ed869a1638aa89792422
This reverts commit a8b3d777fb.
This undoes the revert of PR#6621, which allows nested `follows`, i.e.
{
inputs = {
foo.url = "github:bar/foo";
foo.inputs.bar.inputs.nixpkgs = "nixpkgs";
};
}
does the expected thing now. This is useful to avoid the 1000 instances
of nixpkgs problem without having each flake in the dependency tree to
expose all of its transitive dependencies for modification.
This was in fact part of Nix before and the C++ changes applied w/o
conflicts. However, it got reverted then because people didn't want to
merge lazy-trees against it which was supposed to be merged soon back in
October 2022.
Fixes: lix-project/lix#201
Change-Id: I5ddef914135b695717b2ef88862d57ced5e7aa3c
With Linux kernel >=6.6 & glibc 2.39 a `fchmodat2(2)` is available that
isn't filtered away by the libseccomp sandbox.
Being able to use this to bypass that restriction has surprising results
for some builds such as lxc[1]:
> With kernel ≥6.6 and glibc 2.39, lxc's install phase uses fchmodat2,
> which slips through 9b88e52846/src/libstore/build/local-derivation-goal.cc (L1650-L1663).
> The fixupPhase then uses fchmodat, which fails.
> With older kernel or glibc, setting the suid bit fails in the
> install phase, which is not treated as fatal, and then the
> fixup phase does not try to set it again.
Please note that there are still ways to bypass this sandbox[2] and this is
mostly a fix for the breaking builds.
This change works by creating a syscall filter for the `fchmodat2`
syscall (number 452 on most systems). The problem is that glibc 2.39
is needed to have the correct syscall number available via
`__NR_fchmodat2` / `__SNR_fchmodat2`, but this flake is still on
nixpkgs 23.11. To have this change everywhere and not dependent on the
glibc this package is built against, I added a header
"fchmodat2-compat.hh" that sets the syscall number based on the
architecture. On most platforms its 452 according to glibc with a few
exceptions:
$ rg --pcre2 'define __NR_fchmodat2 (?!452)'
sysdeps/unix/sysv/linux/x86_64/x32/arch-syscall.h
58:#define __NR_fchmodat2 1073742276
sysdeps/unix/sysv/linux/mips/mips64/n32/arch-syscall.h
67:#define __NR_fchmodat2 6452
sysdeps/unix/sysv/linux/mips/mips64/n64/arch-syscall.h
62:#define __NR_fchmodat2 5452
sysdeps/unix/sysv/linux/mips/mips32/arch-syscall.h
70:#define __NR_fchmodat2 4452
sysdeps/unix/sysv/linux/alpha/arch-syscall.h
59:#define __NR_fchmodat2 562
I added a small regression-test to the setuid integration-test that
attempts to set the suid bit on a file using the fchmodat2 syscall.
I confirmed that the test fails without the change in
local-derivation-goal.
Additionally, we require libseccomp 2.5.5 or greater now: as it turns
out, libseccomp maintains an internal syscall table and
validates each rule against it. This means that when using libseccomp
2.5.4 or older, one may pass `452` as syscall number against it, but
since it doesn't exist in the internal structure, `libseccomp` will refuse
to create a filter for that. This happens with nixpkgs-23.11, i.e. on
stable NixOS and when building Lix against the project's flake.
To work around that
* a backport of libseccomp 2.5.5 on upstream nixpkgs has been
scheduled[3].
* the package now uses libseccomp 2.5.5 on its own already. This is to
provide a quick fix since the correct fix for 23.11 is still a staging cycle
away.
We still need the compat header though since `SCMP_SYS(fchmodat2)`
internally transforms this into `__SNR_fchmodat2` which points to
`__NR_fchmodat2` from glibc 2.39, so it wouldn't build on glibc 2.38.
The updated syscall table from libseccomp 2.5.5 is NOT used for that
step, but used later, so we need both, our compat header and their
syscall table 🤷
Relevant PRs in CppNix:
* https://github.com/NixOS/nix/pull/10591
* https://github.com/NixOS/nix/pull/10501
[1] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2031073804
[2] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2030844251
[3] https://github.com/NixOS/nixpkgs/pull/306070
(cherry picked from commit ba6804518772e6afb403dd55478365d4b863c854)
Change-Id: I6921ab5a363188c6bff617750d00bb517276b7fe
Part of #7672
My main motivation is to be able to use `nix.checkConfig`[1]. This
doesn't work with Lix currently since the module uses `nix show-config`
if the Nix version is <2.20pre and `nix config show` otherwise. I think
this is the only instance where nixpkgs checks for which Nix commands
exist that affects us now, so I figured we could just perform the rename
here as well[2] and still provide the current version number[3].
I don't have a strong opinion on whether to deprecate `nix show-config`,
the warning is added there automatically.
(cherry picked from commit f300e11b056dea414d7d77bbc6e5a7dc5d9ddd41)
[1] https://nixos.org/manual/nixos/stable/options.html#opt-nix.checkConfig
[2] I should add that I don't use the "official" ways of installing Lix
because using the flake directly and callPackaging it seemed to fit
better into my workflow: I already have a little mess to make
sure Hydra from the flake uses the correct pkgs.nix and I didn't
want to complicate it further while keeping a single package-set I
can build in CI. Don't get me wrong, I think such a module for a
quick-start is very important, just giving context on why I bother
in the first place :)
[3] When we go public, I think it's worth considering to add support in
nixpkgs itself for Lix.
Change-Id: I47b4239b05cbeda3c370d2fa56ea768b768768ac
* changes:
docs: clarify how ^ works for -E/-f installables
docs: give translation examples from nix-build -E/-A to installables
docs: clarify how the different kinds of installables are selected
docs: guide to installables docs in installable commands' docs
Currently LocalDerivationGoal allows setting `__sandboxProfile`
to add sandbox parameters on Darwin when `sandbox=true`.
This was only supposed to have an effect when `sandbox=relaxed`
Change-Id: Ide44ee82d7e4d6b545285eab26547e7014817d3f
this is used in CA rewriting, replacement of placeholders in
derivations, generating scripts for devShells, and some more
places. in all of these transitive replacements are unsound,
and overlapping replacements would be as well. there even is
a test that transitive replacements do not happen (in the CA
RewriteSink suite), but none for overlapping replacements. a
minimally surprising binary rewriter surely would not do any
of these replacements, the only reason we have not seen this
break yet is probably that rewriteStrings is only called for
store paths and things that look like store paths (and those
should never overlap nor admit such transitive replacements)
Change-Id: I6fc29f939d5061d9f56c752624a823ece8437c07
* changes:
nix3-profile: remove check "name" attr in manifests
Add profile migration test
nix3-profile: make element names stable
getNameFromURL(): Support uppercase characters in attribute names
nix3-profile: remove indices
nix3-profile: allow using human-readable names to select packages
implement parsing human-readable names from URLs
As discussed in the maintainer meeting on 2024-01-29.
Mainly this is to avoid a situation where the name is parsed and
treated as a file name, mostly to protect users.
.-* and ..-* are also considered invalid because they might strip
on that separator to remove versions. Doesn't really work, but that's
what we decided, and I won't argue with it, because .-* probably
doesn't seem to have a real world application anyway.
We do still permit a 1-character name that's just "-", which still
poses a similar risk in such a situation. We can't start disallowing
trailing -, because a non-zero number of users will need it and we've
seen how annoying and painful such a change is.
What matters most is preventing a situation where . or .. can be
injected, and to just get this done.
(cherry picked from commit f1b4663805a9dbcb1ace64ec110092d17c9155e0)
Change-Id: I900a8509933cee662f888c3c76fa8986b0058839
Gen::just is the constant generator. Don't just return that!
(cherry picked from commit 8406da28773f050e00a006e4812e3ecbf919a2a9)
Change-Id: Ibfd0bd40f90942077a4720086ce0cd3bfabef79d
Gen: :just is the constant generator. Don't just return that!
(cherry picked from commit 69bbd5852af9b2f0b794162bd1debcdf64fc6648)
Change-Id: Id6e58141f5a42a1f67bd11d48c87b32a3ebd0500
We didn't even realize you *could* use this syntax with -E and -f, much
less that the attribute path could be *empty*.
Change-Id: Id1a6715609f3a76a5ce477bd43a7832effbbe07b
The installables syntax is not documented in any of the man pages or
docbook pages for any of those individual commands. And while these
commands really should at least peripherally individually document how
installables work, in the meantime we can at least direct people to the
right place.
This commit also clarifies the unexpected fact that `nix profile remove`
and `nix profile upgrade` do *not* take installables.
Change-Id: I3b1453cb197a613bbab639c66a466365c3592c6d
This commit adds a new NixOS VM test, which tests that `nix upgrade-nix`
works on both kinds of profiles (manifest.nix and manifest.json).
Done as a separate commit from 831d18a13, since it relies on the
--store-path argument from 026c90e5f as well.
Change-Id: I5fc94b751d252862cb6cffb541a4c072faad9f3b
nix3-profile automatically migrates any profile its used on to its style
of profile -- the ones with manifest.json instead of manifest.nix. On
non-NixOS systems, Nix is conventionally installed to the profile at
/nix/var/nix/profiles/default, so if a user passed that to `--profile`
of `nix profile`, then it would break upgrade-nix from ever working
again, without recreating the profile.
This commit fixes that, and allows upgrade-nix to work on either kind of
profile.
Fixes#16.
Change-Id: I4c49b1beba93bb50e8f8a107edc451affe08c3f7
Notably, ProfileManifest and ProfileElement are useful generic
profile management code, and nix profile is not the only place in the
codebase where profiles are relevant.
This commit is in preparation for fixing upgrade-nix's interaction with
new-style profiles.
Change-Id: Iefc8bbd34b4bc6012175cb3d6e6a8207973bc792
This replaces the external sandbox-exec call with direct calls into
libsandbox. This API is technically deprecated and is missing some
prototypes, but all major browsers depend on it, so it is unlikely to
materially change without warning.
This commit also ensures the netrc file is only written if the
derivation is in fact meant to be able to access the internet.
This change commits a sin of not actually actively declaring its
dependency on macOS's libsandbox.dylib; this is due to the dylib
cache in macOS making that explicit dependency unnecessary. In the
future this might become a problem, so this commit marks our sins.
Co-authored-by: Artemis Tosini <lix@artem.ist>
Co-authored-by: Lunaphied <lunaphied@lunaphied.me>
Change-Id: Ia302141a53ce7b0327c1aad86a117b6645fe1189
This should have been there from the beginning. As much as nix-env is a
pile of problems we don't need trivial docs papercuts like this adding
to it.
Change-Id: I0c53e4b146af2fefdd0e4743d850672729cb2194
That's expected by `build-remote` and makes sure that errors are
correctly forwarded to the user. For instance, let's say that the
host-key of `example.org` is unknown and
nix-build ../nixpkgs -A hello -j0 --builders 'ssh-ng://example.org'
is issued, then you get the following output:
cannot build on 'ssh-ng://example.org?&': error: failed to start SSH connection to 'example.org'
Failed to find a machine for remote build!
derivation: yh46gakxq3kchrbihwxvpn5bmadcw90b-hello-2.12.1.drv
required (system, features): (x86_64-linux, [])
2 available machines:
[...]
The relevant information (`Host key verification failed`) ends up in the
daemon's log, but that's not very obvious considering that the daemon
isn't very chatty normally.
This can be fixed - the same way as its done for legacy-ssh - by passing
fd 4 to the SSH wrapper. Now you'd get the following error:
cannot build on 'ssh-ng://example.org': error: failed to start SSH connection to 'example.org': Host key verification failed.
Failed to find a machine for remote build!
[...]
...and now it's clear what's wrong.
Please note that this is won't end up in the derivation's log.
For previous discussion about this change see
https://github.com/NixOS/nix/pull/7659.
Change-Id: I5790856dbf58e53ea3e63238b015ea06c347cf92
only decompress the response once all data has been received (in the
fully buffered case), or at least outside of the curl wrapper itself
(in the receive-to-sink case). unfortunately this means we will have
to duplicate decompression logic for these two cases for time being,
but once the curl wrapper has been rewritten to return a real future
or Source we can deduplicate this logic again. the curl wrapper will
have to turn into a proper Source first and use decompression source
logic which also does not currently exist—only decompression *sinks*
Change-Id: I66bc692f07d9b9e69fe10689ee73a2de8d65e35c
this is highly questionable. single-arg download calls will misbehave
with it set, and two-arg download calls will just overwrite it. being
an implementation detail this should not have been in the API at all.
Change-Id: I613772951ee03d8302366085f06a53601d13f132
this lets each implementation of FileTransfer (of which currently only
the one exists at all) implement appropriate handling for its internal
behaviours that are not otherwise exposed. in curl this lets us switch
the buffer-full handling method from "block the entire curl thread" to
"pause just the one transfer", move the non-libcurl body decompression
out of the actual curl wrapper (which will let us eventually morph the
curl wrapper intto an actual source of Sources), and some other things
Change-Id: Id6d3593cde6b4915aab3e90a43b175c103cc3f18
Previously, the garbage collector found runtime roots on Darwin by
shelling out to `lsof -n -w -F n` then parsing the result.
However, this requires an lsof binary and can be extremely slow.
The official Apple lsof returns in a reasonable amount of time,
about 250ms in my tests, but the lsof packaged in nixpkgs is quite slow,
taking about 40 seconds to run the command.
Using libproc directly is about the same speed as Apple lsof,
and allows us to reënable several tests that were disabled on Darwin.
Change-Id: Ifa0adda7984e13c15535693baba835aae79a3577
My main motivation for this change is to limit the amount of compile
jobs to make sure my machine is still usable for something else when
building a fresh Lix locally.
Also made `build` a dependency of `install`: this is analogous to
`make install` in CppNix where this both recompiles changed files and
installs the artifacts into `outputs/out`. May be a little more pleasant
to work with that, especially when you're used to contributing to
CppNix.
Change-Id: I321e2b0daf1c5e20f82c04e2dd158056c80ed86c
just accumulate error data into result.data as we would for successful
transfers without a dataCallback. errorSink and data would contain the
same data in error cases anyway, so splitting them is not very useful.
Change-Id: I00e449866454389ac6a564ab411c903fd357dabf
Meson cross files layer, the last value of each key takes effect.
https: //mesonbuild.com/Machine-files.html#loading-multiple-machine-files
Change-Id: I22d886f71cd51f0ce520d3fc22aed4bcf074bb91
This creates new subclasses of LocalStore for each OS to include
platform-specific functionality. Currently this just includes garbage
collector roots but it could be extended to sandboxing as well.
In order to make sure that the generic LocalStore is not accidentally
constructed, its constructor is protected. A Fallback is provided which
implements no functionality except constructors.
Change-Id: I836a28e90b68309873f75afb83e0f1b2e2c89fb3
This commit makes Meson the default buildsystem for Lix.
The Make buildsystem is now deprecated and will be removed soon, but has
not yet, which will be done in a later commit when all seems good. The
mesonBuild jobs have been removed, and have not been replaced with
equivalent jobs to ensure the Make buildsystem still works.
The full, new commands in a development shell are:
$ meson setup ./build "--prefix=$out" $mesonFlags
(A simple `meson setup ./build` will also build, but will do a different
thing, not having the settings from package.nix applied.)
$ meson compile -C build
$ meson test -C build --suite=check
$ meson install -C build
$ meson test -C build --suite=installcheck
(Check and installcheck may both be done after install, allowing you to
omit the --suite argument entirely, but this is the order package.nix
runs them in.)
If tests fail and Meson helpfully has no output for why, use the
`--print-error-logs` option to `meson test`. Why this is not the default
I cannot explain.
If you change a setting in the buildsystem, most cases will
automatically regenerate the Meson configuration, but some cases, like
trying to build a specific target whose name is new to the buildsystem
(e.g. `meson compile -C build src/libmelt/libmelt.dylib`, when
`libmelt.dylib` did not exist as a target the last time the buildsystem
was generated), then you can reconfigure using new settings but
existing options, and only recompiling stuff affected by the changes:
$ meson setup --reconfigure build
Note that changes to the default values in `meson.options` or in the
`default_options :` argument to project() are NOT propagated with
`--reconfigure`.
If you want a totally clean build, you can use:
$ meson setup --wipe build
That will work regardless of if `./build` exists or not.
Specific, named targets may be addressed in
`meson build -C build <target>` with the "target ID" if there is one,
which is the first string argument passed to target functions that
have one, and unrelated to the variable name, e.g.:
libexpr_dylib = library('nixexpr', …)
can be addressed with:
$ meson compile -C build nixexpr
All targets may be addressed as their output, relative to the build
directory, e.g.:
$ meson compile -C build src/libexpr/libnixexpr.so
But Meson does not consider intermediate files like object files
targets. To build a specific object file, use Ninja directly and
specify the output file relative to the build directory:
$ ninja -C build src/libexpr/libnixexpr.so.p/nixexpr.cc.o
To inspect the canonical source of truth on what the state of the
buildsystem configuration is, use:
$ meson introspect
Have fun!
Change-Id: Ia3e7b1e6fae26daf3162e655b4ded611a5cd57ad
This should fix cross compilation in the base case, but this is
difficult to test as cross compilation is broken in many different
places right now. This should bring Meson back up to cross parity with
the Make buildsystem though.
Change-Id: If09be8142d1fc975a82b994143ff35be1297dad8
don't reimplement header parsing. this was only really needed due to the
ancient github bug we no longer care about, everything else we have done
in custom code can also be done using curl itself. doing this also fixes
possible sources of header smuggling (because the header function didn't
unfold headers and we'd trim them before parsing, which would've made us
read contents of one header as a fully formed header in itself). this is
a slight behavior change because we now honor only the first instance of
a given header where previous behavior was to honor either the last or a
combination of all of them (accept-ranges was logical-or'd by accident).
Change-Id: I93cb93ddb91ab98c8991f846014926f6ef039fdb
this was a workaround for a *github* bug that happend *in 2015*.
not only is github no longer buggy, it shouldn't have been nix's
responsibility to work around these bugs like this to begin with
while we're at it we'll also remove another workaround—again for
github specifically and again for etag handling—from 2021 that's
also not needed any more. future workarounds for serverside bugs
should probably come with an expiration date that mutates into a
build warning after a while, otherwise this *will* happen again.
Change-Id: I74f739ae3e36d40350f78bebcb5869aa8cc9adcd
In hopes of avoiding opaque error messages like the one in
https://buildbot.lix.systems/#/builders/49/builds/1054/steps/1/logs/stdio
Traceback (most recent call last):
File "/nix/store/wj6wh89jhd2492r781qsr09r9wydfs6m-nixos-test-driver-1.1/bin/.nixos-test-driver-wrapped", line 9, in <module>
sys.exit(main())
^^^^^^
File "/nix/store/wj6wh89jhd2492r781qsr09r9wydfs6m-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/__init__.py", line 126, in main
driver.run_tests()
File "/nix/store/wj6wh89jhd2492r781qsr09r9wydfs6m-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/driver.py", line 159, in run_tests
self.test_script()
File "/nix/store/wj6wh89jhd2492r781qsr09r9wydfs6m-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/driver.py", line 151, in test_script
exec(self.tests, symbols, None)
File "<string>", line 13, in <module>
AssertionError
Change-Id: Idd2212a1c3714ce58c7c3a9f34c2ca4313eb6d55
the previous solution to the wakeup problem (adding a pipe and passing
it as an additional fd to curl_multi_wait) worked, but there have been
builtin alternatives for this since 2020. not only do these save code,
they're also a lot more likely to work natively on windows when needed
Change-Id: Iab751b900997110a8d15de45ea3ab0c42f7e5973
the oldest version checked for here is 7.47, which was released in
2016. it's probably safe to say that we do not need these any more
Change-Id: I003411f6b2ce6d56f7ca337390df3ea86bd59a99
With Nix 2.3, it was possible to pass a subpath of a store path to
exportReferencesGraph:
with import <nixpkgs> {};
let
hello = writeShellScriptBin "hello" ''
echo ${toString builtins.currentTime}
'';
in
writeClosure [ "${hello}/bin/hello" ]
This regressed with Nix 2.4, with a very confusing error message, that
presumably indicates it was unintentional:
error: path '/nix/store/3gl7kgjr4pwf03f0x70dgx9ln3bhl7zc-hello/bin/hello' is not in the Nix store
(cherry picked from commit 0774e8ba33c060f56bad3ff696796028249e915a)
Change-Id: I00920fb33077b831a1bb4a1b68d515ba8c3c2a69
The statically embedded busybox is not required for Lix to work, but
package.nix explicitly sets this, which was accidentally being ignored.
Change-Id: Ieeff830ac7d1f5fabe84d1a6cfd82f13d79035bf
This commit adds the capability for building the Doxygen internal API
docs in the Meson buildsystem, and also makes doing so the default for
the internal-api-docs hydra job. Aside from the /nix-support directory,
which differed only by the hash part of a store path, the outputs of
hydraJobs.internal-api-docs before and after this commit were
bit-for-bit identical on my machine.
Change-Id: I98f0017891c25b06866c15f7652fe74f706ec8e1
Either the contents of `line` could cause format errors, or this usage
is Technically safe. However, I trust nothing, especially with
boost::format.
Change-Id: I07933b20bde3b305a6e5d61c2a7bab6ecb042ad9
Previously if isStorePath() was called on anything other than a
top-level /nix/store/some-path, it would throw a BadStorePath exception.
This commit duplicates the absolutely trivial check, into
maybeParseStorePath(), and leaves exception throwing to
parseStorePath(), the function that assumes you're already giving a
valid path instead of the one whose purpose is to check if its valid or
not...
Change-Id: I8dda548f0f88d14ca8c3ee927d64e0ec0681fc7b
Saves us a bunch of thinking about how to handle symlinks, and prevents
the DNS config from changing on the fly under the build, which may or may
not be a good thing?
Change-Id: I071e6ae7e220884690b788d94f480866f428db71
93cc06334 removed nss-cacert from the binary tarball, but they're
necessary for global compatibility (and for our installer). This is what
results in cacerts being in the default profile, so e.g. the daemon has
TLS certs without having to use the system ones.
There's a fallback behavior in the daemon script in case these wind up
missing from the profile, but we don't want to have to rely on that,
since the fallback fails if it doesn't recognize one of a handful of
distros.
Change-Id: I60d8e6f734469548e80d5f38113ef168f67cbf7d
* changes:
meson: fix log-dir
manual: build docs with dummy envs
libcmd: install generated headers as well
docs: redo content generation for mdbook and manual
this was previously used because the macOS docs build would otherwise
pull files out of the host nix store. or something. not sure about it
Change-Id: I76b51eac1ebc5de5f00e2e4be086dd8db3eeb8e6
manpages can be rendered using the markdown output of mdbook, the rest
of the manual can generated out of the main doc/manual source tree. we
still use lowdown to actually render manpages instead of eg mdbook-man
because lowdown does generate reasonably good manpages (though that is
also somewhat debatable, but they're a lot better than mdbook-man).
doing this not only lets us drastically simplify the lowdown pipeline,
but also remove all custom {{#include}} handling since now mdbook does
all of it, even for the manpage builds. even the lowdown wrapper isn't
entirely necessary because lowdown can take all wrapper arguments with
command line flags rather than bits of input file content.
This also implements running mdbook in Meson, in order to generate the
manpages. The mdbook outputs are also installed in the usual location.
Co-authored-by: Qyriad <qyriad@qyriad.me>
Change-Id: I60193f9fd0f15d48872f071af35855cda2a0f40b
this should be a link, not an anchor. it should also point to the
`gloss-store` element, not the `#gloss-store` element.
Change-Id: I1f2803093179549637e10f917ad73399a419131b
Previously, errors while printing values in `nix repl` would be printed
in `«error: ...»` brackets rather than displayed normally:
```
nix-repl> legacyPackages.aarch64-darwin.pythonPackages.APScheduler
«error: Package ‘python-2.7.18.7’ in /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/cpython/2.7/default.nix:335 is marked as insecure, refusing to evaluate.»
```
Now, errors will be displayed normally if they're emitted at the
top-level of an expression:
```
nix-repl> legacyPackages.aarch64-darwin.pythonPackages.APScheduler
error:
… in the condition of the assert statement
at /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/lib/customisation.nix:268:17:
267| in commonAttrs // {
268| drvPath = assert condition; drv.drvPath;
| ^
269| outPath = assert condition; drv.outPath;
… in the left operand of the OR (||) operator
at /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/passthrufun.nix:28:45:
27| if lib.isDerivation value then
28| lib.extendDerivation (valid value || throw "${name} should use `buildPythonPackage` or `toPythonModule` if it is to be part of the Python packages set.") {} value
| ^
29| else
(stack trace truncated; use '--show-trace' to show the full trace)
error: Package ‘python-2.7.18.7’ in /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/cpython/2.7/default.nix:335 is marked as insecure, refusing to evaluate.
```
Errors emitted in nested structures (like e.g. when printing `nixpkgs`)
will still be printed in brackets.
Change-Id: I25aeddf08c017582718cb9772a677bf51b9fc2ad
The configured sysconfdir is used to look for nix.conf, so it needs
to be /etc, and not $out/etc, so we separate out the place where shell
profile files are installed, which is the only other place sysconfdir is
at all used.
See lix-project/lix#231 (comment)
for more info.
Change-Id: Idbed8ba82e711b8a9d6b6127904befa27d58e279
Instead of $sysconfdir.
Fixes#231, but there's more to do in following commits to make
Meson-built Lix actually look in /etc/nix.
Change-Id: Ia8d627070f405843add46e05cff5134b76b8eb48
These scripts were originally written by horrors, and have since been
hacked up a lot by jade. We are putting them up as a CL since it is
better to have checked in benchmarking scripts than to not have
benchmarking scripts.
cc: lix-project/lix#23
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I95c2f9d24753ac468944c5781deec9508fd5cb8c
this isn't strictly necessary, but it'll make it a lot easier to put the
generated files used by the autoconf build system in this directory too.
doing this now already will make the meson transition a lot easier later
Change-Id: I5fb39eade2ff88b6093c9ee436c9e8db793e9448
this would make meson build compatibility unnecessarily hard and
the cli does not change often enough to justify this complexity.
Change-Id: I17b1870cdf8538feeaa01a9945db97af2175a642
not sure why this was done the way it was considering that includes are
a feature the doc toolchain had previously. let's just always have some
kind of entry for the upcoming release in the dev manual builds even if
that means having a completely empty release notes chapter.
the release notes generation script isn't entirely functional right now
due to pre-commit hooks, but it's good enough for time being. we need a
better release process for notes anyway.
Change-Id: Ifda6912cf5233db013f72a30247a62d6f22b1565
Change-Id: I9eb347ec4aabc5be2b816ff0fd3e4be45f93b934
mdbook already does include processing of its own, and the custom
processing code has always admitted as much. we don't need it for
the mdbook build at this point if we run our preprocessors in the
right order, and maybe we can even have mdbook to return complete
pages to us that we only have to pass to lowdown without any more
preprocessing of our own.
Change-Id: Icd978acbc3b1e215fee8f062c53ab2cb2a222ab1
for some reason these three were anchors, not links, but had they been
links they wouldn't've worked because they're not defined anywhere but
here. in the print version of the manual they're duplicated many times
over (creating id collisions), so we should better remove them anyway.
Change-Id: I8988a7c32c812dee0f0b6d4953faa7cd1255228d
Adds a `repl-overlays` option, which specifies files that can overlay
and modify the top-level bindings in `nix repl`. For example, with the
following contents in `~/.config/nix/repl.nix`:
info: final: prev: let
optionalAttrs = predicate: attrs:
if predicate
then attrs
else {};
in
optionalAttrs (prev ? legacyPackages && prev.legacyPackages ? ${info.currentSystem})
{
pkgs = prev.legacyPackages.${info.currentSystem};
}
We can run `nix repl` and use `pkgs` to refer to `legacyPackages.${currentSystem}`:
$ nix repl --repl-overlays ~/.config/nix/repl.nix nixpkgs
Lix 2.90.0
Type :? for help.
Loading installable 'flake:nixpkgs#'...
Added 5 variables.
Loading 'repl-overlays'...
Added 6 variables.
nix-repl> pkgs.bash
«derivation /nix/store/g08b5vkwwh0j8ic9rkmd8mpj878rk62z-bash-5.2p26.drv»
Change-Id: Ic12e0f2f210b2f46e920c33088dfe1083f42391a
This is in our style guide, we can cheaply enforce it, let's do it.
```
$ pre-commit
check-case-conflicts.....................................................Passed
check-executables-have-shebangs..........................................Passed
check-headers............................................................Failed
- hook id: check-headers
- exit code: 1
Missing pattern @file in file src/libexpr/value.hh
We found some header files that don't conform to the style guide.
The Lix style guide requests that header files:
- Begin with `#pragma once` so they only get parsed once
- Contain a doxygen comment (`/**` or `///`) containing `@file`, for
example, `///@file`, which will make doxygen generate docs for them.
When adding that, consider also adding a `@brief` with a sentence
explaining what the header is for.
For more details: https://wiki.lix.systems/link/3#bkmrk-header-files
check-merge-conflicts....................................................Passed
check-shebang-scripts-are-executable.....................................Passed
check-symlinks.......................................(no files to check)Skipped
end-of-file-fixer........................................................Passed
mixed-line-endings.......................................................Passed
no-commit-to-branch......................................................Passed
release-notes........................................(no files to check)Skipped
treefmt..................................................................Passed
trim-trailing-whitespace.................................................Passed
```
Fixes: lix-project/lix#233
Change-Id: I77150b9298c844ffedd0f85cc5250ae9208502e3
This probably snuck in in a refactor using truthiness or so. The
trustedness flag was having the optional fullness checked, rather than
the actual contained trust level.
Also adds some tests.
```
m1@6876551b-255d-4cb0-af02-8a4f17b27e2e ~ % nix store ping
warning: 'nix store ping' is a deprecated alias for 'nix store info'
Store URL: daemon
Version: 2.20.4
Trusted: 0
m1@6876551b-255d-4cb0-af02-8a4f17b27e2e ~ % nix doctor
warning: 'doctor' is a deprecated alias for 'config check'
[PASS] PATH contains only one nix version.
[PASS] All profiles are gcroots.
[PASS] Client protocol matches store protocol.
[INFO] You are trusted by store uri: daemon
```
Fixes: lix-project/lix#232
Change-Id: I21576e2a0a755036edf8814133345987617ba3d0
The flake for pre-commit-checks is rather questionable. We ignored
it so it uses our own nixpkgs and doesn't reimport nixpkgs. This should
save a couple of seconds of eval time!
Change-Id: I4584982beb32e0122f791fa29f6a544bdbb9e201
package.nix previously needed this callPackage'd externally, which
didn't make a lot of sense to us since this is an internal dependency.
Thus we changed it to make it more self contained.
Change-Id: I4935bc0bc80e1a132bc9b1519e917791da95037c
Some of this code existed for installer tests, and indeed its removal is
an indication that our daemon cross-compatibility tests were removed.
Although these are not like, super critical tests, we would like to
restore them.
See: lix-project/lix#33
Change-Id: I75c733b25c00eca3a9676d498703bbfc1d6ec21b
The following command is now sufficient to build Lix from outside of the
flake:
nix-build -E 'let pkgs = import <nixpkgs> { }; in pkgs.callPackage
./package.nix { build-release-notes = false; nix-doc = pkgs.callPackage
./nix-doc/package.nix { }; }'
Change-Id: Ie6b14b446480ac07c7266d4fba20042b04cc35b9
follow-up to 32eaa8a29[1] "flake: move release note checks to hydraJobs",
this commit fixes a load-bearing typo for`checks.rl-next` and
`checks.rl-next-dev`.
[1]: 32eaa8a2910793538deab31f85534faf7e722ef7
Change-Id: I9383ed21f7eccc337c0c2f65525418b735a94a1d
In our view it really doesn't make sense to not have this in in
package.nix in some way. These patches aren't just for performance or
something -- Lix flat out doesn't build without these patches.
(Arguably that makes them a buildsystem responsibility as well, but that
can wait for when we're ready to start adding subproject fallback
dependency resolution to Meson.)
This is a step towards making `package.nix` more self-sufficient and
`callPackage`able without excessive external logic.
With this change the following command is enough to build Lix from out
of the flake:
nix-build -E 'let pkgs = import <nixpkgs> { }; in pkgs.callPackage
./package.nix { build-release-notes = false; inherit (pkgs.lib) fileset;
nix-doc = pkgs.callPackage ./nix-doc/package.nix { }; }'
Change-Id: Ia37fe8171f87d3293033de8be07d9bab12716f1d
`nix eval --write-to` refuses to write to a directory that exists at
all, so now we generate in a temporary directory, and copy the generated
tree to the build directory. This is equivalent to what the Make
buildsystem did, actually, but hopefully more robust.
Future work: documenting the doc generation architecture in the
top-level meson.build outline comment.
Change-Id: Ic3eb6d26e3cc249a1c042fd3ced22d637ac66a69
code blocks, if not surrounded by empty lines, have the language
tags (in these cases, always `nix`) show up in the output of :doc.
for example:
nix-repl> :doc builtins.parseFlakeRef
Synopsis: builtins.parseFlakeRef flake-ref
Parse a flake reference, and return its exploded form.
For example: nix builtins.parseFlakeRef
"github:NixOS/nixpkgs/23.05?dir=lib" evaluates to: nix { dir =
"lib"; owner = "NixOS"; ref = "23.05"; repo = "nixpkgs"; type =
"github"; }
is now instead:
nix-repl> :doc builtins.parseFlakeRef
Synopsis: builtins.parseFlakeRef flake-ref
Parse a flake reference, and return its exploded form.
For example:
| builtins.parseFlakeRef "github:NixOS/nixpkgs/23.05?dir=lib"
evaluates to:
| { dir = "lib"; owner = "NixOS"; ref = "23.05"; repo = "nixpkgs"; type = "github"; }
(closes#225)
Change-Id: I0741aeb1006a5376bb2f663d202c7a4da7e38cce
It is a little bit scuffed, but it seems to produce correct results. We
can run it at a later date when we want to explode every in-flight
commit in existence and then need to filter-branch them.
Fixes: lix-project/lix#188
Change-Id: Id97e4651f78804a941d941df02c7c1b21ce453b6
This is terrible UX, and frankly an eval failure should be a cache
invalidation anyway.
This removes the CachedEvalError type entirely.
Fixes#223.
Change-Id: I91f8003eabd0ea45003024e96d1de3c7ae8e49d8
Commit c21d11ac0 "docs: replace sed invocation with an mdbook
preprocessor for @docroot@" added a direct build dependency on Python.
This has been accidentally working so far because Python is already a
*transitive* dependency of Lix's derivation.
Change-Id: I32d6b4f2665dbbfad7014613457dd58aa4ec73da
* changes:
Revert "libutil: drop Pool resources on exceptional free"
Revert "libutil: remove Pool::Handle::bad"
Revert "libstore: remove one Resource::good flag"
Revert "libstore: using throwing finally in withFramedSink"
Revert "libutil: allow graceful dropping of Pool::Handle"
Revert "libutil: drop Fs{Source,Sink}::good"
libutil: guard Finally against invalid exception throws
- Use a recursive descent parser so that it's easy to extend.
- Add `@args` to enable customizing command-line arguments
- Add `@should-start` to enable `nix repl` tests that error before
entering the REPL
- Make sure to read all stdout output before comparing. This catches
some extra output we were tossing out before!
Change-Id: I5522555df4c313024ab15cd10f9f04e7293bda3a
This reverts commit 491caad6f62c21ffbcdebe662e63ec0f72e6f3a2.
this is not actually legal for nix! throwing exceptions in destructors
is fine, but the way nix is set up we'll end up throwing the exception
we received from the remote *twice* in some cases, and such cases will
cause an immediate terminate without active exception.
Change-Id: I74c46b9f26fd791086e4193ec60eb1deb9a5bb2a
throwing exceptions is fine, but throwing exceptions during exception
handling is hard enough to do correctly that we should just forbid it
entirely out of an overabundance of caution. in cases where terminate
is the correct answer the users of Finally must call it manually now.
Change-Id: Ia51a2cb4a0638500550bfabc89cf01a6d8098983
These were mistakenly labeled `eval-fail-*`.
Note that the `lang.sh` runner passes `parse-fail-*` tests on stdin, so
filenames are removed from error messages.
Change-Id: I7f3a0d78b6cfa87af29aaa1b7af19d5a57fd4ade
We're not entirely clear on why the links preprocessor has to be done
*before* rather than after, but we assume it is probably that as a
builtin preprocessor it does some processing on the raw book source,
and not just the JSON data.
Also a real use for Python pattern matching? I know I was surprised too.
Change-Id: Ibe8b59e7b5bd5f357a655d8b4c5f0b0f58a67d6b
This reverts commit 70954233743a233744787103d3211237a28ddbca.
This seems to have broken running ninja on warm build directories, which
is not what we want. Reverted until we figure out something better
Change-Id: I9623ae078917e7c59a930bf8044a216501d4bb20
This puts the generated files where they are for the make system.
This is in preparation for further meson-mdbook stuff.
Change-Id: I934df6854a80af5ccf381cf1da0bda0187a8bcfc
For a long time `nix repl` has supported displaying documentation set on
builtins, however, it has long been convention to use Markdown comments
on Nix functions themselves for documentation. This exposes that
information to `nix repl` users in a nice and formatted way.
NixOS/rfcs#145 doc-comments are primarily what this feature is intended
to consume, however, support for lambda documentation in the repl is
experimental. We do our best effort to support the RFC here.
These changes are based on [the nix-doc library](https://github.com/lf-/nix-doc) and
are licensed under the terms described in the relevant source files.
Change-Id: Ic6fe947d39a22540705d890737e336c4720b0a22
setting this only on exceptions caused by actual fd access is not
sufficient to diagnose all errors (such as SerialisationError) in
some cases. this usually does not have any negative effects since
those errors will end up killing the process in another way. this
is not a reliable assumption though and we should be using proper
error handling (and closing connections more often, preferring to
close over keeping something open that might be in a weird state)
Change-Id: I1b792cd7ad8ba9ff0f6bd174945ab2575ff2208e
not needed yet, but returning a resource from the exception handling
path that has ownership of a handle is currently not well-supported.
we could also add a default constructor to Handle, but then we would
also need to change the pool reference to a pointer. eventually that
should be done since now resources can be swapped between pools with
clever moves, but since that's not a problem yet we won't do it now.
Change-Id: I26eb06581f7be34569e9e67a33da736128d167af
the duplication of exception handling was added without justification,
so we can only assume that it was done like this because Finally could
not throw exceptions safely. since this has now been rectified we will
deduplicate this handler code again.
Change-Id: I40721f3378c0fd9f34e2914a16d383f6e2713b40
this is supposed to act like a finally block does in other languages. a
finally block should be able to throw exceptions of its own rather than
just crashing the entire program when it throws it own exceptions. even
in the rare case of a finally throwing an unexpected exception it might
be better to report the exception from Finally instead of the original,
at least that can keep our program running instead of letting it crash.
Change-Id: Id42011e46b1df369152b4564938c0e93fa1acf32
usage of this flag previously kept connections open much longer than
necessary, and at the same time obscured that a connection was being
dropped when it *was* set. new variable names clarify this somewhat.
Change-Id: I11f6f08f37a5e4dc04ea6c6036ea589154b121c6
it was used incorrectly (not swapped on handle move), only used in one
place (that is now handled with exception handling detection in Handle
itself), and if ever reintroduced should be replaced with a different,
more understandable mechanism (like an explicit dropAsInvalid method).
Change-Id: Ie3e5d5cfa81d335429cb2ee5c3ad85c74a9df17b
this was never actually used, and bad design in the first place—why
should a bad resource be put back into the idle pool? just drop it.
Change-Id: Idab8774bee19dadae0209d404c4fb86dd4aeba1e
if a scope owning a resource does not gracefully drop that resource
while handling exceptions from deeper down the call stack we should
assume the resource is invalid state and drop it. currently it *is*
true that such cases do not cause resources to be freed, but thanks
to validator misuses this has so far not caused any larger problem.
Change-Id: Ie4f91bcd60a64d05c5ff9d22cc97954816d13b97
Perl has an env hook[1]. Passing the paths manually without putting them
in buildInputs is harder to understand, plays less nicely with dev
shells, and is less build-generic.
Produced identical output on my x86_64-linux machine, and on my
aarch64-darwin machine was identical save for the derivation output path
which gets embedded into the .dylib Mach-O.
Change-Id: Ib313caa5a6f0b0e3154ce6f05379033920d0d290
this notably does *not* install the `nix3-manpages` manpage the old
system generated, mostly because that page was empty and just a bug
with a coat of documentation paint.
Change-Id: I7a4248a72e7bb5e0cc925a6311a33b6b72589569
we'll want to use these for the meson builds, and probably eventually
rewrite them in something that isn't plain shell. diffoscope confirms
that out/share and doc/share are equal before and after these changes
Change-Id: I49aa418fc8615cad86d67328e08c28a7405ec952
The big ones here are `trim-trailing-whitespace` and `end-of-file-fixer`
(which makes sure that every file ends with exactly one newline
character).
Change-Id: Idca73b640883188f068f9903e013cf0d82aa1123
This does involve making a large number of destructors able to throw,
because we had to change it high in the class hierarchy. Oh well.
Change-Id: Ib62d3d6895b755f20322bb8acc9bf43daf0174b2
This has not yet had all the warnings Obliterated, but it is a start and
is not *super* far away from being able to run the current configuration
in CI, which will catch some limited number of mistakes.
I tried the meson clang-tidy target and it seems to fail to find flags
for several files, which seems broken. Unsure what is up with that, but
we can use run-clang-tidy or other tooling instead.
We have an extremely annoying situation with the lexer table, which
means that the lexer probably must be moved to another directory with
its own .clang-tidy file to disable the lints in it, *or* write scuffed
code that prepends a disable comment to the top of the generated file.
None of the comment-based lint disabling features work since yacc dumps
a bunch of non compliant code at the top of the file before anything the
user can control.
Change-Id: I1d2aa6ec32deb1db1fbd581127334db1b972323c
* some things that can throw are marked noexcept
yet the linter seems to think not. Maybe they can't throw in practice.
I would rather not have the UB possibility in pretty obvious cold
paths.
* various default-case-missing complaints
* a fair pile of casts from integer to character, which are in fact
deliberate.
* an instance of <https://clang.llvm.org/extra/clang-tidy/checks/bugprone/move-forwarding-reference.html>
* bugprone-not-null-terminated-result on handing a string to curl in
chunks of bytes. our usage is fine.
* reassigning a unique_ptr by CRIMES instead of using release(), then
using release() and ignoring the result. wild. let's use release() for
its intended purpose.
Change-Id: Ic3e7affef12383576213a8a7c8145c27e662513d
We got confused what formals did and had to briefly figure it out. We
should just have docs, so these are some.
Change-Id: If3e794a401e69d022785cbfa0b0c2e2284f41f58
Since we don't have a ./configure checked in, it makes no sense to keep
any of these scripts since people are already going to be running
autoreconf anyway.
Plus they will be completely deletable when meson shows up.
This fixes `autoreconfPhase` causing git diffs.
Change-Id: Ibb2aee422c562a23faadfdedb55b5c18c41a9420
the autoconf build system defaults to /nix/var, not /nix/var/nix. the
latter is only used in libstore, so we'll move the extra segment there.
Change-Id: Idfbc988ee302355982abdcd51d6d7b5d5d661c0d
Without this, the Meson setup won't bail out if nlohmann_json is
missing, leading to subpar DX (and maybe worse, but I'm not entirely
sure).
Change-Id: I5913111060226b540dcf003257c99a08e84da0de
one headers (args/root.hh) was simply missing, and the generated headers
were not installed. not all of them *should* be installed either, only a
select few (and sadly this needs a custom target for each one, it seems)
Change-Id: I37b25517895d0e5e521abc1202fa65624de57ed1
sometimes these fail with timeouts on loaded machines. let's up the
timeouts until we can pull the tests apart to more reasonable sizes
Change-Id: I2dfff2183cc1f3ff5e6107f43748ac046fe00d05
- Enable parallel builds by default (and allow using environment
variables to override `make` variables)
- Hopefully we can get rid of this once we have Meson
- Set `GTEST_BRIEF=1`
- This only shows failed tests, instead of listing every test on its
own line.
```
$ GTEST_BRIEF=1 make check
[==========] 328 tests from 15 test suites ran. (37 ms total)
[ PASSED ] 328 tests.
```
Change-Id: Id8103a8f24a9681be2be87e1b4df6fd5fdd7e4fd
Functional tests can be run with
`meson test -C build --suite installcheck`.
Notably, functional tests must be run *after* running `meson install`
(Lix's derivation runs the installcheck suite in installCheckPhase so it
does this correctly), due to some quirks between Meson and the testing
system.
As far as I can tell the functional tests are meant to be run after
installing anyway, but unfortunately I can't transparently make
`meson test --suite installcheck` depend on the install targets.
The script that runs the functional tests, meson/run-test.py, checks
that `meson install` has happened and fails fast with a (hopefully)
helpful error message if any of the functional tests are run before
installing.
TODO: this change needs reflection in developer documentation
Change-Id: I8dcb5fdfc0b6cb17580973d24ad930abd57018f6
This was achieved by running maintainers/buildtime_report.sh on the
build directory of a meson build, then asking "why the heck is json
eating our build times", and strategically moving the json using bits
out of widely included headers.
It turns out that putting literally any metrics whatsoever into the
build had immediate and predictable results.
Results are 1382.5s frontend time -> 1175.4s frontend time, back end
time approximately invariant.
Related: lix-project/lix#159
Change-Id: I7edea95c8536203325c8bb4dae5f32d727a21b2d
I didn't enable this by default for clang due to making the build time
10% worse or so. Unfortunate, but tbh devs for whom 10% of build time is
not *that* bad should probably simply enable this.
Change-Id: I8d1e5b6f3f76c649a4e2f115f534f7f97cee46e6
hacking changelog-d to support not just github but also forgejo and
gerrit is a lot more complicated than it's worth, even moreso since
the entire thing can just as well be done with ~60 lines of python.
this new script is also much cheaper to instantiate (being python),
so having it enabled in all shells is far less of a hassle.
we've also adjusted existing release notes that referenced a gerrit
cl to auto-link to the cl in question, making the diff a bit bigger
closes lix-project/lix#176
Change-Id: I8ba7dd0070aad9ba4474401731215fcf5d9d2130
Once this commit lands, we are even more visible in analytics FWIW.
Change-Id: Id7e0c162315d0f191edbea9cb5fb82ce363704b9
Signed-off-by: Raito Bezarius <raito@lix.systems>
HintFmt(string) invokes the HintFmt("%s", literal) constructor,
which is not what we want here. Add a constructor with a proper name
and call that.
Next step: rename all the other ones to HintFmt::literal(string).
Fixes: lix-project/lix#178
Change-Id: If52d2eb8864ceb8663e05992e9d1fffef573d6b8
Unit tests can be run with `meson test -C build --suite check`.
`--suite check` is optional, as right now that's the only test suite,
but when functional tests are added those will be in a separate suite.
Change-Id: I7f22f1cde4b489b3cdb5f9a36a544f0c409fcc1f
parallel meson builds need too much ram. linearize them for now, and
hopefully we can remove the make build system and this hack soonish.
Change-Id: I42c092db8b0c63680e77da2263cdfe9e7f6575be
`nix --version` doesn't require `nix-command` experimental feature to
run and we could all do with less nix-env
Change-Id: I90748d591c574d96eda46591e9f9ce828311da29
Committing a lock file using markFileChanged() required the input to
be writable by the caller in the local filesystem (using the path
returned by getSourcePath()). putFile() abstracts over this.
(cherry picked from commit 95d657c8b3ae4282e24628ba7426edb90c8f3942)
Change-Id: Ie081c5d9eb4e923b229191c5e23ece85145557ff
As I complained in
https://github.com/NixOS/nix/pull/6784#issuecomment-1421777030 (a
comment on the wrong PR, sorry again!), #6693 introduced a second
completions mechanism to fix a bug. Having two completion mechanisms
isn't so nice.
As @thufschmitt also pointed out, it was a bummer to go from `FlakeRef`
to `std::string` when collecting flake refs. Now it is `FlakeRefs`
again.
The underlying issue that sought to work around was that completion of
arguments not at the end can still benefit from the information from
latter arguments.
To fix this better, we rip out that change and simply defer all
completion processing until after all the (regular, already-complete)
arguments have been passed.
In addition, I noticed the original completion logic used some global
variables. I do not like global variables, because even if they save
lines of code, they also obfuscate the architecture of the code.
I got rid of them moved them to a new `RootArgs` class, which now has
`parseCmdline` instead of `Args`. The idea is that we have many argument
parsers from subcommands and what-not, but only one root args that owns
the other per actual parsing invocation. The state that was global is
now part of the root args instead.
This did, admittedly, add a bunch of new code. And I do feel bad about
that. So I went and added a lot of API docs to try to at least make the
current state of things clear to the next person.
--
This is needed for RFC 134 (tracking issue #7868). It was very hard to
modularize `Installable` parsing when there were two completion
arguments. I wouldn't go as far as to say it is *easy* now, but at least
it is less hard (and the completions test finally passed).
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Change-Id: If18cd5be78da4a70635e3fdcac6326dbfeea71a5
(cherry picked from commit 67eb37c1d0de28160cd25376e51d1ec1b1c8305b)
An attrPath prefix of "." indicates no need to try default attrPath prefixes. For example `nixpkgs#legacyPackages.x86_64-linux.ERROR` searches through
```
trying flake output attribute 'packages.x86_64-linux.legacyPackages.x86_64-linux.ERROR'
using cached attrset attribute ''
trying flake output attribute 'legacyPackages.x86_64-linux.legacyPackages.x86_64-linux.ERROR'
using cached attrset attribute 'legacyPackages.x86_64-linux'
trying flake output attribute 'legacyPackages.x86_64-linux.ERROR'
using cached attrset attribute 'legacyPackages.x86_64-linux'
```
And there is no way to specify that one does not want the automatic
search behavior. Now one can specify
`nixpkgs#.legacyPackages.x86_64-linux.ERROR` to only refer to the rooted
attribute path without any default injection of attribute search path or
system.
Change-Id: Iac1334e1470137b7ce11dcf845513810230638ec
(cherry picked from commit d4aed18883b361133607296fb6cd789c47427a38)
protocol versions are sent as u64. on the peer we read them as uint64,
check that the upper half is 0, and throw an exception if not. we then
read an arbitrary amount of data from the peer and dump it to the user
terminal. this is a little bit ridiculous, can never happen in correct
implementation, and is severly untested. let us just drop it entirely.
Change-Id: Ibd2f53a765341ed6439d40d9d1eac11e79c6b5e3
this is not needed and introduces a bunch of memset calls, making up for
3% of valgrind cycle estimation *alone*. real-world impact is a lot
lower on our test machine, but we suspect that less powerful machines
would see an impact from dropping this.
Change-Id: Iad10e9d556e64fdeb0bee0059a4e52520058d11e
If the state SQLite database is configured to use a write-ahead-log, it
creates WAL files in the state directory.
When the state SQLite database is closed by the `nix-daemon` after
builds, those files are removed.
When an unprivileged user would like to open _in read only_ that
database, they cannot do so because they would need to create those WAL
files and they do not have the permission to do so.
For this, SQLite offers a "persistent WAL" feature [1] to leave the WAL
files around, even after closing the database.
This CL enable the persistent WAL mode.
Fixes: https://github.com/NixOS/nix/issues/10300
[1]: https://www.sqlite.org/wal.html
Change-Id: Id8ae534d7d2290457af28782e5215222ae051fe5
Signed-off-by: Raito Bezarius <raito@lix.systems>
This commit adds several meson.build, which successfully build and
install Lix executables, libraries, and headers. Meson does not yet
build docs, Perl bindings, or run tests, which will be added in
following commits. As such, this commit does not remove the existing
build system, or make it the default, and also as such, this commit has
several FIXMEs and TODOs as notes for what should be done before the
existing autoconf + make buildsystem can be removed and Meson made the
default. This commit does not modify any source files.
A Meson-enabled build is also added as a Hydra job, and to
`nix flake check`.
Change-Id: I667c8685b13b7bab91e281053f807a11616ae3d4
this was mostly an inconvenience for error reporting, but fully broke
the debugger (because the debugger does *a lot* of eager position
resolution). copying the line offsets into a local and filling that
local when empty without also storing the calculated offsets back does
kind of ... not cache anything.
fixes lix-project/lix#165
Change-Id: Iccb0ba193ce2f15c832978daecf7b9bebbbe8585
within lix itself this problem is caught by the test suite. outside of
lix itself three cases can be had: either the problem is fully inside
lix libs, fully inside user code, or it exists at the boundary. the
first is caught by the test suite, the second isn't caught at all, and
the third is something lix should not be responsible for.
Change-Id: I95aa35d8cb6f0ef5816a2941c467bc0c15916063
* changes:
Release notes for builtins.nixVersion change
un-nixes ur lix, a little
issue importer: list issues that are *not* closed when finding existing issues
I didn't really go attack the docs because we need to pull a bunch of
PRs. I went looking for strings in the code that called lix nix.
Change-Id: I2138bb4dd239096bc530946b281db7f875195b39
add a reset() method to close the wrapped fd instead of assigning magic
constants. also make the from-fd constructor explicit so you can't
accidentally assign the *wrong* magic constant, or even an unrelated
integer that also just happens to be an fd by pure chance.
Change-Id: I51311b0f6e040240886b5103d39d1794a6acc325
static env association is from expr to its enclosing scope, but let
exprs set their association to their *inner* scope. this skips one level
of envs and will cause segfaults if the parent is a with expr.
fixes#145
Change-Id: I1d22146110f071ede21b4eed7ed34b5850ef2ef3
not doing this exposes the binding name order to the annoying
interference of parse order on symbol order, which wouldn't be so bad if
it didn't make the tests less reliable and, importantly, dependent on
linker behavior (due to primop initialization being done in static
initializer, and the order of static initializers being defined only
within a single translation unit).
fixes#143
Change-Id: I3cf417893fbcf19e9ad3ff8986deb7cbcf3ca511
we now keep not a table of all positions, but a table of all origins and
their sizes. position indices are now direct pointers into the virtual
concatenation of all parsed contents. this slightly reduces memory usage
and time spent in the parser, at the cost of not being able to report
positions if the total input size exceeds 4GiB. this limit is not unique
to nix though, rustc and clang also limit their input to 4GiB (although
at least clang refuses to process inputs that are larger, we will not).
this new 4GiB limit probably will not cause any problems for quite a
while, all of nixpkgs together is less than 100MiB in size and already
needs over 700MiB of memory and multiple seconds just to parse. 4GiB
worth of input will easily take multiple minutes and over 30GiB of
memory without even evaluating anything. if problems *do* arise we can
probably recover the old table-based system by adding some tracking to
Pos::Origin (or increasing the size of PosIdx outright), but for time
being this looks like more complexity than it's worth.
since we now need to read the entire input again to determine the
line/column of a position we'll make unsafeGetAttrPos slightly lazy:
mostly the set it returns is only used to determine the file of origin
of an attribute, not its exact location. the thunks do not add
measurable runtime overhead.
notably this change is necessary to allow changing the parser since
apparently nothing supports nix's very idiosyncratic line ending choice
of "anything goes", making it very hard to calculate line/column
positions in the parser (while byte offsets are very easy).
(cherry picked from commit 5d9fdab3de0ee17c71369ad05806b9ea06dfceda)
Change-Id: Ie0b2430cb120c09097afa8c0101884d94f4bbf34
this needs a string comparison because there seems to be no other way to
get that information out of bison. usually the location info is going to
be correct (pointing at a bad token), but since EOF isn't a token as
such it'll be wrong in that this case.
this hasn't shown up much so far because a single line ending *is* a
token, so any file formatted in the usual manner (ie, ending in a line
ending) would have its EOF position reported correctly.
(cherry picked from commit 855fd5a1bb781e4f722c1d757ba43e866d370132)
Change-Id: I120c56a962f4286b1ae3b71da7b71ce8ec3e0535
the parser treats a plain \r as a newline, error reports do not. this
can lead to interesting divergences if anything makes use of this
feature, with error reports pointing to wrong locations in the input (or
even outside the input altogether).
(cherry picked from commit 2be6b143289e5479cc4a2667bb84e879116c2447)
Change-Id: Ieb7f7655bac8cb0cf5734c60bd41723388f2973c
previously we reported the error at the beginning of the binding
block (for plain inherits) or the beginning of the attr list (for
inherit-from), effectively hiding where exactly the error happened.
this also carries over to runtime positions of attributes in sets as
reported by unsafeGetAttrPos. we're not worried about this changing
observable eval behavior because it *is* marked unsafe, and the new
behavior is much more useful.
(cherry picked from commit 1edd6fada53553b89847ac3981ac28025857ca02)
Change-Id: I2f50eb9f3dc3977db4eb3e3da96f1cb37ccd5174
we already normalize attr order to lexicographic, doing the same for
formals makes sense. doubly so because the order of formals would
otherwise depend on the context of the expression, which is not quite as
useful as one might expect.
(cherry picked from commit 4147ecfb1c51f3fe3b4adcbd4e753fd487dab645)
Change-Id: I3fd0dbdef3ac7447a3a03ff20bb514a0d0f23fb1
the parser modifies its inputs, which means that sharing them between
the error context reporting system and the parser itself can confuse the
reporting system. usually this led to early truncation of error context
reports which, while not dangerous, can be quite confusing.
(cherry picked from commit d384ecd553aa997270b79ee98d02f7cf7e1849e6)
Change-Id: I677646b5675b12b2faa787943646aa36dc6e6ee3
vfork confers a large performance advantage over fork, measured locally
at 16µs per vfork agains 90µs per fork. however nix *almost always*
follows a vfork up with an execve-family call, melting the performance
advantage from 6x to only 15%. in most of those cases it's doing things
that are undefined behavior (like manipulating the heap, or even
throwing exceptions and trashing the parent process stack).
most notably the one place that could benefit from the vfork performance
improvement is linux derivation sandbox setup—which doesn't use vfork.
Change-Id: I2037b7384d5a4ca24da219a569e1b1f39531410e
These now have equivalents in the standard lib in C++20. This change was
performed with a custom clang-tidy check which I will submit later.
Executed like so:
ninja -C build && run-clang-tidy -checks='-*,nix-*' -load=build/libnix-clang-tidy.so -p .. -fix ../tests | tee -a clang-tidy-result
Change-Id: I62679e315ff9e7ce72a40b91b79c3e9fc01b27e9
This builtin is only going to cause us problems because we are not Nix,
so let's just falsify being in the 2.18 series, since that is the
closest target that has any meaning.
In future we might want to have a better feature detection mechanism,
for when we actually add stuff to some builtin's attr set argument. But
builtins.nixVersion is just going to be hopelessly broken and it should
be stubbed out.
Fixes lix-project/lix#144
Change-Id: Id7390b32a29c6147f2977737d81846320de5d67e
diagnose attr duplication at the path the duplication was detected, not
at the path the current attribute wanted to place. doing the latter is
only correct if a leaf attribute was duplicated, not if an attrpath was
set to a non-attrset in one binding and a (potentially implied) attrset
in another binding.
fixes#124
Change-Id: Ic4aa9cc12a9874d4e7897c6f64408f10aa36fc82
-O3 does not measurably improve performance of the resulting binaries,
neither with lto enabled nor with lto disabled. what it does to however
is cause gcc warning spew in libstdc++ that we can't do anything
about (and that upon inspection of libstdc++ source looks like a gcc
bug).
with lto, -O3:
Benchmark 1: GC_INITIAL_HEAP_SIZE=10g nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
Time (mean ± σ): 4.608 s ± 0.027 s [User: 3.866 s, System: 0.522 s]
Range (min … max): 4.579 s … 4.640 s 10 runs
Benchmark 2: nix eval -f <nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix>
Time (mean ± σ): 408.1 ms ± 25.5 ms [User: 360.0 ms, System: 28.1 ms]
Range (min … max): 387.6 ms … 439.0 ms 10 runs
with lto, -O2:
Benchmark 1: GC_INITIAL_HEAP_SIZE=10g nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
Time (mean ± σ): 4.632 s ± 0.044 s [User: 3.874 s, System: 0.544 s]
Range (min … max): 4.563 s … 4.673 s 10 runs
Benchmark 2: nix eval -f <nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix>
Time (mean ± σ): 394.0 ms ± 23.9 ms [User: 351.2 ms, System: 27.6 ms]
Range (min … max): 377.8 ms … 429.3 ms 10 runs
without lto, -O3:
Benchmark 1: GC_INITIAL_HEAP_SIZE=10g nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
Time (mean ± σ): 4.700 s ± 0.024 s [User: 3.906 s, System: 0.559 s]
Range (min … max): 4.663 s … 4.717 s 10 runs
Benchmark 2: nix eval -f <nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix>
Time (mean ± σ): 400.4 ms ± 25.6 ms [User: 353.7 ms, System: 26.8 ms]
Range (min … max): 379.8 ms … 430.6 ms 10 runs
without lto, -O2:
Benchmark 1: GC_INITIAL_HEAP_SIZE=10g nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
Time (mean ± σ): 4.724 s ± 0.030 s [User: 3.924 s, System: 0.570 s]
Range (min … max): 4.687 s … 4.749 s 10 runs
Benchmark 2: nix eval -f <nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix>
Time (mean ± σ): 392.4 ms ± 24.3 ms [User: 350.9 ms, System: 26.4 ms]
Range (min … max): 376.9 ms … 428.0 ms 10 runs
fixes#46
Change-Id: Ib8afad8a07c278f57f2e3317d00cce4f9ec0f338
It happens with some frequency that plugins that might be unimportant to
the evaluation at hand mismatch with the nix version, leading to
spurious load failures. Let's make these non fatal.
Change-Id: Iba10e951d171725ccf1a121bcd9be1e1d6ad69eb
This is because they are unrepresentable in the source files with
commentary but not in the output, so we should just eat them in
normalization. It's ok.
Change-Id: I2cb7e8b3fc7b00874885bb287cbaa200b41cb16b
`macOS` does not have `glibcLocales`:
error:
… while calling the 'derivationStrict' builtin
at /derivation-internal.nix:9:12:
8|
9| strict = derivationStrict drvAttrs;
| ^
10|
… while evaluating derivation 'nix-2.90.0'
whose name attribute is located at /nix/store/y0c95bwyvs80pm69hdd4b11pyq2ghiwh-source
/pkgs/stdenv/generic/make-derivation.nix:348:7
… while evaluating attribute 'LOCALE_ARCHIVE' of derivation 'nix-2.90.0'
at /nix/store/ng5qzbyv4902b4pw7g35caqw5cnmryf9-source/flake.nix:331:15:
330| # Required to make non-NixOS Linux not complain about missing loc
Change-Id: I4464484a0eca12b5e073d49d900b6f25886245c1
using the total-attrs-printed and total-list-items-printed counters to
calculate how many attrs were elided only works properly if no nesting
is involved. once things do nest the global counter can exceed the size
of the currently printed object, leading to unsigned wrapping and great
overestimation of elided counts. counting locally in addition to global
counts fixes this.
these are functional tests because creating these objects requires the
evaluator to not be a huge amount of code, and we also want defaults to
be tested for cli usage.
fixes#14
Change-Id: Icb9a0cb21b2f4bacbc5e9dcdd8c0b9055b4088a7
this lets us set per-test-program environment variables rather than only
a single, global default. this was supported in nix originally but
might've gone partially missing in the upstream backports process?
Change-Id: Iad0919841b1b6d11e0b7ebd3920449a62f544e77
This has some Flaws for sure (like, it is going to be a bit stretched to
use for repl characterization), but it is a start.
Change-Id: I258c8beb3aee236f45818a03be83bcda858120c9
This is definitely not a stable thing, but it does feel slightly crimes
to put it as an experimental feature. Shrug, up for bikeshedding.
Change-Id: I6ef176e3dee6fb1cac9c0a7a60d553a2c63ea728
* changes:
package: cleanup of all intermediaries
package: migrate devShells
package: migrate internal-api-docs
package: migrate testNixVersions
package: use pname, version, and dontBuild (first change with diff hash)
package: refactor Nix out of flake.nix and into package.nix
The src fileset, preConfigure, and separateDebugInfo also respond to doBuild if its overridden
This commit is logically just a continuation of the previous commit's
refactor, but exists separately to delineate when the core Nix
derivation hash changed (this commit).
Change-Id: I67a61bc9608d91b6a833ebc5c3894b2d2e694050
This series takes a somewhat different approach from the flake rework
done in NixOS/nix. The package.nix here does not provide callPackage
options for all the various settings in the build, and instead the other
places Nix derivations are used (like internal-api-docs) will .overrideAttrs
the normal Nix package derivation. This more closely matches how these
things were structured originally, and results in less churn and more
atomicity in these changes.
In the future, package.nix likely will migrate to have more build
options in the callPackage arguments, but we are also planning to
rewrite the build system anyway.
Change-Id: I170c4e5a4184bab62e1fd75e56db876d4ff116cf
This solves the problem of collections of boxed subclasses with virtual
dispatch, which should still be treated as values, since the
indirection is only there due to the virtual dispatch.
Change-Id: I368daedd3f31298e99c6e56a15606337a55494c6
it's no longer widely used and has a rather confusing meaning now that
inherit-from is handled very differently.
(cherry picked from commit 1cd87b7042d14aae1fafa47b1c28db4c5bd20de7)
Change-Id: I90bbebddf06762960d8ca4f621cf042ce8ae83f9
desugaring inherit-from to syntactic duplication of the source expr also
duplicates side effects of the source expr (such as trace calls) and
expensive computations (such as derivationStrict).
(cherry picked from commit cefd0302b55b3360dbca59cfcb4bf6a750d6cdcf)
Change-Id: Iff519f991adef2e51683ba2c552d37a3df7a179e
deduplication does not currently work fully, showing derivations
multiple times if they have different underlying values. this can happen
by selecting the same derivation twice for two different attributes of a
set, using inherit-from (which reduces to the previous), importing
nixpkgs twice, or any other number of things.
since users already have to deal with duplicates for this reason it
won't hurt to add *more* duplicates. the alternative would be to
deduplicate fully, which would drop derivations that are currently
returned and those pose a regression risk.
Change-Id: I64b397351237e10375d270f1bddecb71f62aa131
for plain inherits this is really just a stylistic choice, but for
inherit-from it actually fixes an exponential size increase problem
during expr printing (as may happen during assertion failure reporting,
on during duplicate attr detection in the parser)
(cherry picked from commit ecf8b12d60ad2929f9998666cf0966475b91e291)
Change-Id: Ie55f0cb01a37e766414c31f8d40f51c2c7d106b0
this also has the effect of sorting let bindings lexicographically
rather than by symbol creation order as was previously done, giving a
better canonicalization in the process.
(cherry picked from commit 6c08fba533ef31cad2bdc03ba72ecf58dc8ee5a0)
Change-Id: Ia887f629305645bb8a165fbbc0d32e620912595a
in place of inherited() — not quite useful yet since we don't
distinguish plain and inheritFrom attr kinds so far.
(cherry picked from commit 1f542adb3e18e7078e6a589182a53a47d971748a)
Change-Id: If948c9d43e875de18f213a73a06a36f7c335b536
without these changes the tests will very repeatably (although not very
reliably) wedge in our runs. the ssh command starts, opens a sessions,
does something, the session closes again, but the test does not move on.
adding *just* the redirect and not the unit waits is not sufficient
either, it needs both. this feels like a bug in the nixos testing
framework somewhere, but digging that far is not in the cards right now.
Change-Id: Idab577b83a36cc4899bb5ffbb3d9adc04e83e51c
Do not skip any stack frames when `--show-trace` is given
(cherry picked from commit 0b47783d0a879875d558f0b56e49584f25ceb2d0)
Change-Id: Ia0f18266dbcf97543110110c655c219c7a3e3270
Enter debugger on `builtins.trace` with an option
(cherry picked from commit 774e7ca5847ebc392eac2a124a8f12b24da4f65a)
Change-Id: If01e2110b3a128e639b05143227e365227d149f1
Pretty-print values in the REPL by printing each item in a list or
attrset on a separate line. When possible, single-item lists and
attrsets are printed on one line, as long as they don't contain a nested
list, attrset, or thunk.
Before:
```
{ attrs = { a = { b = { c = { }; }; }; }; list = [ 1 ]; list' = [ 1 2 3 ]; }
```
After:
```
{
attrs = {
a = {
b = {
c = { };
};
};
};
list = [ 1 ];
list' = [
1
2
3
];
}
```
(cherry picked from commit c0a15fb7d03dfb8f53bc6726c414bc88aa362592)
Change-Id: Ia2b41849165a5ddb63f7a8c272a2476b3e4292df
While preparing PRs like #9753, I've had to change error messages in
dozens of code paths. It would be nice if instead of
EvalError("expected 'boolean' but found '%1%'", showType(v))
we could write
TypeError(v, "boolean")
or similar. Then, changing the error message could be a mechanical
refactor with the compiler pointing out places the constructor needs to
be changed, rather than the error-prone process of grepping through the
codebase. Structured errors would also help prevent the "same" error
from having multiple slightly different messages, and could be a first
step towards error codes / an error index.
This PR reworks the exception infrastructure in `libexpr` to
support exception types with different constructor signatures than
`BaseError`. Actually refactoring the exceptions to use structured data
will come in a future PR (this one is big enough already, as it has to
touch every exception in `libexpr`).
The core design is in `eval-error.hh`. Generally, errors like this:
state.error("'%s' is not a string", getAttrPathStr())
.debugThrow<TypeError>()
are transformed like this:
state.error<TypeError>("'%s' is not a string", getAttrPathStr())
.debugThrow()
The type annotation has moved from `ErrorBuilder::debugThrow` to
`EvalState::error`.
(cherry picked from commit c6a89c1a1659b31694c0fbcd21d78a6dd521c732)
Change-Id: Iced91ba4e00ca9e801518071fb43798936cbd05a
Don't print the first bracket in values in magenta in error messages
(cherry picked from commit 46a0625a40aef6946a35f92fdacf0e6b4a14414f)
Change-Id: I8435565c87db182116140eaeea9df1243e67ea94
Enter debugger more reliably in `let` expressions and function calls
(cherry picked from commit c4ed92fa6f836d3d8eb354a48c37a2f9eeecc3aa)
Change-Id: I16d0cad7e898feecd2399723b92ba8df67222fb4
Catch `Error`, not `BaseError` in `ValuePrinter`
BaseError includes Interrupt. We probably don't want the value printer to tell you Ctrl-C was pressed while it was printing.
(cherry picked from commit c291d2d8dda38aa88b004e2ed05b28653c07e342)
Change-Id: I70b105bfb2f52a8f345ae0281d12f022aa36b14e
`nix eval` forces values and prints derivations as attribute sets, so
commands that print derivations (e.g. `nix eval nixpkgs#bash`) will
infinitely loop and segfault.
Printing derivations as `.drv` paths makes `nix eval` complete as
expected. Further work is needed, but this is better than a segfault.
(cherry picked from commit 4910d74086a85876e093136a0e8ebc547b467af7)
Change-Id: I8e1cb39c05db812080759ec183ee7a131760e6ea
these symbols are used a *lot*, so it makes sense to cache them. this
mostly increases clarity of the code (however clear one may wish to call
the parser desugaring here), but it also provides a small performance
benefit.
(cherry picked from commit 09a1128d9e2ff0ae6176784938047350d6f8a782)
Change-Id: I73d9f66be4555168e048cb2d542277251580c2d1
there's no reason the parser itself should be doing semantic analysis
like bindVars. split this bit apart (retaining the previous name in
EvalState) and have the parser really do *only* parsing, decoupled from
EvalState.
(cherry picked from commit b596cc9e7960b9256bcd557334d81e9d555be5a2)
Change-Id: I481a7623afc783e9d28a6eb4627552cf8a780986
most EvalState and Expr members defined here could be elsewhere, where
they'd be easier to maintain (not being embedded in a file with arcane
syntax) and *somewhat* more faithfully placed according to the path of
the file they're defined in.
(cherry picked from commit e1aa585964c3d864ebff0030584f3349a539d615)
Change-Id: Ibc704567462bb40f37cda05d8fadd465519db5f5
most instances of this being used do not refer to the "current"
position, sometimes not even to one reasonably close by. it could also
be called `makePos` instead, but `at` seems clear in context.
(cherry picked from commit 835a6c7bcfd0b22acc16f31de5fc7bb650d52017)
Change-Id: I17cab8a6cc14cac5b64624431957bfcf04140809
ParserState better describes what this struct really is. the parser
really does modify its state (most notably position and symbol tables),
so calling it that rather than obliquely "data" (which implies being
input only) makes sense.
(cherry picked from commit 007605616477f4f0d8a0064c375b1d3cf6188ac5)
Change-Id: I92feaec796530e1d4d0f7d4fba924229591cea95
all of them need access to parser state in some way. make them members
to allow this without fussing so much.
(cherry picked from commit 1b09b80afac27c67157d4b315c237fa7bb9b8d08)
Change-Id: I3145c95666a5617b735eff7cb403c54c0fe86347
since nix doesn't use the bison `error` terminal anywhere any invocation
of yyerror will immediately cause a failure. since we're *already*
leaking tons of memory whatever little bit bison allocates internally
doesn't much matter any more, and we'll be replacing the parser soon anyway.
coincidentally this now also matches the error behavior of URIs when
they are disabled or ~/ paths in pure eval mode, duplicate attr
detection etc.
(cherry picked from commit e8d9de967fe47a7f9324b0022a2ef50df59f419d)
Change-Id: I560c50d11dceddc2d7cf9ed2c6c631a309ce574e
this is a proper subset of Formals anyway, so let's just use those and
avoid the extra allocations and moves.
(cherry picked from commit f07388bf985c2440413f398cf93d5f5840d1ec8c)
Change-Id: I4508c9c9c918cbaaed649dc753eb86f5cafc7ab6
Print the value in `error: cannot coerce` messages
(cherry picked from commit 5b7bfd2d6b89d7dd5f54c1ca6c8072358d31a84e)
===
test taken from 6e8d5983143ae576e3f4b1d2954a5267f2943a49; it was added
previously (and not backported because its pr was a mostly-revert), but
it's useful to have around.
Change-Id: Icbd14b55e3610ce7b774667bf14b82e6dc717982
libexpr: print value of what is attempted to be called as function
(cherry picked from commit 50e5d7b883042852538371237e32a66bb22f0485)
Change-Id: I7cb6290bd8f244e83bfce3b2eed2a4c8b4f16a83
Print the value in `value is X while a Y is expected` error
(cherry picked from commit 5f72a97092da6af28a7d2b2a50d74e9d34fae7e1)
Change-Id: Idb4bc903ae59a0f5b6fb3b1da4d47970fe0a6efe
Just `stdenv.isDarwin` isn't enough because it doesn't apply to the
build platform, which mean that cross packages building from darwin to
another platform will have `isDarwin` set to false.
Replace it by `stdenv.buildPlatform.isDarwin`.
(cherry picked from commit a0cb75d96f76a3be48b9319e26d8ad78ef4e4525)
(h/t jade for finding this one)
Change-Id: If3cb74e6feaa5d51de550d9a140c71683c2214cd
Previously, there were two mostly-identical value printers -- one in
`libexpr/eval.cc` (which didn't force values) and one in
`libcmd/repl.cc` (which did force values and also printed ANSI color
codes).
This PR unifies both of these printers into `print.cc` and provides a
`PrintOptions` struct for controlling the output, which allows for
toggling whether values are forced, whether repeated values are tracked,
and whether ANSI color codes are displayed.
Additionally, `PrintOptions` allows tuning the maximum number of
attributes, list items, and bytes in a string that will be displayed;
this makes it ideal for contexts where printing too much output (e.g.
all of Nixpkgs) is distracting. (As requested by @roberth in
https://github.com/NixOS/nix/pull/9554#issuecomment-1845095735)
Please read the tests for example output.
Future work:
- It would be nice to provide this function as a builtin, perhaps
`builtins.toStringDebug` -- a printing function that never fails would
be useful when debugging Nix code.
- It would be nice to support customizing `PrintOptions` members on the
command line, e.g. `--option to-string-max-attrs 1000`.
(cherry picked from commit 0fa08b451682fb3311fe58112ff05c4fe5bee3a4, )
===
Restore ambiguous value printer for `nix-instantiate`
The Nix team has requested that this output format remain unchanged.
I've added a warning to the man page explaining that `nix-instantiate
--eval` output will not parse correctly in many situations.
(cherry picked from commit df84dd4d8dd3fd6381ac2ca3064432ab31a16b79)
Change-Id: I7cca6b4b53cd0642f2d49af657d5676a8554c9f8
(cherry picked from commit 561a56cd13b4f12e3dfb6c5e3f42e5d8add04ecc)
===
Modified the release notes' synopsis to make it match its contents,
probably a copy-paste.
Co-authored-by: Raito Bezarius <raito@lix.systems>
Change-Id: I03bbff940b93e7df4b6c2fe9159c49a59ed47b55
Don't attempt to `git add` ignored files
(cherry picked from commit 359990dfdc713c80aabd7ea6f7e4528628fbe108)
===
also added a regression test that isn't upstream to be sure we're
actually fixing the bug.
Change-Id: I8267a3d0ece9909d8008b7435b90e7b3eee366f6
Change an `allowPath` call to take a store path again
(cherry picked from commit 7c9ed1b1a325fe64a186e9d454607eaa0a7e8951)
Change-Id: Ia5ec924315a1f2640a0438cfb4b1ee0689cd3558
Factor out `ServeProto::Serialiser<UnkeyedValidPathInfo>` and test
(cherry picked from commit 139982997eec493a0f74105c427953f6be77da6d)
Change-Id: I28e4ba5a681a90d81915a56e6dbaa5456d64f96d
structured attrs: improve support / usage of NIX_ATTRS_{SH,JSON}_FILE
(cherry picked from commit 3c042f3b0b0a7ef9c47bf049f5410dbd4aac9e90)
Change-Id: I7e41838338ee1edf31fff6f9e354c3db2bba6c0e
Also be more consistent with quotes around attribute paths
(cherry picked from commit 9404ce36e4edd1df12892089bdab1ceb7d4d7a97)
Change-Id: Iaa80073b4a07a6ffef106a3c12ecd02b4f6f67aa
Restore `builtins.pathExists` behavior on broken symlinks
(cherry picked from commit d53c8901ef7f2033855dd99063522e3d56a19dab)
===
note that this variant differs markedly from the source commit because
we haven't endured quite as much lazy trees.
Change-Id: I0facf282f21fe0db4134be5c65a8368c1b3a06fc
absPath: Explicitly check if path is empty before accessing it
(cherry picked from commit 6ec08b85f607852eb6f976c1392c4917d0a53787)
Change-Id: Ieeb53fb65d0e334e6017ceb3a48b3b6ae1047843
Include phase reporting in log file for ssh-ng builds
(cherry picked from commit b1e7d7cad625095656fff05ac4aedeb12135110a)
Change-Id: I4076669b0ba160412f7c628ca9113f9abbc8c303
It is possible to exfiltrate a file descriptor out of the build sandbox
of FODs, and use it to modify the store path after it has been
registered. To avoid that issue, don't register the output of the build,
but a copy of it (that will be free of any leaked file descriptor).
Test that we can't leverage abstract unix domain sockets to leak file
descriptors out of the sandbox and modify the path after it has been
registered.
(cherry picked from commit 2dadfeb690e7f4b8f97298e29791d202fdba5ca6)
(tests cherry picked from commit c854ae5b3078ac5d99fa75fe148005044809e18c)
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Theophane Hufschmitt <theophane.hufschmitt@tweag.io>
Co-authored-by: Tom Bereknyei <tomberek@gmail.com>
Change-Id: I87cd58f1c0a4f7b7a610d354206b33301e47b1a4
Combine `AbstractPos`, `PosAdapter`, and `Pos`
(cherry picked from commit 113499d16fc87d53b73fb62fe6242154909756ed)
===
this is a bit cursed because originally it was based on InputAccessor
code that we don't have and moved/patched features we likewise don't
have (fetchToStore caching, all the individual accessors,
ContentAddressMethod). the commit is adjusted accordingly to
match (remove caching, ignore accessors, use FileIngestionMethod).
note that `state.rootPath . CanonPath == abs` and
computeStorePathForPath works relative to cwd, so the slight rewrite in
the moved fetchToStore is legal.
Change-Id: I05fd340c273f0bcc8ffabfebdc4a88b98083bce5
Increase stack size on macOS as well as Linux
(cherry picked from commit efb91d5979a625d5c50558aeabfd24e802ed9173,
4a2444b3f32a2f5d42c4d65302793b987d1ac667)
Change-Id: Ieb72283c61bb9e360683f531d6635697b293c313
Docs build: depend on locally built nix executable and not installed one
(cherry picked from commit ca72e3e7e8f69526f028475a7a9b40812da1acdd)
===
includes changes from (because not doing so removes manpages):
Merge pull request #9976 from alois31/restore-manual-pages
Restore manual pages
(cherry picked from commit d3c1997127e0fc08576e842b2bfe046d8a28d2f4)
Change-Id: I685ff16163ac552a1754570c03c992c63a461d50
Including `config.h` also needs `$(buildprefix)`
(cherry picked from commit 96fdea3394ff61e24c53358644a5064218218d13)
Change-Id: I8b5c0b1826aa007aa681c8b199f9b1489cac6784
Define NixOS tests in `tests/nixos/default.nix` rather than `flake.nix`
(cherry picked from commit c29b8ba142a0650d1182ca838ddc1b2d273dcd2a)
Change-Id: Ieae1b6476d95024485df7067e008013bc5542039
Add position information to `while evaluating the attribute` errors in the debugger
(cherry picked from commit ffe67c86a8ef3695e5c8b9c9800c192ac633dded)
Change-Id: I177ea5ec60898abe09fb9d80d9602b2a32ff8f44
Fix "Failed tcsetattr(TCSADRAIN)" when `nix repl` is not a TTY
(cherry picked from commit 864fc85fc88ff092725ba99907611b2b8d2205fb)
Change-Id: I8198674b935fabd741a349cc74544e61c53ea7b3
`nix`: Fix `haveInternet` to check for proxy
(cherry picked from commit accae60e7710a18f6f2bd7d2f4cd836bcd76b684)
Change-Id: I996dafdcd266f4bc5806386c86b19040120842bf
Say how many channels were unpacked in nix-channel
(cherry picked from commit 9ae665b9e1dc64c507ab6002fc5d7824208f3777)
Change-Id: Ie0950cf32123b550c5b83981a020e513f72a9b7c
When reviewing old PRs, I found that #9997 adds some code to ensure one
particular assert is always present. But, removing asserts isn't
something we do in our own release builds either in the flake here or in
nixpkgs, and is plainly a bad idea that increases support burden,
especially if other distros make bad choices of build flags in their Nix
packaging.
For context, the assert macro in the C standard is defined to do nothing
if NDEBUG is set.
There is no way in our build system to set -DNDEBUG without manually
adding it to CFLAGS, so this is simply a configuration we do not use.
Let's ban it at compile time.
I put this preprocessor directive in src/libutil.cc because it is not
obvious where else to put it, and it seems like the most logical file
since you are not getting a usable nix without it.
Upstream-PR: https://github.com/NixOS/nix/pull/10126
Original-Change-Id: I513cceaac1371decb3d96231e6ef9181c910c218
Change-Id: I531a51f6348a746e8e41d88203b08f614898356c
Expose locals from `let` expressions to the debugger
(cherry picked from commit acef4f17a2daab4ccdf656bdf229792db2f779e4)
Change-Id: Ib3623254f67ac762f4e7230d625e9f87dff38a84
Print positions in `--debugger`, instead of pointers
(cherry picked from commit 49cf090cb2f51d6935756a6cf94d568cab063f81)
Change-Id: Ic27917b2aab3657c28d599898377bf0c14753f8a
Color `diff` output in `tests/functional/lang` tests
(cherry picked from commit 1dc55c0f2f034bce6e3de4a5cda96d686b10a7f8)
Change-Id: Ie9b3fc3446bd3caa0fd8885de88639516a2ff862
Make `StoreConfig::getDefaultSystemFeatures` a static method
(cherry picked from commit 5a9513cdbae31ea5e6f6e7afa7b3c2e3a9a26474)
Change-Id: Ia9c0ae2b7de419bd60aea8bf905154b96c428276
Convert `Machine::speedFactor` from a non-neg int to a non-neg float
(cherry picked from commit 69d0ae27e376e7c7c4f237716b0149223b8a805a)
Change-Id: I2afb5cf9e4fe1384985c58353946135c3d102b42
Make `Machine::systemTypes` a set not vector
(cherry picked from commit f1b030415376e81c5804647c055d71eaba4aa725)
Change-Id: I6d4f5c0bfc226e9bd66c58c360cd99e3fac9a129
Fix crash when NAR is missing from binary cache
(cherry picked from commit 3b20cca9625a1701a10a883735e7315185629563)
Change-Id: I50ff18f4a6de69c323473b4a8e3e098d1f365145
Print a more helpful message if the daemon crashes
(cherry picked from commit 32706b14a7531c2c21b9f96da083a540a0031ec4)
Change-Id: Ief7c465bca7666e2b7e7c9d1dd0c01c5f9014146
Store: :buildPaths(): Fix display of store paths
(cherry picked from commit b5ed36e6633cac844fe4388dcc0cc8055a18ef9e)
Change-Id: Ic6008491088dc6febd4a1e44dc2dbb96c47661f4
tests/functional/lang: Test substring with negative length
(cherry picked from commit 86156d05dd33f856d8804f89669a7fe9b81f1a0d)
Change-Id: I2e2086027a43f8111ba5068ac16590eaa0b798d4
Fix performance of builtins.substring for empty substrings
(cherry picked from commit b2deff1947c2fe57fdbf1a472eb9003eb407f8d3)
Change-Id: I4ddfc8d26a4781c9520fff9807849a073ee7bed8
nix shell: reflect command line order in PATH order
(cherry picked from commit b91c935c2faf08ced2c763dcd2a831f26d84fa86)
Change-Id: If16c120bb74857c2817366e74e5b0877eb997260
Improve error message for fixed-outputs with references.
(cherry picked from commit ff6de4a9ee6c3862db9ee5f09ff9c3f43ae7a088)
Change-Id: I733c49760b9a3f1b76a6bece3b250b8579cd6cac
flake: Go back to regular `nixos-23.05-small`
(cherry picked from commit 1ed245a60672c123c1348a63061fb4d64fb95212)
Change-Id: I33f5fe20cd6bc658a461f560e458b440b3b7e18e
withFramedSink(): Receive interrupts on the stderr thread
(cherry picked from commit 965cfe96886c988c3aa94bfc7fefdd37325f4536)
Change-Id: I8320a96957c01ec0e3450d1b3ae38a3baff78d49
Fix segfault on infinite recursion in some cases
(cherry picked from commit bf1b294bd81ca76c5ec9fe3ecd52196bf52a8300)
Change-Id: Id137541426ec8536567835953fccf986a3aebf16
installer: allow overriding of NIX_FIRST_BUILD_ID on darwin
(cherry picked from commit e85fd92816571ea00abafa3929298d0e091bcb9b)
Change-Id: Ifffc3fedd740079345c205f54c62c76053e24846
Allow access to /dev/stderr in Darwin sandbox
(cherry picked from commit c6d7013583c568590aff285fb7414d1675a745f4)
Change-Id: I5657f6f4ee9dad8c978bad0d71f5cac51584e4f2
Fix building CA derivations with and eval store
(cherry picked from commit dfc0cee7024a082d90a4f68296f55a82dfd52126)
Change-Id: I28feb5a36d4fe75f0ed3e3e2db6eb56b67d0f371
Give `Store::queryDerivationOutputMap` and `evalStore` argument
(cherry picked from commit 8cddda4f892cb42be43e9bd87aa0111572617e78)
Change-Id: I394e7e11c3f2e0cd3dbe0f48d757c14c09835e44
libstore/daemon.cc: note trust model difference in readDerivation()s
(cherry picked from commit 5c917c32048ef185ea0eec352c3505485aa3212c)
Change-Id: I9945bc84e9529b005eafdc5c08b5bf1553335340
rl-next: Fix and support markdown frontmatter syntax
(cherry picked from commit 69b7876a0810269ad71807594cfd99b26cd8a5ff)
Change-Id: I8bfb8967af0943080fdd70d257c34abaf0a9fedf
Give `Derivation::tryResolve` an `evalStore` argument
(cherry picked from commit 36ca6adc60511dc822870f2df43c0a578e481925)
Change-Id: If76b185a01ffa982e4c49cf333a9b5fbf9edebfe
Add option to libarchive so it behaves correctly
(cherry picked from commit c3827ff6348a4d5199eaddf8dbc2ca2e2ef46ec5)
Change-Id: Ib0f928851093f4c644bac071d1c8f8aeec803198
tests: avoid a chroot store without sandbox support
(cherry picked from commit 5910140f252280f6be429d1cb5e91e69999f5b43)
Change-Id: Ic2255261334012f36a465a96073f50669952dc26
config: add included files into parsedContents before applying
(cherry picked from commit 82359eba6b692691ef08a71196ef25a61bc4d3d3)
Change-Id: Idde3177010fec7b8bafe6088c3c23d5caf491845
nix repl: Only hide the progress bar while waiting for user input
(cherry picked from commit 3bebaefcd0c5d650f7edcd39f397bb45c4382f41)
Change-Id: Ie7c0db46f7c2cf5f938e66bdd3c31f0b62bdb104
This allows templates such as `NLOHMANN_DEFINE_TYPE_*` templates and other generators with things like `std::vector<std::optional<T>>`.
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
(cherry picked from commit 02bd821f2e71372d31bbe6700bd68086cc2ee70a)
Change-Id: I8b0ebcf2af4226610dadd565962f2d2327415a03
flakes: bare minimum fix the error message for untracked flake.nix
(cherry picked from commit 7f626dba332f320fafb9f9c749986ea523d20f42)
Change-Id: I470ac56a670a8c11e4164c6b059184a02344d491
Revert use of boost::container::small_vector in the evaluator
(cherry picked from commit 6832d18ac734f4b855f97c07b158491dd01cefcd)
Change-Id: I7f10af0c8b8a8beb4b1e36424120995f4ed82738
Unlock output paths when a derivation is already built
(cherry picked from commit 7ba4e073e8622ca86b52e03d68476e80250ab62f)
Change-Id: I9de077679290d5141a610ac43d99d3a43acff87c
* Fix boost::bad_format_string exception in builtins.addErrorContext
The message passed to addTrace was incorrectly being used as a format
string and this this would cause an exception when the string contained
a '%', which can be hit in places where arbitrary file paths are
interpolated.
* add test
(cherry picked from commit 61d6fe059e959455e156c1d57bb91155d363e983)
Change-Id: Idd671127a9c1ccc8b94e58e727632fcc064f3cbe
fix: make sure `tar` reproducibility flags are set
(cherry picked from commit 28dddde0aca978114eaef00a14a2ab14b2459f4a)
Change-Id: I57c4d4374f5195099e6d763827b6d7d05785b3a8
fix: gcc complains about if which doesn't guard the indented statement
(cherry picked from commit 8d663462938a333a4e81cce1005437f141cd11fa)
Change-Id: Ifa2e65502de4000935549dde82ab1b5867e2f0ed
Turn derivation unit tests into unit characterization tests
(cherry picked from commit a6e587923c9d5d716fe0f0049bed96d1cc210bff)
Change-Id: Ia2a2e65aabfee8d5d52142b8fdaacbae4a27242c
Bindmount files instead of hardlinking or copying to chroot
(cherry picked from commit 622191c2b53882a1675fed5066ff8090b4f01827)
Change-Id: I278ec1baacdfa9044992b58fdec8f14d6d7d09ce
nix-shell: support single quotes in shebangs, fix whitespace parsing
(cherry picked from commit 3b99c6291377cbd22607896af9dfafa857d2f2dc)
Change-Id: I2a431b21c3467eefa1ef95d5a36d672f45b6937a
Give `nix daemon` and `nix-store --serve` protocols separate serializers with version info
(cherry picked from commit 8b68bbb77745fda0d14939b6c23d31cc89da41ce)
Change-Id: Ia3d3b9fbaf9f0ae62ab225020b7d14790e793655
Improve tests and docs prior to refactoring completions
(cherry picked from commit 5442d9b47298389918d1f38d20f768a80ffc2369)
Change-Id: Ief99ac2cd9c92981a9a522d15b9c3daf99182c9d
Factor out bits of the worker protocol to use elsewhere
(cherry picked from commit 4b1a97338f517f45e6169d3d8845c5caa5724e97)
Change-Id: If93afa0f8b1cf9b0e705b34fa71e6fd708752758
Use positive source filtering for the standalone functional tests job and Perl bindings
(cherry picked from commit 6b6bd9003062c86a49d4384381941cf57f269c45)
Change-Id: I896be67654f893d543ed6beb5d0d0d6c6d36e027
Don't run the tests that require building if we're not building
(cherry picked from commit 8cfa582f436db8066eff74cb084990367e014ce6)
Change-Id: I6085ca6107349669407340d7a5e52639a2febc90
Enable most of the third `BuildResult` worker protocol test
(cherry picked from commit d344c112f772282bacacd4c66a75df4022d16e12)
Change-Id: I7b2b72aa84c19a6069f9c12128d901262db6f91c
Test the rest of the worker protocol serializers
(cherry picked from commit 2f1c16dfa2378fd8616bff1b9b7cd0b4d42af69b)
Change-Id: Idfd72d32b21d14a260e02f65531d287cef7464d2
Unit test some worker protocol serializers
(cherry picked from commit c6faef61a6f31c71146aee5d88168e861df9a22a)
Change-Id: I99e36f5f17eb7642211a4e42a16b143424f164b4
do not show configuration override flags for each command
(cherry picked from commit f89b84919c1a5c796512c50311821e7779b3678b)
Change-Id: Ib98b739bd6c9a1e94f94a78a47d84d72e435e7c0
2024-03-04 04:35:54 +01:00
1153 changed files with 34663 additions and 36682 deletions
- To remove the stale label, just leave a new comment.
- _How to find the right people to ping?_→ [`git blame`](https://git-scm.com/docs/git-blame) to the rescue! (or GitHub's history and blame buttons.)
- You can always ask for help on [our Discourse Forum](https://discourse.nixos.org/) or on [Matrix - #nix:nixos.org](https://matrix.to/#/#nix:nixos.org).
## Suggestions for PRs
1. GitHub sometimes doesn't notify people who commented / reviewed a PR previously, when you (force) push commits. If you have addressed the reviews you can [officially ask for a review](https://docs.github.com/en/free-pro-team@latest/github/collaborating-with-issues-and-pull-requests/requesting-a-pull-request-review) from those who commented to you or anyone else.
2. If it is unfinished but you plan to finish it, please mark it as a draft.
3. If you don't expect to work on it any time soon, closing it with a short comment may encourage someone else to pick up your work.
4. To get things rolling again, rebase the PR against the target branch and address valid comments.
5. If you need a review to move forward, ask in [the Discourse thread for PRs that need help](https://discourse.nixos.org/t/prs-in-distress/3604).
6. If all you need is a merge, check the git history to find and [request reviews](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/requesting-a-pull-request-review) from people who usually merge related contributions.
## Suggestions for issues
1. If it is resolved (either for you personally, or in general), please consider closing it.
2. If this might still be an issue, but you are not interested in promoting its resolution, please consider closing it while encouraging others to take over and reopen an issue if they care enough.
3. If you still have interest in resolving it, try to ping somebody who you believe might have an interest in the topic. Consider discussing the problem in [our Discourse Forum](https://discourse.nixos.org/).
4. As with all open source projects, your best option is to submit a Pull Request that addresses this issue. We :heart: this attitude!
**Memorandum on closing issues**
Don't be afraid to close an issue that holds valuable information. Closed issues stay in the system for people to search, read, cross-reference, or even reopen--nothing is lost! Closing obsolete issues is an important way to help maintainers focus their time and effort.
## Useful GitHub search queries
- [Open PRs with any stale-bot interaction](https://github.com/NixOS/nix/pulls?q=is%3Apr+is%3Aopen+commenter%3Aapp%2Fstale+)
- [Open PRs with any stale-bot interaction and `stale`](https://github.com/NixOS/nix/pulls?q=is%3Apr+is%3Aopen+commenter%3Aapp%2Fstale+label%3A%22stale%22)
- [Open PRs with any stale-bot interaction and NOT `stale`](https://github.com/NixOS/nix/pulls?q=is%3Apr+is%3Aopen+commenter%3Aapp%2Fstale+-label%3A%22stale%22+)
- [Open Issues with any stale-bot interaction](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+commenter%3Aapp%2Fstale+)
- [Open Issues with any stale-bot interaction and `stale`](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+commenter%3Aapp%2Fstale+label%3A%22stale%22+)
- [Open Issues with any stale-bot interaction and NOT `stale`](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+commenter%3Aapp%2Fstale+-label%3A%22stale%22+)
Welcome and thank you for your interest in contributing to Nix!
We appreciate your support.
Welcome and thank you for considering contributing to Lix! We're currently in a soft release phase, and your support means a lot to us.
Reading and following these guidelines will help us make the contribution process easy and effective for everyone involved.
To ensure a smooth and effective contribution process, here is a summary of our guidelines:
## Getting help?
If you have any question regarding getting started or reporting bugs, feel free
to reach out to us.
On Matrix, we have a space at `#space:lix.systems`, composed of:
- [`#discuss:lix.systems`](https://matrix.to/#/#discuss:lix.systems) for discussions on Lix.
- [`#dev:lix.systems`](https://matrix.to/#/#dev:lix.systems) for the development channel on Lix.
## Report a bug
1. Check on the [GitHub issue tracker](https://github.com/NixOS/nix/issues) if your bug was already reported.
- Check if your bug has already been reported in the [issue tracker](https://git.lix.systems/lix-project/lix/issues).
- If you can't find the bug or feature, please open a new issue.
2. If you were not able to find the bug or feature [open a new issue](https://github.com/NixOS/nix/issues/new/choose)
3. The issue templates will guide you in specifying your issue.
The more complete the information you provide, the more likely it can be found by others and the more useful it is in the future.
Make sure reported bugs can be reproduced easily.
4. Once submitted, do not expect issues to be picked up or solved right away.
The only way to ensure this, is to [work on the issue yourself](#making-changes-to-nix).
We maintain a copy of the upstream Nix bugs. Their organisation can be read about [here](https://wiki.lix.systems/books/lix-contributors/page/bug-tracker-organisation).
## Report a security vulnerability
Check out the [security policy](https://github.com/NixOS/nix/security/policy).
For security vulnerabilities, reach out by email at `security at lix dot systems`.
## Making changes to Nix
## Making changes to Lix
1. Check for [pull requests](https://github.com/NixOS/nix/pulls) that might already cover the contribution you are about to make.
There are many open pull requests that might already do what you intent to work on.
You can use [labels](https://github.com/NixOS/nix/labels) to filter for relevant topics.
Before diving into making changes, we want to engage with you and your ideas.
2. Search for related issues that cover what you're going to work on. It could help to mention there that you will work on the issue.
We have a few policies in effect; please take the time to familiarize yourself:
Issues labeled [good first issue](https://github.com/NixOS/nix/labels/good-first-issue) should be relatively easy to fix and are likely to get merged quickly.
Pull requests addressing issues labeled [idea approved](https://github.com/NixOS/nix/labels/idea%20approved) are especially welcomed by maintainers and will receive prioritised review.
- [Style guide on code](https://wiki.lix.systems/books/lix-contributors/page/code)
- [Freeze policy and recommended contributions](https://wiki.lix.systems/books/lix-contributors/page/freezes-and-recommended-contributions)
3. Check the [Nix reference manual](https://nixos.org/manual/nix/unstable/contributing/hacking.html) for information on building Nix and running its tests.
To avoid duplication of effort, it may be a good idea to check out the list of
[pending pull requests](https://gerrit.lix.systems/q/status:open+-is:wip) (or "change lists", as Gerrit calls them). Once you have
an idea of what you might want to do, we recommend dropping a message on our
Matrix to ensure your contribution fits with our current schedule and plans
For contributions to the command line interface, please check the [CLI guidelines](https://nixos.org/manual/nix/unstable/contributing/cli-guideline.html).
When you're ready and your changes are ready to go:
4. Make your changes!
- Submit your code.
- Submitting a GitHub PR [on our mirror](https://github.com/lix-project/lix) is totally ok if that's easier for you and your change is relatively small (300 lines or so).
5. [Create a pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request) for your changes.
* Link related issues in your pull request to inform interested parties and future contributors about your change.
* Make sure to have [a clean history of commits on your branch by using rebase](https://www.digitalocean.com/community/tutorials/how-to-rebase-and-update-a-pull-request).
If your pull request closes one or multiple issues, note that in the description using `Closes: #<number>`, as it will then happen automatically when your change is merged.
* [Mark the pull request as draft](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/changing-the-stage-of-a-pull-request) if you're not done with the changes.
We may ask you to resubmit it as a Gerrit CL if it is necessary for the change you're making.
- Our primary code review system is [our Gerrit instance](https://gerrit.lix.systems), where you can open a change list (CL).
If you're new to Gerrit, check out [our wiki page about Gerrit](https://wiki.lix.systems/books/lix-contributors/page/gerrit).
- Make sure to link any related issues.
- If needed, indicate that the change is 'work in progress'.
6. Do not expect your pull request to be reviewed immediately.
Nix maintainers follow a [structured process for reviews and design decisions](https://github.com/NixOS/nix/tree/master/maintainers#project-board-protocol), which may or may not prioritise your work.
Following this checklist will make the process smoother for everyone:
- [ ] Fixes an [idea approved](https://github.com/NixOS/nix/labels/idea%20approved) issue
- [ ] User documentation in the [manual](..doc/manual/src)
- [ ] API documentation in header files
- [ ] Code and comments are self-explanatory
- [ ] Commit message explains **why** the change was made
- [ ] New feature or incompatible change: updated [release notes](./doc/manual/src/release-notes/rl-next.md)
7. If you need additional feedback or help to getting pull request into shape, ask other contributors using [@mentions](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#mentioning-people-and-teams).
## Making changes to the Nix manual
The Nix reference manual is hosted on https://nixos.org/manual/nix.
The underlying source files are located in [`doc/manual/src`](./doc/manual/src).
For small changes you can [use GitHub to edit these files](https://docs.github.com/en/repositories/working-with-files/managing-files/editing-files)
For larger changes see the [Nix reference manual](https://nixos.org/manual/nix/unstable/contributing/hacking.html).
## Getting help
Whenever you're stuck or do not know how to proceed, you can always ask for help.
The appropriate channels to do so can be found on the [NixOS Community](https://nixos.org/community/) page.
You can obtain an account on our platforms by clicking "Sign In with GitHub" on the sign-in page.
**Lix** is an implementation of **Nix**, a powerful package management system for Linux and other Unix systems that makes package management reliable and reproducible.
Nix is a powerful package manager for Linux and other Unix systems that makes package
management reliable and reproducible. Please refer to the [Nix manual](https://nixos.org/nix/manual)
for more details.
Read more about us at https://lix.systems.
## Installation
On Linux and macOS the easiest way to install Nix is to run the following shell command
On Linux and macOS the easiest way to install Lix is to run the following shell command
(as a user other than root):
```console
$ curl -L https://nixos.org/nix/install | sh
$ curl -sSf -L https://install.lix.systems/lix | sh -s -- install
```
Information on additional installation methods is available on the [Nix download page](https://nixos.org/download.html).
For systems that **already have a Nix implementation installed**, such as NixOS systems, read our [install page](https://lix.systems/install)
## Building And Developing
See our [Hacking guide](https://nixos.org/manual/nix/unstable/contributing/hacking.html) in our manual for instruction on how to
to set up a development environment and build Nix from source.
See our [Hacking guide](https://git.lix.systems/lix-project/lix/src/branch/main/doc/manual/src/contributing/hacking.md) in our manual for instruction on how to to set up a development environment and build Lix from source.
## Additional Resources
- [Nix manual](https://nixos.org/nix/manual)
- [Nix jobsets on hydra.nixos.org](https://hydra.nixos.org/project/nix)
[AC_MSG_ERROR([Nix requires libeditline; it was not found via pkg-config, but via its header, but required functions do not work. Maybe it is too old? >= 1.14 is required.])])
AS_HELP_STRING([--disable-cpuid], [Do not determine microarchitecture levels with libcpuid (relevant to x86_64 only)]))
if test "x$enable_cpuid" != "xno"; then
PKG_CHECK_MODULES([LIBCPUID], [libcpuid],
[CXXFLAGS="$LIBCPUID_CFLAGS $CXXFLAGS"
have_libcpuid=1
AC_DEFINE([HAVE_LIBCPUID], [1], [Use libcpuid])]
)
fi
fi
AC_SUBST(HAVE_LIBCPUID, [$have_libcpuid])
# Look for libseccomp, required for Linux sandboxing.
case "$host_os" in
linux*)
AC_ARG_ENABLE([seccomp-sandboxing],
AS_HELP_STRING([--disable-seccomp-sandboxing],[Don't build support for seccomp sandboxing (only recommended if your arch doesn't support libseccomp yet!)
]))
if test "x$enable_seccomp_sandboxing" != "xno"; then
PKG_CHECK_MODULES([LIBSECCOMP], [libseccomp],
[CXXFLAGS="$LIBSECCOMP_CFLAGS $CXXFLAGS"])
have_seccomp=1
AC_DEFINE([HAVE_SECCOMP], [1], [Whether seccomp is available and should be used for sandboxing.])
else
have_seccomp=
fi
;;
*)
have_seccomp=
;;
esac
AC_SUBST(HAVE_SECCOMP, [$have_seccomp])
# Look for aws-cpp-sdk-s3.
AC_LANG_PUSH(C++)
AC_CHECK_HEADERS([aws/s3/S3Client.h],
[AC_DEFINE([ENABLE_S3], [1], [Whether to enable S3 support via aws-sdk-cpp.]) enable_s3=1],
[AC_DEFINE([ENABLE_S3], [0], [Whether to enable S3 support via aws-sdk-cpp.]) enable_s3=])
Thisfunctionisonlyavailableifthe[${experimental-feature}](@docroot@/contributing/experimental-features.md#xp-feature-${experimental-feature}) experimental feature is enabled.
Nix supports remote builds, where a local Nix installation can forward
Lix supports remote builds, where a local Lix installation can forward
Nix builds to other machines. This allows multiple builds to be
performed in parallel and allows Nix to perform multi-platform builds in
performed in parallel and allows Lix to perform multi-platform builds in
a semi-transparent way. For instance, if you perform a build for a
`x86_64-darwin` on an `i686-linux` machine, Nix can automatically
`x86_64-darwin` on an `i686-linux` machine, Lix can automatically
forward the build to a `x86_64-darwin` machine, if available.
To forward a build to a remote machine, it’s required that the remote
@ -38,12 +38,15 @@ contains Nix.
> **Warning**
>
> If you are building via the Nix daemon, it is the Nix daemon user account (that is, `root`) that should have SSH access to a user (not necessarily `root`) on the remote machine.
> If you are building via the Lix daemon (default on Linux and macOS), it is the Lix daemon user account (that is, `root`) that should have SSH access to a user (not necessarily `root`) on the remote machine.
>
> Furthermore, `root` needs to have the public host keys for the remote system in its `.ssh/known_hosts`.
> To add them to `known_hosts` for root, do `ssh-keyscan USER@HOST | sudo tee -a ~root/.ssh/known_hosts`.
>
> If you can’t or don’t want to configure `root` to be able to access the remote machine, you can use a private Nix store instead by passing e.g. `--store ~/my-nix` when running a Nix command from the local machine.
The list of remote machines can be specified on the command line or in
the Nix configuration file. The former is convenient for testing. For
the Lix configuration file. The former is convenient for testing. For
example, the following command allows you to build a derivation for
`x86_64-darwin` on a Linux machine:
@ -84,17 +87,17 @@ default, set it to `-`.
3. The SSH identity file to be used to log in to the remote machine. If
omitted, SSH will use its regular identities.
4. The maximum number of builds that Nix will execute in parallel on
4. The maximum number of builds that Lix will execute in parallel on
the machine. Typically this should be equal to the number of CPU
cores. For instance, the machine `itchy` in the example will execute
up to 8 builds in parallel.
5. The “speed factor”, indicating the relative speed of the machine. If
there are multiple machines of the right type, Nix will prefer the
there are multiple machines of the right type, Lix will prefer the
fastest, taking load into account.
6. A comma-separated list of *supported features*. If a derivation has
the `requiredSystemFeatures` attribute, then Nix will only perform
the `requiredSystemFeatures` attribute, then Lix will only perform
the derivation on a machine that has the specified features. For
Nix supports a variety of configuration settings, which are read from configuration files or taken as command line flags.
Lix supports a variety of configuration settings, which are read from configuration files or taken as command line flags.
## Configuration file
By default Nix reads settings from the following places, in that order:
By default Lix reads settings from the following places, in that order:
1. The system-wide configuration file `sysconfdir/nix/nix.conf` (i.e. `/etc/nix/nix.conf` on most systems), or `$NIX_CONF_DIR/nix.conf` if [`NIX_CONF_DIR`](./env-common.md#env-NIX_CONF_DIR) is set.
@ -17,7 +17,7 @@ By default Nix reads settings from the following places, in that order:
1. If [`NIX_USER_CONF_FILES`](./env-common.md#env-NIX_USER_CONF_FILES) is set, then each path separated by `:` will be loaded in reverse order.
Otherwise it will look for `nix/nix.conf` files in `XDG_CONFIG_DIRS` and [`XDG_CONFIG_HOME`](./env-common.md#env-XDG_CONFIG_HOME).
Otherwise it will look for `nix/nix.conf` files in `XDG_CONFIG_DIRS` and `XDG_CONFIG_HOME`.
If unset, `XDG_CONFIG_DIRS` defaults to `/etc/xdg`, and `XDG_CONFIG_HOME` defaults to `$HOME/.config` as per [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html).
1. If [`NIX_CONFIG`](./env-common.md#env-NIX_CONFIG) is set, its contents are treated as the contents of a configuration file.
@ -68,3 +68,4 @@ The `extra-` prefix is supported for settings that take a list of items (e.g. `-
If `NIX_PATH` is not set at all, Nix will fall back to the following list in [impure](@docroot@/command-ref/conf-file.md#conf-pure-eval) and [unrestricted](@docroot@/command-ref/conf-file.md#conf-restrict-eval) evaluation mode:
If `NIX_PATH` is not set at all, Lix will fall back to the following list in [impure](@docroot@/command-ref/conf-file.md#conf-pure-eval) and [unrestricted](@docroot@/command-ref/conf-file.md#conf-restrict-eval) evaluation mode:
julia
jade
alois31
Pierre Bourdon
Qyriad
Mario Rodas
Linus Heckemann
Nikodem Rabuliński
raito
Lunaphied
terru -
Maximilian Bosch
Ilya K
eldritch horrors
Tom Hubrecht
Artemis Tosini
puck
Yorick
FireFly
Patrick Jackson
Robert Hensing
Rebecca Turner
Jade Lovelace
Winter Cute