not doing this will cause transfers that had their readers disappear to
linger. with lingering transfers the curl thread can't shut down, which
will cause nix itself to not shut down until the transfer finishes some
other way (most likely network timeouts). also add a new test for this.
Change-Id: Id2401b3ac85731c824db05918d4079125be25b57
If unprivileged userns are *believed* to be disabled (such as with
"kernel.unprivileged_userns_clone = 0"), Lix would previously *give up*
on trying to use a user namespace before actually trying it, even if, in
cases such as unprivileged_userns_clone, it would actually be allowed
since Nix has CAP_SYS_ADMIN when running as daemon.
(see, e.g. 25d4709a4f)
We changed it to actually try it first, and then diagnose possible
causes, and also to be more loud about the whole thing, using warnings
instead of debugs. These warnings will only print on the first build run
by the daemon, which is, tbh, eh, shrug.
This is what led to us realizing that no-userns was a poorly exercised
condition.
Change-Id: I8e4f21afc89c574020dc7e89a560cc740ce6573a
This was found when `logrotate.conf` failed to build in a NixOS system
with:
/nix/store/26zdl4pyw5qazppj8if5lm8bjzxlc07l-coreutils-9.3/bin/id: cannot find name for group ID 30000
This was surprising because it seemed to mean that /etc/group was busted
in the sandbox. Indeed it was:
root❌0:
nixbld:!💯
nogroup❌65534:
We diagnosed this to sandboxUid() being called before
usingUserNamespace() was called, in setting up /etc/group inside the
sandbox. This code desperately needs refactoring.
We also moved the /etc/group code to be with the /etc/passwd code, but
honestly this code is all spaghetti'd all over the place and needs some
more serious tidying than we did here.
We also moved some checks to be earlier to improve locality with where
the things they are checking come from.
Change-Id: Ie29798771f3593c46ec313a32960fa955054aceb
This reverts commit a8b3d777fb.
This undoes the revert of PR#6621, which allows nested `follows`, i.e.
{
inputs = {
foo.url = "github:bar/foo";
foo.inputs.bar.inputs.nixpkgs = "nixpkgs";
};
}
does the expected thing now. This is useful to avoid the 1000 instances
of nixpkgs problem without having each flake in the dependency tree to
expose all of its transitive dependencies for modification.
This was in fact part of Nix before and the C++ changes applied w/o
conflicts. However, it got reverted then because people didn't want to
merge lazy-trees against it which was supposed to be merged soon back in
October 2022.
Fixes: lix-project/lix#201
Change-Id: I5ddef914135b695717b2ef88862d57ced5e7aa3c
With Linux kernel >=6.6 & glibc 2.39 a `fchmodat2(2)` is available that
isn't filtered away by the libseccomp sandbox.
Being able to use this to bypass that restriction has surprising results
for some builds such as lxc[1]:
> With kernel ≥6.6 and glibc 2.39, lxc's install phase uses fchmodat2,
> which slips through 9b88e52846/src/libstore/build/local-derivation-goal.cc (L1650-L1663).
> The fixupPhase then uses fchmodat, which fails.
> With older kernel or glibc, setting the suid bit fails in the
> install phase, which is not treated as fatal, and then the
> fixup phase does not try to set it again.
Please note that there are still ways to bypass this sandbox[2] and this is
mostly a fix for the breaking builds.
This change works by creating a syscall filter for the `fchmodat2`
syscall (number 452 on most systems). The problem is that glibc 2.39
is needed to have the correct syscall number available via
`__NR_fchmodat2` / `__SNR_fchmodat2`, but this flake is still on
nixpkgs 23.11. To have this change everywhere and not dependent on the
glibc this package is built against, I added a header
"fchmodat2-compat.hh" that sets the syscall number based on the
architecture. On most platforms its 452 according to glibc with a few
exceptions:
$ rg --pcre2 'define __NR_fchmodat2 (?!452)'
sysdeps/unix/sysv/linux/x86_64/x32/arch-syscall.h
58:#define __NR_fchmodat2 1073742276
sysdeps/unix/sysv/linux/mips/mips64/n32/arch-syscall.h
67:#define __NR_fchmodat2 6452
sysdeps/unix/sysv/linux/mips/mips64/n64/arch-syscall.h
62:#define __NR_fchmodat2 5452
sysdeps/unix/sysv/linux/mips/mips32/arch-syscall.h
70:#define __NR_fchmodat2 4452
sysdeps/unix/sysv/linux/alpha/arch-syscall.h
59:#define __NR_fchmodat2 562
I added a small regression-test to the setuid integration-test that
attempts to set the suid bit on a file using the fchmodat2 syscall.
I confirmed that the test fails without the change in
local-derivation-goal.
Additionally, we require libseccomp 2.5.5 or greater now: as it turns
out, libseccomp maintains an internal syscall table and
validates each rule against it. This means that when using libseccomp
2.5.4 or older, one may pass `452` as syscall number against it, but
since it doesn't exist in the internal structure, `libseccomp` will refuse
to create a filter for that. This happens with nixpkgs-23.11, i.e. on
stable NixOS and when building Lix against the project's flake.
To work around that
* a backport of libseccomp 2.5.5 on upstream nixpkgs has been
scheduled[3].
* the package now uses libseccomp 2.5.5 on its own already. This is to
provide a quick fix since the correct fix for 23.11 is still a staging cycle
away.
We still need the compat header though since `SCMP_SYS(fchmodat2)`
internally transforms this into `__SNR_fchmodat2` which points to
`__NR_fchmodat2` from glibc 2.39, so it wouldn't build on glibc 2.38.
The updated syscall table from libseccomp 2.5.5 is NOT used for that
step, but used later, so we need both, our compat header and their
syscall table 🤷
Relevant PRs in CppNix:
* https://github.com/NixOS/nix/pull/10591
* https://github.com/NixOS/nix/pull/10501
[1] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2031073804
[2] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2030844251
[3] https://github.com/NixOS/nixpkgs/pull/306070
(cherry picked from commit ba6804518772e6afb403dd55478365d4b863c854)
Change-Id: I6921ab5a363188c6bff617750d00bb517276b7fe
Part of #7672
My main motivation is to be able to use `nix.checkConfig`[1]. This
doesn't work with Lix currently since the module uses `nix show-config`
if the Nix version is <2.20pre and `nix config show` otherwise. I think
this is the only instance where nixpkgs checks for which Nix commands
exist that affects us now, so I figured we could just perform the rename
here as well[2] and still provide the current version number[3].
I don't have a strong opinion on whether to deprecate `nix show-config`,
the warning is added there automatically.
(cherry picked from commit f300e11b056dea414d7d77bbc6e5a7dc5d9ddd41)
[1] https://nixos.org/manual/nixos/stable/options.html#opt-nix.checkConfig
[2] I should add that I don't use the "official" ways of installing Lix
because using the flake directly and callPackaging it seemed to fit
better into my workflow: I already have a little mess to make
sure Hydra from the flake uses the correct pkgs.nix and I didn't
want to complicate it further while keeping a single package-set I
can build in CI. Don't get me wrong, I think such a module for a
quick-start is very important, just giving context on why I bother
in the first place :)
[3] When we go public, I think it's worth considering to add support in
nixpkgs itself for Lix.
Change-Id: I47b4239b05cbeda3c370d2fa56ea768b768768ac
* changes:
docs: clarify how ^ works for -E/-f installables
docs: give translation examples from nix-build -E/-A to installables
docs: clarify how the different kinds of installables are selected
docs: guide to installables docs in installable commands' docs
Currently LocalDerivationGoal allows setting `__sandboxProfile`
to add sandbox parameters on Darwin when `sandbox=true`.
This was only supposed to have an effect when `sandbox=relaxed`
Change-Id: Ide44ee82d7e4d6b545285eab26547e7014817d3f
this is used in CA rewriting, replacement of placeholders in
derivations, generating scripts for devShells, and some more
places. in all of these transitive replacements are unsound,
and overlapping replacements would be as well. there even is
a test that transitive replacements do not happen (in the CA
RewriteSink suite), but none for overlapping replacements. a
minimally surprising binary rewriter surely would not do any
of these replacements, the only reason we have not seen this
break yet is probably that rewriteStrings is only called for
store paths and things that look like store paths (and those
should never overlap nor admit such transitive replacements)
Change-Id: I6fc29f939d5061d9f56c752624a823ece8437c07
* changes:
nix3-profile: remove check "name" attr in manifests
Add profile migration test
nix3-profile: make element names stable
getNameFromURL(): Support uppercase characters in attribute names
nix3-profile: remove indices
nix3-profile: allow using human-readable names to select packages
implement parsing human-readable names from URLs
As discussed in the maintainer meeting on 2024-01-29.
Mainly this is to avoid a situation where the name is parsed and
treated as a file name, mostly to protect users.
.-* and ..-* are also considered invalid because they might strip
on that separator to remove versions. Doesn't really work, but that's
what we decided, and I won't argue with it, because .-* probably
doesn't seem to have a real world application anyway.
We do still permit a 1-character name that's just "-", which still
poses a similar risk in such a situation. We can't start disallowing
trailing -, because a non-zero number of users will need it and we've
seen how annoying and painful such a change is.
What matters most is preventing a situation where . or .. can be
injected, and to just get this done.
(cherry picked from commit f1b4663805a9dbcb1ace64ec110092d17c9155e0)
Change-Id: I900a8509933cee662f888c3c76fa8986b0058839
We didn't even realize you *could* use this syntax with -E and -f, much
less that the attribute path could be *empty*.
Change-Id: Id1a6715609f3a76a5ce477bd43a7832effbbe07b
The installables syntax is not documented in any of the man pages or
docbook pages for any of those individual commands. And while these
commands really should at least peripherally individually document how
installables work, in the meantime we can at least direct people to the
right place.
This commit also clarifies the unexpected fact that `nix profile remove`
and `nix profile upgrade` do *not* take installables.
Change-Id: I3b1453cb197a613bbab639c66a466365c3592c6d
nix3-profile automatically migrates any profile its used on to its style
of profile -- the ones with manifest.json instead of manifest.nix. On
non-NixOS systems, Nix is conventionally installed to the profile at
/nix/var/nix/profiles/default, so if a user passed that to `--profile`
of `nix profile`, then it would break upgrade-nix from ever working
again, without recreating the profile.
This commit fixes that, and allows upgrade-nix to work on either kind of
profile.
Fixes#16.
Change-Id: I4c49b1beba93bb50e8f8a107edc451affe08c3f7
Notably, ProfileManifest and ProfileElement are useful generic
profile management code, and nix profile is not the only place in the
codebase where profiles are relevant.
This commit is in preparation for fixing upgrade-nix's interaction with
new-style profiles.
Change-Id: Iefc8bbd34b4bc6012175cb3d6e6a8207973bc792
This replaces the external sandbox-exec call with direct calls into
libsandbox. This API is technically deprecated and is missing some
prototypes, but all major browsers depend on it, so it is unlikely to
materially change without warning.
This commit also ensures the netrc file is only written if the
derivation is in fact meant to be able to access the internet.
This change commits a sin of not actually actively declaring its
dependency on macOS's libsandbox.dylib; this is due to the dylib
cache in macOS making that explicit dependency unnecessary. In the
future this might become a problem, so this commit marks our sins.
Co-authored-by: Artemis Tosini <lix@artem.ist>
Co-authored-by: Lunaphied <lunaphied@lunaphied.me>
Change-Id: Ia302141a53ce7b0327c1aad86a117b6645fe1189
That's expected by `build-remote` and makes sure that errors are
correctly forwarded to the user. For instance, let's say that the
host-key of `example.org` is unknown and
nix-build ../nixpkgs -A hello -j0 --builders 'ssh-ng://example.org'
is issued, then you get the following output:
cannot build on 'ssh-ng://example.org?&': error: failed to start SSH connection to 'example.org'
Failed to find a machine for remote build!
derivation: yh46gakxq3kchrbihwxvpn5bmadcw90b-hello-2.12.1.drv
required (system, features): (x86_64-linux, [])
2 available machines:
[...]
The relevant information (`Host key verification failed`) ends up in the
daemon's log, but that's not very obvious considering that the daemon
isn't very chatty normally.
This can be fixed - the same way as its done for legacy-ssh - by passing
fd 4 to the SSH wrapper. Now you'd get the following error:
cannot build on 'ssh-ng://example.org': error: failed to start SSH connection to 'example.org': Host key verification failed.
Failed to find a machine for remote build!
[...]
...and now it's clear what's wrong.
Please note that this is won't end up in the derivation's log.
For previous discussion about this change see
https://github.com/NixOS/nix/pull/7659.
Change-Id: I5790856dbf58e53ea3e63238b015ea06c347cf92
only decompress the response once all data has been received (in the
fully buffered case), or at least outside of the curl wrapper itself
(in the receive-to-sink case). unfortunately this means we will have
to duplicate decompression logic for these two cases for time being,
but once the curl wrapper has been rewritten to return a real future
or Source we can deduplicate this logic again. the curl wrapper will
have to turn into a proper Source first and use decompression source
logic which also does not currently exist—only decompression *sinks*
Change-Id: I66bc692f07d9b9e69fe10689ee73a2de8d65e35c
this is highly questionable. single-arg download calls will misbehave
with it set, and two-arg download calls will just overwrite it. being
an implementation detail this should not have been in the API at all.
Change-Id: I613772951ee03d8302366085f06a53601d13f132
this lets each implementation of FileTransfer (of which currently only
the one exists at all) implement appropriate handling for its internal
behaviours that are not otherwise exposed. in curl this lets us switch
the buffer-full handling method from "block the entire curl thread" to
"pause just the one transfer", move the non-libcurl body decompression
out of the actual curl wrapper (which will let us eventually morph the
curl wrapper intto an actual source of Sources), and some other things
Change-Id: Id6d3593cde6b4915aab3e90a43b175c103cc3f18
Previously, the garbage collector found runtime roots on Darwin by
shelling out to `lsof -n -w -F n` then parsing the result.
However, this requires an lsof binary and can be extremely slow.
The official Apple lsof returns in a reasonable amount of time,
about 250ms in my tests, but the lsof packaged in nixpkgs is quite slow,
taking about 40 seconds to run the command.
Using libproc directly is about the same speed as Apple lsof,
and allows us to reënable several tests that were disabled on Darwin.
Change-Id: Ifa0adda7984e13c15535693baba835aae79a3577
just accumulate error data into result.data as we would for successful
transfers without a dataCallback. errorSink and data would contain the
same data in error cases anyway, so splitting them is not very useful.
Change-Id: I00e449866454389ac6a564ab411c903fd357dabf
This creates new subclasses of LocalStore for each OS to include
platform-specific functionality. Currently this just includes garbage
collector roots but it could be extended to sandboxing as well.
In order to make sure that the generic LocalStore is not accidentally
constructed, its constructor is protected. A Fallback is provided which
implements no functionality except constructors.
Change-Id: I836a28e90b68309873f75afb83e0f1b2e2c89fb3
jade
eldritch horrors
Maximilian Bosch
Qyriad
Artemis Tosini
Robert Hensing
puck
Ilya K