Compare commits

...

67 commits

Author SHA1 Message Date
V. c347d3df8f Merge changes I609a5898,I7afb53c9 into main
* changes:
  devendor pegtl
  update flake.lock
2024-07-30 20:32:52 +00:00
eldritch horrors 97a389b0be libstore: move Goal::getBuildResult to BuildResult
there are no other uses for this yet, but asking for just a subset of
outputs does seem at least somewhat useful to have as a generic thing

Change-Id: I30ff5055a666c351b1b086b8d05b9d7c9fb1c77a
2024-07-30 16:37:13 +00:00
Max “Goldstein” Siling 53bfcf2586 Merge "src/libcmd/repl.cc: avoid unneeded reload after :e" into main 2024-07-30 16:32:56 +00:00
Max “Goldstein” Siling 1a6d7a3af4 src/libcmd/repl.cc: avoid unneeded reload after :e
If `:edit`ing a store path, don't reload repl afterwards
to avoid losing local variables: store is immutable,
so "editing" a store path is always just viewing it.

Resolves: lix-project/lix#341
Change-Id: I3747f75ce26e0595e953069c39ddc3ee80699718
2024-07-30 16:08:26 +00:00
eldritch horrors d265dd5993 libstore: count all substitutions toward the same limit
limiting CA substitutions was a rather recent addition, and it used a
dedicated counter to not interfere with regular substitutions. though
this works fine it somewhat contradicts the documentation; job limits
should apply to all kinds of substitutions, or be one limit for each.

Change-Id: I1505105b14260ecc1784039b2cc4b7afcf9115c8
2024-07-30 15:37:27 +00:00
eldritch horrors d9af753a7f libstore: always wake up goals on EOF
all goals do this. it makes no sense to not notify a goal of EOF
conditions because this is the universal signal for "child done"

Change-Id: Ic3980de312547e616739c57c6248a8e81308b5ee
2024-07-30 15:37:27 +00:00
eldritch horrors 6c0dcd1220 libstore: simplify substitution handleEOF
both substitution goals add only this single fd to their wait set.

Change-Id: Ibf921f5bb3919106208a0871523b32c8f67fb3d3
2024-07-30 15:37:27 +00:00
eldritch horrors 548c973e82 libstore: remove Worker::updateProgress
just update progress every time a goal has returned from work(). there
seem to be no performance penalties, and the code is much simpler now.

Change-Id: I288ee568b764ee61f40a498d986afda49987cb50
2024-07-29 22:16:11 +00:00
jade 6abad7cb23 Merge "releng: Remove workaround for skopeo feature we didn't know about" into main 2024-07-26 11:09:17 +00:00
V. a98dce2a1f devendor pegtl
Change-Id: I609a58985fc5210806d0959049a48976ae079c30
2024-07-26 11:22:34 +04:00
V. 393794ad92 update flake.lock
Change-Id: I7afb53c929d297061dba6ec4a3ec7c6e3c6a553e
2024-07-26 11:22:34 +04:00
alois31 d945e89e19 Merge changes I45d3895f,I541be3ea,Ibe51416d into main
* changes:
  libstore/build: block io_uring
  libstore/build: use an allowlist approach to syscall filtering
  libstore/build: always treat seccomp setup failures as fatal
2024-07-26 07:08:35 +00:00
Artemis Tosini 60a48311e8 Merge "libutil: Support getSelfExe on FreeBSD" into main 2024-07-25 23:10:30 +00:00
jade c4c7cb7613 Merge changes Ic0dfcfe2,Ibe73851f,Ia7a8df1c,I400b2031 into main
* changes:
  package.nix: remove dead code
  diff-closures: remove gratuitous copy
  tree-wide: NULL -> nullptr
  libutil: rip out GNU Hurd support code
2024-07-25 18:05:41 +00:00
alois31 e7188e211a
libstore/build: block io_uring
Unfortunately, io_uring is totally opaque to seccomp, and while currently there
are no dangerous operations implemented, there is no guarantee that it remains
this way. This means that io_uring should be blocked entirely to ensure that
the sandbox is future-proof. This has not been observed to cause issues in
practice.

Change-Id: I45d3895f95abe1bc103a63969f444c334dbbf50d
2024-07-25 18:24:45 +02:00
alois31 127ee1a101
libstore/build: use an allowlist approach to syscall filtering
Previously, system call filtering (to prevent builders from storing files with
setuid/setgid permission bits or extended attributes) was performed using a
blocklist. While this looks simple at first, it actually carries significant
security and maintainability risks: after all, the kernel may add new syscalls
to achieve the same functionality one is trying to block, and it can even be
hard to actually add the syscall to the blocklist when building against a C
library that doesn't know about it yet. For a recent demonstration of this
happening in practice to Nix, see the introduction of fchmodat2 [0] [1].

The allowlist approach does not share the same drawback. While it does require
a rather large list of harmless syscalls to be maintained in the codebase,
failing to update this list (and roll out the update to all users) in time has
rather benign effects; at worst, very recent programs that already rely on new
syscalls will fail with an error the same way they would on a slightly older
kernel that doesn't support them yet. Most importantly, no unintended new ways
of performing dangerous operations will be silently allowed.

Another possible drawback is reduced system call performance due to the larger
filter created by the allowlist requiring more computation [2]. However, this
issue has not convincingly been demonstrated yet in practice, for example in
systemd or various browsers. To the contrary, it has been measured that the the
actual filter constructed here has approximately the same overhead as a very
simple filter blocking only one system call.

This commit tries to keep the behavior as close to unchanged as possible. The
system call list is in line with libseccomp 2.5.5 and glibc 2.39, which are the
latest versions at the point of writing. Since libseccomp 2.5.5 is already a
requirement and the distributions shipping this together with older versions of
glibc are mostly not a thing any more, this should not lead to more build
failures any more.

[0] https://github.com/NixOS/nixpkgs/issues/300635
[1] https://github.com/NixOS/nix/issues/10424
[2] https://github.com/flatpak/flatpak/pull/4462#issuecomment-1061690607

Change-Id: I541be3ea9b249bcceddfed6a5a13ac10b11e16ad
2024-07-25 18:24:40 +02:00
alois31 233408f677
libstore/build: always treat seccomp setup failures as fatal
In f047e4357b, I missed the behavior that if
building without a dedicated build user (i.e. in single-user setups), seccomp
setup failures are silently ignored. This was introduced without explanation 7
years ago (ff6becafa8). Hopefully the only
use-case nowadays is causing spurious test suite successes when messing up the
seccomp filter during development. Let's try removing it.

Change-Id: Ibe51416d9c7a6dd635c2282990224861adf1ceab
2024-07-25 18:21:26 +02:00
Qyriad 8d12e0fbb7 fix building with Musl, fixing static builds
Musl stdout macro expands¹ to something that isn't a valid identifier,
so we get syntax errors when compiling usage of a method called stdout
with Musl's stdio.h.

[1]: https://git.musl-libc.org/cgit/musl/tree/include/stdio.h?id=ab31e9d6a0fa7c5c408856c89df2dfb12c344039#n67

Change-Id: I10e6f6a49504399bf8edd59c5d9e4e62449469e8
2024-07-24 17:21:40 +00:00
Artemis Tosini 3b96b51cf4
libutil: Support getSelfExe on FreeBSD
getSelfExe is used in a few places re-execute nix.
Current code in this file uses ifdefs to support several
platforms, just keep doing that

Change-Id: Iecc2ada0101aea0c30524e3a1218594f919d74bf
2024-07-24 01:28:03 +00:00
jade 98e8cf9c63 package.nix: remove dead code
Change-Id: Ic0dfcfe27dbf13da4f7f74f5fab8ce6fa718d28f
2024-07-23 21:53:43 +02:00
jade 12a5838d11 diff-closures: remove gratuitous copy
This was done originally because std::smatch does not accept `const char
*` as iterators. However, this was because we should have been using
std::cmatch instead.

Change-Id: Ibe73851fd39755e883df2d33d22fed72ac0a04ae
2024-07-23 21:45:30 +02:00
jade eecc4ff1c0 releng: Remove workaround for skopeo feature we didn't know about
It turns out skopeo *does* support not saying the tag, but I couldn't
find it in the docs.

Asking the author in https://github.com/containers/skopeo/issues/2354
yielded that this can be requested as `@@unknown-digest@@`.

So now we have a perfectly cromulent docker upload chain, yay!!

Change-Id: I256f3cbeef4fe28b3d68d0dda57f02cdaee3996b
2024-07-23 12:11:15 -07:00
jade 2436f2110a tree-wide: NULL -> nullptr
This is slightly more type safe and is more in line with modern C++.

Change-Id: Ia7a8df1c7788085020d1bdc941d6f9cee356144e
2024-07-23 21:06:55 +02:00
jade 916b5c68fb libutil: rip out GNU Hurd support code
Nobody has stepped up to add further support for Hurd since this code
appeared in 2010 or 2014. We don't need it.

Change-Id: I400b2031a225551ea3c71a3ef3ea9fdb599dfba3
2024-07-23 20:52:04 +02:00
Artemis Tosini 53f3e39815
libstore: Add FreeBSD findPlatformRoots
Use libprocstat to find garbage collector roots on FreeBSD.
Tested working on a FreeBSD machine, although there is no CI yet

Change-Id: Id36bac8c3de6cc4de94e2d76e9663dd4b76068a9
2024-07-23 17:49:33 +00:00
Pierre Bourdon 73c013a5df Merge "libexpr/gc-alloc: fix compilation with !HAVE_BOEHMGC" into main 2024-07-22 23:14:59 +00:00
Pierre Bourdon e76245f8e9
libexpr/gc-alloc: fix compilation with !HAVE_BOEHMGC
Fixes: 72ee25b402
Change-Id: Ib59386af1415a8ed4b53af24ec22a4ffa5e5877d
2024-07-23 00:50:09 +02:00
eldritch horrors 472ff1b833 libstore: keep Goal errors as unique_ptrs
Error is pretty large, and most goals do not fail. this alone more than
halves the size of Goal on x86_64-linux, from 720 bytes down to 344. in
derived classes the difference is not as dramatic, but even the largest
derived class (`LocalDerivationGoal`) loses almost 20% of its footprint

Change-Id: Ifda8f94c81b6566eeb3e52d55d9796ec40c7bce8
2024-07-22 19:01:40 +00:00
eldritch horrors 7bf1aff44a libstore: remove an always-defaulted argument
Change-Id: I3c7f17d5492a16bb54480fa1aa384b96fba72d61
2024-07-22 19:01:40 +00:00
eldritch horrors 58a91d70c9 libstore: use std::async instead of Goal threads
the goals are either already using std::async and merely forgot to
remove std::thread vestiges or they emulate async with threads and
promises. we can simply use async directly everywhere for clarity.

Change-Id: I3f05098310a25984f10fff1e68c573329002b500
2024-07-22 19:01:40 +00:00
eldritch horrors ad36fb43ad libstore: remove addToWeakGoals
under owner_less it's equivalent to insert(), only sometimes a little
bit faster because it does not construct a weak_ptr if the goal is in
the set already. this small difference in performance does not matter
here and c++23 will make insert transparent anyway, so we can drop it

Change-Id: I7cbd7d6e0daa95d67145ec58183162f6c4743b15
2024-07-22 19:01:40 +00:00
eldritch horrors d70e045f90 libstore: remove Goal::ecBusy
this should be an optional. "busy" is not an *exit* code!

Change-Id: Ic231cb27b022312b1a7a7b9602f32845b7a9c934
2024-07-22 19:01:40 +00:00
eldritch horrors 20f53346df libstore: remove unused Worker::waitForAnyGoal
Change-Id: Ia3ebd434b17052b6760ce74d8e20025a72148613
2024-07-22 19:01:40 +00:00
V. 85e3b9b871 De-vendor nixfmt
Change-Id: I1a051be495318a507d07f6d0a6b157616e26774c
2024-07-22 21:09:58 +04:00
eldritch horrors c74eb81356 enable -Werror=suggest-override
*accidentally* overriding a function is almost guaranteed to be an
error. overriding a function without labeling it as such is merely
bad style, but bad style that makes the code harder to understand.

Change-Id: Ic0594f3d1604ab6b3c1a75cb5facc246effe45f0
2024-07-22 16:26:55 +00:00
eldritch horrors 0463cf2aef libexpr: fix -Wunused-const-variable warning
Change-Id: Ib986ece0ab2eff83e7abd7f1f915cd8f761827ad
2024-07-22 16:26:55 +00:00
alois31 2d4aca2546
libutil/logging: fix build without precompiled header
Commit 0109368c3f missed to include a required
header, which is not noticed when the precompiled header is enabled because
it's included in that. Also include it in the file so that the build without
precompiled header works too.

Change-Id: Id7a7979684b64f937f7f8191612952d73c113015
2024-07-21 12:54:31 +02:00
alois31 94a8e5fe0d Merge "libstore/binary-cache-store: use correct buffer size for NAR decompression" into main 2024-07-21 10:42:33 +00:00
jade 4fa6961aa2 Merge "gc: refactor the gc server thread out into a class without changing it" into main 2024-07-21 10:36:10 +00:00
alois31 391088900e
libstore/binary-cache-store: use correct buffer size for NAR decompression
Due to a leftover from a previous version where the buffer was allocated on the
stack, the change introduced in commit 4ec87742a1
accidentally passes the size of a pointer as the size of the buffer to the
decompressor. Since the former is much smaller (usually 8 bytes instead of 64
kilobytes), this is safe, but leads to considerable overhead; most notably, due
to excessive progress reports, which happen for each chunk. Pass the proper
buffer size instead.

Change-Id: If4bf472d33e21587acb5235a2d99e3cb10914633
2024-07-21 11:28:23 +02:00
Winter Cute 1917e6c765 Merge "Fix namespace warning being emitted if sandbox is disabled" into main 2024-07-20 22:14:33 +00:00
Qyriad 72ee25b402 libexpr: add a strongly typed version of gcAllocBytes()
This commit adds a new helper template function to gc-alloc.hh (which is
probably where you want to look at first, O great reviewer [custom file
ordering in review diffs when]), which uses a type argument to determine
the size to allocate, rather than making the caller use sizeof().

Change-Id: Ib5d138d91a28bdda304a80db24ea9fb08669ad22
2024-07-20 20:20:01 +00:00
Qyriad e67dac1d74 libexpr: rename confusing makeImmutableString -> gcCopyStringIfNeeded
The purpose of this function has little to do with immutability. Value's
strings are never mutated, and the point of this function is to
singleton empty strings.

Change-Id: Ifd41dd952409d54e4d3de9ab59064e6928b0e480
2024-07-20 20:20:01 +00:00
Qyriad a3361557e3 libexpr: refactor gc-agnostic helpers into one place
Change-Id: Icc4b367e4f670d47256f62a3a002cd248a5c2d3b
2024-07-20 20:20:01 +00:00
eldritch horrors 0109368c3f libutil: make basic loggers thread-safe
SimpleLogger is not fully thread-safe, and all loggers that wrap it are
also not safe accordingly. this does not affect much, but in rare cases
it can cause interleaving of messages on stderr when used with the json
or raw log formats. the fix applied here is a bit of a hack, but fixing
this properly requires rearchitecting the logger infrastructure. nested
loggers are not the most natural abstraction here, and it is biting us.

Change-Id: Ifbf34fe1e85c60e73b59faee50e7411c7b5e7c12
2024-07-20 12:33:49 +00:00
eldritch horrors d8c09b5836 libutil: remove warnOnce macro
it's only used once, and even that one use is highly questionable. more
instances of warnOnce should be much more principled than this has been

Change-Id: I5856570c99cb44462e700d753d0c706a5db03c4b
2024-07-20 12:33:49 +00:00
Winter Cute 3da41fdb82 Fix namespace warning being emitted if sandbox is disabled
If useChroot = false, and user namespaces aren't available for some
reason (e.g. within a Docker container), this fixes a pointless warning
being emitted, as we would never attempt to use them even if they were
available.

Change-Id: Ibcee91c088edd2cd19e70218d5a5802bff8f537b
2024-07-19 19:14:54 -04:00
jade 77ff799cc8 gc: refactor the gc server thread out into a class without changing it
This removes a *whole load* of variables from scope and enforces thread
boundaries with the type system.

There is not much change of significance in here, so the things to watch
out for while reviewing it are primarily that the destructor ordering
may have changed inadvertently, I think.

Change-Id: I3cd87e6d5a08dfcf368637407251db22a8906316
2024-07-19 20:55:55 +00:00
jade 22252825c4 Merge changes Id8b3d289,Ib75ab5b8,I3792eeb3 into main
* changes:
  Fixup a bunch of references to nixos.org manuals
  Add release notes for removing overflow from Nix language
  expr: fix a compiler warning about different signs in comparison
2024-07-19 18:52:46 +00:00
alois31 aba5f19680 Merge changes I829581a3,I0016970d,I5dac8e77,Ib7560fe5 into main
* changes:
  doc/release-notes: add for pretty printing improvements
  libexpr/print: do not show elided nested items when there are none
  libexpr/print: never show empty attrsets or derivations as «repeated»
  libexpr/print: pretty-print idempotently
2024-07-19 06:40:13 +00:00
jade 26e56780ca Fixup a bunch of references to nixos.org manuals
(plus one reference to CppNix github)

Change-Id: Id8b3d2897f3b54e286861805cfd421adc4d5de47
2024-07-18 19:27:33 +00:00
jade 10cc3b288d Add release notes for removing overflow from Nix language
Change-Id: Ib75ab5b8b4d879035d7ee7678f9cd0c491a39c0a
2024-07-18 19:27:33 +00:00
jade 50a63f8435 expr: fix a compiler warning about different signs in comparison
We know that variable is >=0, so we can just cast it to unsigned.

Change-Id: I3792eeb3ca43e6a507cc44c1a70584d42b2acd7b
2024-07-18 19:27:33 +00:00
jade 5ee1e6ea98 Merge changes Ib20e9aa0,I178a038b,I29c7de04 into main
* changes:
  docs: document the actual comparison rules instead of lies
  daemon: remove workaround for macOS kernel bug that seems fixed
  daemon: fix a crash bug "FATAL: exception not rethrown"
2024-07-18 17:40:32 +00:00
alois31 768d1f29a2
doc/release-notes: add for pretty printing improvements
Change-Id: I829581a3f5b8b742e6c866dcdbbc635f91afceb5
2024-07-18 19:08:20 +02:00
alois31 40c39aa5d2
libexpr/print: do not show elided nested items when there are none
When the configured maximum depth has been reached, attribute sets and lists
are printed with ellipsis to indicate the elision of nested items. Previously,
this happened even in case the structure being printed is empty, so that such
items do not in fact exist. This is confusing, so stop doing it.

Change-Id: I0016970dad3e42625e085dc896e6f476b21226c9
2024-07-18 18:41:34 +02:00
alois31 b5da823138
libexpr/print: never show empty attrsets or derivations as «repeated»
The repeated value detection logic exists so that the occurrence of large
common substructures does not fill up the screen or the computer's memory.
However, empty attribute sets and derivations (when their detection is enabled)
are always cheap to print, and in practice I have observed them to make up a
significant majority of the cases where I was annoyed by the repeated value
detection kicking in. Furthermore, `nix-instantiate --eval` already disables
this logic for empty attribute sets, and empty lists are already exempted
everywhere. For these reasons, always print empty attribute sets and
derivations as what they are.

Change-Id: I5dac8e7739f9d726b76fd0521ec46f38af94463f
2024-07-18 18:41:34 +02:00
alois31 81a0624d76
libexpr/print: pretty-print idempotently
When pretty-printing is enabled, previously an unforced thunk would trigger
indentation, even when it subsequently does not evaluate to a nested structure.
The resulting output looked inconsistent, and furthermore pretty-printing was
not idempotent (since pretty-printing the same value again, which is now fully
evaluated, will not trigger indentation).
When strict evaluation is enabled, force the item before inspecting its type,
so that it is properly known whether it contains a nested structure.
Furthermore, there is no need to cause indentation for unforced thunks, since
the very next operation will be printing them as `«thunk»`.

This is mostly a port of https://github.com/NixOS/nix/pull/11100 , but we only
force the item when it's going to be forced anyway due to strict
pretty-printing, and a new test was written since the REPL testing framework in
Lix is different.

Co-Authored-By: Robert Hensing <robert@roberthensing.nl>
Change-Id: Ib7560fe531d09e05ca6b2037a523fe21a26d9d58
2024-07-18 18:41:28 +02:00
alois31 7b1abf8107 Merge "doc/manual: clarify documentation related to the $$ parser bug" into main 2024-07-18 15:01:20 +00:00
alois31 72db9cd67b doc/release-notes: link the upcoming release notes again
The insertion marker comment broke the list into two parts, the first
containing only the link to the upcoming release notes and the second the
past releases. This confused the generator, leading to the first part being
discarded. Indent the marker comment so that it's syntactically part of the
preceding item, and in particular doesn't split the list any more.

Change-Id: I357c51bb03e4e0d79a76d30158615fd9eda95ea8
2024-07-17 22:12:41 +00:00
raito 67f62bcdb4 doc/release-notes: add date for major release
Change-Id: I93aab93c069bb3989c3f8d17e0862899e6f76865
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-07-17 22:12:41 +00:00
alois31 beb231784e doc/manual: clarify documentation related to the $$ parser bug
Due to a mistake in the grammar, a dollar character implicitly escapes a second
dollar character that immediately follows, so that it cannot start an
interpolation. Unfortunately, this behaviour has since come to be relied upon,
so it cannot be fixed. Furthermore, the documentation on regular strings did
not mention this behaviour at all, while in the case of indented strings it was
rather implicit.
Mention it explicitly in both cases, and describe how an interpolation can
follow a dollar character (namely, by escaping that). Since we have to touch
that section anyway, state that any character (other than n, r, and t; but
notably including `$` even if not succeeded by `{`) can be escaped using a
backslash in regular strings.

Change-Id: I7e5d68a9a4130eec98ce8218b485168f4b31a677
2024-07-17 22:01:48 +00:00
Max “Goldstein” Siling 68567206f2 Merge "tests/functional/repl.sh: actually fail test on wrong stdout" into main 2024-07-17 21:50:59 +00:00
Max “Goldstein” Siling 3a36c8bb90 tests/functional/repl.sh: actually fail test on wrong stdout
Previous test implementation assumed that grep supports newlines
in patterns. It doesn't, so tests spuriously passed, even though
some tests outputs were broken.

This patches output (and expected output) before grepping,
so there're no newlines in pattern.

Change-Id: Ie6561f9f2e18b83d976f162269d20136e2595141
2024-07-17 21:48:13 +00:00
jade 702f02c31f docs: document the actual comparison rules instead of lies
Although the comparison rules are ugly and we do not like various parts
of them, we must not hide them away for only catgirls to know about, so
the documentation should actually say how they work.

Change-Id: Ib20e9aa0e7b6486ade4f401035aafd85fbb08c91
2024-07-13 01:17:14 +02:00
jade 69e2ee5b25 daemon: remove workaround for macOS kernel bug that seems fixed
This was filed as https://github.com/nixos/nix/issues/7584, but as far
as I can tell, the previous solution of POLLHUP works just fine on macOS
14. I've also tested on an ancient machine with macOS 10.15.7, which
also has POLLHUP work correctly.

It's possible this might regress some older versions of macOS that have
a kernel bug, but I went looking through the history on the sources and
didn't find anything that looked terribly convincingly like a bug fix
between 2020 and today. If such a broken version exists, it seems pretty
reasonable to suggest simply updating the OS.

Change-Id: I178a038baa000f927ea2cbc4587d69d8ab786843
2024-07-13 01:17:14 +02:00
jade b3fb8d9822 daemon: fix a crash bug "FATAL: exception not rethrown"
This is caused by pthread_cancel effectively throwing a
not-specifically-identifiable C++ exception into the targeted thread,
which, if it is not rethrown, terminates the process entirely.

This is rather "impolite" behaviour, we would say. But thread
cancellation is *always* busted, and we should simply not use it where
unnecessary. It's particularly unnecessary when what we *actually* need
it for is, err, interrupting a poll(2).

That can in turn be achieved by simply listening to more stuff in the
poll, namely, a pipe, which we send a character to when needing to
stop the thread.

While looking at this code, we also investigated whether any of the
poll() madness is required, or was even *ever* required. Curiously we
found in the XNU kernel source code that the thing about needing to
listen to POLLHUP is probably *correct*, but switching it to POLLRDNORM
should not have made any difference at all. We've left a FIXME to look
into that further because what's written here is super janky.

94d3b45284/bsd/kern/sys_generic.c (L1751-L1758)

This is the crash on some Hydra machines:

Thread 1 (Thread 0x7f56b77776c0 (LWP 955542) (Exiting)):
0  0x00007f56b8e9b7dc in __pthread_kill_implementation () from /nix/store/m71p7f0nymb19yn1dascklyya2i96jfw-glibc-2.39-52/lib/libc.so.6
1  0x00007f56b8e49516 in raise () from /nix/store/m71p7f0nymb19yn1dascklyya2i96jfw-glibc-2.39-52/lib/libc.so.6
2  0x00007f56b8e31935 in abort () from /nix/store/m71p7f0nymb19yn1dascklyya2i96jfw-glibc-2.39-52/lib/libc.so.6
3  0x00007f56b8e327f3 in __libc_message_impl.cold () from /nix/store/m71p7f0nymb19yn1dascklyya2i96jfw-glibc-2.39-52/lib/libc.so.6
4  0x00007f56b8e8e8e9 in __libc_fatal () from /nix/store/m71p7f0nymb19yn1dascklyya2i96jfw-glibc-2.39-52/lib/libc.so.6
5  0x00007f56b8ea23c4 in unwind_cleanup () from /nix/store/m71p7f0nymb19yn1dascklyya2i96jfw-glibc-2.39-52/lib/libc.so.6
6  0x00007f56b9d2a1b8 in nix::triggerInterrupt() [clone .cold] () from /nix/store/sahgw550p621m9dy1pd7whl9c5g1g0p7-lix-2.90.0-rc1/lib/liblixutil.so
7  0x00007f56b990ac9d in std:🧵:_State_impl<std:🧵:_Invoker<std::tuple<nix::MonitorFdHup::MonitorFdHup(int)::{lambda()#1}> > >::_M_run() () from /nix/store/sahgw550p621m9dy1pd7whl9c5g1g0p7-lix-2.90.0-rc1/lib/liblixstore.so
8  0x00007f56b90e86d3 in execute_native_thread_routine () from /nix/store/c6r62m84hywf4i6qq1h28f13zv38yqyp-gcc-13.3.0-lib/lib/libstdc++.so.6
9  0x00007f56b8e99a42 in start_thread () from /nix/store/m71p7f0nymb19yn1dascklyya2i96jfw-glibc-2.39-52/lib/libc.so.6
10 0x00007f56b8f1905c in clone3 () from /nix/store/m71p7f0nymb19yn1dascklyya2i96jfw-glibc-2.39-52/lib/libc.so.6

As for testing, we've started a daemon with this change and verified it
deals with HUPs correctly on x86_64-linux, but I don't think we can
easily test the destructor behaviour without whatever Hydra was
doing that broke.

Change-Id: I29c7de0425674494b6e43c075810126c3ff77363
2024-07-13 00:59:33 +02:00
85 changed files with 1789 additions and 807 deletions

View file

@ -57,6 +57,11 @@ ericson:
display_name: John Ericson
github: ericson2314
goldstein:
display_name: goldstein
forgejo: goldstein
github: GoldsteinE
horrors:
display_name: eldritch horrors
forgejo: pennae

View file

@ -0,0 +1,23 @@
---
synopsis: Define integer overflow in the Nix language as an error
issues: [fj#423]
cls: [1594, 1595, 1597, 1609]
category: Fixes
credits: [jade]
---
Previously, integer overflow in the Nix language invoked C++ level signed overflow, which was undefined behaviour, but *probably* manifested as wrapping around on overflow.
Since prior to the public release of Lix, Lix had C++ signed overflow defined to crash the process and nobody noticed this having accidentally removed overflow from the Nix language for three months until it was caught by fiddling around.
Given the significant body of actual Nix code that has been evaluated by Lix in that time, it does not appear that nixpkgs or much of importance depends on integer overflow, so it is safe to turn into an error.
Some other overflows were fixed:
- `builtins.fromJSON` of values greater than the maximum representable value in a signed 64-bit integer will generate an error.
- `nixConfig` in flakes will no longer accept negative values for configuration options.
Integer overflow now looks like the following:
```
» nix eval --expr '9223372036854775807 + 1'
error: integer overflow in adding 9223372036854775807 + 1
```

View file

@ -0,0 +1,12 @@
---
synopsis: "Block io_uring in the Linux sandbox"
cls: 1611
credits: alois31
category: Breaking Changes
---
The io\_uring API has the unfortunate property that it is not possible to selectively decide which operations should be allowed.
This, together with the fact that new operations are routinely added, makes it a hazard to the proper function of the sandbox.
Therefore, any access to io\_uring has been made unavailable inside the sandbox.
As such, attempts to execute any system calls forming part of this API will fail with the error `ENOSYS`, as if io\_uring support had not been configured into the kernel.

View file

@ -0,0 +1,58 @@
---
synopsis: "Eliminate some pretty-printing surprises"
cls: [1616, 1617, 1618]
prs: [11100]
credits: [alois31, roberth]
category: Improvements
---
Some inconsistent and surprising behaviours have been eliminated from the pretty-printing used by the REPL and `nix eval`:
* Lists and attribute sets that contain only a single item without nested structures are no longer sometimes inappropriately indented in the REPL, depending on internal state of the evaluator.
* Empty attribute sets and derivations are no longer shown as `«repeated»`, since they are always cheap to print.
This matches the existing behaviour of `nix-instantiate` on empty attribute sets.
Empty lists were never printed as `«repeated»` already.
* The REPL by default does not print nested attribute sets and lists, and indicates elided items with an ellipsis.
Previously, the ellipsis was printed even when the structure was empty, so that such items do not in fact exist.
Since this behaviour was confusing, it does not happen any more.
Before:
```
nix-repl> :p let x = 1 + 2; in [ [ x ] [ x ] ]
[
[
3
]
[ 3 ]
]
nix-repl> let inherit (import <nixpkgs> { }) hello; in [ hello hello ]
[
«derivation /nix/store/fqs92lzychkm6p37j7fnj4d65nq9fzla-hello-2.12.1.drv»
«repeated»
]
nix-repl> let x = {}; in [ x ]
[
{ ... }
]
```
After:
```
nix-repl> :p let x = 1 + 2; in [ [ x ] [ x ] ]
[
[ 3 ]
[ 3 ]
]
nix-repl> let inherit (import <nixpkgs> { }) hello; in [ hello hello ]
[
«derivation /nix/store/fqs92lzychkm6p37j7fnj4d65nq9fzla-hello-2.12.1.drv»
«derivation /nix/store/fqs92lzychkm6p37j7fnj4d65nq9fzla-hello-2.12.1.drv»
]
nix-repl> let x = {}; in [ x ]
[
{ }
]
```

View file

@ -0,0 +1,11 @@
---
synopsis: "`:edit`ing a file in Nix store no longer reloads the repl"
issues: [fj#341]
cls: [1620]
category: Improvements
credits: [goldstein]
---
Calling `:edit` from the repl now only reloads if the file being edited was outside of Nix store.
That means that all the local variables are now preserved across `:edit`s of store paths.
This is always safe because the store is read-only.

View file

@ -196,8 +196,8 @@
- [C++ style guide](contributing/cxx.md)
- [Release Notes](release-notes/release-notes.md)
- [Upcoming release](release-notes/rl-next.md)
<!-- RELENG-AUTO-INSERTION-MARKER (see releng/release_notes.py) -->
- [Lix 2.90 (FIXME date)](release-notes/rl-2.90.md)
<!-- RELENG-AUTO-INSERTION-MARKER (see releng/release_notes.py) -->
- [Lix 2.90 (2024-07-10)](release-notes/rl-2.90.md)
- [Nix 2.18 (2023-09-20)](release-notes/rl-2.18.md)
- [Nix 2.17 (2023-07-24)](release-notes/rl-2.17.md)
- [Nix 2.16 (2023-05-31)](release-notes/rl-2.16.md)

View file

@ -247,7 +247,6 @@ To ensure that characterization testing doesn't make it harder to intentionally
The integration tests are defined in the Nix flake under the `hydraJobs.tests` attribute.
These tests include everything that needs to interact with external services or run Lix in a non-trivial distributed setup.
Because these tests are expensive and require more than what the standard github-actions setup provides, they only run on the master branch (on <https://hydra.nixos.org/jobset/nix/master>).
You can run them manually with `nix build .#hydraJobs.tests.{testName}` or `nix-build -A hydraJobs.tests.{testName}`

View file

@ -145,19 +145,71 @@ All comparison operators are implemented in terms of `<`, and the following equi
| *a* `>` *b* | *b* `<` *a* |
| *a* `>=` *b* | `! (` *a* `<` *b* `)` |
Note that the above behaviour violates IEEE 754 for floating point numbers with respect to NaN, for instance.
This may be fixed in a future major language revision.
[Comparison]: #comparison-operators
## Equality
- [Attribute sets][attribute set] and [list]s are compared recursively, and therefore are fully evaluated.
- Comparison of [function]s always returns `false`.
- Numbers are type-compatible, see [arithmetic] operators.
- Floating point numbers only differ up to a limited precision.
The following equality comparison rules are followed in order:
- Comparisons are first, sometimes, performed by identity (pointer value), and whether or not this occurs varies depending on the context in which the comparison is performed; for example, through `builtins.elem`, comparison of lists, or other cases.
The exact instances in which this occurs, aside from direct list and attribute set comparisons as discussed below, are too dependent on implementation details to meaningfully document.
See [note on identity comparison](#identity-comparison) below.
- Comparisons between a combination of integers and floating point numbers are first converted to floating point then compared as floating point.
- Comparisons between values of differing types, besides the ones mentioned in the above rule, are unequal.
- Strings are compared as their string values, disregarding string contexts.
- Paths are compared as their absolute form (since they are stored as such).
- [Functions][function] are always considered unequal, including with themselves.
- The following are compared in the typical manner:
- Integers
- Floating point numbers have equality comparison per IEEE 754.
Note that this means that just like in most languages, floating point arithmetic results are not typically equality comparable, and should instead be compared by checking that the absolute difference is less than some error margin.
- Booleans
- Null
- [Attribute sets][attribute set] are compared following these rules in order:
- If both attribute sets have the same identity (via pointer equality), they are considered equal, regardless of whether the contents have reflexive equality (e.g. even if there are functions contained within).
See [note on identity comparison](#identity-comparison) below.
- If both attribute sets have `type = "derivation"` and have an attribute `outPath` that is equal, they are considered equal.
This means that two results of `builtins.derivation`, regardless of other things added to their attributes via `//` afterwards (or `passthru` in nixpkgs), will compare equal if they passed the same arguments to `builtins.derivation`.
- Otherwise, they are compared element-wise in an unspecified order.
Although this order *may* be deterministic in some cases, this is not guaranteed, and correct code must not rely on this ordering behaviour.
The order determines which elements are evaluated first and thus, if there are throwing values in the attribute set, which of those get evaluated, if any, before the comparison returns an unequal result.
- Lists are compared following these rules in order:
- If both lists have the same identity (via pointer equality), they are considered equal, regardless of whether the contents have reflexive equality (e.g. even if there are functions contained within).
See [note on identity comparison](#identity-comparison) below.
- Otherwise, they are compared element-wise in list order.
[function]: ./constructs.md#functions
[Equality]: #equality
### Identity comparison
In the current revision of the Nix language, values are first compared by identity (pointer equality).
This means that values that are not reflexively equal (that is, they do not satisfy `a == a`), such as functions, are nonetheless sometimes compared as equal with themselves if they are placed in attribute sets or lists, or are compared through other indirect means.
Whether identity comparison applies to a given usage of the language aside from direct list and attribute set comparison is strongly dependent on implementation details to the point it is not feasible to document the exact instances.
This is rather unfortunate behaviour which is regrettably load-bearing on nixpkgs (such as with the `type` attribute of NixOS options) and cannot be changed for the time being.
It may be changed in a future major language revision.
Correct code must not rely on this behaviour.
For example:
```
nix-repl> let f = x: 1; s = { func = f; }; in [ (f == f) (s == s) ]
[ false true ]
```
## Logical implication
Equivalent to `!`*b1* `||` *b2*.

View file

@ -7,13 +7,16 @@
*Strings* can be written in three ways.
The most common way is to enclose the string between double quotes,
e.g., `"foo bar"`. Strings can span multiple lines. The special
characters `"` and `\` and the character sequence `${` must be
escaped by prefixing them with a backslash (`\`). Newlines, carriage
returns and tabs can be written as `\n`, `\r` and `\t`,
respectively.
e.g., `"foo bar"`. Strings can span multiple lines. The backslash
(`\`) can be used to escape characters: newlines, carriage returns
and tabs may be written as `\n`, `\r` and `\t` respectively; any
other characters can be preceded by a backslash to remove any
special meaning they may have, like the special characters `"` and
`\` and the character sequence `${`.
You can include the results of other expressions into a string by enclosing them in `${ }`, a feature known as [string interpolation].
Due to a parser issue that has since come to be relied upon, the character sequence `$${` is interpreted literally and does not introduce an interpolation.
To express a `$` character immediately followed by an interpolation, the former must be escaped.
[string interpolation]: ./string-interpolation.md
@ -43,16 +46,16 @@
Note that the whitespace and newline following the opening `''` is
ignored if there is no non-whitespace text on the initial line.
Indented strings support [string interpolation].
Since `${` and `''` have special meaning in indented strings, you
need a way to quote them. `$` can be escaped by prefixing it with
`''` (that is, two single quotes), i.e., `''$`. `''` can be escaped
by prefixing it with `'`, i.e., `'''`. `$` removes any special
meaning from the following `$`. Linefeed, carriage-return and tab
by prefixing it with `'`, i.e., `'''`. Linefeed, carriage-return and tab
characters can be written as `''\n`, `''\r`, `''\t`, and `''\`
escapes any other character.
Indented strings support [string interpolation] using `${ }` the same way regular strings do.
`$${` is interpreted literally in indented strings as well, so the `$` character must be escaped if it is to be followed by an interpolation.
Indented strings are primarily useful in that they allow multi-line
string literals to follow the indentation of the enclosing Nix
expression, and that less escaping is typically necessary for

View file

@ -1,7 +1,7 @@
# Lix 2.90 "Vanilla Ice Cream" (FIXME date)
# Lix 2.90 "Vanilla Ice Cream" (2024-07-10)
# Lix 2.90.0 (FIXME date)
# Lix 2.90.0 (2024-07-10)
## Breaking Changes
- Deprecate the online flake registries and vendor the default registry [fj#183](https://git.lix.systems/lix-project/lix/issues/183) [fj#110](https://git.lix.systems/lix-project/lix/issues/110) [fj#116](https://git.lix.systems/lix-project/lix/issues/116) [#8953](https://github.com/NixOS/nix/issues/8953) [#9087](https://github.com/NixOS/nix/issues/9087) [cl/1127](https://gerrit.lix.systems/c/lix/+/1127)

View file

@ -19,11 +19,11 @@
"nix2container": {
"flake": false,
"locked": {
"lastModified": 1712990762,
"narHash": "sha256-hO9W3w7NcnYeX8u8cleHiSpK2YJo7ecarFTUlbybl7k=",
"lastModified": 1720642556,
"narHash": "sha256-qsnqk13UmREKmRT7c8hEnz26X3GFFyIQrqx4EaRc1Is=",
"owner": "nlewo",
"repo": "nix2container",
"rev": "20aad300c925639d5d6cbe30013c8357ce9f2a2e",
"rev": "3853e5caf9ad24103b13aa6e0e8bcebb47649fe4",
"type": "github"
},
"original": {
@ -34,11 +34,11 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1718111384,
"narHash": "sha256-7tSst0S5FOmcgvNtfy6cjZX5w8CabCVAfAeCkhY4OVg=",
"lastModified": 1721931987,
"narHash": "sha256-1Zg8LY0T5EfXtv0Kf4M6SFnjH7Eto4VV+EKJ/YSnhiI=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "a508a44af0c1b1b57785c34d8b54783536273eeb",
"rev": "e21630230c77140bc6478a21cd71e8bb73706fce",
"type": "github"
},
"original": {
@ -67,11 +67,11 @@
"pre-commit-hooks": {
"flake": false,
"locked": {
"lastModified": 1712055707,
"narHash": "sha256-4XLvuSIDZJGS17xEwSrNuJLL7UjDYKGJSbK1WWX2AK8=",
"lastModified": 1721042469,
"narHash": "sha256-6FPUl7HVtvRHCCBQne7Ylp4p+dpP3P/OYuzjztZ4s70=",
"owner": "cachix",
"repo": "git-hooks.nix",
"rev": "e35aed5fda3cc79f88ed7f1795021e559582093a",
"rev": "f451c19376071a90d8c58ab1a953c6e9840527fd",
"type": "github"
},
"original": {

View file

@ -140,10 +140,7 @@
system = crossSystem;
}
// lib.optionalAttrs (crossSystem == "x86_64-freebsd") { useLLVM = true; };
overlays = [
(overlayFor (p: p.${stdenv}))
(final: prev: { nixfmt = final.callPackage ./nix-support/nixfmt.nix { }; })
];
overlays = [ (overlayFor (p: p.${stdenv})) ];
};
stdenvs = forAllStdenvs (make-pkgs null);
native = stdenvs.stdenvPackages;
@ -167,6 +164,7 @@
nixUnstable = prev.nixUnstable;
check-headers = final.buildPackages.callPackage ./maintainers/check-headers.nix { };
check-syscalls = final.buildPackages.callPackage ./maintainers/check-syscalls.nix { };
default-busybox-sandbox-shell = final.busybox.override {
useMusl = true;
@ -198,8 +196,6 @@
busybox-sandbox-shell = final.busybox-sandbox-shell or final.default-busybox-sandbox-shell;
};
pegtl = final.nix.passthru.pegtl;
# Export the patched version of boehmgc that Lix uses into the overlay
# for consumers of this flake.
boehmgc-nix = final.nix.passthru.boehmgc-nix;

View file

@ -0,0 +1,16 @@
{
runCommandNoCC,
lib,
libseccomp,
writeShellScriptBin,
}:
let
syscalls-csv = runCommandNoCC "syscalls.csv" { } ''
echo ${lib.escapeShellArg libseccomp.src}
tar -xf ${lib.escapeShellArg libseccomp.src} --strip-components=2 ${libseccomp.name}/src/syscalls.csv
mv syscalls.csv "$out"
'';
in
writeShellScriptBin "check-syscalls" ''
${./check-syscalls.sh} ${syscalls-csv}
''

7
maintainers/check-syscalls.sh Executable file
View file

@ -0,0 +1,7 @@
#!/usr/bin/env bash
set -e
diff -u <(awk < src/libstore/build/local-derivation-goal.cc '/BEGIN extract-syscalls/ { extracting = 1; next }
match($0, /allowSyscall\(ctx, SCMP_SYS\(([^)]*)\)\);|\/\/ skip ([^ ]*)/, result) { print result[1] result[2] }
/END extract-syscalls/ { extracting = 0; next }') <(tail -n+2 "$1" | cut -d, -f 1)

View file

@ -314,6 +314,10 @@ nlohmann_json = dependency('nlohmann_json', required : true)
# *absolutely* are not going to make it work)
lix_doc = declare_dependency(link_args : [ '-llix_doc' ])
if is_freebsd
libprocstat = declare_dependency(link_args : [ '-lprocstat' ])
endif
#
# Build-time tools
#
@ -445,6 +449,7 @@ add_project_arguments(
'-Werror=unused-result',
'-Wdeprecated-copy',
'-Wignored-qualifiers',
'-Werror=suggest-override',
# Enable assertions in libstdc++ by default. Harmless on libc++. Benchmarked
# at ~1% overhead in `nix search`.
#

View file

@ -1,23 +0,0 @@
{
stdenv,
cmake,
ninja,
fetchFromGitHub,
}:
stdenv.mkDerivation {
pname = "pegtl";
version = "3.2.7";
src = fetchFromGitHub {
repo = "PEGTL";
owner = "taocpp";
rev = "refs/tags/3.2.7";
hash = "sha256-IV5YNGE4EWVrmg2Sia/rcU8jCuiBynQGJM6n3DCWTQU=";
};
nativeBuildInputs = [
cmake
ninja
];
}

View file

@ -106,7 +106,7 @@ pre-commit-run {
};
treefmt = {
enable = true;
settings.formatters = [ pkgs.nixfmt ];
settings.formatters = [ pkgs.nixfmt-rfc-style ];
};
};
}

View file

@ -1,6 +1,6 @@
[Unit]
Description=Nix Daemon
Documentation=man:nix-daemon https://nixos.org/manual
Documentation=man:nix-daemon https://docs.lix.systems/manual/lix/stable
RequiresMountsFor=@storedir@
RequiresMountsFor=@localstatedir@
RequiresMountsFor=@localstatedir@/nix/db

View file

@ -1,65 +0,0 @@
# Copy of `nixfmt-rfc-style` vendored from `nixpkgs` master:
# https://github.com/NixOS/nixpkgs/blob/ab6071eb54cc9b66dda436111d4f569e4e56cbf4/pkgs/by-name/ni/nixfmt-rfc-style/package.nix
{
haskell,
haskellPackages,
fetchFromGitHub,
}:
let
inherit (haskell.lib.compose) justStaticExecutables;
raw-pkg = haskellPackages.callPackage (
{
mkDerivation,
base,
cmdargs,
directory,
fetchzip,
filepath,
lib,
megaparsec,
mtl,
parser-combinators,
safe-exceptions,
scientific,
text,
transformers,
unix,
}:
mkDerivation {
pname = "nixfmt";
version = "0.6.0-unstable-2024-03-14";
src = fetchFromGitHub {
owner = "serokell";
repo = "nixfmt";
rev = "8d13b593fa8d8d6e5075f541f3231222a08e84df";
hash = "sha256-HtXvzmfN4wk45qiKZ7V+/5WBV7jnTHfd7iBwF4XGl64=";
};
isLibrary = true;
isExecutable = true;
libraryHaskellDepends = [
base
megaparsec
mtl
parser-combinators
scientific
text
transformers
];
executableHaskellDepends = [
base
cmdargs
directory
filepath
safe-exceptions
text
unix
];
jailbreak = true;
homepage = "https://github.com/serokell/nixfmt";
description = "An opinionated formatter for Nix";
license = lib.licenses.mpl20;
mainProgram = "nixfmt";
}
) { };
in
justStaticExecutables raw-pkg

View file

@ -35,7 +35,7 @@
meson,
ninja,
openssl,
pegtl ? __forDefaults.pegtl,
pegtl,
pkg-config,
python3,
rapidcheck,
@ -70,8 +70,6 @@
lix-doc = callPackage ./lix-doc/package.nix { };
build-release-notes = callPackage ./maintainers/build-release-notes.nix { };
pegtl = callPackage ./misc/pegtl.nix { };
},
}:
let
@ -92,31 +90,19 @@ let
# Reimplementation of Nixpkgs' Meson cross file, with some additions to make
# it actually work.
mesonCrossFile =
let
cpuFamily =
platform:
with platform;
if isAarch32 then
"arm"
else if isx86_32 then
"x86"
else
platform.uname.processor;
in
builtins.toFile "lix-cross-file.conf" ''
[properties]
# Meson is convinced that if !buildPlatform.canExecute hostPlatform then we cannot
# build anything at all, which is not at all correct. If we can't execute the host
# platform, we'll just disable tests and doc gen.
needs_exe_wrapper = false
mesonCrossFile = builtins.toFile "lix-cross-file.conf" ''
[properties]
# Meson is convinced that if !buildPlatform.canExecute hostPlatform then we cannot
# build anything at all, which is not at all correct. If we can't execute the host
# platform, we'll just disable tests and doc gen.
needs_exe_wrapper = false
[binaries]
# Meson refuses to consider any CMake binary during cross compilation if it's
# not explicitly specified here, in the cross file.
# https://github.com/mesonbuild/meson/blob/0ed78cf6fa6d87c0738f67ae43525e661b50a8a2/mesonbuild/cmake/executor.py#L72
cmake = 'cmake'
'';
[binaries]
# Meson refuses to consider any CMake binary during cross compilation if it's
# not explicitly specified here, in the cross file.
# https://github.com/mesonbuild/meson/blob/0ed78cf6fa6d87c0738f67ae43525e661b50a8a2/mesonbuild/cmake/executor.py#L72
cmake = 'cmake'
'';
# The internal API docs need these for the build, but if we're not building
# Nix itself, then these don't need to be propagated.
@ -396,13 +382,14 @@ stdenv.mkDerivation (finalAttrs: {
glibcLocales,
just,
llvmPackages,
nixfmt,
nixfmt-rfc-style,
skopeo,
xonsh,
# Lix specific packages
pre-commit-checks,
contribNotice,
check-syscalls,
}:
let
glibcFix = lib.optionalAttrs (buildPlatform.isLinux && glibcLocales != null) {
@ -453,11 +440,12 @@ stdenv.mkDerivation (finalAttrs: {
# `bash` from inside `nix develop`, say, because you are using it
# via direnv, you will by default get bash (unusable edition).
bashInteractive
check-syscalls
pythonEnv
# docker image tool
skopeo
just
nixfmt
nixfmt-rfc-style
# Included above when internalApiDocs is true, but we set that to
# false intentionally to save dev build time.
# To build them in a dev shell, you can set -Dinternal-api-docs=enabled when configuring.

View file

@ -44,23 +44,8 @@ def upload_docker_images(target: DockerTarget, paths: list[Path]):
for path in paths:
digest_file = tmp / (path.name + '.digest')
tmp_image = tmp / 'tmp-image.tar.gz'
# insecure-policy: we don't have any signature policy, we are just uploading an image
#
# Absurd: we copy it into an OCI image first so we can get the hash
# we need to upload it untagged, because skopeo has no "don't tag
# this" option.
# The reason for this is that forgejo's container registry throws
# away old versions of tags immediately, so we cannot use a temp
# tag, and it *does* reduce confusion to not upload tags that
# should not be used.
#
# Workaround for: https://github.com/containers/skopeo/issues/2354
log.info('skopeo copy to temp oci-archive %s', tmp_image)
skopeo --insecure-policy copy --format oci --all --digestfile @(digest_file) docker-archive:@(path) oci-archive:@(tmp_image)
inspection = json.loads($(skopeo inspect oci-archive:@(tmp_image)))
inspection = json.loads($(skopeo inspect docker-archive:@(path)))
docker_arch = inspection['Architecture']
docker_os = inspection['Os']
@ -68,8 +53,9 @@ def upload_docker_images(target: DockerTarget, paths: list[Path]):
log.info('Pushing image %s for %s to %s', path, docker_arch, target.registry_path)
# insecure-policy: we don't have any signature policy, we are just uploading an image
skopeo --insecure-policy copy --digestfile @(digest_file) --all docker-archive:@(path) f'docker://{target.registry_path}@@unknown-digest@@'
digest = digest_file.read_text().strip()
skopeo --insecure-policy copy --preserve-digests --all oci-archive:@(tmp_image) f'docker://{target.registry_path}@{digest}'
# skopeo doesn't give us the manifest size directly, so we just ask the registry
metadata = reg.image_info(target.registry_path, digest)

View file

@ -16,7 +16,7 @@ def add_to_summary(date: str):
if VERSION_RL.exists():
return
MARKER = '<!-- RELENG-AUTO-INSERTION-MARKER'
MARKER = ' <!-- RELENG-AUTO-INSERTION-MARKER'
new_lines = []
for line in SUMMARY.read_text().splitlines():

View file

@ -236,9 +236,9 @@ static int main_build_remote(int argc, char * * argv)
}
#if __APPLE__
futimes(bestSlotLock.get(), NULL);
futimes(bestSlotLock.get(), nullptr);
#else
futimens(bestSlotLock.get(), NULL);
futimens(bestSlotLock.get(), nullptr);
#endif
lock.reset();

View file

@ -652,9 +652,12 @@ ProcessLineResult NixRepl::processLine(std::string line)
// using runProgram2 to allow editors to display their UI
runProgram2(RunOptions { .program = editor, .searchPath = true, .args = args }).wait();
// Reload right after exiting the editor
state->resetFileCache();
reloadFiles();
// Reload right after exiting the editor if path is not in store
// Store is immutable, so there could be no changes, so there's no need to reload
if (!state->store->isInStore(path.resolveSymlinks().path.abs())) {
state->resetFileCache();
reloadFiles();
}
}
else if (command == ":t") {

View file

@ -1,5 +1,6 @@
#include "attr-set.hh"
#include "eval-inline.hh"
#include "eval.hh"
#include "gc-alloc.hh"
#include <algorithm>
@ -19,7 +20,7 @@ Bindings * EvalState::allocBindings(size_t capacity)
throw Error("attribute set of size %d is too big", capacity);
nrAttrsets++;
nrAttrsInAttrsets += capacity;
return new (allocBytes(sizeof(Bindings) + sizeof(Attr) * capacity)) Bindings((Bindings::size_t) capacity);
return new (gcAllocBytes(sizeof(Bindings) + sizeof(Attr) * capacity)) Bindings((Bindings::size_t) capacity);
}

View file

@ -4,26 +4,10 @@
#include "print.hh"
#include "eval.hh"
#include "eval-error.hh"
#include "gc-alloc.hh"
namespace nix {
/**
* Note: Various places expect the allocated memory to be zeroed.
*/
[[gnu::always_inline]]
inline void * allocBytes(size_t n)
{
void * p;
#if HAVE_BOEHMGC
p = GC_MALLOC(n);
#else
p = calloc(n, 1);
#endif
if (!p) throw std::bad_alloc();
return p;
}
[[gnu::always_inline]]
Value * EvalState::allocValue()
{
@ -43,7 +27,7 @@ Value * EvalState::allocValue()
*valueAllocCache = GC_NEXT(p);
GC_NEXT(p) = nullptr;
#else
void * p = allocBytes(sizeof(Value));
void * p = gcAllocBytes(sizeof(Value));
#endif
nrValues++;
@ -73,7 +57,7 @@ Env & EvalState::allocEnv(size_t size)
env = (Env *) p;
} else
#endif
env = (Env *) allocBytes(sizeof(Env) + size * sizeof(Value *));
env = (Env *) gcAllocBytes(sizeof(Env) + size * sizeof(Value *));
/* We assume that env->values has been cleared by the allocator; maybeThunk() and lookupVar fromWith expect this. */

View file

@ -9,6 +9,7 @@
#include "store-api.hh"
#include "derivations.hh"
#include "downstream-placeholder.hh"
#include "gc-alloc.hh"
#include "globals.hh"
#include "eval-inline.hh"
#include "filetransfer.hh"
@ -48,35 +49,6 @@ using json = nlohmann::json;
namespace nix {
static char * allocString(size_t size)
{
char * t;
#if HAVE_BOEHMGC
t = (char *) GC_MALLOC_ATOMIC(size);
#else
t = (char *) malloc(size);
#endif
if (!t) throw std::bad_alloc();
return t;
}
// When there's no need to write to the string, we can optimize away empty
// string allocations.
// This function handles makeImmutableString(std::string_view()) by returning
// the empty string.
static const char * makeImmutableString(std::string_view s)
{
const size_t size = s.size();
if (size == 0)
return "";
auto t = allocString(size + 1);
memcpy(t, s.data(), size);
t[size] = '\0';
return t;
}
RootValue allocRootValue(Value * v)
{
#if HAVE_BOEHMGC
@ -797,7 +769,7 @@ DebugTraceStacker::DebugTraceStacker(EvalState & evalState, DebugTrace t)
void Value::mkString(std::string_view s)
{
mkString(makeImmutableString(s));
mkString(gcCopyStringIfNeeded(s));
}
@ -805,10 +777,9 @@ static void copyContextToValue(Value & v, const NixStringContext & context)
{
if (!context.empty()) {
size_t n = 0;
v.string.context = (const char * *)
allocBytes((context.size() + 1) * sizeof(char *));
v.string.context = gcAllocType<char const *>(context.size() + 1);
for (auto & i : context)
v.string.context[n++] = makeImmutableString(i.to_string());
v.string.context[n++] = gcCopyStringIfNeeded(i.to_string());
v.string.context[n] = 0;
}
}
@ -828,7 +799,7 @@ void Value::mkStringMove(const char * s, const NixStringContext & context)
void Value::mkPath(const SourcePath & path)
{
mkPath(makeImmutableString(path.path.abs()));
mkPath(gcCopyStringIfNeeded(path.path.abs()));
}
@ -862,7 +833,7 @@ void EvalState::mkList(Value & v, size_t size)
{
v.mkList(size);
if (size > 2)
v.bigList.elems = (Value * *) allocBytes(size * sizeof(Value *));
v.bigList.elems = gcAllocType<Value *>(size);
nrListElems += size;
}
@ -1901,7 +1872,7 @@ void EvalState::autoCallFunction(Bindings & args, Value & fun, Value & res)
Nix attempted to evaluate a function as a top level expression; in
this case it must have its arguments supplied either by default
values, or passed explicitly with '--arg' or '--argstr'. See
https://nixos.org/manual/nix/stable/language/constructs.html#functions.)", symbols[i.name])
https://docs.lix.systems/manual/lix/stable/language/constructs.html#functions)", symbols[i.name])
.atPos(i.pos).withFrame(*fun.lambda.env, *fun.lambda.fun).debugThrow();
}
}
@ -2076,7 +2047,7 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
Value. allocating a GC'd string directly and moving it into a
Value lets us avoid an allocation and copy. */
const auto c_str = [&] {
char * result = allocString(sSize + 1);
char * result = gcAllocString(sSize + 1);
char * tmp = result;
for (const auto & part : s) {
memcpy(tmp, part->data(), part->size());

View file

@ -3,6 +3,7 @@
#include "attr-set.hh"
#include "eval-error.hh"
#include "gc-alloc.hh"
#include "types.hh"
#include "value.hh"
#include "nixexpr.hh"
@ -37,22 +38,6 @@ namespace eval_cache {
class EvalCache;
}
/** Alias for std::map which uses boehmgc's allocator conditional on us actually
* using boehmgc in this build.
*/
#if HAVE_BOEHMGC
template<typename KeyT, typename ValueT>
using GcMap = std::map<
KeyT,
ValueT,
std::less<KeyT>,
traceable_allocator<std::pair<KeyT const, ValueT>>
>;
#else
using GcMap = std::map<KeyT, ValueT>
#endif
/**
* Function that implements a primop.
*/

23
src/libexpr/gc-alloc.cc Normal file
View file

@ -0,0 +1,23 @@
#include "gc-alloc.hh"
#include <cstring>
#include <string_view>
namespace nix
{
char const * gcCopyStringIfNeeded(std::string_view toCopyFrom)
{
if (toCopyFrom.empty()) {
return "";
}
size_t const size = toCopyFrom.size();
char * cstr = gcAllocString(size + 1);
memcpy(cstr, toCopyFrom.data(), size);
cstr[size] = '\0';
return cstr;
}
}

150
src/libexpr/gc-alloc.hh Normal file
View file

@ -0,0 +1,150 @@
#pragma once
/// @file Aliases and wrapper functions that are transparently GC-enabled
/// if Lix is compiled with BoehmGC enabled.
#include <cstddef>
#include <cstring>
#include <list>
#include <map>
#include <new>
#include <string_view>
#include <vector>
#include "checked-arithmetic.hh"
#if HAVE_BOEHMGC
#include <functional> // std::less
#include <utility> // std::pair
#define GC_INCLUDE_NEW
#include <gc/gc.h>
#include <gc/gc_allocator.h>
#include <gc/gc_cpp.h>
/// calloc, transparently GC-enabled.
#define LIX_GC_CALLOC(size) GC_MALLOC(size)
/// strdup, transaprently GC-enabled.
#define LIX_GC_STRDUP(str) GC_STRDUP(str)
/// Atomic GC malloc() with GC enabled, or regular malloc() otherwise.
#define LIX_GC_MALLOC_ATOMIC(size) GC_MALLOC_ATOMIC(size)
namespace nix
{
/// Alias for std::map which uses BoehmGC's allocator conditional on this Lix
/// build having GC enabled.
template<typename KeyT, typename ValueT>
using GcMap = std::map<
KeyT,
ValueT,
std::less<KeyT>,
traceable_allocator<std::pair<KeyT const, ValueT>>
>;
/// Alias for std::vector which uses BoehmGC's allocator conditional on this Lix
/// build having GC enabled.
template<typename ItemT>
using GcVector = std::vector<ItemT, traceable_allocator<ItemT>>;
/// Alias for std::list which uses BoehmGC's allocator conditional on this Lix
/// build having GC enabled.
template<typename ItemT>
using GcList = std::list<ItemT, traceable_allocator<ItemT>>;
}
#else
#include <cstdlib>
/// calloc, transparently GC-enabled.
#define LIX_GC_CALLOC(size) calloc(size, 1)
/// strdup, transparently GC-enabled.
#define LIX_GC_STRDUP(str) strdup(str)
/// Atomic GC malloc() with GC enabled, or regular malloc() otherwise.
/// The returned memory must never contain pointers.
#define LIX_GC_MALLOC_ATOMIC(size) malloc(size)
namespace nix
{
/// Alias for std::map which uses BoehmGC's allocator conditional on this Lix
/// build having GC enabled.
template<typename KeyT, typename ValueT>
using GcMap = std::map<KeyT, ValueT>;
/// Alias for std::vector which uses BoehmGC's allocator conditional on this Lix
/// build having GC enabled.
template<typename ItemT>
using GcVector = std::vector<ItemT>;
/// Alias for std::list which uses BoehmGC's allocator conditional on this Lix
/// build having GC enabled.
template<typename ItemT>
using GcList = std::list<ItemT>;
}
#endif
namespace nix
{
[[gnu::always_inline]]
inline void * gcAllocBytes(size_t n)
{
// Note: various places expect the allocated memory to be zero.
// Hence: calloc().
void * ptr = LIX_GC_CALLOC(n);
if (ptr == nullptr) {
throw std::bad_alloc();
}
return ptr;
}
/// Typed, safe wrapper around calloc() (transparently GC-enabled). Allocates
/// enough for the requested count of the specified type. Also checks for
/// nullptr (and throws @ref std::bad_alloc), and casts the void pointer to
/// a pointer of the specified type, for type-convenient goodness.
template<typename T>
[[gnu::always_inline]]
inline T * gcAllocType(size_t howMany = 1)
{
// NOTE: size_t * size_t, which can definitely overflow.
// Unsigned integer overflow is definitely a bug, but isn't undefined
// behavior, so we can just check if we overflowed after the fact.
// However, people can and do request zero sized allocations, so we need
// to check that neither of our multiplicands were zero before complaining
// about it.
auto checkedSz = checked::Checked<size_t>(howMany) * sizeof(T);
size_t sz = checkedSz.valueWrapping();
if (checkedSz.overflowed()) {
// Congrats, you done did an overflow.
throw std::bad_alloc();
}
return static_cast<T *>(gcAllocBytes(sz));
}
/// GC-transparently allocates a buffer for a C-string of @ref size *bytes*,
/// meaning you should include the size needed by the NUL terminator in the
/// passed size. Memory allocated with this function must never contain other
/// pointers.
inline char * gcAllocString(size_t size)
{
char * cstr = static_cast<char *>(LIX_GC_MALLOC_ATOMIC(size));
if (cstr == nullptr) {
throw std::bad_alloc();
}
return cstr;
}
/// Returns a C-string copied from @ref toCopyFrom, or a single, static empty
/// string if @ref toCopyFrom is also empty.
char const * gcCopyStringIfNeeded(std::string_view toCopyFrom);
}

View file

@ -80,13 +80,7 @@ public:
bool hasFailed() { return failed; };
};
#if HAVE_BOEHMGC
typedef std::list<DrvInfo, traceable_allocator<DrvInfo>> DrvInfos;
#else
typedef std::list<DrvInfo> DrvInfos;
#endif
using DrvInfos = GcList<DrvInfo>;
/**
* If value `v` denotes a derivation, return a DrvInfo object

View file

@ -82,28 +82,28 @@ class JSONSax : nlohmann::json_sax<json> {
public:
JSONSax(EvalState & state, Value & v) : state(state), rs(new JSONState(&v)) {};
bool null()
bool null() override
{
rs->value(state).mkNull();
rs->add();
return true;
}
bool boolean(bool val)
bool boolean(bool val) override
{
rs->value(state).mkBool(val);
rs->add();
return true;
}
bool number_integer(number_integer_t val)
bool number_integer(number_integer_t val) override
{
rs->value(state).mkInt(val);
rs->add();
return true;
}
bool number_unsigned(number_unsigned_t val_)
bool number_unsigned(number_unsigned_t val_) override
{
if (val_ > std::numeric_limits<NixInt::Inner>::max()) {
throw Error("unsigned json number %1% outside of Nix integer range", val_);
@ -114,14 +114,14 @@ public:
return true;
}
bool number_float(number_float_t val, const string_t & s)
bool number_float(number_float_t val, const string_t & s) override
{
rs->value(state).mkFloat(val);
rs->add();
return true;
}
bool string(string_t & val)
bool string(string_t & val) override
{
rs->value(state).mkString(val);
rs->add();
@ -129,7 +129,7 @@ public:
}
#if NLOHMANN_JSON_VERSION_MAJOR >= 3 && NLOHMANN_JSON_VERSION_MINOR >= 8
bool binary(binary_t&)
bool binary(binary_t&) override
{
// This function ought to be unreachable
assert(false);
@ -137,35 +137,37 @@ public:
}
#endif
bool start_object(std::size_t len)
bool start_object(std::size_t len) override
{
rs = std::make_unique<JSONObjectState>(std::move(rs));
return true;
}
bool key(string_t & name)
bool key(string_t & name) override
{
dynamic_cast<JSONObjectState*>(rs.get())->key(name, state);
return true;
}
bool end_object() {
bool end_object() override {
rs = rs->resolve(state);
rs->add();
return true;
}
bool end_array() {
bool end_array() override {
return end_object();
}
bool start_array(size_t len) {
bool start_array(size_t len) override {
rs = std::make_unique<JSONListState>(std::move(rs),
len != std::numeric_limits<size_t>::max() ? len : 128);
return true;
}
bool parse_error(std::size_t, const std::string&, const nlohmann::detail::exception& ex) {
bool
parse_error(std::size_t, const std::string &, const nlohmann::detail::exception & ex) override
{
throw JSONParseError("%s", ex.what());
}
};

View file

@ -22,6 +22,7 @@ libexpr_sources = files(
'eval.cc',
'function-trace.cc',
'get-drvs.cc',
'gc-alloc.cc',
'json-to-value.cc',
'nixexpr.cc',
'parser/parser.cc',
@ -58,6 +59,7 @@ libexpr_headers = files(
'function-trace.hh',
'gc-small-vector.hh',
'get-drvs.hh',
'gc-alloc.hh',
'json-to-value.hh',
'nixexpr.hh',
'parser/change_head.hh',

View file

@ -43,7 +43,6 @@ error_message_for(p::one<'}'>) = "expecting '}'";
error_message_for(p::one<'"'>) = "expecting '\"'";
error_message_for(p::one<';'>) = "expecting ';'";
error_message_for(p::one<')'>) = "expecting ')'";
error_message_for(p::one<'='>) = "expecting '='";
error_message_for(p::one<']'>) = "expecting ']'";
error_message_for(p::one<':'>) = "expecting ':'";
error_message_for(p::string<'\'', '\''>) = "expecting \"''\"";

View file

@ -622,14 +622,13 @@ struct CompareValues
}
};
/// NOTE: this type must NEVER be outside of GC-scanned memory.
#if HAVE_BOEHMGC
typedef std::list<Value *, gc_allocator<Value *>> ValueList;
using UnsafeValueList = std::list<Value *, gc_allocator<Value *>>;
#else
typedef std::list<Value *> ValueList;
using UnsafeValueList = std::list<Value *>;
#endif
static Bindings::iterator getAttr(
EvalState & state,
Symbol attrSym,
@ -652,7 +651,7 @@ static void prim_genericClosure(EvalState & state, const PosIdx pos, Value * * a
state.forceList(*startSet->value, noPos, "while evaluating the 'startSet' attribute passed as argument to builtins.genericClosure");
ValueList workSet;
UnsafeValueList workSet;
for (auto elem : startSet->value->listItems())
workSet.push_back(elem);
@ -668,7 +667,7 @@ static void prim_genericClosure(EvalState & state, const PosIdx pos, Value * * a
/* Construct the closure by applying the operator to elements of
`workSet', adding the result to `workSet', continuing until
no new elements are found. */
ValueList res;
UnsafeValueList res;
// `doneKeys' doesn't need to be a GC root, because its values are
// reachable from res.
auto cmp = CompareValues(state, noPos, "while comparing the `key` attributes of two genericClosure elements");
@ -3275,10 +3274,12 @@ static RegisterPrimOp primop_all({
static void prim_genList(EvalState & state, const PosIdx pos, Value * * args, Value & v)
{
auto len = state.forceInt(*args[1], pos, "while evaluating the second argument passed to builtins.genList").value;
auto len_ = state.forceInt(*args[1], pos, "while evaluating the second argument passed to builtins.genList").value;
if (len < 0)
state.error<EvalError>("cannot create list of size %1%", len).atPos(pos).debugThrow();
if (len_ < 0)
state.error<EvalError>("cannot create list of size %1%", len_).atPos(pos).debugThrow();
size_t len = len_;
// More strict than striclty (!) necessary, but acceptable
// as evaluating map without accessing any values makes little sense.

View file

@ -264,22 +264,24 @@ private:
return true;
}
if (options.force) {
// The item is going to be forced during printing anyway, but we need its type now.
state.forceValue(*item, item->determinePos(noPos));
}
// Pretty-print single-item attrsets only if they contain nested
// structures.
auto itemType = item->type();
return itemType == nList || itemType == nAttrs || itemType == nThunk;
return itemType == nList || itemType == nAttrs;
}
void printAttrs(Value & v, size_t depth)
{
if (seen && !seen->insert(v.attrs).second) {
printRepeated();
return;
}
if (options.force && options.derivationPaths && state.isDerivation(v)) {
printDerivation(v);
} else if (depth < options.maxDepth) {
} else if (seen && !v.attrs->empty() && !seen->insert(v.attrs).second) {
printRepeated();
} else if (depth < options.maxDepth || v.attrs->empty()) {
increaseIndent();
output << "{";
@ -335,10 +337,15 @@ private:
return true;
}
if (options.force) {
// The item is going to be forced during printing anyway, but we need its type now.
state.forceValue(*item, item->determinePos(noPos));
}
// Pretty-print single-item lists only if they contain nested
// structures.
auto itemType = item->type();
return itemType == nList || itemType == nAttrs || itemType == nThunk;
return itemType == nList || itemType == nAttrs;
}
void printList(Value & v, size_t depth)
@ -348,7 +355,7 @@ private:
return;
}
if (depth < options.maxDepth) {
if (depth < options.maxDepth || v.listSize() == 0) {
increaseIndent();
output << "[";
auto listItems = v.listItems();

View file

@ -4,6 +4,7 @@
#include <cassert>
#include <climits>
#include "gc-alloc.hh"
#include "symbol-table.hh"
#include "value/context.hh"
#include "input-accessor.hh"
@ -11,9 +12,6 @@
#include "print-options.hh"
#include "checked-arithmetic.hh"
#if HAVE_BOEHMGC
#include <gc/gc_allocator.h>
#endif
#include <nlohmann/json_fwd.hpp>
namespace nix {
@ -464,17 +462,9 @@ void Value::mkBlackhole()
thunk.expr = (Expr*) &eBlackHole;
}
#if HAVE_BOEHMGC
typedef std::vector<Value *, traceable_allocator<Value *>> ValueVector;
typedef std::map<Symbol, Value *, std::less<Symbol>, traceable_allocator<std::pair<const Symbol, Value *>>> ValueMap;
typedef std::map<Symbol, ValueVector, std::less<Symbol>, traceable_allocator<std::pair<const Symbol, ValueVector>>> ValueVectorMap;
#else
typedef std::vector<Value *> ValueVector;
typedef std::map<Symbol, Value *> ValueMap;
typedef std::map<Symbol, ValueVector> ValueVectorMap;
#endif
using ValueVector = GcVector<Value *>;
using ValueMap = GcMap<Symbol, Value *>;
using ValueVectorMap = std::map<Symbol, ValueVector>;
/**
* A value allocated in traceable memory.

View file

@ -695,7 +695,7 @@ struct GitInputScheme : InputScheme
});
Finally const _wait([&] { proc.wait(); });
unpackTarfile(*proc.stdout(), tmpDir);
unpackTarfile(*proc.getStdout(), tmpDir);
}
auto storePath = store->addToStore(name, tmpDir, FileIngestionMethod::Recursive, htSHA256, filter);

View file

@ -91,7 +91,7 @@ void ProgressBar::resume()
nextWakeup = draw(*state, {});
state.wait_for(quitCV, std::chrono::milliseconds(50));
}
writeToStderr("\r\e[K");
writeLogsToStderr("\r\e[K");
});
}
@ -124,7 +124,7 @@ void ProgressBar::log(State & state, Verbosity lvl, std::string_view s)
} else {
auto s2 = s + ANSI_NORMAL "\n";
if (!isTTY) s2 = filterANSIEscapes(s2, true);
writeToStderr(s2);
writeLogsToStderr(s2);
}
}
@ -322,9 +322,9 @@ void ProgressBar::eraseProgressDisplay(State & state)
{
if (printMultiline && (state.lastLines >= 1)) {
// FIXME: make sure this works on windows
writeToStderr(fmt("\e[G\e[%dF\e[J", state.lastLines));
writeLogsToStderr(fmt("\e[G\e[%dF\e[J", state.lastLines));
} else {
writeToStderr("\r\e[K");
writeLogsToStderr("\r\e[K");
}
}
@ -346,7 +346,7 @@ std::chrono::milliseconds ProgressBar::draw(State & state, const std::optional<s
state.lastLines = 0;
if (s != std::nullopt)
writeToStderr(filterANSIEscapes(s.value(), !isTTY) + ANSI_NORMAL "\n");
writeLogsToStderr(filterANSIEscapes(s.value(), !isTTY) + ANSI_NORMAL "\n");
std::string line;
std::string status = getStatus(state);
@ -356,7 +356,7 @@ std::chrono::milliseconds ProgressBar::draw(State & state, const std::optional<s
line += "]";
}
if (printMultiline && !line.empty()) {
writeToStderr(filterANSIEscapes(line, false, width) + ANSI_NORMAL "\n");
writeLogsToStderr(filterANSIEscapes(line, false, width) + ANSI_NORMAL "\n");
state.lastLines++;
}
@ -398,7 +398,7 @@ std::chrono::milliseconds ProgressBar::draw(State & state, const std::optional<s
if (printMultiline) {
if (state.lastLines < (height -1)) {
writeToStderr(filterANSIEscapes(activity_line, false, width) + ANSI_NORMAL "\n");
writeLogsToStderr(filterANSIEscapes(activity_line, false, width) + ANSI_NORMAL "\n");
state.lastLines++;
} else moreActivities++;
}
@ -406,7 +406,7 @@ std::chrono::milliseconds ProgressBar::draw(State & state, const std::optional<s
}
if (printMultiline && moreActivities)
writeToStderr(fmt("And %d more...", moreActivities));
writeLogsToStderr(fmt("And %d more...", moreActivities));
if (!printMultiline) {
if (!line.empty()) {
@ -414,7 +414,7 @@ std::chrono::milliseconds ProgressBar::draw(State & state, const std::optional<s
}
line += activity_line;
if (!line.empty()) {
writeToStderr(filterANSIEscapes(line, false, width) + ANSI_NORMAL);
writeLogsToStderr(filterANSIEscapes(line, false, width) + ANSI_NORMAL);
}
}

View file

@ -33,9 +33,13 @@ void printGCWarning()
{
if (!gcWarning) return;
static bool haveWarned = false;
warnOnce(haveWarned,
"you did not specify '--add-root'; "
"the result might be removed by the garbage collector");
if (!haveWarned) {
haveWarned = true;
warn(
"you did not specify '--add-root'; "
"the result might be removed by the garbage collector"
);
}
}

View file

@ -336,12 +336,13 @@ WireFormatGenerator BinaryCacheStore::narFromPath(const StorePath & storePath)
try {
auto file = getFile(info->url);
return [](auto info, auto file, auto & stats) -> WireFormatGenerator {
std::unique_ptr<char[]> buf(new char[65536]);
constexpr size_t buflen = 65536;
auto buf = std::make_unique<char []>(buflen);
size_t total = 0;
auto decompressor = makeDecompressionSource(info->compression, *file);
try {
while (true) {
const auto len = decompressor->read(buf.get(), sizeof(buf));
const auto len = decompressor->read(buf.get(), buflen);
co_yield std::span{buf.get(), len};
total += len;
}

View file

@ -15,4 +15,27 @@ GENERATE_CMP_EXT(
me->cpuUser,
me->cpuSystem);
KeyedBuildResult BuildResult::restrictTo(DerivedPath path) const
{
KeyedBuildResult res{*this, std::move(path)};
if (auto pbp = std::get_if<DerivedPath::Built>(&res.path)) {
auto & bp = *pbp;
/* Because goals are in general shared between derived paths
that share the same derivation, we need to filter their
results to get back just the results we care about.
*/
for (auto it = res.builtOutputs.begin(); it != res.builtOutputs.end();) {
if (bp.outputs.contains(it->first))
++it;
else
it = res.builtOutputs.erase(it);
}
}
return res;
}
}

View file

@ -11,6 +11,8 @@
namespace nix {
struct KeyedBuildResult;
struct BuildResult
{
/**
@ -112,6 +114,18 @@ struct BuildResult
{
throw Error("%s", errorMsg);
}
/**
* Project a BuildResult with just the information that pertains to
* the given path.
*
* A `BuildResult` may hold information for multiple derived paths;
* this function discards information about outputs not relevant in
* `path`. Build `Goal`s in particular may contain more outputs for
* a single build result than asked for directly, it's necessary to
* remove any such additional result to not leak other build infos.
*/
KeyedBuildResult restrictTo(DerivedPath path) const;
};
/**

View file

@ -79,7 +79,6 @@ DerivationGoal::DerivationGoal(const StorePath & drvPath,
trace("created");
mcExpectedBuilds = std::make_unique<MaintainCount<uint64_t>>(worker.expectedBuilds);
worker.updateProgress();
}
@ -100,7 +99,6 @@ DerivationGoal::DerivationGoal(const StorePath & drvPath, const BasicDerivation
trace("created");
mcExpectedBuilds = std::make_unique<MaintainCount<uint64_t>>(worker.expectedBuilds);
worker.updateProgress();
/* Prevent the .chroot directory from being
garbage-collected. (See isActiveTempFile() in gc.cc.) */
@ -670,7 +668,6 @@ void DerivationGoal::started()
act = std::make_unique<Activity>(*logger, lvlInfo, actBuild, msg,
Logger::Fields{worker.store.printStorePath(drvPath), hook ? machineName : "", 1, 1});
mcRunningBuilds = std::make_unique<MaintainCount<uint64_t>>(worker.runningBuilds);
worker.updateProgress();
}
void DerivationGoal::tryToBuild()
@ -781,7 +778,7 @@ void DerivationGoal::tryLocalBuild() {
throw Error(
"unable to build with a primary store that isn't a local store; "
"either pass a different '--store' or enable remote builds."
"\nhttps://nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html");
"\nhttps://docs.lix.systems/manual/lix/stable/advanced-topics/distributed-builds.html");
}
@ -932,7 +929,7 @@ void runPostBuildHook(
Finally const _wait([&] { proc.wait(); });
// FIXME just process the data, without a wrapper sink class
proc.stdout()->drainInto(sink);
proc.getStdout()->drainInto(sink);
}
void DerivationGoal::buildDone()
@ -1166,7 +1163,7 @@ HookReply DerivationGoal::tryBuildHook()
}
else {
s += "\n";
writeToStderr(s);
writeLogsToStderr(s);
}
}
@ -1366,7 +1363,6 @@ void DerivationGoal::handleChildOutput(int fd, std::string_view data)
void DerivationGoal::handleEOF(int fd)
{
if (!currentLogLine.empty()) flushLine();
worker.wakeUp(shared_from_this());
}
@ -1537,8 +1533,6 @@ void DerivationGoal::done(
worker.failedBuilds++;
}
worker.updateProgress();
auto traceBuiltOutputsFile = getEnv("_NIX_TRACE_BUILT_OUTPUTS").value_or("");
if (traceBuiltOutputsFile != "") {
std::fstream fs;
@ -1566,7 +1560,7 @@ void DerivationGoal::waiteeDone(GoalPtr waitee, ExitCode result)
auto & outputs = nodeP->value;
for (auto & outputName : outputs) {
auto buildResult = dg->getBuildResult(DerivedPath::Built {
auto buildResult = dg->buildResult.restrictTo(DerivedPath::Built {
.drvPath = makeConstantStorePathRef(dg->drvPath),
.outputs = OutputsSpec::Names { outputName },
});

View file

@ -41,14 +41,13 @@ void DrvOutputSubstitutionGoal::tryNext()
/* Make sure that we are allowed to start a substitution. Note that even
if maxSubstitutionJobs == 0, we still allow a substituter to run. This
prevents infinite waiting. */
if (worker.runningCASubstitutions >= std::max(1U, settings.maxSubstitutionJobs.get())) {
if (worker.runningSubstitutions >= std::max(1U, settings.maxSubstitutionJobs.get())) {
worker.waitForBuildSlot(shared_from_this());
return;
}
maintainRunningSubstitutions =
std::make_unique<MaintainCount<uint64_t>>(worker.runningCASubstitutions);
worker.updateProgress();
std::make_unique<MaintainCount<uint64_t>>(worker.runningSubstitutions);
if (subs.size() == 0) {
/* None left. Terminate this goal and let someone else deal
@ -62,7 +61,6 @@ void DrvOutputSubstitutionGoal::tryNext()
if (substituterFailed) {
worker.failedSubstitutions++;
worker.updateProgress();
}
return;
@ -164,10 +162,5 @@ void DrvOutputSubstitutionGoal::work()
(this->*state)();
}
void DrvOutputSubstitutionGoal::handleEOF(int fd)
{
if (fd == downloadState->outPipe.readSide.get()) worker.wakeUp(shared_from_this());
}
}

View file

@ -4,7 +4,6 @@
#include "store-api.hh"
#include "goal.hh"
#include "realisation.hh"
#include <thread>
#include <future>
namespace nix {
@ -41,11 +40,6 @@ class DrvOutputSubstitutionGoal : public Goal {
*/
std::shared_ptr<Store> sub;
/**
* The substituter thread.
*/
std::thread thr;
std::unique_ptr<MaintainCount<uint64_t>> maintainRunningSubstitutions;
struct DownloadState
@ -78,7 +72,6 @@ public:
std::string key() override;
void work() override;
void handleEOF(int fd) override;
JobCategory jobCategory() const override {
return JobCategory::Substitution;

View file

@ -22,7 +22,7 @@ void Store::buildPaths(const std::vector<DerivedPath> & reqs, BuildMode buildMod
if (ex)
logError(i->ex->info());
else
ex = std::move(i->ex);
ex = std::move(*i->ex);
}
if (i->exitCode != Goal::ecSuccess) {
if (auto i2 = dynamic_cast<DerivationGoal *>(i.get()))
@ -62,10 +62,7 @@ std::vector<KeyedBuildResult> Store::buildPathsWithResults(
std::vector<KeyedBuildResult> results;
for (auto & [req, goalPtr] : state)
results.emplace_back(KeyedBuildResult {
goalPtr->getBuildResult(req),
/* .path = */ req,
});
results.emplace_back(goalPtr->buildResult.restrictTo(req));
return results;
}
@ -78,7 +75,7 @@ BuildResult Store::buildDerivation(const StorePath & drvPath, const BasicDerivat
try {
worker.run(Goals{goal});
return goal->getBuildResult(DerivedPath::Built {
return goal->buildResult.restrictTo(DerivedPath::Built {
.drvPath = makeConstantStorePathRef(drvPath),
.outputs = OutputsSpec::All {},
});

View file

@ -11,41 +11,10 @@ bool CompareGoalPtrs::operator() (const GoalPtr & a, const GoalPtr & b) const {
}
BuildResult Goal::getBuildResult(const DerivedPath & req) const {
BuildResult res { buildResult };
if (auto pbp = std::get_if<DerivedPath::Built>(&req)) {
auto & bp = *pbp;
/* Because goals are in general shared between derived paths
that share the same derivation, we need to filter their
results to get back just the results we care about.
*/
for (auto it = res.builtOutputs.begin(); it != res.builtOutputs.end();) {
if (bp.outputs.contains(it->first))
++it;
else
it = res.builtOutputs.erase(it);
}
}
return res;
}
void addToWeakGoals(WeakGoals & goals, GoalPtr p)
{
if (goals.find(p) != goals.end())
return;
goals.insert(p);
}
void Goal::addWaitee(GoalPtr waitee)
{
waitees.insert(waitee);
addToWeakGoals(waitee->waiters, shared_from_this());
waitee->waiters.insert(shared_from_this());
}
@ -79,15 +48,14 @@ void Goal::waiteeDone(GoalPtr waitee, ExitCode result)
void Goal::amDone(ExitCode result, std::optional<Error> ex)
{
trace("done");
assert(exitCode == ecBusy);
assert(result == ecSuccess || result == ecFailed || result == ecNoSubstituters || result == ecIncompleteClosure);
assert(!exitCode.has_value());
exitCode = result;
if (ex) {
if (!waiters.empty())
logError(ex->info());
else
this->ex = std::move(*ex);
this->ex = std::make_unique<Error>(std::move(*ex));
}
for (auto & i : waiters) {

View file

@ -53,7 +53,7 @@ enum struct JobCategory {
struct Goal : public std::enable_shared_from_this<Goal>
{
typedef enum {ecBusy, ecSuccess, ecFailed, ecNoSubstituters, ecIncompleteClosure} ExitCode;
typedef enum {ecSuccess, ecFailed, ecNoSubstituters, ecIncompleteClosure} ExitCode;
/**
* Backlink to the worker.
@ -96,9 +96,8 @@ struct Goal : public std::enable_shared_from_this<Goal>
/**
* Whether the goal is finished.
*/
ExitCode exitCode = ecBusy;
std::optional<ExitCode> exitCode;
protected:
/**
* Build result.
*/
@ -106,22 +105,10 @@ protected:
public:
/**
* Project a `BuildResult` with just the information that pertains
* to the given request.
*
* In general, goals may be aliased between multiple requests, and
* the stored `BuildResult` has information for the union of all
* requests. We don't want to leak what the other request are for
* sake of both privacy and determinism, and this "safe accessor"
* ensures we don't.
*/
BuildResult getBuildResult(const DerivedPath &) const;
/**
* Exception containing an error message, if any.
*/
std::optional<Error> ex;
std::unique_ptr<Error> ex;
Goal(Worker & worker, DerivedPath path)
: worker(worker)
@ -145,7 +132,6 @@ public:
virtual void handleEOF(int fd)
{
abort();
}
void trace(std::string_view s);
@ -175,6 +161,4 @@ public:
virtual JobCategory jobCategory() const = 0;
};
void addToWeakGoals(WeakGoals & goals, GoalPtr p);
}

View file

@ -45,7 +45,6 @@
#include <sys/prctl.h>
#include <sys/syscall.h>
#if HAVE_SECCOMP
#include "linux/fchmodat2-compat.hh"
#include <seccomp.h>
#endif
#define pivot_root(new_root, put_old) (syscall(SYS_pivot_root, new_root, put_old))
@ -221,12 +220,12 @@ void LocalDerivationGoal::tryLocalBuild()
}
#if __linux__
// FIXME: should user namespaces being unsupported also require
// sandbox-fallback to be allowed? I don't think so, since they aren't a
// huge security win to have enabled.
usingUserNamespace = userNamespacesSupported();
if (useChroot) {
// FIXME: should user namespaces being unsupported also require
// sandbox-fallback to be allowed? I don't think so, since they aren't a
// huge security win to have enabled.
usingUserNamespace = userNamespacesSupported();
if (!mountAndPidNamespacesSupported()) {
if (!settings.sandboxFallback)
throw Error("this system does not support the kernel namespaces that are required for sandboxing; use '--no-sandbox' to disable sandboxing. Pass --debug for diagnostics on what is broken.");
@ -976,7 +975,7 @@ bool LocalDerivationGoal::isAllowed(const DerivedPath & req)
struct RestrictedStoreConfig : virtual LocalFSStoreConfig
{
using LocalFSStoreConfig::LocalFSStoreConfig;
const std::string name() { return "Restricted Store"; }
const std::string name() override { return "Restricted Store"; }
};
/* A wrapper around LocalStore that only allows building/querying of
@ -1363,6 +1362,20 @@ void LocalDerivationGoal::chownToBuilder(const Path & path)
throw SysError("cannot change ownership of '%1%'", path);
}
#if HAVE_SECCOMP
static void allowSyscall(scmp_filter_ctx ctx, int syscall) {
if (seccomp_rule_add(ctx, SCMP_ACT_ALLOW, syscall, 0) != 0)
throw SysError("unable to add seccomp rule");
}
#define ALLOW_CHMOD_IF_SAFE(ctx, syscall, modePos) \
if (seccomp_rule_add(ctx, SCMP_ACT_ALLOW, syscall, 1, SCMP_A##modePos(SCMP_CMP_MASKED_EQ, S_ISUID | S_ISGID, 0)) != 0 || \
seccomp_rule_add(ctx, SCMP_ACT_ERRNO(EPERM), syscall, 1, SCMP_A##modePos(SCMP_CMP_MASKED_EQ, S_ISUID, S_ISUID)) != 0 || \
seccomp_rule_add(ctx, SCMP_ACT_ERRNO(EPERM), syscall, 1, SCMP_A##modePos(SCMP_CMP_MASKED_EQ, S_ISGID, S_ISGID)) != 0) \
throw SysError("unable to add seccomp rule");
#endif
void setupSeccomp()
{
@ -1370,7 +1383,9 @@ void setupSeccomp()
#if HAVE_SECCOMP
scmp_filter_ctx ctx;
if (!(ctx = seccomp_init(SCMP_ACT_ALLOW)))
// Pretend that syscalls we don't yet know about don't exist.
// This is the best option for compatibility: after all, they did in fact not exist not too long ago.
if (!(ctx = seccomp_init(SCMP_ACT_ERRNO(ENOSYS))))
throw SysError("unable to initialize seccomp mode 2");
Finally cleanup([&]() {
@ -1405,28 +1420,514 @@ void setupSeccomp()
seccomp_arch_add(ctx, SCMP_ARCH_MIPSEL64N32) != 0)
printError("unable to add mips64el-*abin32 seccomp architecture");
/* Prevent builders from creating setuid/setgid binaries. */
for (int perm : { S_ISUID, S_ISGID }) {
if (seccomp_rule_add(ctx, SCMP_ACT_ERRNO(EPERM), SCMP_SYS(chmod), 1,
SCMP_A1(SCMP_CMP_MASKED_EQ, (scmp_datum_t) perm, (scmp_datum_t) perm)) != 0)
throw SysError("unable to add seccomp rule");
// This list is intended for machine consumption.
// Please keep its format, order and BEGIN/END markers.
//
// Currently, it is up to date with libseccomp 2.5.5 and glibc 2.39.
// Run check-syscalls to determine which new syscalls should be added.
// New syscalls must be audited and handled in a way that blocks the following dangerous operations:
// * Creation of non-empty setuid/setgid files
// * Creation of extended attributes (including ACLs)
//
// BEGIN extract-syscalls
allowSyscall(ctx, SCMP_SYS(accept));
allowSyscall(ctx, SCMP_SYS(accept4));
allowSyscall(ctx, SCMP_SYS(access));
allowSyscall(ctx, SCMP_SYS(acct));
allowSyscall(ctx, SCMP_SYS(add_key));
allowSyscall(ctx, SCMP_SYS(adjtimex));
allowSyscall(ctx, SCMP_SYS(afs_syscall));
allowSyscall(ctx, SCMP_SYS(alarm));
allowSyscall(ctx, SCMP_SYS(arch_prctl));
allowSyscall(ctx, SCMP_SYS(arm_fadvise64_64));
allowSyscall(ctx, SCMP_SYS(arm_sync_file_range));
allowSyscall(ctx, SCMP_SYS(bdflush));
allowSyscall(ctx, SCMP_SYS(bind));
allowSyscall(ctx, SCMP_SYS(bpf));
allowSyscall(ctx, SCMP_SYS(break));
allowSyscall(ctx, SCMP_SYS(breakpoint));
allowSyscall(ctx, SCMP_SYS(brk));
allowSyscall(ctx, SCMP_SYS(cachectl));
allowSyscall(ctx, SCMP_SYS(cacheflush));
allowSyscall(ctx, SCMP_SYS(cachestat));
allowSyscall(ctx, SCMP_SYS(capget));
allowSyscall(ctx, SCMP_SYS(capset));
allowSyscall(ctx, SCMP_SYS(chdir));
// skip chmod (dangerous)
allowSyscall(ctx, SCMP_SYS(chown));
allowSyscall(ctx, SCMP_SYS(chown32));
allowSyscall(ctx, SCMP_SYS(chroot));
allowSyscall(ctx, SCMP_SYS(clock_adjtime));
allowSyscall(ctx, SCMP_SYS(clock_adjtime64));
allowSyscall(ctx, SCMP_SYS(clock_getres));
allowSyscall(ctx, SCMP_SYS(clock_getres_time64));
allowSyscall(ctx, SCMP_SYS(clock_gettime));
allowSyscall(ctx, SCMP_SYS(clock_gettime64));
allowSyscall(ctx, SCMP_SYS(clock_nanosleep));
allowSyscall(ctx, SCMP_SYS(clock_nanosleep_time64));
allowSyscall(ctx, SCMP_SYS(clock_settime));
allowSyscall(ctx, SCMP_SYS(clock_settime64));
allowSyscall(ctx, SCMP_SYS(clone));
allowSyscall(ctx, SCMP_SYS(clone3));
allowSyscall(ctx, SCMP_SYS(close));
allowSyscall(ctx, SCMP_SYS(close_range));
allowSyscall(ctx, SCMP_SYS(connect));
allowSyscall(ctx, SCMP_SYS(copy_file_range));
allowSyscall(ctx, SCMP_SYS(creat));
allowSyscall(ctx, SCMP_SYS(create_module));
allowSyscall(ctx, SCMP_SYS(delete_module));
allowSyscall(ctx, SCMP_SYS(dup));
allowSyscall(ctx, SCMP_SYS(dup2));
allowSyscall(ctx, SCMP_SYS(dup3));
allowSyscall(ctx, SCMP_SYS(epoll_create));
allowSyscall(ctx, SCMP_SYS(epoll_create1));
allowSyscall(ctx, SCMP_SYS(epoll_ctl));
allowSyscall(ctx, SCMP_SYS(epoll_ctl_old));
allowSyscall(ctx, SCMP_SYS(epoll_pwait));
allowSyscall(ctx, SCMP_SYS(epoll_pwait2));
allowSyscall(ctx, SCMP_SYS(epoll_wait));
allowSyscall(ctx, SCMP_SYS(epoll_wait_old));
allowSyscall(ctx, SCMP_SYS(eventfd));
allowSyscall(ctx, SCMP_SYS(eventfd2));
allowSyscall(ctx, SCMP_SYS(execve));
allowSyscall(ctx, SCMP_SYS(execveat));
allowSyscall(ctx, SCMP_SYS(exit));
allowSyscall(ctx, SCMP_SYS(exit_group));
allowSyscall(ctx, SCMP_SYS(faccessat));
allowSyscall(ctx, SCMP_SYS(faccessat2));
allowSyscall(ctx, SCMP_SYS(fadvise64));
allowSyscall(ctx, SCMP_SYS(fadvise64_64));
allowSyscall(ctx, SCMP_SYS(fallocate));
allowSyscall(ctx, SCMP_SYS(fanotify_init));
allowSyscall(ctx, SCMP_SYS(fanotify_mark));
allowSyscall(ctx, SCMP_SYS(fchdir));
// skip fchmod (dangerous)
// skip fchmodat (dangerous)
// skip fchmodat2 (dangerous)
allowSyscall(ctx, SCMP_SYS(fchown));
allowSyscall(ctx, SCMP_SYS(fchown32));
allowSyscall(ctx, SCMP_SYS(fchownat));
allowSyscall(ctx, SCMP_SYS(fcntl));
allowSyscall(ctx, SCMP_SYS(fcntl64));
allowSyscall(ctx, SCMP_SYS(fdatasync));
allowSyscall(ctx, SCMP_SYS(fgetxattr));
allowSyscall(ctx, SCMP_SYS(finit_module));
allowSyscall(ctx, SCMP_SYS(flistxattr));
allowSyscall(ctx, SCMP_SYS(flock));
allowSyscall(ctx, SCMP_SYS(fork));
allowSyscall(ctx, SCMP_SYS(fremovexattr));
allowSyscall(ctx, SCMP_SYS(fsconfig));
// skip fsetxattr (dangerous)
allowSyscall(ctx, SCMP_SYS(fsmount));
allowSyscall(ctx, SCMP_SYS(fsopen));
allowSyscall(ctx, SCMP_SYS(fspick));
allowSyscall(ctx, SCMP_SYS(fstat));
allowSyscall(ctx, SCMP_SYS(fstat64));
allowSyscall(ctx, SCMP_SYS(fstatat64));
allowSyscall(ctx, SCMP_SYS(fstatfs));
allowSyscall(ctx, SCMP_SYS(fstatfs64));
allowSyscall(ctx, SCMP_SYS(fsync));
allowSyscall(ctx, SCMP_SYS(ftime));
allowSyscall(ctx, SCMP_SYS(ftruncate));
allowSyscall(ctx, SCMP_SYS(ftruncate64));
allowSyscall(ctx, SCMP_SYS(futex));
allowSyscall(ctx, SCMP_SYS(futex_requeue));
allowSyscall(ctx, SCMP_SYS(futex_time64));
allowSyscall(ctx, SCMP_SYS(futex_wait));
allowSyscall(ctx, SCMP_SYS(futex_waitv));
allowSyscall(ctx, SCMP_SYS(futex_wake));
allowSyscall(ctx, SCMP_SYS(futimesat));
allowSyscall(ctx, SCMP_SYS(getcpu));
allowSyscall(ctx, SCMP_SYS(getcwd));
allowSyscall(ctx, SCMP_SYS(getdents));
allowSyscall(ctx, SCMP_SYS(getdents64));
allowSyscall(ctx, SCMP_SYS(getegid));
allowSyscall(ctx, SCMP_SYS(getegid32));
allowSyscall(ctx, SCMP_SYS(geteuid));
allowSyscall(ctx, SCMP_SYS(geteuid32));
allowSyscall(ctx, SCMP_SYS(getgid));
allowSyscall(ctx, SCMP_SYS(getgid32));
allowSyscall(ctx, SCMP_SYS(getgroups));
allowSyscall(ctx, SCMP_SYS(getgroups32));
allowSyscall(ctx, SCMP_SYS(getitimer));
allowSyscall(ctx, SCMP_SYS(get_kernel_syms));
allowSyscall(ctx, SCMP_SYS(get_mempolicy));
allowSyscall(ctx, SCMP_SYS(getpeername));
allowSyscall(ctx, SCMP_SYS(getpgid));
allowSyscall(ctx, SCMP_SYS(getpgrp));
allowSyscall(ctx, SCMP_SYS(getpid));
allowSyscall(ctx, SCMP_SYS(getpmsg));
allowSyscall(ctx, SCMP_SYS(getppid));
allowSyscall(ctx, SCMP_SYS(getpriority));
allowSyscall(ctx, SCMP_SYS(getrandom));
allowSyscall(ctx, SCMP_SYS(getresgid));
allowSyscall(ctx, SCMP_SYS(getresgid32));
allowSyscall(ctx, SCMP_SYS(getresuid));
allowSyscall(ctx, SCMP_SYS(getresuid32));
allowSyscall(ctx, SCMP_SYS(getrlimit));
allowSyscall(ctx, SCMP_SYS(get_robust_list));
allowSyscall(ctx, SCMP_SYS(getrusage));
allowSyscall(ctx, SCMP_SYS(getsid));
allowSyscall(ctx, SCMP_SYS(getsockname));
allowSyscall(ctx, SCMP_SYS(getsockopt));
allowSyscall(ctx, SCMP_SYS(get_thread_area));
allowSyscall(ctx, SCMP_SYS(gettid));
allowSyscall(ctx, SCMP_SYS(gettimeofday));
allowSyscall(ctx, SCMP_SYS(get_tls));
allowSyscall(ctx, SCMP_SYS(getuid));
allowSyscall(ctx, SCMP_SYS(getuid32));
allowSyscall(ctx, SCMP_SYS(getxattr));
allowSyscall(ctx, SCMP_SYS(gtty));
allowSyscall(ctx, SCMP_SYS(idle));
allowSyscall(ctx, SCMP_SYS(init_module));
allowSyscall(ctx, SCMP_SYS(inotify_add_watch));
allowSyscall(ctx, SCMP_SYS(inotify_init));
allowSyscall(ctx, SCMP_SYS(inotify_init1));
allowSyscall(ctx, SCMP_SYS(inotify_rm_watch));
allowSyscall(ctx, SCMP_SYS(io_cancel));
allowSyscall(ctx, SCMP_SYS(ioctl));
allowSyscall(ctx, SCMP_SYS(io_destroy));
allowSyscall(ctx, SCMP_SYS(io_getevents));
allowSyscall(ctx, SCMP_SYS(ioperm));
allowSyscall(ctx, SCMP_SYS(io_pgetevents));
allowSyscall(ctx, SCMP_SYS(io_pgetevents_time64));
allowSyscall(ctx, SCMP_SYS(iopl));
allowSyscall(ctx, SCMP_SYS(ioprio_get));
allowSyscall(ctx, SCMP_SYS(ioprio_set));
allowSyscall(ctx, SCMP_SYS(io_setup));
allowSyscall(ctx, SCMP_SYS(io_submit));
// skip io_uring_enter (may become dangerous)
// skip io_uring_register (may become dangerous)
// skip io_uring_setup (may become dangerous)
allowSyscall(ctx, SCMP_SYS(ipc));
allowSyscall(ctx, SCMP_SYS(kcmp));
allowSyscall(ctx, SCMP_SYS(kexec_file_load));
allowSyscall(ctx, SCMP_SYS(kexec_load));
allowSyscall(ctx, SCMP_SYS(keyctl));
allowSyscall(ctx, SCMP_SYS(kill));
allowSyscall(ctx, SCMP_SYS(landlock_add_rule));
allowSyscall(ctx, SCMP_SYS(landlock_create_ruleset));
allowSyscall(ctx, SCMP_SYS(landlock_restrict_self));
allowSyscall(ctx, SCMP_SYS(lchown));
allowSyscall(ctx, SCMP_SYS(lchown32));
allowSyscall(ctx, SCMP_SYS(lgetxattr));
allowSyscall(ctx, SCMP_SYS(link));
allowSyscall(ctx, SCMP_SYS(linkat));
allowSyscall(ctx, SCMP_SYS(listen));
allowSyscall(ctx, SCMP_SYS(listxattr));
allowSyscall(ctx, SCMP_SYS(llistxattr));
allowSyscall(ctx, SCMP_SYS(_llseek));
allowSyscall(ctx, SCMP_SYS(lock));
allowSyscall(ctx, SCMP_SYS(lookup_dcookie));
allowSyscall(ctx, SCMP_SYS(lremovexattr));
allowSyscall(ctx, SCMP_SYS(lseek));
// skip lsetxattr (dangerous)
allowSyscall(ctx, SCMP_SYS(lstat));
allowSyscall(ctx, SCMP_SYS(lstat64));
allowSyscall(ctx, SCMP_SYS(madvise));
allowSyscall(ctx, SCMP_SYS(map_shadow_stack));
allowSyscall(ctx, SCMP_SYS(mbind));
allowSyscall(ctx, SCMP_SYS(membarrier));
allowSyscall(ctx, SCMP_SYS(memfd_create));
allowSyscall(ctx, SCMP_SYS(memfd_secret));
allowSyscall(ctx, SCMP_SYS(migrate_pages));
allowSyscall(ctx, SCMP_SYS(mincore));
allowSyscall(ctx, SCMP_SYS(mkdir));
allowSyscall(ctx, SCMP_SYS(mkdirat));
allowSyscall(ctx, SCMP_SYS(mknod));
allowSyscall(ctx, SCMP_SYS(mknodat));
allowSyscall(ctx, SCMP_SYS(mlock));
allowSyscall(ctx, SCMP_SYS(mlock2));
allowSyscall(ctx, SCMP_SYS(mlockall));
allowSyscall(ctx, SCMP_SYS(mmap));
allowSyscall(ctx, SCMP_SYS(mmap2));
allowSyscall(ctx, SCMP_SYS(modify_ldt));
allowSyscall(ctx, SCMP_SYS(mount));
allowSyscall(ctx, SCMP_SYS(mount_setattr));
allowSyscall(ctx, SCMP_SYS(move_mount));
allowSyscall(ctx, SCMP_SYS(move_pages));
allowSyscall(ctx, SCMP_SYS(mprotect));
allowSyscall(ctx, SCMP_SYS(mpx));
allowSyscall(ctx, SCMP_SYS(mq_getsetattr));
allowSyscall(ctx, SCMP_SYS(mq_notify));
allowSyscall(ctx, SCMP_SYS(mq_open));
allowSyscall(ctx, SCMP_SYS(mq_timedreceive));
allowSyscall(ctx, SCMP_SYS(mq_timedreceive_time64));
allowSyscall(ctx, SCMP_SYS(mq_timedsend));
allowSyscall(ctx, SCMP_SYS(mq_timedsend_time64));
allowSyscall(ctx, SCMP_SYS(mq_unlink));
allowSyscall(ctx, SCMP_SYS(mremap));
allowSyscall(ctx, SCMP_SYS(msgctl));
allowSyscall(ctx, SCMP_SYS(msgget));
allowSyscall(ctx, SCMP_SYS(msgrcv));
allowSyscall(ctx, SCMP_SYS(msgsnd));
allowSyscall(ctx, SCMP_SYS(msync));
allowSyscall(ctx, SCMP_SYS(multiplexer));
allowSyscall(ctx, SCMP_SYS(munlock));
allowSyscall(ctx, SCMP_SYS(munlockall));
allowSyscall(ctx, SCMP_SYS(munmap));
allowSyscall(ctx, SCMP_SYS(name_to_handle_at));
allowSyscall(ctx, SCMP_SYS(nanosleep));
allowSyscall(ctx, SCMP_SYS(newfstatat));
allowSyscall(ctx, SCMP_SYS(_newselect));
allowSyscall(ctx, SCMP_SYS(nfsservctl));
allowSyscall(ctx, SCMP_SYS(nice));
allowSyscall(ctx, SCMP_SYS(oldfstat));
allowSyscall(ctx, SCMP_SYS(oldlstat));
allowSyscall(ctx, SCMP_SYS(oldolduname));
allowSyscall(ctx, SCMP_SYS(oldstat));
allowSyscall(ctx, SCMP_SYS(olduname));
allowSyscall(ctx, SCMP_SYS(open));
allowSyscall(ctx, SCMP_SYS(openat));
allowSyscall(ctx, SCMP_SYS(openat2));
allowSyscall(ctx, SCMP_SYS(open_by_handle_at));
allowSyscall(ctx, SCMP_SYS(open_tree));
allowSyscall(ctx, SCMP_SYS(pause));
allowSyscall(ctx, SCMP_SYS(pciconfig_iobase));
allowSyscall(ctx, SCMP_SYS(pciconfig_read));
allowSyscall(ctx, SCMP_SYS(pciconfig_write));
allowSyscall(ctx, SCMP_SYS(perf_event_open));
allowSyscall(ctx, SCMP_SYS(personality));
allowSyscall(ctx, SCMP_SYS(pidfd_getfd));
allowSyscall(ctx, SCMP_SYS(pidfd_open));
allowSyscall(ctx, SCMP_SYS(pidfd_send_signal));
allowSyscall(ctx, SCMP_SYS(pipe));
allowSyscall(ctx, SCMP_SYS(pipe2));
allowSyscall(ctx, SCMP_SYS(pivot_root));
allowSyscall(ctx, SCMP_SYS(pkey_alloc));
allowSyscall(ctx, SCMP_SYS(pkey_free));
allowSyscall(ctx, SCMP_SYS(pkey_mprotect));
allowSyscall(ctx, SCMP_SYS(poll));
allowSyscall(ctx, SCMP_SYS(ppoll));
allowSyscall(ctx, SCMP_SYS(ppoll_time64));
allowSyscall(ctx, SCMP_SYS(prctl));
allowSyscall(ctx, SCMP_SYS(pread64));
allowSyscall(ctx, SCMP_SYS(preadv));
allowSyscall(ctx, SCMP_SYS(preadv2));
allowSyscall(ctx, SCMP_SYS(prlimit64));
allowSyscall(ctx, SCMP_SYS(process_madvise));
allowSyscall(ctx, SCMP_SYS(process_mrelease));
allowSyscall(ctx, SCMP_SYS(process_vm_readv));
allowSyscall(ctx, SCMP_SYS(process_vm_writev));
allowSyscall(ctx, SCMP_SYS(prof));
allowSyscall(ctx, SCMP_SYS(profil));
allowSyscall(ctx, SCMP_SYS(pselect6));
allowSyscall(ctx, SCMP_SYS(pselect6_time64));
allowSyscall(ctx, SCMP_SYS(ptrace));
allowSyscall(ctx, SCMP_SYS(putpmsg));
allowSyscall(ctx, SCMP_SYS(pwrite64));
allowSyscall(ctx, SCMP_SYS(pwritev));
allowSyscall(ctx, SCMP_SYS(pwritev2));
allowSyscall(ctx, SCMP_SYS(query_module));
allowSyscall(ctx, SCMP_SYS(quotactl));
allowSyscall(ctx, SCMP_SYS(quotactl_fd));
allowSyscall(ctx, SCMP_SYS(read));
allowSyscall(ctx, SCMP_SYS(readahead));
allowSyscall(ctx, SCMP_SYS(readdir));
allowSyscall(ctx, SCMP_SYS(readlink));
allowSyscall(ctx, SCMP_SYS(readlinkat));
allowSyscall(ctx, SCMP_SYS(readv));
allowSyscall(ctx, SCMP_SYS(reboot));
allowSyscall(ctx, SCMP_SYS(recv));
allowSyscall(ctx, SCMP_SYS(recvfrom));
allowSyscall(ctx, SCMP_SYS(recvmmsg));
allowSyscall(ctx, SCMP_SYS(recvmmsg_time64));
allowSyscall(ctx, SCMP_SYS(recvmsg));
allowSyscall(ctx, SCMP_SYS(remap_file_pages));
allowSyscall(ctx, SCMP_SYS(removexattr));
allowSyscall(ctx, SCMP_SYS(rename));
allowSyscall(ctx, SCMP_SYS(renameat));
allowSyscall(ctx, SCMP_SYS(renameat2));
allowSyscall(ctx, SCMP_SYS(request_key));
allowSyscall(ctx, SCMP_SYS(restart_syscall));
allowSyscall(ctx, SCMP_SYS(riscv_flush_icache));
allowSyscall(ctx, SCMP_SYS(rmdir));
allowSyscall(ctx, SCMP_SYS(rseq));
allowSyscall(ctx, SCMP_SYS(rtas));
allowSyscall(ctx, SCMP_SYS(rt_sigaction));
allowSyscall(ctx, SCMP_SYS(rt_sigpending));
allowSyscall(ctx, SCMP_SYS(rt_sigprocmask));
allowSyscall(ctx, SCMP_SYS(rt_sigqueueinfo));
allowSyscall(ctx, SCMP_SYS(rt_sigreturn));
allowSyscall(ctx, SCMP_SYS(rt_sigsuspend));
allowSyscall(ctx, SCMP_SYS(rt_sigtimedwait));
allowSyscall(ctx, SCMP_SYS(rt_sigtimedwait_time64));
allowSyscall(ctx, SCMP_SYS(rt_tgsigqueueinfo));
allowSyscall(ctx, SCMP_SYS(s390_guarded_storage));
allowSyscall(ctx, SCMP_SYS(s390_pci_mmio_read));
allowSyscall(ctx, SCMP_SYS(s390_pci_mmio_write));
allowSyscall(ctx, SCMP_SYS(s390_runtime_instr));
allowSyscall(ctx, SCMP_SYS(s390_sthyi));
allowSyscall(ctx, SCMP_SYS(sched_getaffinity));
allowSyscall(ctx, SCMP_SYS(sched_getattr));
allowSyscall(ctx, SCMP_SYS(sched_getparam));
allowSyscall(ctx, SCMP_SYS(sched_get_priority_max));
allowSyscall(ctx, SCMP_SYS(sched_get_priority_min));
allowSyscall(ctx, SCMP_SYS(sched_getscheduler));
allowSyscall(ctx, SCMP_SYS(sched_rr_get_interval));
allowSyscall(ctx, SCMP_SYS(sched_rr_get_interval_time64));
allowSyscall(ctx, SCMP_SYS(sched_setaffinity));
allowSyscall(ctx, SCMP_SYS(sched_setattr));
allowSyscall(ctx, SCMP_SYS(sched_setparam));
allowSyscall(ctx, SCMP_SYS(sched_setscheduler));
allowSyscall(ctx, SCMP_SYS(sched_yield));
allowSyscall(ctx, SCMP_SYS(seccomp));
allowSyscall(ctx, SCMP_SYS(security));
allowSyscall(ctx, SCMP_SYS(select));
allowSyscall(ctx, SCMP_SYS(semctl));
allowSyscall(ctx, SCMP_SYS(semget));
allowSyscall(ctx, SCMP_SYS(semop));
allowSyscall(ctx, SCMP_SYS(semtimedop));
allowSyscall(ctx, SCMP_SYS(semtimedop_time64));
allowSyscall(ctx, SCMP_SYS(send));
allowSyscall(ctx, SCMP_SYS(sendfile));
allowSyscall(ctx, SCMP_SYS(sendfile64));
allowSyscall(ctx, SCMP_SYS(sendmmsg));
allowSyscall(ctx, SCMP_SYS(sendmsg));
allowSyscall(ctx, SCMP_SYS(sendto));
allowSyscall(ctx, SCMP_SYS(setdomainname));
allowSyscall(ctx, SCMP_SYS(setfsgid));
allowSyscall(ctx, SCMP_SYS(setfsgid32));
allowSyscall(ctx, SCMP_SYS(setfsuid));
allowSyscall(ctx, SCMP_SYS(setfsuid32));
allowSyscall(ctx, SCMP_SYS(setgid));
allowSyscall(ctx, SCMP_SYS(setgid32));
allowSyscall(ctx, SCMP_SYS(setgroups));
allowSyscall(ctx, SCMP_SYS(setgroups32));
allowSyscall(ctx, SCMP_SYS(sethostname));
allowSyscall(ctx, SCMP_SYS(setitimer));
allowSyscall(ctx, SCMP_SYS(set_mempolicy));
allowSyscall(ctx, SCMP_SYS(set_mempolicy_home_node));
allowSyscall(ctx, SCMP_SYS(setns));
allowSyscall(ctx, SCMP_SYS(setpgid));
allowSyscall(ctx, SCMP_SYS(setpriority));
allowSyscall(ctx, SCMP_SYS(setregid));
allowSyscall(ctx, SCMP_SYS(setregid32));
allowSyscall(ctx, SCMP_SYS(setresgid));
allowSyscall(ctx, SCMP_SYS(setresgid32));
allowSyscall(ctx, SCMP_SYS(setresuid));
allowSyscall(ctx, SCMP_SYS(setresuid32));
allowSyscall(ctx, SCMP_SYS(setreuid));
allowSyscall(ctx, SCMP_SYS(setreuid32));
allowSyscall(ctx, SCMP_SYS(setrlimit));
allowSyscall(ctx, SCMP_SYS(set_robust_list));
allowSyscall(ctx, SCMP_SYS(setsid));
allowSyscall(ctx, SCMP_SYS(setsockopt));
allowSyscall(ctx, SCMP_SYS(set_thread_area));
allowSyscall(ctx, SCMP_SYS(set_tid_address));
allowSyscall(ctx, SCMP_SYS(settimeofday));
allowSyscall(ctx, SCMP_SYS(set_tls));
allowSyscall(ctx, SCMP_SYS(setuid));
allowSyscall(ctx, SCMP_SYS(setuid32));
// skip setxattr (dangerous)
allowSyscall(ctx, SCMP_SYS(sgetmask));
allowSyscall(ctx, SCMP_SYS(shmat));
allowSyscall(ctx, SCMP_SYS(shmctl));
allowSyscall(ctx, SCMP_SYS(shmdt));
allowSyscall(ctx, SCMP_SYS(shmget));
allowSyscall(ctx, SCMP_SYS(shutdown));
allowSyscall(ctx, SCMP_SYS(sigaction));
allowSyscall(ctx, SCMP_SYS(sigaltstack));
allowSyscall(ctx, SCMP_SYS(signal));
allowSyscall(ctx, SCMP_SYS(signalfd));
allowSyscall(ctx, SCMP_SYS(signalfd4));
allowSyscall(ctx, SCMP_SYS(sigpending));
allowSyscall(ctx, SCMP_SYS(sigprocmask));
allowSyscall(ctx, SCMP_SYS(sigreturn));
allowSyscall(ctx, SCMP_SYS(sigsuspend));
allowSyscall(ctx, SCMP_SYS(socket));
allowSyscall(ctx, SCMP_SYS(socketcall));
allowSyscall(ctx, SCMP_SYS(socketpair));
allowSyscall(ctx, SCMP_SYS(splice));
allowSyscall(ctx, SCMP_SYS(spu_create));
allowSyscall(ctx, SCMP_SYS(spu_run));
allowSyscall(ctx, SCMP_SYS(ssetmask));
allowSyscall(ctx, SCMP_SYS(stat));
allowSyscall(ctx, SCMP_SYS(stat64));
allowSyscall(ctx, SCMP_SYS(statfs));
allowSyscall(ctx, SCMP_SYS(statfs64));
allowSyscall(ctx, SCMP_SYS(statx));
allowSyscall(ctx, SCMP_SYS(stime));
allowSyscall(ctx, SCMP_SYS(stty));
allowSyscall(ctx, SCMP_SYS(subpage_prot));
allowSyscall(ctx, SCMP_SYS(swapcontext));
allowSyscall(ctx, SCMP_SYS(swapoff));
allowSyscall(ctx, SCMP_SYS(swapon));
allowSyscall(ctx, SCMP_SYS(switch_endian));
allowSyscall(ctx, SCMP_SYS(symlink));
allowSyscall(ctx, SCMP_SYS(symlinkat));
allowSyscall(ctx, SCMP_SYS(sync));
allowSyscall(ctx, SCMP_SYS(sync_file_range));
allowSyscall(ctx, SCMP_SYS(sync_file_range2));
allowSyscall(ctx, SCMP_SYS(syncfs));
allowSyscall(ctx, SCMP_SYS(syscall));
allowSyscall(ctx, SCMP_SYS(_sysctl));
allowSyscall(ctx, SCMP_SYS(sys_debug_setcontext));
allowSyscall(ctx, SCMP_SYS(sysfs));
allowSyscall(ctx, SCMP_SYS(sysinfo));
allowSyscall(ctx, SCMP_SYS(syslog));
allowSyscall(ctx, SCMP_SYS(sysmips));
allowSyscall(ctx, SCMP_SYS(tee));
allowSyscall(ctx, SCMP_SYS(tgkill));
allowSyscall(ctx, SCMP_SYS(time));
allowSyscall(ctx, SCMP_SYS(timer_create));
allowSyscall(ctx, SCMP_SYS(timer_delete));
allowSyscall(ctx, SCMP_SYS(timerfd));
allowSyscall(ctx, SCMP_SYS(timerfd_create));
allowSyscall(ctx, SCMP_SYS(timerfd_gettime));
allowSyscall(ctx, SCMP_SYS(timerfd_gettime64));
allowSyscall(ctx, SCMP_SYS(timerfd_settime));
allowSyscall(ctx, SCMP_SYS(timerfd_settime64));
allowSyscall(ctx, SCMP_SYS(timer_getoverrun));
allowSyscall(ctx, SCMP_SYS(timer_gettime));
allowSyscall(ctx, SCMP_SYS(timer_gettime64));
allowSyscall(ctx, SCMP_SYS(timer_settime));
allowSyscall(ctx, SCMP_SYS(timer_settime64));
allowSyscall(ctx, SCMP_SYS(times));
allowSyscall(ctx, SCMP_SYS(tkill));
allowSyscall(ctx, SCMP_SYS(truncate));
allowSyscall(ctx, SCMP_SYS(truncate64));
allowSyscall(ctx, SCMP_SYS(tuxcall));
allowSyscall(ctx, SCMP_SYS(ugetrlimit));
allowSyscall(ctx, SCMP_SYS(ulimit));
allowSyscall(ctx, SCMP_SYS(umask));
allowSyscall(ctx, SCMP_SYS(umount));
allowSyscall(ctx, SCMP_SYS(umount2));
allowSyscall(ctx, SCMP_SYS(uname));
allowSyscall(ctx, SCMP_SYS(unlink));
allowSyscall(ctx, SCMP_SYS(unlinkat));
allowSyscall(ctx, SCMP_SYS(unshare));
allowSyscall(ctx, SCMP_SYS(uselib));
allowSyscall(ctx, SCMP_SYS(userfaultfd));
allowSyscall(ctx, SCMP_SYS(usr26));
allowSyscall(ctx, SCMP_SYS(usr32));
allowSyscall(ctx, SCMP_SYS(ustat));
allowSyscall(ctx, SCMP_SYS(utime));
allowSyscall(ctx, SCMP_SYS(utimensat));
allowSyscall(ctx, SCMP_SYS(utimensat_time64));
allowSyscall(ctx, SCMP_SYS(utimes));
allowSyscall(ctx, SCMP_SYS(vfork));
allowSyscall(ctx, SCMP_SYS(vhangup));
allowSyscall(ctx, SCMP_SYS(vm86));
allowSyscall(ctx, SCMP_SYS(vm86old));
allowSyscall(ctx, SCMP_SYS(vmsplice));
allowSyscall(ctx, SCMP_SYS(vserver));
allowSyscall(ctx, SCMP_SYS(wait4));
allowSyscall(ctx, SCMP_SYS(waitid));
allowSyscall(ctx, SCMP_SYS(waitpid));
allowSyscall(ctx, SCMP_SYS(write));
allowSyscall(ctx, SCMP_SYS(writev));
// END extract-syscalls
if (seccomp_rule_add(ctx, SCMP_ACT_ERRNO(EPERM), SCMP_SYS(fchmod), 1,
SCMP_A1(SCMP_CMP_MASKED_EQ, (scmp_datum_t) perm, (scmp_datum_t) perm)) != 0)
throw SysError("unable to add seccomp rule");
// chmod family: prevent adding setuid/setgid bits to existing files.
// The Nix store does not support setuid/setgid, and even their temporary creation can weaken the security of the sandbox.
ALLOW_CHMOD_IF_SAFE(ctx, SCMP_SYS(chmod), 1);
ALLOW_CHMOD_IF_SAFE(ctx, SCMP_SYS(fchmod), 1);
ALLOW_CHMOD_IF_SAFE(ctx, SCMP_SYS(fchmodat), 2);
ALLOW_CHMOD_IF_SAFE(ctx, SCMP_SYS(fchmodat2), 2);
if (seccomp_rule_add(ctx, SCMP_ACT_ERRNO(EPERM), SCMP_SYS(fchmodat), 1,
SCMP_A2(SCMP_CMP_MASKED_EQ, (scmp_datum_t) perm, (scmp_datum_t) perm)) != 0)
throw SysError("unable to add seccomp rule");
if (seccomp_rule_add(ctx, SCMP_ACT_ERRNO(EPERM), NIX_SYSCALL_FCHMODAT2, 1,
SCMP_A2(SCMP_CMP_MASKED_EQ, (scmp_datum_t) perm, (scmp_datum_t) perm)) != 0)
throw SysError("unable to add seccomp rule");
}
/* Prevent builders from creating EAs or ACLs. Not all filesystems
support these, and they're not allowed in the Nix store because
they're not representable in the NAR serialisation. */
// setxattr family: prevent creation of extended attributes or ACLs.
// Not all filesystems support them, and they're incompatible with the NAR format.
if (seccomp_rule_add(ctx, SCMP_ACT_ERRNO(ENOTSUP), SCMP_SYS(setxattr), 0) != 0 ||
seccomp_rule_add(ctx, SCMP_ACT_ERRNO(ENOTSUP), SCMP_SYS(lsetxattr), 0) != 0 ||
seccomp_rule_add(ctx, SCMP_ACT_ERRNO(ENOTSUP), SCMP_SYS(fsetxattr), 0) != 0)
@ -1460,11 +1961,7 @@ void LocalDerivationGoal::runChild()
commonChildInit();
try {
setupSeccomp();
} catch (...) {
if (buildUser) throw;
}
setupSeccomp();
bool setUser = true;
@ -1882,7 +2379,7 @@ void LocalDerivationGoal::runChild()
sandboxArgs.push_back("_ALLOW_LOCAL_NETWORKING");
sandboxArgs.push_back("1");
}
if (sandbox_init_with_parameters(sandboxProfile.c_str(), 0, stringsToCharPtrs(sandboxArgs).data(), NULL)) {
if (sandbox_init_with_parameters(sandboxProfile.c_str(), 0, stringsToCharPtrs(sandboxArgs).data(), nullptr)) {
writeFull(STDERR_FILENO, "failed to configure sandbox\n");
_exit(1);
}

View file

@ -86,7 +86,6 @@ void PathSubstitutionGoal::tryNext()
if (substituterFailed) {
worker.failedSubstitutions++;
worker.updateProgress();
}
return;
@ -150,8 +149,6 @@ void PathSubstitutionGoal::tryNext()
? std::make_unique<MaintainCount<uint64_t>>(worker.expectedDownloadSize, narInfo->fileSize)
: nullptr;
worker.updateProgress();
/* Bail out early if this substituter lacks a valid
signature. LocalStore::addToStore() also checks for this, but
only after we've downloaded the path. */
@ -210,13 +207,10 @@ void PathSubstitutionGoal::tryToRun()
}
maintainRunningSubstitutions = std::make_unique<MaintainCount<uint64_t>>(worker.runningSubstitutions);
worker.updateProgress();
outPipe.create();
promise = std::promise<void>();
thr = std::thread([this]() {
thr = std::async(std::launch::async, [this]() {
auto & fetchPath = subPath ? *subPath : storePath;
try {
ReceiveInterrupts receiveInterrupts;
@ -230,16 +224,12 @@ void PathSubstitutionGoal::tryToRun()
copyStorePath(
*sub, worker.store, fetchPath, repair, sub->isTrusted ? NoCheckSigs : CheckSigs
);
promise.set_value();
} catch (const EndOfFile &) {
promise.set_exception(std::make_exception_ptr(EndOfFile(
throw EndOfFile(
"NAR for '%s' fetched from '%s' is incomplete",
sub->printStorePath(fetchPath),
sub->getUri()
)));
} catch (...) {
promise.set_exception(std::current_exception());
);
}
});
@ -253,11 +243,10 @@ void PathSubstitutionGoal::finished()
{
trace("substitute finished");
thr.join();
worker.childTerminated(this);
try {
promise.get_future().get();
thr.get();
} catch (std::exception & e) {
printError(e.what());
@ -296,8 +285,6 @@ void PathSubstitutionGoal::finished()
worker.doneNarSize += maintainExpectedNar->delta;
maintainExpectedNar.reset();
worker.updateProgress();
done(ecSuccess, BuildResult::Substituted);
}
@ -307,18 +294,12 @@ void PathSubstitutionGoal::handleChildOutput(int fd, std::string_view data)
}
void PathSubstitutionGoal::handleEOF(int fd)
{
if (fd == outPipe.readSide.get()) worker.wakeUp(shared_from_this());
}
void PathSubstitutionGoal::cleanup()
{
try {
if (thr.joinable()) {
if (thr.valid()) {
// FIXME: signal worker thread to quit.
thr.join();
thr.get();
worker.childTerminated(this);
}

View file

@ -50,9 +50,7 @@ struct PathSubstitutionGoal : public Goal
/**
* The substituter thread.
*/
std::thread thr;
std::promise<void> promise;
std::future<void> thr;
/**
* Whether to try to repair a valid path.
@ -112,7 +110,6 @@ public:
* Callback used by the worker to write to the log.
*/
void handleChildOutput(int fd, std::string_view data) override;
void handleEOF(int fd) override;
/* Called by destructor, can't be overridden */
void cleanup() override final;

View file

@ -157,21 +157,13 @@ void Worker::removeGoal(GoalPtr goal)
if (goal->exitCode == Goal::ecFailed && !settings.keepGoing)
topGoals.clear();
}
/* Wake up goals waiting for any goal to finish. */
for (auto & i : waitingForAnyGoal) {
GoalPtr goal = i.lock();
if (goal) wakeUp(goal);
}
waitingForAnyGoal.clear();
}
void Worker::wakeUp(GoalPtr goal)
{
goal->trace("woken up");
addToWeakGoals(awake, goal);
awake.insert(goal);
}
@ -213,7 +205,7 @@ void Worker::childStarted(GoalPtr goal, const std::set<int> & fds,
}
void Worker::childTerminated(Goal * goal, bool wakeSleepers)
void Worker::childTerminated(Goal * goal)
{
auto i = std::find_if(children.begin(), children.end(),
[&](const Child & child) { return child.goal2 == goal; });
@ -236,16 +228,13 @@ void Worker::childTerminated(Goal * goal, bool wakeSleepers)
children.erase(i);
if (wakeSleepers) {
/* Wake up goals waiting for a build slot. */
for (auto & j : wantingToBuild) {
GoalPtr goal = j.lock();
if (goal) wakeUp(goal);
}
wantingToBuild.clear();
/* Wake up goals waiting for a build slot. */
for (auto & j : wantingToBuild) {
GoalPtr goal = j.lock();
if (goal) wakeUp(goal);
}
wantingToBuild.clear();
}
@ -257,21 +246,14 @@ void Worker::waitForBuildSlot(GoalPtr goal)
(isSubstitutionGoal && getNrSubstitutions() < settings.maxSubstitutionJobs))
wakeUp(goal); /* we can do it right away */
else
addToWeakGoals(wantingToBuild, goal);
}
void Worker::waitForAnyGoal(GoalPtr goal)
{
debug("wait for any goal");
addToWeakGoals(waitingForAnyGoal, goal);
wantingToBuild.insert(goal);
}
void Worker::waitForAWhile(GoalPtr goal)
{
debug("wait for a while");
addToWeakGoals(waitingForAWhile, goal);
waitingForAWhile.insert(goal);
}
@ -318,6 +300,19 @@ void Worker::run(const Goals & _topGoals)
for (auto & goal : awake2) {
checkInterrupt();
goal->work();
actDerivations.progress(
doneBuilds, expectedBuilds + doneBuilds, runningBuilds, failedBuilds
);
actSubstitutions.progress(
doneSubstitutions,
expectedSubstitutions + doneSubstitutions,
runningSubstitutions,
failedSubstitutions
);
act.setExpected(actFileTransfer, expectedDownloadSize + doneDownloadSize);
act.setExpected(actCopyPath, expectedNarSize + doneNarSize);
if (topGoals.empty()) break; // stuff may have been cancelled
}
}
@ -333,11 +328,11 @@ void Worker::run(const Goals & _topGoals)
if (getMachines().empty())
throw Error("unable to start any build; either increase '--max-jobs' "
"or enable remote builds."
"\nhttps://nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html");
"\nhttps://docs.lix.systems/manual/lix/stable/advanced-topics/distributed-builds.html");
else
throw Error("unable to start any build; remote machines may not have "
"all required system features."
"\nhttps://nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html");
"\nhttps://docs.lix.systems/manual/lix/stable/advanced-topics/distributed-builds.html");
}
assert(!awake.empty());
@ -429,7 +424,7 @@ void Worker::waitForInput()
GoalPtr goal = j->goal.lock();
assert(goal);
if (goal->exitCode == Goal::ecBusy &&
if (!goal->exitCode.has_value() &&
0 != settings.maxSilentTime &&
j->respectTimeouts &&
after - j->lastOutput >= std::chrono::seconds(settings.maxSilentTime))
@ -440,7 +435,7 @@ void Worker::waitForInput()
continue;
}
else if (goal->exitCode == Goal::ecBusy &&
else if (!goal->exitCode.has_value() &&
0 != settings.buildTimeout &&
j->respectTimeouts &&
after - j->timeStarted >= std::chrono::seconds(settings.buildTimeout))
@ -464,6 +459,7 @@ void Worker::waitForInput()
if (rd == 0 || (rd == -1 && errno == EIO)) {
debug("%1%: got EOF", goal->getName());
goal->handleEOF(k);
wakeUp(goal);
j->fds.erase(k);
} else if (rd == -1) {
if (errno != EINTR)

View file

@ -90,11 +90,6 @@ private:
std::map<StorePath, std::weak_ptr<PathSubstitutionGoal>> substitutionGoals;
std::map<DrvOutput, std::weak_ptr<DrvOutputSubstitutionGoal>> drvOutputSubstitutionGoals;
/**
* Goals waiting for busy paths to be unlocked.
*/
WeakGoals waitingForAnyGoal;
/**
* Goals sleeping for a few seconds (polling a lock).
*/
@ -151,7 +146,6 @@ public:
uint64_t doneSubstitutions = 0;
uint64_t failedSubstitutions = 0;
uint64_t runningSubstitutions = 0;
uint64_t runningCASubstitutions = 0;
uint64_t expectedDownloadSize = 0;
uint64_t doneDownloadSize = 0;
uint64_t expectedNarSize = 0;
@ -228,12 +222,9 @@ public:
bool inBuildSlot, bool respectTimeouts);
/**
* Unregisters a running child process. `wakeSleepers` should be
* false if there is no sense in waking up goals that are sleeping
* because they can't run yet (e.g., there is no free build slot,
* or the hook would still say `postpone`).
* Unregisters a running child process.
*/
void childTerminated(Goal * goal, bool wakeSleepers = true);
void childTerminated(Goal * goal);
/**
* Put `goal` to sleep until a build slot becomes available (which
@ -241,12 +232,6 @@ public:
*/
void waitForBuildSlot(GoalPtr goal);
/**
* Wait for any goal to finish. Pretty indiscriminate way to
* wait for some resource that some other goal is holding.
*/
void waitForAnyGoal(GoalPtr goal);
/**
* Wait for a few seconds and then retry this goal. Used when
* waiting for a lock held by another process. This kind of
@ -295,14 +280,6 @@ public:
bool pathContentsGood(const StorePath & path);
void markContentsGood(const StorePath & path);
void updateProgress()
{
actDerivations.progress(doneBuilds, expectedBuilds + doneBuilds, runningBuilds, failedBuilds);
actSubstitutions.progress(doneSubstitutions, expectedSubstitutions + doneSubstitutions, runningSubstitutions, failedSubstitutions);
act.setExpected(actFileTransfer, expectedDownloadSize + doneDownloadSize);
act.setExpected(actCopyPath, expectedNarSize + doneNarSize);
}
};
}

View file

@ -158,7 +158,7 @@ struct TunnelSink : Sink
{
Sink & to;
TunnelSink(Sink & to) : to(to) { }
void operator () (std::string_view data)
void operator () (std::string_view data) override
{
to << STDERR_WRITE << data;
}

View file

@ -22,8 +22,8 @@
namespace nix {
static std::string gcSocketPath = "/gc-socket/socket";
static std::string gcRootsDir = "gcroots";
constexpr static const std::string_view gcSocketPath = "/gc-socket/socket";
constexpr static const std::string_view gcRootsDir = "gcroots";
static void makeSymlink(const Path & link, const Path & target)
@ -359,16 +359,34 @@ void LocalStore::findRuntimeRoots(Roots & roots, bool censor)
}
struct GCLimitReached { };
struct GCLimitReached : std::exception { };
void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
{
bool shouldDelete = options.action == GCOptions::gcDeleteDead || options.action == GCOptions::gcDeleteSpecific;
bool gcKeepOutputs = settings.gcKeepOutputs;
bool gcKeepDerivations = settings.gcKeepDerivations;
/**
* Delegate class to expose just the operations required to perform GC on a store.
*/
class GCStoreDelegate {
LocalStore const & store;
StorePathSet roots, dead, alive;
public:
GCStoreDelegate(LocalStore const & store) : store(store) {}
std::optional<StorePath> maybeParseStorePath(std::string_view path) const {
return store.maybeParseStorePath(path);
}
};
/**
* Class holding a server to receive new GC roots.
*/
class GCOperation {
const GCStoreDelegate store;
std::thread serverThread;
Pipe shutdownPipe;
AutoCloseFD fdServer;
struct Shared
{
@ -381,9 +399,165 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
std::optional<std::string> pending;
};
Sync<Shared> _shared;
void runServerThread();
std::condition_variable wakeup;
Sync<Shared> _shared;
public:
GCOperation(LocalStore const & store, Path stateDir) : store(store)
{
/* Start the server for receiving new roots. */
shutdownPipe.create();
auto socketPath = stateDir + gcSocketPath;
createDirs(dirOf(socketPath));
fdServer = createUnixDomainSocket(socketPath, 0666);
if (fcntl(fdServer.get(), F_SETFL, fcntl(fdServer.get(), F_GETFL) | O_NONBLOCK) == -1) {
throw SysError("making socket '%1%' non-blocking", socketPath);
}
serverThread = std::thread([this]() { runServerThread(); });
}
void addTempRoot(std::string rootHashPart)
{
_shared.lock()->tempRoots.insert(rootHashPart);
}
void releasePending()
{
auto shared(_shared.lock());
shared->pending.reset();
wakeup.notify_all();
}
/**
* Marks a path as pending deletion if it is not in tempRoots.
*
* Returns whether it was marked for deletion.
*/
bool markPendingIfPresent(std::string const & hashPart)
{
auto shared(_shared.lock());
if (shared->tempRoots.count(hashPart)) {
return false;
}
shared->pending = hashPart;
return true;
}
~GCOperation();
};
void GCOperation::runServerThread()
{
Sync<std::map<int, std::thread>> connections;
Finally cleanup([&]() {
debug("GC roots server shutting down");
fdServer.close();
while (true) {
auto item = remove_begin(*connections.lock());
if (!item) break;
auto & [fd, thread] = *item;
shutdown(fd, SHUT_RDWR);
thread.join();
}
});
while (true) {
std::vector<struct pollfd> fds;
fds.push_back({.fd = shutdownPipe.readSide.get(), .events = POLLIN});
fds.push_back({.fd = fdServer.get(), .events = POLLIN});
auto count = poll(fds.data(), fds.size(), -1);
assert(count != -1);
if (fds[0].revents)
/* Parent is asking us to quit. */
break;
if (fds[1].revents) {
/* Accept a new connection. */
assert(fds[1].revents & POLLIN);
AutoCloseFD fdClient{accept(fdServer.get(), nullptr, nullptr)};
if (!fdClient) continue;
debug("GC roots server accepted new client");
/* Process the connection in a separate thread. */
auto fdClient_ = fdClient.get();
std::thread clientThread([&, fdClient = std::move(fdClient)]() {
Finally cleanup([&]() {
auto conn(connections.lock());
auto i = conn->find(fdClient.get());
if (i != conn->end()) {
i->second.detach();
conn->erase(i);
}
});
/* On macOS, accepted sockets inherit the
non-blocking flag from the server socket, so
explicitly make it blocking. */
if (fcntl(fdClient.get(), F_SETFL, fcntl(fdClient.get(), F_GETFL) & ~O_NONBLOCK) == -1)
abort();
while (true) {
try {
auto path = readLine(fdClient.get());
auto storePath = store.maybeParseStorePath(path);
if (storePath) {
debug("got new GC root '%s'", path);
auto hashPart = std::string(storePath->hashPart());
auto shared(_shared.lock());
shared->tempRoots.insert(hashPart);
/* If this path is currently being
deleted, then we have to wait until
deletion is finished to ensure that
the client doesn't start
re-creating it before we're
done. FIXME: ideally we would use a
FD for this so we don't block the
poll loop. */
while (shared->pending == hashPart) {
debug("synchronising with deletion of path '%s'", path);
shared.wait(wakeup);
}
} else
printError("received garbage instead of a root from client");
writeFull(fdClient.get(), "1", false);
} catch (Error & e) {
debug("reading GC root from client: %s", e.msg());
break;
}
}
});
connections.lock()->insert({fdClient_, std::move(clientThread)});
}
}
}
GCOperation::~GCOperation()
{
writeFull(shutdownPipe.writeSide.get(), "x", false);
{
auto shared(_shared.lock());
wakeup.notify_all();
}
if (serverThread.joinable()) serverThread.join();
}
void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
{
bool shouldDelete = options.action == GCOptions::gcDeleteDead || options.action == GCOptions::gcDeleteSpecific;
bool gcKeepOutputs = settings.gcKeepOutputs;
bool gcKeepDerivations = settings.gcKeepDerivations;
StorePathSet roots, dead, alive;
/* Using `--ignore-liveness' with `--delete' can have unintended
consequences if `keep-outputs' or `keep-derivations' are true
@ -395,7 +569,7 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
}
if (shouldDelete)
deletePath(reservedPath);
deletePath(reservedSpacePath);
/* Acquire the global GC root. Note: we don't use fdGCLock
here because then in auto-gc mode, another thread could
@ -408,110 +582,7 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
if (auto p = getEnv("_NIX_TEST_GC_SYNC_1"))
readFile(*p);
/* Start the server for receiving new roots. */
auto socketPath = stateDir.get() + gcSocketPath;
createDirs(dirOf(socketPath));
auto fdServer = createUnixDomainSocket(socketPath, 0666);
if (fcntl(fdServer.get(), F_SETFL, fcntl(fdServer.get(), F_GETFL) | O_NONBLOCK) == -1)
throw SysError("making socket '%1%' non-blocking", socketPath);
Pipe shutdownPipe;
shutdownPipe.create();
std::thread serverThread([&]() {
Sync<std::map<int, std::thread>> connections;
Finally cleanup([&]() {
debug("GC roots server shutting down");
fdServer.close();
while (true) {
auto item = remove_begin(*connections.lock());
if (!item) break;
auto & [fd, thread] = *item;
shutdown(fd, SHUT_RDWR);
thread.join();
}
});
while (true) {
std::vector<struct pollfd> fds;
fds.push_back({.fd = shutdownPipe.readSide.get(), .events = POLLIN});
fds.push_back({.fd = fdServer.get(), .events = POLLIN});
auto count = poll(fds.data(), fds.size(), -1);
assert(count != -1);
if (fds[0].revents)
/* Parent is asking us to quit. */
break;
if (fds[1].revents) {
/* Accept a new connection. */
assert(fds[1].revents & POLLIN);
AutoCloseFD fdClient{accept(fdServer.get(), nullptr, nullptr)};
if (!fdClient) continue;
debug("GC roots server accepted new client");
/* Process the connection in a separate thread. */
auto fdClient_ = fdClient.get();
std::thread clientThread([&, fdClient = std::move(fdClient)]() {
Finally cleanup([&]() {
auto conn(connections.lock());
auto i = conn->find(fdClient.get());
if (i != conn->end()) {
i->second.detach();
conn->erase(i);
}
});
/* On macOS, accepted sockets inherit the
non-blocking flag from the server socket, so
explicitly make it blocking. */
if (fcntl(fdClient.get(), F_SETFL, fcntl(fdClient.get(), F_GETFL) & ~O_NONBLOCK) == -1)
abort();
while (true) {
try {
auto path = readLine(fdClient.get());
auto storePath = maybeParseStorePath(path);
if (storePath) {
debug("got new GC root '%s'", path);
auto hashPart = std::string(storePath->hashPart());
auto shared(_shared.lock());
shared->tempRoots.insert(hashPart);
/* If this path is currently being
deleted, then we have to wait until
deletion is finished to ensure that
the client doesn't start
re-creating it before we're
done. FIXME: ideally we would use a
FD for this so we don't block the
poll loop. */
while (shared->pending == hashPart) {
debug("synchronising with deletion of path '%s'", path);
shared.wait(wakeup);
}
} else
printError("received garbage instead of a root from client");
writeFull(fdClient.get(), "1", false);
} catch (Error & e) {
debug("reading GC root from client: %s", e.msg());
break;
}
}
});
connections.lock()->insert({fdClient_, std::move(clientThread)});
}
}
});
Finally stopServer([&]() {
writeFull(shutdownPipe.writeSide.get(), "x", false);
wakeup.notify_all();
if (serverThread.joinable()) serverThread.join();
});
GCOperation gcServer {*this, stateDir.get()};
/* Find the roots. Since we've grabbed the GC lock, the set of
permanent roots cannot increase now. */
@ -527,7 +598,7 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
Roots tempRoots;
findTempRoots(tempRoots, true);
for (auto & root : tempRoots) {
_shared.lock()->tempRoots.insert(std::string(root.first.hashPart()));
gcServer.addTempRoot(std::string(root.first.hashPart()));
roots.insert(root.first);
}
@ -580,9 +651,7 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
/* Wake up any GC client waiting for deletion of the paths in
'visited' to finish. */
Finally releasePending([&]() {
auto shared(_shared.lock());
shared->pending.reset();
wakeup.notify_all();
gcServer.releasePending();
});
auto enqueue = [&](const StorePath & path) {
@ -629,14 +698,9 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
&& !options.pathsToDelete.count(*path))
return;
{
auto hashPart = std::string(path->hashPart());
auto shared(_shared.lock());
if (shared->tempRoots.count(hashPart)) {
debug("cannot delete '%s' because it's a temporary root", printStorePath(*path));
return markAlive();
}
shared->pending = hashPart;
if (!gcServer.markPendingIfPresent(std::string(path->hashPart()))) {
debug("cannot delete '%s' because it's a temporary root", printStorePath(*path));
return markAlive();
}
if (isValidPath(*path)) {

View file

@ -177,14 +177,14 @@ static bool hasVirt() {
size_t size;
size = sizeof(hasVMM);
if (sysctlbyname("kern.hv_vmm_present", &hasVMM, &size, NULL, 0) == 0) {
if (sysctlbyname("kern.hv_vmm_present", &hasVMM, &size, nullptr, 0) == 0) {
if (hasVMM)
return false;
}
// whether the kernel and hardware supports virt
size = sizeof(hvSupport);
if (sysctlbyname("kern.hv_support", &hvSupport, &size, NULL, 0) == 0) {
if (sysctlbyname("kern.hv_support", &hvSupport, &size, nullptr, 0) == 0) {
return hvSupport == 1;
} else {
return false;

View file

@ -1,35 +0,0 @@
/*
* Determine the syscall number for `fchmodat2`.
*
* On most platforms this is 452. Exceptions can be found on
* a glibc git checkout via `rg --pcre2 'define __NR_fchmodat2 (?!452)'`.
*
* The problem is that glibc 2.39 and libseccomp 2.5.5 are needed to
* get the syscall number. However, a Lix built against nixpkgs 23.11
* (glibc 2.38) should still have the issue fixed without depending
* on the build environment.
*
* To achieve that, the macros below try to determine the platform and
* set the syscall number which is platform-specific, but
* in most cases 452.
*
* TODO: remove this when 23.11 is EOL and the entire (supported) ecosystem
* is on glibc 2.39.
*/
#pragma once
///@file
#if defined(__alpha__)
# define NIX_SYSCALL_FCHMODAT2 562
#elif defined(__x86_64__) && SIZE_MAX == 0xFFFFFFFF // x32
# define NIX_SYSCALL_FCHMODAT2 1073742276
#elif defined(__mips__) && defined(__mips64) && defined(_ABIN64) // mips64/n64
# define NIX_SYSCALL_FCHMODAT2 5452
#elif defined(__mips__) && defined(__mips64) && defined(_ABIN32) // mips64/n32
# define NIX_SYSCALL_FCHMODAT2 6452
#elif defined(__mips__) && defined(_ABIO32) // mips32
# define NIX_SYSCALL_FCHMODAT2 4452
#else
# define NIX_SYSCALL_FCHMODAT2 452
#endif

View file

@ -181,7 +181,7 @@ LocalStore::LocalStore(const Params & params)
, LocalFSStore(params)
, dbDir(stateDir + "/db")
, linksDir(realStoreDir + "/.links")
, reservedPath(dbDir + "/reserved")
, reservedSpacePath(dbDir + "/reserved")
, schemaPath(dbDir + "/schema")
, tempRootsDir(stateDir + "/temproots")
, fnTempRoots(fmt("%s/%d", tempRootsDir, getpid()))
@ -259,10 +259,10 @@ LocalStore::LocalStore(const Params & params)
before doing a garbage collection. */
try {
struct stat st;
if (stat(reservedPath.c_str(), &st) == -1 ||
if (stat(reservedSpacePath.c_str(), &st) == -1 ||
st.st_size != settings.reservedSize)
{
AutoCloseFD fd{open(reservedPath.c_str(), O_WRONLY | O_CREAT | O_CLOEXEC, 0600)};
AutoCloseFD fd{open(reservedSpacePath.c_str(), O_WRONLY | O_CREAT | O_CLOEXEC, 0600)};
int res = -1;
#if HAVE_POSIX_FALLOCATE
res = posix_fallocate(fd.get(), 0, settings.reservedSize);
@ -557,7 +557,7 @@ void LocalStore::openDB(State & state, bool create)
if (sqlite3_exec(db, "pragma main.journal_size_limit = 1099511627776;", 0, 0, 0) != SQLITE_OK)
SQLiteError::throw_(db, "setting journal_size_limit");
int enable = 1;
if (sqlite3_file_control(db, NULL, SQLITE_FCNTL_PERSIST_WAL, &enable) != SQLITE_OK)
if (sqlite3_file_control(db, nullptr, SQLITE_FCNTL_PERSIST_WAL, &enable) != SQLITE_OK)
SQLiteError::throw_(db, "setting persistent WAL mode");
}

View file

@ -119,7 +119,8 @@ public:
const Path dbDir;
const Path linksDir;
const Path reservedPath;
/** Path kept around to reserve some filesystem space to be able to begin a garbage collection */
const Path reservedSpacePath;
const Path schemaPath;
const Path tempRootsDir;
const Path fnTempRoots;

View file

@ -167,6 +167,9 @@ if host_machine.system() == 'linux'
elif host_machine.system() == 'darwin'
libstore_sources += files('platform/darwin.cc')
libstore_headers += files('platform/darwin.hh')
elif host_machine.system() == 'freebsd'
libstore_sources += files('platform/freebsd.cc')
libstore_headers += files('platform/freebsd.hh')
else
libstore_sources += files('platform/fallback.cc')
libstore_headers += files('platform/fallback.hh')
@ -202,23 +205,29 @@ foreach name, value : cpp_str_defines
]
endforeach
dependencies = [
libarchive,
liblixutil, # Internal.
seccomp,
sqlite,
sodium,
curl,
openssl,
aws_sdk,
aws_s3,
aws_sdk_transfer,
nlohmann_json,
]
if host_machine.system() == 'freebsd'
dependencies += [ libprocstat ]
endif
libstore = library(
'lixstore',
libstore_generated_headers,
libstore_sources,
dependencies : [
libarchive,
liblixutil, # Internal.
seccomp,
sqlite,
sodium,
curl,
openssl,
aws_sdk,
aws_s3,
aws_sdk_transfer,
nlohmann_json,
],
dependencies : dependencies,
cpp_args : cpp_args,
cpp_pch : cpp_pch,
install : true,

View file

@ -5,6 +5,8 @@
#include "platform/linux.hh"
#elif __APPLE__
#include "platform/darwin.hh"
#elif __FreeBSD__
#include "platform/freebsd.hh"
#else
#include "platform/fallback.hh"
#endif
@ -16,6 +18,8 @@ std::shared_ptr<LocalStore> LocalStore::makeLocalStore(const Params & params)
return std::shared_ptr<LocalStore>(new LinuxLocalStore(params));
#elif __APPLE__
return std::shared_ptr<LocalStore>(new DarwinLocalStore(params));
#elif __FreeBSD__
return std::shared_ptr<LocalStore>(new FreeBSDLocalStore(params));
#else
return std::shared_ptr<LocalStore>(new FallbackLocalStore(params));
#endif
@ -32,6 +36,8 @@ std::shared_ptr<LocalDerivationGoal> LocalDerivationGoal::makeLocalDerivationGoa
return std::make_shared<LinuxLocalDerivationGoal>(drvPath, wantedOutputs, worker, buildMode);
#elif __APPLE__
return std::make_shared<DarwinLocalDerivationGoal>(drvPath, wantedOutputs, worker, buildMode);
#elif __FreeBSD__
return std::make_shared<FreeBSDLocalDerivationGoal>(drvPath, wantedOutputs, worker, buildMode);
#else
return std::make_shared<FallbackLocalDerivationGoal>(drvPath, wantedOutputs, worker, buildMode);
#endif
@ -53,6 +59,10 @@ std::shared_ptr<LocalDerivationGoal> LocalDerivationGoal::makeLocalDerivationGoa
return std::make_shared<DarwinLocalDerivationGoal>(
drvPath, drv, wantedOutputs, worker, buildMode
);
#elif __FreeBSD__
return std::make_shared<FreeBSDLocalDerivationGoal>(
drvPath, drv, wantedOutputs, worker, buildMode
);
#else
return std::make_shared<FallbackLocalDerivationGoal>(
drvPath, drv, wantedOutputs, worker, buildMode

View file

@ -235,15 +235,15 @@ void DarwinLocalDerivationGoal::execBuilder(std::string builder, Strings args, S
if (drv->platform == "aarch64-darwin") {
// Unset kern.curproc_arch_affinity so we can escape Rosetta
int affinity = 0;
sysctlbyname("kern.curproc_arch_affinity", NULL, NULL, &affinity, sizeof(affinity));
sysctlbyname("kern.curproc_arch_affinity", nullptr, nullptr, &affinity, sizeof(affinity));
cpu_type_t cpu = CPU_TYPE_ARM64;
posix_spawnattr_setbinpref_np(&attrp, 1, &cpu, NULL);
posix_spawnattr_setbinpref_np(&attrp, 1, &cpu, nullptr);
} else if (drv->platform == "x86_64-darwin") {
cpu_type_t cpu = CPU_TYPE_X86_64;
posix_spawnattr_setbinpref_np(&attrp, 1, &cpu, NULL);
posix_spawnattr_setbinpref_np(&attrp, 1, &cpu, nullptr);
}
posix_spawn(NULL, builder.c_str(), NULL, &attrp, stringsToCharPtrs(args).data(), stringsToCharPtrs(envStrs).data());
posix_spawn(nullptr, builder.c_str(), nullptr, &attrp, stringsToCharPtrs(args).data(), stringsToCharPtrs(envStrs).data());
}
}

View file

@ -0,0 +1,142 @@
#include "platform/freebsd.hh"
#include "regex.hh"
#include <sys/param.h>
#include <sys/queue.h>
#include <sys/socket.h>
#include <sys/sysctl.h>
#include <sys/user.h>
#include <libprocstat.h>
namespace nix {
static void readSysctlRoots(const char * name, UncheckedRoots & unchecked)
{
size_t len = 0;
std::string value;
if (int err = sysctlbyname(name, nullptr, &len, nullptr, 0) < 0) {
if (err == ENOENT || err == EACCES) {
return;
} else {
throw SysError(err, "sysctlbyname %1%", name);
}
}
value.resize(len, ' ');
if (int err = sysctlbyname(name, value.data(), &len, nullptr, 0) < 0) {
if (err == ENOENT || err == EACCES) {
return;
} else {
throw SysError(err, "sysctlbyname %1%", name);
}
}
for (auto & path : tokenizeString<Strings>(value, ";")) {
unchecked[path].emplace(fmt("{{sysctl:%1%}}", name));
}
}
struct ProcstatDeleter
{
void operator()(struct procstat * ps)
{
procstat_close(ps);
}
};
template<auto del>
struct ProcstatReferredDeleter
{
struct procstat * ps;
ProcstatReferredDeleter(struct procstat * ps) : ps(ps) {}
template<typename T>
void operator()(T * p)
{
del(ps, p);
}
};
void FreeBSDLocalStore::findPlatformRoots(UncheckedRoots & unchecked)
{
readSysctlRoots("kern.module_path", unchecked);
auto storePathRegex = regex::storePathRegex(storeDir);
auto ps = std::unique_ptr<struct procstat, ProcstatDeleter>(procstat_open_sysctl());
if (!ps) {
throw SysError("procstat_open_sysctl");
}
auto procs = std::unique_ptr<struct kinfo_proc[], ProcstatReferredDeleter<procstat_freeprocs>>(
nullptr, ps.get()
);
auto files = std::unique_ptr<struct filestat_list, ProcstatReferredDeleter<procstat_freefiles>>(
nullptr, ps.get()
);
unsigned int numprocs = 0;
procs.reset(procstat_getprocs(ps.get(), KERN_PROC_PROC, 0, &numprocs));
if (!procs || numprocs == 0) {
throw SysError("procstat_getprocs");
};
for (unsigned int procidx = 0; procidx < numprocs; procidx++) {
// Includes file descriptors, executable, cwd,
// and mmapped files (including dynamic libraries)
files.reset(procstat_getfiles(ps.get(), &procs[procidx], 1));
// We only have permission if we're root so just skip it if we fail
if (!files) {
continue;
}
for (struct filestat * file = files->stqh_first; file; file = file->next.stqe_next) {
if (!file->fs_path) {
continue;
}
std::string role;
if (file->fs_uflags & PS_FST_UFLAG_CTTY) {
role = "ctty";
} else if (file->fs_uflags & PS_FST_UFLAG_CDIR) {
role = "cwd";
} else if (file->fs_uflags & PS_FST_UFLAG_JAIL) {
role = "jail";
} else if (file->fs_uflags & PS_FST_UFLAG_RDIR) {
role = "root";
} else if (file->fs_uflags & PS_FST_UFLAG_TEXT) {
role = "text";
} else if (file->fs_uflags & PS_FST_UFLAG_TRACE) {
role = "trace";
} else if (file->fs_uflags & PS_FST_UFLAG_MMAP) {
role = "mmap";
} else {
role = fmt("fd/%1%", file->fs_fd);
}
unchecked[file->fs_path].emplace(fmt("{procstat:%1%/%2%}", procs[procidx].ki_pid, role)
);
}
auto env_name = fmt("{procstat:%1%/env}", procs[procidx].ki_pid);
// No need to free, the buffer is reused on next call and deallocated in procstat_close
char ** env = procstat_getenvv(ps.get(), &procs[procidx], 0);
if (env == nullptr) {
continue;
}
for (size_t i = 0; env[i]; i++) {
auto envString = std::string(env[i]);
auto envEnd = std::sregex_iterator{};
for (auto match =
std::sregex_iterator{envString.begin(), envString.end(), storePathRegex};
match != envEnd;
match++)
{
unchecked[match->str()].emplace(env_name);
}
}
}
}
}

View file

@ -0,0 +1,47 @@
#pragma once
///@file
#include "build/local-derivation-goal.hh"
#include "gc-store.hh"
#include "local-store.hh"
namespace nix {
/**
* FreeBSD-specific implementation of LocalStore
*/
class FreeBSDLocalStore : public LocalStore
{
public:
FreeBSDLocalStore(const Params & params)
: StoreConfig(params)
, LocalFSStoreConfig(params)
, LocalStoreConfig(params)
, Store(params)
, LocalFSStore(params)
, LocalStore(params)
{
}
FreeBSDLocalStore(const std::string scheme, std::string path, const Params & params)
: FreeBSDLocalStore(params)
{
throw UnimplementedError("FreeBSDLocalStore");
}
private:
void findPlatformRoots(UncheckedRoots & unchecked) override;
};
/**
* FreeBSD-specific implementation of LocalDerivationGoal
*/
class FreeBSDLocalDerivationGoal : public LocalDerivationGoal
{
public:
using LocalDerivationGoal::LocalDerivationGoal;
private:
};
}

View file

@ -15,6 +15,11 @@
# include <sys/resource.h>
#endif
#if __FreeBSD__
# include <sys/param.h>
# include <sys/sysctl.h>
#endif
#include <sys/mount.h>
#include <cgroup.hh>
@ -102,6 +107,24 @@ std::optional<Path> getSelfExe()
return buf;
else
return std::nullopt;
#elif __FreeBSD__
int sysctlName[] = {
CTL_KERN,
KERN_PROC,
KERN_PROC_PATHNAME,
-1,
};
size_t pathLen = 0;
if (sysctl(sysctlName, sizeof(sysctlName) / sizeof(sysctlName[0]), nullptr, &pathLen, nullptr, 0) < 0) {
return std::nullopt;
}
std::vector<char> path(pathLen);
if (sysctl(sysctlName, sizeof(sysctlName) / sizeof(sysctlName[0]), path.data(), &pathLen, nullptr, 0) < 0) {
return std::nullopt;
}
return Path(path.begin(), path.end());
#else
return std::nullopt;
#endif

View file

@ -21,22 +21,14 @@ Path absPath(Path path, std::optional<PathView> dir, bool resolveSymlinks)
{
if (path.empty() || path[0] != '/') {
if (!dir) {
#ifdef __GNU__
/* GNU (aka. GNU/Hurd) doesn't have any limitation on path
lengths and doesn't define `PATH_MAX'. */
char *buf = getcwd(NULL, 0);
if (buf == NULL)
#else
char buf[PATH_MAX];
if (!getcwd(buf, sizeof(buf)))
#endif
if (!getcwd(buf, sizeof(buf))) {
throw SysError("cannot get cwd");
}
path = concatStrings(buf, "/", path);
#ifdef __GNU__
free(buf);
#endif
} else
} else {
path = concatStrings(*dir, "/", path);
}
}
return canonPath(path, resolveSymlinks);
}

View file

@ -7,6 +7,7 @@
#include <algorithm>
#include <atomic>
#include <mutex>
#include <sstream>
#include <nlohmann/json.hpp>
@ -77,7 +78,7 @@ public:
prefix = std::string("<") + c + ">";
}
writeToStderr(prefix + filterANSIEscapes(s, !tty) + "\n");
writeLogsToStderr(prefix + filterANSIEscapes(s, !tty) + "\n");
}
void logEI(const ErrorInfo & ei) override
@ -117,8 +118,17 @@ Verbosity verbosityFromIntClamped(int val)
return static_cast<Verbosity>(clamped);
}
void writeToStderr(std::string_view s)
void writeLogsToStderr(std::string_view s)
{
static std::mutex lock;
// make sure only one thread uses this function at any given time.
// multiple concurrent threads can have deleterious effects on log
// output, especially when layering structured formats (like JSON)
// on top of a SimpleLogger which is itself not thread-safe. every
// Logger instance should be thread-safe in an ideal world, but we
// cannot really enforce that on a per-logger level at this point.
std::unique_lock _lock(lock);
try {
writeFull(STDERR_FILENO, s, false);
} catch (SysError & e) {

View file

@ -283,12 +283,6 @@ inline void warn(const std::string & fs, const Args & ... args)
logger->warn(HintFmt(fs, args...).str());
}
#define warnOnce(haveWarned, args...) \
if (!haveWarned) { \
haveWarned = true; \
warn(args); \
}
void writeToStderr(std::string_view s);
void writeLogsToStderr(std::string_view s);
}

View file

@ -10,6 +10,8 @@
#include <unistd.h>
#include <signal.h>
#include "error.hh"
#include "file-descriptor.hh"
#include "signals.hh"
namespace nix {
@ -19,28 +21,46 @@ class MonitorFdHup
{
private:
std::thread thread;
/**
* Pipe used to interrupt the poll()ing in the monitoring thread.
*/
Pipe terminatePipe;
std::atomic_bool quit = false;
public:
MonitorFdHup(int fd)
{
thread = std::thread([fd]() {
while (true) {
terminatePipe.create();
auto &quit_ = this->quit;
int terminateFd = terminatePipe.readSide.get();
thread = std::thread([fd, terminateFd, &quit_]() {
while (!quit_) {
/* Wait indefinitely until a POLLHUP occurs. */
struct pollfd fds[1];
struct pollfd fds[2];
fds[0].fd = fd;
/* Polling for no specific events (i.e. just waiting
for an error/hangup) doesn't work on macOS
anymore. So wait for read events and ignore
them. */
fds[0].events =
#ifdef __APPLE__
POLLRDNORM
#else
0
#endif
;
auto count = poll(fds, 1, -1);
if (count == -1) abort(); // can't happen
// There is a POSIX violation on macOS: you have to listen for
// at least POLLHUP to receive HUP events for a FD. POSIX says
// this is not so, and you should just receive them regardless,
// however, as of our testing on macOS 14.5, the events do not
// get delivered in such a case.
//
// This is allegedly filed as rdar://37537852.
//
// Relevant code, which backs this up:
// https://github.com/apple-oss-distributions/xnu/blob/94d3b452840153a99b38a3a9659680b2a006908e/bsd/kern/sys_generic.c#L1751-L1758
fds[0].events = POLLHUP;
fds[1].fd = terminateFd;
fds[1].events = POLLIN;
auto count = poll(fds, 2, -1);
if (count == -1) {
if (errno == EINTR || errno == EAGAIN) {
// These are best dealt with by just trying again.
continue;
} else {
throw SysError("in MonitorFdHup poll()");
}
}
/* This shouldn't happen, but can on macOS due to a bug.
See rdar://37550628.
@ -53,9 +73,16 @@ public:
triggerInterrupt();
break;
}
/* This will only happen on macOS. We sleep a bit to
avoid waking up too often if the client is sending
input. */
// No reason to actually look at the pipe FD if that's what
// woke us, the only thing that actually matters is the quit
// flag.
if (quit_) {
break;
}
// On macOS, it is possible (although not observed on macOS
// 14.5) that in some limited cases on buggy kernel versions,
// all the non-POLLHUP events for the socket get delivered.
// Sleeping avoids pointlessly spinning a thread on those.
sleep(1);
}
});
@ -63,8 +90,12 @@ public:
~MonitorFdHup()
{
pthread_cancel(thread.native_handle());
thread.join();
quit = true;
// Poke the thread out of its poll wait
writeFull(terminatePipe.writeSide.get(), "*", false);
if (thread.joinable()) {
thread.join();
}
}
};

View file

@ -251,7 +251,7 @@ std::pair<int, std::string> runProgram(RunOptions && options)
try {
auto proc = runProgram2(options);
Finally const _wait([&] { proc.wait(); });
stdout = proc.stdout()->drain();
stdout = proc.getStdout()->drain();
} catch (ExecError & e) {
status = e.status;
}

View file

@ -106,7 +106,7 @@ public:
void wait();
Source * stdout() const { return stdoutSource.get(); }
Source * getStdout() const { return stdoutSource.get(); };
};
std::pair<int, std::string> runProgram(RunOptions && options);

View file

@ -211,7 +211,7 @@ struct TeeSink : Sink
{
Sink & sink1, & sink2;
TeeSink(Sink & sink1, Sink & sink2) : sink1(sink1), sink2(sink2) { }
virtual void operator () (std::string_view data)
virtual void operator () (std::string_view data) override
{
sink1(data);
sink2(data);
@ -228,7 +228,7 @@ struct TeeSource : Source
Sink & sink;
TeeSource(Source & orig, Sink & sink)
: orig(orig), sink(sink) { }
size_t read(char * data, size_t len)
size_t read(char * data, size_t len) override
{
size_t n = orig.read(data, len);
sink({data, n});
@ -245,7 +245,7 @@ struct SizedSource : Source
size_t remain;
SizedSource(Source & orig, size_t size)
: orig(orig), remain(size) { }
size_t read(char * data, size_t len)
size_t read(char * data, size_t len) override
{
if (this->remain <= 0) {
throw EndOfFile("sized: unexpected end-of-file");
@ -338,7 +338,7 @@ struct GeneratorSource : Source
{
GeneratorSource(Generator<Bytes> && g) : g(std::move(g)) {}
virtual size_t read(char * data, size_t len)
virtual size_t read(char * data, size_t len) override
{
// we explicitly do not poll the generator multiple times to fill the
// buffer, only to produce some output at all. this is allowed by the

View file

@ -54,7 +54,7 @@ TarArchive::TarArchive(Source & source, bool raw) : buffer(65536)
archive_read_support_format_raw(archive);
archive_read_support_format_empty(archive);
}
archive_read_set_option(archive, NULL, "mac-ext", NULL);
archive_read_set_option(archive, nullptr, "mac-ext", nullptr);
check(archive_read_open(archive, (void *)this, callback_open, callback_read, callback_close), "Failed to open archive (%s)");
}
@ -65,7 +65,7 @@ TarArchive::TarArchive(const Path & path)
archive_read_support_filter_all(archive);
archive_read_support_format_all(archive);
archive_read_set_option(archive, NULL, "mac-ext", NULL);
archive_read_set_option(archive, nullptr, "mac-ext", nullptr);
check(archive_read_open_filename(archive, path.c_str(), 16384), "failed to open archive: %s");
}

View file

@ -24,24 +24,17 @@ GroupedPaths getClosureInfo(ref<Store> store, const StorePath & toplevel)
GroupedPaths groupedPaths;
for (auto & path : closure) {
for (auto const & path : closure) {
/* Strip the output name. Unfortunately this is ambiguous (we
can't distinguish between output names like "bin" and
version suffixes like "unstable"). */
static std::regex regex("(.*)-([a-z]+|lib32|lib64)");
std::smatch match;
std::cmatch match;
std::string name{path.name()};
// Used to keep name alive through being potentially overwritten below
// (to not invalidate the references from the regex result)
//
// n.b. cannot be just path.name().{begin,end}() since that returns const
// char *, which does not, for some reason, convert as required on
// libstdc++. Seems like a libstdc++ bug or standard bug to me... we
// can afford the allocation in any case.
const std::string origName{path.name()};
std::string_view const origName = path.name();
std::string outputName;
if (std::regex_match(origName, match, regex)) {
if (std::regex_match(origName.begin(), origName.end(), match, regex)) {
name = match[1];
outputName = match[2];
}

View file

@ -48,7 +48,7 @@ std::set<std::string> runResolver(const Path & filename)
return {};
}
char* obj = (char*) mmap(NULL, st.st_size, PROT_READ, MAP_SHARED, fd.get(), 0);
char* obj = (char*) mmap(nullptr, st.st_size, PROT_READ, MAP_SHARED, fd.get(), 0);
if (!obj)
throw SysError("mmapping '%s'", filename);

View file

@ -33,7 +33,7 @@ sleep 2
pid2=$!
# Start a build. This should not be blocked by the GC in progress.
outPath=$(nix-build --max-silent-time 60 -o "$TEST_ROOT/result" -E "
outPath=$(nix-build --max-silent-time 60 --debug -o "$TEST_ROOT/result" -E "
with import ./config.nix;
mkDerivation {
name = \"non-blocking\";

View file

@ -62,21 +62,31 @@ stripColors () {
testReplResponseGeneral () {
local grepMode="$1"; shift
local commands="$1"; shift
local expectedResponse="$1"; shift
local response="$(nix repl "$@" <<< "$commands" | stripColors)"
echo "$response" | grepQuiet "$grepMode" -s "$expectedResponse" \
|| fail "repl command set:
# Expected response can contain newlines.
# grep can't handle multiline patterns, so replace newlines with TEST_NEWLINE
# in both expectedResponse and response.
# awk ORS always adds a trailing record separator, so we strip it with sed.
local expectedResponse="$(printf '%s' "$1" | awk 1 ORS=TEST_NEWLINE | sed 's/TEST_NEWLINE$//')"; shift
# We don't need to strip trailing record separator here, since extra data is ok.
local response="$(nix repl "$@" <<< "$commands" 2>&1 | stripColors | awk 1 ORS=TEST_NEWLINE)"
printf '%s' "$response" | grepQuiet "$grepMode" -s "$expectedResponse" \
|| fail "$(echo "repl command set:
$commands
does not respond with:
---
$expectedResponse
---
but with:
---
$response
"
---
" | sed 's/TEST_NEWLINE/\n/g')"
}
testReplResponse () {
@ -179,7 +189,7 @@ testReplResponseNoRegex '
let x = { y = { a = 1; }; inherit x; }; in x
' \
'{
x = { ... };
x = «repeated»;
y = { ... };
}
'
@ -231,6 +241,33 @@ testReplResponseNoRegex '
' \
'{
x = «repeated»;
y = { a = 1 };
y = { a = 1; };
}
'
# Test that editing a store path does not reload...
echo '{ identity = a: a; }' > repl-test.nix
repl_test_store="$(nix-store --add repl-test.nix)"
EDITOR=true testReplResponseNoRegex "
a = ''test string that we'll grep later''
:l $repl_test_store
:e identity
a
" "test string that we'll grep later"
# ...even through symlinks
ln -s "$repl_test_store" repl-test-link.nix
EDITOR=true testReplResponseNoRegex "
a = ''test string that we'll grep later''
:l repl-test-link.nix
:e identity
a
" "test string that we'll grep later"
# Test that editing a local file does reload
EDITOR=true testReplResponseNoRegex "
a = ''test string that we'll grep later''
:l repl-test.nix
:e identity
a
" "undefined variable"

View file

@ -0,0 +1,13 @@
A previously unforced thunk in an attribute set does not lead to indentation when it won't evaluate to a nested structure:
nix-repl> :p let x = 1 + 2; in [ { inherit x; } { inherit x; } ]
[
{ x = 3; }
{ x = 3; }
]
Same for a list:
nix-repl> :p let x = 1 + 2; in [ [ x ] [ x ] ]
[
[ 3 ]
[ 3 ]
]

View file

@ -185,5 +185,6 @@ REPL_TEST(repl_overlays_error);
REPL_TEST(repl_printing);
REPL_TEST(stack_vars);
REPL_TEST(errors);
REPL_TEST(idempotent);
}; // namespace nix

View file

@ -155,4 +155,6 @@ in
broken-userns = runNixOSTestFor "x86_64-linux" ./broken-userns.nix;
coredumps = runNixOSTestFor "x86_64-linux" ./coredumps;
io_uring = runNixOSTestFor "x86_64-linux" ./io_uring;
}

View file

@ -0,0 +1,7 @@
let
inherit (import ../util.nix) mkNixBuildTest;
in
mkNixBuildTest {
name = "io_uring";
expressionFile = ./package.nix;
}

View file

@ -0,0 +1,19 @@
{ runCommandCC }:
runCommandCC "io_uring-is-blocked" { } ''
cat > test.c <<EOF
#include <errno.h>
#include <sys/syscall.h>
#include <unistd.h>
int main() {
int res = syscall(SYS_io_uring_setup, 0, NULL);
return res == -1 && errno == ENOSYS ? 0 : 1;
}
EOF
"$CC" -o test test.c
if ! ./test; then
echo "Oh no! io_uring is available!"
exit 1
fi
touch "$out"
''

View file

@ -12,10 +12,7 @@ int main(void) {
fprintf(fd, "henlo :3");
fclose(fd);
// FIXME use something nicer here that's less
// platform-dependent as soon as we go to 24.05
// and the glibc is new enough to support fchmodat2
long rs = syscall(452, NULL, name, S_ISUID, 0);
long rs = syscall(SYS_fchmodat2, NULL, name, S_ISUID, 0);
assert(rs == -1);
assert(errno == EPERM);
}

View file

@ -193,6 +193,9 @@ TEST_F(ValuePrintingTests, vBlackhole)
TEST_F(ValuePrintingTests, depthAttrs)
{
Value vZero;
vZero.mkInt(0);
Value vOne;
vOne.mkInt(1);
@ -203,10 +206,16 @@ TEST_F(ValuePrintingTests, depthAttrs)
Value vAttrsEmpty;
vAttrsEmpty.mkAttrs(builderEmpty.finish());
BindingsBuilder builderNested(state, state.allocBindings(1));
builderNested.insert(state.symbols.create("zero"), &vZero);
Value vAttrsNested;
vAttrsNested.mkAttrs(builderNested.finish());
BindingsBuilder builder(state, state.allocBindings(10));
builder.insert(state.symbols.create("one"), &vOne);
builder.insert(state.symbols.create("two"), &vTwo);
builder.insert(state.symbols.create("nested"), &vAttrsEmpty);
builder.insert(state.symbols.create("empty"), &vAttrsEmpty);
builder.insert(state.symbols.create("nested"), &vAttrsNested);
Value vAttrs;
vAttrs.mkAttrs(builder.finish());
@ -220,9 +229,9 @@ TEST_F(ValuePrintingTests, depthAttrs)
vNested.mkAttrs(builder2.finish());
test(vNested, "{ nested = { ... }; one = 1; two = 2; }", PrintOptions { .maxDepth = 1 });
test(vNested, "{ nested = { nested = { ... }; one = 1; two = 2; }; one = 1; two = 2; }", PrintOptions { .maxDepth = 2 });
test(vNested, "{ nested = { nested = { }; one = 1; two = 2; }; one = 1; two = 2; }", PrintOptions { .maxDepth = 3 });
test(vNested, "{ nested = { nested = { }; one = 1; two = 2; }; one = 1; two = 2; }", PrintOptions { .maxDepth = 4 });
test(vNested, "{ nested = { empty = { }; nested = { ... }; one = 1; two = 2; }; one = 1; two = 2; }", PrintOptions { .maxDepth = 2 });
test(vNested, "{ nested = { empty = { }; nested = { zero = 0; }; one = 1; two = 2; }; one = 1; two = 2; }", PrintOptions { .maxDepth = 3 });
test(vNested, "{ nested = { empty = { }; nested = { zero = 0; }; one = 1; two = 2; }; one = 1; two = 2; }", PrintOptions { .maxDepth = 4 });
}
TEST_F(ValuePrintingTests, depthList)
@ -641,20 +650,24 @@ TEST_F(ValuePrintingTests, ansiColorsBlackhole)
TEST_F(ValuePrintingTests, ansiColorsAttrsRepeated)
{
BindingsBuilder emptyBuilder(state, state.allocBindings(1));
Value vZero;
vZero.mkInt(0);
Value vEmpty;
vEmpty.mkAttrs(emptyBuilder.finish());
BindingsBuilder innerBuilder(state, state.allocBindings(1));
innerBuilder.insert(state.symbols.create("x"), &vZero);
Value vInner;
vInner.mkAttrs(innerBuilder.finish());
BindingsBuilder builder(state, state.allocBindings(10));
builder.insert(state.symbols.create("a"), &vEmpty);
builder.insert(state.symbols.create("b"), &vEmpty);
builder.insert(state.symbols.create("a"), &vInner);
builder.insert(state.symbols.create("b"), &vInner);
Value vAttrs;
vAttrs.mkAttrs(builder.finish());
test(vAttrs,
"{ a = { }; b = " ANSI_MAGENTA "«repeated»" ANSI_NORMAL "; }",
"{ a = { x = " ANSI_CYAN "0" ANSI_NORMAL "; }; b = " ANSI_MAGENTA "«repeated»" ANSI_NORMAL "; }",
PrintOptions {
.ansiColors = true
});
@ -662,19 +675,23 @@ TEST_F(ValuePrintingTests, ansiColorsAttrsRepeated)
TEST_F(ValuePrintingTests, ansiColorsListRepeated)
{
BindingsBuilder emptyBuilder(state, state.allocBindings(1));
Value vZero;
vZero.mkInt(0);
Value vEmpty;
vEmpty.mkAttrs(emptyBuilder.finish());
BindingsBuilder innerBuilder(state, state.allocBindings(1));
innerBuilder.insert(state.symbols.create("x"), &vZero);
Value vInner;
vInner.mkAttrs(innerBuilder.finish());
Value vList;
state.mkList(vList, 3);
vList.bigList.elems[0] = &vEmpty;
vList.bigList.elems[1] = &vEmpty;
vList.bigList.elems[0] = &vInner;
vList.bigList.elems[1] = &vInner;
vList.bigList.size = 2;
test(vList,
"[ { } " ANSI_MAGENTA "«repeated»" ANSI_NORMAL " ]",
"[ { x = " ANSI_CYAN "0" ANSI_NORMAL "; } " ANSI_MAGENTA "«repeated»" ANSI_NORMAL " ]",
PrintOptions {
.ansiColors = true
});
@ -682,20 +699,24 @@ TEST_F(ValuePrintingTests, ansiColorsListRepeated)
TEST_F(ValuePrintingTests, listRepeated)
{
BindingsBuilder emptyBuilder(state, state.allocBindings(1));
Value vZero;
vZero.mkInt(0);
Value vEmpty;
vEmpty.mkAttrs(emptyBuilder.finish());
BindingsBuilder innerBuilder(state, state.allocBindings(1));
innerBuilder.insert(state.symbols.create("x"), &vZero);
Value vInner;
vInner.mkAttrs(innerBuilder.finish());
Value vList;
state.mkList(vList, 3);
vList.bigList.elems[0] = &vEmpty;
vList.bigList.elems[1] = &vEmpty;
vList.bigList.elems[0] = &vInner;
vList.bigList.elems[1] = &vInner;
vList.bigList.size = 2;
test(vList, "[ { } «repeated» ]", PrintOptions { });
test(vList, "[ { x = 0; } «repeated» ]", PrintOptions { });
test(vList,
"[ { } { } ]",
"[ { x = 0; } { x = 0; } ]",
PrintOptions {
.trackRepeated = false
});