Ever read gdb output and you just kinda get a headache because you have
to infer what a thread is by reading the stack trace? It's not hard, but
we could also just never have to do that again, which is also not hard.
Sample:
(gdb) info thr
Id Target Id Frame
* 1 LWP 3719283 "nix-daemon" 0x00007e558587da0f in accept ()
from target:/nix/store/c10zhkbp6jmyh0xc5kd123ga8yy2p4hk-glibc-2.39-52/lib/libc.so.6
2 LWP 3719284 "signal handler" 0x00007e55857b2bea in sigtimedwait ()
from target:/nix/store/c10zhkbp6jmyh0xc5kd123ga8yy2p4hk-glibc-2.39-52/lib/libc.so.6
The API design for this is forced by the macOS pthread_setname_np only
being able to change the current thread's name, but if we just conform
everything to that, it works everywhere.
Change-Id: I2b1d6ed41e3c94170cb0b4e73ad66f239ebd9c88
file:// urls are no longer handled by curl itself, and this is supposed
to test the curl wrapper. use http to force the wrapper to be involved.
Change-Id: Ib13087db07b3b2f1ae44ce8e3ec7a96d935b1bab
this only ever worked for empty uploads, and there it worked only by
complete accident: curl was asked to send more data than the wrapper
would provide, which curl would not like and report as an error. the
error would cause a retry with even less data to send, until finally
failing by running into the retry limit. let's just forbid all this.
Change-Id: I229a94b3b8b33e2c6cdb8ea19edd57cd6740e6c6
This adds a new temp-dir setting for controlling the temporary directory
without having to change the TMPDIR env var. This can be used to e.g.
use a path on a case-sensitive store on macOS for temporary files
without changing the TMPDIR var used by interactive shells or commands
invoked with `nix run`.
This also stops unsetting `TMPDIR` on darwin when the env var value
starts with `/var/folders/`, preferring instead to just do the check
when reading `TMPDIR`. This way the inherited `TMPDIR` env var is
preserved for child processes (such as interactive shells).
As a side effect this changes the behavior of `nix-build -o ''` to act
like `nix-build --no-out-link` instead of failing with an error caused
by trying to create a symlink at the cwd.
Fixes: #253
Fixes: #112
Change-Id: I9ee826323f2deca62854715a77ca7a373a948a29
This reverts commit 02c35ea9df.
Reason for revert: this code path is also used for `Input::getRev()`, i.e. flakes VCS revision validation, which, in the case of Git, are using SHA1.
As a result, this cause too much noise due to SHA1 revisions in Flakes.
Change-Id: I8064c1ebc26e4e83b627f0803a7a9ba56cfe1f37
don't pause the entire curl thread. we have multiple consumer threads
after all, not just one, so stalling all of them is likely not great.
note that libcurl advises against using transfer pauses if compressed
encodings are allowed and automatically decoded. this should not lead
to problems in practice because our data is usually not compressed to
such a degree that curl buffering *uncompressed* data matters. should
this cause problems we can reintroduce the whole-thread pause, but we
will probably get away with this until the entire file transfer class
is made kj::Promise-using async (and *then* curl can be hardpaused if
it cannot get rid of its data, solving the problem once and for all).
Change-Id: I218e41bfa5a27c7454eafb0bdb54f2a29a7f6493
using UTF-8 bullets in the sample avoids locale confusion where Bash
doesn't know to treat `•` as a single character.
Signed-off-by: Dusk Banks <me@bb010g.com>
Change-Id: I829019b66e93e6d33ac3a6641df07d0dd2332a5a
It's only used in a couple of tests, and only in such a way that
replacing it with a random command suffices.
I also removed a few pointless uses of the variable.
Fixes: #376
Change-Id: I90aedb61d64b02f7c9b007e72f9d614cc1b37a2e
Before:
error: derivation '/nix/store/4spy3nz1661zm15gkybsy1h5f36aliwx-python3.11-test-1.0.0.drv' may not be deterministic: output '/nix/store/ccqcp01zg18wp9iadzmzimqzdi3ll08d-python3.11-test-1.0.0-dist' differs from '/nix/store/ccqcp01zg18wp9iadzmzimqzdi3ll08d-python3.11-test-1.0.0-dist.check'
After:
error: derivation '4spy3nz1661zm15gkybsy1h5f36aliwx-python3.11-test-1.0.0.drv' may not be deterministic: outputs differ
output differs: output '/nix/store/ccqcp01zg18wp9iadzmzimqzdi3ll08d-python3.11-test-1.0.0-dist' differs from '/nix/store/ccqcp01zg18wp9iadzmzimqzdi3ll08d-python3.11-test-1.0.0-dist.check'
output differs: output '/nix/store/yl59v08356i841c560alb0zmk7q16klb-python3.11-test-1.0.0' differs from '/nix/store/yl59v08356i841c560alb0zmk7q16klb-python3.11-test-1.0.0.check'
Change-Id: Ib2871fa602bf1fa9c00e2565b3a2e1b26f908152
it's no longer needed. `download` can do everything `enqueueDownload`
did, and a lot more. e.g. not block the calling thread, for instance.
Change-Id: I4b36235ed707c92d117b4c33efa3db50d26f9a84
return it as a separate item in a pair instead. this will let us remove
enqueueDownload() in favor of returning metadata from download() itself
Change-Id: I74fad2ca15f920da1eefabc950c2baa2c360f2ba
it's just a uri and some headers now. those can be function arguments
with no loss of clarity. *actual* additional arguments, for example a
TLS context with additional certificates, could be added on a new and
improved FileTransfer class that carries not just a backend reference
but some real, visible context for its transfers. curl not being very
multi-threading-friendly when using multi handles will make sharing a
bit hard anyway once we drop the single global download worker thread
Change-Id: Id2112c95cbd118c6d920488f38d272d7da926460
we don't even need this outside of tests. maybe we should not do
automatic retries at this level at all and use retrying wrappers
instead? at some point we may have to do this, but not just yet.
Change-Id: If0088aa55215be81f1770c25b3bb1b5268c65cf8
this is fragile, slow as fuck, breaks constantly under high concurrency,
and completely unnecessary since ssh bypasses the stdio file descriptors
*anyway*. we do still check that we see ssh messages to ensure that none
of our subprocess handling messes with ssh's /dev/tty, but that's it now
Change-Id: Ib8e31e1999f813d07a27efc63a9d3454a9e4fcdd
* changes:
treewide: make more settings conditionally available
libstore/build: only send overridden settings to the build hook
treewide: consistently mark overridden settings as such
The old behavior results in lots of concatenations happening for no good
reason and is an artifact of the technical limitations of the old parser
(combined with some lack of care for such details).
Change-Id: I0d78d6220ca6aeaa10bc437e48e08bf7922e0bb3
This is only a minor semantical distinction, but we should be able to
properly test it, and the parser tests rely on show for that.
Change-Id: I25e868cf9544e30cdff17deb5fd50a434e0f367e
Only overridden settings are sent to the daemon, and we're going to do the same
for the build hook to. It needs to be ensured that overridden settings are in
fact consistently marked as such, so that they actually get sent.
Change-Id: I7cd58d925702f86cf2c35ad121eb191ceb62a355
Before this change, expressions like:
with import <nixpkgs> {};
runCommand "foo" {} ''
echo '@nix {}' >&$NIX_LOG_FD
''
would result in Lix crashing, because accessing nonexistent fields of
a JSON object throws an exception.
Rather than handling each field individually, we just catch JSON
exceptions wholesale. Since these log messages are an unusual
circumstance, log a warning when this happens.
Fixes#544.
Change-Id: Idc2d8acf6e37046b3ec212f42e29269163dca893
This was always a terrible idea independently of whether it crashes.
Stop doing it!
This commit was verified by running nix-shell on a trivial derivation
with --debug --verbose to get the vomit-level output of the shell rc
file and then diffing it before/after this change. I have reasonable
confidence it did not regress anything, though this code is genuinely
really hard to follow (which is a second reason that I split it into two
fmt calls).
Fixes: #533
Change-Id: I8e11ddbece2b12749fda13efe0b587a71b00bfe5
This change feels kind of gross and reveals a fair bit about the
disorganization of our tests, but I think it makes parts of it a bit
better.
Change-Id: Idb8d9a00cbd75d5c156678c6b408b42b59d5e4d7
A better fix than in 104448e75d, hence a
revert + the fix.
It turns out that this commit has the side-effect that when having e.g.
`StrictHostKeyChecking=accept-new` for a remote builder, the warnings à la
Warning: Permanently added 'builder' (ED25519) to the list of known hosts.
actually end up in the derivation's log whereas hostkey verification
errors don't, but only in the stderr of the `nix-build` invocation
(which was the motivation for the patch).
This change writes the stderr from the build-hook to
* the daemon's stderr, so that the SSH errors appear in the journal
(which was the case before 104448e75d)
* the client's stderr, as a log message
* NOT to the drv log (this is handled via `handleJSONLogMessage`)
I tried to fix the issue for legacy-ssh as well, but failed and
ultimately decided to not bother.
I know that we'll sooner or later replace the entire component, however
this is the part of the patch I have working for a while, so I figured I
might still submit it for the time being.
Change-Id: I21ca1aa0d8ae281d2eacddf26e0aa825272707e5
When the git default branch is not set to master the installcheck
test suite fails. This patch adjusts the test setup scripts to
ignore the system and user git config files.
GIT_CONFIG_SYSTEM is set to /dev/null to ignore /etc/gitconfig
GIT_CONFIG_GLOBAL is not set because the global config files
are loaded from $HOME or $XDG_CONFIG_HOME which we already
reset.
git documentation: https://git-scm.com/docs/git#Documentation/git.txt-codeGITCONFIGGLOBALcode
Change-Id: Ie73bbed1db9419c9885b9d57e4edb7a4047d5cce
There is absolutely no good reason these should show up in NARs besides
misconfigured systems and as long as the case hack exists, unpacking
such a NAR will cause its repacking to be wrong on systems with case
hack enabled.
This should not have any security impact on Lix to fix, but it was one
of the vectors for CVE-2024-45593:
https://github.com/NixOS/nix/security/advisories/GHSA-h4vv-h3jq-v493
Change-Id: I85b6075aacc069ee7039240b0f525804a2d8edcb
This is capped at 12 because 3.7 seconds of startup is painful enough
and 5.5 seconds with 24 was more annoying.
Change-Id: I327db40fd98deaa5330cd9cf6de99fb07b2c1cb0
This also rewrites a lot of the command handling in the fixtures
library, since we want to more precisely control which way that the nix
store is set up in the tests, rather than the previous method of
renaming /nix/store to some temp dir (which allows builds but does not
allow any /nix/store paths or stability across runs, which is a
significant issue for snapshot testing).
It uses a builder to reduce the amount of state carelessly thrown
around.
The evil NARs are inspired by CVE-2024-45593
(https://github.com/NixOS/nix/security/advisories/GHSA-h4vv-h3jq-v493).
No bugs were found in this endeavor.
Change-Id: Iee41b055fa96529c5a3c761f680ed1d0667ba5da
jade
eldritch horrors
Emilia Bopp
Lily Ballard
raito
Aria
Linus Heckemann
V.
Dusk Banks
alois31
piegames
Maximilian Bosch