Crash during nixos-rebuild #542

New issue

Closed

opened 2024-10-08 16:01:39 +00:00 by cobaltcause · 3 comments

cobaltcause commented

2024-10-08 16:01:39 +00:00

Member

Describe the bug

building the system configuration...
copying 80 paths...
copying path '/nix/store/09wn41pjfaq75q1wdjnqfln3v4fylwzw-unit-script-sshd-pre-start' to 'ssh://192.168.0.27'...
error: cannot add path '/nix/store/09wn41pjfaq75q1wdjnqfln3v4fylwzw-unit-script-sshd-pre-start' because it lacks a signature by a trusted key
error (ignored): error: writing to file: Broken pipe
copying path '/nix/store/0kal2k73inviikxv9f1ciaj39lkl9a87-etc-os-release' to 'ssh://192.168.0.27'...
Lix crashed. This is a bug. We would appreciate if you report it along with what caused it at https://git.lix.systems/lix-project/lix/issues with the following information included:

error (ignored): error: interrupted by the user
Exception: nix::Interrupted: error: interrupted by the user
Stack trace:
 0# nix::printStackTrace() in /nix/store/ghxr2ykqc3rrfcy8rzdys0rzx9ah5fqj-lix-2.92.0-dev-pre20241005-ed9b7f4/lib/liblixutil.so
 1# 0x00007FA7A12947B6 in /nix/store/ghxr2ykqc3rrfcy8rzdys0rzx9ah5fqj-lix-2.92.0-dev-pre20241005-ed9b7f4/lib/liblixmain.so
 2# 0x00007FA7A00BC20A in /nix/store/22nxhmsfcv2q2rpkmfvzwg2w5z1l231z-gcc-13.3.0-lib/lib/libstdc++.so.6
 3# 0x00007FA7A00BC275 in /nix/store/22nxhmsfcv2q2rpkmfvzwg2w5z1l231z-gcc-13.3.0-lib/lib/libstdc++.so.6
 4# __cxa_rethrow in /nix/store/22nxhmsfcv2q2rpkmfvzwg2w5z1l231z-gcc-13.3.0-lib/lib/libstdc++.so.6
 5# nix::ignoreExceptionExceptInterrupt(nix::Verbosity) in /nix/store/ghxr2ykqc3rrfcy8rzdys0rzx9ah5fqj-lix-2.92.0-dev-pre20241005-ed9b7f4/lib/liblixutil.so
 6# nix::ThreadPool::doWork(bool) in /nix/store/ghxr2ykqc3rrfcy8rzdys0rzx9ah5fqj-lix-2.92.0-dev-pre20241005-ed9b7f4/lib/liblixutil.so
 7# 0x00007FA7A00E86D3 in /nix/store/22nxhmsfcv2q2rpkmfvzwg2w5z1l231z-gcc-13.3.0-lib/lib/libstdc++.so.6
 8# 0x00007FA79FE99A42 in /nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib/libc.so.6
 9# 0x00007FA79FF1905C in /nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib/libc.so.6

/nix/store/rcj58g6fk1qpvgjvwwb4x0fi418dp5n0-nixos-rebuild/bin/nixos-rebuild: line 192: 38418 Aborted                 (core dumped) "$@"

Steps To Reproduce

Run nixos-rebuild boot --target-host 192.168.0.27 --use-remote-sudo --flake .#worky as a normal non-root user
See error

I'm not actually sure how reproducible this is, I only tried/saw this once.

Expected behavior

Lix shouldn't crash. However, the error about lacking a trusted signature is correct, I accidentally did --target-host 192.168.0.27 instead of --target-host root@192.168.0.27.

`nix --version` output

nix (Lix, like Nix) 2.92.0-dev-pre20241005-ed9b7f4

Additional context

#229 can probably be closed now :P

## Describe the bug ``` building the system configuration... copying 80 paths... copying path '/nix/store/09wn41pjfaq75q1wdjnqfln3v4fylwzw-unit-script-sshd-pre-start' to 'ssh://192.168.0.27'... error: cannot add path '/nix/store/09wn41pjfaq75q1wdjnqfln3v4fylwzw-unit-script-sshd-pre-start' because it lacks a signature by a trusted key error (ignored): error: writing to file: Broken pipe copying path '/nix/store/0kal2k73inviikxv9f1ciaj39lkl9a87-etc-os-release' to 'ssh://192.168.0.27'... Lix crashed. This is a bug. We would appreciate if you report it along with what caused it at https://git.lix.systems/lix-project/lix/issues with the following information included: error (ignored): error: interrupted by the user Exception: nix::Interrupted: error: interrupted by the user Stack trace: 0# nix::printStackTrace() in /nix/store/ghxr2ykqc3rrfcy8rzdys0rzx9ah5fqj-lix-2.92.0-dev-pre20241005-ed9b7f4/lib/liblixutil.so 1# 0x00007FA7A12947B6 in /nix/store/ghxr2ykqc3rrfcy8rzdys0rzx9ah5fqj-lix-2.92.0-dev-pre20241005-ed9b7f4/lib/liblixmain.so 2# 0x00007FA7A00BC20A in /nix/store/22nxhmsfcv2q2rpkmfvzwg2w5z1l231z-gcc-13.3.0-lib/lib/libstdc++.so.6 3# 0x00007FA7A00BC275 in /nix/store/22nxhmsfcv2q2rpkmfvzwg2w5z1l231z-gcc-13.3.0-lib/lib/libstdc++.so.6 4# __cxa_rethrow in /nix/store/22nxhmsfcv2q2rpkmfvzwg2w5z1l231z-gcc-13.3.0-lib/lib/libstdc++.so.6 5# nix::ignoreExceptionExceptInterrupt(nix::Verbosity) in /nix/store/ghxr2ykqc3rrfcy8rzdys0rzx9ah5fqj-lix-2.92.0-dev-pre20241005-ed9b7f4/lib/liblixutil.so 6# nix::ThreadPool::doWork(bool) in /nix/store/ghxr2ykqc3rrfcy8rzdys0rzx9ah5fqj-lix-2.92.0-dev-pre20241005-ed9b7f4/lib/liblixutil.so 7# 0x00007FA7A00E86D3 in /nix/store/22nxhmsfcv2q2rpkmfvzwg2w5z1l231z-gcc-13.3.0-lib/lib/libstdc++.so.6 8# 0x00007FA79FE99A42 in /nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib/libc.so.6 9# 0x00007FA79FF1905C in /nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib/libc.so.6 /nix/store/rcj58g6fk1qpvgjvwwb4x0fi418dp5n0-nixos-rebuild/bin/nixos-rebuild: line 192: 38418 Aborted (core dumped) "$@" ``` ## Steps To Reproduce 1. Run `nixos-rebuild boot --target-host 192.168.0.27 --use-remote-sudo --flake .#worky` as a normal non-root user 2. See error I'm not actually sure how reproducible this is, I only tried/saw this once. ## Expected behavior Lix shouldn't crash. However, the error about lacking a trusted signature is correct, I accidentally did `--target-host 192.168.0.27` instead of `--target-host root@192.168.0.27`. ## `nix --version` output nix (Lix, like Nix) 2.92.0-dev-pre20241005-ed9b7f4 ## Additional context https://git.lix.systems/lix-project/lix/issues/229 can probably be closed now :P

cobaltcause added the

bug

label 2024-10-08 16:01:39 +00:00

jade commented

2024-10-08 18:45:54 +00:00

Owner

oh this is a bug but it's not the bug you'd think. i think nix::Interrupted gets caught in the new error printing machinery in spite of not being a bug in lix.

the broken pipe error is semi expected in that we know the daemon hangs up with certain errors (but possibly a regression that it's printed, will have to contemplate a bit).

oh this is a bug but it's not the bug you'd think. i think nix::Interrupted gets caught in the new error printing machinery in spite of not being a bug in lix. the broken pipe error is semi expected in that we know the daemon hangs up with certain errors (but possibly a regression that it's printed, will have to contemplate a bit).

👍 1 👀 1

jade commented

2024-10-08 22:49:04 +00:00

Owner

either we have a bad cherry pick or the cherry picked code completely overlooked that exceptions could fall out of the thread and didn't make it a clean thread termination. either way, goofy, this is a lix bug. the thread pool should eat terminated exceptions and cleanly stop the thread.

(the cool part about this infrastructure is that it telling you it's a lix bug is tautologically true! either it's a lix bug that it's telling you it's a lix bug, or it's a lix bug!)

oh. the Interrupted exception fell out of the end of the thread, as i observe when actually reading the stack trace; gods i am so glad i wrote that thing. that's 100% the thread pool code's fault and is 100% a bug. cc @rbt. either we have a bad cherry pick or the cherry picked code completely overlooked that exceptions could fall out of the thread and didn't make it a clean thread termination. either way, *goofy*, this *is* a lix bug. the thread pool should eat terminated exceptions and cleanly stop the thread. (the cool part about this infrastructure is that it telling you it's a lix bug is tautologically true! either it's a lix bug that it's telling you it's a lix bug, or it's a lix bug!)

lix-bot commented

2024-10-09 22:51:13 +00:00

Member

This issue was mentioned on Gerrit on the following CLs:

commit message in cl/2055 ("Fix std::terminate call in thread pool")

This issue was mentioned on Gerrit on the following CLs: * commit message in [cl/2055](https://gerrit.lix.systems/c/lix/+/2055) ("Fix std::terminate call in thread pool")