lix-project/lix
22
85
Fork 58
Code Issues 523 Code Review (Gerrit) Projects 8 Releases Packages Wiki Activity

tests.remoteBuilds is flakey #522

New issue
Open
opened 2024-09-14 00:09:15 +00:00 by jade · 2 comments
jade commented 2024-09-14 00:09:15 +00:00
Owner
Copy link

I don't know why network-online.target has no ActiveState, that's pretty weird. https://buildbot.lix.systems/#/builders/43/builds/3461/steps/1/logs/stdio

Failure log 
vm-test-run-remote-builds> (finished: must succeed: systemctl start network-online.target, in 15.97 seconds)
vm-test-run-remote-builds> builder2: must succeed: systemctl start network-online.target
vm-test-run-remote-builds> builder2: waiting for the VM to finish booting
vm-test-run-remote-builds> builder2: Guest shell says: b'Spawning backdoor root shell...\n'
vm-test-run-remote-builds> builder2: connected to guest root shell
vm-test-run-remote-builds> builder2: (connecting took 0.00 seconds)
vm-test-run-remote-builds> (finished: waiting for the VM to finish booting, in 0.00 seconds)
vm-test-run-remote-builds> builder1 # [   15.720508] systemd[1]: Started Name Service Cache Daemon (nsncd).
vm-test-run-remote-builds> builder1 # [   15.722824] nsncd[856]: Sep 11 08:25:57.287 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 3s }, path: "/var/run/nscd/socket"
vm-test-run-remote-builds> builder1 # [   15.729654] systemd[1]: Reached target Host and Network Name Lookups.
vm-test-run-remote-builds> builder1 # [   15.731157] systemd[1]: Reached target User and Group Name Lookups.
vm-test-run-remote-builds> builder2 # [   16.031812] NET: Registered PF_PACKET protocol family
vm-test-run-remote-builds> builder2 # [   16.230692] dhcpcd[672]: eth0: offered 10.0.2.15 from 10.0.2.2
vm-test-run-remote-builds> builder2 # [   16.238376] dhcpcd[672]: eth0: leased 10.0.2.15 for 86400 seconds
vm-test-run-remote-builds> builder2 # [   16.239764] dhcpcd[672]: eth0: adding route to 10.0.2.0/24
vm-test-run-remote-builds> builder2 # [   16.240928] dhcpcd[672]: eth0: adding default route via 10.0.2.2
vm-test-run-remote-builds> builder2 # [   18.897544] systemd[1]: Stopped target Host and Network Name Lookups.
vm-test-run-remote-builds> builder2 # [   18.899672] systemd[1]: Stopping Host and Network Name Lookups...
vm-test-run-remote-builds> builder2 # [   18.902800] systemd[1]: Stopped target User and Group Name Lookups.
vm-test-run-remote-builds> builder2 # [   18.904396] systemd[1]: Stopping User and Group Name Lookups...
vm-test-run-remote-builds> builder2 # [   18.906920] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...
vm-test-run-remote-builds> builder2 # [   18.919834] systemd[1]: nscd.service: Deactivated successfully.
vm-test-run-remote-builds> builder2 # [   18.922445] systemd[1]: Stopped Name Service Cache Daemon (nsncd).
vm-test-run-remote-builds> builder2 # [   18.936975] systemd[1]: Starting Name Service Cache Daemon (nsncd)...
vm-test-run-remote-builds> builder2 # [   19.000255] systemd[1]: Started DHCP Client.
vm-test-run-remote-builds> builder2 # [   19.016495] dhcpcd[672]: eth0: Router Advertisement from fe80::2
vm-test-run-remote-builds> builder2 # [   19.017851] dhcpcd[672]: eth0: adding address fec0::5054:ff:fe12:3456/64
vm-test-run-remote-builds> builder2 # [   19.019303] dhcpcd[672]: eth0: adding route to fec0::/64
vm-test-run-remote-builds> builder2 # [   19.021833] dhcpcd[672]: eth0: adding default route via fe80::2
vm-test-run-remote-builds> builder2 # [   19.045712] systemd[1]: Started Name Service Cache Daemon (nsncd).
vm-test-run-remote-builds> builder2 # [   19.047452] systemd[1]: Reached target Host and Network Name Lookups.
vm-test-run-remote-builds> builder2 # [   19.049856] systemd[1]: Reached target User and Group Name Lookups.
vm-test-run-remote-builds> builder2 # [   19.053856] nsncd[820]: Sep 11 08:26:00.630 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 3s }, path: "/var/run/nscd/socket"
vm-test-run-remote-builds> builder2 # [   20.101188] systemd[1]: Finished Networking Setup.
vm-test-run-remote-builds> builder2 # [   20.127918] systemd[1]: Starting Extra networking commands....
vm-test-run-remote-builds> builder2 # [   20.286099] systemd[1]: Finished Extra networking commands..
vm-test-run-remote-builds> builder1 # [   20.423682] sshd-pre-start[869]: Generating public/private rsa key pair.
vm-test-run-remote-builds> builder2 # [   20.303406] systemd[1]: Reached target Network.
vm-test-run-remote-builds> builder2 # [   20.312350] systemd[1]: Reached target Network is Online.
vm-test-run-remote-builds> builder1 # [   20.425195] sshd-pre-start[869]: Your identification has been saved in /etc/ssh/ssh_host_rsa_key
vm-test-run-remote-builds> builder1 # [   20.426871] sshd-pre-start[869]: Your public key has been saved in /etc/ssh/ssh_host_rsa_key.pub
vm-test-run-remote-builds> builder1 # [   20.428354] sshd-pre-start[869]: The key fingerprint is:
vm-test-run-remote-builds> builder1 # [   20.429367] sshd-pre-start[869]: SHA256:+r+/xAwAFbg7b/vOTjfqeiqdKiYMIjmYggK3yRRxWQQ root@builder1
vm-test-run-remote-builds> builder1 # [   20.431578] sshd-pre-start[869]: The key's randomart image is:
vm-test-run-remote-builds> builder1 # [   20.432675] sshd-pre-start[869]: +---[RSA 4096]----+
vm-test-run-remote-builds> builder1 # [   20.433633] sshd-pre-start[869]: |  ..E=+oo.       |
vm-test-run-remote-builds> builder1 # [   20.438647] sshd-pre-start[869]: |  ... ..         |
vm-test-run-remote-builds> builder1 # [   20.439596] sshd-pre-start[869]: |   .   ..        |
vm-test-run-remote-builds> builder1 # [   20.440562] sshd-pre-start[869]: |. o   .  .       |
vm-test-run-remote-builds> builder2 # [   20.335280] systemd[1]: Starting SSH Daemon...
vm-test-run-remote-builds> builder1 # [   20.441547] sshd-pre-start[869]: |+* o   .S .      |
vm-test-run-remote-builds> builder1 # [   20.450717] sshd-pre-start[869]: |% +   o.   +     |
vm-test-run-remote-builds> builder1 # [   20.451759] sshd-pre-start[869]: |=+    .+ .. *    |
vm-test-run-remote-builds> builder1 # [   20.452814] sshd-pre-start[869]: |  o o ..=o.+ .   |
vm-test-run-remote-builds> builder1 # [   20.453807] sshd-pre-start[869]: |   o ..+=@Xoo.   |
vm-test-run-remote-builds> builder1 # [   20.467056] sshd-pre-start[869]: +----[SHA256]-----+
vm-test-run-remote-builds> builder2 # [   20.370060] systemd[1]: Starting Permit User Sessions...
vm-test-run-remote-builds> builder1 # [   20.567132] systemd[1]: Finished Virtual Console Setup.
vm-test-run-remote-builds> builder1 # [   20.716531] sshd-pre-start[892]: Generating public/private ed25519 key pair.
vm-test-run-remote-builds> (finished: must succeed: systemctl start network-online.target, in 5.01 seconds)
vm-test-run-remote-builds> client: must succeed: systemctl start network-online.target
vm-test-run-remote-builds> client: waiting for the VM to finish booting
vm-test-run-remote-builds> client: Guest shell says: b'Spawning backdoor root shell...\n'
vm-test-run-remote-builds> client: connected to guest root shell
vm-test-run-remote-builds> client: (connecting took 0.00 seconds)
vm-test-run-remote-builds> (finished: waiting for the VM to finish booting, in 0.00 seconds)
vm-test-run-remote-builds> builder1 # [   20.717984] sshd-pre-start[892]: Your identification has been saved in /etc/ssh/ssh_host_ed25519_key
vm-test-run-remote-builds> builder1 # [   20.720539] sshd-pre-start[892]: Your public key has been saved in /etc/ssh/ssh_host_ed25519_key.pub
vm-test-run-remote-builds> builder1 # [   20.726583] sshd-pre-start[892]: The key fingerprint is:
vm-test-run-remote-builds> builder1 # [   20.727667] sshd-pre-start[892]: SHA256:cNg9hbYD6GG7JqbleNdzVe2+gSsfzJw+H+fI9rnIC8s root@builder1
vm-test-run-remote-builds> builder1 # [   20.729243] sshd-pre-start[892]: The key's randomart image is:
vm-test-run-remote-builds> builder1 # [   20.730906] sshd-pre-start[892]: +--[ED25519 256]--+
vm-test-run-remote-builds> builder1 # [   20.731871] sshd-pre-start[892]: |       .   ..    |
vm-test-run-remote-builds> builder1 # [   20.732872] sshd-pre-start[892]: |      +o..o.     |
vm-test-run-remote-builds> builder1 # [   20.733929] sshd-pre-start[892]: |     oooooo.   . |
vm-test-run-remote-builds> builder1 # [   20.735384] sshd-pre-start[892]: |      oo  o.  . .|
vm-test-run-remote-builds> builder1 # [   20.736358] sshd-pre-start[892]: |       .S  . . . |
vm-test-run-remote-builds> builder1 # [   20.737350] sshd-pre-start[892]: |    + o     = o .|
vm-test-run-remote-builds> builder1 # [   20.738640] sshd-pre-start[892]: |   * o .   o B +.|
vm-test-run-remote-builds> builder1 # [   20.739595] sshd-pre-start[892]: |  o o . o o.=o=o*|
vm-test-run-remote-builds> builder1 # [   20.740527] sshd-pre-start[892]: |   . .   o EoO**=|
vm-test-run-remote-builds> builder1 # [   20.741488] sshd-pre-start[892]: +----[SHA256]-----+
vm-test-run-remote-builds> (finished: must succeed: systemctl start network-online.target, in 0.37 seconds)
vm-test-run-remote-builds> builder1: waiting for unit network-online.target
vm-test-run-remote-builds> builder1 # [   21.164847] systemd[1]: Started SSH Daemon.
vm-test-run-remote-builds> builder1 # [   21.166038] systemd[1]: Reached target Multi-User System.
vm-test-run-remote-builds> builder1 # [   21.171856] systemd[1]: Startup finished in 2.608s (kernel) + 18.236s (userspace) = 20.845s.
vm-test-run-remote-builds> builder1 # [   21.639829] sshd[894]: Server listening on 0.0.0.0 port 22.
vm-test-run-remote-builds> builder1 # [   21.641076] sshd[894]: Server listening on :: port 22.
vm-test-run-remote-builds> (finished: waiting for unit network-online.target, in 0.63 seconds)
vm-test-run-remote-builds> builder2: waiting for unit network-online.target
vm-test-run-remote-builds> builder2 # Failed to get properties: Transport endpoint is not connected
vm-test-run-remote-builds> cleanup
vm-test-run-remote-builds> kill machine (pid 9)
vm-test-run-remote-builds> qemu-kvm: terminating on signal 15 from pid 6 (/nix/store/pfv4raslwhi3101k342752v65zxkwrxq-python3-3.11.9/bin/python3.11)
vm-test-run-remote-builds> kill machine (pid 30)
vm-test-run-remote-builds> qemu-kvm: terminating on signal 15 from pid 6 (/nix/store/pfv4raslwhi3101k342752v65zxkwrxq-python3-3.11.9/bin/python3.11)
vm-test-run-remote-builds> kill machine (pid 51)
vm-test-run-remote-builds> qemu-kvm: terminating on signal 15 from pid 6 (/nix/store/pfv4raslwhi3101k342752v65zxkwrxq-python3-3.11.9/bin/python3.11)
vm-test-run-remote-builds> (finished: cleanup, in 0.02 seconds)
vm-test-run-remote-builds> Traceback (most recent call last):
vm-test-run-remote-builds>   File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/bin/.nixos-test-driver-wrapped", line 9, in <module>
vm-test-run-remote-builds>     sys.exit(main())
vm-test-run-remote-builds>              ^^^^^^
vm-test-run-remote-builds>   File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/__init__.py", line 146, in main
vm-test-run-remote-builds>     driver.run_tests()
vm-test-run-remote-builds>   File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/driver.py", line 167, in run_tests
vm-test-run-remote-builds>     self.test_script()
vm-test-run-remote-builds>   File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/driver.py", line 159, in test_script
vm-test-run-remote-builds>     exec(self.tests, symbols, None)
vm-test-run-remote-builds>   File "<string>", line 10, in <module>
vm-test-run-remote-builds>   File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 374, in wait_for_unit
vm-test-run-remote-builds>     retry(check_active, timeout)
vm-test-run-remote-builds>   File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 129, in retry
vm-test-run-remote-builds>     if fn(False):
vm-test-run-remote-builds>        ^^^^^^^^^
vm-test-run-remote-builds>   File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 355, in check_active
vm-test-run-remote-builds>     state = self.get_unit_property(unit, "ActiveState", user)
vm-test-run-remote-builds>             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
vm-test-run-remote-builds>   File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 409, in get_unit_property
vm-test-run-remote-builds>     raise Exception(
vm-test-run-remote-builds> Exception: retrieving systemctl property "ActiveState" for unit "network-online.target" failed with exit code 1
vm-test-run-remote-builds> kill vlan (pid 7)
error: build of '/nix/store/n8b5n4k6v0102q37888viwv5xw4sgsrd-vm-test-run-remote-builds.drv' on 'ssh-ng://nix@epyc.infra.newtype.fr' failed: builder for '/nix/store/n8b5n4k6v0102q37888viwv5xw4sgsrd-vm-test-run-remote-builds.drv' failed with exit code 1;
       last 25 log lines:
       > qemu-kvm: terminating on signal 15 from pid 6 (/nix/store/pfv4raslwhi3101k342752v65zxkwrxq-python3-3.11.9/bin/python3.11)
       > (finished: cleanup, in 0.02 seconds)
       > Traceback (most recent call last):
       >   File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/bin/.nixos-test-driver-wrapped", line 9, in <module>
       >     sys.exit(main())
       >              ^^^^^^
       >   File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/__init__.py", line 146, in main
       >     driver.run_tests()
       >   File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/driver.py", line 167, in run_tests
       >     self.test_script()
       >   File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/driver.py", line 159, in test_script
       >     exec(self.tests, symbols, None)
       >   File "<string>", line 10, in <module>
       >   File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 374, in wait_for_unit
       >     retry(check_active, timeout)
       >   File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 129, in retry
       >     if fn(False):
       >        ^^^^^^^^^
       >   File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 355, in check_active
       >     state = self.get_unit_property(unit, "ActiveState", user)
       >             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       >   File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 409, in get_unit_property
       >     raise Exception(
       > Exception: retrieving systemctl property "ActiveState" for unit "network-online.target" failed with exit code 1
       > kill vlan (pid 7)
       For full logs, run 'nix log /nix/store/n8b5n4k6v0102q37888viwv5xw4sgsrd-vm-test-run-remote-builds.drv'.
error: builder for '/nix/store/n8b5n4k6v0102q37888viwv5xw4sgsrd-vm-test-run-remote-builds.drv' failed with exit code 1
I don't know why network-online.target has no ActiveState, that's pretty weird. https://buildbot.lix.systems/#/builders/43/builds/3461/steps/1/logs/stdio <details><summary>Failure log</summary> ``` vm-test-run-remote-builds> (finished: must succeed: systemctl start network-online.target, in 15.97 seconds) vm-test-run-remote-builds> builder2: must succeed: systemctl start network-online.target vm-test-run-remote-builds> builder2: waiting for the VM to finish booting vm-test-run-remote-builds> builder2: Guest shell says: b'Spawning backdoor root shell...\n' vm-test-run-remote-builds> builder2: connected to guest root shell vm-test-run-remote-builds> builder2: (connecting took 0.00 seconds) vm-test-run-remote-builds> (finished: waiting for the VM to finish booting, in 0.00 seconds) vm-test-run-remote-builds> builder1 # [ 15.720508] systemd[1]: Started Name Service Cache Daemon (nsncd). vm-test-run-remote-builds> builder1 # [ 15.722824] nsncd[856]: Sep 11 08:25:57.287 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 3s }, path: "/var/run/nscd/socket" vm-test-run-remote-builds> builder1 # [ 15.729654] systemd[1]: Reached target Host and Network Name Lookups. vm-test-run-remote-builds> builder1 # [ 15.731157] systemd[1]: Reached target User and Group Name Lookups. vm-test-run-remote-builds> builder2 # [ 16.031812] NET: Registered PF_PACKET protocol family vm-test-run-remote-builds> builder2 # [ 16.230692] dhcpcd[672]: eth0: offered 10.0.2.15 from 10.0.2.2 vm-test-run-remote-builds> builder2 # [ 16.238376] dhcpcd[672]: eth0: leased 10.0.2.15 for 86400 seconds vm-test-run-remote-builds> builder2 # [ 16.239764] dhcpcd[672]: eth0: adding route to 10.0.2.0/24 vm-test-run-remote-builds> builder2 # [ 16.240928] dhcpcd[672]: eth0: adding default route via 10.0.2.2 vm-test-run-remote-builds> builder2 # [ 18.897544] systemd[1]: Stopped target Host and Network Name Lookups. vm-test-run-remote-builds> builder2 # [ 18.899672] systemd[1]: Stopping Host and Network Name Lookups... vm-test-run-remote-builds> builder2 # [ 18.902800] systemd[1]: Stopped target User and Group Name Lookups. vm-test-run-remote-builds> builder2 # [ 18.904396] systemd[1]: Stopping User and Group Name Lookups... vm-test-run-remote-builds> builder2 # [ 18.906920] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... vm-test-run-remote-builds> builder2 # [ 18.919834] systemd[1]: nscd.service: Deactivated successfully. vm-test-run-remote-builds> builder2 # [ 18.922445] systemd[1]: Stopped Name Service Cache Daemon (nsncd). vm-test-run-remote-builds> builder2 # [ 18.936975] systemd[1]: Starting Name Service Cache Daemon (nsncd)... vm-test-run-remote-builds> builder2 # [ 19.000255] systemd[1]: Started DHCP Client. vm-test-run-remote-builds> builder2 # [ 19.016495] dhcpcd[672]: eth0: Router Advertisement from fe80::2 vm-test-run-remote-builds> builder2 # [ 19.017851] dhcpcd[672]: eth0: adding address fec0::5054:ff:fe12:3456/64 vm-test-run-remote-builds> builder2 # [ 19.019303] dhcpcd[672]: eth0: adding route to fec0::/64 vm-test-run-remote-builds> builder2 # [ 19.021833] dhcpcd[672]: eth0: adding default route via fe80::2 vm-test-run-remote-builds> builder2 # [ 19.045712] systemd[1]: Started Name Service Cache Daemon (nsncd). vm-test-run-remote-builds> builder2 # [ 19.047452] systemd[1]: Reached target Host and Network Name Lookups. vm-test-run-remote-builds> builder2 # [ 19.049856] systemd[1]: Reached target User and Group Name Lookups. vm-test-run-remote-builds> builder2 # [ 19.053856] nsncd[820]: Sep 11 08:26:00.630 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 3s }, path: "/var/run/nscd/socket" vm-test-run-remote-builds> builder2 # [ 20.101188] systemd[1]: Finished Networking Setup. vm-test-run-remote-builds> builder2 # [ 20.127918] systemd[1]: Starting Extra networking commands.... vm-test-run-remote-builds> builder2 # [ 20.286099] systemd[1]: Finished Extra networking commands.. vm-test-run-remote-builds> builder1 # [ 20.423682] sshd-pre-start[869]: Generating public/private rsa key pair. vm-test-run-remote-builds> builder2 # [ 20.303406] systemd[1]: Reached target Network. vm-test-run-remote-builds> builder2 # [ 20.312350] systemd[1]: Reached target Network is Online. vm-test-run-remote-builds> builder1 # [ 20.425195] sshd-pre-start[869]: Your identification has been saved in /etc/ssh/ssh_host_rsa_key vm-test-run-remote-builds> builder1 # [ 20.426871] sshd-pre-start[869]: Your public key has been saved in /etc/ssh/ssh_host_rsa_key.pub vm-test-run-remote-builds> builder1 # [ 20.428354] sshd-pre-start[869]: The key fingerprint is: vm-test-run-remote-builds> builder1 # [ 20.429367] sshd-pre-start[869]: SHA256:+r+/xAwAFbg7b/vOTjfqeiqdKiYMIjmYggK3yRRxWQQ root@builder1 vm-test-run-remote-builds> builder1 # [ 20.431578] sshd-pre-start[869]: The key's randomart image is: vm-test-run-remote-builds> builder1 # [ 20.432675] sshd-pre-start[869]: +---[RSA 4096]----+ vm-test-run-remote-builds> builder1 # [ 20.433633] sshd-pre-start[869]: | ..E=+oo. | vm-test-run-remote-builds> builder1 # [ 20.438647] sshd-pre-start[869]: | ... .. | vm-test-run-remote-builds> builder1 # [ 20.439596] sshd-pre-start[869]: | . .. | vm-test-run-remote-builds> builder1 # [ 20.440562] sshd-pre-start[869]: |. o . . | vm-test-run-remote-builds> builder2 # [ 20.335280] systemd[1]: Starting SSH Daemon... vm-test-run-remote-builds> builder1 # [ 20.441547] sshd-pre-start[869]: |+* o .S . | vm-test-run-remote-builds> builder1 # [ 20.450717] sshd-pre-start[869]: |% + o. + | vm-test-run-remote-builds> builder1 # [ 20.451759] sshd-pre-start[869]: |=+ .+ .. * | vm-test-run-remote-builds> builder1 # [ 20.452814] sshd-pre-start[869]: | o o ..=o.+ . | vm-test-run-remote-builds> builder1 # [ 20.453807] sshd-pre-start[869]: | o ..+=@Xoo. | vm-test-run-remote-builds> builder1 # [ 20.467056] sshd-pre-start[869]: +----[SHA256]-----+ vm-test-run-remote-builds> builder2 # [ 20.370060] systemd[1]: Starting Permit User Sessions... vm-test-run-remote-builds> builder1 # [ 20.567132] systemd[1]: Finished Virtual Console Setup. vm-test-run-remote-builds> builder1 # [ 20.716531] sshd-pre-start[892]: Generating public/private ed25519 key pair. vm-test-run-remote-builds> (finished: must succeed: systemctl start network-online.target, in 5.01 seconds) vm-test-run-remote-builds> client: must succeed: systemctl start network-online.target vm-test-run-remote-builds> client: waiting for the VM to finish booting vm-test-run-remote-builds> client: Guest shell says: b'Spawning backdoor root shell...\n' vm-test-run-remote-builds> client: connected to guest root shell vm-test-run-remote-builds> client: (connecting took 0.00 seconds) vm-test-run-remote-builds> (finished: waiting for the VM to finish booting, in 0.00 seconds) vm-test-run-remote-builds> builder1 # [ 20.717984] sshd-pre-start[892]: Your identification has been saved in /etc/ssh/ssh_host_ed25519_key vm-test-run-remote-builds> builder1 # [ 20.720539] sshd-pre-start[892]: Your public key has been saved in /etc/ssh/ssh_host_ed25519_key.pub vm-test-run-remote-builds> builder1 # [ 20.726583] sshd-pre-start[892]: The key fingerprint is: vm-test-run-remote-builds> builder1 # [ 20.727667] sshd-pre-start[892]: SHA256:cNg9hbYD6GG7JqbleNdzVe2+gSsfzJw+H+fI9rnIC8s root@builder1 vm-test-run-remote-builds> builder1 # [ 20.729243] sshd-pre-start[892]: The key's randomart image is: vm-test-run-remote-builds> builder1 # [ 20.730906] sshd-pre-start[892]: +--[ED25519 256]--+ vm-test-run-remote-builds> builder1 # [ 20.731871] sshd-pre-start[892]: | . .. | vm-test-run-remote-builds> builder1 # [ 20.732872] sshd-pre-start[892]: | +o..o. | vm-test-run-remote-builds> builder1 # [ 20.733929] sshd-pre-start[892]: | oooooo. . | vm-test-run-remote-builds> builder1 # [ 20.735384] sshd-pre-start[892]: | oo o. . .| vm-test-run-remote-builds> builder1 # [ 20.736358] sshd-pre-start[892]: | .S . . . | vm-test-run-remote-builds> builder1 # [ 20.737350] sshd-pre-start[892]: | + o = o .| vm-test-run-remote-builds> builder1 # [ 20.738640] sshd-pre-start[892]: | * o . o B +.| vm-test-run-remote-builds> builder1 # [ 20.739595] sshd-pre-start[892]: | o o . o o.=o=o*| vm-test-run-remote-builds> builder1 # [ 20.740527] sshd-pre-start[892]: | . . o EoO**=| vm-test-run-remote-builds> builder1 # [ 20.741488] sshd-pre-start[892]: +----[SHA256]-----+ vm-test-run-remote-builds> (finished: must succeed: systemctl start network-online.target, in 0.37 seconds) vm-test-run-remote-builds> builder1: waiting for unit network-online.target vm-test-run-remote-builds> builder1 # [ 21.164847] systemd[1]: Started SSH Daemon. vm-test-run-remote-builds> builder1 # [ 21.166038] systemd[1]: Reached target Multi-User System. vm-test-run-remote-builds> builder1 # [ 21.171856] systemd[1]: Startup finished in 2.608s (kernel) + 18.236s (userspace) = 20.845s. vm-test-run-remote-builds> builder1 # [ 21.639829] sshd[894]: Server listening on 0.0.0.0 port 22. vm-test-run-remote-builds> builder1 # [ 21.641076] sshd[894]: Server listening on :: port 22. vm-test-run-remote-builds> (finished: waiting for unit network-online.target, in 0.63 seconds) vm-test-run-remote-builds> builder2: waiting for unit network-online.target vm-test-run-remote-builds> builder2 # Failed to get properties: Transport endpoint is not connected vm-test-run-remote-builds> cleanup vm-test-run-remote-builds> kill machine (pid 9) vm-test-run-remote-builds> qemu-kvm: terminating on signal 15 from pid 6 (/nix/store/pfv4raslwhi3101k342752v65zxkwrxq-python3-3.11.9/bin/python3.11) vm-test-run-remote-builds> kill machine (pid 30) vm-test-run-remote-builds> qemu-kvm: terminating on signal 15 from pid 6 (/nix/store/pfv4raslwhi3101k342752v65zxkwrxq-python3-3.11.9/bin/python3.11) vm-test-run-remote-builds> kill machine (pid 51) vm-test-run-remote-builds> qemu-kvm: terminating on signal 15 from pid 6 (/nix/store/pfv4raslwhi3101k342752v65zxkwrxq-python3-3.11.9/bin/python3.11) vm-test-run-remote-builds> (finished: cleanup, in 0.02 seconds) vm-test-run-remote-builds> Traceback (most recent call last): vm-test-run-remote-builds> File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/bin/.nixos-test-driver-wrapped", line 9, in <module> vm-test-run-remote-builds> sys.exit(main()) vm-test-run-remote-builds> ^^^^^^ vm-test-run-remote-builds> File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/__init__.py", line 146, in main vm-test-run-remote-builds> driver.run_tests() vm-test-run-remote-builds> File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/driver.py", line 167, in run_tests vm-test-run-remote-builds> self.test_script() vm-test-run-remote-builds> File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/driver.py", line 159, in test_script vm-test-run-remote-builds> exec(self.tests, symbols, None) vm-test-run-remote-builds> File "<string>", line 10, in <module> vm-test-run-remote-builds> File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 374, in wait_for_unit vm-test-run-remote-builds> retry(check_active, timeout) vm-test-run-remote-builds> File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 129, in retry vm-test-run-remote-builds> if fn(False): vm-test-run-remote-builds> ^^^^^^^^^ vm-test-run-remote-builds> File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 355, in check_active vm-test-run-remote-builds> state = self.get_unit_property(unit, "ActiveState", user) vm-test-run-remote-builds> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ vm-test-run-remote-builds> File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 409, in get_unit_property vm-test-run-remote-builds> raise Exception( vm-test-run-remote-builds> Exception: retrieving systemctl property "ActiveState" for unit "network-online.target" failed with exit code 1 vm-test-run-remote-builds> kill vlan (pid 7) error: build of '/nix/store/n8b5n4k6v0102q37888viwv5xw4sgsrd-vm-test-run-remote-builds.drv' on 'ssh-ng://nix@epyc.infra.newtype.fr' failed: builder for '/nix/store/n8b5n4k6v0102q37888viwv5xw4sgsrd-vm-test-run-remote-builds.drv' failed with exit code 1; last 25 log lines: > qemu-kvm: terminating on signal 15 from pid 6 (/nix/store/pfv4raslwhi3101k342752v65zxkwrxq-python3-3.11.9/bin/python3.11) > (finished: cleanup, in 0.02 seconds) > Traceback (most recent call last): > File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/bin/.nixos-test-driver-wrapped", line 9, in <module> > sys.exit(main()) > ^^^^^^ > File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/__init__.py", line 146, in main > driver.run_tests() > File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/driver.py", line 167, in run_tests > self.test_script() > File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/driver.py", line 159, in test_script > exec(self.tests, symbols, None) > File "<string>", line 10, in <module> > File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 374, in wait_for_unit > retry(check_active, timeout) > File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 129, in retry > if fn(False): > ^^^^^^^^^ > File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 355, in check_active > state = self.get_unit_property(unit, "ActiveState", user) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/nix/store/3gb4ds3a3vh0ikf4dcn0jb5sx0jmwzgg-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 409, in get_unit_property > raise Exception( > Exception: retrieving systemctl property "ActiveState" for unit "network-online.target" failed with exit code 1 > kill vlan (pid 7) For full logs, run 'nix log /nix/store/n8b5n4k6v0102q37888viwv5xw4sgsrd-vm-test-run-remote-builds.drv'. error: builder for '/nix/store/n8b5n4k6v0102q37888viwv5xw4sgsrd-vm-test-run-remote-builds.drv' failed with exit code 1 ``` </details>
lix-bot commented 2024-09-14 00:10:04 +00:00
Member
Copy link

This issue was mentioned on Gerrit on the following CLs:

  • comment in cl/1908 ("archive: rename ParseSink to NARParseVisitor")
<!-- GERRIT_LINKBOT: {"cls": [{"backlink": "https://gerrit.lix.systems/c/lix/+/1908", "number": 1908, "kind": "comment"}], "cl_meta": {"1908": {"change_title": "archive: rename ParseSink to NARParseVisitor"}}} --> This issue was mentioned on Gerrit on the following CLs: * comment in [cl/1908](https://gerrit.lix.systems/c/lix/+/1908) ("archive: rename ParseSink to NARParseVisitor")
pennae commented 2025-10-17 12:25:49 +00:00
Owner
Copy link

if these tests ran on a single vm with multiple containers rather than multiple vms that need prepared by copying files around we might be able to fix them after all

if these tests ran on a *single* vm with multiple containers rather than multiple vms that need prepared by copying files around we might be able to fix them after all
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
sb/raitobezarius/cgroups-observability
sb/jade/wip-tars
release-2.94
sb/raito/flakes-out-of-core
sb/raito/eval-explorations
release-2.93
ci-config
sb/pennae/ci-config
release-2.91
release-2.92
sb/raito/darwin-kqueue-disablement
sb/raito/hydra-merge
sb/raito/phantom-referrers-gc
repl-overlay-example
reuse-compliance
release-2.90
sb/pennae/io-rewrite
git-series/activity-cleanup
sb/tom-hubrecht/primops.cc-v2
sb/bacchanalia/cache-strlen%topic=substr-perf
sb/qyriad/beta-1
sb/qyriad/2.90-beta.0
sb/rbt/test-branch
sb/rbt/justfile
sb/rbt/repl-overlays
sb/qyriad/maint-meson
sb/qyriad/maint/meson
sb/puck/meson-c-api
sb/jade/burn-it-all
dependabot/github_actions/zeebe-io/backport-action-2.4.1
dependabot/github_actions/cachix/install-nix-action-25
dependabot/github_actions/actions/labeler-5
dependabot/github_actions/cachix/cachix-action-14
2.94.0
2.93.3
2.92.3
2.93.2
2.91.3
2.92.2
2.91.2
2.93.1
2.93.0
2.92.0
2.91.1
2.91.0
2.90.0
2.90.0-rc1
2.90-beta.1
2.90-beta.0
Labels
Clear labels   
Affects/CppNix

Probably affects CppNix. This is a best-effort tag for troubles in old code; it exists to inform the CppNix team about problems they might care about also.

  
Affects/Nightly

Found in Lix nightly.

  
Affects/Only nightly

Tag for regressions in Lix nightly. We intend to fix all of these prior to releases.

  
Affects/Stable

Affects the stable release of Lix. Usually such issues affect nightly too but there is no guarantee.

  
Area/build-packaging

regarding the building or packaging of Lix itself

  
Area/cli

The command line interface of Lix

  
Area/evaluator

libexpr and anything else related to the evaluation of Nix expressions

  
Area/fetching

Fetching sources or other materials from the World Wide Web in Lix

  
Area/flakes

   
Area/language

language design, features, extensions, etc

  
Area/lix ci

Lix's own CI and deployment pipeline

  
Area/nix-eval-jobs

nix-eval-jobs for Lix

  
Area/profiles

profiles: nix-env, nix profile, and installer flavors all alike

  
Area/protocol

The protocol, used for communicating between the client and the daemon

  
Area/releng

Release engineering

  
Area/remote-builds

remote builds

  
Area/repl

nix repl

  
Area/repl/debugger

The --debugger that brings up a Lix repl

  
Area/store

libstore and anything else relating to the Nix store

  
awaiting
author
needs more information from the issue author

  
awaiting
contributors
needs more information from contributors

  
bug

it's a bug

  
Context
contributors
This issue might require some context that isn't written down. It might be possible to handle as a person who already worked on this area.

  
Context
drive-by
This issue contains all the necessary information to do a drive-by contribution: that is, you don't have to talk with anyone at all to successfully solve it!

  
Context
maintainers
This issue requires a lot of know-how. Attention from maintainers is needed to proceed.

  
Context
RFD
This issue needs discussion from multiple maintainers and interested parties: it's a Request For Discussion.

  
crash 💥

Lix crashes (possibly bad error handling)

  
Cross Compilation

   
devx

development experience issues

  
docs

requires or is about documentation changes

  
Downstream Dependents

regarding an external tool's usage with Lix (e.g. nix-eval-jobs)

  
E/easy

bug is relatively simple and could simply be fixed with some minor effort

  
E/hard

this issue should probably be solved with close involvement of an expert

  
E/help wanted

maintainers may not get to this soon, but we want it fixed

  
E/reproducible

This bug is reproducible and obvious.

  
E/requires rearchitecture

this bug cannot be fixed properly without fixing a known architectural flaw

  
Feature/S3

S3 support in Lix

  
imported

Issue imported from upstream nix (please do not remove this label, it will cause dupes if import is rerun)

  
Language/Bash

   
Language/C++

   
Language/NixLang

   
Language/Python

   
Language/Rust

   
Needs Langver

We can't get to this until we implement language versioning

  
OS/Linux

Affects specifically Lix on Linux

  
OS/macOS

Affects specifically Lix on macOS

  
performance

make lix go faster

  
regression

This is a regression since some previous version of Lix

  
release-blocker

Blocks public release

  
stability

Fixing this makes the software more stable

  
Status
blocked
Blocked on something

  
Status
invalid
not our bug, duplicate, similar

  
Status
postponed
closed but should maybe look at later (generally with S-wontfix)

  
Status
wontfix
We chose not to change this (see issue comments for more detail).

  
testing

Tests that should be added

  
testing/flakey

Tests that are unreliable/flakey (not to be confused with flakes that test)

  
Topic/Large Scale Installations

Large scale installations of Lix, e.g. NixOS Hydra, Lix CI, big corporate installations etc.

  
ux

makes the user experience better :)

No labels
Affects/CppNix
Affects/Nightly
Affects/Only nightly
Affects/Stable
Area/build-packaging
Area/cli
Area/evaluator
Area/fetching
Area/flakes
Area/language
Area/lix ci
Area/nix-eval-jobs
Area/profiles
Area/protocol
Area/releng
Area/remote-builds
Area/repl
Area/repl/debugger
Area/store
awaiting
author
awaiting
contributors
bug
Context
contributors
Context
drive-by
Context
maintainers
Context
RFD
crash 💥
Cross Compilation
devx
docs
Downstream Dependents
E/easy
E/hard
E/help wanted
E/reproducible
E/requires rearchitecture
Feature/S3
imported
Language/Bash
Language/C++
Language/NixLang
Language/Python
Language/Rust
Needs Langver
OS/Linux
OS/macOS
performance
regression
release-blocker
stability
Status
blocked
Status
invalid
Status
postponed
Status
wontfix
testing
testing/flakey
Topic/Large Scale Installations
ux
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lix-project/lix#522
No description provided.