Flaky tests: `nix-copy` and `nix-copy-closure` #490

New issue

Open

opened 2024-08-27 16:54:52 +00:00 by rbt · 2 comments

rbt commented

2024-08-27 16:54:52 +00:00

Owner

The nix-copy and nix-copy-closure NixOS tests in tests/nixos seem to be flaky.

Build failures here: https://buildbot.lix.systems/#/buildrequests/167235, https://buildbot.lix.systems/#/buildrequests/167245, https://buildbot.lix.systems/#/buildrequests/167036

	`nix-copy-closure = runNixOSTestFor "x86_64-linux" ./nix-copy-closure.nix;`

	`nix-copy = runNixOSTestFor "x86_64-linux" ./nix-copy.nix;`

lix-project/lix@0dc486a5bf/tests/nixos/nix-copy-closure.nix

lix-project/lix@0dc486a5bf/tests/nixos/nix-copy.nix

The `nix-copy` and `nix-copy-closure` NixOS tests in `tests/nixos` seem to be flaky. Build failures here: https://buildbot.lix.systems/#/buildrequests/167235, https://buildbot.lix.systems/#/buildrequests/167245, https://buildbot.lix.systems/#/buildrequests/167036 https://git.lix.systems/lix-project/lix/src/commit/0dc486a5bf218870aafd5552586ab4330881647e/tests/nixos/default.nix#L125-L127 https://git.lix.systems/lix-project/lix/src/commit/0dc486a5bf218870aafd5552586ab4330881647e/tests/nixos/nix-copy-closure.nix https://git.lix.systems/lix-project/lix/src/commit/0dc486a5bf218870aafd5552586ab4330881647e/tests/nixos/nix-copy.nix

rbt added the

bug

label

2024-08-27 16:54:52 +00:00

jade commented

2024-08-27 17:03:21 +00:00

Owner

These regularly break because of garbage collection, seemingly. I'm not sure what the exact reason is but I have my suspicions it could be either a gc bug or a design flaw.

i believe that the place where the paths are vanishing too early is on the coordinator.

These regularly break because of garbage collection, seemingly. I'm not sure what the exact reason is but I have my suspicions it could be either a gc bug or a design flaw. i believe that the place where the paths are vanishing too early is on the coordinator.

jade referenced this issue

2024-09-09 03:16:15 +00:00

Race condition with gc in remote builds causes paths to vanish #505

jade added the

testing/flakey

testing

labels

2024-09-14 19:22:20 +00:00

pennae commented

2025-10-12 15:05:23 +00:00

Owner

we've taken a good long look and this problem seems to be straight up unfixable in the current test infra without removing bits of the test. the test requires that nix copy can correctly pass ssh password prompts to the user (or at least it wants to require that, it does even less because the previous attempt was more broken), but because there's no sane way to wrap ssh in ways that simulate user interaction this is extremely busted.

perhaps we should just deprecate password authentication for remote store interaction entirely. doing it through the daemon does not work anyway, and the mere chance that two ssh sessions may be open simulatenously and asking for passwords is a complete stopper too

we've taken a good long look and this problem seems to be straight up unfixable in the current test infra without removing bits of the test. the test requires that `nix copy` can correctly pass ssh password prompts to the user (or at least it *wants* to require that, it does even less because the previous attempt was *more* broken), but because there's no sane way to wrap ssh in ways that simulate user interaction this is *extremely busted*. perhaps we should just deprecate password authentication for remote store interaction entirely. doing it through the daemon does not work anyway, and the mere *chance* that two ssh sessions may be open simulatenously and asking for passwords is a complete stopper too