`nix copy --substitute-on-destination` shouldn't fail if a substituter isn't reachable #686

New issue

Open

opened 2025-02-21 14:31:01 +00:00 by ma27 · 1 comment

ma27 commented

2025-02-21 14:31:01 +00:00

Member

Describe the bug

Part of my deployment process is to do nix copy --substitute-on-destination /nix/store.. --to ssh-ng://target-host.

If said host has broken DNS or my binary cache is down, it fails like this:

$ nix copy --substitute-on-destination --to ssh-ng://target-host /nix/store/crrf4bysvarqp0g9pyhcrj47689g4i5l-hello-2.12.1
warning: error: unable to download 'https://cache.nixos.org/crrf4bysvarqp0g9pyhcrj47689g4i5l.narinfo': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 308 ms
warning: error: unable to download 'https://cache.nixos.org/crrf4bysvarqp0g9pyhcrj47689g4i5l.narinfo': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 603 ms
warning: error: unable to download 'https://cache.nixos.org/crrf4bysvarqp0g9pyhcrj47689g4i5l.narinfo': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1122 ms
warning: error: unable to download 'https://cache.nixos.org/crrf4bysvarqp0g9pyhcrj47689g4i5l.narinfo': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 2154 ms
error: unable to download 'https://cache.nixos.org/crrf4bysvarqp0g9pyhcrj47689g4i5l.narinfo': Could not resolve hostname (6) Could not resolve host: cache.nixos.org

This is a little problematic because I usually encounter these situations when I broke something with a bad configuration and I try to deploy a fix.

I do have a connect-timeout configured and I know that this works as documented. Given the logs of the progress bar it seems as if queryValidPaths on the remote side is what's failing hard.

Steps To Reproduce

Do systemctl stop kresd@1 on the target host (or whatever you need to do to not have DNS)
nix copy --substitute-on-destination $(nix-build '<nixpkgs>' -A hello) --to ssh-ng://target-host

Expected behavior

In the case above the store-path in question existed on my local machine. Hence, I'd expect Nix to ignore the unavailable substituter and copy the path.

If the path doesn't exist on my local machine, it should fail hard, but with a different error (something along the lines of "remote host can't reach the substituter, path cannot be uploaded since it's not in $store").

`nix --version` output

local side:

nix (Lix, like Nix) 2.93.0-devpre20250208_72326c4
System type: x86_64-linux
Additional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux, x86_64-v4-linux
Features: gc, signed-caches
System configuration file: /etc/nix/nix.conf
User configuration files: /home/ma27/.config/nix/nix.conf:/etc/xdg/nix/nix.conf:/home/ma27/.nix-profile/etc/xdg/nix/nix.conf:/nix/profile/etc/xdg/nix/nix.conf:/home/ma27/.local/state/nix/profile/etc/xdg/nix/nix.conf:/etc/profiles/per-user/ma27/etc/xdg/nix/nix.conf:/nix/var/nix/profiles/default/etc/xdg/nix/nix.conf:/run/current-system/sw/etc/xdg/nix/nix.conf
Store directory: /nix/store
State directory: /nix/var/nix
Data directory: /nix/store/blfkh48f3zp6vc1nscc5426qcx109iv1-lix-2.93.0-devpre20250208_72326c4/share

remote side:

nix (Lix, like Nix) 2.93.0-devpre20250208_72326c4
System type: x86_64-linux
Additional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux, x86_64-v4-linux
Features: gc, signed-caches
System configuration file: /etc/nix/nix.conf
User configuration files: /root/.config/nix/nix.conf:/etc/xdg/nix/nix.conf:/root/.nix-profile/etc/xdg/nix/nix.conf:/nix/profile/etc/xdg/nix/nix.conf:/root/.local/state/nix/profile/etc/xdg/nix/nix.conf:/etc/profiles/per-user/root/etc/xdg/nix/nix.conf:/nix/var/nix/profiles/default/etc/xdg/nix/nix.conf:/run/current-system/sw/etc/xdg/nix/nix.conf
Store directory: /nix/store
State directory: /nix/var/nix
Data directory: /nix/store/blfkh48f3zp6vc1nscc5426qcx109iv1-lix-2.93.0-devpre20250208_72326c4/share

Additional context

Add any other context about the problem here.

## Describe the bug Part of my deployment process is to do `nix copy --substitute-on-destination /nix/store.. --to ssh-ng://target-host`. If said host has broken DNS or my binary cache is down, it fails like this: ``` $ nix copy --substitute-on-destination --to ssh-ng://target-host /nix/store/crrf4bysvarqp0g9pyhcrj47689g4i5l-hello-2.12.1 warning: error: unable to download 'https://cache.nixos.org/crrf4bysvarqp0g9pyhcrj47689g4i5l.narinfo': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 308 ms warning: error: unable to download 'https://cache.nixos.org/crrf4bysvarqp0g9pyhcrj47689g4i5l.narinfo': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 603 ms warning: error: unable to download 'https://cache.nixos.org/crrf4bysvarqp0g9pyhcrj47689g4i5l.narinfo': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1122 ms warning: error: unable to download 'https://cache.nixos.org/crrf4bysvarqp0g9pyhcrj47689g4i5l.narinfo': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 2154 ms error: unable to download 'https://cache.nixos.org/crrf4bysvarqp0g9pyhcrj47689g4i5l.narinfo': Could not resolve hostname (6) Could not resolve host: cache.nixos.org ``` This is a little problematic because I usually encounter these situations when I broke something with a bad configuration and I try to deploy a fix. I do have a `connect-timeout` configured and I know that this works as documented. Given the logs of the progress bar it seems as if `queryValidPaths` on the remote side is what's failing hard. ## Steps To Reproduce 1. Do `systemctl stop kresd@1` on the target host (or whatever you need to do to not have DNS) 2. `nix copy --substitute-on-destination $(nix-build '<nixpkgs>' -A hello) --to ssh-ng://target-host` ## Expected behavior In the case above the store-path in question existed on my local machine. Hence, I'd expect Nix to ignore the unavailable substituter and copy the path. If the path doesn't exist on my local machine, it should fail hard, but with a different error (something along the lines of "remote host can't reach the substituter, path cannot be uploaded since it's not in $store"). ## `nix --version` output local side: ``` nix (Lix, like Nix) 2.93.0-devpre20250208_72326c4 System type: x86_64-linux Additional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux, x86_64-v4-linux Features: gc, signed-caches System configuration file: /etc/nix/nix.conf User configuration files: /home/ma27/.config/nix/nix.conf:/etc/xdg/nix/nix.conf:/home/ma27/.nix-profile/etc/xdg/nix/nix.conf:/nix/profile/etc/xdg/nix/nix.conf:/home/ma27/.local/state/nix/profile/etc/xdg/nix/nix.conf:/etc/profiles/per-user/ma27/etc/xdg/nix/nix.conf:/nix/var/nix/profiles/default/etc/xdg/nix/nix.conf:/run/current-system/sw/etc/xdg/nix/nix.conf Store directory: /nix/store State directory: /nix/var/nix Data directory: /nix/store/blfkh48f3zp6vc1nscc5426qcx109iv1-lix-2.93.0-devpre20250208_72326c4/share ``` remote side: ``` nix (Lix, like Nix) 2.93.0-devpre20250208_72326c4 System type: x86_64-linux Additional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux, x86_64-v4-linux Features: gc, signed-caches System configuration file: /etc/nix/nix.conf User configuration files: /root/.config/nix/nix.conf:/etc/xdg/nix/nix.conf:/root/.nix-profile/etc/xdg/nix/nix.conf:/nix/profile/etc/xdg/nix/nix.conf:/root/.local/state/nix/profile/etc/xdg/nix/nix.conf:/etc/profiles/per-user/root/etc/xdg/nix/nix.conf:/nix/var/nix/profiles/default/etc/xdg/nix/nix.conf:/run/current-system/sw/etc/xdg/nix/nix.conf Store directory: /nix/store State directory: /nix/var/nix Data directory: /nix/store/blfkh48f3zp6vc1nscc5426qcx109iv1-lix-2.93.0-devpre20250208_72326c4/share ``` ## Additional context Add any other context about the problem here.