lix-project/lix

Fork 54

fetchTree results for `file` inputs are non-reproducibly influenced by store state by not considering the hash mode when grabbing existing paths #750

New issue

Closed

opened 2025-03-19 21:20:02 +00:00 by jade · 4 comments

jade commented

2025-03-19 21:20:02 +00:00

Owner

I was working on flake-compat to support a feature for work and I found a flakes bug I wanted to unsee:

$ bash -x repro.sh
+ nix store delete /nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source
1 store paths deleted, 0.00 MiB freed
+ nix eval --raw .#lix-manifest
/nix/store/9hagxf1i39ikalx8yi3nfcya81rhr7a6-source

+ nix build --print-out-paths --no-link -f wtf.nix
/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source
+ nix eval --raw .#lix-manifest
/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source

This goes back to at least 2.18, we didn't create this bug. CppNix fixed this somewhere between 2.19 and 2.24.

Repro: clone this and run bash -x repro.sh:

https://gist.github.com/lf-/a412b1e1ab65a54454f46bde9e9be515

The reason for this is:

$ nix path-info --json /nix/store/9hagxf1i39ikalx8yi3nfcya81rhr7a6-source | jq .
[
  {
    "ca": "fixed:sha256:08q1zv32rqqjl5y07plzq1kxdpbk1zh568py7j9rr93a19lzb8n6",
    "narHash": "sha256-WmkitUnTwTCCdq0a08EdUBC/UTo/i4+xwD3mk19NA1Q=",
    "narSize": 488,
    "path": "/nix/store/9hagxf1i39ikalx8yi3nfcya81rhr7a6-source",
    "references": [],
    "registrationTime": 1742414805,
    "valid": true
  }
]

$ nix path-info --json /nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source | jq .
[
  {
    "ca": "fixed:r:sha256:0m039mgr7rixq2qqz2rz798vy42h3p0x66mdfs131hfk96sj4sas",
    "deriver": "/nix/store/ir4hbwrm698nhrihj9yv68l0pggn6f6h-source.drv",
    "narHash": "sha256-WmkitUnTwTCCdq0a08EdUBC/UTo/i4+xwD3mk19NA1Q=",
    "narSize": 488,
    "path": "/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source",
    "references": [],
    "registrationTime": 1742418772,
    "ultimate": true,
    "valid": true
  }
]

Equivalently replacing the flake bits with an unflaked.nix with the following contents works:

builtins.fetchTree {
  narHash = "sha256-WmkitUnTwTCCdq0a08EdUBC/UTo/i4+xwD3mk19NA1Q=";
  type = "file";
  url = "https://releases.lix.systems/lix/lix-2.90.0/manifest.nix";
}

I was working on flake-compat to support a feature for work and I found a flakes bug I wanted to unsee: ``` $ bash -x repro.sh + nix store delete /nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source 1 store paths deleted, 0.00 MiB freed + nix eval --raw .#lix-manifest /nix/store/9hagxf1i39ikalx8yi3nfcya81rhr7a6-source + nix build --print-out-paths --no-link -f wtf.nix /nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source + nix eval --raw .#lix-manifest /nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source ``` This goes back to at least 2.18, we didn't create this bug. CppNix fixed this somewhere between 2.19 and 2.24. Repro: clone this and run `bash -x repro.sh`: https://gist.github.com/lf-/a412b1e1ab65a54454f46bde9e9be515 The reason for this is: ``` $ nix path-info --json /nix/store/9hagxf1i39ikalx8yi3nfcya81rhr7a6-source | jq . [ { "ca": "fixed:sha256:08q1zv32rqqjl5y07plzq1kxdpbk1zh568py7j9rr93a19lzb8n6", "narHash": "sha256-WmkitUnTwTCCdq0a08EdUBC/UTo/i4+xwD3mk19NA1Q=", "narSize": 488, "path": "/nix/store/9hagxf1i39ikalx8yi3nfcya81rhr7a6-source", "references": [], "registrationTime": 1742414805, "valid": true } ] $ nix path-info --json /nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source | jq . [ { "ca": "fixed:r:sha256:0m039mgr7rixq2qqz2rz798vy42h3p0x66mdfs131hfk96sj4sas", "deriver": "/nix/store/ir4hbwrm698nhrihj9yv68l0pggn6f6h-source.drv", "narHash": "sha256-WmkitUnTwTCCdq0a08EdUBC/UTo/i4+xwD3mk19NA1Q=", "narSize": 488, "path": "/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source", "references": [], "registrationTime": 1742418772, "ultimate": true, "valid": true } ] ``` Equivalently replacing the flake bits with an `unflaked.nix` with the following contents works: ``` builtins.fetchTree { narHash = "sha256-WmkitUnTwTCCdq0a08EdUBC/UTo/i4+xwD3mk19NA1Q="; type = "file"; url = "https://releases.lix.systems/lix/lix-2.90.0/manifest.nix"; } ```

jade added the

bug

Area/flakes

labels

2025-03-19 21:20:02 +00:00

jade commented

2025-03-19 21:30:03 +00:00

Author

Owner

~~Soundness bug in the fetcher cache it appears?~~ ok no, because it also happens if you wipe that cache.

first time:

substitution of '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source': created
entered goal loop
substitution of '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source': init
substitution of '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source': trying next substituter
substitution of '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source': trying next substituter
path '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source' is required, but there is no substituter that can build it
substitution of '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source': done
substitution of '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source': goal destroyed
substitution of input 'https://releases.lix.systems/lix/lix-2.90.0/manifest.nix?narHash=sha256-WmkitUnTwTCCdq0a08EdUBC/UTo/i4%2BxwD
3mk19NA1Q%3D' failed: error: path '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source' does not exist and cannot be created
performing daemon worker op: 11
performing daemon worker op: 1
using cache entry '{"name":"source","type":"file","url":"https://releases.lix.systems/lix/lix-2.90.0/manifest.nix"}' -> '{"etag":"\
"5e95ab3ab152ab733a58bceed8cc4b81\"","url":"https://releases.lix.systems/lix/lix-2.90.0/manifest.nix"}', '/nix/store/9hagxf1i39ikal
x8yi3nfcya81rhr7a6-source'

second time:

using substituted/cached input 'https://releases.lix.systems/lix/lix-2.90.0/manifest.nix?narHash=sha256-WmkitUnTwTCCdq0a08EdUBC/UTo/i4%2BxwD3mk19NA1Q%3D' in '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source'

okay so it tries for a store path matching the expected fixed output hash and then grabs it if it finds one.

~~Soundness bug in the fetcher cache it appears?~~ ok no, because it also happens if you wipe that cache. first time: ``` substitution of '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source': created entered goal loop substitution of '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source': init substitution of '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source': trying next substituter substitution of '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source': trying next substituter path '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source' is required, but there is no substituter that can build it substitution of '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source': done substitution of '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source': goal destroyed substitution of input 'https://releases.lix.systems/lix/lix-2.90.0/manifest.nix?narHash=sha256-WmkitUnTwTCCdq0a08EdUBC/UTo/i4%2BxwD 3mk19NA1Q%3D' failed: error: path '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source' does not exist and cannot be created performing daemon worker op: 11 performing daemon worker op: 1 using cache entry '{"name":"source","type":"file","url":"https://releases.lix.systems/lix/lix-2.90.0/manifest.nix"}' -> '{"etag":"\ "5e95ab3ab152ab733a58bceed8cc4b81\"","url":"https://releases.lix.systems/lix/lix-2.90.0/manifest.nix"}', '/nix/store/9hagxf1i39ikal x8yi3nfcya81rhr7a6-source' ``` second time: ``` using substituted/cached input 'https://releases.lix.systems/lix/lix-2.90.0/manifest.nix?narHash=sha256-WmkitUnTwTCCdq0a08EdUBC/UTo/i4%2BxwD3mk19NA1Q%3D' in '/nix/store/q9prszvqfgn3h1asa5vgr665q26z1ch2-source' ``` okay so it tries for a store path matching the expected fixed output hash and then grabs it if it finds one.

jade commented

2025-03-19 21:34:57 +00:00

Author

Owner

Aha. This is completely unsound because it does not consider whether something actually is supposed to be recursive-mode hashed lol (do we want to force it to be? what is narHash supposed to mean, by design? always recursive?):

https://git.lix.systems/lix-project/lix/src/81d55286461803206240ffd8a5e3d979706b2944/lix/libfetchers/fetchers.cc#L249-L275

Aha. This is completely unsound because it does not consider whether something actually is supposed to be recursive-mode hashed lol (do we want to force it to be? what is `narHash` supposed to mean, by design? always recursive?): https://git.lix.systems/lix-project/lix/src/81d55286461803206240ffd8a5e3d979706b2944/lix/libfetchers/fetchers.cc#L249-L275

jade commented

2025-03-19 21:37:06 +00:00

Author

Owner

@raito or @piegames: thoughts on how to deal with the hash stability impacts of fixing this? it seems like following with cppnix (always using the recursive path q9pr...) is probably the correct bad choice among the bad choices. soundness bugs...

jade referenced this issue from a commit

2025-03-20 19:36:51 +00:00

Introduce file inputs and a whole dev env/tests

jade referenced this issue from lix-project/flake-compat

2025-03-20 19:38:47 +00:00

Introduce file inputs and a whole dev env/tests #69

lix-bot commented

2025-03-21 00:21:50 +00:00

Member

This issue was mentioned on Gerrit on the following CLs:

commit message in cl/2864 ("fix!: file type flake inputs are always recursive hashed")

This issue was mentioned on Gerrit on the following CLs: * commit message in [cl/2864](https://gerrit.lix.systems/c/lix/+/2864) ("fix!: `file` type flake inputs are always recursive hashed")

jade added the

Area/fetching

label

2025-03-21 00:35:52 +00:00