[Nix#10104] `fetchTree` performs fetch even when a `narHash` is provided #52

New issue

Open

opened 2024-03-16 06:44:39 +00:00 by lix-bot · 0 comments

lix-bot commented

2024-03-16 06:44:39 +00:00

Member

Upstream-Issue: NixOS/nix#10104

Describe the bug

According to the docs, when you add narHash to a fetchTree call, the result won't be fetched if it's already available: 864fc85fc8/src/libexpr/primops/fetchTree.cc (L218)

Well, that's not happening.

Steps To Reproduce

The 1st time you fetch, it downloads ✅:

➤ nix eval --no-eval-cache --debug --expr 'builtins.fetchTree {type="github"; owner="OCA"; repo="account-analytic"; rev="bc8dbde0477634d64915d027093756defcf0ebc3"; narHash="sha256-sD75xS86JRr7q1be+U8FMI7ijDj4tgexlyNGVEa8fpc=";}'
did not find cache entry for '{"rev":"bc8dbde0477634d64915d027093756defcf0ebc3","type":"git-tarball"}'
did not find cache entry for '{"name":"source","type":"tarball","url":"https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz"}'
did not find cache entry for '{"name":"source","type":"file","url":"https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz"}'
downloading 'https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz'...
starting download of https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz
finished download of 'https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz'; curl status = 0, HTTP status = 200, body = 1089274 bytes
performing daemon worker op: 39
acquiring write lock on '/nix/var/nix/temproots/575565'
performing daemon worker op: 7
performing daemon worker op: 26
{ lastModified = 1697447248; lastModifiedDate = "20231016090728"; narHash = "sha256-sD75xS86JRr7q1be+U8FMI7ijDj4tgexlyNGVEa8fpc="; outPath = "/nix/store/q3fw9j28hdlyal51xgqlrac9vcb59p2f-source"; rev = "bc8dbde0477634d64915d027093756defcf0ebc3"; shortRev = "bc8dbde"; }
download thread shutting down

The next time you download, it says "using cache entry". That's suspicious 🕵️‍♂️. It shouldn't need any cache because there's a narHash parameter passed:

➤ nix eval --no-eval-cache --debug --expr 'builtins.fetchTree {type="github"; owner="OCA"; repo="account-analytic"; rev="bc8dbde0477634d64915d027093756defcf0ebc3"; narHash="sha256-sD75xS86JRr7q1be+U8FMI7ijDj4tgexlyNGVEa8fpc=";}'
performing daemon worker op: 11
acquiring write lock on '/nix/var/nix/temproots/575669'
performing daemon worker op: 1
using cache entry '{"rev":"bc8dbde0477634d64915d027093756defcf0ebc3","type":"git-tarball"}' -> '{"lastModified":1697447248,"rev":"bc8dbde0477634d64915d027093756defcf0ebc3"}', '/nix/store/q3fw9j28hdlyal51xgqlrac9vcb59p2f-source'
performing daemon worker op: 26
{ lastModified = 1697447248; lastModifiedDate = "20231016090728"; narHash = "sha256-sD75xS86JRr7q1be+U8FMI7ijDj4tgexlyNGVEa8fpc="; outPath = "/nix/store/q3fw9j28hdlyal51xgqlrac9vcb59p2f-source"; rev = "bc8dbde0477634d64915d027093756defcf0ebc3"; shortRev = "bc8dbde"; }

Let's remove cache and retry. We can see how fetches again. ❌ It shouldn't!:

➤ rm -rf ~/.cache/nix

➤ nix eval --no-eval-cache --debug --expr 'builtins.fetchTree {type="github"; owner="OCA"; repo="account-analytic"; rev="bc8dbde0477634d64915d027093756defcf0ebc3"; narHash="sha256-sD75xS86JRr7q1be+U8FMI7ijDj4tgexlyNGVEa8fpc=";}'
did not find cache entry for '{"rev":"bc8dbde0477634d64915d027093756defcf0ebc3","type":"git-tarball"}'
did not find cache entry for '{"name":"source","type":"tarball","url":"https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz"}'
did not find cache entry for '{"name":"source","type":"file","url":"https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz"}'
downloading 'https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz'...
starting download of https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz
finished download of 'https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz'; curl status = 0, HTTP status = 200, body = 1089274 bytes
performing daemon worker op: 39
acquiring write lock on '/nix/var/nix/temproots/575717'
performing daemon worker op: 7
performing daemon worker op: 26
{ lastModified = 1697447248; lastModifiedDate = "20231016090728"; narHash = "sha256-sD75xS86JRr7q1be+U8FMI7ijDj4tgexlyNGVEa8fpc="; outPath = "/nix/store/q3fw9j28hdlyal51xgqlrac9vcb59p2f-source"; rev = "bc8dbde0477634d64915d027093756defcf0ebc3"; shortRev = "bc8dbde"; }
download thread shutting down

Expected behavior

When calling with a NAR hash, it shouldn't need to even check the cache. Nix should just check the store and skip the fetching/caching completely.

nix-env --version output
nix-env (Nix) 2.18.1

Additional context

This might be duplicate of https://github.com/NixOS/nix/issues/9077 or https://github.com/NixOS/nix/issues/9570, but I'm not sure so I open separately.

Priorities

Add 👍 to issues you find important.

Upstream-Issue: https://git.lix.systems/NixOS/nix/issues/10104 **Describe the bug** According to the docs, when you add `narHash` to a `fetchTree` call, the result won't be fetched if it's already available: https://github.com/NixOS/nix/blob/864fc85fc88ff092725ba99907611b2b8d2205fb/src/libexpr/primops/fetchTree.cc#L218 Well, that's not happening. **Steps To Reproduce** The 1st time you fetch, it downloads ✅: ```shell ➤ nix eval --no-eval-cache --debug --expr 'builtins.fetchTree {type="github"; owner="OCA"; repo="account-analytic"; rev="bc8dbde0477634d64915d027093756defcf0ebc3"; narHash="sha256-sD75xS86JRr7q1be+U8FMI7ijDj4tgexlyNGVEa8fpc=";}' did not find cache entry for '{"rev":"bc8dbde0477634d64915d027093756defcf0ebc3","type":"git-tarball"}' did not find cache entry for '{"name":"source","type":"tarball","url":"https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz"}' did not find cache entry for '{"name":"source","type":"file","url":"https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz"}' downloading 'https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz'... starting download of https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz finished download of 'https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz'; curl status = 0, HTTP status = 200, body = 1089274 bytes performing daemon worker op: 39 acquiring write lock on '/nix/var/nix/temproots/575565' performing daemon worker op: 7 performing daemon worker op: 26 { lastModified = 1697447248; lastModifiedDate = "20231016090728"; narHash = "sha256-sD75xS86JRr7q1be+U8FMI7ijDj4tgexlyNGVEa8fpc="; outPath = "/nix/store/q3fw9j28hdlyal51xgqlrac9vcb59p2f-source"; rev = "bc8dbde0477634d64915d027093756defcf0ebc3"; shortRev = "bc8dbde"; } download thread shutting down ``` The next time you download, it says "using cache entry". That's suspicious 🕵️‍♂️. It shouldn't need any cache because there's a `narHash` parameter passed: ```shell ➤ nix eval --no-eval-cache --debug --expr 'builtins.fetchTree {type="github"; owner="OCA"; repo="account-analytic"; rev="bc8dbde0477634d64915d027093756defcf0ebc3"; narHash="sha256-sD75xS86JRr7q1be+U8FMI7ijDj4tgexlyNGVEa8fpc=";}' performing daemon worker op: 11 acquiring write lock on '/nix/var/nix/temproots/575669' performing daemon worker op: 1 using cache entry '{"rev":"bc8dbde0477634d64915d027093756defcf0ebc3","type":"git-tarball"}' -> '{"lastModified":1697447248,"rev":"bc8dbde0477634d64915d027093756defcf0ebc3"}', '/nix/store/q3fw9j28hdlyal51xgqlrac9vcb59p2f-source' performing daemon worker op: 26 { lastModified = 1697447248; lastModifiedDate = "20231016090728"; narHash = "sha256-sD75xS86JRr7q1be+U8FMI7ijDj4tgexlyNGVEa8fpc="; outPath = "/nix/store/q3fw9j28hdlyal51xgqlrac9vcb59p2f-source"; rev = "bc8dbde0477634d64915d027093756defcf0ebc3"; shortRev = "bc8dbde"; } ``` Let's remove cache and retry. We can see how fetches again. ❌ It shouldn't!: ```shell ➤ rm -rf ~/.cache/nix ➤ nix eval --no-eval-cache --debug --expr 'builtins.fetchTree {type="github"; owner="OCA"; repo="account-analytic"; rev="bc8dbde0477634d64915d027093756defcf0ebc3"; narHash="sha256-sD75xS86JRr7q1be+U8FMI7ijDj4tgexlyNGVEa8fpc=";}' did not find cache entry for '{"rev":"bc8dbde0477634d64915d027093756defcf0ebc3","type":"git-tarball"}' did not find cache entry for '{"name":"source","type":"tarball","url":"https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz"}' did not find cache entry for '{"name":"source","type":"file","url":"https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz"}' downloading 'https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz'... starting download of https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz finished download of 'https://github.com/OCA/account-analytic/archive/bc8dbde0477634d64915d027093756defcf0ebc3.tar.gz'; curl status = 0, HTTP status = 200, body = 1089274 bytes performing daemon worker op: 39 acquiring write lock on '/nix/var/nix/temproots/575717' performing daemon worker op: 7 performing daemon worker op: 26 { lastModified = 1697447248; lastModifiedDate = "20231016090728"; narHash = "sha256-sD75xS86JRr7q1be+U8FMI7ijDj4tgexlyNGVEa8fpc="; outPath = "/nix/store/q3fw9j28hdlyal51xgqlrac9vcb59p2f-source"; rev = "bc8dbde0477634d64915d027093756defcf0ebc3"; shortRev = "bc8dbde"; } download thread shutting down ``` **Expected behavior** When calling with a NAR hash, it shouldn't need to even check the cache. Nix should just check the store and skip the fetching/caching completely. **`nix-env --version` output** nix-env (Nix) 2.18.1 **Additional context** This might be duplicate of https://github.com/NixOS/nix/issues/9077 or https://github.com/NixOS/nix/issues/9570, but I'm not sure so I open separately. **Priorities** Add :+1: to [issues you find important](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc).

lix-bot added the

bug

imported

labels 2024-03-16 06:44:39 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: lix-project/lix#52

No description provided.

[Nix#10104] fetchTree performs fetch even when a narHash is provided #52

[Nix#10104] `fetchTree` performs fetch even when a `narHash` is provided #52