nix copy "querying info about /nix/store/PATH" is extremely slow and probably serial on s3 store #366

New issue

Closed

opened 2024-05-31 22:12:40 +00:00 by jade · 2 comments

jade commented

2024-05-31 22:12:40 +00:00

Owner

CACHE_STORE='s3://staging-cache?region=garage&endpoint=s3.lix.systems&want-mass-query=true&write-nar-listing=true&ls-compression=zstd&narinfo-compression=zstd&compression=zstd&parallel-compression=true&secret-key=./temp.key'
nix copy -v --to "$CACHE_STORE" ./result

I am currently watching nix copy do something utterly foolish, namely, hundreds of lines of these, seemingly in a serial fashion:

downloaded 's3://staging-cache/shisq40zmv9k6b03h9ywjyl1ki8k8bfw.narinfo' (881 bytes) in 147 ms
querying info about '/nix/store/sj0qllprnrmk1cqnnk57vvn2cqgjynbx-reenable_DT_HASH.patch' on 's3://staging-cache'...
downloaded 's3://staging-cache/sj0qllprnrmk1cqnnk57vvn2cqgjynbx.narinfo' (417 bytes) in 148 ms
querying info about '/nix/store/sjlm8agj6m3cpglc5v11d40cj7j6kin2-fix-static.patch.drv' on 's3://staging-cache'...

This should not be the case. At the very least this could be a breadth-first traversal with parallelism?

``` CACHE_STORE='s3://staging-cache?region=garage&endpoint=s3.lix.systems&want-mass-query=true&write-nar-listing=true&ls-compression=zstd&narinfo-compression=zstd&compression=zstd&parallel-compression=true&secret-key=./temp.key' nix copy -v --to "$CACHE_STORE" ./result ``` I am currently watching `nix copy` do something utterly foolish, namely, hundreds of lines of these, seemingly in a serial fashion: ``` downloaded 's3://staging-cache/shisq40zmv9k6b03h9ywjyl1ki8k8bfw.narinfo' (881 bytes) in 147 ms querying info about '/nix/store/sj0qllprnrmk1cqnnk57vvn2cqgjynbx-reenable_DT_HASH.patch' on 's3://staging-cache'... downloaded 's3://staging-cache/sj0qllprnrmk1cqnnk57vvn2cqgjynbx.narinfo' (417 bytes) in 148 ms querying info about '/nix/store/sjlm8agj6m3cpglc5v11d40cj7j6kin2-fix-static.patch.drv' on 's3://staging-cache'... ``` This should not be the case. At the very least this could be a breadth-first traversal with parallelism?