Download files only roughly in substitution order #19

New issue

Open

opened 2024-03-11 19:58:52 +00:00 by jade · 3 comments

jade commented

2024-03-11 19:58:52 +00:00

Owner

Might require some serious violence to our scheduling system though.... who even does the substitution? the store ... oh no.

related
NixOS/nix#5118
NixOS/nix#4533

Currently Nix blocks downloading the next file to substitute on its dependencies being substituted. This is very silly. We should just download in correct order and then send it to the store as dependencies are available. Might require some serious violence to our scheduling system though.... who even does the substitution? the store ... oh no. related https://git.lix.systems/NixOS/nix/issues/5118 https://git.lix.systems/NixOS/nix/issues/4533

jade added the

performance

label 2024-03-11 20:02:00 +00:00

jade commented

2024-03-11 20:08:22 +00:00

Author

Owner

probably the way to go about this is to have a substitution only scheduler that substitutable builds just submit all their deps to, then await the result. we'd have to ensure it's in a preorder traversal (i think)

i think part of the reason this might be cursed is I'm not sure if you can actually feed something signed into the store without being the store yourself. and i bet you the store scheduler around substitution is just the build scheduler and. sigh. probably the way to go about this is to have a substitution only scheduler that substitutable builds just submit all their deps to, then await the result. we'd have to ensure it's in a preorder traversal (i think)

jade commented

2024-03-11 20:09:03 +00:00

Author

Owner

another bug NixOS/nix#8758

another bug https://git.lix.systems/NixOS/nix/issues/8758

jade commented

2024-04-04 06:00:45 +00:00

Author

Owner

related #139

related https://git.lix.systems/lix-project/lix/issues/139