Find all the `builtins.fetchTarball` or other eval dependency invocations in a given evaluation and allow uploading them to cache #972

New issue

Open

opened 2025-08-20 18:36:01 +00:00 by jade · 3 comments

jade commented

2025-08-20 18:36:01 +00:00

Owner

Because of silliness, without nix flake archive (which is not a complete picture if you're using anything but a flake), it's seemingly impossible to make sure that eval dependencies land in nix cache. Normally you can use post-build-hook, but that doesn't do anything for eval dependencies.

Ideal situation: some way to trace what these fetches are within an evaluation such that they can be uploaded later. Perhaps this looks like the evaluator being able to send RPCs when it adds paths to the store. Perhaps something else.

The reason that we want this is that we had a build failure at work caused by (probably) git.lix.systems experiencing AI crawling and returning 502, which was traffic we weren't even intending to be sending to git.lix.systems (and which is rather rude of us to be sending repeatedly for ~no reason! we would rather not be sending it!).

Today, this can be implemented by parsing logs, but yuck (and also being able to pull out the logs, but this means redirecting lix's output and etc, which is no fun).

Because of silliness, without `nix flake archive` (which is not a complete picture if you're using anything but a flake), it's seemingly impossible to make sure that eval dependencies land in nix cache. Normally you can use `post-build-hook`, but that doesn't do anything for eval dependencies. Ideal situation: some way to trace what these fetches are within an evaluation such that they can be uploaded later. Perhaps this looks like the evaluator being able to send RPCs when it adds paths to the store. Perhaps something else. The reason that we want this is that we had a build failure at work caused by (probably) git.lix.systems experiencing AI crawling and returning 502, which was traffic we weren't even *intending* to be sending to git.lix.systems (and which is rather rude of us to be sending repeatedly for ~no reason! we would rather not be sending it!). Today, this can be implemented by parsing logs, but *yuck* (and also being able to pull out the logs, but this means redirecting lix's output and etc, which is no fun).

jade added the

Area/evaluator

label

2025-08-20 18:36:01 +00:00

raito commented

2025-08-20 18:37:38 +00:00

Owner

https://github.com/NixOS/nixpkgs/blob/master/maintainers/scripts/find-tarballs.nix (obviously doesn't work with everything as it relies heavily on FODs metadata.)

jade commented

2025-08-20 20:54:27 +00:00

Author

Owner

I just had a horrible related idea: abuse scopedImport to inject calls to builtins.trace to log the eval dependencies. I don't think that script can do eval time deps (since in the most common case they simply go directly into import where they're not inspectable).