lix-project/lix

Fork 54

Evaluation result is not cached for multiple nix subcommands #925

New issue

Open

opened 2025-07-21 12:58:37 +00:00 by gepbird · 3 comments

gepbird commented

2025-07-21 12:58:37 +00:00

Describe the bug

Evaluating a derivation with flakes doesn't populate the eval cache.

If you have a problem with a specific package or NixOS,
you probably want to file an issue at https://github.com/NixOS/nixpkgs/issues.

Steps To Reproduce

(Clear your ~/.cache/nix/eval-cache-v5 if the following steps are already cached somehow)
Run time nix eval nixpkgs/e364990b50adfbb8c32c1316903652f37834538c#firefox
Run the previous command again
Notice that for the second command, Lix evaluates the derivation again which is slow:

~ ❯ time nix eval nixpkgs/e364990b50adfbb8c32c1316903652f37834538c#firefox
«derivation /nix/store/15slf7h63nnf7pa0dqnln56p3qj8x131-firefox-140.0.4.drv»
nix eval nixpkgs/e364990b50adfbb8c32c1316903652f37834538c#firefox  3.30s user 0.62s system 77% cpu 5.048 total
~ ❯ time nix eval nixpkgs/e364990b50adfbb8c32c1316903652f37834538c#firefox
«derivation /nix/store/15slf7h63nnf7pa0dqnln56p3qj8x131-firefox-140.0.4.drv»
nix eval nixpkgs/e364990b50adfbb8c32c1316903652f37834538c#firefox  3.41s user 0.60s system 77% cpu 5.153 total

Expected behavior

Running the second command should be much quicker

`nix --version` output

nix (Lix, like Nix) 2.93.2
System type: x86_64-linux
Additional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux
Features: gc, signed-caches
System configuration file: /etc/nix/nix.conf
User configuration files: /home/gep/.config/nix/nix.conf:/etc/xdg/nix/nix.conf:/home/gep/.nix-profile/etc/xdg/nix/nix.conf:/nix/profile/etc/xdg/nix/nix.conf:/home/gep/.local/state/nix/profile/etc/xdg/nix/nix.conf:/etc/profiles/per-user/gep/etc/xdg/nix/nix.conf:/nix/var/nix/profiles/default/etc/xdg/nix/nix.conf:/run/current-system/sw/etc/xdg/nix/nix.conf
Store directory: /nix/store
State directory: /nix/var/nix
Data directory: /nix/store/pknhjvaknz0vc04zkjrw0hwld41z1s9r-lix-2.93.2/share

Additional context

I'm getting varying results with nix eval (some data gets written to the eval cache db) nix shell (only an empty table gets created in eval cache db) and nix build (seems to work well, but only for some flakes, maybe it's about pinning?)

My main problem I'm trying to solve, is evaluating our server configuration on a slower Raspberry Pi 4 takes 3-4 minutes. This configuration uses some flakes that steal ~70% of the eval time, and I'd like them to be cached. First, I want to understand why is Lix not caching even a single flake output with nix eval.

I've attached the output of nix eval nixpkgs/e364990b50adfbb8c32c1316903652f37834538c#firefox -vvvv.

## Describe the bug Evaluating a derivation with flakes doesn't populate the eval cache. If you have a problem with a specific package or NixOS, you probably want to file an issue at https://github.com/NixOS/nixpkgs/issues. ## Steps To Reproduce 0. (Clear your `~/.cache/nix/eval-cache-v5` if the following steps are already cached somehow) 1. Run `time nix eval nixpkgs/e364990b50adfbb8c32c1316903652f37834538c#firefox` 2. Run the previous command again 3. Notice that for the second command, Lix evaluates the derivation again which is slow: ```console ~ ❯ time nix eval nixpkgs/e364990b50adfbb8c32c1316903652f37834538c#firefox «derivation /nix/store/15slf7h63nnf7pa0dqnln56p3qj8x131-firefox-140.0.4.drv» nix eval nixpkgs/e364990b50adfbb8c32c1316903652f37834538c#firefox 3.30s user 0.62s system 77% cpu 5.048 total ~ ❯ time nix eval nixpkgs/e364990b50adfbb8c32c1316903652f37834538c#firefox «derivation /nix/store/15slf7h63nnf7pa0dqnln56p3qj8x131-firefox-140.0.4.drv» nix eval nixpkgs/e364990b50adfbb8c32c1316903652f37834538c#firefox 3.41s user 0.60s system 77% cpu 5.153 total ``` ## Expected behavior Running the second command should be much quicker ## `nix --version` output ```console nix (Lix, like Nix) 2.93.2 System type: x86_64-linux Additional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux Features: gc, signed-caches System configuration file: /etc/nix/nix.conf User configuration files: /home/gep/.config/nix/nix.conf:/etc/xdg/nix/nix.conf:/home/gep/.nix-profile/etc/xdg/nix/nix.conf:/nix/profile/etc/xdg/nix/nix.conf:/home/gep/.local/state/nix/profile/etc/xdg/nix/nix.conf:/etc/profiles/per-user/gep/etc/xdg/nix/nix.conf:/nix/var/nix/profiles/default/etc/xdg/nix/nix.conf:/run/current-system/sw/etc/xdg/nix/nix.conf Store directory: /nix/store State directory: /nix/var/nix Data directory: /nix/store/pknhjvaknz0vc04zkjrw0hwld41z1s9r-lix-2.93.2/share ``` ## Additional context I'm getting varying results with `nix eval` (some data gets written to the eval cache db) `nix shell` (only an empty table gets created in eval cache db) and `nix build` (seems to work well, but only for some flakes, maybe it's about pinning?) My main problem I'm trying to solve, is evaluating our server [configuration](https://github.com/tchfoo/raspi-dotfiles) on a slower Raspberry Pi 4 takes 3-4 minutes. This configuration uses some flakes that steal ~70% of the eval time, and I'd like them to be cached. First, I want to understand *why* is Lix not caching even a single flake output with `nix eval`. I've attached the output of `nix eval nixpkgs/e364990b50adfbb8c32c1316903652f37834538c#firefox -vvvv`.

nix-eval-vvvv.txt

8.2 MiB

gepbird added the

bug

label

2025-07-21 12:58:37 +00:00

gepbird changed title from ~~Nothing is saved to eval-cache databases~~ to Evaluation result is not cached for multiple nix subcommands

2025-07-21 13:04:24 +00:00

pennae commented

2025-07-21 14:16:46 +00:00

Owner

we do have plans to fix all of this, but it's a long way out. caching shells the way we cache built installables is in theory possible, doing it for eval in general is not feasible because the cache drops far too much data.

the eval cache mechanism is indeed very badly busted, and it's possible that it cannot be fixed at all. your test case of evaluating eg firefox can probably be cached with enough reworking of the cache infrastructure, but due to how the system works today it is *completely impossible* to cache partial evaluation results in the way you're asking. the flake eval cache is applied at the very end of the process, it's an all-or-nothing deal. (yes, flakes are Absolutely Perfect In Every Single Imaginable Way like that.) we do have plans to fix all of this, but it's a *long* way out. caching shells the way we cache built installables is in *theory* possible, doing it for eval in general is not feasible because the cache drops far too much data.

gepbird commented

2025-07-21 17:18:56 +00:00

Author

Thanks for the explanation :)

Unfortunate that this is the state of flake eval caching, I think I'll try to implement a similar hack to https://github.com/DavHau/nix-eval-cache/blob/main/lib.nix, which puts the evaled values in a derivation which is cached. With this implementation, my problem was that it relied on pkgs to build that derivation in a way that nixpkgs had to be copied to the store on every eval. At least it cuts down all the eval time to the time of copying nixpkgs to the store, which is ~5 seconds on my drive.

Thanks for the explanation :) Unfortunate that this is the state of flake eval caching, I think I'll try to implement a similar hack to https://github.com/DavHau/nix-eval-cache/blob/main/lib.nix, which puts the evaled values in a derivation which is cached. With this implementation, my problem was that it relied on `pkgs` to build that derivation in a way that `nixpkgs` had to be copied to the store on every eval. At least it cuts down all the eval time to the time of copying nixpkgs to the store, which is ~5 seconds on my drive.

jade added the

Area/flakes

label

2025-08-03 12:01:39 +00:00

gepbird commented

2025-08-10 15:50:49 +00:00

Author

I made a "userspace" eval cache at https://git.lix.systems/gepbird/lix/src/branch/nix-cache-with-trace, but it's far from being upstreamable.

It allows caching a package based on a cache identifier that the user provides. For ease of prototyping and backwards compatibility, I modified the trace primop, that does the caching with a special message. For example evaluating builtins.trace "cache=${nixpkgs.narHash}-${pkgs.hello.name}" pkgs.hello will act as pkgs.hello for the first time, but it also saves the resulting drvPath of the package to a cache. For the second time it won't evaluate pkgs.hello, but pull the drvPath from cache. Example on how to use it: ea741ad199/trace-cache-test/flake.nix

For simple use cases like declaring packages in environment.systemPackages this works, but it quickly falls apart if you try to do something more advanced, the limitations I know of so far:

modifying the package that doesn't cause a cache invalidation (eg. overriding an input or attribute), will pull an older package from the cache, but this is left to the user to make a good cache identifier
trying to do anything else with the cached package other than getting the drvPath (eg. getting an attr like .version) will fail, as only this info is stored right now. For example putting the cached package in a pkgs.mkShell will fail, because some nixpkgs function tries to access other attrs. Caching all the attrs of a package seems too much, especially as there are many thunks there, I'm not sure how can that be serialized and later reused, or we'd need to evaluate them, costing more eval time. I imagine caching all the attrs (even excluding thunks) would bloat the cache a lot, quickly reaching gigabytes of data

Of course the best solution would be to improve the already existing eval-cache itself, (which I played around with for a few hours), but it looked more intimidating and complicated, so I made this hacky workaround. I'm pretty new to the Lix codebase and C++ itself, this is my half-baked implementation.

I think I'll try to implement a similar hack to https://github.com/DavHau/nix-eval-cache/blob/main/lib.nix, which puts the evaled values in a derivation which is cached.

This didn't work out in the end, especially without recursive-nix.

I made a "userspace" eval cache at https://git.lix.systems/gepbird/lix/src/branch/nix-cache-with-trace, but it's far from being upstreamable. It allows caching a package based on a cache identifier that the user provides. For ease of prototyping and backwards compatibility, I modified the `trace` primop, that does the caching with a special message. For example evaluating `builtins.trace "cache=${nixpkgs.narHash}-${pkgs.hello.name}" pkgs.hello` will act as `pkgs.hello` for the first time, but it also saves the resulting `drvPath` of the package to a cache. For the second time it won't evaluate `pkgs.hello`, but pull the `drvPath` from cache. Example on how to use it: https://git.lix.systems/gepbird/lix/src/commit/ea741ad199a742c742d9a3f7a8f9888e8aed798a/trace-cache-test/flake.nix For simple use cases like declaring packages in `environment.systemPackages` this works, but it quickly falls apart if you try to do something more advanced, the limitations I know of so far: - modifying the package that doesn't cause a cache invalidation (eg. overriding an input or attribute), will pull an older package from the cache, but this is left to the user to make a good cache identifier - trying to do anything else with the cached package other than getting the `drvPath` (eg. getting an attr like `.version`) will fail, as only this info is stored right now. For example putting the cached package in a `pkgs.mkShell` will fail, because some nixpkgs function tries to access other attrs. Caching all the attrs of a package seems too much, especially as there are many thunks there, I'm not sure how can that be serialized and later reused, or we'd need to evaluate them, costing more eval time. I imagine caching all the attrs (even excluding thunks) would bloat the cache a lot, quickly reaching gigabytes of data Of course the best solution would be to improve the already existing eval-cache itself, (which I played around with for a few hours), but it looked more intimidating and complicated, so I made this hacky workaround. I'm pretty new to the Lix codebase and C++ itself, [this](https://git.lix.systems/gepbird/lix/src/commit/ea741ad199a742c742d9a3f7a8f9888e8aed798a/lix/libexpr/primops.cc#L739-L840) is my half-baked implementation. > I think I'll try to implement a similar hack to https://github.com/DavHau/nix-eval-cache/blob/main/lib.nix, which puts the evaled values in a derivation which is cached. This didn't work out in the end, especially without `recursive-nix`.