Lix should disallow auto-optimise-store on darwin for now #477

New issue

Closed

opened 2024-08-20 09:14:36 +00:00 by weethet · 5 comments

weethet commented

2024-08-20 09:14:36 +00:00

Right now, to my knowledge, auto-optimise-store causes nix-store corruption on darwin systems. Until it is fixed, lix should not allow users to turn it on darwin.

Describe the solution you'd like

Any error when auto-optimise-store is enabled on darwin would be nice

Describe alternatives you've considered

Investigate and fix the underlying issue that causes auto-optimise-store to be broken

Additional context

See a screenshot of a discussion in the Nix on macOS matrix room attached

## Is your feature request related to a problem? Please describe. Right now, to my knowledge, auto-optimise-store causes nix-store corruption on darwin systems. Until it is fixed, lix should not allow users to turn it on darwin. ## Describe the solution you'd like Any error when auto-optimise-store is enabled on darwin would be nice ## Describe alternatives you've considered Investigate and fix the underlying issue that causes auto-optimise-store to be broken ## Additional context See a screenshot of a discussion in the Nix on macOS matrix room attached

Screenshot 2024-08-20 at 12.13.47.png

166 KiB

weethet commented

2024-08-20 09:24:45 +00:00

Author

Upstream issue on the store corruption (does not discuss prohibiting this option)

[Upstream issue on the store corruption](https://github.com/NixOS/nix/issues/7273) (does not discuss prohibiting this option)

jade added the

bug

label 2024-08-20 16:31:48 +00:00

jade added the

E/help wanted

E/easy

labels 2024-08-23 00:29:11 +00:00

jade added the

OS/macOS

label 2024-10-18 22:11:15 +00:00

lilyball commented

2024-10-20 22:19:41 +00:00

Member

Should we hard error, or should we perhaps print a warning and ignore the setting? My concern is if this is an error then when the user hits this they can't run nix profile rollback or nix-env --rollback, and if their nix.conf is managed by something like nix-darwin then that makes recovery more annoying.

Should we hard error, or should we perhaps print a warning and ignore the setting? My concern is if this is an error then when the user hits this they can't run `nix profile rollback` or `nix-env --rollback`, and if their `nix.conf` is managed by something like nix-darwin then that makes recovery more annoying.

lilyball commented

2024-10-21 02:27:07 +00:00

Member

Digging into the underlying code, it's hard to trace through all this but it looks like addToStore() can end up invoked from multiple threads, and random() is fundamentally not a thread-safe API (it uses global state), so my best guess is we're simply getting collisions. I just tried a test where I spun up 10 threads and had them generate only 100 numbers each and checked for dupes and I got a lot.

Digging into the underlying code, it's hard to trace through all this but it looks like `addToStore()` can end up invoked from multiple threads, and `random()` is fundamentally not a thread-safe API (it uses global state), so my best guess is we're simply getting collisions. I just tried a test where I spun up 10 threads and had them generate only 100 numbers each and checked for dupes and I got a _lot_.

lilyball commented

2024-10-21 05:39:53 +00:00

Member

I submitted https://gerrit.lix.systems/c/lix/+/2100 as a proposed fix for the underlying issue, though I don't know how to actually test the store corruption to make sure this solves it.