feat: enable use-xdg-base-directories by default #796
Labels
No labels
Affects/CppNix
Affects/Nightly
Affects/Only nightly
Affects/Stable
Area/build-packaging
Area/cli
Area/evaluator
Area/fetching
Area/flakes
Area/language
Area/lix ci
Area/nix-eval-jobs
Area/profiles
Area/protocol
Area/releng
Area/remote-builds
Area/repl
Area/repl/debugger
Area/store
bug
Context
contributors
Context
drive-by
Context
maintainers
Context
RFD
crash 💥
Cross Compilation
devx
docs
Downstream Dependents
E/easy
E/hard
E/help wanted
E/reproducible
E/requires rearchitecture
Feature/S3
imported
Language/Bash
Language/C++
Language/NixLang
Language/Python
Language/Rust
Needs Langver
OS/Linux
OS/macOS
performance
regression
release-blocker
stability
Status
blocked
Status
invalid
Status
postponed
Status
wontfix
testing
testing/flakey
Topic/Large Scale Installations
ux
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
lix-project/lix#796
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
XDG Base Directory support has been stalled in CppNix for years. Let's see what is left to do, and fix it.
The setting is currently used in a few places: nix-channel, nix-env, eval settings, profiles. Eval setting is also used in default NIX_PATH, nix-channel and nix-env.
This internal API is error-prone: it expects the user to handle the migration manually, which can come with many caveats. We need to carefully handle this in a centralized manner.
Centralizing User Files
First, we need to move
getNixDefExprfrom eval settings tolibutil/users.cc, which is already used bygetNixDefExpr. SimilargetNixChannelsandgetNixProfilefunctions should be created there, and should be used in previous places whereuseXDGBaseDirectorywas mentioned explicitly.Then, the functionality of
getNixDefExprand similar functions should be refactored. Right now,getNixDefExpris a simple conditional onuseXDGBaseDirectory. We need more:useXDGBaseDirectory; write relevant subpaths (depending on the function) tousedPathandpossibleUnusedPathvariablesusedPathexists on disk and write the result to a variablepossibleUnusedPathexists on disk and write the result to a variableusedPathandpossibleUnusedPath:5. If
usedPathdoesn't exist andpossibleUnusedPathalso doesn't exist: callcreateNixStateDirwith relevant subpath6. If
usedPathdoesn't exist andpossibleUnusedPathexists: movepossibleUnusedPathtousedPath. This is the migration path7. If
usedPathexists andpossibleUnusedPathdoesn't exist: do nothing. This is backwards compat flow8. If
usedPathexists andpossibleUnusedPathalso exists: TBD, also backwards compat flowusedPathMigration Path and Backwards Compat
There are a few options here. What's described above is remove old, use new option.
A potentially better flow is symlink old to new option. This would write a symlink to old location that points to new location. It needs testing to see if this option works for the ecosystem. I suggest starting with this one and testing the ecosystem. The major benefit, aside from potentially better backwards compat, is that it might support Lix downgrade operations. This, too, needs to be tested.
The least preferred flow is keep both. This will potentially break UX in awful ways when doing downgrades or using some of the ecosystem options.
Known Issues
There are several things that are already reported to be broken with
use-xdg-base-directories.nix-build is busted. More accurately, this seems to be a
NIX_PATHissue, seems to originate here.fish completions are busted - seems to be resolved.
nix-collect-garbage issues - not actually related.
HM issues: the HM logic is faulty in those lines. First logic branch applies when HM is a NixOS/nix-darwin module, second if Nix impl is managed with HM. If Nix is not managed with HM, then HM defaults to legacy path and doesn't consider XDG paths at all, which is a bug.
Testing
This is a large change that needs extensive testing. There are several use cases that need to be checked on VMs. Like mentioned above, symlink old to new approach is ideal; so this approach should be taken first and discarded if related tests fail and it's a major issue:
use-xdg-base-directoriesfalse -> trueuse-xdg-base-directoriestrue -> falseuse-xdg-base-directoriestrue on a patched Lix version -> downgrade to non-patched and verify that it worksuse-xdg-base-directoriesfalse on a patched Lix version -> downgrade to non-patched and verify that it worksuse-xdg-base-directoriestrue on a patched Lix version -> downgrade to non-patched and verify that it worksuse-xdg-base-directoriestrue -> see if HM standalone worksuse-xdg-base-directoriestrue -> see ifnix-buildworksuse-xdg-base-directoriestrue -> see if Hydra and lix-eval-jobs workuse-xdg-base-directoriestrue -> see if packages that use nixForLinking workuse-xdg-base-directoriestrue -> see if packages that can be built with Lix dependency workuse-xdg-base-directoriestrue -> see if commands likenixos-rebuild,nix-env,nix-channelworkuse-xdg-base-directoriestrue -> downgrade to non-patched and see if commands likenixos-rebuild,nix-env,nix-channelworkDiscussed in Matrix: supposedly, this flag also screws up profiles by changing symlink structure from
~/.nix-profile -> /nix/var/nix/profiles/per-user/foo/bar -> /nix/store/...to~/.local/share/nix/something -> /nix/store/...Also, while checking each other's setups, we might have found a lot of differences in when and how the setup is done...
Sigh, the list of things to do grows ever larger. Things to check:
Additionally, the feature matrix between (
nix-envornix profile, non-XDG or XDG) symlink trees needs checking.Need to also investigate on what this CL is about: https://gerrit.lix.systems/c/lix/+/1101
Okay, totally unscientific research here is done. Methodology:
wget https://releases.nixos.org/nix/nix-<version>/install && chmod +x install && ./install --daemonfor script installsI used "vanilla releases" (ending on
.0, like2.3.0), because keeping up with minor patches is too much of a pain. Results:/root/.nix-channelsand/root/.nix-defexpr,/root/.nix-defexpr/channels -> /nix/var/nix/profiles/per-user/root/channels, nothing in user homedir. Relog sets up~/.nix-defexpr/channels_root -> /nix/var/nix/profiles/per-user/root/channelsand~/.nix-profile -> /nix/var/nix/profiles/per-user/user/profile. Trying to runnix-envgives errorerror: opening lock file '/nix/var/nix/db/big-lock': Permission denied. This isn't fixed withtrusted-usersor reboot, lol/root/.nix-profile-> /nix/var/nix/profiles/default. Relog also sets up~/.nix-defexpr/channels -> /nix/var/nix/profiles/per-user/user/channels, but homedir setup requires reboot, and runningnix-envgives errorerror: could not set permissions on '/nix/var/nix/profiles/per-user' to 755: Operation not permitted. And it is actually 755 already, just owned byroot:root. Gross.nix-env -iA nixpkgs.hellosets everything up. Channels are a bit weird: it sets up~/.nix-defexpr/root_channelbut~/.nix-defexpr/channelsexists and point to/nix/var/nix/profiles/per-user/user/channels(which doesn't exist). This should resolve itself when runningnix-channel --add, though.~/.nix-profile -> ~/.local/state/nix/profiles/profile -> ~/.local/state/nix/profiles/profile-1-link -> /nix/store/...-user-environment. Bizarrely enough,/root/.nix-profile->/nix/var/nix/profiles/per-user/root/profile, with/nix/var/nix/profiles/defaultalso pointing to itsudo nix-channelgives "no command found"? wtf? Reboot doesn't fix itI'll write a summary of what this means in practice later in a separate comment, because this drained life out of me.
Pre-2.14 layout looks like this:
/root/.nix-channels/root/.nix-defexpr(directory)/root/.nix-defexpr/channels -> /nix/var/nix/profiles/per-user/root/channels/root/.nix-profile -> /nix/var/nix/profiles/default -> /nix/var/nix/profiles/per-user/root/profile~/.nix-defexpr(directory)~/.nix-defexpr/channels_root -> /nix/var/nix/profiles/per-user/root/channels~/.nix-defexpr/channels -> /nix/var/nix/profiles/per-user/user/channels~/.nix-profile -> /nix/var/nix/profiles/per-user/user/profile -> /nix/var/nix/profiles/per-user/user/profile-1 -> /nix/store/...-user-environmentPost-2.14:
/root/.nix-channels/root/.nix-defexpr(directory)/root/.nix-defexpr/channels -> /nix/var/nix/profiles/per-user/root/channels/root/.nix-profile -> /nix/var/nix/profiles/default -> /nix/var/nix/profiles/per-user/root/profile~/.nix-defexpr(directory)~/.nix-defexpr/channels_root -> /nix/var/nix/profiles/per-user/root/channels~/.nix-defexpr/channels -> /nix/var/nix/profiles/per-user/user/channels~/.nix-profile -> ~/.local/state/nix/profiles/profile -> ~/.local/state/nix/profiles/profile-1 -> /nix/store/...-user-environmentTo revert 2.14 change (@qyriad CL at https://gerrit.lix.systems/c/lix/+/1101 , CppNix PR: https://github.com/NixOS/nix/pull/5226 ), we need to change daemon stuff, probably
nix-collect-garbage, and we need to do something equivalent tomkdir -p /nix/var/nix/profiles/per-user/user && cp ~/.local/state/nix/profiles/* /nix/var/nix/profiles/per-user/user/ && ln -s /nix/var/nix/profiles/per-user/user/profile ~/.nix-profilewhen running commands.Additional consideration: we need to figure out root channels, because modern installers don't even set them up. Some middle ground needs to be found, where it's impossible to footgun yourself with rootless/rootful
nix-channelusage, yet other stuff doesn't break.After that, we can actually get to doing this issue, which works as documented.
Related: #151 #215