feat: enable use-xdg-base-directories by default				#796
		
		
	
	
		Labels
		
	
	
	
	No labels
	
		
			
	
	Affects/CppNix
		
			Affects/Nightly
		
			Affects/Only nightly
		
			Affects/Stable
		
			Area/build-packaging
		
			Area/cli
		
			Area/evaluator
		
			Area/fetching
		
			Area/flakes
		
			Area/language
		
			Area/lix ci
		
			Area/nix-eval-jobs
		
			Area/profiles
		
			Area/protocol
		
			Area/releng
		
			Area/remote-builds
		
			Area/repl
		
			Area/repl/debugger
		
			Area/store
		
			bug
		
			Context
contributors
		
			Context
drive-by
		
			Context
maintainers
		
			Context
RFD
		
			crash 💥
		
			Cross Compilation
		
			devx
		
			docs
		
			Downstream Dependents
		
			E/easy
		
			E/hard
		
			E/help wanted
		
			E/reproducible
		
			E/requires rearchitecture
		
			Feature/S3
		
			imported
		
			Language/Bash
		
			Language/C++
		
			Language/NixLang
		
			Language/Python
		
			Language/Rust
		
			Needs Langver
		
			OS/Linux
		
			OS/macOS
		
			performance
		
			regression
		
			release-blocker
		
			stability
		
			Status
blocked
		
			Status
invalid
		
			Status
postponed
		
			Status
wontfix
		
			testing
		
			testing/flakey
		
			Topic/Large Scale Installations
		
			ux
		
		
	
		No milestone
		
			
		
	No project
	
		
	
	
	
	
		No assignees
		
	
	
		
			
		
	
	
	
		1 participant
	
	
		
		
	Notifications
	
		
	
	
	
		
	
	
	Due date
No due date set.
	
		Dependencies
		
		
	
	
	No dependencies set.
	
	
		
	
	
		
			Reference
		
	
	
		
	
	
			lix-project/lix#796
			
		
	
		Loading…
	
	Add table
		Add a link
		
	
		Reference in a new issue
	
	
	No description provided.
		
		Delete branch "%!s()"
	 
	Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
XDG Base Directory support has been stalled in CppNix for years. Let's see what is left to do, and fix it.
The setting is currently used in a few places: nix-channel, nix-env, eval settings, profiles. Eval setting is also used in default NIX_PATH, nix-channel and nix-env.
This internal API is error-prone: it expects the user to handle the migration manually, which can come with many caveats. We need to carefully handle this in a centralized manner.
Centralizing User Files
First, we need to move
getNixDefExprfrom eval settings tolibutil/users.cc, which is already used bygetNixDefExpr. SimilargetNixChannelsandgetNixProfilefunctions should be created there, and should be used in previous places whereuseXDGBaseDirectorywas mentioned explicitly.Then, the functionality of
getNixDefExprand similar functions should be refactored. Right now,getNixDefExpris a simple conditional onuseXDGBaseDirectory. We need more:useXDGBaseDirectory; write relevant subpaths (depending on the function) tousedPathandpossibleUnusedPathvariablesusedPathexists on disk and write the result to a variablepossibleUnusedPathexists on disk and write the result to a variableusedPathandpossibleUnusedPath:5. If
usedPathdoesn't exist andpossibleUnusedPathalso doesn't exist: callcreateNixStateDirwith relevant subpath6. If
usedPathdoesn't exist andpossibleUnusedPathexists: movepossibleUnusedPathtousedPath. This is the migration path7. If
usedPathexists andpossibleUnusedPathdoesn't exist: do nothing. This is backwards compat flow8. If
usedPathexists andpossibleUnusedPathalso exists: TBD, also backwards compat flowusedPathMigration Path and Backwards Compat
There are a few options here. What's described above is remove old, use new option.
A potentially better flow is symlink old to new option. This would write a symlink to old location that points to new location. It needs testing to see if this option works for the ecosystem. I suggest starting with this one and testing the ecosystem. The major benefit, aside from potentially better backwards compat, is that it might support Lix downgrade operations. This, too, needs to be tested.
The least preferred flow is keep both. This will potentially break UX in awful ways when doing downgrades or using some of the ecosystem options.
Known Issues
There are several things that are already reported to be broken with
use-xdg-base-directories.nix-build is busted. More accurately, this seems to be a
NIX_PATHissue, seems to originate here.fish completions are busted - seems to be resolved.
nix-collect-garbage issues - not actually related.
HM issues: the HM logic is faulty in those lines. First logic branch applies when HM is a NixOS/nix-darwin module, second if Nix impl is managed with HM. If Nix is not managed with HM, then HM defaults to legacy path and doesn't consider XDG paths at all, which is a bug.
Testing
This is a large change that needs extensive testing. There are several use cases that need to be checked on VMs. Like mentioned above, symlink old to new approach is ideal; so this approach should be taken first and discarded if related tests fail and it's a major issue:
use-xdg-base-directoriesfalse -> trueuse-xdg-base-directoriestrue -> falseuse-xdg-base-directoriestrue on a patched Lix version -> downgrade to non-patched and verify that it worksuse-xdg-base-directoriesfalse on a patched Lix version -> downgrade to non-patched and verify that it worksuse-xdg-base-directoriestrue on a patched Lix version -> downgrade to non-patched and verify that it worksuse-xdg-base-directoriestrue -> see if HM standalone worksuse-xdg-base-directoriestrue -> see ifnix-buildworksuse-xdg-base-directoriestrue -> see if Hydra and lix-eval-jobs workuse-xdg-base-directoriestrue -> see if packages that use nixForLinking workuse-xdg-base-directoriestrue -> see if packages that can be built with Lix dependency workuse-xdg-base-directoriestrue -> see if commands likenixos-rebuild,nix-env,nix-channelworkuse-xdg-base-directoriestrue -> downgrade to non-patched and see if commands likenixos-rebuild,nix-env,nix-channelworkDiscussed in Matrix: supposedly, this flag also screws up profiles by changing symlink structure from
~/.nix-profile -> /nix/var/nix/profiles/per-user/foo/bar -> /nix/store/...to~/.local/share/nix/something -> /nix/store/...Also, while checking each other's setups, we might have found a lot of differences in when and how the setup is done...
Sigh, the list of things to do grows ever larger. Things to check:
Additionally, the feature matrix between (
nix-envornix profile, non-XDG or XDG) symlink trees needs checking.Need to also investigate on what this CL is about: https://gerrit.lix.systems/c/lix/+/1101
Okay, totally unscientific research here is done. Methodology:
wget https://releases.nixos.org/nix/nix-<version>/install && chmod +x install && ./install --daemonfor script installsI used "vanilla releases" (ending on
.0, like2.3.0), because keeping up with minor patches is too much of a pain. Results:/root/.nix-channelsand/root/.nix-defexpr,/root/.nix-defexpr/channels -> /nix/var/nix/profiles/per-user/root/channels, nothing in user homedir. Relog sets up~/.nix-defexpr/channels_root -> /nix/var/nix/profiles/per-user/root/channelsand~/.nix-profile -> /nix/var/nix/profiles/per-user/user/profile. Trying to runnix-envgives errorerror: opening lock file '/nix/var/nix/db/big-lock': Permission denied. This isn't fixed withtrusted-usersor reboot, lol/root/.nix-profile-> /nix/var/nix/profiles/default. Relog also sets up~/.nix-defexpr/channels -> /nix/var/nix/profiles/per-user/user/channels, but homedir setup requires reboot, and runningnix-envgives errorerror: could not set permissions on '/nix/var/nix/profiles/per-user' to 755: Operation not permitted. And it is actually 755 already, just owned byroot:root. Gross.nix-env -iA nixpkgs.hellosets everything up. Channels are a bit weird: it sets up~/.nix-defexpr/root_channelbut~/.nix-defexpr/channelsexists and point to/nix/var/nix/profiles/per-user/user/channels(which doesn't exist). This should resolve itself when runningnix-channel --add, though.~/.nix-profile -> ~/.local/state/nix/profiles/profile -> ~/.local/state/nix/profiles/profile-1-link -> /nix/store/...-user-environment. Bizarrely enough,/root/.nix-profile->/nix/var/nix/profiles/per-user/root/profile, with/nix/var/nix/profiles/defaultalso pointing to itsudo nix-channelgives "no command found"? wtf? Reboot doesn't fix itI'll write a summary of what this means in practice later in a separate comment, because this drained life out of me.
Pre-2.14 layout looks like this:
/root/.nix-channels/root/.nix-defexpr(directory)/root/.nix-defexpr/channels -> /nix/var/nix/profiles/per-user/root/channels/root/.nix-profile -> /nix/var/nix/profiles/default -> /nix/var/nix/profiles/per-user/root/profile~/.nix-defexpr(directory)~/.nix-defexpr/channels_root -> /nix/var/nix/profiles/per-user/root/channels~/.nix-defexpr/channels -> /nix/var/nix/profiles/per-user/user/channels~/.nix-profile -> /nix/var/nix/profiles/per-user/user/profile -> /nix/var/nix/profiles/per-user/user/profile-1 -> /nix/store/...-user-environmentPost-2.14:
/root/.nix-channels/root/.nix-defexpr(directory)/root/.nix-defexpr/channels -> /nix/var/nix/profiles/per-user/root/channels/root/.nix-profile -> /nix/var/nix/profiles/default -> /nix/var/nix/profiles/per-user/root/profile~/.nix-defexpr(directory)~/.nix-defexpr/channels_root -> /nix/var/nix/profiles/per-user/root/channels~/.nix-defexpr/channels -> /nix/var/nix/profiles/per-user/user/channels~/.nix-profile -> ~/.local/state/nix/profiles/profile -> ~/.local/state/nix/profiles/profile-1 -> /nix/store/...-user-environmentTo revert 2.14 change (@qyriad CL at https://gerrit.lix.systems/c/lix/+/1101 , CppNix PR: https://github.com/NixOS/nix/pull/5226 ), we need to change daemon stuff, probably
nix-collect-garbage, and we need to do something equivalent tomkdir -p /nix/var/nix/profiles/per-user/user && cp ~/.local/state/nix/profiles/* /nix/var/nix/profiles/per-user/user/ && ln -s /nix/var/nix/profiles/per-user/user/profile ~/.nix-profilewhen running commands.Additional consideration: we need to figure out root channels, because modern installers don't even set them up. Some middle ground needs to be found, where it's impossible to footgun yourself with rootless/rootful
nix-channelusage, yet other stuff doesn't break.After that, we can actually get to doing this issue, which works as documented.
Related: #151 #215