lix-project/lix
22
79
Fork 54
Code Issues 516 Code Review (Gerrit) Projects 8 Releases Packages Wiki Activity

Ban resolving registry indirect flake inputs #170

New issue
Open
opened 2024-03-24 04:33:13 +00:00 by jade · 21 comments
jade commented 2024-03-24 04:33:13 +00:00
Owner
Copy link

Currently you can specify a flake without inputs.*.url and it will use the flake registry. This is utterly broken behaviour as it means that updating a flake depends on host specific state such as flake registry pins, which may be completely broken if such pins are to a filesystem path such as is the default on NixOS unstable.

Let's just ban nix flake update on such inputs.

Currently you can specify a flake without `inputs.*.url` and it will use the flake registry. This is utterly broken behaviour as it means that updating a flake depends on host specific state such as flake registry pins, which may be completely broken if such pins are to a filesystem path such as is the default on NixOS unstable. Let's just ban nix flake update on such inputs.
❤️ 1
jade added this to the v2.91 milestone 2024-03-24 04:33:13 +00:00
gabriella439 commented 2024-03-25 15:58:27 +00:00
Member
Copy link

Just to clarify: is this issue to ban indirect flake inputs entirely or just banning updating those inputs? The reason why is that the title sounds more like the former whereas the description sounds more like the latter.

Just to clarify: is this issue to ban indirect flake inputs entirely or just banning updating those inputs? The reason why is that the title sounds more like the former whereas the description sounds more like the latter.
jade commented 2024-03-25 16:03:36 +00:00
Author
Owner
Copy link

I would like to ban updating them or resolving them. If they're already locked and we block that, that is a much bigger breakage to ban them than to just ban updating them.

I would like to ban updating them or resolving them. If they're already locked and we block that, that is a much bigger breakage to ban them than to just ban updating them.
jade changed title from Ban registry indirect flake inputs to Ban resolving registry indirect flake inputs 2024-03-25 16:03:59 +00:00
gabriella439 commented 2024-03-25 16:05:36 +00:00
Member
Copy link

Do you mean that we're grandfathering existing flake.nix files that use indirect flake inputs if they've already been locked in flake.lock, but we refuse to lock new indirect flake inputs going forward?

Do you mean that we're grandfathering existing `flake.nix` files that use indirect flake inputs if they've already been locked in `flake.lock`, but we refuse to lock new indirect flake inputs going forward?
jade commented 2024-03-25 16:20:32 +00:00
Author
Owner
Copy link

Yes. It seems like the way that ratchets correctness without breaking users who aren't actively doing something to the code.

hmmm this should go in the stability policy notes, this is a general view

edit: in fact, we could even print what it resolved to, to put in the flake file and then refuse to proceed.

Yes. It seems like the way that ratchets correctness without breaking users who aren't actively doing something to the code. hmmm this should go in the stability policy notes, this is a general view edit: in fact, we could even print what it resolved to, to put in the flake file and then refuse to proceed.
gabriella439 commented 2024-03-25 16:24:02 +00:00
Member
Copy link

Do we intend to support the flake registry at all? The reason I'm asking is that the arguments you're making against unspecified flake inputs seem to equally apply as reasons to drop the flake registry entirely. Like, even if the user explicitly added an input of the form:

  inputs.nixpkgs.url = "nixpkgs";

… that would still be victim to the problem you describe that "updating a flake depends on host specific state such as flake registry pins".

Do we intend to support the flake registry at all? The reason I'm asking is that the arguments you're making against unspecified flake inputs seem to equally apply as reasons to drop the flake registry entirely. Like, even if the user explicitly added an input of the form: ```nix inputs.nixpkgs.url = "nixpkgs"; ``` … that would still be victim to the problem you describe that "updating a flake depends on host specific state such as flake registry pins".
qyriad commented 2024-03-25 16:25:54 +00:00
Owner
Copy link

The flake registry is still really handy for command-line usage so I can't imagine wanting to drop it entirely. But command-line usage doesn't require actually locking flakes, which is the thing we want to avoid doing on the registry

The flake registry is still really handy for command-line usage so I can't imagine wanting to drop it entirely. But command-line usage doesn't require actually locking flakes, which is the thing we want to avoid doing on the registry
jade commented 2024-03-25 16:28:55 +00:00
Author
Owner
Copy link

Oh yes I would like to ban that too. Any usages of the flake registry in flake.nix files should be banned with prejudice.

The only place the registry should be a going concern is in direct CLI usages and even those I honestly just want to fix the thing in place by locking the flake registry in time and including it with the distribution as a default value of a configuration option. I'll file an issue for that.

Oh yes I would like to ban that too. Any usages of the flake registry in flake.nix files should be banned with prejudice. The only place the registry should be a going concern is in direct CLI usages and even those I honestly just want to fix the thing in place by locking the flake registry in time and including it with the distribution as a default value of a configuration option. I'll file an issue for that.
👍 1
gabriella439 commented 2024-03-25 16:33:45 +00:00
Member
Copy link

@qyriad: Command-line usage does conceptually lock a flake even if it doesn't create a flake.lock file per se. For example:

$ nix flake metadata nixpkgs
Resolved URL:  github:NixOS/nixpkgs/nixpkgs-unstable
Locked URL:    github:NixOS/nixpkgs/e1d501922fd7351da4200e1275dfcf5faaad1220
Description:   A collection of packages for the Nix package manager
Path:          /nix/store/r66pr19ny80412ghfmn4wa9a8yzxj3c7-source
Revision:      e1d501922fd7351da4200e1275dfcf5faaad1220
Last modified: 2024-03-23 17:08:43

That will remain locked until the tarball-ttl expires

@qyriad: Command-line usage does conceptually lock a flake even if it doesn't create a `flake.lock` file *per se*. For example: ```bash $ nix flake metadata nixpkgs Resolved URL: github:NixOS/nixpkgs/nixpkgs-unstable Locked URL: github:NixOS/nixpkgs/e1d501922fd7351da4200e1275dfcf5faaad1220 Description: A collection of packages for the Nix package manager Path: /nix/store/r66pr19ny80412ghfmn4wa9a8yzxj3c7-source Revision: e1d501922fd7351da4200e1275dfcf5faaad1220 Last modified: 2024-03-23 17:08:43 ``` That will remain locked until the `tarball-ttl` expires
qyriad commented 2024-03-25 17:35:08 +00:00
Owner
Copy link

Yes in the codebase it is also technically a flake lock, even absent any notion of tarball-ttl its technically the same mechanism

Yes in the codebase it is also technically a flake lock, even absent any notion of `tarball-ttl` its technically the same mechanism
goldstein commented 2024-08-05 11:39:57 +00:00
Member
Copy link

Oh yes I would like to ban that too. Any usages of the flake registry in flake.nix files should be banned with prejudice.

I think it makes it harder to use flakes for personal devshells that are not intended for sharing with other machines. If every flake will be required to separately pin nixpkgs, I’d end up with a lot more copies of stuff in my store, since I’d probably forget to manually update half the flakes.

It could be mitigated by making it easy to update the flake to point to the same version as the system nixpkgs though, maybe via some command that can be put into .envrc. (I guess it could already be done by using jq on /etc/nix/registry.json, but that’s not really ergonomic).

> Oh yes I would like to ban that too. Any usages of the flake registry in flake.nix files should be banned with prejudice. I think it makes it harder to use flakes for personal devshells that are not intended for sharing with other machines. If every flake will be required to separately pin nixpkgs, I’d end up with _a lot_ more copies of stuff in my store, since I’d probably forget to manually update half the flakes. It could be mitigated by making it easy to update the flake to point to the same version as the system nixpkgs though, maybe via some command that can be put into `.envrc`. (I guess it could already be done by using jq on `/etc/nix/registry.json`, but that’s not really ergonomic).
pennae commented 2024-08-05 13:57:34 +00:00
Owner
Copy link

I think it makes it harder to use flakes for personal devshells that are not intended for sharing with other machines.

this demonstrates a critical design flaw of flakes as a whole: there is no such thing as an ephemeral environment outside of the (severly lacking) nix shell and nix run commands. you are correct that using a flake purely for dev shells would suffer if the registry were outlawed for inputs. the takeaway from that shouldn't be to not disallow registry use though, but rather to add restore functionality of shell.nix that was needlessly thrown away. for example, why should nix shell without arguments not read shell.nix and auto-call it with entries of the flake registry?

> I think it makes it harder to use flakes for personal devshells that are not intended for sharing with other machines. this demonstrates a critical design flaw of flakes as a whole: there is no such thing as an ephemeral environment outside of the (severly lacking) `nix shell` and `nix run` commands. you are correct that using a flake purely for dev shells would suffer if the registry were outlawed for inputs. the takeaway from that shouldn't be to not disallow registry use though, but rather to add restore functionality of `shell.nix` that was needlessly thrown away. for example, why should `nix shell` without arguments *not* read `shell.nix` and auto-call it with entries of the flake registry?
❤️ 1
goldstein commented 2024-08-05 14:06:55 +00:00
Member
Copy link

That’s fair, it would be a valid replacement for this use case. In general, I think shells can have both updatable and non-updatable inputs (for example, I want a specific version of some utility that’s not in nixpkgs, but I’m fine with whatever nixpkgs I currently have). Using old-style shell.nix for all personal devshells makes it a bit harder to use updatable inputs (you’d need to either use something niv-like or manually update versions), but given that it’d probably run in impure mode, I don’t think it’s that bad: it’d be enough to just update version tag, without painstakingly recalculating hashes.

That’s fair, it would be a valid replacement for this use case. In general, I think shells can have both updatable and non-updatable inputs (for example, I want a specific version of some utility that’s not in nixpkgs, but I’m fine with whatever nixpkgs I currently have). Using old-style `shell.nix` for all personal devshells makes it a bit harder to use updatable inputs (you’d need to either use something niv-like or manually update versions), but given that it’d probably run in impure mode, I don’t think it’s that bad: it’d be enough to just update version tag, without painstakingly recalculating hashes.
goldstein commented 2024-08-05 14:08:31 +00:00
Member
Copy link

for example, why should nix shell without arguments not read shell.nix and auto-call it with entries of the flake registry?

I am potentially interested in hacking on this part, is there an issue?

> for example, why should nix shell without arguments not read shell.nix and auto-call it with entries of the flake registry? I am potentially interested in hacking on this part, is there an issue?
pennae commented 2024-08-05 19:13:14 +00:00
Owner
Copy link

don't think there is one, but we also don't have a full design (yet); what we said earlier was more to illuminate the problem space than suggest actual semantics. we could conceivably just take an implementation and mark it experimental until we're satisfied though?

don't think there is one, but we also don't have a full design (yet); what we said earlier was more to illuminate the problem space than suggest actual semantics. we could conceivably just take an implementation and mark it experimental until we're satisfied though?
goldstein commented 2024-08-06 16:59:04 +00:00
Member
Copy link

I asked about an issue because there’re some design considerations I’d like to discuss (I’ll share them here, but feel free to point me to a more suitable venue).

old-style shell.nix doesn’t currently accept any arguments, it just returns a derivation (you kinda can pass arguments via --arg, but they require default values). that means that adding auto-filling for arguments with no default values is not a breaking change, which is nice. there’re some options as to which arguments get passed though:

  • flakes from the registry (so basically only nixpkgs for most shells). requires the shell to import nixpkgs manually every time, but allows e.g. overlays, so is the most flexible.

  • packages via callPackage, as nix-build does. very concise and pleasant for simple shells, but less flexible and doesn’t provide a clean way to access other flakes from the registry.

  • somehow both? placing them into the same namespace feels bad though, name conflicts are bound to happen. something like

    { flakes = { someFlake }, hello, mkDerivation }: ...

    could potentially solve this problem, but that’s very much not a syntax today.

I’m leaning towards the first option, as it’s flexible, has simple semantics and doesn’t require new syntax, and I’ll try to write a basic implementation to see how that works. I’d be interested to hear another opinions on the issue though.

I asked about an issue because there’re some design considerations I’d like to discuss (I’ll share them here, but feel free to point me to a more suitable venue). old-style `shell.nix` doesn’t currently accept any arguments, it just returns a derivation (you kinda can pass arguments via `--arg`, but they require default values). that means that adding auto-filling for arguments with no default values is not a breaking change, which is nice. there’re some options as to which arguments get passed though: - flakes from the registry (so basically only `nixpkgs` for most shells). requires the shell to `import nixpkgs` manually every time, but allows e.g. overlays, so is the most flexible. - packages via `callPackage`, as `nix-build` does. very concise and pleasant for simple shells, but less flexible and doesn’t provide a clean way to access other flakes from the registry. - somehow both? placing them into the same namespace feels bad though, name conflicts are bound to happen. something like ```nix { flakes = { someFlake }, hello, mkDerivation }: ... ``` could potentially solve this problem, but that’s very much not a syntax today. I’m leaning towards the first option, as it’s flexible, has simple semantics and doesn’t require new syntax, and I’ll try to write a basic implementation to see how that works. I’d be interested to hear another opinions on the issue though.
jade modified the milestone from v2.91 to post-release 2024-08-13 01:19:36 +00:00
whovian9369 commented 2024-10-17 00:15:19 +00:00
Member
Copy link

--inputs-from [flake-url] (which has an official description of "Use the inputs of the specified flake as registry entries.") seems to fall under this want for deprecation.

[`--inputs-from [flake-url]`](https://nix.dev/manual/nix/2.18/command-ref/new-cli/nix3-flake-archive#opt-inputs-from) (which has an official description of "Use the inputs of the specified flake as registry entries.") seems to fall under this want for deprecation.
jade commented 2025-01-23 06:10:13 +00:00
Author
Owner
Copy link

https://github.com/NixOS/nix/pull/12019 Nix 2.26 has a PR for most of this

https://github.com/NixOS/nix/pull/12019 Nix 2.26 has a PR for most of this
suhr commented 2025-05-01 11:41:17 +00:00
Copy link

I use a flake for home-manager style package configuration. This change would break this workflow.

I had to rollback to Nix 2.24 because of this change.

I use a flake for home-manager style package configuration. This change would break this workflow. I had to rollback to Nix 2.24 because of this change.
jade commented 2025-05-01 23:14:57 +00:00
Author
Owner
Copy link

It would not break that workflow unless I am misunderstanding. It would just make you write inputs.nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable". Or do you mean that you're trying to use system nixpkgs from a flake?

That um, does not work. Not really, at least. It results in still having the lock file of the home-manager thingy potentially getting desynced with system nixpkgs and more than likely a path type input in the flake.lock file, which is the exact behaviour motivating this change! The solution here is to either actually use <nixpkgs> or build it from the same flake as your system, which has a real pin.

It would not break that workflow unless I am misunderstanding. It would just make you write `inputs.nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable"`. Or do you mean that you're trying to use system nixpkgs from a flake? That um, does not work. Not really, at least. It results in still having the lock file of the home-manager thingy potentially getting desynced with system nixpkgs *and* more than likely a `path` type input in the flake.lock file, which is the exact behaviour motivating this change! The solution here is to either *actually* use `<nixpkgs>` or build it from the same flake as your system, which has a real pin.
suhr commented 2025-05-02 09:01:18 +00:00
Copy link

It does desync, but I do nix flake update after system update, so no problem here.

The solution here is to either actually use <nixpkgs>

Can I use it in a flake based system? A home flake can be easily installed with nix profile.

or build it from the same flake as your system, which has a real pin

Does that allow changing the home flake without sudo? Not having to do nixos-rebuild is the very point of this setup.

It does desync, but I do `nix flake update` after system update, so no problem here. > The solution here is to either actually use `<nixpkgs>` Can I use it in a flake based system? A home flake can be easily installed with `nix profile`. > or build it from the same flake as your system, which has a real pin Does that allow changing the home flake without `sudo`? Not having to do `nixos-rebuild` is the very point of this setup.
jade commented 2025-05-03 07:49:40 +00:00
Author
Owner
Copy link

Can I use it in a flake based system? A home flake can be easily installed with nix profile.

Yes likely, though I'm not certain what the entry point setup looks like. This is not a very well supported configuration.

Does that allow changing the home flake without sudo? Not having to do nixos-rebuild is the very point of this setup.

Yes, this is fully supported. You just have a homeConfigurations exported from the same flake as the nixosSystem and you build it as normal for a home manager config in a flake.

> Can I use it in a flake based system? A home flake can be easily installed with nix profile. Yes likely, though I'm not certain what the entry point setup looks like. This is not a very well supported configuration. > Does that allow changing the home flake without sudo? Not having to do nixos-rebuild is the very point of this setup. Yes, this is fully supported. You just have a homeConfigurations exported from the same flake as the nixosSystem and you build it as normal for a home manager config in a flake.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
ci-config
release-2.91
release-2.92
release-2.93
sb/raito/darwin-kqueue-disablement
sb/raito/hydra-merge
sb/pennae/ci-config
sb/raito/phantom-referrers-gc
repl-overlay-example
reuse-compliance
release-2.90
sb/pennae/io-rewrite
git-series/activity-cleanup
sb/tom-hubrecht/primops.cc-v2
sb/bacchanalia/cache-strlen%topic=substr-perf
sb/qyriad/beta-1
sb/qyriad/2.90-beta.0
sb/rbt/test-branch
sb/rbt/justfile
sb/rbt/repl-overlays
sb/qyriad/maint-meson
sb/qyriad/maint/meson
sb/puck/meson-c-api
sb/jade/burn-it-all
dependabot/github_actions/zeebe-io/backport-action-2.4.1
dependabot/github_actions/cachix/install-nix-action-25
dependabot/github_actions/actions/labeler-5
dependabot/github_actions/cachix/cachix-action-14
2.93.3
2.92.3
2.93.2
2.91.3
2.92.2
2.91.2
2.93.1
2.93.0
2.92.0
2.91.1
2.91.0
2.90.0
2.90.0-rc1
2.90-beta.1
2.90-beta.0
Labels
Clear labels   
Affects/CppNix

Probably affects CppNix. This is a best-effort tag for troubles in old code; it exists to inform the CppNix team about problems they might care about also.

  
Affects/Nightly

Found in Lix nightly.

  
Affects/Only nightly

Tag for regressions in Lix nightly. We intend to fix all of these prior to releases.

  
Affects/Stable

Affects the stable release of Lix. Usually such issues affect nightly too but there is no guarantee.

  
Area/build-packaging

regarding the building or packaging of Lix itself

  
Area/cli

The command line interface of Lix

  
Area/evaluator

libexpr and anything else related to the evaluation of Nix expressions

  
Area/fetching

Fetching sources or other materials from the World Wide Web in Lix

  
Area/flakes

   
Area/language

language design, features, extensions, etc

  
Area/lix ci

Lix's own CI and deployment pipeline

  
Area/nix-eval-jobs

nix-eval-jobs for Lix

  
Area/profiles

profiles: nix-env, nix profile, and installer flavors all alike

  
Area/protocol

The protocol, used for communicating between the client and the daemon

  
Area/releng

Release engineering

  
Area/remote-builds

remote builds

  
Area/repl

nix repl

  
Area/repl/debugger

The --debugger that brings up a Lix repl

  
Area/store

libstore and anything else relating to the Nix store

  
bug

it's a bug

  
Context
contributors
This issue might require some context that isn't written down. It might be possible to handle as a person who already worked on this area.

  
Context
drive-by
This issue contains all the necessary information to do a drive-by contribution: that is, you don't have to talk with anyone at all to successfully solve it!

  
Context
maintainers
This issue requires a lot of know-how. Attention from maintainers is needed to proceed.

  
Context
RFD
This issue needs discussion from multiple maintainers and interested parties: it's a Request For Discussion.

  
crash 💥

Lix crashes (possibly bad error handling)

  
Cross Compilation

   
devx

development experience issues

  
docs

requires or is about documentation changes

  
Downstream Dependents

regarding an external tool's usage with Lix (e.g. nix-eval-jobs)

  
E/easy

bug is relatively simple and could simply be fixed with some minor effort

  
E/hard

this issue should probably be solved with close involvement of an expert

  
E/help wanted

maintainers may not get to this soon, but we want it fixed

  
E/reproducible

This bug is reproducible and obvious.

  
E/requires rearchitecture

this bug cannot be fixed properly without fixing a known architectural flaw

  
imported

Issue imported from upstream nix (please do not remove this label, it will cause dupes if import is rerun)

  
Language/Bash

   
Language/C++

   
Language/NixLang

   
Language/Python

   
Language/Rust

   
Needs Langver

We can't get to this until we implement language versioning

  
OS/Linux

Affects specifically Lix on Linux

  
OS/macOS

Affects specifically Lix on macOS

  
performance

make lix go faster

  
regression

This is a regression since some previous version of Lix

  
release-blocker

Blocks public release

  
stability

Fixing this makes the software more stable

  
Status
blocked
Blocked on something

  
Status
invalid
not our bug, duplicate, similar

  
Status
postponed
closed but should maybe look at later (generally with S-wontfix)

  
Status
wontfix
We chose not to change this (see issue comments for more detail).

  
testing

Tests that should be added

  
testing/flakey

Tests that are unreliable/flakey (not to be confused with flakes that test)

  
Topic/Large Scale Installations

Large scale installations of Lix, e.g. NixOS Hydra, Lix CI, big corporate installations etc.

  
ux

makes the user experience better :)

No labels
Affects/CppNix
Affects/Nightly
Affects/Only nightly
Affects/Stable
Area/build-packaging
Area/cli
Area/evaluator
Area/fetching
Area/flakes
Area/language
Area/lix ci
Area/nix-eval-jobs
Area/profiles
Area/protocol
Area/releng
Area/remote-builds
Area/repl
Area/repl/debugger
Area/store
bug
Context
contributors
Context
drive-by
Context
maintainers
Context
RFD
crash 💥
Cross Compilation
devx
docs
Downstream Dependents
E/easy
E/hard
E/help wanted
E/reproducible
E/requires rearchitecture
imported
Language/Bash
Language/C++
Language/NixLang
Language/Python
Language/Rust
Needs Langver
OS/Linux
OS/macOS
performance
regression
release-blocker
stability
Status
blocked
Status
invalid
Status
postponed
Status
wontfix
testing
testing/flakey
Topic/Large Scale Installations
ux
Milestone
Clear milestone
No items
No milestone
post-release
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
7 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: lix-project/lix#170
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?