Ban resolving registry indirect flake inputs

jade commented

2024-03-24 04:33:13 +00:00

Owner

Currently you can specify a flake without inputs.*.url and it will use the flake registry. This is utterly broken behaviour as it means that updating a flake depends on host specific state such as flake registry pins, which may be completely broken if such pins are to a filesystem path such as is the default on NixOS unstable.

Let's just ban nix flake update on such inputs.

Currently you can specify a flake without `inputs.*.url` and it will use the flake registry. This is utterly broken behaviour as it means that updating a flake depends on host specific state such as flake registry pins, which may be completely broken if such pins are to a filesystem path such as is the default on NixOS unstable. Let's just ban nix flake update on such inputs.

❤️ 1

jade added this to the v2.91 milestone 2024-03-24 04:33:13 +00:00

jade added the

labels 2024-03-24 04:33:50 +00:00

gabriella439 commented

2024-03-25 15:58:27 +00:00

Member

Just to clarify: is this issue to ban indirect flake inputs entirely or just banning updating those inputs? The reason why is that the title sounds more like the former whereas the description sounds more like the latter.

jade commented

2024-03-25 16:03:36 +00:00

Author

Owner

I would like to ban updating them or resolving them. If they're already locked and we block that, that is a much bigger breakage to ban them than to just ban updating them.

jade changed title from ~~Ban registry indirect flake inputs~~ to Ban resolving registry indirect flake inputs

2024-03-25 16:03:59 +00:00

gabriella439 commented

2024-03-25 16:05:36 +00:00

Member

Do you mean that we're grandfathering existing flake.nix files that use indirect flake inputs if they've already been locked in flake.lock, but we refuse to lock new indirect flake inputs going forward?

Do you mean that we're grandfathering existing `flake.nix` files that use indirect flake inputs if they've already been locked in `flake.lock`, but we refuse to lock new indirect flake inputs going forward?

lunaphied referenced this issue

2024-03-25 16:09:13 +00:00

Flake registry entries can overlap with fully-qualified flake inputs #181

jade commented

2024-03-25 16:20:32 +00:00

Author

Owner

Yes. It seems like the way that ratchets correctness without breaking users who aren't actively doing something to the code.

hmmm this should go in the stability policy notes, this is a general view

edit: in fact, we could even print what it resolved to, to put in the flake file and then refuse to proceed.

Yes. It seems like the way that ratchets correctness without breaking users who aren't actively doing something to the code. hmmm this should go in the stability policy notes, this is a general view edit: in fact, we could even print what it resolved to, to put in the flake file and then refuse to proceed.

gabriella439 commented

2024-03-25 16:24:02 +00:00

Member

Do we intend to support the flake registry at all? The reason I'm asking is that the arguments you're making against unspecified flake inputs seem to equally apply as reasons to drop the flake registry entirely. Like, even if the user explicitly added an input of the form:

  inputs.nixpkgs.url = "nixpkgs";

… that would still be victim to the problem you describe that "updating a flake depends on host specific state such as flake registry pins".

Do we intend to support the flake registry at all? The reason I'm asking is that the arguments you're making against unspecified flake inputs seem to equally apply as reasons to drop the flake registry entirely. Like, even if the user explicitly added an input of the form: ```nix inputs.nixpkgs.url = "nixpkgs"; ``` … that would still be victim to the problem you describe that "updating a flake depends on host specific state such as flake registry pins".

qyriad commented

2024-03-25 16:25:54 +00:00

Owner

The flake registry is still really handy for command-line usage so I can't imagine wanting to drop it entirely. But command-line usage doesn't require actually locking flakes, which is the thing we want to avoid doing on the registry

jade commented

2024-03-25 16:28:55 +00:00

Author

Owner

Oh yes I would like to ban that too. Any usages of the flake registry in flake.nix files should be banned with prejudice.

The only place the registry should be a going concern is in direct CLI usages and even those I honestly just want to fix the thing in place by locking the flake registry in time and including it with the distribution as a default value of a configuration option. I'll file an issue for that.

Oh yes I would like to ban that too. Any usages of the flake registry in flake.nix files should be banned with prejudice. The only place the registry should be a going concern is in direct CLI usages and even those I honestly just want to fix the thing in place by locking the flake registry in time and including it with the distribution as a default value of a configuration option. I'll file an issue for that.

👍 1

gabriella439 commented

2024-03-25 16:33:45 +00:00

Member

@qyriad: Command-line usage does conceptually lock a flake even if it doesn't create a flake.lock file per se. For example:

$ nix flake metadata nixpkgs
Resolved URL:  github:NixOS/nixpkgs/nixpkgs-unstable
Locked URL:    github:NixOS/nixpkgs/e1d501922fd7351da4200e1275dfcf5faaad1220
Description:   A collection of packages for the Nix package manager
Path:          /nix/store/r66pr19ny80412ghfmn4wa9a8yzxj3c7-source
Revision:      e1d501922fd7351da4200e1275dfcf5faaad1220
Last modified: 2024-03-23 17:08:43

That will remain locked until the tarball-ttl expires

@qyriad: Command-line usage does conceptually lock a flake even if it doesn't create a `flake.lock` file *per se*. For example: ```bash $ nix flake metadata nixpkgs Resolved URL: github:NixOS/nixpkgs/nixpkgs-unstable Locked URL: github:NixOS/nixpkgs/e1d501922fd7351da4200e1275dfcf5faaad1220 Description: A collection of packages for the Nix package manager Path: /nix/store/r66pr19ny80412ghfmn4wa9a8yzxj3c7-source Revision: e1d501922fd7351da4200e1275dfcf5faaad1220 Last modified: 2024-03-23 17:08:43 ``` That will remain locked until the `tarball-ttl` expires

jade referenced this issue

2024-03-25 16:39:28 +00:00

Deprecate online flake registries #183

qyriad commented

2024-03-25 17:35:08 +00:00

Owner

Yes in the codebase it is also technically a flake lock, even absent any notion of tarball-ttl its technically the same mechanism

Yes in the codebase it is also technically a flake lock, even absent any notion of `tarball-ttl` its technically the same mechanism

qyriad referenced this issue

2024-03-25 19:12:46 +00:00

Deprecate online flake registries #183

jade added the

Area/flakes

label 2024-03-30 00:04:22 +00:00

goldstein commented

2024-08-05 11:39:57 +00:00

Member

Oh yes I would like to ban that too. Any usages of the flake registry in flake.nix files should be banned with prejudice.

I think it makes it harder to use flakes for personal devshells that are not intended for sharing with other machines. If every flake will be required to separately pin nixpkgs, I’d end up with a lot more copies of stuff in my store, since I’d probably forget to manually update half the flakes.

It could be mitigated by making it easy to update the flake to point to the same version as the system nixpkgs though, maybe via some command that can be put into .envrc. (I guess it could already be done by using jq on /etc/nix/registry.json, but that’s not really ergonomic).

> Oh yes I would like to ban that too. Any usages of the flake registry in flake.nix files should be banned with prejudice. I think it makes it harder to use flakes for personal devshells that are not intended for sharing with other machines. If every flake will be required to separately pin nixpkgs, I’d end up with _a lot_ more copies of stuff in my store, since I’d probably forget to manually update half the flakes. It could be mitigated by making it easy to update the flake to point to the same version as the system nixpkgs though, maybe via some command that can be put into `.envrc`. (I guess it could already be done by using jq on `/etc/nix/registry.json`, but that’s not really ergonomic).

pennae commented

2024-08-05 13:57:34 +00:00

Owner

I think it makes it harder to use flakes for personal devshells that are not intended for sharing with other machines.

this demonstrates a critical design flaw of flakes as a whole: there is no such thing as an ephemeral environment outside of the (severly lacking) nix shell and nix run commands. you are correct that using a flake purely for dev shells would suffer if the registry were outlawed for inputs. the takeaway from that shouldn't be to not disallow registry use though, but rather to add restore functionality of shell.nix that was needlessly thrown away. for example, why should nix shell without arguments not read shell.nix and auto-call it with entries of the flake registry?

> I think it makes it harder to use flakes for personal devshells that are not intended for sharing with other machines. this demonstrates a critical design flaw of flakes as a whole: there is no such thing as an ephemeral environment outside of the (severly lacking) `nix shell` and `nix run` commands. you are correct that using a flake purely for dev shells would suffer if the registry were outlawed for inputs. the takeaway from that shouldn't be to not disallow registry use though, but rather to add restore functionality of `shell.nix` that was needlessly thrown away. for example, why should `nix shell` without arguments *not* read `shell.nix` and auto-call it with entries of the flake registry?

❤️ 1

goldstein commented

2024-08-05 14:06:55 +00:00

Member

That’s fair, it would be a valid replacement for this use case. In general, I think shells can have both updatable and non-updatable inputs (for example, I want a specific version of some utility that’s not in nixpkgs, but I’m fine with whatever nixpkgs I currently have). Using old-style shell.nix for all personal devshells makes it a bit harder to use updatable inputs (you’d need to either use something niv-like or manually update versions), but given that it’d probably run in impure mode, I don’t think it’s that bad: it’d be enough to just update version tag, without painstakingly recalculating hashes.

That’s fair, it would be a valid replacement for this use case. In general, I think shells can have both updatable and non-updatable inputs (for example, I want a specific version of some utility that’s not in nixpkgs, but I’m fine with whatever nixpkgs I currently have). Using old-style `shell.nix` for all personal devshells makes it a bit harder to use updatable inputs (you’d need to either use something niv-like or manually update versions), but given that it’d probably run in impure mode, I don’t think it’s that bad: it’d be enough to just update version tag, without painstakingly recalculating hashes.

goldstein commented

2024-08-05 14:08:31 +00:00

Member

for example, why should nix shell without arguments not read shell.nix and auto-call it with entries of the flake registry?

I am potentially interested in hacking on this part, is there an issue?

> for example, why should nix shell without arguments not read shell.nix and auto-call it with entries of the flake registry? I am potentially interested in hacking on this part, is there an issue?

pennae commented

2024-08-05 19:13:14 +00:00

Owner

don't think there is one, but we also don't have a full design (yet); what we said earlier was more to illuminate the problem space than suggest actual semantics. we could conceivably just take an implementation and mark it experimental until we're satisfied though?

goldstein commented

2024-08-06 16:59:04 +00:00

Member

I asked about an issue because there’re some design considerations I’d like to discuss (I’ll share them here, but feel free to point me to a more suitable venue).

old-style shell.nix doesn’t currently accept any arguments, it just returns a derivation (you kinda can pass arguments via --arg, but they require default values). that means that adding auto-filling for arguments with no default values is not a breaking change, which is nice. there’re some options as to which arguments get passed though:

flakes from the registry (so basically only nixpkgs for most shells). requires the shell to import nixpkgs manually every time, but allows e.g. overlays, so is the most flexible.
packages via callPackage, as nix-build does. very concise and pleasant for simple shells, but less flexible and doesn’t provide a clean way to access other flakes from the registry.
somehow both? placing them into the same namespace feels bad though, name conflicts are bound to happen. something like
```
{ flakes = { someFlake }, hello, mkDerivation }: ...
```
could potentially solve this problem, but that’s very much not a syntax today.

I’m leaning towards the first option, as it’s flexible, has simple semantics and doesn’t require new syntax, and I’ll try to write a basic implementation to see how that works. I’d be interested to hear another opinions on the issue though.

I asked about an issue because there’re some design considerations I’d like to discuss (I’ll share them here, but feel free to point me to a more suitable venue). old-style `shell.nix` doesn’t currently accept any arguments, it just returns a derivation (you kinda can pass arguments via `--arg`, but they require default values). that means that adding auto-filling for arguments with no default values is not a breaking change, which is nice. there’re some options as to which arguments get passed though: - flakes from the registry (so basically only `nixpkgs` for most shells). requires the shell to `import nixpkgs` manually every time, but allows e.g. overlays, so is the most flexible. - packages via `callPackage`, as `nix-build` does. very concise and pleasant for simple shells, but less flexible and doesn’t provide a clean way to access other flakes from the registry. - somehow both? placing them into the same namespace feels bad though, name conflicts are bound to happen. something like ```nix { flakes = { someFlake }, hello, mkDerivation }: ... ``` could potentially solve this problem, but that’s very much not a syntax today. I’m leaning towards the first option, as it’s flexible, has simple semantics and doesn’t require new syntax, and I’ll try to write a basic implementation to see how that works. I’d be interested to hear another opinions on the issue though.

pennae referenced this issue

2024-08-06 21:24:17 +00:00

shell.nix for new cli #463

jade modified the milestone from v2.91 to post-release

2024-08-13 01:19:36 +00:00

Ban resolving registry indirect flake inputs #170