Change nixpgks overlay so I do not have to enumerate all the packages that are included

dwt commented

2025-09-06 14:10:03 +00:00

When I hardcode the nixpkgs packages that I want to get from the overlay, it means I have to check with each upgrade that the list is up to date.

With that I mean this code:

{ pkgs, ... }:
{
  nixpkgs.overlays = [ (final: prev: {
    inherit (final.lixPackageSets.stable)
      nixpkgs-review
      nix-direnv
      nix-eval-jobs
      nix-fast-build
      colmena;
  }) ];

  nix.package = pkgs.lixPackageSets.stable.lix;
}

Replicating these packages: nixpkgs-review, nix-direnv, nix-eval-jobs, nix-fast-build, colmena; in every nix configuration is what I do not like.

Describe the solution you'd like

Would be much nicer to the current way for power users who need only specific packages, and default to something that takes all packages from the overlay.

Something like

nixpkgs.overlays = [
  pkgs.lixPackageSets.stable.overlay
];

Describe alternatives you've considered

The existing approach is nice if you want just some of the packages, but I have a hard time coming up with use cases where you want the original package in some cases.

Additional context

…

## Is your feature request related to a problem? Please describe. When I hardcode the nixpkgs packages that I want to get from the overlay, it means I have to check with each upgrade that the list is up to date. With that I mean this code: ```nix { pkgs, ... }: { nixpkgs.overlays = [ (final: prev: { inherit (final.lixPackageSets.stable) nixpkgs-review nix-direnv nix-eval-jobs nix-fast-build colmena; }) ]; nix.package = pkgs.lixPackageSets.stable.lix; } ``` Replicating these packages: `nixpkgs-review`, `nix-direnv`, `nix-eval-jobs`, `nix-fast-build`, `colmena;` in every nix configuration is what I do not like. ## Describe the solution you'd like Would be much nicer to the current way for power users who need only specific packages, and default to something that takes all packages from the overlay. Something like ```nix nixpkgs.overlays = [ pkgs.lixPackageSets.stable.overlay ]; ``` ## Describe alternatives you've considered The existing approach is nice if you want just some of the packages, but I have a hard time coming up with use cases where you want the original package in some cases. ## Additional context …

dwt commented

2025-09-06 14:12:54 +00:00

Author

This might need to be a bug report in nixpkgs now that I write it here, just send me there if I'm wrong here.

pennae commented

2025-09-06 14:14:53 +00:00

Owner

yeah, that's a nixpkgs thing more than a lix repo thing. does sound like a useful thing though!

jade commented

2025-09-06 16:53:52 +00:00

Owner

this is intended to be fixed e.g. with config.nix.implementation while importing nixpkgs. it's just not gotten done due to resource constraints and needing to stop the bleeding with respect to reactive work. it'll happen eventually in due course.

dwt commented

2025-09-07 10:33:07 +00:00

Author

@jade If you can point me to a pull request for nixpkgs where this is worked on, or give some general guidance how to proceed, I would like to help.

raito commented

2025-09-08 14:41:28 +00:00

Owner

@dwt There's no such pull request, if you are interested, I can jump in a call to explain you the strategy or we can discuss it in a public channel.

The idea is to enable import <nixpkgs> { config.nix.implementation = "lix"; } and have a package set that uses Lix for everything.

@dwt There's no such pull request, if you are interested, I can jump in a call to explain you the strategy or we can discuss it in a public channel. The idea is to enable `import <nixpkgs> { config.nix.implementation = "lix"; }` and have a package set that uses Lix for everything.

jade added the

Area/build-packaging

label

2025-09-11 17:33:34 +00:00

yu-re-ka commented

2025-09-11 18:33:44 +00:00

Member

@raito do you think adding more options to nixpkgs.config is the way to go?

For me, this sounds like a recipe for failure just like all the other nixpkgs.config options are not well-maintained / well-tested.

A much better approach seems to be linuxPackages / pythonPackages, where we have one shared packagesFor (which would be shared among nix and lix) and don't even expose colmena / nixpkgs-review / ... under the top-level/all-packages.nix anymore, thus forcing the user to choose explicitly. This also avoids any issues caused by having multiple possible interpretations of nixpkgs, which has never really been a thing for upstream nixpkgs features.

Instead, I suggest managing this on a NixOS/nix-darwin level: A NixOS module can abstract on top of that to choose a Nix/Lix packagesSet and use the packages from there:

{ config, ... }:

{
  nix.packageSet = pkgs.lixPackageSets.latest;

  environment.systemPackages = [
    config.nix.packageSet.colmena
  ];
}

@raito do you think adding more options to nixpkgs.config is the way to go? For me, this sounds like a recipe for failure just like all the other nixpkgs.config options are not well-maintained / well-tested. A much better approach seems to be linuxPackages / pythonPackages, where we have one shared packagesFor (which would be shared among nix and lix) and don't even expose colmena / nixpkgs-review / ... under the top-level/all-packages.nix anymore, thus forcing the user to choose explicitly. This also avoids any issues caused by having multiple possible interpretations of nixpkgs, which has never really been a thing for upstream nixpkgs features. Instead, I suggest managing this on a NixOS/nix-darwin level: A NixOS module can abstract on top of that to choose a Nix/Lix packagesSet and use the packages from there: ``` { config, ... }: { nix.packageSet = pkgs.lixPackageSets.latest; environment.systemPackages = [ config.nix.packageSet.colmena ]; } ```

raito commented

2025-09-11 18:52:04 +00:00

Owner

@yu-re-ka wrote in #989 (comment):

@raito do you think adding more options to nixpkgs.config is the way to go?

For me, this sounds like a recipe for failure just like all the other nixpkgs.config options are not well-maintained / well-tested.

A much better approach seems to be linuxPackages / pythonPackages, where we have one shared packagesFor (which would be shared among nix and lix) and don't even expose colmena / nixpkgs-review / ... under the top-level/all-packages.nix anymore, thus forcing the user to choose explicitly. This also avoids any issues caused by having multiple possible interpretations of nixpkgs, which has never really been a thing for upstream nixpkgs features.

Instead, I suggest managing this on a NixOS/nix-darwin level: A NixOS module can abstract on top of that to choose a Nix/Lix packagesSet and use the packages from there:
{ config, ... }:

{
  nix.packageSet = pkgs.lixPackageSets.latest;

  environment.systemPackages = [
    config.nix.packageSet.colmena
  ];
}

We can do that but this means that each implementation of an OS needs to manage their glue code vs. a single entrypoint that works automatically for all OS implementations.

I say that because I am a user of non-mainstream OS implementations that are neither NixOS neither nix-darwin.

Lix is already using the linuxPackages system BTW. All that is missing is the Lix module in NixOS.

We can start there, but I surmise that the destination may be config.nix.implementation.

@yu-re-ka wrote in https://git.lix.systems/lix-project/lix/issues/989#issuecomment-14902: > @raito do you think adding more options to nixpkgs.config is the way to go? > > For me, this sounds like a recipe for failure just like all the other nixpkgs.config options are not well-maintained / well-tested. > > A much better approach seems to be linuxPackages / pythonPackages, where we have one shared packagesFor (which would be shared among nix and lix) and don't even expose colmena / nixpkgs-review / ... under the top-level/all-packages.nix anymore, thus forcing the user to choose explicitly. This also avoids any issues caused by having multiple possible interpretations of nixpkgs, which has never really been a thing for upstream nixpkgs features. > > Instead, I suggest managing this on a NixOS/nix-darwin level: A NixOS module can abstract on top of that to choose a Nix/Lix packagesSet and use the packages from there: > > ```text > { config, ... }: > > { > nix.packageSet = pkgs.lixPackageSets.latest; > > environment.systemPackages = [ > config.nix.packageSet.colmena > ]; > } > ``` We can do that but this means that each implementation of an OS needs to manage their glue code vs. a single entrypoint that works automatically for *all* OS implementations. I say that because I am a user of non-mainstream OS implementations that are neither NixOS neither nix-darwin. Lix is already using the linuxPackages system BTW. All that is missing is the Lix module in NixOS. We can start there, but I surmise that the destination may be `config.nix.implementation`.

raito commented

2025-09-11 18:52:54 +00:00

Owner

Also, I do not want to ask CppNix upstream to standardize on anything of the sort, my spoons are already spent for any standardizing efforts we tried in the past with them. So, this would be a Lix exclusive.

dwt commented

2025-09-12 13:52:41 +00:00

Author

@raito Just to understand this better, what would setting config.nix.implemenation actually do?

My understanding of @yu-re-ka is currently, that the packageset is required so that hydra actually builds these variants of the packages. (We already do that) but now to activate that, what would that setting actually change? I understood @yu-re-ka that nixpkgs usually has no stomach to accept an overlay in nixpkgs ... so what would be upstream able in nixpkgs that allows users to say pkgs.colmena to actually get pkgs.lixPackages.latest.colmena?

@raito Just to understand this better, what would setting `config.nix.implemenation` actually do? My understanding of @yu-re-ka is currently, that the packageset is required so that hydra actually builds these variants of the packages. (We already do that) but now to activate that, what would that setting actually change? I understood @yu-re-ka that nixpkgs usually has no stomach to accept an overlay in nixpkgs ... so what would be upstream able in nixpkgs that allows users to say `pkgs.colmena` to actually get `pkgs.lixPackages.latest.colmena`?

raito commented

2025-09-12 16:23:07 +00:00

Owner

config.nix.implementation would offer a flag across all of nixpkgs so that the top-level could inject pkgs.lixPackageSets.stable.lix as nix by default instead of nix by default.

So that pkgs.colmena would use Lix by default.

`config.nix.implementation` would offer a flag across all of nixpkgs so that the top-level could inject `pkgs.lixPackageSets.stable.lix` as `nix` by default instead of `nix` by default. So that `pkgs.colmena` would use Lix by default.

dwt commented

2025-09-12 17:57:05 +00:00

Author

How does that work to maintain the binary caches ability to function? Does that only require that the package was built as pkgs.lixPackageSets.stable.colmena via hydra?

And where in nixpkgs would that be set? Do you have an example?

How does that work to maintain the binary caches ability to function? Does that only require that the package was built as `pkgs.lixPackageSets.stable.colmena` via hydra? And where in nixpkgs would that be set? Do you have an example?

dwt commented

2025-09-16 06:18:55 +00:00

Author

@raito @yu-re-ka Thinking about this a bit further, the plans you outline seem like they require quite a bit of know how in nixpkgs and perhaps even quite a bit clout in the nixpkgs project to get merged.

Would you oppose some incremental improvement? I am thinking of refining the package set, so it is possible to easily apply all packages within it as an overlay yourself. Something like:

overlay = final: prev: final.lixPackageSet.latest

To my understanding that would require modifying the current set to get rid of the lix package inside it, as well as the storeDir,
stateDir and confDir variable.

@raito @yu-re-ka Thinking about this a bit further, the plans you outline seem like they require quite a bit of know how in nixpkgs and perhaps even quite a bit clout in the nixpkgs project to get merged. Would you oppose some incremental improvement? I am thinking of refining the package set, so it is possible to easily apply all packages within it as an overlay yourself. Something like: ```nix overlay = final: prev: final.lixPackageSet.latest ``` To my understanding that would require modifying the current set to get rid of the `lix` package inside it, as well as the `storeDir`, `stateDir` and `confDir` variable.

pennae commented

2025-09-16 10:52:36 +00:00

Owner

that is squarely a nixpkgs problem at this point, maybe it should be discussed there. we may also want to move this issue to the nixos-module repo rather than keeping it here

raito referenced this issue from lix-project/nixos-module

2025-09-21 10:26:52 +00:00

Have a nixpkgs overlay to avoid enumerating packages #94

raito commented

2025-09-21 10:27:11 +00:00

Owner

Forgejo doesn't support issue transfers, closing for now, let's continue the discussion in lix-project/nixos-module#94.

Forgejo doesn't support issue transfers, closing for now, let's continue the discussion in https://git.lix.systems/lix-project/nixos-module/issues/94.

raito closed this issue

2025-09-21 10:27:11 +00:00

raito referenced this issue from lix-project/nixos-module

2025-09-21 10:28:20 +00:00

Have a nixpkgs overlay to avoid enumerating packages #94

raito referenced this issue from lix-project/nixos-module

2025-09-23 15:03:14 +00:00

Have a nixpkgs overlay to avoid enumerating packages #94