feat: allow optional moving to nix store

yajo commented

2022-03-31 10:58:29 +00:00

(Migrated from github.com)

According to https://github.com/NixOS/nix/issues/3121#issuecomment-1050510805, flake-compat should be that issue's workaround.

However, it tries to move src to the nix store always. That is a sane default, as it imitates closer what flakes do. However, it does not let this tool to be used comfortably as a workaround for that issue.

With this change, the default behaviour is preserved. However, the user can pass moveSrcToStore = false to avoid it.

@moduon MT-83

According to https://github.com/NixOS/nix/issues/3121#issuecomment-1050510805, flake-compat should be that issue's workaround. However, it tries to move src to the nix store always. That is a sane default, as it imitates closer what flakes do. However, it does not let this tool to be used comfortably as a workaround for that issue. With this change, the default behaviour is preserved. However, the user can pass `moveSrcToStore = false` to avoid it. @moduon MT-83

👍 3

edolstra commented

2022-04-14 08:47:43 +00:00

(Migrated from github.com)

flake-compat is intended to emulate the regular flake interface, so I'd prefer not to add functionality that isn't provided by flakes (like this moveSrcToStore flag). The problem is that people may come to depend on this extended interface, which would prevent them from switching to the new CLI.

`flake-compat` is intended to emulate the regular flake interface, so I'd prefer not to add functionality that isn't provided by flakes (like this `moveSrcToStore` flag). The problem is that people may come to depend on this extended interface, which would prevent them from switching to the new CLI.

lovesegfault commented

2022-04-14 09:48:12 +00:00

(Migrated from github.com)

@edolstra That's totally fair IMO, but what is the solution here for people who want to use Flakes but have to deal with large repos?

It's really hard to sell Nix to my team if it means their disk is filled after a handful of builds.

I suppose I could go back to the "traditional" approach of using something like niv and ignoring flakes altogether, but that feels like a huge sacrifice. Do you have any better suggestions?

@edolstra That's totally fair IMO, but what is the solution here for people who want to use Flakes but have to deal with large repos? It's really hard to sell Nix to my team if it means their disk is filled after a handful of builds. I suppose I could go back to the "traditional" approach of using something like `niv` and ignoring flakes altogether, but that feels like a huge sacrifice. Do you have any better suggestions?

yajo commented

2022-04-14 10:30:42 +00:00

(Migrated from github.com)

I've been testing these days by setting auto-optimise-store = true in nix.conf. I don't have measures to tell if it does the trick, but so far it seems to do it.

I've been testing these days by setting `auto-optimise-store = true` in `nix.conf`. I don't have measures to tell if it does the trick, but so far it seems to do it.

edolstra commented

2022-04-14 10:58:39 +00:00

(Migrated from github.com)

@lovesegfault This will be addressed by https://github.com/NixOS/nix/issues/3121 (copying flakes to the store lazily). It will probably be hard to replicate that functionality in flake-compat though.

lovesegfault commented

2022-04-14 17:56:24 +00:00

(Migrated from github.com)

@lovesegfault This will be addressed by NixOS/nix#3121 (copying flakes to the store lazily). It will probably be hard to replicate that functionality in flake-compat though.

That's a three-year-old issue, though. Do you have any expectation of when that could get done?

Just trying to figure out which direction to go.

> @lovesegfault This will be addressed by [NixOS/nix#3121](https://github.com/NixOS/nix/issues/3121) (copying flakes to the store lazily). It will probably be hard to replicate that functionality in flake-compat though. That's a three-year-old issue, though. Do you have any expectation of when that could get done? Just trying to figure out which direction to go.

edolstra commented

2022-04-14 18:50:42 +00:00

(Migrated from github.com)

Hopefully in Nix 2.9, see https://github.com/edolstra/nix/tree/lazy-trees.

👍 4

sigprof commented

2022-08-10 09:33:51 +00:00

(Migrated from github.com)

One possible workaround is replacing { src = ./.; } with { src = { outPath = ./.;}; } in shell.nix — this disables source tree cleaning without adding an extra argument for that. Of course, this also disables the protection against referencing untracked files, but this might be acceptable for a compatibility mode.

Another workaround which is safer against accidentally referring to untracked files at the expense of needing to maintain an explicit list of files used by the flake (this might be acceptable if the flake is used only to provide the development environment):

let
  flakeManifest = [
    ./flake.lock
    ./flake.nix
  ];

  lock = builtins.fromJSON (builtins.readFile ./flake.lock);
  flake-compat = fetchTarball {
    url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
    sha256 = lock.nodes.flake-compat.locked.narHash;
  };

  src = builtins.path {
    path = ./.;
    name = "source";
    filter = path: type: builtins.any (x: (toString x) == path) flakeManifest;
  };
in
  (import flake-compat {inherit src;}).shellNix.default

This obviously still copies files listed in flakeManifest to the store, but at least that's not the whole repo. Although once you need to do something like nix flake update, you would again end up with the whole thing in the store until lazy trees are implemented.

One possible workaround is replacing `{ src = ./.; }` with `{ src = { outPath = ./.;}; }` in `shell.nix` — this disables source tree cleaning without adding an extra argument for that. Of course, this also disables the protection against referencing untracked files, but this might be acceptable for a compatibility mode. Another workaround which is safer against accidentally referring to untracked files at the expense of needing to maintain an explicit list of files used by the flake (this might be acceptable if the flake is used only to provide the development environment): ```nix let flakeManifest = [ ./flake.lock ./flake.nix ]; lock = builtins.fromJSON (builtins.readFile ./flake.lock); flake-compat = fetchTarball { url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz"; sha256 = lock.nodes.flake-compat.locked.narHash; }; src = builtins.path { path = ./.; name = "source"; filter = path: type: builtins.any (x: (toString x) == path) flakeManifest; }; in (import flake-compat {inherit src;}).shellNix.default ``` This obviously still copies files listed in `flakeManifest` to the store, but at least that's not the whole repo. Although once you need to do something like `nix flake update`, you would again end up with the whole thing in the store until lazy trees are implemented.

yajo commented

2023-02-02 12:36:47 +00:00

(Migrated from github.com)

Closing, as this will obviously not be merged, and the comments explained viable alternatives. Thanks everyone!

Pull request closed

This pull request cannot be reopened because the branch was deleted.

feat: allow optional moving to nix store #39

Pull request closed