[Nix#8907] Ran out of disk space building NixOS 23.05 on 22.11 and corrupted 178 store paths #118

New issue

Open

opened 2024-03-16 06:44:59 +00:00 by lix-bot · 0 comments

lix-bot commented

2024-03-16 06:44:59 +00:00

Member

Upstream-Issue: NixOS/nix#8907

Describe the bug

Was running 22.11. Updated system config for 23.05 and did a nixos-rebuild boot. System ran out of disk space part way through. Freed more disk space and reissued nixos-rebuild boot. It completed. Booting failed.

Eventually tracked the issue down to ...-util-linux-2.38.1-bin having a butch a zero length executables under bin. Ran nix store verify --all and discovered there were 178 corrupted store paths including ...-util-linux-2.38.1-bin. Presumably the others also had zero length files, but didn't check to make sure.

Fixed these all with nix store repair .... System boots fine now. Would seem that the nix store path unpacker did not handle running out of disk space correctly. In case it is relevant, my root filesystem is btrfs with zstd compression.

  fileSystems."/" =
    { device = "/dev/disk/by-uuid/9fbc9b87-0168-4675-98f4-256f22c2d6fc";
      fsType = "btrfs";
      options = [ "subvol=root" "compress=zstd" ];
    };

Steps To Reproduce

Presumably install a package under 22.11 when you don't have enough disk space for it? Possibly you need a filesystem with compression enabled (all though I can't imagine why that would be an issue as that generally has no effect beyond it underestimating free space). Not sure if this is an issue under the newer nix in 23.05.

Expected behavior

If space is exhausted when unpacking a store path, it should not be put it in place with a bunch of the files zeroed.

nix-env --version output

Running this on the old 22.11 system link gives

nix-env (Nix) 2.11.1

Upstream-Issue: https://git.lix.systems/NixOS/nix/issues/8907 **Describe the bug** Was running 22.11. Updated system config for 23.05 and did a `nixos-rebuild boot`. System ran out of disk space part way through. Freed more disk space and reissued `nixos-rebuild boot`. It completed. Booting failed. Eventually tracked the issue down to `...-util-linux-2.38.1-bin` having a butch a zero length executables under `bin`. Ran `nix store verify --all` and discovered there were 178 corrupted store paths including `...-util-linux-2.38.1-bin`. Presumably the others also had zero length files, but didn't check to make sure. Fixed these all with `nix store repair ...`. System boots fine now. Would seem that the nix store path unpacker did not handle running out of disk space correctly. In case it is relevant, my root filesystem is btrfs with zstd compression. ``` fileSystems."/" = { device = "/dev/disk/by-uuid/9fbc9b87-0168-4675-98f4-256f22c2d6fc"; fsType = "btrfs"; options = [ "subvol=root" "compress=zstd" ]; }; ``` **Steps To Reproduce** Presumably install a package under 22.11 when you don't have enough disk space for it? Possibly you need a filesystem with compression enabled (all though I can't imagine why that would be an issue as that generally has no effect beyond it underestimating free space). Not sure if this is an issue under the newer nix in 23.05. **Expected behavior** If space is exhausted when unpacking a store path, it should not be put it in place with a bunch of the files zeroed. **`nix-env --version` output** Running this on the old 22.11 system link gives ``` nix-env (Nix) 2.11.1 ```