lix installer fails in debian 13 with nix-vm-test

nim65s commented

2024-09-27 12:03:56 +00:00

Hi,

I'm trying to use lix in a debian VM with the following:

{
  description = "Debian VM for and from nix";

  inputs.nix-vm-test.url = "github:numtide/nix-vm-test";

  outputs =
    inputs:
    let
      system = "x86_64-linux";
      vmTest = inputs.nix-vm-test.lib.${system}.debian."13" {
        sharedDirs = { };
        testScript = ''
          vm.wait_for_unit("multi-user.target")
          vm.succeed("apt update && apt install -qy curl")
          vm.succeed("curl -sSf -L https://install.lix.systems/lix | sh -s -- install --no-confirm")
        '';
      };
    in
    {
      packages.${system}.default = vmTest.driver;
    };
}

And this fail with:

Error

Error: 
   0: Install failure
   1: Error executing action
   2: Action `provision_nix` errored
   3: Action `move_unpacked_nix` errored
   4: Rename `/nix/temp-install-dir/lix-2.91.0-x86_64-linux/store/lbp0rbrmvsh6nc5r0in8nm8qzr0z0ss0-libkrb5-1.21.3` to `/nix/store/lbp0rbrmvsh6nc5r0in8nm8qzr0z0ss0-libkrb5-1.21.3`
   5: Cross-device link (os error 18)

Metadata

key	value
version	0.17.1
os	linux
arch	x86_64

Same issue with detsys, so sorry in advance if that's only me missing something.

Hi, I'm trying to use lix in a debian VM with the following: ```nix { description = "Debian VM for and from nix"; inputs.nix-vm-test.url = "github:numtide/nix-vm-test"; outputs = inputs: let system = "x86_64-linux"; vmTest = inputs.nix-vm-test.lib.${system}.debian."13" { sharedDirs = { }; testScript = '' vm.wait_for_unit("multi-user.target") vm.succeed("apt update && apt install -qy curl") vm.succeed("curl -sSf -L https://install.lix.systems/lix | sh -s -- install --no-confirm") ''; }; in { packages.${system}.default = vmTest.driver; }; } ``` And this fail with: ## Error ``` Error: 0: Install failure 1: Error executing action 2: Action `provision_nix` errored 3: Action `move_unpacked_nix` errored 4: Rename `/nix/temp-install-dir/lix-2.91.0-x86_64-linux/store/lbp0rbrmvsh6nc5r0in8nm8qzr0z0ss0-libkrb5-1.21.3` to `/nix/store/lbp0rbrmvsh6nc5r0in8nm8qzr0z0ss0-libkrb5-1.21.3` 5: Cross-device link (os error 18) ``` ## Metadata |key|value| |--|--| |**version**|0.17.1| |**os**|linux| |**arch**|x86_64| Same issue with detsys, so sorry in advance if that's only me missing something.

kfearsoff commented

2024-09-28 17:25:05 +00:00

Member

Thanks for the report!

My theory was that /nix/store is on a different filesystem. rename only works for the same filesystem, so it fails with a somewhat cryptic error message. But alas, I ran your repro with lsblk command to list filesystems on the VM and lix-installer with -vv for trace logging and... apparently, it's the same filesystem. What the hell.

The renaming issue should be fairly easy to fix, but I want to spend some more time trying to figure out why it fails to begin with.

Thanks for the report! My theory was that `/nix/store` is on a different filesystem. `rename` only works for the same filesystem, so it fails with a somewhat cryptic error message. But alas, I ran your repro with `lsblk` command to list filesystems on the VM and `lix-installer` with `-vv` for trace logging and... apparently, it's the same filesystem. What the hell. The renaming issue should be fairly easy to fix, but I want to spend some more time trying to figure out why it fails to begin with.

kfearsoff commented

2024-09-28 17:53:33 +00:00

Member

Oh, apparently you need to use findmnt to correctly list mounts. So I managed to figure it out: since nix-vm-test uses NixOS test framework, it still mounts the /nix/store from the host filesystem. So the root filesystem is ext4, and /nix/store is 9p, so those are different filesystems and the rename fails.

The renaming issue should be pretty easy, but I'm not 100% sure if we'll be able to support nix-vm-test use case (or if we want to). I'll talk to people more experienced with NixOS test framework to see if there are any landmines that will cause issues, and test it again once I've fixed renaming.

Repro flake:

{
  description = "Debian VM for and from nix";

  inputs.nix-vm-test.url = "github:numtide/nix-vm-test";

  outputs =
    inputs:
    let
      system = "x86_64-linux";
      vmTest = inputs.nix-vm-test.lib.${system}.debian."13" {
        sharedDirs = { };
        testScript = ''
          vm.wait_for_unit("multi-user.target")
          vm.succeed("apt update && apt install -qy curl")
          vm.succeed("findmnt 1>&2")
          vm.fail("curl -sSf -L https://install.lix.systems/lix | sh -s -- install -vv --no-confirm")
        '';
      };
    in
    {
      packages.${system}.default = vmTest.driver;
    };
}

Logs (note / being ext4, /nix/.ro-store being 9p, /nix/.rw-store being tmpfs and /nix/store being overlay: all of this is expected with how NixOS framework is designed to be used):

vm: must succeed: findmnt 1>&2
vm # TARGET                       SOURCE   FSTYPE  OPTIONS
vm # /                            /dev/sda1
vm # │                                     ext4    rw,relatime,discard,errors=remount
vm # ├─/sys                       sysfs    sysfs   rw,nosuid,nodev,noexec,relatime
vm # │ ├─/sys/kernel/security     securityfs
vm # │ │                                   securit rw,nosuid,nodev,noexec,relatime
vm # │ ├─/sys/fs/cgroup           cgroup2  cgroup2 rw,nosuid,nodev,noexec,relatime,ns
vm # │ ├─/sys/fs/pstore           pstore   pstore  rw,nosuid,nodev,noexec,relatime
vm # │ ├─/sys/fs/bpf              bpf      bpf     rw,nosuid,nodev,noexec,relatime,mo
vm # │ ├─/sys/kernel/debug        none     debugfs rw,nosuid,nodev,noexec,relatime
vm # │ ├─/sys/kernel/tracing      tracefs  tracefs rw,nosuid,nodev,noexec,relatime
vm # │ ├─/sys/fs/fuse/connections fusectl  fusectl rw,nosuid,nodev,noexec,relatime
vm # │ └─/sys/kernel/config       configfs configf rw,nosuid,nodev,noexec,relatime
vm # ├─/proc                      proc     proc    rw,nosuid,nodev,noexec,relatime
vm # │ └─/proc/sys/fs/binfmt_misc systemd-1
vm # │   │                                 autofs  rw,relatime,fd=39,pgrp=1,timeout=0
vm # │   └─/proc/sys/fs/binfmt_misc
vm # │                            binfmt_misc
vm # │                                     binfmt_ rw,nosuid,nodev,noexec,relatime
vm # ├─/dev                       udev     devtmpf rw,nosuid,relatime,size=474656k,nr
vm # │ ├─/dev/pts                 devpts   devpts  rw,nosuid,noexec,relatime,gid=5,mo
vm # │ ├─/dev/shm                 tmpfs    tmpfs   rw,nosuid,nodev,inode64
vm # │ ├─/dev/mqueue              mqueue   mqueue  rw,nosuid,nodev,noexec,relatime
vm # │ └─/dev/hugepages           hugetlbfs
vm # │                                     hugetlb rw,nosuid,nodev,relatime,pagesize=
vm # ├─/run                       tmpfs    tmpfs   rw,nosuid,nodev,noexec,relatime,si
vm # │ ├─/run/lock                tmpfs    tmpfs   rw,nosuid,nodev,noexec,relatime,si
vm # │ ├─/run/credentials/systemd-journald.service
vm # │ │                          tmpfs    tmpfs   ro,nosuid,nodev,noexec,relatime,no
vm # │ ├─/run/credentials/systemd-network-generator.service
vm # │ │                          tmpfs    tmpfs   ro,nosuid,nodev,noexec,relatime,no
vm # │ ├─/run/credentials/systemd-udev-load-credentials.service
vm # │ │                          tmpfs    tmpfs   ro,nosuid,nodev,noexec,relatime,no
vm # │ ├─/run/credentials/systemd-sysctl.service
vm # │ │                          tmpfs    tmpfs   ro,nosuid,nodev,noexec,relatime,no
vm # │ ├─/run/credentials/systemd-tmpfiles-setup-dev-early.service
vm # │ │                          tmpfs    tmpfs   ro,nosuid,nodev,noexec,relatime,no
vm # │ ├─/run/credentials/systemd-sysusers.service
vm # │ │                          tmpfs    tmpfs   ro,nosuid,nodev,noexec,relatime,no
vm # │ ├─/run/credentials/systemd-tmpfiles-setup-dev.service
vm # │ │                          tmpfs    tmpfs   ro,nosuid,nodev,noexec,relatime,no
vm # │ ├─/run/credentials/systemd-networkd.service
vm # │ │                          tmpfs    tmpfs   ro,nosuid,nodev,noexec,relatime,no
vm # │ ├─/run/credentials/systemd-tmpfiles-setup.service
vm # │ │                          tmpfs    tmpfs   ro,nosuid,nodev,noexec,relatime,no
vm # │ ├─/run/credentials/systemd-resolved.service
vm # │ │                          tmpfs    tmpfs   ro,nosuid,nodev,noexec,relatime,no
vm # │ └─/run/credentials/getty@tty1.service
vm # │                            tmpfs    tmpfs   ro,nosuid,nodev,noexec,relatime,no
vm # ├─/tmp                       tmpfs    tmpfs   rw,nosuid,nodev,size=495516k,nr_in
vm # ├─/boot/efi                  /dev/sda15
vm # │                                     vfat    rw,relatime,fmask=0022,dmask=0022,
vm # ├─/nix/.ro-store             nix-store
vm # │                                     9p      rw,relatime,cache=f,access=client,
vm # ├─/nix/.rw-store             tmpfs    tmpfs   rw,relatime,inode64
vm # └─/nix/store                 overlay  overlay rw,relatime,lowerdir=/nix/.ro-stor
(finished: must succeed: findmnt 1>&2, in 0.05 seconds)

Oh, apparently you need to use `findmnt` to correctly list mounts. So I managed to figure it out: since `nix-vm-test` uses NixOS test framework, it still mounts the `/nix/store` from the host filesystem. So the root filesystem is `ext4`, and `/nix/store` is `9p`, so those are different filesystems and the rename fails. The renaming issue should be pretty easy, but I'm not 100% sure if we'll be able to support `nix-vm-test` use case (or if we want to). I'll talk to people more experienced with NixOS test framework to see if there are any landmines that will cause issues, and test it again once I've fixed renaming. Repro flake: ```nix { description = "Debian VM for and from nix"; inputs.nix-vm-test.url = "github:numtide/nix-vm-test"; outputs = inputs: let system = "x86_64-linux"; vmTest = inputs.nix-vm-test.lib.${system}.debian."13" { sharedDirs = { }; testScript = '' vm.wait_for_unit("multi-user.target") vm.succeed("apt update && apt install -qy curl") vm.succeed("findmnt 1>&2") vm.fail("curl -sSf -L https://install.lix.systems/lix | sh -s -- install -vv --no-confirm") ''; }; in { packages.${system}.default = vmTest.driver; }; } ``` Logs (note `/` being `ext4`, `/nix/.ro-store` being `9p`, `/nix/.rw-store` being `tmpfs` and `/nix/store` being `overlay`: all of this is expected with how NixOS framework is designed to be used): ``` vm: must succeed: findmnt 1>&2 vm # TARGET SOURCE FSTYPE OPTIONS vm # / /dev/sda1 vm # │ ext4 rw,relatime,discard,errors=remount vm # ├─/sys sysfs sysfs rw,nosuid,nodev,noexec,relatime vm # │ ├─/sys/kernel/security securityfs vm # │ │ securit rw,nosuid,nodev,noexec,relatime vm # │ ├─/sys/fs/cgroup cgroup2 cgroup2 rw,nosuid,nodev,noexec,relatime,ns vm # │ ├─/sys/fs/pstore pstore pstore rw,nosuid,nodev,noexec,relatime vm # │ ├─/sys/fs/bpf bpf bpf rw,nosuid,nodev,noexec,relatime,mo vm # │ ├─/sys/kernel/debug none debugfs rw,nosuid,nodev,noexec,relatime vm # │ ├─/sys/kernel/tracing tracefs tracefs rw,nosuid,nodev,noexec,relatime vm # │ ├─/sys/fs/fuse/connections fusectl fusectl rw,nosuid,nodev,noexec,relatime vm # │ └─/sys/kernel/config configfs configf rw,nosuid,nodev,noexec,relatime vm # ├─/proc proc proc rw,nosuid,nodev,noexec,relatime vm # │ └─/proc/sys/fs/binfmt_misc systemd-1 vm # │ │ autofs rw,relatime,fd=39,pgrp=1,timeout=0 vm # │ └─/proc/sys/fs/binfmt_misc vm # │ binfmt_misc vm # │ binfmt_ rw,nosuid,nodev,noexec,relatime vm # ├─/dev udev devtmpf rw,nosuid,relatime,size=474656k,nr vm # │ ├─/dev/pts devpts devpts rw,nosuid,noexec,relatime,gid=5,mo vm # │ ├─/dev/shm tmpfs tmpfs rw,nosuid,nodev,inode64 vm # │ ├─/dev/mqueue mqueue mqueue rw,nosuid,nodev,noexec,relatime vm # │ └─/dev/hugepages hugetlbfs vm # │ hugetlb rw,nosuid,nodev,relatime,pagesize= vm # ├─/run tmpfs tmpfs rw,nosuid,nodev,noexec,relatime,si vm # │ ├─/run/lock tmpfs tmpfs rw,nosuid,nodev,noexec,relatime,si vm # │ ├─/run/credentials/systemd-journald.service vm # │ │ tmpfs tmpfs ro,nosuid,nodev,noexec,relatime,no vm # │ ├─/run/credentials/systemd-network-generator.service vm # │ │ tmpfs tmpfs ro,nosuid,nodev,noexec,relatime,no vm # │ ├─/run/credentials/systemd-udev-load-credentials.service vm # │ │ tmpfs tmpfs ro,nosuid,nodev,noexec,relatime,no vm # │ ├─/run/credentials/systemd-sysctl.service vm # │ │ tmpfs tmpfs ro,nosuid,nodev,noexec,relatime,no vm # │ ├─/run/credentials/systemd-tmpfiles-setup-dev-early.service vm # │ │ tmpfs tmpfs ro,nosuid,nodev,noexec,relatime,no vm # │ ├─/run/credentials/systemd-sysusers.service vm # │ │ tmpfs tmpfs ro,nosuid,nodev,noexec,relatime,no vm # │ ├─/run/credentials/systemd-tmpfiles-setup-dev.service vm # │ │ tmpfs tmpfs ro,nosuid,nodev,noexec,relatime,no vm # │ ├─/run/credentials/systemd-networkd.service vm # │ │ tmpfs tmpfs ro,nosuid,nodev,noexec,relatime,no vm # │ ├─/run/credentials/systemd-tmpfiles-setup.service vm # │ │ tmpfs tmpfs ro,nosuid,nodev,noexec,relatime,no vm # │ ├─/run/credentials/systemd-resolved.service vm # │ │ tmpfs tmpfs ro,nosuid,nodev,noexec,relatime,no vm # │ └─/run/credentials/getty@tty1.service vm # │ tmpfs tmpfs ro,nosuid,nodev,noexec,relatime,no vm # ├─/tmp tmpfs tmpfs rw,nosuid,nodev,size=495516k,nr_in vm # ├─/boot/efi /dev/sda15 vm # │ vfat rw,relatime,fmask=0022,dmask=0022, vm # ├─/nix/.ro-store nix-store vm # │ 9p rw,relatime,cache=f,access=client, vm # ├─/nix/.rw-store tmpfs tmpfs rw,relatime,inode64 vm # └─/nix/store overlay overlay rw,relatime,lowerdir=/nix/.ro-stor (finished: must succeed: findmnt 1>&2, in 0.05 seconds) ```

👍 1

lix-bot commented

2024-09-28 20:16:45 +00:00

Member

This issue was mentioned on Gerrit on the following CLs:

commit message in cl/1993 ("fix: rename logic")

This issue was mentioned on Gerrit on the following CLs: * commit message in [cl/1993](https://gerrit.lix.systems/c/lix-installer/+/1993) ("fix: rename logic")

lix installer fails in debian 13 with nix-vm-test #27

Error

Metadata