lix-project/lix

Fork 58

macOS: tarball unpacking is extremely slow on some machines (2-5 mins to unpack Nixpkgs) #1072

New issue

Open

opened 2025-12-10 21:38:03 +00:00 by jade · 5 comments

jade commented

2025-12-10 21:38:03 +00:00

Owner

Discussion: https://zulip.lix.systems/#narrow/channel/27-Darwin/topic/slow.20archive.20extraction.20in.20macOS.2026

Repro: install Lix 2.94 using the installer with flakes enabled. nix-instantiate --eval --expr '<nixpkgs>'. This will get stuck unpacking a tarball for two minutes (!!!!!!).

Alternate repro: cd "$(mktemp -d)" && nix flake init && time nix flake lock --debug.

We intend to work on fixing this in the next week.

I don't know why this is happening; it's not a regression on our end as it happens on ancient nixes and also on lixes that have at one point not had the issue. It seems to happen on macOS 15.6.1 and 15.7.2 and also 26. GitHub tarball change? I don't know.

Seems to happen on my machine also. It seems to happen on machines that contain Enterprise Nonsense (SentinelOne EDR) and also VMs that don't.

~ » time nix-prefetch-url --unpack https://channels.nixos.org/nixos-25.05/nixexprs.tar.xz
path is '/nix/store/s5bdcwl39qavk84lg05q0y6s34qwpbsz-nixexprs.tar.xz'
1dji447hls0cm4531v7x16akf027843g78smrsvrn8c1y2zcn3jf
nix-prefetch-url --unpack   5.93s user 24.22s system 16% cpu 3:01.14 total

~ »·time nix-prefetch-url --unpack https://channels.nixos.org/nixos-24.05/nixexprs.tar.xz
path is '/nix/store/ix1vzmmzzj91r4gwy4xa15gjvh4914gi-nixexprs.tar.xz'
1f8j7fh0nl4qmqlxn6lis8zf7dnckm6jri4rwmj0qm1qivhr58lv
nix-prefetch-url --unpack   4.89s user 25.86s system 18% cpu 2:46.79 total

configs/nix » time nix flake update lix lix-module nixpkgs
warning: updating lock file '/Users/jade/.dotfiles/configs/nix/flake.lock':
• Updated input 'lix':
    'https://git.lix.systems/api/v1/repos/lix-project/lix/archive/6da351929744863686bf81771242d796653d018c.tar.gz?narHash=sha256-MLAOvbsXOMRhBf51309kLOzvNJKDOcwFVL7D%2BGEKIWk%3D&rev=6da351929744863686bf81771242d796653d018c' (2025-11-24)
  → 'https://git.lix.systems/api/v1/repos/lix-project/lix/archive/36a61d922c51a16b57a88f44833960bf32c92d39.tar.gz?narHash=sha256-IMbgXXV7%2BzcRIG7GDERX%2B9vNIlgE%2BZs1gI2bfy96EO8%3D&rev=36a61d922c51a16b57a88f44833960bf32c92d39' (2025-12-09)
• Updated input 'lix-module':
    'https://git.lix.systems/api/v1/repos/lix-project/nixos-module/archive/192c92b603731fbc1bade6c1b18c8d8a0086f703.tar.gz?narHash=sha256-i2467FddWfd19q5Qoj%2B1/BAeg6LZmM5m4mYGRSQn/as%3D&rev=192c92b603731fbc1bade6c1b18c8d8a0086f703' (2025-11-18)
  → 'https://git.lix.systems/api/v1/repos/lix-project/nixos-module/archive/6c95c0b6f73f831226453fc6905c216ab634c30f.tar.gz?narHash=sha256-XnNABKfIYKSimQVvKc9FnlC2H0LurOhd9MS6l0Z67lE%3D&rev=6c95c0b6f73f831226453fc6905c216ab634c30f' (2025-11-30)
• Updated input 'nixpkgs':
    'https://releases.nixos.org/nixos/unstable-small/nixos-25.11pre901556.146bbcdf0a08/nixexprs.tar.xz?lastModified=1763985748&narHash=sha256-Mzkwmnc351sV9q%2BJUE/Fv9q4Gz1qer7OC0m8G%2B6Mv9I%3D&rev=146bbcdf0a0829fa8e39bacac942f3f2ee210b03' (2025-11-24)
  → 'https://releases.nixos.org/nixos/unstable-small/nixos-26.05pre909797.004943ed3cf9/nixexprs.tar.xz?lastModified=1765357816&narHash=sha256-wFcIt2NuDVi9Ux9hIL1SN/2GKbU%2BxdSwSyfFkQx/P9c%3D&rev=004943ed3cf9de5805a0da377599d1bfdd47a98a' (2025-12-10)
nix flake update lix lix-module nixpkgs  5.11s user 28.61s system 15% cpu 3:43.38 total

this is nuts. what is going on here. nix (Lix, like Nix) 2.95.0-pre20251124-dev_6da3519 but it's also on 2.93 and older versions. It seems to have started happening in the last ~month?

Discussion: https://zulip.lix.systems/#narrow/channel/27-Darwin/topic/slow.20archive.20extraction.20in.20macOS.2026 Repro: install Lix 2.94 using the installer with flakes enabled. `nix-instantiate --eval --expr '<nixpkgs>'`. This will get stuck unpacking a tarball for two minutes (!!!!!!). Alternate repro: `cd "$(mktemp -d)" && nix flake init && time nix flake lock --debug`. We intend to work on fixing this in the next week. I don't know why this is happening; it's not a regression on our end as it happens on ancient nixes and also on lixes that have at one point not had the issue. It seems to happen on macOS 15.6.1 and 15.7.2 and also 26. GitHub tarball change? I don't know. Seems to happen on my machine also. It seems to happen on machines that contain Enterprise Nonsense (SentinelOne EDR) and also VMs that don't. ``` ~ » time nix-prefetch-url --unpack https://channels.nixos.org/nixos-25.05/nixexprs.tar.xz path is '/nix/store/s5bdcwl39qavk84lg05q0y6s34qwpbsz-nixexprs.tar.xz' 1dji447hls0cm4531v7x16akf027843g78smrsvrn8c1y2zcn3jf nix-prefetch-url --unpack 5.93s user 24.22s system 16% cpu 3:01.14 total ~ »·time nix-prefetch-url --unpack https://channels.nixos.org/nixos-24.05/nixexprs.tar.xz path is '/nix/store/ix1vzmmzzj91r4gwy4xa15gjvh4914gi-nixexprs.tar.xz' 1f8j7fh0nl4qmqlxn6lis8zf7dnckm6jri4rwmj0qm1qivhr58lv nix-prefetch-url --unpack 4.89s user 25.86s system 18% cpu 2:46.79 total configs/nix » time nix flake update lix lix-module nixpkgs warning: updating lock file '/Users/jade/.dotfiles/configs/nix/flake.lock': • Updated input 'lix': 'https://git.lix.systems/api/v1/repos/lix-project/lix/archive/6da351929744863686bf81771242d796653d018c.tar.gz?narHash=sha256-MLAOvbsXOMRhBf51309kLOzvNJKDOcwFVL7D%2BGEKIWk%3D&rev=6da351929744863686bf81771242d796653d018c' (2025-11-24) → 'https://git.lix.systems/api/v1/repos/lix-project/lix/archive/36a61d922c51a16b57a88f44833960bf32c92d39.tar.gz?narHash=sha256-IMbgXXV7%2BzcRIG7GDERX%2B9vNIlgE%2BZs1gI2bfy96EO8%3D&rev=36a61d922c51a16b57a88f44833960bf32c92d39' (2025-12-09) • Updated input 'lix-module': 'https://git.lix.systems/api/v1/repos/lix-project/nixos-module/archive/192c92b603731fbc1bade6c1b18c8d8a0086f703.tar.gz?narHash=sha256-i2467FddWfd19q5Qoj%2B1/BAeg6LZmM5m4mYGRSQn/as%3D&rev=192c92b603731fbc1bade6c1b18c8d8a0086f703' (2025-11-18) → 'https://git.lix.systems/api/v1/repos/lix-project/nixos-module/archive/6c95c0b6f73f831226453fc6905c216ab634c30f.tar.gz?narHash=sha256-XnNABKfIYKSimQVvKc9FnlC2H0LurOhd9MS6l0Z67lE%3D&rev=6c95c0b6f73f831226453fc6905c216ab634c30f' (2025-11-30) • Updated input 'nixpkgs': 'https://releases.nixos.org/nixos/unstable-small/nixos-25.11pre901556.146bbcdf0a08/nixexprs.tar.xz?lastModified=1763985748&narHash=sha256-Mzkwmnc351sV9q%2BJUE/Fv9q4Gz1qer7OC0m8G%2B6Mv9I%3D&rev=146bbcdf0a0829fa8e39bacac942f3f2ee210b03' (2025-11-24) → 'https://releases.nixos.org/nixos/unstable-small/nixos-26.05pre909797.004943ed3cf9/nixexprs.tar.xz?lastModified=1765357816&narHash=sha256-wFcIt2NuDVi9Ux9hIL1SN/2GKbU%2BxdSwSyfFkQx/P9c%3D&rev=004943ed3cf9de5805a0da377599d1bfdd47a98a' (2025-12-10) nix flake update lix lix-module nixpkgs 5.11s user 28.61s system 15% cpu 3:43.38 total ``` this is *nuts*. what is going on here. `nix (Lix, like Nix) 2.95.0-pre20251124-dev_6da3519` but it's also on 2.93 and older versions. It seems to have started happening in the last ~month?

jade added the

OS/macOS

bug

labels

2025-12-10 21:38:03 +00:00

lix-bot commented

2025-12-12 01:32:17 +00:00

Member

This issue was mentioned on Gerrit on the following CLs:

commit message in cl/4772 ("tarfile: improve perf by writing to disk less")

This issue was mentioned on Gerrit on the following CLs: * commit message in [cl/4772](https://gerrit.lix.systems/c/lix/+/4772) ("tarfile: improve perf by writing to disk less")

rbt commented

2025-12-12 18:45:23 +00:00

Owner

Here are some conditions we've seen this bug on:

Lix versions:

2.93.0
2.94.0

CppNix versions:

2.33.0
IIRC some "ancient" CppNix versions, but I don't have numbers on hand.

macOS versions:

15.7.2
26.1

NixOS versions:

Latest nixos-unstable, e.g. addf7cf5f383a3101ecfba091b98d0a1263dc9b8
nixos-24.05

We definitely started seeing this bug by December 4th, but vague early reports from November 26th may have been the same bug.

Here are some conditions we've seen this bug on: Lix versions: - 2.93.0 - 2.94.0 CppNix versions: - 2.33.0 - IIRC some "ancient" CppNix versions, but I don't have numbers on hand. macOS versions: - 15.7.2 - 26.1 NixOS versions: - Latest `nixos-unstable`, e.g. `addf7cf5f383a3101ecfba091b98d0a1263dc9b8` - nixos-24.05 We definitely started seeing this bug by December 4th, but vague early reports from November 26th may have been the same bug.

pennae commented

2025-12-12 18:58:42 +00:00

Owner

so far it sounds like this being Very Clearly not a lix bug though, but a macos bug? right?

matthewbauer commented

2025-12-12 20:38:33 +00:00

It seems like it. Although things like normal "tar" & even libarchive's "bsdtar" don't seem to be affected? Caveat that it's a little hard to compare since Lix is doing a little more in nix-prefetch-url (download/deletion/other things).

jade commented

2025-12-14 07:12:42 +00:00

Author

Owner

While the regression and sense of urgency in fixing the perf here are new, extracting nixpkgs on macOS has always had bad performance, it's just not risen to the level of "so egregiously bad it's our first priority to go fix it" until Apple screwed up. (note: even if Apple fixes the part of this that's a regression, the status quo was a constant background mild frustration! also holding ones breath about apple fixing bugs is ill advised. one should assume their bugs are permanent)

The proposed fix of not using the problem code obviously will get rid of the problem by nuclear option, but it's not an optimization ultimately coming from nowhere; if macOS support were better funded all along we might have certainly done something like this earlier. It also should be a significant improvement on Linux, just less night and day because the cost per file extracted is lower.

While the regression and sense of urgency in fixing the perf here are new, extracting nixpkgs on macOS has always had *bad* performance, it's just not risen to the level of "so egregiously bad it's our first priority to go fix it" until Apple screwed up. (note: even if Apple fixes the part of this that's a regression, the status quo was a constant background mild frustration! also holding ones breath about apple fixing bugs is ill advised. one should assume their bugs are permanent) The proposed fix of not using the problem code obviously will get rid of the problem by nuclear option, but it's not an optimization ultimately coming from nowhere; if macOS support were better funded all along we might have certainly done something like this earlier. It also should be a significant improvement on Linux, just less night and day because the cost per file extracted is lower.

rbt changed title from ~~Extremely poor performance fetching GitHub tarballs on macOS~~ to macOS: tarball unpacking is extremely slow on some machines (2-5 mins to unpack Nixpkgs)

2025-12-31 20:43:23 +00:00