Excessive darwin sandbox entries can cause Apple's sandbox code to throw an assert #756

New issue

Open

opened 2025-03-21 21:44:32 +00:00 by jade · 4 comments

jade commented

2025-03-21 21:44:32 +00:00

Owner

Test case:

{ n ? 1500 }:
let
  # nixpkgs lib.range
  range =
    first:
    last:
    if first > last then
      []
    else
      builtins.genList (n: first + n) (last - first + 1);

  deps = builtins.map (n: builtins.toFile "input-${toString n}" "meow") (range 0 n);
in builtins.derivation {
  name = "dependent";
  inherit deps;
  builder = "/bin/sh";
  system = "aarch64-darwin";
  args = ["-c" "echo ok"];
}

Or: https://gerrit.lix.systems/c/lix/+/2870

+(darwin-sandbox-big.sh:15) nix-build --no-out-link ./darwin-sandbox-big.nix --option sandbox true --argstr seed 2761
this derivation will be built:
  /private/var/folders/wk/92h51hd93rz0mkf92yxxhr5w0000gn/T/nix-test/darwin-sandbox-big/store/b0rwnymxin3p8idsj5ax4kpm640s5myw-depe
ndent.drv
error:
       … while waiting for the build environment for '/private/var/folders/wk/92h51hd93rz0mkf92yxxhr5w0000gn/T/nix-test/darwin-san
dbox-big/store/b0rwnymxin3p8idsj5ax4kpm640s5myw-dependent.drv' to initialize (failed due to signal 6 (Abort trap: 6), previous mes
sages: Assertion failed: (diff <= INSTR_JUMP_NE_MAX_LENGTH), function push_jne_instr, file serialize.c, line 240.)

       error: unexpected EOF reading a line

This seems to only throw an assert when NIX_STORE is rebound to cause the paths to be much longer? idk. Gremlins.

Weirdly enough this seems to also break when I nix build --sandbox -f test.nix even with the sandbox patch.

Test case: ```nix { n ? 1500 }: let # nixpkgs lib.range range = first: last: if first > last then [] else builtins.genList (n: first + n) (last - first + 1); deps = builtins.map (n: builtins.toFile "input-${toString n}" "meow") (range 0 n); in builtins.derivation { name = "dependent"; inherit deps; builder = "/bin/sh"; system = "aarch64-darwin"; args = ["-c" "echo ok"]; } ``` Or: https://gerrit.lix.systems/c/lix/+/2870 ``` +(darwin-sandbox-big.sh:15) nix-build --no-out-link ./darwin-sandbox-big.nix --option sandbox true --argstr seed 2761 this derivation will be built: /private/var/folders/wk/92h51hd93rz0mkf92yxxhr5w0000gn/T/nix-test/darwin-sandbox-big/store/b0rwnymxin3p8idsj5ax4kpm640s5myw-depe ndent.drv error: … while waiting for the build environment for '/private/var/folders/wk/92h51hd93rz0mkf92yxxhr5w0000gn/T/nix-test/darwin-san dbox-big/store/b0rwnymxin3p8idsj5ax4kpm640s5myw-dependent.drv' to initialize (failed due to signal 6 (Abort trap: 6), previous mes sages: Assertion failed: (diff <= INSTR_JUMP_NE_MAX_LENGTH), function push_jne_instr, file serialize.c, line 240.) error: unexpected EOF reading a line ``` This seems to only throw an assert when `NIX_STORE` is rebound to cause the paths to be much longer? idk. Gremlins. Weirdly enough this seems to also break when I `nix build --sandbox -f test.nix` even with the sandbox patch.

jade added the

OS/macOS

Area/store

labels

2025-03-21 21:44:32 +00:00

jade commented

2025-03-21 21:46:19 +00:00

Author

Owner

cc #752

cc https://git.lix.systems/lix-project/lix/issues/752

jade commented

2025-03-22 00:14:08 +00:00

Author

Owner

I have tried to create an independent repro but it doesn't work. idk what i am doing wrong that causes it to not have problems.

#!/usr/bin/env python3

import subprocess
from pathlib import Path
import tempfile
import os


def make_profile(stuff_dir: Path):
    stuff_dir.mkdir(parents=True)
    evils_list = []
    for n in range(3000):
        f = stuff_dir / f'evil-evil-evil-very-evil-no-good-very-bad-evil-evil-evil-kitty-evil-{n}'
        with f.open('w') as h:
            h.write('meow')
        evils_list.append(f)

    evils_profile = '(allow file-read* file-write* process-exec\n'
    evils_profile += '\n'.join(
        f'(literal "{p!s}")' for p in evils_list
    )
    evils_profile += '\n)'

    return """
(version 1)
;(deny default (with no-log))


(define TMPDIR (param "_GLOBAL_TMP_DIR"))

(deny default)

; Disallow creating setuid/setgid binaries, since that
; would allow breaking build user isolation.
(deny file-write-setugid)

; Allow forking.
(allow process-fork)

; Allow reading system information like #CPUs, etc.
(allow sysctl-read)

; Allow POSIX semaphores and shared memory.
(allow ipc-posix*)

; Allow socket creation.
(allow system-socket)

; Allow sending signals within the sandbox.
(allow signal (target same-sandbox))

; Allow getpwuid.
(allow mach-lookup (global-name "com.apple.system.opendirectoryd.libinfo"))

; Access to /tmp.
; The network-outbound/network-inbound ones are for unix domain sockets, which
; we allow access to in TMPDIR (but if we allow them more broadly, you could in
; theory escape the sandbox)
(allow file* process-exec network-outbound network-inbound
       (literal "/tmp") (subpath TMPDIR))

; Some packages like to read the system version.
(allow file-read*
       (literal "/System/Library/CoreServices/SystemVersion.plist")
       (literal "/System/Library/CoreServices/SystemVersionCompat.plist"))

; Without this line clang cannot write to /dev/null, breaking some configure tests.
(allow file-read-metadata (literal "/dev"))

; Many packages like to do local networking in their test suites, but let's only
; allow it if the package explicitly asks for it.
(if (param "_ALLOW_LOCAL_NETWORKING")
    (begin
      (allow network* (local ip) (local tcp) (local udp))

      ; Allow access to /etc/resolv.conf (which is a symlink to
      ; /private/var/run/resolv.conf).
      ; TODO: deduplicate with sandbox-network.sb
      (allow file-read-metadata
             (literal "/var")
             (literal "/etc")
             (literal "/etc/resolv.conf")
             (literal "/private/etc/resolv.conf"))

      (allow file-read*
             (literal "/private/var/run/resolv.conf"))

      ; Allow DNS lookups. This is even needed for localhost, which lots of tests rely on
      (allow file-read-metadata (literal "/etc/hosts"))
      (allow file-read*         (literal "/private/etc/hosts"))
      (allow network-outbound (remote unix-socket (path-literal "/private/var/run/mDNSResponder")))))

; Standard devices.
(allow file*
       (literal "/dev/null")
       (literal "/dev/random")
       (literal "/dev/stderr")
       (literal "/dev/stdin")
       (literal "/dev/stdout")
       (literal "/dev/tty")
       (literal "/dev/urandom")
       (literal "/dev/zero")
       (subpath "/dev/fd"))

; Allow pseudo-terminals.
(allow file*
       (literal "/dev/ptmx")
       (regex #"^/dev/pty[a-z]+")
       (regex #"^/dev/ttys[0-9]+"))

; Does nothing, but reduces build noise.
(allow file* (literal "/dev/dtracehelper"))

(allow file* (literal "/"))

; Allow access to zoneinfo since libSystem needs it.
(allow file-read* (subpath "/usr/share/zoneinfo"))

(allow file-read* (subpath "/usr/share/locale"))

; This is mostly to get more specific log messages when builds try to
; access something in /etc or /var.
(allow file-read-metadata
       (literal "/etc")
       (literal "/var")
       (literal "/private/var/tmp"))

; This is used by /bin/sh on macOS 10.15 and later.
(allow file*
       (literal "/private/var/select/sh"))

; Allow Rosetta 2 to run x86_64 binaries on aarch64-darwin (and vice versa).
(allow file-read*
       (subpath "/Library/Apple/usr/libexec/oah")
       (subpath "/System/Library/Apple/usr/libexec/oah")
       (subpath "/System/Library/LaunchDaemons/com.apple.oahd.plist")
       (subpath "/Library/Apple/System/Library/LaunchDaemons/com.apple.oahd.plist"))

;(allow file-read* file-write* process-exec
;    (subpath "/nix/store/gwcs72vb6jlc3b36y11ywrz4ga45sz5k-dependent")
;)
(allow file-read* file-write* process-exec
 (subpath "/System/Library/Frameworks")
 (subpath "/System/Library/PrivateFrameworks")
  (literal "/bin/bash")
  (literal "/bin/echo")
  (literal "/bin/sh"))
    """ + evils_profile

with tempfile.TemporaryDirectory() as d:
    d = Path(d)
    new_env = os.environ.copy()
    # new_env['_GLOBAL_TMP_DIR'] = '/tmp'
    # new_env['_ALLOW_LOCAL_NETWORKING'] = '0'

    profile = make_profile(d / 'stuff')
    # print(profile)
    with open(d / 'profile.sb', 'w') as h:
        h.write(profile)

    subprocess.check_call(
        ['sandbox-exec', '-D' '_GLOBAL_TMP_DIR=/tmp', '-f', d / 'profile.sb', '/bin/echo', 'kitty']
    )

I have tried to create an independent repro but it doesn't work. idk what i am doing wrong that causes it to not have problems. ```python #!/usr/bin/env python3 import subprocess from pathlib import Path import tempfile import os def make_profile(stuff_dir: Path): stuff_dir.mkdir(parents=True) evils_list = [] for n in range(3000): f = stuff_dir / f'evil-evil-evil-very-evil-no-good-very-bad-evil-evil-evil-kitty-evil-{n}' with f.open('w') as h: h.write('meow') evils_list.append(f) evils_profile = '(allow file-read* file-write* process-exec\n' evils_profile += '\n'.join( f'(literal "{p!s}")' for p in evils_list ) evils_profile += '\n)' return """ (version 1) ;(deny default (with no-log)) (define TMPDIR (param "_GLOBAL_TMP_DIR")) (deny default) ; Disallow creating setuid/setgid binaries, since that ; would allow breaking build user isolation. (deny file-write-setugid) ; Allow forking. (allow process-fork) ; Allow reading system information like #CPUs, etc. (allow sysctl-read) ; Allow POSIX semaphores and shared memory. (allow ipc-posix*) ; Allow socket creation. (allow system-socket) ; Allow sending signals within the sandbox. (allow signal (target same-sandbox)) ; Allow getpwuid. (allow mach-lookup (global-name "com.apple.system.opendirectoryd.libinfo")) ; Access to /tmp. ; The network-outbound/network-inbound ones are for unix domain sockets, which ; we allow access to in TMPDIR (but if we allow them more broadly, you could in ; theory escape the sandbox) (allow file* process-exec network-outbound network-inbound (literal "/tmp") (subpath TMPDIR)) ; Some packages like to read the system version. (allow file-read* (literal "/System/Library/CoreServices/SystemVersion.plist") (literal "/System/Library/CoreServices/SystemVersionCompat.plist")) ; Without this line clang cannot write to /dev/null, breaking some configure tests. (allow file-read-metadata (literal "/dev")) ; Many packages like to do local networking in their test suites, but let's only ; allow it if the package explicitly asks for it. (if (param "_ALLOW_LOCAL_NETWORKING") (begin (allow network* (local ip) (local tcp) (local udp)) ; Allow access to /etc/resolv.conf (which is a symlink to ; /private/var/run/resolv.conf). ; TODO: deduplicate with sandbox-network.sb (allow file-read-metadata (literal "/var") (literal "/etc") (literal "/etc/resolv.conf") (literal "/private/etc/resolv.conf")) (allow file-read* (literal "/private/var/run/resolv.conf")) ; Allow DNS lookups. This is even needed for localhost, which lots of tests rely on (allow file-read-metadata (literal "/etc/hosts")) (allow file-read* (literal "/private/etc/hosts")) (allow network-outbound (remote unix-socket (path-literal "/private/var/run/mDNSResponder"))))) ; Standard devices. (allow file* (literal "/dev/null") (literal "/dev/random") (literal "/dev/stderr") (literal "/dev/stdin") (literal "/dev/stdout") (literal "/dev/tty") (literal "/dev/urandom") (literal "/dev/zero") (subpath "/dev/fd")) ; Allow pseudo-terminals. (allow file* (literal "/dev/ptmx") (regex #"^/dev/pty[a-z]+") (regex #"^/dev/ttys[0-9]+")) ; Does nothing, but reduces build noise. (allow file* (literal "/dev/dtracehelper")) (allow file* (literal "/")) ; Allow access to zoneinfo since libSystem needs it. (allow file-read* (subpath "/usr/share/zoneinfo")) (allow file-read* (subpath "/usr/share/locale")) ; This is mostly to get more specific log messages when builds try to ; access something in /etc or /var. (allow file-read-metadata (literal "/etc") (literal "/var") (literal "/private/var/tmp")) ; This is used by /bin/sh on macOS 10.15 and later. (allow file* (literal "/private/var/select/sh")) ; Allow Rosetta 2 to run x86_64 binaries on aarch64-darwin (and vice versa). (allow file-read* (subpath "/Library/Apple/usr/libexec/oah") (subpath "/System/Library/Apple/usr/libexec/oah") (subpath "/System/Library/LaunchDaemons/com.apple.oahd.plist") (subpath "/Library/Apple/System/Library/LaunchDaemons/com.apple.oahd.plist")) ;(allow file-read* file-write* process-exec ; (subpath "/nix/store/gwcs72vb6jlc3b36y11ywrz4ga45sz5k-dependent") ;) (allow file-read* file-write* process-exec (subpath "/System/Library/Frameworks") (subpath "/System/Library/PrivateFrameworks") (literal "/bin/bash") (literal "/bin/echo") (literal "/bin/sh")) """ + evils_profile with tempfile.TemporaryDirectory() as d: d = Path(d) new_env = os.environ.copy() # new_env['_GLOBAL_TMP_DIR'] = '/tmp' # new_env['_ALLOW_LOCAL_NETWORKING'] = '0' profile = make_profile(d / 'stuff') # print(profile) with open(d / 'profile.sb', 'w') as h: h.write(profile) subprocess.check_call( ['sandbox-exec', '-D' '_GLOBAL_TMP_DIR=/tmp', '-f', d / 'profile.sb', '/bin/echo', 'kitty'] ) ```

jade commented

2025-03-22 00:30:46 +00:00

Author

Owner

Well, whatever, here's a reproducer obtained by just extracting the profile from what Lix was putting in there, idk if it works on anyone else's machine, but it works on mine: https://gist.github.com/lf-/8ec3d714fd638873fe5324838c4dae4d#file-repro-md