Commit graph

3492 commits

Author SHA1 Message Date
Eelco Dolstra
224b56f10e Move creation of the temp roots file into its own function
This also moves the file handle into its own Sync object so we're not
holding the _state while acquiring the file lock. There was no real
deadlock risk here since locking a newly created file cannot block,
but it's still a bit nicer.
2023-01-03 14:51:23 +01:00
Eelco Dolstra
ae31b5f50f
Merge pull request #7497 from rski/master
src/libstore: Print the reason opening the DB failed
2023-01-03 12:44:14 +01:00
Théophane Hufschmitt
fb8fc6fda6
Merge pull request #7478 from hercules-ci/make-sure-initNix-called
libstore: Make sure that initNix has been called
2023-01-02 14:12:49 +01:00
Théophane Hufschmitt
e8a3e58171
Merge pull request #7521 from ncfavier/migration-deadlock
Release shared lock before acquiring exclusive lock
2023-01-02 11:08:43 +01:00
Steven Shaw
84b0893725
Fix error message 2023-01-01 12:37:43 +10:00
Eelco Dolstra
8c52f8ea9d
Merge pull request #7524 from ncfavier/sandbox-paths-closure
doc: sandbox-paths computes closures
2022-12-29 19:45:51 +01:00
Naïm Favier
d5d2f50ebb
doc: sandbox-paths computes closures 2022-12-28 17:09:20 +01:00
Naïm Favier
81c3f99b36
Release shared lock before acquiring exclusive lock
In principle, this should avoid deadlocks where two instances of Nix are
holding a shared lock on big-lock and are both waiting to get an
exclusive lock.

However, it seems like `flock(2)` is supposed to do this automatically,
so it's not clear whether this is actually where the problem comes from.
2022-12-27 15:58:14 +01:00
aba6eb348e libstore: Make sure that initNix has been called
Prevent bugs like https://github.com/cachix/cachix/pull/477
2022-12-24 14:39:30 +01:00
Eelco Dolstra
c164d304f3 nix develop: Set personality
This makes 'nix develop' set the Linux personality in the same way
that the actual build does, allowing a command like 'nix develop
nix#devShells.i686-linux.default' on x86_64-linux to work correctly.
2022-12-23 16:33:55 +01:00
rski
d034ed1891 src/libstore: Print the reason opening the DB failed
Without this, the error is lost, and it makes for a hard to debug
situation. Also remove some of the busyness inside the sqlite_open_v2
args.

The errcode returned is not the extended one. The only way to make open
return an extended code, would be to add SQLITE_OPEN_EXRESCODE to the
flags. In the future it might be worth making this change,
which would also simplify the existing SQLiteError code.
2022-12-23 02:55:51 +02:00
mupdt
bc8ab21c5a [PDT] TDE-3114: prevent a race-condition when creating the S3 cache 2022-12-21 04:50:40 -05:00
c965f35de7 Improve sqlite error messages
They did not include the detailed error message, losing essential
information for troubleshooting.

Example message:

    warning: creating statement 'insert or rplace into NARs(cache, hashPart, namePart, url, compression, fileHash, fileSize, narHash, narSize, refs, deriver, sigs, ca, timestamp, present) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 1)': at offset 10: SQL logic error, near "rplace": syntax error (in '/tmp/nix-shell.grQ6f7/nix-test/tests/binary-cache/test-home/.cache/nix/binary-cache-v6.sqlite')

It's not the best example; more important information will be in
the message for e.g. a constraint violation.

I don't see why this specific error is printed as a warning, but
that's for another commit.
2022-12-17 14:51:37 +01:00
Eelco Dolstra
0687e16c4a Fix a crash in DerivedPath::Built::toJSON() with impure derivations
The use of 'nullptr' here didn't result in a null JSON value, but in a
nullptr being cast to a string, which aborts.
2022-12-15 16:02:27 +01:00
Naïm Favier
1f3c0a3c1d
Allow disabling build users by unsetting build-users-group
Unsetting `build-users-group` (without `auto-allocate-uids` enabled)
gives the following error:

```
src/libstore/lock.cc:25: static std::unique_ptr<nix::UserLock> nix::SimpleUserLock::acquire(): Assertion `settings.buildUsersGroup != ""' failed.
```

Fix the logic in `useBuildUsers` and document the default value
for `build-users-group`.
2022-12-14 00:40:30 +01:00
John Ericson
5273cf4c97 Merge remote-tracking branch 'upstream/master' into indexed-store-path-outputs 2022-12-12 17:40:49 -05:00
John Ericson
dabb03b8d0 Merge remote-tracking branch 'upstream/master' into indexed-store-path-outputs 2022-12-12 17:36:02 -05:00
John Ericson
dc075dcdd0
Apply suggestions from code review
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-12-12 16:26:10 -05:00
Eelco Dolstra
ae5f62a894 Move isUri() and resolveUri() out of filetransfer.cc
These are purely related to NIX_PATH / -I command line parsing, so put
them in libexpr.
2022-12-12 14:05:35 +01:00
Eelco Dolstra
7396844676
Merge pull request #7421 from edolstra/lazy-trees-trivial-changes
Trivial changes from the lazy-trees branch
2022-12-12 13:52:56 +01:00
John Ericson
1879c7c95e
Merge branch 'master' into indexed-store-path-outputs 2022-12-12 07:33:36 -05:00
Théophane Hufschmitt
2affb19c92
Merge pull request #7409 from tweag/fix-6383
check the store for input before failing (hopefully fix #6383)
2022-12-09 06:33:30 +01:00
Eelco Dolstra
703d863a48 Trivial changes from the lazy-trees branch 2022-12-07 14:06:34 +01:00
8e0946e8df Remove repeat and enforce-determinism options
These only functioned if a very narrow combination of conditions held:

- The result path does not yet exist (--check did not result in
  repeated builds), AND
- The result path is not available from any configured substituters, AND
- No remote builders that can build the path are available.

If any of these do not hold, a derivation would be built 0 or 1 times
regardless of the repeat option. Thus, remove it to avoid confusion.
2022-12-07 11:36:48 +01:00
Taeer Bar-Yam
1c8de7d3d0 improve style 2022-12-06 11:25:38 -05:00
Eelco Dolstra
54906bc93c
Merge pull request #7382 from fricklerhandwerk/doc-automatic-uid
move documentation on `auto-allocate-uids` to options docs
2022-12-06 11:31:34 +01:00
Eelco Dolstra
484578d3f9
Tweak option descriptions 2022-12-06 10:30:36 +01:00
Taeer Bar-Yam
8c7661da09 check the store for input before failing (hopefully fix #6383) 2022-12-05 23:22:38 -05:00
Taeer Bar-Yam
e4f9f3bf24 check the store for input before failing (hopefully fix #6700) 2022-12-05 11:27:47 -05:00
Eelco Dolstra
1e6a5d1ff6 Clean up cgroup handling in getMaxCPU()
Also, don't assume in LocalDerivationGoal that cgroups are mounted on
/sys/fs/cgroup.
2022-12-02 12:59:13 +01:00
Eelco Dolstra
1211e59a03 Move cgroup.{cc,hh} to libutil 2022-12-02 12:38:03 +01:00
Valentin Gagarin
0ea62670ed move documentation on auto-allocate-uids to options docs
this is where it belongs and can be found together with the other
options.
2022-12-01 04:40:02 +01:00
Eelco Dolstra
fbc53e97ed
Merge pull request #3600 from NixOS/auto-uid-allocation
Automatic UID allocation
2022-11-29 14:01:42 +01:00
Eelco Dolstra
4f762e2b02 Restore ownership of / for non-uid-range builds 2022-11-29 13:10:53 +01:00
Eelco Dolstra
67bcb99700 Add a setting for enabling cgroups 2022-11-28 21:54:02 +01:00
Eelco Dolstra
ff12d1c1a1 Check that auto-allocated UIDs don't clash with existing accounts 2022-11-28 20:49:17 +01:00
Eelco Dolstra
dbf78a7ada
Merge pull request #7313 from yorickvP/nlohmann-everywhere
Replace src/libutil/json.cc with nlohmann
2022-11-28 15:03:48 +01:00
Eelco Dolstra
5b798f6cae Fix random client failures during GC server shutdown
We need to close the GC server socket before shutting down the active
GC client connections, otherwise a client may (re)connect and get
ECONNRESET. But also handle ECONNRESET for resilience.

Fixes random failures like

  GC socket disconnected
  connecting to '/tmp/nix-shell.y07M0H/nix-test/default/var/nix/gc-socket/socket'
  sending GC root '/tmp/nix-shell.y07M0H/nix-test/default/store/kb5yzija0f1x5xkqkgclrdzldxj6nnc6-non-blocking'
  reading GC root from client: error: unexpected EOF reading a line
  1 store paths deleted, 0.00 MiB freed
  error: reading from file: Connection reset by peer

in gc-non-blocking.sh.
2022-11-27 12:57:18 +01:00
John Ericson
26534f141c
Merge branch 'master' into indexed-store-path-outputs 2022-11-25 08:14:32 -05:00
Eelco Dolstra
6292d5616e Merge remote-tracking branch 'origin/master' into auto-uid-allocation 2022-11-23 11:16:09 +01:00
Eelco Dolstra
05d0892443
Merge pull request #7328 from edolstra/nix-build-stats
nix build --json: Include build statistics
2022-11-22 14:41:15 +01:00
Eelco Dolstra
3d23b9d032 SimpleUserLock::getSupplementaryGIDs(): Filter out main gid
This avoids having the user's gid in the supplementary group list as
well.
2022-11-22 10:26:17 +01:00
Eelco Dolstra
b37c2d84b6 Always call setgroups()
We shouldn't skip this if the supplementary group list is empty,
because then the sandbox won't drop the supplementary groups of the
parent (like "root").
2022-11-22 10:26:17 +01:00
Eelco Dolstra
02c02ee7c3
Merge pull request #6456 from amjoseph-nixpkgs/seccomp-mips
local-derivation-goal.cc: enable seccomp filters for mips{32,64}
2022-11-21 23:03:00 +01:00
Eelco Dolstra
c776dfbb35
Use hex for startId
Co-authored-by: Linus Heckemann <git@sphalerite.org>
2022-11-21 18:46:55 +01:00
Eelco Dolstra
9d17ce07e8 AutoUserLock: If sandboxing is disabled, use the build users group
We have to use a gid that has write access to the Nix store.
2022-11-21 12:55:49 +01:00
Eelco Dolstra
f0baa5c128 nix build --json: Include build statistics
Example:

  # nix build -L --extra-experimental-features cgroups --impure --expr 'with import <nixpkgs> {}; runCommand "foo" {} "dd if=/dev/urandom bs=1M count=1024 | md5sum; mkdir $out"' --json
  [
    {
      "cpuSystem": 1.911431,
      "cpuUser": 1.214249,
      "drvPath": "/nix/store/xzdqz67xba18hljhycp0hwfigzrs2z69-foo.drv",
      "outputs": {
        "out": "/nix/store/rh9mc9l2gkpq8kn2sgzndr6ll7ffjh6l-foo"
      },
      "startTime": 1669024076,
      "stopTime": 1669024079
    }
  ]
2022-11-21 12:06:01 +01:00
Eelco Dolstra
e7a5b76844 Rename derivedPathsWithHintsToJSON -> builtPathsToJSON 2022-11-21 11:56:20 +01:00
Eelco Dolstra
82d5cf2a76 Fix macOS build 2022-11-21 11:45:41 +01:00
Eelco Dolstra
653b32a78f Merge remote-tracking branch 'origin/master' into auto-uid-allocation 2022-11-21 11:33:23 +01:00