Commit graph

12468 commits

Author SHA1 Message Date
Théophane Hufschmitt 90f9680733 Only use renameFile where needed
In most places the fallback to copying isn’t needed and can actually be
bad, so we’d rather not transparently fallback
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt d71d9e9fbf moveFile -> renameFile
`move` tends to have this `mv` connotation of “I will copy it for you if
needs be”
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt a4f0fd633c Link against c++fs on darwin
Required by the old clang version
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt c5db1821a9 Re-implement the recursive directory copy
The recursive copy from the stl doesn’t exactly do what we need because
1. It doesn’t delete things as we go
2. It doesn’t keep the mtime, which change the nars

So re-implement it ourselves. A bit dull, but that way we have what we want
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt 6f89fb6008 rename: Fallback to a copy if the filesystems mismatch
In `nix::rename`, if the call to `rename` fails with `EXDEV` (failure
because the source and the destination are in a different filesystems)
switch to copying and removing the source.

To avoid having to re-implement the copy manually, I switched the
function to use the c++17 `filesystem` library (which has a `copy`
function that should do what we want).

Fix #6262
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt c2de0a232c Create a wrapper around stdlib’s rename
Directly takes some c++ strings, and gently throws an exception on error
(rather than having to inline this logic everywhere)
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt 8119390abc Move some fs-related functions to their own file
Unclutter `util.cc` a bit
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt 780a479386
Merge pull request #6851 from K900/patch-1
doc/distributed-builds: don't use deprecated alias
2022-08-02 07:17:09 +02:00
Ilya K f675ba5331
doc/distributed-builds: don't use deprecated alias
`nix ping-store` -> `nix store ping`.
2022-08-01 13:50:35 +03:00
Rok Garbas e9178d7d4a
Merge pull request #6844 from centromere/custom-nix-conf
docker.nix: Allow Nix configuration to be customized
2022-07-29 13:47:24 +02:00
Eelco Dolstra 86fcd4f692
Merge pull request #6845 from fricklerhandwerk/attrset
manual: set -> attribute set
2022-07-28 16:55:03 +02:00
Valentin Gagarin 85cdaebcd6 manual: set -> attribute set
reword description to have shorter sentences.
2022-07-28 16:10:24 +02:00
Valentin Gagarin be4654c344 manual: fix section title in table of contents 2022-07-28 15:55:41 +02:00
Alex Wied 228028fc1a docker.nix: Allow Nix configuration to be customized 2022-07-28 03:36:39 -04:00
Théophane Hufschmitt 2805439335
Merge pull request #6814 from amjoseph-nixpkgs/pr/sandbox-error-messages
local-derivation-goal.cc: improve error messages when sandboxing fails
2022-07-22 13:27:52 +02:00
Théophane Hufschmitt e10807cdbb
Merge pull request #6813 from centromere/cgroup-cpu-detection
libstore/globals.cc: Automatically set cores based on cgroup CPU limit
2022-07-22 10:15:32 +02:00
Alex Wied 722de8ddcc libstore/globals.cc: Move cgroup detection to libutil 2022-07-19 16:25:53 -04:00
Alex Wied 1af5d798a4 libstore/globals.cc: Automatically set cores based on cgroup CPU limit
By default, Nix sets the "cores" setting to the number of CPUs which are
physically present on the machine. If cgroups are used to limit the CPU
and memory consumption of a large Nix build, the OOM killer may be
invoked.

For example, consider a GitLab CI pipeline which builds a large software
package. The GitLab runner spawns a container whose CPU is limited to 4
cores and whose memory is limited to 16 GiB. If the underlying machine
has 64 cores, Nix will invoke the build with -j64. In many cases, that
level of parallelism will invoke the OOM killer and the build will
completely fail.

This change sets the default value of "cores" to be
ceil(cpu_quota / cpu_period), with a fallback to
std:🧵:hardware_concurrency() if cgroups v2 is not detected.
2022-07-19 16:03:58 -04:00
Adam Joseph 36e1383b6b local-derivation-goal.cc: save global errno to the stack before performing tests which might clobber it 2022-07-19 03:53:20 -07:00
Adam Joseph a9e75eca00 error.hh: add additional constructor with explicit errno argument 2022-07-19 03:49:33 -07:00
Adam Joseph 99fcc91f67 as requested by @thufschmitt https://github.com/NixOS/nix/pull/6814#discussion_r924275777 2022-07-19 03:33:12 -07:00
Adam Joseph 5f51539f88 change warn() to notice() 2022-07-19 03:30:52 -07:00
Théophane Hufschmitt fbd0a6c6e2
Merge pull request #6784 from tweag/completion-test
Add some tests for the CLI completion
2022-07-18 20:32:14 +02:00
Eelco Dolstra 2584c151bd
Merge pull request #6812 from lovesegfault/rosetta-paths
fix(libstore): allow Nix to access all Rosetta 2 paths on MacOS
2022-07-18 14:09:54 +02:00
Adam Joseph c8c6203c2c local-derivation-goal.cc: detect unprivileged_userns_clone failure mode
The workaround for "Some distros patch Linux" mentioned in
local-derivation-goal.cc will not help in the `--option
sandbox-fallback false` case.  To provide the user more helpful
guidance on how to get the sandbox working, let's check to see if the
`/proc` node created by the aforementioned patch is present and
configured in a way that will cause us problems.  If so, give the user
a suggestion for how to troubleshoot the problem.
2022-07-17 01:27:22 -07:00
Adam Joseph 6fc56318bf local-derivation-goal.cc: add comment re: CLONE_NEWUSER
local-derivation-goal.cc contains a comment stating that "Some distros
patch Linux to not allow unprivileged user namespaces."  Let's give a
pointer to a common version of this patch for those who want more
details about this failure mode.
2022-07-17 01:23:32 -07:00
Adam Joseph 8d35f387dc local-derivation-goal.cc: warn if failing and /proc/self/ns/user missing
This commit causes nix to `warn()` if sandbox setup has failed and
`/proc/self/ns/user` does not exist.  This is usually a sign that the
kernel was compiled without `CONFIG_USER_NS=y`, which is required for
sandboxing.
2022-07-16 19:37:27 -07:00
Adam Joseph 90830b1074 local-derivation-goal.cc: warn if failing due to max_user_namespaces==0
This commit uses `warn()` to notify the user if sandbox setup fails
with errno==EPERM and /proc/sys/user/max_user_namespaces is missing or
zero, since that is at least part of the reason why sandbox setup
failed.

Note that `echo -n 0 > /proc/sys/user/max_user_namespaces` or
equivalent at boot time has been the recommended mitigation for
several Linux LPE vulnerabilities over the past few years.  Many users
have applied this mitigation and then forgotten that they have done
so.
2022-07-16 19:30:53 -07:00
Adam Joseph 8ea3a911aa local-derivation-goal.cc: improve error messages when sandboxing fails
The failure modes for nix's sandboxing setup are pretty complicated.
When nix is unable to set up the sandbox, let's provide more detail
about what went wrong.  Specifically:

* Make sure the error message includes the word "sandbox" so the user
  knows that the failure was related to sandboxing.

* If `--option sandbox-fallback false` was provided, and removing it
  would have allowed further attempts to make progress, let the user
  know.
2022-07-16 14:56:24 -07:00
Alex Wied b88fb50e21 fix(libstore): allow Nix to access all Rosetta 2 paths on MacOS
Fixes: #5884
2022-07-15 12:10:56 -07:00
Eelco Dolstra 59764eb842
Merge pull request #6810 from jfly/jfly/do-not-assume-savedvars-exist
nix develop: do not assume that saved vars are set
2022-07-15 13:59:25 +02:00
Eelco Dolstra 0621e99414
Merge pull request #6811 from edolstra/fix-auto-chroot
Disable auto-chroot if $NIX_STATE_DIR is set
2022-07-15 13:11:08 +02:00
Eelco Dolstra 3bcd7a5474 Disable auto-chroot if $NIX_STATE_DIR is set
Issue #6732.
2022-07-15 12:32:29 +02:00
Jeremy Fleischman 04386f7d69
nix develop: do not assume that saved vars are set
This fixes https://github.com/NixOS/nix/issues/6809
2022-07-14 23:25:39 -07:00
Domen Kožar de287964d5
Merge pull request #6807 from NixOS/curl-patch
curl: patch for netrc regression in Nix
2022-07-14 19:30:14 -05:00
Domen Kožar 99208bb8cc curl: patch for netrc regression in Nix 2022-07-14 17:45:02 -05:00
Eelco Dolstra ca4d5bee09
Merge pull request #6804 from edolstra/fix-auto-chroot
Disable auto-chroot if $NIX_STORE_DIR is set
2022-07-14 18:24:08 +02:00
Eelco Dolstra ff49c75502 Disable auto-chroot if $NIX_STORE_DIR is set
Fixes #6732.
2022-07-14 17:47:09 +02:00
Eelco Dolstra 73ff9b863c
Merge pull request #6803 from edolstra/test-stack-trace
On test failures, print a bash stack trace
2022-07-14 15:56:06 +02:00
Eelco Dolstra 2532fee157 On test failures, print a bash stack trace
This makes it easier to identify what command failed. It looks like:

  follow-paths.sh: test failed at:
    main in follow-paths.sh:54
2022-07-14 15:07:19 +02:00
Eelco Dolstra 819615c7f4
Merge pull request #6802 from edolstra/split-flakes-tests
Split flakes tests
2022-07-14 09:15:58 +02:00
Eelco Dolstra b15c4fdbde Split off 'nix flake check' tests 2022-07-13 21:01:16 +02:00
Eelco Dolstra 752158a8ef Move flake-searching.sh and make it less dependent on git 2022-07-13 20:55:17 +02:00
Eelco Dolstra 6ba45f81a8 Move flake-local-settings.sh 2022-07-13 20:51:28 +02:00
Eelco Dolstra 7abcafcfea Move the 'nix bundle' tests
Note: these were previously not actually called.
2022-07-13 20:49:07 +02:00
Eelco Dolstra d16f1070f4 Split off following paths tests 2022-07-13 20:46:22 +02:00
Eelco Dolstra a094259d35 Split off 'nix flake init' tests 2022-07-13 20:37:40 +02:00
Eelco Dolstra f011c269c9 Split off the circular flake import tests 2022-07-13 20:37:32 +02:00
Eelco Dolstra c591efafd3 Split off the Mercurial flake tests 2022-07-13 15:06:57 +02:00
Eelco Dolstra 420957e149 Move flakes tests to a subdirectory 2022-07-13 15:06:54 +02:00