Commit graph

3957 commits

Author SHA1 Message Date
Daiderd Jordan 97baf32fbc
build: add exit code for hash and check mismatches
Makes it easier to identify the failure reason in other tooling, eg.
differentiate between a non-deterministic --check vs a failed build.

	$ nix-build '<nix/fetchurl.nix>' --argstr url http://example.org --argstr sha256 0000000000000000000000000000000000000000000000000000
	hash mismatch in fixed-output derivation '/nix/store/nzi9ck45rwlxzcwr25is7qlf3hs5xl83-example.org':
	  wanted: sha256:0000000000000000000000000000000000000000000000000000
	  got:    sha256:08y4734bm2zahw75b16bcmcg587vvyvh0n11gwiyir70divwp1rm
	$ echo $?
	102

	$ nix-build -E 'with import <nixpkgs> {}; runCommand "foo" {} "date +%s > $out"' --check
	warning: rewriting hashes in '/nix/store/g3k47g0399fvjmbm0p0mnad74k4w8vkz-foo'; cross fingers
	error: derivation '/nix/store/mggc8dz13ackb49qca6m23zq4fpq132q-foo.drv' may not be deterministic: output '/nix/store/g3k47g0399fvjmbm0p0mnad74k4w8vkz-foo' differs
	$ echo $?
	104
2019-07-02 00:12:34 +02:00
Aniket Deshpande ec58ba38c5
Fix http2 = false having no effect. Fixes #2971.
Setting `http2 = false` in nix config (e.g. /etc/nix/nix.conf)
had no effect, and `nix-env -vvvvv -i hello` still downloaded .nar
packages using HTTP/2.

In `src/libstore/download.cc`, the `CURL_HTTP_VERSION_2TLS` option was
being explicitly set when `downloadSettings.enableHttp2` was `true`,
but, `CURL_HTTP_VERSION_1_1` option was not being explicitly set when
`downloadSettings.enableHttp2` was `false`.

This may be because `https://curl.haxx.se/libcurl/c/libcurl-env.html` states:
"You have to set this option if you want to use libcurl's HTTP/2 support."
but, also, in the changelog, states:
"DEFAULT
Since curl 7.62.0: CURL_HTTP_VERSION_2TLS
Before that: CURL_HTTP_VERSION_1_1"

So, the default setting for `libcurl` is HTTP/2 for version >= 7.62.0.

In this commit, option `CURLOPT_HTTP_VERSION` is explicitly set to
`CURL_HTTP_VERSION_1_1` when `downloadSettings.enableHttp2` nix config
setting is `false`.

This can be tested by running `nix-env -vvvvv -i hello | grep HTTP`
2019-06-28 20:44:46 +05:30
Eelco Dolstra 6847c92788
Fix macOS build failure
Issue #2976.
2019-06-28 15:38:23 +02:00
Eelco Dolstra 324a5dc92f
ProgressBar: Fix updating
'updateCV.notify_one()' does nothing if the update thread is not
waiting for updateCV (in particular this happens when it is sleeping
on quitCV). So also set a variable to ensure that the update isn't
lost.
2019-06-25 21:59:20 +02:00
Eelco Dolstra 88571219d9
nix-channel: Don't fetch binary-cache-url
This has been ignored since the Perl->C++ rewrite.
2019-06-25 13:27:16 +02:00
Eelco Dolstra 09dde33c19
Automatically use --no-net if there are no network interfaces
(cherry picked from commit 04a5976996)
2019-06-25 12:45:59 +02:00
Eelco Dolstra 5600b070a7
Add "warning" verbosity level
This ensures that "nix" shows warnings. Previously these were hidden
because they were at "info" level.

(cherry picked from commit 615a9d031d)
2019-06-25 12:44:17 +02:00
Eelco Dolstra 64ec087f58
Fix 32-bit overflow with --no-net
--no-net causes tarballTtl to be set to the largest 32-bit integer,
which causes comparison like 'time + tarballTtl < other_time' to
fail on 32-bit systems. So cast them to 64-bit first.

https://hydra.nixos.org/build/95076624
(cherry picked from commit 29ccb2e969)
2019-06-24 22:16:43 +02:00
Eelco Dolstra f8b30338ac
Refactor downloadCached() interface
(cherry picked from commit df3f5a78d5)
2019-06-24 22:12:26 +02:00
Eelco Dolstra 7b9c68766d
Add '--no-net' convenience flag
This flag

* Disables substituters.

* Sets the tarball-ttl to infinity (ensuring e.g. that the flake
  registry and any downloaded flakes are considered current).

* Disables retrying downloads and sets the connection timeout to the
  minimum. (So it doesn't completely disable downloads at the moment.)

(cherry picked from commit 8ea842260b)
2019-06-24 22:07:29 +02:00
Eelco Dolstra 78fa47a7f0
Fix 'error 9 while decompressing xz file'
Once we've started writing data to a Sink, we can't restart a download
request, because then we end up writing duplicate data to the
Sink. Therefore we shouldn't handle retries in Downloader but at a
higher level (in particular, in copyStorePath()).

Fixes #2952.

(cherry picked from commit a67cf5a358)
2019-06-24 21:59:51 +02:00
Eelco Dolstra 2fef4dd296
Downloader: Propagate exceptions from decompressionSink->finish()
(cherry picked from commit 15fa70cd1b)
2019-06-24 21:59:31 +02:00
Eelco Dolstra b43e1e186e
CachedDownloadResult: Include store path
Also, make fetchGit and fetchMercurial update allowedPaths properly.

(Maybe the evaluator, rather than the caller of the evaluator, should
apply toRealPath(), but that's a bigger change.)

(cherry picked from commit 5c34d66538)
2019-06-24 21:59:27 +02:00
Eelco Dolstra dc29e9fb47
downloadCached: Return ETag
(cherry picked from commit 529add316c)
2019-06-24 21:58:33 +02:00
Eelco Dolstra 94f11d0a61
Fix abort in fromTOML
Fixes #2969.
2019-06-24 17:09:09 +02:00
Eelco Dolstra 3cc1125595
Another attempt at getting pseudoterminals to work on macOS 2019-06-17 08:08:04 +02:00
Eelco Dolstra 2743bf0bb1 Hopefully fix macOS tests 2019-06-16 20:02:40 +02:00
Eelco Dolstra 82ca6ef390 Set $TERM 2019-06-16 17:36:30 +02:00
Eelco Dolstra e84c265645 Run builds in a pseudo-terminal
This allows many programs (e.g. gcc, clang, cmake) to print colorized
log output (assuming $TERM is set to a value like "xterm").

There are other ways to get colors, in particular setting
CLICOLOR_FORCE, but they're less widely supported and can break
programs that parse tool output.
2019-06-16 17:36:30 +02:00
Eelco Dolstra b693029ca0
Style fix 2019-06-16 09:43:20 +02:00
Eelco Dolstra 26bc876ae6
nix: Add -L alias for --print-build-logs 2019-06-15 16:45:00 +02:00
Eelco Dolstra 34fa8ce917
nix: Support -j flag 2019-06-15 16:34:06 +02:00
Félix Baylac-Jacqué 9e0f5f803f
Daemon: warn when an untrusted user cannot override a setting
In a daemon-based Nix setup, some options cannot be overridden by a
client unless the client's user is considered trusted.

Currently, if an untrusted user tries to override one of those
options, we are silently ignoring it.

This can be pretty confusing in certain situations.

e.g. a user thinks he disabled the sandbox when in reality he did not.

We are now sending a warning message letting know the user some options
have been ignored.

Related to #1761.
2019-06-15 03:59:12 -04:00
Eelco Dolstra aec545c20b Fix segfault in builtin fetchurl with hashed mirrors + SRI hashes 2019-06-01 15:27:43 +02:00
Ding Xiang Fei abdedcdb38 bump cpptoml to v0.1.1 2019-05-29 17:01:39 +08:00
Eelco Dolstra 22f2744afd
Iterate over references 2019-05-28 23:05:08 +02:00
Eelco Dolstra 9eaebbf575
Merge branch 'attrPaths' of https://github.com/NinjaTrappeur/nix 2019-05-28 22:59:05 +02:00
Eelco Dolstra 4d829916e7
Merge pull request #2800 from flokli/progress-bar-hide-unknown-expected
progress-bar: hide expected if expected is 0 (unknown)
2019-05-21 13:29:28 +02:00
Eelco Dolstra 8f6c72faee
Merge pull request #2810 from NixOS/print-build-logs
nix: Add --print-build-logs flag
2019-05-15 20:38:38 +02:00
Eelco Dolstra 66b8a62101 nix: Add --print-build-logs flag
This causes 'nix' to print build log output to stderr rather than
showing the last log line in the progress bar. Log lines are prefixed
by the name of the derivation (minus the version string), e.g.

  binutils> make[1]: Leaving directory '/build/binutils-2.31.1'
  binutils-wrapper> unpacking sources
  binutils-wrapper> patching sources
  ...
  binutils-wrapper> Using dynamic linker: '/nix/store/kr51dlsj9v5cr4n8700jliyz8v5b2q7q-bootstrap-stage0-glibc/lib/ld-linux-x86-64.so.2'
  bootstrap-stage2-gcc-wrapper> unpacking sources
  ...
  linux-headers> unpacking sources
  linux-headers> unpacking source archive /nix/store/8javli69jhj3bkql2c35gsj5vl91p382-linux-4.19.16.tar.xz
2019-05-15 17:33:56 +02:00
Eelco Dolstra 5f6840fbb4
Merge pull request #2802 from LnL7/fix-needs-hashrewrite
build: move needsHashRewrite initialization to startBuilder
2019-05-13 08:53:48 +02:00
Graham Christensen f1b8e9efe7
runProgram: Uncomment chdir support 2019-05-12 17:03:01 -04:00
Daiderd Jordan ce02fc74b2
build: make needsHashRewrite a method 2019-05-12 22:56:27 +02:00
Graham Christensen 73b797c207
handleDiffHook: stop passing allowVfork 2019-05-12 13:44:22 -04:00
Graham Christensen b4a05edbfe
runProgram: support gid, uid, chdir 2019-05-12 13:17:27 -04:00
Graham Christensen dde8eeb39a
chdir, setgroups 2019-05-12 13:17:26 -04:00
Graham Christensen 6df61db060
diff hook: execute as the build user, and pass the temp dir 2019-05-12 13:17:26 -04:00
Graham Christensen c78686e411
build: run diff-hook under --check and document diff-hook 2019-05-12 13:17:24 -04:00
Daiderd Jordan ff6867ab94
build: move needsHashRewrite initialization to startBuilder
The value of useChroot is not set yet in the constructor, resulting in
hash rewriting being enabled in certain cases where it should not be.

Fixes #2801
2019-05-12 15:53:40 +02:00
Florian Klink 6ade7ec022 progress-bar: hide expected if expected is 0 (unknown)
Sometimes, "expected" can be "0", but in fact means "unknown".

This is for example the case when downloading a file while the http
server doesn't send the `Content-Length` header, like when running `nix
build` pointing to a nixpkgs checkout streamed from GitHub:

⇒  nix build -f https://github.com/NixOS/nixpkgs/archive/master.tar.gz hello
[1.8/0.0 MiB DL] downloading 'https://github.com/NixOS/nixpkgs/archive/master.tar.gz'

In that case, don't show that weird progress bar, but only the (slowly
increasing) downloaded size ("done").

⇒  nix build -f https://github.com/NixOS/nixpkgs/archive/master.tar.gz hello
[1.8 MiB DL] downloading 'https://github.com/NixOS/nixpkgs/archive/master.tar.gz'

This commit also updates fmt calls with three numbers (when something is
currently 'running' too) - I'm not sure if this can be provoked, but
showing "0" as expected doesn't make any sense, as we're obviously doing
more than nothing.
2019-05-11 12:34:39 +02:00
Eelco Dolstra 7c6391ddc7
nix-shell: Don't absolutize '-p' expressions
This prevents spurious syscalls like

  25011 lstat("/home/eelco/with import <nixpkgs> { }; (pkgs.runCommandCC or pkgs.runCommand) \"shell\" { buildInputs = [ (hello) ]; } \"\"", 0x7ffe9c67f580) = -1 ENOENT (No such file or directory)
2019-05-08 14:29:36 +02:00
Eelco Dolstra 92caa60c49
Merge branch 'repl/ctrlc' of https://github.com/xbreak/nix 2019-05-08 10:18:45 +02:00
Eelco Dolstra 3e940bbf2d
Merge pull request #2790 from samueldr/fix/minor-comment-NIX_ROOT_FINDER
findRootsNoTemp: fixes comment about findRuntimeRoots
2019-05-08 10:13:09 +02:00
Daniel Schaefer 3f192ac80c Add builtins.hashFile
For text files it is possible to do it like so:
`builtins.hashString "sha256" (builtins.readFile /tmp/a)`
but that doesn't work for binary files.

With builtins.hashFile any kind of file can be conveniently hashed.
2019-05-03 17:23:36 +02:00
Eelco Dolstra f9a2ea4486
Fix "Bad system call" running i686-linux binaries on x86_64-linux
To determine which seccomp filters to install, we were incorrectly
using settings.thisSystem, which doesn't denote the actual system when
--system is used.

Fixes #2791.
2019-05-03 10:48:33 +02:00
Samuel Dionne-Riel cbc7d9a412 findRootsNoTemp: fixes comment about findRuntimeRoots
The NIX_ROOT_FINDER environment variable was removed in
3c46fe62b8 when porting from perl to C.
2019-04-30 22:43:24 -04:00
Daiderd Jordan b614e0e53d
build: only skip hash rewriting for sandboxing on linux
The sandbox on darwin, and possibly other future platforms, doesn't have
enough isolation to redirect outputs without hash rewriting.
2019-04-17 23:41:23 +02:00
Eelco Dolstra bb6e6923f2 Add environment variable NIX_SHOW_SYMBOLS for dumping the symbol table 2019-04-11 23:04:13 +02:00
Eelco Dolstra 41ba5135e0 primeCache(): Barf if builds are needed but not allowed
Fixes #2753.
2019-04-01 21:09:49 +02:00
Eelco Dolstra 2bc6304793 getMachines(): Cache result 2019-04-01 21:09:35 +02:00