Commit graph

6447 commits

Author SHA1 Message Date
Eelco Dolstra 293220bed5
Merge pull request #4440 from Ma27/misc-pos-fixes
Miscellaneous improvements for positioning in eval-errors
2021-04-23 11:10:59 +02:00
Eelco Dolstra d9864be4b7 Merge branch 'master' of github.com:NixOS/nix 2021-04-22 10:33:00 +02:00
Eelco Dolstra 78a2303b3d Merge branch 'nix-derivation-query-store-path' of https://github.com/matthewbauer/nix 2021-04-22 10:29:18 +02:00
Eelco Dolstra dd83b29b90
Merge pull request #4724 from NixOS/nix-shell-missing-outputs
Make `nix shell` fallback to static outputs when needed
2021-04-22 10:25:55 +02:00
Eelco Dolstra 5385755227 Remove makeDecompressionSource()
This function doesn't support all compression methods (i.e. 'none' and
'br') so it shouldn't be exposed.

Also restore the original decompress() as a wrapper around
makeDecompressionSink().
2021-04-22 10:23:20 +02:00
Eelco Dolstra a6eebcff36 Merge branch 's3-decompress' of https://github.com/lukegb/nix 2021-04-22 10:20:40 +02:00
Luke Granger-Brown 97dde3cdd9 libutil: allow decompression with none/empty method
The S3 store relies on the ability to be able to decompress things with
an empty method, because it just passes the value of the Content-Encoding
directly to decompress.

If the file is not compressed, then this will cause the compression
routine to get confused.

This caused NixOS/nixpkgs#120120.
2021-04-22 02:33:59 +00:00
Antoine Martin ac38c4cdc2 Fix typo in flake doc 2021-04-22 03:07:57 +02:00
Matthew Bauer 428062e578 Get deriver when passing --derivation
This makes Nix look up paths derivations when they are passed as a
store paths. So:

$ nix path-info --derivation /nix/store/0pisd259nldh8yfjvw663mspm60cn5v8-hello-2.10

now gives

/nix/store/qp724w90516n4bk5r9gfb37vzmqdh3z7-hello-2.10.drv

instead of "".

If no deriver is available, Nix now errors instead of silently
ignoring that argument.
2021-04-21 15:32:33 -05:00
regnat 8d66f5f110 Make nix shell fallback to static outputs when needed
In case of pure input-addressed derivations, the build loop doesn't
guaranty that the realisations are stored in the db (if the output paths
are already there or can be substituted, the realisations won't be
registered). This caused `nix shell` to fail in some cases because it
was assuming that the realisations were always existing.

A better (but more involved) fix would probably to ensure that we always
register the realisations, but in the meantime this patches the surface
issue.

Fix #4721
2021-04-21 17:46:27 +02:00
Eelco Dolstra 8d651a1f68 Fix brotli compression of files > 128 KiB
This has been broken since faa31f4084.
2021-04-20 22:54:49 +02:00
Eelco Dolstra b60b0d62d6
Merge pull request #4720 from alyssais/wait
Include sys/wait.h everywhere WIFEXITED etc is used
2021-04-20 16:09:30 +02:00
Alyssa Ross 9ac6534f7c
Include sys/wait.h everywhere WIFEXITED etc is used
This is required on NetBSD, and I think FreeBSD too.
2021-04-19 18:31:58 +00:00
Maximilian Bosch 6905291daf
libcmd/installables: force re-evaluation of cached failures
I think that it's not very helpful to get "cached failures" in a wrong
`flake.nix`. This can be very confusing when debugging a Nix expression.
See for instance NixOS/nixpkgs#118115.

In fact, the eval cache allows a forced reevaluation which is used for
e.g. `nix eval`.

This change makes sure that this is the case for `nix build` as well. So
rather than

    λ ma27 [~/Projects/exp] → ../nix/outputs/out/bin/nix build -L --rebuild --experimental-features 'nix-command flakes'
    error: cached failure of attribute 'defaultPackage.x86_64-linux'

the evaluation of already-evaluated (and failed) attributes looks like
this now:

    λ ma27 [~/Projects/exp] → ../nix/outputs/out/bin/nix build -L --rebuild --experimental-features 'nix-command flakes'
    error: attribute 'hell' missing

           at /nix/store/mrnvi9ss8zn5wj6gpn4bcd68vbh42mfh-source/flake.nix:6:35:

                5|
                6|     packages.x86_64-linux.hello = nixpkgs.legacyPackages.x86_64-linux.hell;
                 |                                   ^
                7|
    (use '--show-trace' to show detailed location information)
2021-04-19 19:51:04 +02:00
Eelco Dolstra 3ee0ecdda0
Merge pull request #4679 from ony/feature/one-pass-canon-path
Optimize canonPath to resolve relative symlinks in one pass
2021-04-15 14:11:51 +02:00
Eelco Dolstra 15f4d4fd43 Drop libbz2 / zlib / lzma dependency + style fixes 2021-04-15 13:55:22 +02:00
Eelco Dolstra 6fb7582413 Merge branch 'libarchive-decompress' of https://github.com/serokell/nix 2021-04-15 13:39:04 +02:00
Eelco Dolstra f716779c1f Merge branch 'revertRemoveJSON' of https://github.com/cgohla/nix 2021-04-15 13:26:56 +02:00
Eelco Dolstra 709a60a045
Merge pull request #4707 from grahamc/defaul-path-restricted-eval
EvalSettings::getDefaultNixPath: respect {restrict,pure}Eval
2021-04-14 11:02:58 +02:00
Eelco Dolstra b47b5f4061
Merge pull request #4708 from NixOS/fix-remote-registerDrvOutput
Fix registerDrvOutput with the daemon
2021-04-14 11:01:36 +02:00
Maximilian Bosch 864ef0e93d
libexpr/primops: review 2021-04-13 23:12:39 +02:00
Maximilian Bosch ad7b47dc85
primops/libexpr: use new attr-call extractor everywhere; use function's pos if attr-set pos == noPos 2021-04-13 23:12:38 +02:00
Maximilian Bosch f60473a077
libexpr/primops: Move attr name extraction into its own function
This now takes care of providing positioning for both the faulting value
and the faulting function call in case of an error.
2021-04-13 23:12:38 +02:00
Maximilian Bosch 7c76964daa
libexpr: misc improvements for proper error position
When working on some more complex Nix code, there are sometimes rather
unhelpful or misleading error messages, especially if coerce-errors are
thrown.

This patch is a first steps towards improving that. I'm happy to file
more changes after that, but I'd like to gather some feedback first.

To summarize, this patch does the following things:

* Attrsets (a.k.a. `Bindings` in `libexpr`) now have a `Pos`. This is
  helpful e.g. to identify which attribute-set in `listToAttrs` is
  invalid.

* The `Value`-struct has a new method named `determinePos` which tries
  to guess the position of a value and falls back to a default if that's
  not possible.

  This can be used to provide better messages if a coercion fails.

* The new `determinePos`-API is used by `builtins.concatMap` now. With
  that change, Nix shows the exact position in the error where a wrong
  value was returned by the lambda.

  To make sure it's still obvious that `concatMap` is the problem,
  another stack-frame was added.

* The changes described above can be added to every other `primop`, but
  first I'd like to get some feedback about the overall approach.
2021-04-13 23:12:38 +02:00
Maximilian Bosch 3550a32b25
primops/derivation: use position of currently evaluated attribute
* The position of the `name`-attribute appears in the trace.
* If e.g. `meta` has no `outPath`-attribute, a `cannot coerce set to
  string` error will be thrown where `pos` points to `name =` which is
  highly misleading.
2021-04-13 23:12:38 +02:00
regnat ed29610cc6 Fix registerDrvOutput with the daemon
Resolve a protocol issue that caused the daemon to endlessly wait for
some information that the client doesn't ever send
2021-04-13 13:58:22 +02:00
Graham Christensen d04969ffa5 EvalSettings::getDefaultNixPath: respect {restrict,pure}Eval
Otherwise Nix may look to invalid locations for channels.
2021-04-12 12:37:15 -04:00
Domen Kožar c2dfda007e
add tests for zstd compression 2021-04-09 23:13:08 +02:00
Eelco Dolstra 42f0246698
Revert "libfetchers/tarball: Lock on effectiveUrl"
This reverts commit fc6bfb261d.

Fixes #4672.
2021-04-09 13:51:57 +02:00
Eelco Dolstra 906adadacd Restore stack size in child processes
Fixes #4673.
2021-04-07 13:40:13 +02:00
Eelco Dolstra 9b9e703df4 restoreSignals() + restoreAffinity() -> restoreProcessContext() 2021-04-07 13:10:02 +02:00
Eelco Dolstra 8a29052cb2 PathSubstitutionGoal: Clean up pipe
If there were many top-level goals (which are not destroyed until the
very end), commands like

  $ nix copy --to 'ssh://localhost?remote-store=/tmp/nix' \
    /run/current-system --no-check-sigs --substitute-on-destination

could fail with "Too many open files". So now we do some explicit
cleanup from amDone(). It would be cleaner to separate goals from
their temporary internal state, but that would be a bigger refactor.
2021-04-07 12:21:31 +02:00
Björn Gohla 9f28dd97ae Revert "Use upstream nlohmann_json"
This reverts commit 4145cd2da0.
2021-04-05 21:24:55 +01:00
John Ericson 125a824228 Document the derived path types. 2021-04-05 10:56:48 -04:00
John Ericson d8fa7517fa buildable.{cc,hh} -> derived-path.{cc,hh} 2021-04-05 10:33:28 -04:00
John Ericson 179582872d Make DerivedPathWithHints a newtype
This allows us to namespace its constructors under it.
2021-04-05 10:05:21 -04:00
John Ericson 9b805d36ac Rename Buildable 2021-04-05 09:52:25 -04:00
John Ericson 9dfb97c987 "newtype" BuildableReq
This makes for better types errors and allows us to give it methods.
2021-04-05 09:35:55 -04:00
John Ericson 4fe41c6db3 No templates for Buildable and BuildableReq 2021-04-05 09:15:25 -04:00
John Ericson 255d145ba7 Use BuildableReq for buildPaths and ensurePath
This avoids an ambiguity where the `StorePathWithOutputs { drvPath, {}
}` could mean "build `brvPath`" or "substitute `drvPath`" depending on
context.

It also brings the internals closer in line to the new CLI, by
generalizing the `Buildable` type is used there and makes that
distinction already.

In doing so, relegate `StorePathWithOutputs` to being a type just for
backwards compatibility (CLI and RPC).
2021-04-05 08:33:00 -04:00
John Ericson 32f4454b9f Move StorePathWithOutput utilities out of store class
These are by no means part of the notion of a store, but rather are
things that happen to use stores. (Or put another way, there's no way
we'd make them virtual methods any time soon.) It's better to move them
out of that too-big class then.

Also, this helps us remove StorePathWithOutputs from the Store interface
altogether next commit.
2021-04-05 08:31:37 -04:00
John Ericson 7a2b566dc8 Move StorePathWithOutputs into its own header/file
In the following commits it will become less prevalent.
2021-04-05 08:31:37 -04:00
John Ericson f7d9f7c338 Pull out Buildable into its own file/header in libnixstore 2021-04-05 08:31:37 -04:00
Mykola Orliuk f3f228700a canonPath in one pass 2021-03-31 04:58:49 +02:00
Bjørn Forsman edd606ae62 fetchGit: don't prefix "refs/heads/" on ref = "HEAD"
This fixes builtins.fetchGit { url = ...; ref = "HEAD"; }, that works in
stable nix (v2.3.10), but is broken in nix master:

  $ ./result/bin/nix repl
  Welcome to Nix version 2.4pre19700101_dd77f71. Type :? for help.

  nix-repl> builtins.fetchGit { url = "https://github.com/NixOS/nix"; ref = "HEAD"; }
  fetching Git repository 'https://github.com/NixOS/nix'fatal: couldn't find remote ref refs/heads/HEAD
  error: program 'git' failed with exit code 128

The documentation for builtins.fetchGit says ref = "HEAD" is the
default, so it should also be supported to explicitly pass it.

I came across this issue because poetry2nix can use ref = "HEAD" in some
situations.

Fixes #4674.
2021-03-30 11:21:38 +02:00
Eelco Dolstra dd77f71afe LocalBinaryCacheStore::upsertFile(): Fix race
When multiple threads try to upsert the same file, this could fail.

Fixes #4667.
2021-03-26 17:10:15 +01:00
Eelco Dolstra 4638bcfb2c Fix some typos
Fixes #4671.
2021-03-26 16:14:38 +01:00
Eelco Dolstra d1cb956bf2
Merge pull request #4650 from samueldr/fix/registry-config-dir
Use the appropriate config dir for the registry
2021-03-24 13:37:19 +01:00
Eelco Dolstra 1581c3e8ef
Merge pull request #4603 from DavHau/davhau-improve-nix-conf-help-builders
improve man page for nix.conf (builders)
2021-03-23 16:13:02 +01:00
Eelco Dolstra 6749d9e057
Merge pull request #4646 from Ma27/support-scoped-v6
Fix Nix to properly work with stores using a scoped IPv6 address
2021-03-23 15:03:47 +01:00