Commit graph

6435 commits

Author SHA1 Message Date
Eelco Dolstra a6eebcff36 Merge branch 's3-decompress' of https://github.com/lukegb/nix 2021-04-22 10:20:40 +02:00
Luke Granger-Brown 97dde3cdd9 libutil: allow decompression with none/empty method
The S3 store relies on the ability to be able to decompress things with
an empty method, because it just passes the value of the Content-Encoding
directly to decompress.

If the file is not compressed, then this will cause the compression
routine to get confused.

This caused NixOS/nixpkgs#120120.
2021-04-22 02:33:59 +00:00
Antoine Martin ac38c4cdc2 Fix typo in flake doc 2021-04-22 03:07:57 +02:00
Eelco Dolstra 8d651a1f68 Fix brotli compression of files > 128 KiB
This has been broken since faa31f4084.
2021-04-20 22:54:49 +02:00
Eelco Dolstra b60b0d62d6
Merge pull request #4720 from alyssais/wait
Include sys/wait.h everywhere WIFEXITED etc is used
2021-04-20 16:09:30 +02:00
Alyssa Ross 9ac6534f7c
Include sys/wait.h everywhere WIFEXITED etc is used
This is required on NetBSD, and I think FreeBSD too.
2021-04-19 18:31:58 +00:00
Maximilian Bosch 6905291daf
libcmd/installables: force re-evaluation of cached failures
I think that it's not very helpful to get "cached failures" in a wrong
`flake.nix`. This can be very confusing when debugging a Nix expression.
See for instance NixOS/nixpkgs#118115.

In fact, the eval cache allows a forced reevaluation which is used for
e.g. `nix eval`.

This change makes sure that this is the case for `nix build` as well. So
rather than

    λ ma27 [~/Projects/exp] → ../nix/outputs/out/bin/nix build -L --rebuild --experimental-features 'nix-command flakes'
    error: cached failure of attribute 'defaultPackage.x86_64-linux'

the evaluation of already-evaluated (and failed) attributes looks like
this now:

    λ ma27 [~/Projects/exp] → ../nix/outputs/out/bin/nix build -L --rebuild --experimental-features 'nix-command flakes'
    error: attribute 'hell' missing

           at /nix/store/mrnvi9ss8zn5wj6gpn4bcd68vbh42mfh-source/flake.nix:6:35:

                5|
                6|     packages.x86_64-linux.hello = nixpkgs.legacyPackages.x86_64-linux.hell;
                 |                                   ^
                7|
    (use '--show-trace' to show detailed location information)
2021-04-19 19:51:04 +02:00
Eelco Dolstra 3ee0ecdda0
Merge pull request #4679 from ony/feature/one-pass-canon-path
Optimize canonPath to resolve relative symlinks in one pass
2021-04-15 14:11:51 +02:00
Eelco Dolstra 15f4d4fd43 Drop libbz2 / zlib / lzma dependency + style fixes 2021-04-15 13:55:22 +02:00
Eelco Dolstra 6fb7582413 Merge branch 'libarchive-decompress' of https://github.com/serokell/nix 2021-04-15 13:39:04 +02:00
Eelco Dolstra f716779c1f Merge branch 'revertRemoveJSON' of https://github.com/cgohla/nix 2021-04-15 13:26:56 +02:00
Eelco Dolstra 709a60a045
Merge pull request #4707 from grahamc/defaul-path-restricted-eval
EvalSettings::getDefaultNixPath: respect {restrict,pure}Eval
2021-04-14 11:02:58 +02:00
Eelco Dolstra b47b5f4061
Merge pull request #4708 from NixOS/fix-remote-registerDrvOutput
Fix registerDrvOutput with the daemon
2021-04-14 11:01:36 +02:00
regnat ed29610cc6 Fix registerDrvOutput with the daemon
Resolve a protocol issue that caused the daemon to endlessly wait for
some information that the client doesn't ever send
2021-04-13 13:58:22 +02:00
Graham Christensen d04969ffa5 EvalSettings::getDefaultNixPath: respect {restrict,pure}Eval
Otherwise Nix may look to invalid locations for channels.
2021-04-12 12:37:15 -04:00
Domen Kožar c2dfda007e
add tests for zstd compression 2021-04-09 23:13:08 +02:00
Eelco Dolstra 42f0246698
Revert "libfetchers/tarball: Lock on effectiveUrl"
This reverts commit fc6bfb261d.

Fixes #4672.
2021-04-09 13:51:57 +02:00
Eelco Dolstra 906adadacd Restore stack size in child processes
Fixes #4673.
2021-04-07 13:40:13 +02:00
Eelco Dolstra 9b9e703df4 restoreSignals() + restoreAffinity() -> restoreProcessContext() 2021-04-07 13:10:02 +02:00
Eelco Dolstra 8a29052cb2 PathSubstitutionGoal: Clean up pipe
If there were many top-level goals (which are not destroyed until the
very end), commands like

  $ nix copy --to 'ssh://localhost?remote-store=/tmp/nix' \
    /run/current-system --no-check-sigs --substitute-on-destination

could fail with "Too many open files". So now we do some explicit
cleanup from amDone(). It would be cleaner to separate goals from
their temporary internal state, but that would be a bigger refactor.
2021-04-07 12:21:31 +02:00
Björn Gohla 9f28dd97ae Revert "Use upstream nlohmann_json"
This reverts commit 4145cd2da0.
2021-04-05 21:24:55 +01:00
John Ericson 125a824228 Document the derived path types. 2021-04-05 10:56:48 -04:00
John Ericson d8fa7517fa buildable.{cc,hh} -> derived-path.{cc,hh} 2021-04-05 10:33:28 -04:00
John Ericson 179582872d Make DerivedPathWithHints a newtype
This allows us to namespace its constructors under it.
2021-04-05 10:05:21 -04:00
John Ericson 9b805d36ac Rename Buildable 2021-04-05 09:52:25 -04:00
John Ericson 9dfb97c987 "newtype" BuildableReq
This makes for better types errors and allows us to give it methods.
2021-04-05 09:35:55 -04:00
John Ericson 4fe41c6db3 No templates for Buildable and BuildableReq 2021-04-05 09:15:25 -04:00
John Ericson 255d145ba7 Use BuildableReq for buildPaths and ensurePath
This avoids an ambiguity where the `StorePathWithOutputs { drvPath, {}
}` could mean "build `brvPath`" or "substitute `drvPath`" depending on
context.

It also brings the internals closer in line to the new CLI, by
generalizing the `Buildable` type is used there and makes that
distinction already.

In doing so, relegate `StorePathWithOutputs` to being a type just for
backwards compatibility (CLI and RPC).
2021-04-05 08:33:00 -04:00
John Ericson 32f4454b9f Move StorePathWithOutput utilities out of store class
These are by no means part of the notion of a store, but rather are
things that happen to use stores. (Or put another way, there's no way
we'd make them virtual methods any time soon.) It's better to move them
out of that too-big class then.

Also, this helps us remove StorePathWithOutputs from the Store interface
altogether next commit.
2021-04-05 08:31:37 -04:00
John Ericson 7a2b566dc8 Move StorePathWithOutputs into its own header/file
In the following commits it will become less prevalent.
2021-04-05 08:31:37 -04:00
John Ericson f7d9f7c338 Pull out Buildable into its own file/header in libnixstore 2021-04-05 08:31:37 -04:00
Mykola Orliuk f3f228700a canonPath in one pass 2021-03-31 04:58:49 +02:00
Bjørn Forsman edd606ae62 fetchGit: don't prefix "refs/heads/" on ref = "HEAD"
This fixes builtins.fetchGit { url = ...; ref = "HEAD"; }, that works in
stable nix (v2.3.10), but is broken in nix master:

  $ ./result/bin/nix repl
  Welcome to Nix version 2.4pre19700101_dd77f71. Type :? for help.

  nix-repl> builtins.fetchGit { url = "https://github.com/NixOS/nix"; ref = "HEAD"; }
  fetching Git repository 'https://github.com/NixOS/nix'fatal: couldn't find remote ref refs/heads/HEAD
  error: program 'git' failed with exit code 128

The documentation for builtins.fetchGit says ref = "HEAD" is the
default, so it should also be supported to explicitly pass it.

I came across this issue because poetry2nix can use ref = "HEAD" in some
situations.

Fixes #4674.
2021-03-30 11:21:38 +02:00
Eelco Dolstra dd77f71afe LocalBinaryCacheStore::upsertFile(): Fix race
When multiple threads try to upsert the same file, this could fail.

Fixes #4667.
2021-03-26 17:10:15 +01:00
Eelco Dolstra 4638bcfb2c Fix some typos
Fixes #4671.
2021-03-26 16:14:38 +01:00
Eelco Dolstra d1cb956bf2
Merge pull request #4650 from samueldr/fix/registry-config-dir
Use the appropriate config dir for the registry
2021-03-24 13:37:19 +01:00
Eelco Dolstra 1581c3e8ef
Merge pull request #4603 from DavHau/davhau-improve-nix-conf-help-builders
improve man page for nix.conf (builders)
2021-03-23 16:13:02 +01:00
Eelco Dolstra 6749d9e057
Merge pull request #4646 from Ma27/support-scoped-v6
Fix Nix to properly work with stores using a scoped IPv6 address
2021-03-23 15:03:47 +01:00
DavHau 0f40561c78 nix.conf builders: refer to manual page 2021-03-23 10:19:00 +07:00
John Ericson f44206e719 Fix typos in the last PR #4656 2021-03-22 15:18:48 +00:00
John Ericson 9d309de0de Clean up serialization for BuildResult
A few versioning mistakes were corrected:

- In 27b5747ca7, Daemon protocol had some
  version `>= 0xc` that should have been `>= 0x1c`, or `28` since the
  other conditions used decimal.

- In a2b69660a9, legacy SSH gated new CAS
  info on version 6, but version 5 in the server. It is now 6
  everywhere.

Additionally, legacy ssh was sending over more metadata than the daemon
one was. The daemon now sends that data too.

CC @regnat

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2021-03-22 14:57:41 +00:00
Samuel Dionne-Riel 66b857244f Use the appropriate config dir for the registry 2021-03-19 15:20:47 -04:00
Eelco Dolstra 3e0e443181 ProgressBar: Respect verbosity level
This makes its behaviour consistent with SimpleLogger.
2021-03-19 17:56:39 +01:00
Eelco Dolstra ef83ced4e1 Restore 'nix flake info' as a deprecated alias 2021-03-19 17:21:37 +01:00
Eelco Dolstra 8a5203d3b8
Merge pull request #4648 from hercules-ci/remove-unimplemented-hashAlgoOpt
Remove unimplemented hashAlgoOpt
2021-03-17 20:51:09 +01:00
Robert Hensing a61112aadf Remove unimplemented hashAlgoOpt
It was in the header but never implemented.
2021-03-17 11:27:11 +01:00
Maximilian Bosch ddcef3bcbb
Fix Nix to properly work with stores using a scoped IPv6 address
According to RFC4007[1], IPv6 addresses can have a so-called zone_id
separated from the actual address with `%` as delimiter. In contrast to
Nix 2.3, the version on `master` doesn't recognize it as such:

    $ nix ping-store --store ssh://root@fe80::1%18 --experimental-features nix-command
    warning: 'ping-store' is a deprecated alias for 'store ping'
    error: --- Error ----------------------------------------------------------------- nix
    don't know how to open Nix store 'ssh://root@fe80::1%18'

I modified the IPv6 match-regex accordingly to optionally detect this
part of the address. As we don't seem to do anything special with it, I
decided to leave it as part of the URL for now.

Fixes #4490

[1] https://tools.ietf.org/html/rfc4007
2021-03-16 19:14:42 +01:00
Eelco Dolstra 66fa1c7375 Merge 'nix flake {info,list-inputs}' into 'nix flake metadata'
Fixes #4613.
2021-03-16 17:19:04 +01:00
Eelco Dolstra 77f5d171e1 --override-input: Imply --no-write-lock-file
Fixes #3779.
2021-03-16 16:53:39 +01:00
Eelco Dolstra ccb8a403ee
Merge pull request #4587 from obsidiansystems/derivation-goal-detect-invalid-output
Throw error for derivation goal with bogus wanted output
2021-03-15 16:49:44 +01:00