Commit graph

11489 commits

Author SHA1 Message Date
Matthew Bauer 2f63cc02de Create parent trusted list path before writing
This makes sure that $HOME/.local/share/nix exists before we try to
write to it.
2021-05-08 21:31:28 -05:00
regnat b66234134f Add a realisations disk cache
Similar to the nar-info disk cache (and using the same db).
This makes rebuilds muuch faster.

- This works regardless of the ca-derivations experimental feature.
  I could modify the logic to not touch the db if the flag isn’t there,
  but given that this is a trash-able local cache, it doesn’t seem to be
  really worth it.
- We could unify the `NARs` and `Realisation` tables to only have one
  generic kv table. This is left as an exercise to the reader.
- I didn’t update the cache db version number as the new schema just
  adds a new table to the previous one, so the db will be transparently
  migrated and is backwards-compatible.

Fix #4746
2021-05-06 17:38:20 +02:00
Eelco Dolstra db6ab75cae
Merge pull request #4778 from matthewbauer/fix-nix-profile-install-first-output-2
Use derivation output name from toDerivation
2021-05-06 15:35:24 +02:00
Matthew Bauer 1e3a9033b7 Use derivation output name from toDerivation
This was previously done in https://github.com/NixOS/nix/pull/4515 but
got clobbered away in https://github.com/NixOS/nix/pull/4594.

--------------------------------------------------------------------------------

This fixes an issue where derivations with a primary output that is
not "out" would fail with:

$ nix profile install nixpkgs#sqlite
error: opening directory '/nix/store/2a2ydlgyydly5czcc8lg12n6qqkfz863-sqlite-3.34.1-bin': No such file or directory

This happens because while derivations produce every output when
built, you might not have them if you didn't build the derivation
yourself (for instance, the store path was fetch from a binary cache).
This uses outputName provided from DerivationInfo which appears to
match the first output of the derivation.
2021-05-05 16:33:05 -05:00
regnat bf485dcf46 Properly normalize the content-addressed paths
Make sure that their timestamp are always normalized.
Otherwise, strange − and non-deterministic − things might happen, like
https://github.com/NixOS/nixpkgs/issues/121813

Fix #4775
2021-05-05 21:00:08 +02:00
Alyssa Ross 5f07f2ff2b
doc: builtins: use a definition list
This looks a lot better (and is also more semantically meaningful).

Since this list is generated in a Nix expression, I don't think using
HTML here is going to be the thing that puts people off modifying this
part of the documentation!
2021-05-05 14:34:46 +00:00
Anders Kaseorg 4f493faf80 launchd: Use exec to avoid leaving the extra shell wrapper running
Before:

  UID   PID  PPID   C STIME   TTY           TIME CMD
    0  1737     1   0  2:28PM ??         0:00.00 /bin/sh -c /bin/wait4path /nix/var/nix/profiles/default/bin/nix-daemon && /nix/var/nix/profiles/default/bin/nix-daemon
    0  1739  1737   0  2:28PM ??         0:00.05 /nix/var/nix/profiles/default/bin/nix-daemon

After:

  UID   PID  PPID   C STIME   TTY           TIME CMD
    0  1763     1   0  2:29PM ??         0:00.05 /nix/var/nix/profiles/default/bin/nix-daemon

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2021-05-04 14:33:47 -07:00
Eelco Dolstra fe3a10a9b2
Merge pull request #4767 from NixOS/fix-s3-ca-caching
Fix the double-slash in the realisations path
2021-05-04 15:38:52 +02:00
regnat dadfbce318 Fix the double-slash in the realisations path
Make sure that we always access the realisations under
`binaryCacheUrl/realisations` and not `binaryCacheUrl//realisations`

Fix #4766
2021-05-04 10:35:34 +02:00
Eelco Dolstra ef13c9c223
Merge pull request #4750 from NinjaTrappeur/nin-build-dry-run-json
nix build: make dry-run to print a json output if --json is enabled
2021-05-03 16:14:29 +02:00
Eelco Dolstra 40378fbcba
Merge pull request #4761 from emilazy/issue-4658-mark-impure-host-deps-as-optional
Mark `__impureHostDeps` paths as optional
2021-05-03 16:13:36 +02:00
Eelco Dolstra 6d2553ae14
Merge pull request #4763 from Ma27/fix-nix-shell-error
Bump version number for `DerivedPath` changes
2021-05-03 16:11:45 +02:00
Maximilian Bosch e5951a6b2f
Bump version number for DerivedPath changes
I guess I misunderstood John's initial explanation about why wildcards
for outputs are sent to older stores[1]. My `nix-daemon` from 2021-03-26
also has version 1.29, but misses the wildcard[2]. So bumping seems to
be the right call.

[1] https://github.com/NixOS/nix/pull/4759#issuecomment-830812464
[2] 255d145ba7
2021-05-03 01:12:23 +02:00
Emily c4355a52fa Mark __impureHostDeps paths as optional
Starting in macOS 11, the on-disk dylib bundles are no longer available,
but nixpkgs needs to be able to keep compatibility with older versions
that require `/usr/lib/libSystem.B.dylib` in `__impureHostDeps`. Allow
it to keep backwards compatibility with these versions by marking these
dependencies as optional.

Fixes #4658.
2021-05-02 05:30:50 +01:00
Graham Christensen d15a1962cb
Merge pull request #4289 from abathur/encrypt_darwin_volume
darwin: encrypt nix volume if filevault is enabled
2021-04-29 15:59:12 -04:00
Travis A. Everett eab14a642c darwin: encrypt nix volume if filevault is enabled 2021-04-29 13:26:51 -05:00
Domen Kožar e3e78ee2a2
Merge pull request #4751 from Ma27/storepath-pos
primops/storePath: add trace to pure mode error
2021-04-27 19:05:23 +02:00
Maximilian Bosch e6a360a4d9
primops/storePath: add trace to pure mode error
As described in #4745 it's otherwise fairly hard to understand where
this is coming from. Say you have an expression which uses e.g.
`types.package`:

``` nix
{ outputs = { self, nixpkgs }: {
    packages.x86_64-linux.hello = let
      foo = nixpkgs.lib.evalModules {
        modules = [
          {
            options.foo.bar = with nixpkgs.lib; mkOption { type = types.package; };
          }
          {
            foo.bar = ./.;
          }
        ];
      };
    in builtins.trace foo.config.foo.bar.outPath nixpkgs.legacyPackages.x86_64-linux.hello;
    defaultPackage.x86_64-linux = self.packages.x86_64-linux.hello;
  };
}
```

Then you'll get an error trace like this:

```
error: 'builtins.storePath' is not allowed in pure evaluation mode

       at /nix/store/p4h2x6r80njkb0j2rc1xjhhl99yri3zb-source/lib/attrsets.nix:328:15:

          327|     let
          328|       path' = builtins.storePath path;
             |               ^
          329|       res =

       … while evaluating the attribute 'config.foo.bar.outPath'

       at /nix/store/p4h2x6r80njkb0j2rc1xjhhl99yri3zb-source/lib/attrsets.nix:332:11:

          331|           name = sanitizeDerivationName (builtins.substring 33 (-1) (baseNameOf path'));
          332|           outPath = path';
             |           ^
          333|           outputs = [ "out" ];

       … while evaluating the attribute 'packages.x86_64-linux.hello'

       at /nix/store/6c1rfsqzrhjw1235palzjmf5vihcpci7-source/flake.nix:3:5:

            2| { outputs = { self, nixpkgs }: {
            3|     packages.x86_64-linux.hello = let
             |     ^
            4|       foo = nixpkgs.lib.evalModules {
```

Fixes #4745
2021-04-27 17:24:38 +02:00
Félix Baylac-Jacqué 7dca9c24c4
nix build: make dry-run to print a json output if --json is enabled 2021-04-27 11:28:47 +02:00
Eelco Dolstra fe2bf464cf Merge branch 'remove-trailing-spaces' of github.com:NixOS/nix 2021-04-23 15:09:44 +02:00
regnat 31313d1401 Replace the trailing markdown spaces by a backslash
They are equivalent according to
<https://spec.commonmark.org/0.29/#hard-line-breaks>,
and the trailing spaces tend to be a pain (because the make git
complain, editors tend to want to remove them − the `.editorconfig`
actually specifies that − etc..).
2021-04-23 14:37:21 +02:00
Eelco Dolstra 4549d65b4f
Merge pull request #4732 from NixOS/4725-always-register-the-realisations
Aways register the realisations
2021-04-23 14:22:03 +02:00
Eelco Dolstra 3996ca42cf
Merge pull request #4735 from NixOS/document-ca-derivations
(briefly) document how to build ca derivations
2021-04-23 14:21:25 +02:00
Eelco Dolstra 788008385e Use lowdown 0.8.4 2021-04-23 14:20:32 +02:00
Eelco Dolstra e65f604ea9 Set more man sections 2021-04-23 14:20:28 +02:00
Eelco Dolstra 5a1feddfe7 Merge branch 'man1' of https://github.com/alyssais/nix 2021-04-23 14:20:17 +02:00
Alyssa Ross 29132f99c5
doc: fix section in nix3 man page metadata
These man pages said they were in section 7, even though we were
installing them to section 1 (which is the right place for them).
2021-04-23 10:52:54 +00:00
Eelco Dolstra 293220bed5
Merge pull request #4440 from Ma27/misc-pos-fixes
Miscellaneous improvements for positioning in eval-errors
2021-04-23 11:10:59 +02:00
regnat 2207018df5 (briefly) document how to build ca derivations 2021-04-23 10:28:26 +02:00
regnat d3df747cb6 Revert "Make nix shell fallback to static outputs when needed"
This reverts commit 8d66f5f110.

This fix isn't needed anymore now that the realisations are always
properly registered
2021-04-23 09:34:43 +02:00
regnat 6ea9c65aec fixup! Add a regression test for #4725 2021-04-23 09:34:16 +02:00
p01arst0rm 45473d02c9 unified macro style for ENABLE_S3 2021-04-23 07:30:05 +01:00
regnat 9161e02039 Always register the realisations of input-addressed drvs
Fix #4725
2021-04-22 20:07:02 +02:00
regnat b1711071d1 Add a regression test for #4725 2021-04-22 20:07:02 +02:00
Eelco Dolstra d9864be4b7 Merge branch 'master' of github.com:NixOS/nix 2021-04-22 10:33:00 +02:00
Eelco Dolstra 78a2303b3d Merge branch 'nix-derivation-query-store-path' of https://github.com/matthewbauer/nix 2021-04-22 10:29:18 +02:00
Eelco Dolstra dd83b29b90
Merge pull request #4724 from NixOS/nix-shell-missing-outputs
Make `nix shell` fallback to static outputs when needed
2021-04-22 10:25:55 +02:00
Eelco Dolstra 5385755227 Remove makeDecompressionSource()
This function doesn't support all compression methods (i.e. 'none' and
'br') so it shouldn't be exposed.

Also restore the original decompress() as a wrapper around
makeDecompressionSink().
2021-04-22 10:23:20 +02:00
Eelco Dolstra a6eebcff36 Merge branch 's3-decompress' of https://github.com/lukegb/nix 2021-04-22 10:20:40 +02:00
Eelco Dolstra 7c90552879
Merge pull request #4729 from alarsyo/fix-flake-doc-typo
Fix typo in flake doc
2021-04-22 09:56:22 +02:00
Luke Granger-Brown 97dde3cdd9 libutil: allow decompression with none/empty method
The S3 store relies on the ability to be able to decompress things with
an empty method, because it just passes the value of the Content-Encoding
directly to decompress.

If the file is not compressed, then this will cause the compression
routine to get confused.

This caused NixOS/nixpkgs#120120.
2021-04-22 02:33:59 +00:00
Antoine Martin ac38c4cdc2 Fix typo in flake doc 2021-04-22 03:07:57 +02:00
Matthew Bauer 428062e578 Get deriver when passing --derivation
This makes Nix look up paths derivations when they are passed as a
store paths. So:

$ nix path-info --derivation /nix/store/0pisd259nldh8yfjvw663mspm60cn5v8-hello-2.10

now gives

/nix/store/qp724w90516n4bk5r9gfb37vzmqdh3z7-hello-2.10.drv

instead of "".

If no deriver is available, Nix now errors instead of silently
ignoring that argument.
2021-04-21 15:32:33 -05:00
regnat 8d66f5f110 Make nix shell fallback to static outputs when needed
In case of pure input-addressed derivations, the build loop doesn't
guaranty that the realisations are stored in the db (if the output paths
are already there or can be substituted, the realisations won't be
registered). This caused `nix shell` to fail in some cases because it
was assuming that the realisations were always existing.

A better (but more involved) fix would probably to ensure that we always
register the realisations, but in the meantime this patches the surface
issue.

Fix #4721
2021-04-21 17:46:27 +02:00
Eelco Dolstra 8d651a1f68 Fix brotli compression of files > 128 KiB
This has been broken since faa31f4084.
2021-04-20 22:54:49 +02:00
Eelco Dolstra b60b0d62d6
Merge pull request #4720 from alyssais/wait
Include sys/wait.h everywhere WIFEXITED etc is used
2021-04-20 16:09:30 +02:00
Eelco Dolstra 3cbad4b035
Merge pull request #4688 from Ma27/force-reevaluation
libcmd/installables: force re-evaluation of cached failures
2021-04-19 20:38:17 +02:00
Alyssa Ross 9ac6534f7c
Include sys/wait.h everywhere WIFEXITED etc is used
This is required on NetBSD, and I think FreeBSD too.
2021-04-19 18:31:58 +00:00
Maximilian Bosch 6905291daf
libcmd/installables: force re-evaluation of cached failures
I think that it's not very helpful to get "cached failures" in a wrong
`flake.nix`. This can be very confusing when debugging a Nix expression.
See for instance NixOS/nixpkgs#118115.

In fact, the eval cache allows a forced reevaluation which is used for
e.g. `nix eval`.

This change makes sure that this is the case for `nix build` as well. So
rather than

    λ ma27 [~/Projects/exp] → ../nix/outputs/out/bin/nix build -L --rebuild --experimental-features 'nix-command flakes'
    error: cached failure of attribute 'defaultPackage.x86_64-linux'

the evaluation of already-evaluated (and failed) attributes looks like
this now:

    λ ma27 [~/Projects/exp] → ../nix/outputs/out/bin/nix build -L --rebuild --experimental-features 'nix-command flakes'
    error: attribute 'hell' missing

           at /nix/store/mrnvi9ss8zn5wj6gpn4bcd68vbh42mfh-source/flake.nix:6:35:

                5|
                6|     packages.x86_64-linux.hello = nixpkgs.legacyPackages.x86_64-linux.hell;
                 |                                   ^
                7|
    (use '--show-trace' to show detailed location information)
2021-04-19 19:51:04 +02:00
Eelco Dolstra 76980a1f3d Merge branch 'build-with-strictDeps' of https://github.com/hercules-ci/nix 2021-04-15 14:15:21 +02:00