Commit graph

431 commits

Author SHA1 Message Date
Claudio Bley e7b9dc7c89 nix-store: Use long for narSize in graphml output 2023-03-24 17:25:17 +01:00
Eelco Dolstra 8f96720b9f Fix another uninitialized variable
https://hydra.nixos.org/build/211811494
2023-03-09 15:30:53 +01:00
Eelco Dolstra f0908f592c
Merge pull request #7942 from edolstra/remove-format
Remove FormatOrString and remaining uses of format()
2023-03-03 10:22:11 +01:00
Valentin Gagarin 37c61907d7
Merge pull request #7605 from tweag/comments
add comments
2023-03-03 06:38:29 +01:00
Eelco Dolstra 29abc8e764 Remove FormatOrString and remaining uses of format() 2023-03-02 15:57:54 +01:00
Timothy DeHerrera df643051e2
nix-store: read paths from standard input
Resolves #7437 for new `nix-store` by adding a `--stdin` flag.
2023-02-28 12:29:16 -07:00
Taeer Bar-Yam b2752a4f74 add comments 2023-01-17 08:28:56 -05:00
Eelco Dolstra 17f81d3215 Fix unused variable warning 2022-12-12 16:41:46 +01:00
Eelco Dolstra 7396844676
Merge pull request #7421 from edolstra/lazy-trees-trivial-changes
Trivial changes from the lazy-trees branch
2022-12-12 13:52:56 +01:00
Andreas Rammhold dbc8547664 Ignore the enforceDeterminism value
We used to set enforceDeterminism to true in the settings (by default)
and thus did send a non-zero value over the wire. The value should
probably be ignored as it should only matter if nrRounds is non-zero
as well.

Having the old code here where the value is expected to be zero only
works with the same version of Nix where we are sending zero. We
should always test this against older Nix versions being client or
server as otherwise upgrade in larger networks might be a pain.

Fixes 8e0946e8df
2022-12-10 17:55:07 +01:00
Eelco Dolstra 703d863a48 Trivial changes from the lazy-trees branch 2022-12-07 14:06:34 +01:00
Linus Heckemann 8e0946e8df Remove repeat and enforce-determinism options
These only functioned if a very narrow combination of conditions held:

- The result path does not yet exist (--check did not result in
  repeated builds), AND
- The result path is not available from any configured substituters, AND
- No remote builders that can build the path are available.

If any of these do not hold, a derivation would be built 0 or 1 times
regardless of the repeat option. Thus, remove it to avoid confusion.
2022-12-07 11:36:48 +01:00
Eelco Dolstra 6259fd7ea6 Fix indentation 2022-11-04 13:29:39 +01:00
Eelco Dolstra b95faccf03 Merge remote-tracking branch 'origin/master' into auto-uid-allocation 2022-11-03 17:43:40 +01:00
Rickard Nilsson c2d7456926 Fix a misplaced parenthese in serve protocol check
This issue made it impossible for clients using a serve protocol of
version <= 2.3 to use the `cmdBuildDerivation` command of servers using
a protocol of version >= 2.6. The faulty version check makes the server
send back build outputs that the client is not expecting.
2022-08-24 01:54:43 +02:00
Sidharth Kshatriya 3efea3d28a nix-store: small std::move() optimization 2022-06-09 16:25:26 +05:30
Naïm Favier 1461e6cdda
Stop the logger properly in legacy commands
Ensures the logger is stopped on exit in legacy commands. Without this,
when using `nix-build --log-format bar` and stopping nix with CTRL+C,
the bar is not cleared from the screen.
2022-05-11 12:58:45 +02:00
John Ericson a03b1fd7f6 Deduplicate the Store downcasting with a template 2022-03-11 13:32:33 +00:00
John Ericson 678d1c2aa0 Factor out a LogStore interface
Continue progress on #5729.

Just as I hoped, this uncovered an issue: the daemon protocol is missing
a way to query build logs. This doesn't effect `unix://`, but does
effect `ssh://`. A FIXME is left for this, so we come back to it later.
2022-03-11 13:32:16 +00:00
John Ericson 073e134de6 Rename requireGcStore to GcStore::require
I should have done this to begin with. This will be nicer once more
Store sub-interfaces exist too, to illustrate the pattern.
2022-03-11 13:27:38 +00:00
John Ericson 6636202356 Factor out a GcStore interface
Starts progress on #5729.

The idea is that we should not have these default methods throwing
"unimplemented". This is a small step in that direction.

I kept `addTempRoot` because it is a no-op, rather than failure. Also,
as a practical matter, it is called all over the place, while doing
other tasks, so the downcasting would be annoying.

Maybe in the future I could move the "real" `addTempRoot` to `GcStore`,
and the existing usecases use a `tryAddTempRoot` wrapper to downcast or
do nothing, but I wasn't sure whether that was a good idea so with a
bias to less churn I didn't do it yet.
2022-03-03 19:01:25 +00:00
John Ericson e862833ec6 Move BuildResult defintion to its own header
Just like we did for `ValidPathInfo` in
d92d4f85a5.
2022-03-01 19:43:07 +00:00
John Ericson dc92b01885 Merge remote-tracking branch 'upstream/master' into auto-uid-allocation 2022-02-28 23:54:20 +00:00
Eelco Dolstra df552ff53e Remove std::string alias (for real this time)
Also use std::string_view in a few more places.
2022-02-25 16:13:02 +01:00
Maximilian Bosch 50edbc4ddf
nix-store --serve: pass on settings.keepFailed from SSH store
When doing e.g.

    nix-build -A package --keep-failed --option \
      builders \
      'ssh://mfhydra?remote-store=/home/bosch/store x86_64-linux - 10 4 big-parallel'

this doesn't work properly because this build-setting is ignored.

I changed this behavior by passing the `settings.keepFailed` through the
serve-protocol to remote machines to make sure that I can introspect the
build-directory (which is particularly helpful when I have to look at a
`config.log` from a failed build for instance).
2021-08-31 13:11:46 +02:00
John Ericson e023c985d5 Merge remote-tracking branch 'upstream/master' into auto-uid-allocation 2021-04-27 19:06:58 +00:00
John Ericson 9b805d36ac Rename Buildable 2021-04-05 09:52:25 -04:00
John Ericson 255d145ba7 Use BuildableReq for buildPaths and ensurePath
This avoids an ambiguity where the `StorePathWithOutputs { drvPath, {}
}` could mean "build `brvPath`" or "substitute `drvPath`" depending on
context.

It also brings the internals closer in line to the new CLI, by
generalizing the `Buildable` type is used there and makes that
distinction already.

In doing so, relegate `StorePathWithOutputs` to being a type just for
backwards compatibility (CLI and RPC).
2021-04-05 08:33:00 -04:00
John Ericson 32f4454b9f Move StorePathWithOutput utilities out of store class
These are by no means part of the notion of a store, but rather are
things that happen to use stores. (Or put another way, there's no way
we'd make them virtual methods any time soon.) It's better to move them
out of that too-big class then.

Also, this helps us remove StorePathWithOutputs from the Store interface
altogether next commit.
2021-04-05 08:31:37 -04:00
John Ericson 9d309de0de Clean up serialization for BuildResult
A few versioning mistakes were corrected:

- In 27b5747ca7, Daemon protocol had some
  version `>= 0xc` that should have been `>= 0x1c`, or `28` since the
  other conditions used decimal.

- In a2b69660a9, legacy SSH gated new CAS
  info on version 6, but version 5 in the server. It is now 6
  everywhere.

Additionally, legacy ssh was sending over more metadata than the daemon
one was. The daemon now sends that data too.

CC @regnat

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2021-03-22 14:57:41 +00:00
Eelco Dolstra 94637cd7e5
Merge pull request #4477 from NixOS/ca/build-remote
Build ca derivations remotely
2021-02-26 16:54:44 +01:00
Shea Levy 98d1b64400
Initialize plugins after handling initial command line flags
This is technically a breaking change, since attempting to set plugin
files after the first non-flag argument will now throw an error. This
is acceptable given the relative lack of stability in a plugin
interface and the need to tie the knot somewhere once plugins can
actually define new subcommands.
2021-02-24 08:22:17 -05:00
regnat a2b69660a9 LegacySSHStore: Send back the new realisations
To allow it to build ca derivations remotely
2021-02-23 08:04:03 +01:00
Shea Levy 6af6e41df0
Move command plugin interface to libnixcmd 2021-01-26 06:22:24 -05:00
Eelco Dolstra 8d4268d190 Improve error formatting
Changes:

* The divider lines are gone. These were in practice a bit confusing,
  in particular with --show-trace or --keep-going, since then there
  were multiple lines, suggesting a start/end which wasn't the case.

* Instead, multi-line error messages are now indented to align with
  the prefix (e.g. "error: ").

* The 'description' field is gone since we weren't really using it.

* 'hint' is renamed to 'msg' since it really wasn't a hint.

* The error is now printed *before* the location info.

* The 'name' field is no longer printed since most of the time it
  wasn't very useful since it was just the name of the exception (like
  EvalError). Ideally in the future this would be a unique, easily
  googleable error ID (like rustc).

* "trace:" is now just "…". This assumes error contexts start with
  something like "while doing X".

Example before:

  error: --- AssertionError ---------------------------------------------------------------------------------------- nix
  at: (7:7) in file: /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix

       6|
       7|   x = assert false; 1;
        |       ^
       8|

  assertion 'false' failed
  ----------------------------------------------------- show-trace -----------------------------------------------------
  trace: while evaluating the attribute 'x' of the derivation 'hello-2.10'
  at: (192:11) in file: /home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/make-derivation.nix

     191|         // (lib.optionalAttrs (!(attrs ? name) && attrs ? pname && attrs ? version)) {
     192|           name = "${attrs.pname}-${attrs.version}";
        |           ^
     193|         } // (lib.optionalAttrs (stdenv.hostPlatform != stdenv.buildPlatform && !dontAddHostSuffix && (attrs ? name || (attrs ? pname && attrs ? version)))) {

Example after:

  error: assertion 'false' failed

         at: (7:7) in file: /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix

              6|
              7|   x = assert false; 1;
               |       ^
              8|

         … while evaluating the attribute 'x' of the derivation 'hello-2.10'

         at: (192:11) in file: /home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/make-derivation.nix

            191|         // (lib.optionalAttrs (!(attrs ? name) && attrs ? pname && attrs ? version)) {
            192|           name = "${attrs.pname}-${attrs.version}";
               |           ^
            193|         } // (lib.optionalAttrs (stdenv.hostPlatform != stdenv.buildPlatform && !dontAddHostSuffix && (attrs ? name || (attrs ? pname && attrs ? version)))) {
2021-01-21 11:02:09 +01:00
Eelco Dolstra 61216d32e1 Add 'nix store repair' command 2021-01-13 23:27:39 +01:00
Eelco Dolstra 555152ffe8 crypto.cc: API cleanup and add generate() / to_string() methods 2021-01-06 17:04:46 +01:00
Eelco Dolstra 146af4ee9b Move sodium_init() call 2021-01-06 16:43:09 +01:00
Maximilian Bosch 3a63fc6cd5
Allow substituting paths when building remotely using ssh-ng://
Until now, it was not possible to substitute missing paths from e.g.
`https://cache.nixos.org` on a remote server when building on it using
the new `ssh-ng` protocol.

This is because every store implementation except legacy `ssh://`
ignores the substitution flag passed to `Store::queryValidPaths` while
the `legacy-ssh-store` substitutes the remote store using
`cmdQueryValidPaths` when the remote store is opened with `nix-store
--serve`.

This patch slightly modifies the daemon protocol to allow passing an
integer value suggesting whether to substitute missing paths during
`wopQueryValidPaths`. To implement this on the daemon-side, the
substitution logic from `nix-store --serve` has been moved into a
protected method named `Store::substitutePaths` which gets currently
called from `LocalStore::queryValidPaths` and `Store::queryValidPaths`
if `maybeSubstitute` is `true`.

Fixes #2770
2020-11-05 20:12:37 +01:00
Eelco Dolstra e8c379555f LocalStore: Get rid of recursive_mutex 2020-11-03 14:45:24 +01:00
John Ericson 2546c63373 Merge commit 'f66bbd8c7bb1472facf8917e58e3cd4f6ddfa1b5' into auto-uid-allocation 2020-10-17 17:25:17 +00:00
Eelco Dolstra 85c8be6286 Remove static variable name clashes
This was useful for an experiment with building Nix as a single
compilation unit. It's not very useful otherwise but also doesn't
hurt...
2020-10-06 13:49:20 +02:00
John Ericson b759701652 nix::worker_proto -> worker_proto 2020-09-30 00:41:18 +00:00
John Ericson 45a0ed82f0 Revert "Use template structs instead of phantoms"
This reverts commit 9ab07e99f5.
2020-09-30 00:39:06 +00:00
John Ericson b92d3b2edd Merge remote-tracking branch 'upstream/master' into templated-daemon-protocol 2020-09-22 00:45:55 +00:00
John Ericson e7d93e7ece Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-09-03 15:43:17 +00:00
Eelco Dolstra 82b77a7726
addPermRoot(): Remove indirect flag 2020-09-03 11:29:34 +02:00
Eelco Dolstra 00d25e8457
Remove the --indirect flag
All GC roots are now indirect.
2020-09-03 11:22:00 +02:00
John Ericson 45a2f1baab Rename drv output querying functions, like master
- `queryDerivationOutputMapAssumeTotal` -> `queryPartialDerivationOutputMap`
 - `queryDerivationOutputMapAssumeTotal` -> `queryDerivationOutputMap
2020-08-20 18:14:12 +00:00
John Ericson be0d429b95 Merge branch 'master' of github.com:NixOS/nix into templated-daemon-protocol 2020-08-19 03:17:41 +00:00