Commit graph

8677 commits

Author SHA1 Message Date
Théophane Hufschmitt b8a0e9a9b8 Move the getBuildLog implementation to its own implementation file
Keep the header minimal and clean
2023-01-13 11:05:44 +01:00
Théophane Hufschmitt bdeb6de889
Merge pull request #7430 from tweag/ca/fix-nix-log
Ca/fix nix log
2023-01-13 11:00:56 +01:00
John Ericson d8512653d4 Write more (extended) output spec tests 2023-01-12 22:05:55 -05:00
John Ericson d29eb08563 Assert on construction that OutputsSpec::Names is non-empty 2023-01-12 20:52:29 -05:00
John Ericson e947aa5401 Unit test OuputsSpec::{union_, isSubsetOf} 2023-01-12 20:33:50 -05:00
John Ericson 31875bcfb7 Split OutputsSpec::merge into OuputsSpec::{union_, isSubsetOf}
Additionally get rid of the evil time we made an empty
`OutputSpec::Names()`.
2023-01-12 20:20:27 -05:00
Cole Helbling 1fc74afbba nix/show-config: allow getting the value of a specific setting
Instead of needing to run `nix show-config --json | jq -r
'."warn-dirty".value'` to view the value of `warn-dirty`, you can now
run `nix show-config warn-dirty`.
2023-01-12 13:56:35 -08:00
Théophane Hufschmitt eaa20f2574
Merge pull request #7590 from fricklerhandwerk/remove-unnecessary-cast
remove unncessary cast
2023-01-12 14:00:43 +01:00
Valentin Gagarin 48b2a3a0d0 remove unncessary cast 2023-01-12 13:23:32 +01:00
John Ericson 0faf5326bd Improve tests for OutputsSpec 2023-01-11 19:09:21 -05:00
John Ericson 5ba6e5d0d9 Remove default constructor from OutputsSpec
This forces us to be explicit.

It also requires to rework how `from_json` works. A `JSON_IMPL` is added
to assist with this.
2023-01-11 19:08:19 -05:00
John Ericson 114a6e2b09 Make it hard to construct an empty OutputsSpec::Names
This should be a non-empty set, and so we don't want people doing this
by accident. We remove the zero-0 constructor with a little inheritance
trickery.
2023-01-11 19:08:19 -05:00
John Ericson 8a3b1b7ced Simplify and document store path installable parsing 2023-01-11 19:08:19 -05:00
John Ericson ce2f91d356 Split OutputsSpec and ExtendedOutputsSpec, use the former more
`DerivedPath::Built` and `DerivationGoal` were previously using a
regular set with the convention that the empty set means all outputs.
But it is easy to forget about this rule when processing those sets.
Using `OutputSpec` forces us to get it right.
2023-01-11 18:57:18 -05:00
John Ericson a7c0cff07f Rename OutputPath -> ExtendedOutputPath
Do this prior to making a new more limitted `OutputPath` we will use in
more places.
2023-01-11 18:55:29 -05:00
John Ericson a8f45b5e5a Improve OutputsSpec slightly
A few little changes preparing for the rest.
2023-01-11 18:54:50 -05:00
Eelco Dolstra 9fc8d00d74 MonitorFdHup: Make it work on macOS again
It appears that on current macOS versions, our use of poll() to detect
client disconnects no longer works. As a workaround, poll() for
POLLRDNORM, since this *will* wake up when the client has
disconnected. The downside is that it also wakes up when input is
available. So just sleep for a bit in that case.  This means that on
macOS, a client disconnect may take up to a second to be detected,
but that's better than not being detected at all.

Fixes #7584.
2023-01-11 10:48:40 -08:00
Eelco Dolstra 7515617ad0 Backport getLine tests from lazy-trees 2023-01-11 13:49:39 +01:00
Théophane Hufschmitt a3ba80357d
Merge pull request #7543 from obsidiansystems/typed-string-context
Parse string context elements properly
2023-01-11 07:09:37 +01:00
Robert Hensing f58c301112
Merge pull request #7541 from hercules-ci/check-manual-links
Check links in the manual
2023-01-10 23:07:38 +01:00
Robert Hensing fefa3a49ce doc/manual: Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-01-10 22:30:41 +01:00
Robert Hensing e79f935718 doc/manual: Fix broken internal links
The targets I could find.
2023-01-10 22:30:41 +01:00
Robert Hensing 34a1e0d29b doc/manual: Introduce @docroot@ as a stable base for includable snippets
This way the links are clearly within the manual (ie not absolute paths),
while allowing snippets to reference the documentation root reliably,
regardless of at which base url they're included.
2023-01-10 22:30:41 +01:00
John Ericson 5576d5e987 Parse string context elements properly
Prior to this change, we had a bunch of ad-hoc string manipulation code
scattered around. This made it hard to figure out what data model for
string contexts is.

Now, we still store string contexts most of the time as encoded strings
--- I was wary of the performance implications of changing that --- but
whenever we parse them we do so only through the
`NixStringContextElem::parse` method, which handles all cases. This
creates a data type that is very similar to `DerivedPath` but:

 - Represents the funky `=<drvpath>` case as properly distinct from the
   others.

 - Only encodes a single output, no wildcards and no set, for the
   "built" case.

(I would like to deprecate `=<path>`, after which we are in spitting
distance of `DerivedPath` and could maybe get away with fewer types, but
that is another topic for another day.)
2023-01-10 13:10:49 -05:00
John Ericson da64f026dd Make clear that StorePathWithOutputs is a deprecated type
- Add a comment

- Put `OutputsSpec` in a different header (First part of #6815)

- Make a few stray uses of it in new code use `DerivedPath` instead.
2023-01-10 11:27:19 -05:00
Eelco Dolstra 59cc920cc0 Add a FIXME 2023-01-10 15:20:30 +01:00
Eelco Dolstra 7f1af270dd Clean up toDerivedPaths() logic 2023-01-10 15:08:46 +01:00
Eelco Dolstra 1123c42f90
Apply suggestions from code review
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2023-01-10 14:57:35 +01:00
Eelco Dolstra b4dc68f0be Show string in error message 2023-01-10 14:56:03 +01:00
Eelco Dolstra b80e4b57da ExtraInfo -> ExtraPathInfo 2023-01-10 14:52:49 +01:00
Eelco Dolstra 8e923bf4c5 Merge remote-tracking branch 'origin/master' into fix-7417 2023-01-10 14:35:06 +01:00
Will Bush 05b13aff3d
Fix typo in example for builtin function map 2023-01-06 23:04:43 -06:00
Théophane Hufschmitt 8d88c3b347
Merge pull request #7307 from hercules-ci/derivation-goal-improve-comment
libstore/derivation-goal: Elaborate a TODO for performance concern
2023-01-06 13:07:57 +01:00
Eelco Dolstra 420ccecc1e
Merge pull request #7557 from NixOS/fix-7529
On macOS with auto-uid-allocation and sandboxing, use the correct gid
2023-01-06 12:35:55 +01:00
Naïm Favier f1ee4ece80
Don't check NixOS modules
NixOS modules can be paths. Rather than dig further down into the layer
violation, don't check anything specific to NixOS modules.
2023-01-05 18:23:30 +01:00
Eelco Dolstra 3a98107170
Merge pull request #7542 from edolstra/gc-deadlock
Fix deadlock between auto-GC and addTempRoot()
2023-01-05 17:08:23 +01:00
Eelco Dolstra 0fe2b222d5
Merge pull request #7539 from tweag/fix-nix-why-depends--derivation
Fix `nix why-depends --derivation`
2023-01-05 15:32:04 +01:00
Eelco Dolstra 4e84b532ed On macOS with auto-uid-allocation and sandboxing, use the correct gid
macOS doesn't have user namespacing, so the gid of the builder needs
to be nixbld. The logic got "has sandboxing enabled" confused with
"has user namespaces".

Fixes #7529.
2023-01-05 04:58:55 -08:00
Eelco Dolstra 6991e558dd Move macOS sandbox files to sr/libstore/build 2023-01-04 04:50:45 -08:00
Eelco Dolstra 609a7dc059 Include macOS sandbox files in the Nix binary
This basically reverts 6e5165b773.
It fixes errors like

  sandbox-exec: <internal init prelude>:292:47: unable to open sandbox-minimal.sb: not found

when trying to run a development Nix installed in a user's home
directory.

Also, we're trying to minimize the number of installed files
to make it possible to deploy Nix as a single statically-linked
binary.
2023-01-04 04:36:07 -08:00
Naïm Favier 3c968191f1
move unsafeDiscardReferences out of outputChecks
It's not a check.
2023-01-03 18:53:01 +01:00
Eelco Dolstra d4d1ca8b11 nix --version: Print the data directory 2023-01-03 08:30:49 -08:00
Naïm Favier 15f7fa59be
unsafeDiscardReferences
Adds a new boolean structured attribute
`outputChecks.<output>.unsafeDiscardReferences` which disables scanning
an output for runtime references.

    __structuredAttrs = true;
    outputChecks.out.unsafeDiscardReferences = true;

This is useful when creating filesystem images containing their own embedded Nix
store: they are self-contained blobs of data with no runtime dependencies.

Setting this attribute requires the experimental feature
`discard-references` to be enabled.
2023-01-03 17:19:16 +01:00
Eelco Dolstra 28d5b5cd45 Fix deadlock between auto-GC and addTempRoot()
Previously addTempRoot() acquired the LocalStore state lock and waited
for the garbage collector to reply. If the garbage collector is in the
same process (as it the case with auto-GC), this would deadlock as
soon as the garbage collector thread needs the LocalStore state lock.

So now addTempRoot() uses separate Syncs for the state that it
needs. As long at the auto-GC thread doesn't call addTempRoot() (which
it shouldn't), it shouldn't deadlock.

Fixes #3224.
2023-01-03 15:20:21 +01:00
Eelco Dolstra 224b56f10e Move creation of the temp roots file into its own function
This also moves the file handle into its own Sync object so we're not
holding the _state while acquiring the file lock. There was no real
deadlock risk here since locking a newly created file cannot block,
but it's still a bit nicer.
2023-01-03 14:51:23 +01:00
Eelco Dolstra ae31b5f50f
Merge pull request #7497 from rski/master
src/libstore: Print the reason opening the DB failed
2023-01-03 12:44:14 +01:00
Eelco Dolstra d33d15a48b Put the --show-trace hint in the logical place 2023-01-02 20:53:58 +01:00
Eelco Dolstra c548e35498 Don't use state.positions[noPos]
This caused traces 'at «none»:0: (source not available)'.
2023-01-02 20:53:58 +01:00
Eelco Dolstra 6b69652385 Merge remote-tracking branch 'origin/master' into coerce-string 2023-01-02 20:53:39 +01:00
Théophane Hufschmitt 8cac451fce Fix why-depends for CA derivations (again)
This has the same goal as b13fd4c58e81b2b2b0d72caa5ce80de861622610,but
achieves it in a different way in order to not break
`nix why-depends --derivation`.
2023-01-02 17:42:22 +01:00
Théophane Hufschmitt 105d74eb81 Revert "Fix why-depends for CA derivations"
This reverts commit b13fd4c58e.
2023-01-02 15:44:04 +01:00
Théophane Hufschmitt 9af16c5f74
Merge pull request #5941 from hercules-ci/optimize-intersectAttrs
Optimize intersectAttrs performance
2023-01-02 15:22:38 +01:00
Valentin Gagarin e0c4a95611 antiquotation -> string interpolation
as proposed by @mkaito[1] and @tazjin[2] and discussed with @edolstra
and Nix maintainers

[1]: https://github.com/NixOS/nix.dev/pull/267#issuecomment-1270076332
[2]: https://github.com/NixOS/nix.dev/pull/267#issuecomment-1270201979

Co-authored-by: John Ericson <git@JohnEricson.me>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2023-01-02 14:38:57 +01:00
Théophane Hufschmitt fb8fc6fda6
Merge pull request #7478 from hercules-ci/make-sure-initNix-called
libstore: Make sure that initNix has been called
2023-01-02 14:12:49 +01:00
Valentin Gagarin 9cdf8ededb remove redundant re-definition of store derivations 2023-01-02 13:37:59 +01:00
Théophane Hufschmitt cfd6c7fc9b
Merge pull request #7485 from fricklerhandwerk/doc-store-derivation
define "store derivation"
2023-01-02 13:26:41 +01:00
Théophane Hufschmitt b3285c7722
Merge pull request #7351 from NaN-git/fix-mkString
cleanup eval.hh/eval.cc
2023-01-02 11:41:52 +01:00
Théophane Hufschmitt e8a3e58171
Merge pull request #7521 from ncfavier/migration-deadlock
Release shared lock before acquiring exclusive lock
2023-01-02 11:08:43 +01:00
Steven Shaw 84b0893725
Fix error message 2023-01-01 12:37:43 +10:00
Eelco Dolstra 8c52f8ea9d
Merge pull request #7524 from ncfavier/sandbox-paths-closure
doc: sandbox-paths computes closures
2022-12-29 19:45:51 +01:00
Eelco Dolstra 8227fe819e
Merge pull request #7504 from edolstra/nix-develop-personality
nix develop: Set personality
2022-12-29 19:42:45 +01:00
Naïm Favier d5d2f50ebb
doc: sandbox-paths computes closures 2022-12-28 17:09:20 +01:00
Naïm Favier 81c3f99b36
Release shared lock before acquiring exclusive lock
In principle, this should avoid deadlocks where two instances of Nix are
holding a shared lock on big-lock and are both waiting to get an
exclusive lock.

However, it seems like `flock(2)` is supposed to do this automatically,
so it's not clear whether this is actually where the problem comes from.
2022-12-27 15:58:14 +01:00
Robert Hensing 336908cf4c Optimize intersectAttrs performance
Always traverse the shortest set.
2022-12-24 14:51:05 +01:00
Robert Hensing aba6eb348e libstore: Make sure that initNix has been called
Prevent bugs like https://github.com/cachix/cachix/pull/477
2022-12-24 14:39:30 +01:00
Philipp Otterbein 8af839f48c remove undefined function 2022-12-24 12:19:53 +01:00
Philipp Otterbein a6e9d9cb2f remove function makeImmutableStringWithLen 2022-12-24 12:09:06 +01:00
Eelco Dolstra c164d304f3 nix develop: Set personality
This makes 'nix develop' set the Linux personality in the same way
that the actual build does, allowing a command like 'nix develop
nix#devShells.i686-linux.default' on x86_64-linux to work correctly.
2022-12-23 16:33:55 +01:00
Eelco Dolstra 14f7dae3e4
Merge pull request #7503 from edolstra/fix-dirOf
Fix CanonPath::dirOf() returning a string_view of a temporary
2022-12-23 16:17:48 +01:00
Eelco Dolstra 64c60f7241 Fix CanonPath::dirOf() returning a string_view of a temporary
https://hydra.nixos.org/build/202837872
2022-12-23 15:32:54 +01:00
rski d034ed1891 src/libstore: Print the reason opening the DB failed
Without this, the error is lost, and it makes for a hard to debug
situation. Also remove some of the busyness inside the sqlite_open_v2
args.

The errcode returned is not the extended one. The only way to make open
return an extended code, would be to add SQLITE_OPEN_EXRESCODE to the
flags. In the future it might be worth making this change,
which would also simplify the existing SQLiteError code.
2022-12-23 02:55:51 +02:00
Eelco Dolstra c9eee5a84d
Merge pull request #7493 from mupdt/primops-storeDir-test-non-standard-path
primops `storeDir` test uses `settings.nixStore`
2022-12-22 16:18:48 +01:00
mupdt a33e45b60b primops storeDir test uses settings.nixStore 2022-12-21 07:01:57 -05:00
Valentin Gagarin 7797661a70 link "store derivation" to glossary definition 2022-12-21 11:42:50 +01:00
Valentin Gagarin 3a66d82e1d update description of "store derivation" in installables section
a store derivation is not a store path itself, it has a store path.
2022-12-21 11:42:22 +01:00
mupdt bc8ab21c5a [PDT] TDE-3114: prevent a race-condition when creating the S3 cache 2022-12-21 04:50:40 -05:00
Eelco Dolstra 5c97b5a398 InstallableFlake::toDerivedPaths(): Support paths and store paths
This makes 'nix build' work on paths (which will be copied to the
store) and store paths (returned as is). E.g. the following flake
output attributes can be built using 'nix build .#foo':

  foo = ./src;
  foo = self.outPath;
  foo = builtins.fetchTarball { ... };
  foo = (builtins.fetchTree { .. }).outPath;
  foo = builtins.fetchTree { .. } + "/README.md";
  foo = builtins.storePath /nix/store/...;

Note that this is potentially risky, e.g.

  foo = /.;

will cause Nix to try to copy the entire file system to the store.

What doesn't work yet:

  foo = self;
  foo = builtins.fetchTree { .. };

because we don't handle attrsets with an outPath attribute in it yet,
and

  foo = builtins.storePath /nix/store/.../README.md;

since result symlinks have to point to a store path currently (rather
than a file inside a store path).

Fixes #7417.
2022-12-20 15:11:44 +01:00
Eelco Dolstra bda879170f EvalState::copyPathToStore(): Return a StorePath 2022-12-20 14:58:39 +01:00
Eelco Dolstra 845fc3f605 Merge toDerivations() into toDerivedPaths()
toDerivedPaths() now returns DerivedPathWithInfo, which is DerivedPath
with some attributes needed by 'nix profile' etc.

Preparation for #7417.
2022-12-20 14:24:14 +01:00
Eelco Dolstra 8332ac6a1d
Merge pull request #7451 from edolstra/abstract-pos
Introduce AbstractPos
2022-12-20 12:55:13 +01:00
Théophane Hufschmitt 000dd77d8d
Merge pull request #7473 from hercules-ci/sqlite-error
Improve sqlite error messages
2022-12-20 11:27:30 +01:00
Théophane Hufschmitt c3d522dc51
Merge pull request #7480 from nrdxp/fix-develop-drv
fix(develop): make `nix develop` drv recreatable
2022-12-20 07:41:46 +01:00
Timothy DeHerrera 94cf0da7b2
fix(develop): make nix develop drv recreatable 2022-12-19 13:16:06 -07:00
Domen Kožar f118e661e0
Merge pull request #7462 from rapenne-s/improve_messaging_reusing_trusted_substituter
Inform user instead of warning them when using a trusted substituter
2022-12-19 16:04:01 +00:00
Eelco Dolstra d00bfe4833
Merge pull request #7450 from edolstra/canon-path
Add CanonPath wrapper to represent canonicalized paths
2022-12-19 16:21:11 +01:00
Robert Hensing c965f35de7 Improve sqlite error messages
They did not include the detailed error message, losing essential
information for troubleshooting.

Example message:

    warning: creating statement 'insert or rplace into NARs(cache, hashPart, namePart, url, compression, fileHash, fileSize, narHash, narSize, refs, deriver, sigs, ca, timestamp, present) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 1)': at offset 10: SQL logic error, near "rplace": syntax error (in '/tmp/nix-shell.grQ6f7/nix-test/tests/binary-cache/test-home/.cache/nix/binary-cache-v6.sqlite')

It's not the best example; more important information will be in
the message for e.g. a constraint violation.

I don't see why this specific error is printed as a warning, but
that's for another commit.
2022-12-17 14:51:37 +01:00
Solène Rapenne d0660c6c0b
printMsg replacement by printInfo
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-12-16 09:34:22 +01:00
Taeer Bar-Yam e5eb05c599 getBuildLog: factor out resolving derivations 2022-12-15 15:58:54 -05:00
Eelco Dolstra 0687e16c4a Fix a crash in DerivedPath::Built::toJSON() with impure derivations
The use of 'nullptr' here didn't result in a null JSON value, but in a
nullptr being cast to a string, which aborts.
2022-12-15 16:02:27 +01:00
Solène Rapenne 98e01da0b1 warnings: switch to info level when using a saved substituter 2022-12-14 14:11:21 +01:00
Naïm Favier 1f3c0a3c1d
Allow disabling build users by unsetting build-users-group
Unsetting `build-users-group` (without `auto-allocate-uids` enabled)
gives the following error:

```
src/libstore/lock.cc:25: static std::unique_ptr<nix::UserLock> nix::SimpleUserLock::acquire(): Assertion `settings.buildUsersGroup != ""' failed.
```

Fix the logic in `useBuildUsers` and document the default value
for `build-users-group`.
2022-12-14 00:40:30 +01:00
Eelco Dolstra 46b3c026fc
Merge pull request #7455 from rapenne-s/documentation_fix
Add anchors and links to definition
2022-12-13 17:26:52 +01:00
Eelco Dolstra e2a4e7aecd
Merge pull request #4543 from obsidiansystems/indexed-store-path-outputs
Low level `<drvPath>^<outputName>` installable syntax to match existing `<highLevelInstallable>^<outputNames>` syntax
2022-12-13 17:22:29 +01:00
Eelco Dolstra c9b0a85b08 Restore display of source lines for stdin/string inputs 2022-12-13 16:00:44 +01:00
Eelco Dolstra aea97f07a3 Fix compilation 2022-12-13 15:23:12 +01:00
Eelco Dolstra 1315133b50
Improve cast safety
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-12-13 12:38:33 +01:00
Solène Rapenne 09860c16ce documentation: use sections instead of list items 2022-12-13 11:47:44 +01:00
Solène Rapenne e43b0f5b12 documentation: link flake URL term to definition 2022-12-13 11:47:38 +01:00
Solène Rapenne ae27181f16 documentation: fix link to definition 2022-12-13 11:47:22 +01:00
Théophane Hufschmitt 6e31d27cba
Merge pull request #6741 from Mindavi/nix-no-gc
support building with --enable-gc=no
2022-12-13 10:36:59 +01:00
Eelco Dolstra b3fdab28a2 Introduce AbstractPos
This makes the position object used in exceptions abstract, with a
method getSource() to get the source code of the file in which the
error originated. This is needed for lazy trees because source files
don't necessarily exist in the filesystem, and we don't want to make
libutil depend on the InputAccessor type in libfetcher.
2022-12-13 00:50:43 +01:00
John Ericson f61d575810 Merge branch 'indexed-store-path-outputs' of github.com:obsidiansystems/nix into indexed-store-path-outputs 2022-12-12 17:43:10 -05:00
John Ericson 5273cf4c97 Merge remote-tracking branch 'upstream/master' into indexed-store-path-outputs 2022-12-12 17:40:49 -05:00
John Ericson 32ae715db1
Fix typos in the docs
Thanks!

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-12-12 17:37:45 -05:00
John Ericson dabb03b8d0 Merge remote-tracking branch 'upstream/master' into indexed-store-path-outputs 2022-12-12 17:36:02 -05:00
John Ericson d8c1c24c78 Adjust docs 2022-12-12 17:32:24 -05:00
Rick van Schijndel 672ee88231 support building with --enable-gc=no
Some minor changes fixing the build without boehm.
Fixes NixOS#6250
2022-12-12 23:31:30 +01:00
John Ericson dc075dcdd0
Apply suggestions from code review
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-12-12 16:26:10 -05:00
Eelco Dolstra 900b854084 Add CanonPath wrapper to represent canonicalized paths 2022-12-12 19:57:32 +01:00
Florian Friesdorf 8618c6cc75 Simplify loop, feedback from @tfc and @Ericson2314 2022-12-12 18:41:00 +00:00
John Ericson 7b122d43a4 Fix stack context notes to not rely on order
Make everything be in the form "while ..." (most things were already),
and in particular *don't* use other propositions that must go after or
before specific "while ..." clauses to make sense.
2022-12-12 18:41:00 +00:00
Florian Friesdorf d269976be6 Show stack trace above error message
Save developers from scrolling by displaying the error message last,
below the stack trace.
2022-12-12 18:41:00 +00:00
Florian Friesdorf 173dcb0af9 Don't reverse stack trace when showing
When debugging nix expressions the outermost trace tends to be more useful
than the innermost. It is therefore printed last to save developers from
scrolling.
2022-12-12 18:41:00 +00:00
Eelco Dolstra e408af82ab
Merge pull request #7436 from edolstra/enable-lang-tests
Enable some language tests that were accidentally disabled
2022-12-12 17:39:02 +01:00
Eelco Dolstra 17f81d3215 Fix unused variable warning 2022-12-12 16:41:46 +01:00
Benoit de Chezelles a456630a5a Allow to disable global flake-registry with "" 2022-12-12 15:32:02 +01:00
Eelco Dolstra 7a85199f87 Add docs from the lazy-trees branch 2022-12-12 14:06:13 +01:00
Eelco Dolstra fd0ed75118 Support flake references in the old CLI
Fixes #7026.
2022-12-12 14:05:52 +01:00
Eelco Dolstra ae5f62a894 Move isUri() and resolveUri() out of filetransfer.cc
These are purely related to NIX_PATH / -I command line parsing, so put
them in libexpr.
2022-12-12 14:05:35 +01:00
Eelco Dolstra 7396844676
Merge pull request #7421 from edolstra/lazy-trees-trivial-changes
Trivial changes from the lazy-trees branch
2022-12-12 13:52:56 +01:00
John Ericson 1879c7c95e
Merge branch 'master' into indexed-store-path-outputs 2022-12-12 07:33:36 -05:00
Eelco Dolstra e558e089ba -I description: Use -I examples 2022-12-12 12:51:23 +01:00
Eelco Dolstra 877ea1dab8
Use get_ptr()
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-12-12 12:46:13 +01:00
Eelco Dolstra 037d5c4299
Manual improvements
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-12-12 12:43:44 +01:00
Eelco Dolstra 786402365e Cleanup 2022-12-12 12:40:51 +01:00
Eelco Dolstra f3d1e92856
Update URL
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-12-12 12:37:55 +01:00
Eelco Dolstra 8272cd9dec
Optimize string concatenation
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-12-12 12:36:19 +01:00
Andreas Rammhold dbc8547664 Ignore the enforceDeterminism value
We used to set enforceDeterminism to true in the settings (by default)
and thus did send a non-zero value over the wire. The value should
probably be ignored as it should only matter if nrRounds is non-zero
as well.

Having the old code here where the value is expected to be zero only
works with the same version of Nix where we are sending zero. We
should always test this against older Nix versions being client or
server as otherwise upgrade in larger networks might be a pain.

Fixes 8e0946e8df
2022-12-10 17:55:07 +01:00
Théophane Hufschmitt 2affb19c92
Merge pull request #7409 from tweag/fix-6383
check the store for input before failing (hopefully fix #6383)
2022-12-09 06:33:30 +01:00
Taeer Bar-Yam 3b27181ee5 fix missing function after rebase 2022-12-08 16:59:21 -05:00
regnat 04b113f6cb Fix nix log with CA derivations
Fix #6209

When trying to run `nix log <installable>`, try first to resolve the derivation pointed to
by `<installable>` as it is the resolved one that holds the build log.

This has a couple of shortcomings:
1. It’s expensive as it requires re-reading the derivation
2. It’s brittle because if the derivation doesn’t exist anymore or can’t
   be resolved (which is the case if any one of its build inputs is missing),
   then we can’t access the log anymore

However, I don’t think we can do better (at least not right now).
The alternatives I see are:
1. Copy the build log for the un-resolved derivation. But that means a
   lot of duplication
2. Store the results of the resolving in the db. Which might be the best
   long-term solution, but leads to a whole new class of potential
   issues.
2022-12-08 16:03:20 -05:00
Eelco Dolstra 703d863a48 Trivial changes from the lazy-trees branch 2022-12-07 14:06:34 +01:00
amesgen a5ab253e8f
Mention host parameter for GitHub 2022-12-07 12:38:18 +01:00
Linus Heckemann 8e0946e8df Remove repeat and enforce-determinism options
These only functioned if a very narrow combination of conditions held:

- The result path does not yet exist (--check did not result in
  repeated builds), AND
- The result path is not available from any configured substituters, AND
- No remote builders that can build the path are available.

If any of these do not hold, a derivation would be built 0 or 1 times
regardless of the repeat option. Thus, remove it to avoid confusion.
2022-12-07 11:36:48 +01:00
Taeer Bar-Yam 1c8de7d3d0 improve style 2022-12-06 11:25:38 -05:00
Eelco Dolstra 54906bc93c
Merge pull request #7382 from fricklerhandwerk/doc-automatic-uid
move documentation on `auto-allocate-uids` to options docs
2022-12-06 11:31:34 +01:00
amesgen 105677140d
Document gitlab input scheme 2022-12-06 10:36:39 +01:00
Eelco Dolstra 484578d3f9
Tweak option descriptions 2022-12-06 10:30:36 +01:00
Taeer Bar-Yam 8c7661da09 check the store for input before failing (hopefully fix #6383) 2022-12-05 23:22:38 -05:00
Théophane Hufschmitt bfcf30f0ab
Merge pull request #7390 from Radvendii/fix-6383
check the store for input before failing (hopefully fix #6700)
2022-12-05 18:04:41 +01:00
Taeer Bar-Yam e4f9f3bf24 check the store for input before failing (hopefully fix #6700) 2022-12-05 11:27:47 -05:00
Eelco Dolstra e0ab2069c9
Consistent capitalisation 2022-12-05 16:55:55 +01:00
Jörg Thalheim cccd57c022 getMaxCPU: fix cgroup path
Given this typo I am not sure if it has been tested.
2022-12-04 18:22:12 +01:00
Eelco Dolstra c582150360
Merge pull request #7394 from edolstra/fix-7268
Lower verbosity of exceptions in getMaxCPU()
2022-12-02 17:02:32 +01:00
Eelco Dolstra 3a8341f57e
Merge pull request #7395 from obsidiansystems/evaluating-to-calling
Change "while evaluating <fun>" to "while *calling*" in trace
2022-12-02 15:53:57 +01:00
John Ericson 19c5394971 Change "while evaluating <fun>" to "while *calling*" in trace
The old way was not correct.

Here is an example:
```
 $ nix-instantiate --eval --expr 'let x = a: throw "asdf"; in x 1' --show-trace
error: asdf

       … while evaluating 'x'

       at «string»:1:9:

            1| let x = a: throw "asdf"; in x 1
             |         ^

       … from call site

       at «string»:1:29:

            1| let x = a: throw "asdf"; in x 1
             |                             ^
```
and yet also:
```
 $ nix-instantiate --eval --expr 'let x = a: throw "asdf"; in x' --show-trace
<LAMBDA>
```

Here is the thing: in both cases we are evaluating `x`!

Nix is a higher-order languages, and functions are a sort of value. When
we write `x = a: ...`, `a: ...` is the expression that `x` is being
defined to be, and that is already a value. Therefore, we should *never*
get an trace that says "while evaluating `x`", because evaluating `a:
...` is *trival* and nothing happens during it!

What is actually happening here is we are applying `x` and evaluating
its *body* with arguments substituted for parameters. I think the
simplest way to say is just "while *calling* `x`", and so that is what I
changed it to.
2022-12-02 09:14:12 -05:00
Eelco Dolstra fa99ef6a87 getMaxCPU(): Lower verbosity level for ignored exceptions
Fixes #7268.
2022-12-02 15:03:40 +01:00
Valentin Gagarin 0ce5742bec
Merge pull request #7381 from aakropotkin/doc-ltoa-conflicts
doc: listToAttrs: document repeated keys
2022-12-02 14:19:34 +01:00
Eelco Dolstra 1e6a5d1ff6 Clean up cgroup handling in getMaxCPU()
Also, don't assume in LocalDerivationGoal that cgroups are mounted on
/sys/fs/cgroup.
2022-12-02 12:59:13 +01:00
Eelco Dolstra 1211e59a03 Move cgroup.{cc,hh} to libutil 2022-12-02 12:38:03 +01:00
Alex Ameen ef524013aa doc: listToAttrs: add extra whitespace 2022-12-01 10:32:45 -06:00
Eelco Dolstra f1e1ba9fe0 Really fix 'nix store make-content-addressed --json'
https://hydra.nixos.org/log/mcgypcf9vj4n8vdmw7lj3l05c899v73w-nix-2.12.0pre20221201_16b03f0-x86_64-unknown-linux-musl.drv
2022-12-01 16:29:09 +01:00
Alex Ameen ec18b7d09b doc: listToAttrs: fix line wrapping 2022-11-30 23:21:09 -06:00
Alex Ameen ad46726546 doc: listToAttrs: document repeated keys 2022-11-30 22:53:41 -06:00
Valentin Gagarin 0ea62670ed move documentation on auto-allocate-uids to options docs
this is where it belongs and can be found together with the other
options.
2022-12-01 04:40:02 +01:00
Eelco Dolstra 0b092bd87f nix store make-content-addressed: Fix JSON construction
Fixes

  error: [json.exception.type_error.301] cannot create object from initializer list

in tests/fetchClosure.sh.
2022-11-30 13:46:33 +01:00
Eelco Dolstra fbc53e97ed
Merge pull request #3600 from NixOS/auto-uid-allocation
Automatic UID allocation
2022-11-29 14:01:42 +01:00
Eelco Dolstra 4f762e2b02 Restore ownership of / for non-uid-range builds 2022-11-29 13:10:53 +01:00
Eelco Dolstra af8a32143b
Merge pull request #7358 from ncfavier/repl-exit-newline
repl: print a newline on ctrl-D
2022-11-29 11:31:43 +01:00
Eelco Dolstra 67bcb99700 Add a setting for enabling cgroups 2022-11-28 21:54:02 +01:00
Eelco Dolstra ff12d1c1a1 Check that auto-allocated UIDs don't clash with existing accounts 2022-11-28 20:49:17 +01:00
Eelco Dolstra dbf78a7ada
Merge pull request #7313 from yorickvP/nlohmann-everywhere
Replace src/libutil/json.cc with nlohmann
2022-11-28 15:03:48 +01:00
Naïm Favier 9b35cc716b
use logger->cout
in order to avoid potential problems with the progress bar

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-11-28 14:59:06 +01:00
Naïm Favier 04ec157517
repl: print a newline on ctrl-D 2022-11-28 10:38:23 +01:00
Eelco Dolstra 5b798f6cae Fix random client failures during GC server shutdown
We need to close the GC server socket before shutting down the active
GC client connections, otherwise a client may (re)connect and get
ECONNRESET. But also handle ECONNRESET for resilience.

Fixes random failures like

  GC socket disconnected
  connecting to '/tmp/nix-shell.y07M0H/nix-test/default/var/nix/gc-socket/socket'
  sending GC root '/tmp/nix-shell.y07M0H/nix-test/default/store/kb5yzija0f1x5xkqkgclrdzldxj6nnc6-non-blocking'
  reading GC root from client: error: unexpected EOF reading a line
  1 store paths deleted, 0.00 MiB freed
  error: reading from file: Connection reset by peer

in gc-non-blocking.sh.
2022-11-27 12:57:18 +01:00
Eelco Dolstra 0b4c4d7434 Don't use GC_STRNDUP
It calls strlen() on the input (rather than simply copying at most
`size` bytes), which can fail if the input is not zero-terminated and
is inefficient in any case.

Fixes #7347.
2022-11-25 22:30:56 +01:00
John Ericson 26534f141c
Merge branch 'master' into indexed-store-path-outputs 2022-11-25 08:14:32 -05:00
Théophane Hufschmitt bc9692a6b7
Merge pull request #7337 from Radvendii/why-depends-ca
Fix why-depends for CA derivations
2022-11-23 20:16:14 +01:00
Taeer Bar-Yam bd8571a5c3 add explanation and test 2022-11-23 12:06:47 -05:00
Taeer Bar-Yam b13fd4c58e Fix why-depends for CA derivations
why-depends assumed that we knew the output path of the second argument.
For CA derivations, we might not know until it's built. One way to solve
this would be to build the second installable to get the output path.

In this case we don't need to, though. If the first installable (A)
depends on the second (B), then getting the store path of A will
necessitate having the store path B. The contrapositive is, if the store
path of B is not known (i.e. it's a CA derivation which hasn't been
built), then A does not depend on B.
2022-11-23 11:39:50 -05:00
Eelco Dolstra 6292d5616e Merge remote-tracking branch 'origin/master' into auto-uid-allocation 2022-11-23 11:16:09 +01:00
Eelco Dolstra 05d0892443
Merge pull request #7328 from edolstra/nix-build-stats
nix build --json: Include build statistics
2022-11-22 14:41:15 +01:00
Eelco Dolstra 3d23b9d032 SimpleUserLock::getSupplementaryGIDs(): Filter out main gid
This avoids having the user's gid in the supplementary group list as
well.
2022-11-22 10:26:17 +01:00
Eelco Dolstra b37c2d84b6 Always call setgroups()
We shouldn't skip this if the supplementary group list is empty,
because then the sandbox won't drop the supplementary groups of the
parent (like "root").
2022-11-22 10:26:17 +01:00
Eelco Dolstra 02c02ee7c3
Merge pull request #6456 from amjoseph-nixpkgs/seccomp-mips
local-derivation-goal.cc: enable seccomp filters for mips{32,64}
2022-11-21 23:03:00 +01:00
Eelco Dolstra c776dfbb35
Use hex for startId
Co-authored-by: Linus Heckemann <git@sphalerite.org>
2022-11-21 18:46:55 +01:00
Eelco Dolstra 9d17ce07e8 AutoUserLock: If sandboxing is disabled, use the build users group
We have to use a gid that has write access to the Nix store.
2022-11-21 12:55:49 +01:00
Eelco Dolstra f0baa5c128 nix build --json: Include build statistics
Example:

  # nix build -L --extra-experimental-features cgroups --impure --expr 'with import <nixpkgs> {}; runCommand "foo" {} "dd if=/dev/urandom bs=1M count=1024 | md5sum; mkdir $out"' --json
  [
    {
      "cpuSystem": 1.911431,
      "cpuUser": 1.214249,
      "drvPath": "/nix/store/xzdqz67xba18hljhycp0hwfigzrs2z69-foo.drv",
      "outputs": {
        "out": "/nix/store/rh9mc9l2gkpq8kn2sgzndr6ll7ffjh6l-foo"
      },
      "startTime": 1669024076,
      "stopTime": 1669024079
    }
  ]
2022-11-21 12:06:01 +01:00
Eelco Dolstra e7a5b76844 Rename derivedPathsWithHintsToJSON -> builtPathsToJSON 2022-11-21 11:56:20 +01:00
Eelco Dolstra 82d5cf2a76 Fix macOS build 2022-11-21 11:45:41 +01:00
Eelco Dolstra 653b32a78f Merge remote-tracking branch 'origin/master' into auto-uid-allocation 2022-11-21 11:33:23 +01:00
Eelco Dolstra ec45f4b82e Fix indentation 2022-11-21 11:12:45 +01:00
Eelco Dolstra 300753d594 nix build --json: Include build statistics
Example:

  # nix build -L --extra-experimental-features cgroups --impure --expr 'with import <nixpkgs> {}; runCommand "foo" {} "dd if=/dev/urandom bs=1M count=1024 | md5sum; mkdir $out"' --json
  [
    {
      "cpuSystem": 1.911431,
      "cpuUser": 1.214249,
      "drvPath": "/nix/store/xzdqz67xba18hljhycp0hwfigzrs2z69-foo.drv",
      "outputs": {
        "out": "/nix/store/rh9mc9l2gkpq8kn2sgzndr6ll7ffjh6l-foo"
      },
      "startTime": 1669024076,
      "stopTime": 1669024079
    }
  ]
2022-11-21 10:49:01 +01:00
Eelco Dolstra f538ee4342 Rename derivedPathsWithHintsToJSON -> builtPathsToJSON 2022-11-21 09:38:08 +01:00
Eelco Dolstra e6b71f84a0 Use cgroup.kill to quickly kill cgroups 2022-11-18 16:59:36 +01:00
Eelco Dolstra fa68eb367e Get CPU stats from the cgroup 2022-11-18 13:40:59 +01:00
Eelco Dolstra 20f66c6889 Indentation 2022-11-18 13:40:48 +01:00
Eelco Dolstra 128910ba23 Separate cgroup support from auto-uid-allocation
The new experimental feature 'cgroups' enables the use of cgroups for
all builds. This allows better containment and enables setting
resource limits and getting some build stats.
2022-11-18 10:39:28 +01:00
Eelco Dolstra f1ab082ac4 createTempDir(): Use std::atomic 2022-11-18 09:37:11 +01:00
Eelco Dolstra f423d4425f Fix segfault in unprivileged mode 2022-11-17 11:56:45 +01:00
Yorick 09f00dd4d0
Replace src/libutil/json.cc with nlohmann json generation 2022-11-16 16:50:50 +01:00
Théophane Hufschmitt 62960f3291
Merge pull request #7134 from yorickvP/disable-dbg-on-complete
Temporarily disable the debugger during completion evaluation
2022-11-16 11:28:40 +01:00
Théophane Hufschmitt 60dea270d0
Swallow the error in a more idiomatic way 2022-11-16 10:34:32 +01:00
Robert Hensing bcd298d39b libstore/derivation-goal: Elaborate a TODO for performance concern 2022-11-15 17:57:40 +01:00
Théophane Hufschmitt 4bf70b74a7
Merge pull request #7294 from tobim/support-aws-sdk-1.10
libstore: link to aws-crt-cpp
2022-11-15 16:51:09 +01:00
Théophane Hufschmitt 3ade5f5d60
Merge pull request #7283 from hercules-ci/issue-6572
Fix #6572 `requires non-existent output`
2022-11-15 16:24:24 +01:00
Théophane Hufschmitt daf1423a4a
Merge pull request #7260 from ncfavier/readFile-scan-references
Restrict `readFile` context to references that appear in the string
2022-11-15 16:22:28 +01:00
Robert Hensing 7e162c69fe derivation-goal: Fix requires non-existing output error
It occurred when a output of the dependency was already available,
so it didn't need rebuilding and didn't get added to the
inputDrvOutputs.
This process-related info wasn't suitable for the purpose of finding
the actual input paths for the builder. It is better to do this in
absolute terms by querying the store.
2022-11-14 17:52:55 +01:00
Théophane Hufschmitt 8b4352d79b Merge remote-tracking branch 'nixos/master' into readFile-scan-references 2022-11-14 15:00:05 +01:00
Et7f3 efadeee8fd
build: use pkg-config for lowdown 2022-11-12 23:04:58 +01:00
Tobias Mayer 07f2cb1e8f
libstore: link to aws-crt-cpp
This change is needed to support aws-sdk-cpp 1.10 and newer.

I opted not to make this dependent on the sdk version because
the crt dependency has been in the interface of the older
sdk as well, and it was only coincidence that libstore didn't
make use of any privately defined symbols directly.
2022-11-12 14:34:23 +01:00
Naïm Favier e7ed9ae0c7
Restrict readFile context to references that appear in the string
When calling `builtins.readFile` on a store path, the references of that
path are currently added to the resulting string's context.

This change makes those references the *possible* context of the string,
but filters them to keep only the references whose hash actually appears
in the string, similarly to what is done for determining the runtime
references of a path.
2022-11-11 13:04:34 +01:00
Théophane Hufschmitt 9550b1d519
Merge pull request #7258 from ncfavier/fix-eval-error-fmt
Fix printing of eval errors with two format placeholders
2022-11-10 18:41:16 +01:00
Eelco Dolstra 6c6eff8ac4 Remove the SystemdCgroup feature 2022-11-10 17:24:12 +01:00
Eelco Dolstra 05d258667d Fix build on macOS 2022-11-08 08:00:29 -08:00
Eelco Dolstra 2fde7e0108 Split auto UID allocation from cgroups
Cgroups are now only used for derivations that require the uid-range
range feature. This allows auto UID allocation even on systems that
don't have cgroups (like macOS).

Also, make things work on modern systems that use cgroups v2 (where
there is a single hierarchy and no "systemd" controller).
2022-11-08 16:03:42 +01:00
Eelco Dolstra b378876894
Merge pull request #7264 from patricksjackson/remote-brackets
build-remote: Add brackets to error message
2022-11-07 16:59:29 +01:00
Eelco Dolstra 995f5f2e41
Merge pull request #7267 from fricklerhandwerk/build-task
manual: build action -> build task
2022-11-07 16:58:47 +01:00
Valentin Gagarin f7ab93b068 manual: build action -> build task
after discussing this with multiple people, I'm convinced that "build
task" is more precise: a derivation is not an action, but inert until it
is built. also it's easier to pronounce.

proposal: use "build task" for the generic concept "description of how
to derive new files from the contents of existing files". then it will
be easier to distinguish what we mean by "derivation" (a specific data
structure and Nix language value type) and "store derivation" (a
serialisation of a derivation into a file in the Nix store).
2022-11-06 13:28:18 +01:00
Patrick Jackson 907f52c337 build-remote: Add brackets to error message 2022-11-04 10:49:44 -07:00
Eelco Dolstra 40911d7dec Remove stray tab 2022-11-04 13:30:35 +01:00
Eelco Dolstra 6259fd7ea6 Fix indentation 2022-11-04 13:29:39 +01:00
Naïm Favier dad859ba0f
Fix printing of eval errors with two format placeholders 2022-11-04 12:41:38 +01:00
Eelco Dolstra b95faccf03 Merge remote-tracking branch 'origin/master' into auto-uid-allocation 2022-11-03 17:43:40 +01:00
Yorick 16f1720fd2
SourceExprCommand: swallow EvalError, add tests for this
Completing things that would error would print an ugly error in
the middle of your command line. Avoid printing this error.
2022-11-03 10:11:28 +01:00
Yorick 444af85552
Temporarily disable the debugger during completion evaluation
readline is not re-entrant, so entering the debugger from the
completioncallback results in an eventual segfault.

The workaround is to temporarily disable the debugger when searching
for possible completions.
2022-11-02 13:02:24 +01:00
Yorick 34ea0e2e7b
tarfile: set directory mode to at least 0500, don't extract fflags
We don't need SGID, or any ACL's. We also want to keep every dir +rx.
2022-11-01 16:01:38 +01:00
Eelco Dolstra 0edba4cc1e
Merge pull request #7231 from jfroche/fix/savedArgv-access
Move savedArgv into libmain
2022-10-31 16:35:08 +01:00
Théophane Hufschmitt 90ed041677
Merge pull request #7039 from Mic92/libfetchers-variant
libfetchers: avoid api.github.com ratelimit if no github token is set
2022-10-31 11:48:51 +01:00
John Ericson 13f2a6f38d
Merge branch 'master' into indexed-store-path-outputs 2022-10-28 23:22:18 +01:00
Jean-François Roche cd86eeb693
Move savedArgv into libmain
`savedArgv` is not accessible by plugins when defined in main binary.
Moving it into one of the nix lib fix the problem.
2022-10-28 12:19:37 +02:00
Eelco Dolstra b7e8a3bf4c
Merge pull request #7217 from NixOS/restore-nix-build-remote
Fix `nix __build-remote`
2022-10-27 13:33:51 +02:00
Théophane Hufschmitt f8d0193383 Pass the right argv when calling the build hook
Call it as `['nix', '__build-remote', ... ]` rather than the previous
`["__build-remote", "nix __build-remote", ... ]` which seemed to have
been most likely unintended
2022-10-27 11:53:04 +02:00
Eelco Dolstra 9323d139b0
Merge pull request #7174 from agbrooks/git-tag-bug
Prevent fetchGit from using incorrect cached rev for different refs
2022-10-26 16:48:48 +02:00
Théophane Hufschmitt 9bff7e8ee2 Fix nix __build-remote
Because of a wrong index, `nix __build-remote` wasn't working.

Fix the index to restore the command (and the build hook).
2022-10-26 11:53:46 +02:00
Yorick aff6d10934
nix run: fix "'defaultApp.x86_64-linux' should have type 'derivation'" 2022-10-26 10:05:27 +02:00
Théophane Hufschmitt 899878f77a
Merge pull request #7206 from akiekintveld/master
Defer to SSH config files for ForwardAgent option
2022-10-25 20:26:52 +02:00
Eelco Dolstra fed1700754
Merge pull request #7207 from NixOS/flake-type
Mark flakes with ._type = "flake".
2022-10-25 17:31:23 +02:00
Eelco Dolstra da2c61637b
Use _type
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2022-10-25 16:48:31 +02:00
Guillaume Maudoux d6f5734c63 Complete genericClosure tests 2022-10-25 01:57:18 +02:00
Guillaume Maudoux e93bf69b44 Rework error throwing, and test it 2022-10-25 01:46:10 +02:00
Michal Sojka a9a868fe6a Improve --profile description
The description of the --profile option talks about the "update" operation.
This is probably meant for operations such as "nix profile install", but the
same option is reused in other subcommands, which do not update the profile,
such as "nix profile {list,history,diff-closures}".

We update the description to make sense in both contexts.
2022-10-24 08:49:46 +02:00
Shea Levy 334fa81d08
Mark flakes with .type = "flake".
Fixes #7186
2022-10-23 06:54:11 -04:00
Austin Kiekintveld 8e7804273c Defer to SSH config files for ForwardAgent option
Currently, Nix passes `-a` when it runs commands on a remote machine via
SSH, which disables agent forwarding. This causes issues when the
`ForwardAgent` option is set in SSH config files, as the command line
operation always overrides those.

In particular, this causes issues if the command being run is `sudo`
and the remote machine is configured with the equivalent of NixOS's
`security.pam.enableSSHAgentAuth` option. Not allowing SSH agent
forwarding can cause authentication to fail unexpectedly.

This can currently be worked around by setting `NIX_SSHOPTS="-A"`, but
we should defer to the options in the SSH config files to be least
surprising for users.
2022-10-22 19:51:22 -05:00
Guillaume Maudoux 8bd8583bc7 Try to please clang with convoluted templates 2022-10-23 00:11:44 +02:00
Guillaume Maudoux 8c3afd2d68 Introduce an Error builder to tackle complexity 2022-10-22 23:37:54 +02:00
Graham Bennett 4563e80363 Fix C++20 warnings 2022-10-22 15:16:46 +01:00
Graham Bennett c5fd34a14e Build with C++20 2022-10-22 14:24:25 +01:00
Guillaume Maudoux 4a909c142c Rollback unneeded throwFrameErrorWithTrace function 2022-10-20 14:25:11 +02:00
Guillaume Maudoux 31ce52a045 Fix context message being printed twice with forceStringNoCtx 2022-10-20 14:18:35 +02:00
Guillaume Maudoux 512f6be9b5 Reword incomparable types message 2022-10-20 14:17:05 +02:00
Guillaume Maudoux 520404f450 Revert custom position of 'if' blocks 2022-10-20 13:55:15 +02:00
Eelco Dolstra 95331cb9c9
Merge pull request #7183 from n8henrie/patch-1
Fix typo -- dashes not underscores
2022-10-19 11:21:52 +02:00
Eelco Dolstra e136d57f26
Implement BinaryCacheStore::queryPathFromHashPart() 2022-10-18 17:48:09 +02:00
Eelco Dolstra 61f89e954a
Add command 'nix store path-from-hash-part'
This exposes the Store::queryPathFromHashPart() interface in the CLI.
2022-10-18 16:51:12 +02:00
Nathan Henrie af9c9504ca
Fix typo -- dashes not underscores 2022-10-17 14:15:32 -06:00
Guillaume Maudoux b945b844a9 Initial frames support 2022-10-17 03:05:02 +02:00
Guillaume Maudoux 3f9f6ae127 Merge remote-tracking branch 'origin/master' into coerce-string 2022-10-16 20:39:19 +02:00
sternenseemann d0f2da214b primops: make nature of foldl' strictness clearer
* Clarify the documentation of foldl': That the arguments are forced
  before application (?) of `op` is necessarily true. What is important
  to stress is that we force every application of `op`, even when the
  value turns out to be unused.

* Move the example before the comment about strictness to make it less
  confusing: It is a general example and doesn't really showcase anything
  about foldl' strictness.

* Add test cases which nail down aspects of foldl' strictness:
  * The initial accumulator value is not forced unconditionally.
  * Applications of op are forced.
  * The list elements are not forced unconditionally.
2022-10-16 14:29:12 +02:00
Andrew Brooks a259084c50 Fix #7146
When fetching a non-local git repo by ref (and no rev), don't consider unrelated
cached revs for the same repository.
2022-10-14 18:04:47 -05:00
Eelco Dolstra 3093bd3a85
Merge pull request #7168 from NixOS/rosetta-test
Improve Rosetta detection
2022-10-14 17:35:57 +02:00
Robert Hensing ab4eb39386 libmain: Make the entire stack overflow handler pluggable 2022-10-14 12:53:46 +02:00
Robert Hensing 0d75675787 libmain: Add extraStackOverflowHandler 2022-10-14 12:53:07 +02:00
Eelco Dolstra 285277a61a Remove useless debug statements
We haven't parsed the '-v' command line flags yet when this code executes,
so we can't actually get debug output here.
2022-10-14 00:35:33 -07:00
Eelco Dolstra ddd5503950 Use /usr/bin/true 2022-10-14 00:34:31 -07:00
Eelco Dolstra 0359d6d123 Fix error display if execve() in the builder fails
After we've send "\2\n" to the parent, we can't send a serialized
exception anymore. It will show up garbled like

  $ nix-build --store /tmp/nix --expr 'derivation { name = "foo"; system = "x86_64-linux"; builder = "/foo/bar"; }'
  this derivation will be built:
    /nix/store/xmdip0z5x1zqpp6gnxld3vqng7zbpapp-foo.drv
  building '/nix/store/xmdip0z5x1zqpp6gnxld3vqng7zbpapp-foo.drv'...

  ErrorErrorEexecuting '/foo/bar': No such file or directory
  error: builder for '/nix/store/xmdip0z5x1zqpp6gnxld3vqng7zbpapp-foo.drv' failed with exit code 1
2022-10-13 21:35:16 +02:00
Eelco Dolstra 96eb5ef156 Improve Rosetta detection
Turns out that one of those *.plist files can exist even if Rosetta is
not installed. So let's just try to run an x86_64-darwin binary
directly.
2022-10-13 11:46:16 -07:00
Eelco Dolstra 59a304a9a8 Fix clang warnings 2022-10-13 11:25:49 -07:00
Eelco Dolstra 27ed3d0458
Merge pull request #7140 from crawford/chroot-store
Make warning about chroot store location more accurate
2022-10-13 16:35:31 +02:00
Steam Deck User a86916eb72 Make warning about chroot store location more accurate
While trying to use an alternate directory for my Nix installation, I
noticed that nix's output didn't reflect the updated state
directory. This patch corrects that and now prints the warning before
attempting to create the directory (if the directory creation fails,
it wouldn't have been obvious why nix was attempting to create the
directory in the first place).

With this patch, I now get the following warning:

    warning: '/home/deck/.var/app/org.nixos.nix/var/nix' does not
    exist, so Nix will use '/home/deck/.local/share/nix/root' as a
    chroot store
2022-10-12 12:12:12 -07:00
Eelco Dolstra eba610956b Move some options into a misc category
This unclutters the per-command options a bit by moving out some
global options.
2022-10-12 15:09:17 +02:00
Eelco Dolstra ae2d330455 Merge branch 'print-common-flags' of github.com:Hoverbear/nix 2022-10-12 13:43:11 +02:00
Eelco Dolstra f3193edd87
Merge pull request #7149 from amjoseph-nixpkgs/pr/intersectAttrs/values
parseDrvName: remove doc/impl discrepancy, add test covering the gap
2022-10-12 10:00:39 +02:00
Adam Joseph 7ef71cd21f src/libexpr/primops.cc: parseDrvName: make documentation follow implementation
The documentation for `parseDrvName` does not agree with the implementation when
the derivation name contains a dash which is followed by something that is
neither a letter nor a digit.  This commit corrects the documentation to agree
with the implementation.
2022-10-08 17:21:34 -07:00
Ana Hobden 069409d167 Print common flags in --help 2022-10-07 09:07:22 -07:00
Valentin Gagarin 927234cfb2
Merge pull request #6870 from amjoseph-nixpkgs/pr/doc/explain-local-remote-binary-substituter 2022-10-05 09:01:42 +02:00
Jörg Thalheim d9abce4ad4 libfetchers: avoid api.github.com ratelimit if no github token is set
If we don't have any github token, we won't be able to fetch private
repos, but we are also more likely to run into API limits since
we don't have a token. To mitigate this only ever use the github api
if we actually have a token.
2022-10-01 10:38:09 +02:00
Eelco Dolstra 89ca75c9f9
Merge pull request #7080 from squalus/nar-close-file
archive: check close errors when extracting nars
2022-09-30 12:58:28 +02:00
Linus Heckemann 19f3ecd830 nix-build: remove unused --add-root arg
Fixes #1982
2022-09-27 00:07:42 +02:00
Eelco Dolstra 6806f20157
Merge pull request #7089 from NixOS/mention-accept-flake-config-in-warning
Mention `--accept-flake-config` in the related warning
2022-09-26 11:13:02 +02:00
Adam Joseph dc205c75a2 src/libexpr/primops.cc: correct definition for intersectAttrs
The current definition of `intersectAttrs` is incorrect:

> Return a set consisting of the attributes in the set e2 that also exist in the
> set e1.

Recall that (Nix manual, section 5.1):

> An attribute set is a collection of name-value-pairs (called attributes)

According to the existing description of `intersectAttrs`, the following should
evaluate to the empty set, since no key-value *pair* (i.e. attribute) exists in
both sets:

```
builtins.intersectAttrs { x=3; } {x="foo";}
```

And yet:

```
nix-repl> builtins.intersectAttrs { x=3; } {x="foo";}
{ x = "foo"; }
```

Clearly the intent here was for the *names* of the resulting attribute set to be
the intersection of the *names* of the two arguments, and for the values of the
resulting attribute set to be the values from the second argument.

This commit corrects the definition, making it match the implementation and intent.
2022-09-24 23:51:09 -07:00
Théophane Hufschmitt 371013c08d
Merge pull request #7010 from edolstra/ignore-shell
nix develop: Ignore stdenv's $SHELL
2022-09-24 12:55:53 +02:00
Théophane Hufschmitt db29ddd113
Merge pull request #7078 from obsidiansystems/trustworthy-signature
"valid signature" -> "signature by a trusted key"
2022-09-24 12:52:35 +02:00
Théophane Hufschmitt 74276cb354
Merge pull request #7079 from matthewbauer/allow-untrusted-settings
Allow pass max-silent-time and build-poll-interval to daemon untrusted
2022-09-24 12:47:49 +02:00
Théophane Hufschmitt 1a5d094be7
Mention --accept-flake-config in the related warning
Make sure that people who run Nix in non-interactive mode (and so don't have the possibility to interactively accept the individual flake configuration settings) are aware of this flag.

Fix #7086
2022-09-24 12:11:26 +02:00
Eelco Dolstra c13007f012
Merge pull request #7059 from NixOS/remove-useless-ca-file-message
Remove a useless debug message in filetransfer.cc
2022-09-24 10:39:23 +02:00
John Ericson 60e23c8bae
Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Rune K. Svendsen <runesvend@gmail.com>
2022-09-23 13:57:57 -04:00
squalus 223f8dace0 archive: check close errors when extracting nars 2022-09-22 12:50:32 -07:00
Matthew Bauer 6e049ae607 Allow pass max-silent-time and build-poll-interval to daemon untrusted
These settings seem harmless, they control the same polling
functionality that timeout does, but with different behavior. Should
be safe for untrusted users to pass in.
2022-09-22 13:59:16 -05:00
John Ericson a2a8cb10ac Dodge "trusted" vs "trustworthy" by being explicit
Hopefully this is best!
2022-09-22 14:37:52 -04:00
John Ericson 752f967c0f "valid signature" -> "trustworthy signature"
I just had a colleague get confused by the previous phrase for good
reason. "valid" sounds like an *objective* criterion, e.g. and *invalid
signature* would be one that would be trusted by no one, e.g. because it
misformatted or something.

What is actually going is that there might be a signature which is
perfectly valid to *someone else*, but not to the user, because they
don't trust the corresponding public key. This is a *subjective*
criterion, because it depends on the arbitrary and personal choice of
which public keys to trust.

I therefore think "trustworthy" is a better adjective to use. Whether
something is worthy of trust is clearly subjective, and then "trust"
within that word nicely evokes `trusted-public-keys` and friends.
2022-09-22 10:49:31 -04:00
Théophane Hufschmitt f704c2720f
Merge pull request #7070 from alyssais/repl-history
nix repl: warn if creating dir for history fails
2022-09-21 13:50:49 +02:00
Alyssa Ross d234d01f01
nix repl: warn if creating dir for history fails
The history is not critical to the functionality of nix repl, so it's
enough to warn here, rather than refuse to start if the directory Nix
thinks the history should live in can't be created.
2022-09-20 11:01:40 +00:00
squalus 1b595026e1 Improve durability of schema version file writes
- call close explicitly in writeFile to prevent the close exception
  from being ignored
- fsync after writing schema file to flush data to disk
- fsync schema file parent to flush metadata to disk

https://github.com/NixOS/nix/issues/7064
2022-09-19 20:13:30 -07:00
Théophane Hufschmitt 0f977bf91e
Remove a useless debug message in filetransfer.cc
Remove the `verify TLS: Nix CA file = 'blah'` message that Nix used to print when fetching anything as it's both useless (`libcurl` prints the same info in its logs) and misleading (gives the impression that a new TLS connection is being established which might not be the case because of multiplexing. See #7011 )
2022-09-19 08:42:43 +02:00
Adam Joseph fb985f855c fetchurl.nix: change other use of __impure 2022-09-16 01:54:24 -07:00
Adam Joseph 673fd21b7c
Update src/libexpr/fetchurl.nix
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-09-16 08:51:14 +00:00
Adam Joseph b3550d9179 libexpr/fetchurl.nix: allow __impure fetch
This commit adds an optional `__impure` parameter to fetchurl.nix, which allows
the caller to use `libfetcher`'s fetcher in an impure derivation.  This allows
nixpkgs' patch-normalizing fetcher (fetchpatch) to be rewritten to use nix's
internal fetchurl, thereby eliminating the awkward "you can't use fetchpatch
here" banners scattered all over the place.

See also: https://github.com/NixOS/nixpkgs/pull/188587
2022-09-16 00:48:01 -07:00
Eelco Dolstra d365cced4f
Trim option descriptions
This removes unintended blank lines in Markdown when the description
is a multiline string literal.
2022-09-13 16:58:32 +02:00
Eelco Dolstra c6ff33ff5c
RunPager: Stop the progress bar
In particular, the progress bar was interfering with 'less' rendering
in '--help' (e.g. run 'nix --help' and hit '/' to search).
2022-09-13 15:29:13 +02:00
Théophane Hufschmitt 0f64bf445a
Merge pull request #6994 from agbrooks/master
Prevent tempdir from being GC-ed before addToStoreFromDump completes
2022-09-13 09:23:16 +02:00
Andrew Brooks 565d888e0f Address PR feedback on #6694 2022-09-12 11:33:23 -05:00
Guillaume Maudoux 96f2dd99d3 fix remaining foo stuff 2022-09-11 23:09:36 +02:00
Guillaume Maudoux e412bb6d30 fix remaining fixtures 2022-09-11 22:58:59 +02:00
Guillaume Maudoux c7b901fd33 Cleanup error strings rebase 2022-09-11 01:34:19 +02:00
Valentin Gagarin dceb4f9dbb
Merge pull request #6825 from rapenne-s/nix_shell_example
nix shell: document how to invoke multiple commands from the command …
2022-09-09 13:35:23 +02:00
Théophane Hufschmitt c8e8eea95e
Merge pull request #7008 from jonringer/add-command-docs
Docs: Add nix develop --command entry
2022-09-08 13:34:40 +02:00
Max 6da4cc92d8 showForeign -> showAllSystems 2022-09-07 20:28:30 +02:00
Andrew Brooks 84fe75a12a Keep created temp dirs inside store, but protect from GC
Implements the approach suggested by feedback on PR #6994, where
tempdir paths are created in the store (now with an exclusive lock).

As part of this work, the currently-broken and unused
`createTempDirInStore` function is updated to create an exclusive lock
on the temp directory in the store.

The GC now makes a non-blocking attempt to lock any store directories
that "look like" the temp directories created by this function, and if
it can't acquire one, ignores the directory.
2022-09-06 17:48:00 -05:00
Guillaume Maudoux eb460a9529 WIP: broken merge but need a git checkpoint 2022-09-07 00:34:03 +02:00
Eelco Dolstra ece12a97d9 lockfile -> lock file for consistency 2022-09-06 19:20:31 +02:00
Eelco Dolstra 27be54ca53 nix develop: Ignore stdenv's $SHELL
Stdenv sets this to a bash that doesn't have readline/completion
support, so running 'nix (develop|shell)' inside a 'nix develop' gives
you a crippled shell. So let's just ignore the derivation's $SHELL.

This could break interactive use of build phases that use $SHELL, but
they appear to be fairly rare.
2022-09-06 18:27:39 +02:00
Eelco Dolstra 9600109879
Merge pull request #6995 from matthewbauer/disable-sa-restart-on-macos
Disable SA_RESTART for some signals on macOS
2022-09-06 17:36:31 +02:00
Jonathan Ringer 6ce2e96c88
Docs: Add nix develop --command entry
Add example of nix develop being used to execuate a series of script
commands. This is common when doing things like CI/CD, and should be
represented in the official documentation.

Also useful for people looking for the 'nix develop' equivalent of
'nix-shell --run'.

Related:
 - https://github.com/NixOS/nix/issues/6908
 - https://github.com/NixOS/nix/issues/6908#issuecomment-1229266853
2022-09-06 08:18:29 -07:00
Matthew Bauer 77d3d3d18d Remove some signals 2022-09-06 09:47:53 -05:00
Matthew Bauer a47b5476e1 Add more signals 2022-09-03 16:06:33 -05:00
Matthew Bauer 102434e4cb Disable SA_RESTART for some signals on macOS
Disables the SA_RESTART behavior on macOS which causes:

> Restarting of pending calls is requested by setting the SA_RESTART bit
> in sa_flags. The affected system calls include read(2), write(2),
> sendto(2), recvfrom(2), sendmsg(2) and recvmsg(2) on a communications
> channel or a slow device (such as a terminal, but not a regular file)
> and during a wait(2) or ioctl(2).

From: https://man.openbsd.org/sigaction#SA_RESTART

This being set on macOS caused a bug where read() calls to the daemon
socket were blocking after a SIGINT was received. As a result,
checkInterrupt was never reached even though the signal was received
by the signal handler thread.

On Linux, SA_RESTART is disabled by default. This probably effects
other BSDs but I don’t have the ability to test it there right now.
2022-09-03 00:27:16 -05:00
Andrew Brooks 1f041ac54f Prevent tempdir from being GC-ed before addToStoreFromDump has renamed it
This fixes issue 6823 by placing the tempdir used in LocalStore::addToStoreFromDump
outside the Nix store, where automatic GC is no longer a concern.
2022-09-02 18:32:35 -05:00
Matthew Bauer 4894e567fb Don’t readDerivation if impure derivations feature is disabled
readDerivation is pretty slow, and while it may not be significant for
some use cases, on things like ghc-nix where we have thousands of
derivations is really slows things down.

So, this just doesn’t do the impure derivation check if the impure
derivation experimental feature is disabled. Perhaps we could cache
the result of isPure() and keep the check, but this is a quick fix to
for the slowdown introduced with impure derivations features in 2.8.0.
2022-09-02 11:46:34 -05:00
Adam Joseph 1ab913467e linkify mention of other options 2022-09-01 18:03:35 -07:00
Adam Joseph 59dc8346ca move substituter signature-checking conditions to configuration file documentation 2022-09-01 17:51:56 -07:00
Max 02bff90e7b nix flake show: don't evaluate derivations for foreign systems by default 2022-09-01 22:04:22 +02:00
Eelco Dolstra a8b3d777fb Revert "Merge pull request #6621 from Kha/nested-follows"
This reverts commit c530cda345, reversing
changes made to 4adcdff5c1.
2022-09-01 15:26:19 +02:00
Théophane Hufschmitt c530cda345
Merge pull request #6621 from Kha/nested-follows
Fix nested flake input overrides
2022-09-01 12:04:00 +02:00
Eelco Dolstra 1f2b12b30e
Merge pull request #6968 from fricklerhandwerk/man-no-weblinks
do not render relative links in help pages
2022-08-29 08:46:35 +02:00
Sebastian Ullrich 6f65c11780 Fix overlapping flake follows 2022-08-28 11:50:25 +02:00
Sebastian Ullrich 2b9d381301 Fix nested flake input overrides 2022-08-28 11:47:25 +02:00
Valentin Gagarin e7dcacbd7c do not render relative links in help pages
this simplifies the setup a lot, and avoids weird looking `./file.md`
links showing up.

it also does not show regular URLs any more. currently the command
reference only has few of them, and not showing them in the offline
documentation is hopefully not a big deal.

instead of building more special-case solutions, clumsily preprocessing
the input, or issuing verbal rules on dealing with URLs, should better
be solved sustainably by not rendering relative links in `lowdown`:

https://github.com/kristapsdz/lowdown/issues/105
2022-08-26 17:10:41 +02:00
Eelco Dolstra 7918adbb62
Merge pull request #6954 from winterqt/darwin-sandbox-trustd
fix(libstore): allow access to trustd on macOS
2022-08-26 11:45:00 +02:00
Eelco Dolstra bb411e4ae1 Fix progress bar flicker with -L
This was caused by -L calling setLogFormat() again, which caused the
creation of a new progress bar without destroying the old one. So we
had two progress bars clobbering each other.

We should change 'logger' to be a smart pointer, but I'll do that in a
future PR.

Fixes #6931.
2022-08-24 22:36:40 +02:00
Eelco Dolstra a17ce0a8a9 Fix evaluation cache
98e361ad4c introduced a regression where
previously stored attributes were replaced by placeholders. As a
result, a command like 'nix build nixpkgs#hello' had to be executed at
least twice to get caching.

This code does not seem necessary for suggestions to work.
2022-08-24 21:19:43 +02:00
Winter Cute 8e5659423e fix(libstore): allow access to trustd on macOS 2022-08-24 13:09:44 -04:00
Eelco Dolstra 04e74f7c8b
Merge pull request #6612 from NixOS/parallel-nix-copy
Make nix copy parallel again
2022-08-24 15:31:42 +02:00
Eelco Dolstra 56d97d4b4d Remove redundant Finally 2022-08-24 14:49:58 +02:00
Eelco Dolstra 8d906b1f3b Fix macOS build 2022-08-24 14:11:03 +02:00
Rickard Nilsson c2d7456926 Fix a misplaced parenthese in serve protocol check
This issue made it impossible for clients using a serve protocol of
version <= 2.3 to use the `cmdBuildDerivation` command of servers using
a protocol of version >= 2.6. The faulty version check makes the server
send back build outputs that the client is not expecting.
2022-08-24 01:54:43 +02:00
Eelco Dolstra 4a0c4ca186
Merge pull request #6944 from edolstra/bash-vars
nix develop: Ignore some more bash special variables
2022-08-23 20:15:18 +02:00
Eelco Dolstra 0e54fab0dd
Merge pull request #6909 from ncfavier/json-paths
JSON: print paths as strings without copying them to the store
2022-08-23 15:19:33 +02:00
Eelco Dolstra db026103b1 nix develop: Ignore some more bash special variables
Fixes #6940.
2022-08-23 14:57:08 +02:00
Eelco Dolstra db71f5646f
Merge pull request #6887 from winterqt/specify-bashinteractive-outputs
nix-shell: specify which outputs from bashInteractive to build
2022-08-23 14:44:23 +02:00
Naïm Favier ff0b5a778c
Revert to copyStore = true for nix-instantiate and nix-env 2022-08-23 14:40:27 +02:00
Eelco Dolstra f0358ed465 Fix a hang in nix-copy-ssh.sh
This hang for some reason didn't trigger in the Nix build, but did
running 'make installcheck' interactively. What happened:

* Store::addMultipleToStore() calls a SinkToSource object to copy a
  path, which in turn calls LegacySSHStore::narFromPath(), which
  acquires a connection.

* The SinkToSource object is not destroyed after the last bytes has
  been read, so the coroutine's stack is still alive and its
  destructors are not run. So the connection is not released.

* Then when the next path is copied, because max-connections = 1,
  LegacySSHStore::narFromPath() hangs forever waiting for a connection
  to be released.

The fix is to make sure that the source object is destroyed when we're
done with it.
2022-08-23 14:19:53 +02:00
Eelco Dolstra f865048332 Indentation 2022-08-22 15:32:53 +02:00
Eelco Dolstra 4c96761c2b Merge remote-tracking branch 'origin/master' into parallel-nix-copy 2022-08-22 15:29:10 +02:00
Eelco Dolstra 7c3ab57515
Merge pull request #6937 from edolstra/fix-repl
nix repl: Stop the progress bar
2022-08-22 15:10:01 +02:00
Naïm Favier 4c2ff4a0f4
JSON: add missing newlines
after `nix eval --json` and `nix-instantiate --eval --json`.
2022-08-22 15:07:52 +02:00
Naïm Favier 062e4fcdde
JSON: print paths as strings without copying them to the store
Makes `printValueAsJSON` not copy paths to the store for `nix eval
--json`, `nix-instantiate --eval --json` and `nix-env --json`.

Fixes https://github.com/NixOS/nix/issues/5612
2022-08-22 15:01:35 +02:00
Eelco Dolstra 8d84634e26
Merge pull request #6926 from rapenne-s/download_limit
add a nix.conf option to set a download speed limit
2022-08-22 14:57:19 +02:00
Eelco Dolstra 590660c1c6
Merge pull request #6932 from jack1142/patch-1
Fix default profile path for root in nix profile documentation
2022-08-22 14:55:04 +02:00
Eelco Dolstra 0d2163c6dc nix repl: Stop the progress bar
The repl was broken since c3769c6846. In
general, the progress bar is incompatible with the repl.
2022-08-22 14:27:36 +02:00
Eelco Dolstra 9ddef9f0be
Merge pull request #6923 from ncfavier/nix-env-incomplete-json
Avoid producing incomplete JSON
2022-08-22 14:15:31 +02:00
Eelco Dolstra c21b1a7e67
Spelling 2022-08-22 14:14:14 +02:00
Solène Rapenne caad87e6db
Better documentation wording
Co-authored-by: Anderson Torres <torres.anderson.85@protonmail.com>
2022-08-20 18:21:36 +02:00
Jakub Kuczys 7d800909e9
Fix default profile path for root in nix profile documentation 2022-08-20 03:48:42 +02:00
Solene Rapenne 0d2bf7acf9 add a nix.conf option to set a download speed limit 2022-08-19 12:40:22 +02:00
pennae 7d934f7880 don't read outputs into memory for output rewriting
RewritingSink can handle being fed input where a reference crosses a
chunk boundary. we don't need to load the whole source into memory, and
in fact *not* loading the whole source lets nix build FODs that do not
fit into memory (eg fetchurl'ing data files larger than system memory).
2022-08-19 11:26:26 +02:00
Naïm Favier 7535ee345d
nix-env: don't output incomplete JSON 2022-08-19 01:00:10 +02:00
Naïm Favier 8188b1d0ab
json: write null on abnormal placeholder destruction
Avoids leaving dangling attributes like

    {
        "foo":
    }

in case of exceptions.
2022-08-19 01:00:10 +02:00
Eelco Dolstra c80a74b7d5 Don't pass --force to 'git add'
Fixes #5810.
2022-08-17 16:59:02 +02:00
Eelco Dolstra 53e7b7e8ac Remove warnLargeDump()
This message was unhelpful (#1184) and probably misleading since
memory is O(1) in most cases now.
2022-08-17 11:32:01 +02:00
Eelco Dolstra c3769c6846 ProgressBar: Delay before showing a new activity
Some activities are numerous but usually very short (e.g. copying a
source file to the store) which would cause a lot of flickering. So
only show activities that have been running for at least 10 ms.
2022-08-17 11:31:44 +02:00
Eelco Dolstra e62160579f nix flake metadata: Don't show "Inputs" if there are no inputs 2022-08-17 11:31:34 +02:00
Eelco Dolstra 3d4489b623 Show when we're evaluating a flake 2022-08-17 11:31:27 +02:00
David Arnold b58785ab83
docfix: bundlers 2022-08-10 18:49:29 -05:00
Théophane Hufschmitt 8f3fdef1e0
Merge pull request #6850 from NinjaTrappeur/nin/build-check
Fix Nix build --check flag
2022-08-10 17:44:06 +02:00
Winter Cute 5f37c5191a nix-shell: specify which outputs from bashInteractive to build 2022-08-09 22:54:22 -04:00
Théophane Hufschmitt 5192bb093a Merge remote-tracking branch 'upstream/master' into fix-mv-in-different-filesystems 2022-08-08 15:42:56 +02:00
Théophane Hufschmitt f4a8426098 Remove the explicit c++fs linkage on darwin
Doesn't seem needed on a recent-enough clang anymore (and even seems to
break stuff)
2022-08-08 14:34:22 +02:00
Dave Nicponski cb6794a0d9 Do not spam logs if the owned-homedir check results in a noop 2022-08-07 10:13:11 -04:00
Erik Arvstedt 4c8441be0a docs/flake-update: fix example 2022-08-04 09:45:30 +02:00
Erik Arvstedt 53833dfb40 libexpr/flake: remove FIXME
Line 593 checks that all overrides (i.e. all elements of
`lockFlags.inputOverrides`) are members of `overridesUsed`.
2022-08-04 09:45:29 +02:00
Erik Arvstedt c9f446ede1 flakeref: fix comment 2022-08-04 09:45:28 +02:00
Eelco Dolstra ccbd906c86 Fix NIX_COUNT_CALLS=1
Also, make the JSON writer support std::string_view.

Fixes #6857.
2022-08-03 17:46:51 +02:00
Théophane Hufschmitt 1ba5b3e001 Make moveFile more atomic
Rather than directly copying the source to its dest, copy it first to a
temporary location, and eventually move that temporary.
That way, the move is at least atomic from the point-of-view of the destination
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt 90f9680733 Only use renameFile where needed
In most places the fallback to copying isn’t needed and can actually be
bad, so we’d rather not transparently fallback
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt d71d9e9fbf moveFile -> renameFile
`move` tends to have this `mv` connotation of “I will copy it for you if
needs be”
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt a4f0fd633c Link against c++fs on darwin
Required by the old clang version
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt c5db1821a9 Re-implement the recursive directory copy
The recursive copy from the stl doesn’t exactly do what we need because
1. It doesn’t delete things as we go
2. It doesn’t keep the mtime, which change the nars

So re-implement it ourselves. A bit dull, but that way we have what we want
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt 6f89fb6008 rename: Fallback to a copy if the filesystems mismatch
In `nix::rename`, if the call to `rename` fails with `EXDEV` (failure
because the source and the destination are in a different filesystems)
switch to copying and removing the source.

To avoid having to re-implement the copy manually, I switched the
function to use the c++17 `filesystem` library (which has a `copy`
function that should do what we want).

Fix #6262
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt c2de0a232c Create a wrapper around stdlib’s rename
Directly takes some c++ strings, and gently throws an exception on error
(rather than having to inline this logic everywhere)
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt 8119390abc Move some fs-related functions to their own file
Unclutter `util.cc` a bit
2022-08-03 10:27:25 +02:00
Félix Baylac-Jacqué 1467a98d4c
derivation-goal.cc: remove bmCheck custom return branch on buildDone
Once a derivation goal has been completed, we check whether or not
this goal was meant to be repeated to check its output.

An early return branch was preventing the worker to reach that repeat
code branch, hence breaking the --check command (#2619).

It seems like this early return branch is an artifact of a passed
refactoring. As far as I can tell, buildDone's main branch also
cleanup the tmp directory before returning.
2022-08-01 11:39:19 +02:00
Théophane Hufschmitt 2805439335
Merge pull request #6814 from amjoseph-nixpkgs/pr/sandbox-error-messages
local-derivation-goal.cc: improve error messages when sandboxing fails
2022-07-22 13:27:52 +02:00
Solène Rapenne 92bae33ca5
nix shell: example shouldn't use an absolute path for the shell
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-07-21 14:25:07 +02:00
Solène Rapenne 64404220f5 nix shell: document how to invoke multiple commands from the command line 2022-07-20 14:54:46 +02:00
Théophane Hufschmitt 7ed91d6c6a
Merge branch 'master' into parallel-nix-copy 2022-07-20 10:05:34 +02:00
Alex Wied 722de8ddcc libstore/globals.cc: Move cgroup detection to libutil 2022-07-19 16:25:53 -04:00
Alex Wied 1af5d798a4 libstore/globals.cc: Automatically set cores based on cgroup CPU limit
By default, Nix sets the "cores" setting to the number of CPUs which are
physically present on the machine. If cgroups are used to limit the CPU
and memory consumption of a large Nix build, the OOM killer may be
invoked.

For example, consider a GitLab CI pipeline which builds a large software
package. The GitLab runner spawns a container whose CPU is limited to 4
cores and whose memory is limited to 16 GiB. If the underlying machine
has 64 cores, Nix will invoke the build with -j64. In many cases, that
level of parallelism will invoke the OOM killer and the build will
completely fail.

This change sets the default value of "cores" to be
ceil(cpu_quota / cpu_period), with a fallback to
std:🧵:hardware_concurrency() if cgroups v2 is not detected.
2022-07-19 16:03:58 -04:00
Théophane Hufschmitt 56f6f3725f Don't ultimately trust the signed paths
Like the old implem did (and like you'd want it to be anyways)
2022-07-19 19:46:00 +02:00
Adam Joseph 36e1383b6b local-derivation-goal.cc: save global errno to the stack before performing tests which might clobber it 2022-07-19 03:53:20 -07:00
Adam Joseph a9e75eca00 error.hh: add additional constructor with explicit errno argument 2022-07-19 03:49:33 -07:00
Adam Joseph 99fcc91f67 as requested by @thufschmitt https://github.com/NixOS/nix/pull/6814#discussion_r924275777 2022-07-19 03:33:12 -07:00
Adam Joseph 5f51539f88 change warn() to notice() 2022-07-19 03:30:52 -07:00
Adam Joseph c8c6203c2c local-derivation-goal.cc: detect unprivileged_userns_clone failure mode
The workaround for "Some distros patch Linux" mentioned in
local-derivation-goal.cc will not help in the `--option
sandbox-fallback false` case.  To provide the user more helpful
guidance on how to get the sandbox working, let's check to see if the
`/proc` node created by the aforementioned patch is present and
configured in a way that will cause us problems.  If so, give the user
a suggestion for how to troubleshoot the problem.
2022-07-17 01:27:22 -07:00
Adam Joseph 6fc56318bf local-derivation-goal.cc: add comment re: CLONE_NEWUSER
local-derivation-goal.cc contains a comment stating that "Some distros
patch Linux to not allow unprivileged user namespaces."  Let's give a
pointer to a common version of this patch for those who want more
details about this failure mode.
2022-07-17 01:23:32 -07:00
Adam Joseph 8d35f387dc local-derivation-goal.cc: warn if failing and /proc/self/ns/user missing
This commit causes nix to `warn()` if sandbox setup has failed and
`/proc/self/ns/user` does not exist.  This is usually a sign that the
kernel was compiled without `CONFIG_USER_NS=y`, which is required for
sandboxing.
2022-07-16 19:37:27 -07:00
Adam Joseph 90830b1074 local-derivation-goal.cc: warn if failing due to max_user_namespaces==0
This commit uses `warn()` to notify the user if sandbox setup fails
with errno==EPERM and /proc/sys/user/max_user_namespaces is missing or
zero, since that is at least part of the reason why sandbox setup
failed.

Note that `echo -n 0 > /proc/sys/user/max_user_namespaces` or
equivalent at boot time has been the recommended mitigation for
several Linux LPE vulnerabilities over the past few years.  Many users
have applied this mitigation and then forgotten that they have done
so.
2022-07-16 19:30:53 -07:00
Adam Joseph 8ea3a911aa local-derivation-goal.cc: improve error messages when sandboxing fails
The failure modes for nix's sandboxing setup are pretty complicated.
When nix is unable to set up the sandbox, let's provide more detail
about what went wrong.  Specifically:

* Make sure the error message includes the word "sandbox" so the user
  knows that the failure was related to sandboxing.

* If `--option sandbox-fallback false` was provided, and removing it
  would have allowed further attempts to make progress, let the user
  know.
2022-07-16 14:56:24 -07:00
Alex Wied b88fb50e21 fix(libstore): allow Nix to access all Rosetta 2 paths on MacOS
Fixes: #5884
2022-07-15 12:10:56 -07:00
John Ericson 12461e246b Leverage existing docs for new store-path^outputs syntax 2022-07-15 14:04:22 +00:00
John Ericson 0e4ec98ae8
Fix typo in docs
Thanks!

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-07-15 09:49:23 -04:00
John Ericson 279ecf7cde Remove computed-derivations experimental feature
We don't need it yet.
2022-07-15 13:42:17 +00:00
Eelco Dolstra 59764eb842
Merge pull request #6810 from jfly/jfly/do-not-assume-savedvars-exist
nix develop: do not assume that saved vars are set
2022-07-15 13:59:25 +02:00
Eelco Dolstra 3bcd7a5474 Disable auto-chroot if $NIX_STATE_DIR is set
Issue #6732.
2022-07-15 12:32:29 +02:00
Jeremy Fleischman 04386f7d69
nix develop: do not assume that saved vars are set
This fixes https://github.com/NixOS/nix/issues/6809
2022-07-14 23:25:39 -07:00
John Ericson 8735f55dec Fix bug, test more, document more 2022-07-14 20:23:43 -04:00
John Ericson f3262bc216 Combine InstallableStorePath with InstallableIndexedStorePath
No behavior should be changed, the `isDerivation` logic is moved from
the methods to the constructor.
2022-07-14 19:37:55 -04:00
John Ericson 6cafe308c9 Merge remote-tracking branch 'upstream/master' into indexed-store-path-outputs 2022-07-14 16:15:37 -04:00
Eelco Dolstra ff49c75502 Disable auto-chroot if $NIX_STORE_DIR is set
Fixes #6732.
2022-07-14 17:47:09 +02:00
Eelco Dolstra 12df8885cc Simplify the check for overrides on non-existent inputs 2022-07-13 13:40:40 +02:00