Commit graph

8999 commits

Author SHA1 Message Date
Valentin Gagarin a1cf16563f
Fixup description of substituters (#8291)
Introduce what substituters "are" in the configuration option entry.
Remove arbitrary line breaks for easier editing in the future.
Link glossary some more.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-06-14 20:49:58 +00:00
Michal Sojka a0c4d58549 Remove RegisterPrimOp constructor without support for documentation
The remaining constructor RegisterPrimOp::RegisterPrimOp(Info && info)
allows specifying the documentation in .args and .doc members of the
Info structure.

Commit 8ec1ba0210 removed all uses of the removed constructor in the
nix binary. Here, we remove the constructor completely as well as its
use in a plugin test. According to #8515, we didn't promis to maintain
compatibility with external plugins.

Fixes #8515
2023-06-14 22:37:52 +02:00
John Ericson 05eb06a1de
Merge pull request #8490 from flox/stdin_handling
fix: Do not apply default installables when using --stdin
2023-06-14 20:41:22 +02:00
John Ericson e377746842
Merge pull request #8491 from wentasah/builtins-doc
Document several undocumented builtin functions
2023-06-14 20:32:37 +02:00
Daniel Asaturov 468add5aa0
Remove dead code (#8504)
`filesystem.cc` is the only place where `createSymlink()` is used with three arguments:
in the definition of `replaceSymlink()` with three parameters that _is not used at all_.

Closes #8495
2023-06-14 14:09:11 -04:00
John Ericson ff905cb796
Merge pull request #4803 from ShamrockLee/nix-channel-list-generations
Add `nix-channel --list-generations`
2023-06-14 18:30:35 +02:00
John Ericson 61a3e1f2e2
Merge pull request #4282 from tweag/fix-ca-hash-rewriting
fix the hash rewriting for ca-derivations
2023-06-14 18:25:00 +02:00
Michal Sojka c6d7c4f9ec Document fromTOML, hasContext and getContext builtins
Until now, these functions were completely missing in the Nix manual.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-13 21:53:03 +02:00
Eelco Dolstra 1ad3328c5e Allow tarball URLs to redirect to a lockable immutable URL
Previously, for tarball flakes, we recorded the original URL of the
tarball flake, rather than the URL to which it ultimately
redirects. Thus, a flake URL like
http://example.org/patchelf-latest.tar that redirects to
http://example.org/patchelf-<revision>.tar was not really usable. We
couldn't record the redirected URL, because sites like GitHub redirect
to CDN URLs that we can't rely on to be stable.

So now we use the redirected URL only if the server returns the
`x-nix-is-immutable` or `x-amz-meta-nix-is-immutable` headers in its
response.
2023-06-13 14:17:45 +02:00
Eelco Dolstra 3402b650cd Add a generic check for rev attribute mismatches 2023-06-13 14:13:50 +02:00
Tom Bereknyei 0e3849dc65 test: add test for non-defaulting for stding installable input 2023-06-12 08:40:17 -04:00
Eelco Dolstra 87c66f6b0f Don't include uds-remote-store.md from a header file
Closes #8484.
2023-06-12 11:10:55 +02:00
Michal Sojka 8ec1ba0210 Register all PrimOps via the Info structure
This will allow documenting them (in later commits).

Note that we keep the old constructor even if it is no longer used by
Nix code, because it is used in tests/plugins/plugintest.cc, which
suggests that it might be used by some external plugin.
2023-06-11 21:40:43 +02:00
Tom Bereknyei 08089fdd32 fix: Do not apply default installables when using --stdin 2023-06-11 13:33:38 -04:00
Eelco Dolstra e54538c461 restoreMountNamespace(): Restore the original root directory
This is necessary when we're in a chroot environment, where the
process root is not the same as the root of the mount namespace
(e.g. in nixos-enter).

Fixes #7602.
2023-06-09 16:09:29 +02:00
Eelco Dolstra 381a32981b
Merge branch 'master' into angerman/mac-fix-recursive-nix 2023-06-09 13:06:47 +02:00
Andrea Bedini 3c78920f73
Parse TOML timestamps (#8120)
Currently `fromTOML` throws an exception when encountering a timestamp
since the Nix language lacks a way to represent them.

This patch changes this beaviour and makes `fromTOML` parse timestamps as
attrsets of the format

  { _type = "timestamp"; value = "1979-05-27T07:32:00Z"; }

This is guarded by an experimental feature flag to leave room for iterating on the representation.
2023-06-09 11:53:18 +02:00
Eelco Dolstra b56319a139
Merge pull request #8391 from aneeshusa/remove-wrong-default-value-in-docs-for-hashed-mirrors
Remove old default from docs for `hashed-mirrors`
2023-06-06 12:54:28 +02:00
Eelco Dolstra 864ab1a7ac
Merge pull request #8438 from zhaofengli/armv5tel-linux32
Also set the PER_LINUX32 personality flag on armv5tel-linux
2023-06-06 12:47:16 +02:00
Eelco Dolstra 51b3576855
Merge pull request #8456 from emilytrau/include-select
Add missing <sys/select.h> include
2023-06-06 12:45:51 +02:00
Eelco Dolstra f5c6b29940 Fix SourcePath::resolveSymlinks()
This fixes handling of symlinks that start with '..', and symlink
targets that contain symlinks themselves.
2023-06-06 11:24:10 +02:00
Emily Trau b37dd43db4 Add missing <sys/select.h> include
`select()` may not be ambiently available for use on every platform
2023-06-05 20:18:24 -07:00
figsoda b64450ed4e Fix code block formatting in man page 2023-06-04 14:50:47 -04:00
Alexander Bantyev a15b2c01c0
Document manual migration for use-xdg-base-directories (#8044)
* Document manual migration for use-xdg-base-directories

As there's currently no automatic migration for use-xdg-base-directories
option, add instructions for manual migration to the option's
description.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-02 15:38:17 +00:00
Zhaofeng Li 48632e6139 Also set the PER_LINUX32 personality flag on armv5tel-linux 2023-06-02 03:04:13 -06:00
Shamrock Lee b292177eec Add nix-channel --list-generations
Add support to --list-generations
as another way to say
nix-env --profile /nix/var/nix/profiles/per-user/$USER/channels --list-generations
the way we did for nix-channel --rollback [generation id]
2023-06-02 10:12:35 +02:00
Eelco Dolstra 9a5b9c588f
Merge pull request #8421 from fricklerhandwerk/doc-replaceStrings
update documentation according to release notes
2023-05-31 12:10:32 +02:00
Valentin Gagarin 738c0d5064
Merge pull request #8318 from fricklerhandwerk/doc-currentTime
document `builtins.currentTime`
2023-05-31 03:15:54 +02:00
Valentin Gagarin 9afd525e18 update documentation according to release notes 2023-05-30 22:58:15 +02:00
Robert Hensing bed2fe2312
Merge pull request #8398 from polykernel/perf/lazy-eval-replacements-replacestrings
primops: lazy evaluation of replaceStrings replacements
2023-05-30 22:49:42 +02:00
Eelco Dolstra ecae62020b
Merge pull request #8406 from NixOS/fix-ca-attrs-false
Restore Nix 2.3 behaviour for {__impure,__contentAddressed} = false
2023-05-30 11:59:14 +02:00
Eelco Dolstra 61ddfa154b
Merge pull request #8399 from tweag/fix-chrooted-stores-error-path
Properly report build errors on chrooted stores
2023-05-27 17:55:57 +02:00
Eelco Dolstra e4aebccf20 Restore Nix 2.3 behaviour for {__impure,__contentAddressed} = false
Fixes #8405.
2023-05-27 17:53:30 +02:00
Yorick 2c462486fe
create pathAccessible, use it to infer default dirs 2023-05-26 15:36:47 +02:00
Yorick a6c78ba367
getDefaultNixPath: ignore EPERM 2023-05-26 15:36:46 +02:00
Yorick b7cde90c6b
Call getDefaultSSLCertFile() only when none is specified
This does pathExists on various paths, which crashes on EPERM in the
macOS sandbox.
2023-05-26 15:36:45 +02:00
Silvan Mosberger be48907470
ci: Always run with sandbox, even on Darwin
And fix a test failure in the sandbox due to /home
existing on Darwin but not being accessible in the sandbox since it's a
symlink to /System/Volumes/Data/home, see
https://github.com/NixOS/nix/actions/runs/4205378453/jobs/7297384658#step:6:2127:

    C++ exception with description "error: getting status of /home/schnitzel/darmstadt/pommes: Operation not permitted" thrown in the test body.

On Linux this wasn't a problem because there /home doesn't exist in the sandbox
2023-05-26 15:36:44 +02:00
polykernel a382919d7d
primops: lazy evaluation of replaceStrings replacements
The primop `builtins.replaceStrings` currently always strictly evaluates the
replacement strings, however time and space are wasted for their computation
if the corresponding pattern do not occur in the input string. This commit
makes the evaluation of the replacement strings lazy by deferring their
evaluation to when the corresponding pattern are matched and memoize the result
for efficient retrieval on subsequent matches.

The testcases for replaceStrings was updated to check for lazy evaluation
of the replacements. A note was also added in the release notes to
document the behavior change.
2023-05-25 18:35:23 -04:00
Théophane Hufschmitt d16a1994fb Properly report build errors on chrooted stores
When encountering a build error, Nix moves the output paths out of the
chroot into their final location (for “easier debugging of build
failures”). However this was broken for chroot stores as it was moving
it to the _logical_ location, not the _physical_ one.

Fix it by moving to the physical (_real_) location.

Fix https://github.com/NixOS/nix/issues/8395
2023-05-25 16:38:29 +02:00
Aneesh Agrawal c694f1a2f3 Remove old default from docs for hashed-mirrors
The `hashed-mirrors` option did use to have this default value,
but it was removed and re-added with an empty default value.
As the autogenerated docs show the (actual) default values from code,
remove this incorrect reference from the docs.

I was updating my nix.conf settings after a few years and noticed this.
2023-05-24 11:05:40 -04:00
Théophane Hufschmitt 34e1b464f0 Normalize the hash-rewriting process when building derivations 2023-05-24 14:11:50 +02:00
Théophane Hufschmitt a917fb0d53 Use a RewritingSink in derivation goal
Possibly this will make it stream
2023-05-24 14:11:50 +02:00
Théophane Hufschmitt 3ebe1341ab Make RewritingSink accept a map of rewrites
Giving it the same semantics as `rewriteStrings`.
Also add some tests for it
2023-05-24 14:11:50 +02:00
oxalica 303858afad Fix typo in error message of too long store path 2023-05-24 01:24:28 +08:00
Peter Becich a420ccc6a8
nix flake check: skip derivations for foreign systems (#7759)
`nix flake show` now skips derivations for foreign systems: https://github.com/NixOS/nix/pull/6988

This commit borrows from that to implement the same behavior for `nix flake check`.

See "nix flake check breaks on IFD in multi-platform flake" https://github.com/NixOS/nix/issues/4265
2023-05-23 06:59:44 +02:00
Théophane Hufschmitt 494a09c6df
Merge pull request #8377 from layus/fix-ssl-cert-mount
Make mounting ssl cert file optional
2023-05-22 14:06:57 +02:00
Théophane Hufschmitt 673fe85976
Merge pull request #8365 from obsidiansystems/proto-structs
Revert "Revert "Use template structs instead of phantoms""
2023-05-22 09:34:34 +02:00
Guillaume Maudoux b14fea6fff Shortcircuit for empty caFile 2023-05-19 23:30:35 +02:00
Guillaume Maudoux 36b7e30c11 Make mounting ssl cert file optional 2023-05-19 22:47:40 +02:00
Eelco Dolstra 3305fd0cb1
Merge pull request #8354 from KasyanDiGris/git-fetcher-ask-credentials
Ask for git credentials in fetcher
2023-05-19 13:40:43 +02:00
Eelco Dolstra 34381d5747
Merge pull request #8215 from obsidiansystems/general-repair-path
Support `repairPath` on most stores.
2023-05-19 13:39:48 +02:00
Eelco Dolstra e31d9b910d
Merge pull request #7312 from layus/fixed-output-system-cert
Allow system certs access to fixed-output derivations
2023-05-19 13:05:16 +02:00
Robert Hensing b1c34c0ee8
Merge pull request #8366 from obsidiansystems/worker-proto-forward-decl-types
Don't use `store-api.hh` in `worker-protocol.hh`
2023-05-18 14:33:52 +02:00
Konstantin Vukolov 31ffd0c1fe Remove unused 2023-05-18 13:26:23 +03:00
Konstantin Vukolov 4c4ae887b8 Add option isInteractive 2023-05-18 13:18:34 +03:00
John Ericson 9923403d90 Don't use store-api.hh in worker-protocol.hh
Using abstract types like can help cut down on compilation time, both
from scratch, and especially incremental builds during development. The
idea is that `worker-protocol.hh` can declare all the (de)serializers, but
only again abstract types; when code needs to use some (de)serializers, it can
include headers just for the data types it needs to (de)serialize.

`store-api.hh` in particular is a bit of a sledgehammer, and the data
types we want to serialize have their own headers.
2023-05-18 00:20:24 -04:00
John Ericson cb5052d98f Revert "Revert "Use template structs instead of phantoms""
This is the more typically way to do [Argument-dependent
lookup](https://en.cppreference.com/w/cpp/language/adl)-leveraging
generic serializers in C++. It makes the relationship between the `read`
and `write` methods more clear and rigorous, and also looks more
familiar to users coming from other languages that do not have C++'s
libertine ad-hoc overloading.

I am returning to this because during the review in
https://github.com/NixOS/nix/pull/6223, it came up as something that
would make the code easier to read --- easier today hopefully already,
but definitely easier if we were have multiple codified protocols with
code sharing between them as that PR seeks to accomplish.

If I recall correctly, the main criticism of this the first time around
(in 2020) was that having to specify the type when writing, e.g.
`WorkerProto<MyType>::write`, was too verbose and cumbersome. This is
now addressed with the `workerProtoWrite` wrapper function.

This method is also the way `nlohmann::json`, which we have used for a
number of years now, does its serializers, for what its worth.

This reverts commit 45a0ed82f0. That
commit in turn reverted 9ab07e99f5.
2023-05-17 22:44:47 -04:00
John Ericson b9e5ce4a27 Upgrade downstreamPlaceholder to a type with methods
This gets us ready for dynamic derivation dependencies (part of RFC 92).
2023-05-17 17:41:16 -04:00
John Ericson e7c1113a37 Add test for downstreamPlaceholder
This is good in general, but in particular ensures when we heavily
refactor it in the next commit there is less likelihood for an
unintentional change in behavior to sneak in.
2023-05-17 17:31:33 -04:00
John Ericson 32dc77ba5d
Merge pull request #8349 from tweag/fix-control-master
Fix ControlMaster behaviour
2023-05-17 12:17:09 -04:00
John Ericson 05cb93442c
Merge pull request #8360 from obsidiansystems/worker-protocol.cc
Move `worker_proto` defs out of `remote-store.cc` to own file
2023-05-17 11:04:29 -04:00
John Ericson 904878d6d2 Move worker_proto defs out of remote-store.cc to own file
These items are not templates, and they declared in
`worker-protocol.hh`; therefore they should live in a
`worker-protocol.cc`.

Anything else needlessly diverges from convention. After all, it is not
like this code is only used in `remote-store.cc`; it is also used in
`daemon.cc`. There is no good reason to place it with the client
implementation or the server implementation when it used equally by
both.
2023-05-17 10:36:03 -04:00
Valentin Gagarin 6a5a8f51bb add cross-references to pure evaluation mode
use consistent wording everywhere.
add some details on the configuration option documentation.
2023-05-17 15:01:54 +02:00
Alexander Schmolck afd9bd787d Also use long options in src/nix/*.md 2023-05-17 08:10:30 +01:00
Konstantin Vukolov 25434df0d9 Ask for git credentials in fetcher 2023-05-17 02:00:32 +03:00
John Ericson 5fd161189d
Merge pull request #8346 from tweag/fix-nix-profile-install-conflict-segfault
Fix the segfault on `nix profile install` with conflict
2023-05-16 15:49:43 -04:00
Alexander Bantyev 61cdb0b057
Fix ControlMaster behaviour 2023-05-16 18:50:09 +04:00
John Ericson 0a715ff9cf
Merge pull request #8154 from tweag/delete-old-on-all-profiles-dir
undefined
2023-05-16 09:51:41 -04:00
Théophane Hufschmitt e997512523 Fix the printing of the installables on nix profile install conflict
- If the element comes from a flake, print the full flakeref (with the
  fragment part) and not just the reference to the flake itself
- If the element doesn't come from a flake, print its store path(s)

This is a bit too verbose, but has the advantages of being correct (and
not crashing), so it's strictly better than the previous situation

Fix https://github.com/NixOS/nix/issues/8284
2023-05-16 11:41:58 +02:00
Théophane Hufschmitt 3d144e778e Rename ProfileElement.describe() to .identifier
This method isn't used to describe what the element is, but to return a
unique identifier for it whithin the current profile
2023-05-16 11:41:58 +02:00
John Ericson f8a6a9e473
Merge pull request #8341 from obsidiansystems/dedup-gen-hh
Dedup some markdown -> C++ big literal stuff in build system
2023-05-15 11:44:35 -04:00
Robert Hensing 41591b33a9
Merge pull request #8340 from NixOS/delete-commited-build-artifacts
Delete commited build artifacts
2023-05-15 17:30:20 +02:00
John Ericson 98afd6ff76 Delete commited build artifacts
They were improperly added in 8a93b5a551.

They were not `.gitignore`d because they were stale in that commit --
build artifacts no longer used that name by then and so `.gitignore` was
updated accordingly.
2023-05-15 10:50:33 -04:00
John Ericson bbd7d5de09 Fix some bounds in rapid check instances
`inRange` is exclusive not inclusive:
https://github.com/emil-e/rapidcheck/blob/master/doc/generators.md#usage

Furthermore, use `std::variant_size_v` so we use the right number
automatically.

Finally, make the `switch` assert the discriminant is in bounds as
expected.
2023-05-15 10:41:44 -04:00
John Ericson f7c896cfd8 Dedup some markdown -> C++ big literal stuff in build system
This pattern rule was unwisely hidden in `src/libstore/local.mk`. Now it
is properly in `mk/` and we reuse it for the profile docs too.
2023-05-15 10:38:11 -04:00
John Ericson d2162e7acd Make more string values work as installables
As discussed in #7417, it would be good to make more string values work
as installables. That is to say, if an installable refers to a value,
and the value is a string, it used to not work at all, since #7484, it
works somewhat, and this PR make it work some more.

The new cases that are added for `BuiltPath` contexts:

- Fixed input- or content-addressed derivation:

  ```
  nix-repl> hello.out.outPath
  "/nix/store/jppfl2bp1zhx8sgs2mgifmsx6dv16mv2-hello-2.12"

  nix-repl> :p builtins.getContext hello.out.outPath
  { "/nix/store/c7jrxqjhdda93lhbkanqfs07x2bzazbm-hello-2.12.drv" = { outputs = [ "out" ]; }; }

  The string matches the specified single output of that derivation, so
  it should also be valid.

- Floating content-addressed derivation:

  ```
  nix-repl> (hello.overrideAttrs (_: { __contentAddressed = true; })).out.outPath
  "/1a08j26xqc0zm8agps8anxpjji410yvsx4pcgyn4bfan1ddkx2g0"

  nix-repl> :p builtins.getContext (hello.overrideAttrs (_: { __contentAddressed = true; })).out.outPath
  { "/nix/store/qc645pyf9wl37c6qvqzaqkwsm1gp48al-hello-2.12.drv" = { outputs = [ "out" ]; }; }
  ```

  The string is not a path but a placeholder, however it also matches
  the context, and because it is a CA derivation we have no better
  option. This should also be valid.

We may also want to think about richer attrset based values (also
discussed in that issue and #6507), but this change "completes" our
string-based building blocks, from which the others can be desugared
into or at least described/document/taught in terms of.

Progress towards #7417

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-05-15 09:03:38 -04:00
John Ericson 5a23b80b0a Create EvalState::coerceToDerivedPath
This gives us some round trips to test.

`EvalState::coerceToDerivedPathUnchecked` is a factored out helper just
for unit testing.
2023-05-15 09:03:37 -04:00
John Ericson 8e1a990268 Expose mkOutputString as method of EvalState 2023-05-15 09:03:37 -04:00
John Ericson 0a9afce3b9 Split mkOutputString in two
This well help us with some unit testing
2023-05-15 09:03:37 -04:00
John Ericson 9550c3862f Remove unneeded argument for mkOutputString 2023-05-15 09:03:37 -04:00
John Ericson 914672dc4f
Merge pull request #8141 from tweag/user-files-doc
Document user files of nix
2023-05-15 07:11:47 -04:00
Théophane Hufschmitt 64b0cc5bc4 Make nix-collect-garbage -d look into more places
Make it look into the new-style profiles dir, the old-style one, and the
target of `~/.nix-profile` to be sure that we don't miss anything
2023-05-15 11:36:53 +02:00
Matej Urbas c66a7af0c6 max-substitution-jobs release note entry 2023-05-14 09:51:11 +01:00
Eelco Dolstra 643b8d2126
Merge pull request #8299 from urbas/max-substitution-jobs
`max-substitution-jobs` setting
2023-05-12 12:24:27 +02:00
John Ericson a93110ab19 Fix nix print-dev-env & nix develop with drv paths
Fixes #8309

This regression was because both `CmdDevelop` and `CmdPrintDevEnv` were
switched to be `InstallableValueCommand` subclasses, but actually
neither should have been.

The `nixpkgsFlakeRef` method should indeed not be on the base
installable class, because "flake refs" and "nixpkgs" are not
installable-wide notions, but that doesn't mean these commands should
only accept installable values.
2023-05-10 11:29:45 -04:00
John Ericson 53a1354acf
Merge pull request #3959 from obsidiansystems/ca-drv-exotic
Derivations can output "text-hashed" data
2023-05-10 10:41:59 -04:00
Théophane Hufschmitt 85ff212051
Merge pull request #7721 from yorickvP/post-build-hook
Also pass unwanted outputs to post-build-hook
2023-05-10 14:30:42 +02:00
John Ericson 6a3a87a714 Improve error message for self reference with text hashing
The `ContentAddressWithReferences` method is made total, with error
handling now squarely the caller's job. This is better.
2023-05-09 14:44:08 -04:00
John Ericson e514b3939a Add name to some error messages 2023-05-09 13:24:53 -04:00
John Ericson 753fc1661d Cleanups to content address types 2023-05-09 13:05:38 -04:00
John Ericson d3c125e5a8
Apply suggestions from code review
Thanks!

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-05-09 12:45:51 -04:00
John Ericson 6513f4fe92 Fix bug, newInfo -> newInfo0
It appears we were checking a variable in the process of definining it.
2023-05-09 12:31:36 -04:00
John Ericson 35dcbe1c21 Fix spurious change
Didn't mean to use the private name that shouldn't be exposed.
2023-05-09 12:19:03 -04:00
Alex Ameen 82296f8113
prevent double quotation 2023-05-09 09:59:18 -05:00
Alex Ameen b72bc4a972
libexpr: quote reserved keys when printing
This fixes a bug in commands like `nix eval' which would emit invalid attribute
sets if they contained reserved keywords such as "assert", "let", etc.

These keywords will not be quoted when printed, making them valid expressions.
All keywords recognized by the lexer are quoted except "or", which does not
require quotation.
2023-05-09 09:45:12 -05:00
Matej Urbas 13185133bc introduces Goal::jobCategory 2023-05-08 19:45:46 +01:00
Matej Urbas 1ea1e378de removes MaxSubstitutionJobsSetting 2023-05-08 19:21:57 +01:00
John Ericson b5d9ef0a4c
Merge pull request #3921 from obsidiansystems/trustless-remote-builder-simple
Trustless remote building for input-addressed drvs
2023-05-08 10:43:37 -04:00
John Ericson df53a7d268 Split comment, match with each variable 2023-05-08 10:08:01 -04:00
John Ericson cd0d8e0bd5
Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-05-08 09:57:05 -04:00