Commit graph

7111 commits

Author SHA1 Message Date
regnat cbbd21ec07 Factor out the path realisation bit of IFD 2021-12-21 09:36:19 +01:00
pennae 09b245690a bulk-allocate Value instances in the evaluator
calling GC_malloc for each value is significantly more expensive than
allocating a bunch of values at once with GC_malloc_many. "a bunch" here
is a GC block size, ie 16KiB or less.

this gives a 1.5% performance boost when evaluating our nixos system.

tested with

nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'

 # on master

  Time (mean ± σ):      3.335 s ±  0.007 s    [User: 2.774 s, System: 0.293 s]
  Range (min … max):    3.315 s …  3.347 s    50 runs

 # with this change

  Time (mean ± σ):      3.288 s ±  0.006 s    [User: 2.728 s, System: 0.292 s]
  Range (min … max):    3.274 s …  3.307 s    50 runs
2021-12-20 23:01:28 +01:00
Nicolas Mattia 9dea5768ef Clean up toml parsing code 2021-12-20 21:39:37 +01:00
Naïm Favier 1827cf5a9b
Add shell completion for why-depends
Fixes #5806
2021-12-20 19:58:30 +01:00
Nicolas Mattia 5d06836b9f Use long in fromTOML 2021-12-17 23:15:26 +01:00
Nicolas Mattia 10a8b5d3ae Update primops/fromTOML.cc to use toml11 2021-12-17 22:13:04 +01:00
Nicolas Mattia c3f68b5db7 Replace cpptoml with toml11 2021-12-17 22:03:33 +01:00
Eelco Dolstra 6e6e998930
Merge pull request #5787 from edolstra/unshare-fs
Ignore EPERM when unsharing FS state
2021-12-17 12:00:09 +01:00
Théophane Hufschmitt 23ea1e46cc
Merge pull request #5047 from symphorien/fix-nix-channel
nix-channel: use nix-env -i --remove-all to upgrade
2021-12-17 10:05:48 +01:00
regnat ab902521b1 Allow paths in flake local settings
Fix #5505
2021-12-17 09:31:13 +01:00
Eelco Dolstra ec8f24ed3a Ignore EPERM when unsharing FS state
On Docker (but not podman), unshare(CLONE_FS) fails with EPERM. So
let's ignore it and hope nothing bad happens.

Attempted fix for #5777.
2021-12-16 22:02:50 +01:00
Noah Snelson c3f9344cae Fix typo in src/nix/shell.md
Changed `If not command` to `If no command`, also specified that the
default shell launched in `nix shell` can be specified with `$SHELL`.
2021-12-15 23:40:37 -08:00
Eelco Dolstra 18e4851752
Merge pull request #5769 from NixOS/ca/get-build-stats
Add a crude tracing mechansim for the build results
2021-12-14 10:53:51 +01:00
Théophane Hufschmitt 8868da45a8
Merge pull request #5623 from yorickvP/fix-5621
flakes: fix boolean and int nixConfig values
2021-12-14 10:35:37 +01:00
Théophane Hufschmitt 99d617bcde
Merge pull request #5771 from edolstra/single-file-flake-inputs
Re-allow inputs.x.url = "/path/to/file"
2021-12-14 09:17:24 +01:00
Théophane Hufschmitt 6234e1c811
Merge pull request #5702 from baloo/baloo/hide-non-reproducible-settings
reproducibility: hide non-reproducible settings from manual
2021-12-14 09:14:25 +01:00
Théophane Hufschmitt 3fc8042f77
Merge pull request #5366 from trofi/speedup-ca-query
ca-specific-schema.sql: add index on RealisationsRefs(referrer)
2021-12-14 09:08:17 +01:00
Eelco Dolstra 95bd5da341 queryMissing(): Use a bigger thread pool
This is a temporary hack until we make queryMissing() async.

Closes #5324.
2021-12-13 21:11:37 +01:00
Eelco Dolstra ade870764a
Merge pull request #5725 from ConnorBaker/master
Add support for .tar.gz shorthand (.tgz) to TarballInputScheme
2021-12-13 21:08:06 +01:00
Eelco Dolstra 32a62b0d25 Re-allow inputs.x.url = "/path/to/file" 2021-12-13 20:59:33 +01:00
Eelco Dolstra 46d86e06ba Simplify 2021-12-13 20:28:53 +01:00
Eelco Dolstra 9529de2eed Merge branch 'arm32-personality' of https://github.com/cleverca22/nix 2021-12-13 19:58:40 +01:00
Eelco Dolstra 6ae5f39ea0
Merge pull request #5770 from edolstra/gc-fixes
Fix macOS GC failures
2021-12-13 19:55:32 +01:00
Eelco Dolstra e2e5f3a78f Respect /etc/hosts
https://hydra.nixos.org/build/161439329
2021-12-13 19:37:30 +01:00
Eelco Dolstra cc6406cc59 Merge branch 'better-interpolation-error-location' of https://github.com/greedy/nix 2021-12-13 19:20:48 +01:00
Eelco Dolstra 2567b74c66 nix run: Tweak docs 2021-12-13 19:14:32 +01:00
Eelco Dolstra 775bdc0d9e Merge branch 'pname-main-program' of https://github.com/figsoda/nix 2021-12-13 19:09:36 +01:00
Eelco Dolstra d17d46cfc2 More debug info 2021-12-13 19:03:08 +01:00
Eelco Dolstra 05fcc681ac Retry on ECONNREFUSED
https://hydra.nixos.org/build/161439235
2021-12-13 19:00:19 +01:00
Eelco Dolstra 6d30f9e6fe Explicitly make GC roots client sockets blocking
On macOS / BSD, these sockets inherit the non-blocking flag of the
server soocket, which is not what we want.

https://github.com/dotnet/runtime/issues/25069
https://bugs.python.org/issue7995

https://hydra.nixos.org/build/161439304
2021-12-13 18:59:50 +01:00
regnat 2eec2f765a Add a crude tracing mechansim for the build results
Add a `_NIX_TRACE_BUILT_OUTPUTS` environment variable that can be set to
a filename in which the result of each build will be logged.

This is intentionally crude and undocumented as it’s only meant to be a
temporary thing to assess the usefulness of CA derivations.
Any other use would need a cleaner re-implementation first.
2021-12-13 17:02:14 +01:00
regnat 55dbb7f1cc More properly track the status of CA builds
Make the build of unresolved derivations return the same status as the
resolved one, except in the case of an `AlreadyValid` in which case it
will return `ResolvesToAlreadyValid` to mean that the outputs of the unresolved
derivation weren’t known, but the resolved one is.
2021-12-13 17:02:13 +01:00
Eelco Dolstra 84497119ad
Merge pull request #5761 from yorickvP/fix-5741
extract_archive: fix "Hard-link target '...'" error
2021-12-13 13:37:40 +01:00
Yorick 33e96820d5
EvalCommand::getEvalState: use gc tracable allocator for EvalState 2021-12-11 16:02:08 +01:00
Gavin Ray 3542d4fe16
Incorporate suggestions from @edolstra 2021-12-10 19:02:22 -05:00
Yorick 0e0de90b35
extract_archive: fix "Hard-link target '...'" error
Fixes #5741
2021-12-10 17:03:51 +01:00
Yorick be1055f2cc
extract_archive: use copy_pathname instead of set_pathname.
Libarchive documentation mentions that archive_entry_set_pathname
expects us to keep the passed string alive, which we don't.
2021-12-10 17:00:32 +01:00
Eelco Dolstra 20b1290103 Ignore system.nfs4_acl
Closes #1584.
2021-12-10 15:12:55 +01:00
Eelco Dolstra f9437b4f18 Merge branch 'master' of https://github.com/MagicRB/nix 2021-12-10 14:49:25 +01:00
regnat 55e3a1108c Properly take the last assignment in the REPL
When a variable is assigned in the REPL, make sure to remove any possible reference to the old one so that we correctly pick the new one afterwards

Fix #5706
2021-12-09 16:35:27 +01:00
Eelco Dolstra bc57b3854e
Merge pull request #5695 from obsidiansystems/tidy-logging
Tidy up the logging
2021-12-09 15:31:08 +01:00
Gavin Ray 971382cab0
Better diagnostics if no valid signature found
I downloaded Nix tonight, and immediately broke it by accidentally removing the default binary caching.
After figuring this out, I also failed to fix it properly, due to using the wrong key for Nix's default binary cache

If the diagnostic message would have been clearer about what/where a "signature" for a "substituter" is + comes from, it probably would have saved me a few hours.
Maybe we can save other noobs the same pain?
2021-12-08 19:55:34 -05:00
Eelco Dolstra 92ff5b4254 Tweak warning 2021-12-07 22:22:02 +01:00
Eelco Dolstra 21206caf13 Merge remote-tracking branch 'origin/5712-ca-drvs-with-old-daemon' 2021-12-07 22:06:22 +01:00
Eelco Dolstra 8b5088b62f Merge branch 'balsoft/nix-repl-log' of https://github.com/tweag/nix 2021-12-07 21:33:32 +01:00
Jan Tojnar ae21aab456 Update manual links
Fixes: https://github.com/NixOS/nixos-homepage/issues/762
2021-12-06 16:42:57 +01:00
Alexander Bantyev 33926ed1e7
nix repl: add :log
Add a :log command that shows logs for a derivation.

Closes https://github.com/NixOS/nix/issues/3504

Co-authored-by: Taeer Bar-Yam <taeer@bar-yam.me>
2021-12-06 16:52:50 +03:00
Alexander Bantyev 0a2fa2d684
RunPager: restore stdout upon pager exit
Before this change, stdout was closed after the pager exits. This is
fine for non-interactive commands where we want to exit right after
the pager exits anyways, but for interactive things (e.g. nix repl)
this breaks the output after we quit the pager.

Keep the initial stdout fd as part of RunPager, and restore it in
RunPager::~RunPager using dup2.
2021-12-06 16:51:55 +03:00
regnat fa58bff1ab Fallback when the daemon is too old for CA derivations
Fix #5712
2021-12-06 11:45:18 +01:00
Connor Baker 52f5fa948a Add support for .tar.gz shorthand (.tgz) to TarballInputScheme 2021-12-03 22:16:02 -05:00
Silvan Mosberger 90700736c7 Introduce builtins.groupBy primop
This function is very useful in nixpkgs, but its implementation in Nix
itself is rather slow due to it requiring a lot of attribute set and
list appends.
2021-12-02 21:54:51 +01:00
Eelco Dolstra 2ff71b0213
Merge pull request #5713 from tweag/balsoft/another-follows-bugfix
Flakes: computeLocks: pass correct LockParent when reusing oldLock
2021-12-02 13:17:50 +01:00
Eelco Dolstra 77b3e4b4ee
Merge pull request #5708 from trofi/fix-build-on-gcc-11
src/libutil/util.hh: fix build on gcc-11
2021-12-02 13:16:15 +01:00
Alexander Bantyev 8cbf862e6f
Flakes: computeLocks: pass correct LockParent when reusing oldLock
Previously, when we were attempting to reuse the old lockfile
information in the computeLocks function, we have passed the parent of
the current input to the next computeLocks call. This was incorrect,
since the follows are resolved relative to the parent. This caused
issues when we tried to reuse oldLock but couldn't for some
reason (read: mustRefetch is true), in that case the follows were
resolved incorrectly.

Fix this by passing the correct parent, and adding some tests to
prevent this particular regression from happening again.

Closes https://github.com/NixOS/nix/issues/5697
2021-12-02 12:02:34 +03:00
David Arnold 7489811736
Fix parent path check boundary
- Previous to this commit the boundary was exclusive of the
  top level flake.
- This is wrong since the top level flake is still a valid
  relative reference.
- Now, the check boundary is inclusive of the top level flake.

Signed-off-by: Timothy DeHerrera <tim.deh@pm.me>
2021-12-01 17:01:48 -07:00
Sergei Trofimovich c32a5f4d38 src/libutil/util.hh: fix build on gcc-11
Due to missing <atomic> declaration the build fails as:

    src/libutil/util.hh:350:24: error: no match for 'operator||' (operand types are 'std::atomic<bool>' and 'bool')
      350 |     if (_isInterrupted || (interruptCheck && interruptCheck()))
          |         ~~~~~~~~~~~~~~ ^~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          |         |                                 |
          |         std::atomic<bool>                 bool
2021-12-01 22:08:05 +00:00
Eelco Dolstra 782837d934 Merge branch 'nix-env-install-suggestions' of https://github.com/tweag/nix 2021-12-01 21:47:28 +01:00
Eelco Dolstra fb662e0acf
Merge pull request #5696 from obsidiansystems/fix-5299
Fix #5299 and remove uncesssary unbounded buffer
2021-12-01 21:32:26 +01:00
Arthur Gautier 21520297da reproducibility: hide non-reproducible settings from manual
Because the manual is generated from default values which are themselves
generated from various sources (cpuid, bios settings (kvm), number of
cores). This commit hides non-reproducible settings from the manual
output.
2021-12-01 17:25:58 +01:00
Théophane Hufschmitt f3f32f0c30
Merge pull request #5599 from fzakaria/faridzakaria/fix-progress-bar-leak
Fix heap use after free in progress-bar.cc
2021-12-01 16:28:38 +01:00
John Ericson 2f5c913d4a Fix #5299
No matter what, we need to resize the buffer to not have any scratch
space after we do the `read`. In the end of file case, `got` will be 0
from it's initial value.

Before, we forgot to resize in the EOF case with the break. Yes, we know
we didn't recieve any data in that case, but we still have the scatch
space to undo.

Co-Authored-By: Will Fancher <Will.Fancher@Obsidian.Systems>
2021-11-30 22:11:46 +00:00
John Ericson 95157b4e66 Push wopAddToStore old style stream adapters into smaller scopes
This doesn't fix the bug, but makes the code less difficult to read.
Also improve the comments, now that it is clear what part is needed in
each code path.
2021-11-30 22:10:58 +00:00
John Ericson 70a717f7a8 Nix daemon stream old wopAddToStore
No more buffering in string.
2021-11-30 21:55:53 +00:00
John Ericson 1f15441103 Tidy up the logging
Use the macros more, so we properly skip work when the log level
excludes. Also log the daemon operation number on the daemon side.
2021-11-30 20:23:13 +00:00
Yorick bd628cf3da
flakes: fix boolean and int nixConfig values
Some type confusion was causing ints to be pointers, and bools
to be ints. Fixes #5621
2021-11-29 15:53:22 +01:00
Eelco Dolstra 6e0cbc666b createTempFile(): Mark file as CLOEEXEC
Fixes #5674.
2021-11-29 11:20:50 +01:00
Eelco Dolstra e1420c66a4
Merge pull request #5680 from andir/libexpr-dont-move-primop-args
Don't move the arguments of the primOp
2021-11-29 11:11:54 +01:00
Eelco Dolstra 3d36f48acb
Merge pull request #5667 from NixOS/5661-dont-complete-flakes-without-xp-feature
Don’t try to complete flakes is the feature isn’t enabled
2021-11-29 10:51:41 +01:00
Andreas Rammhold 90d8178009
Don't move the arguments of the primOp
Moving arguments of the primOp into the registration structure makes it
impossible to initialize a second EvalState with the correct primOp
registration. It will end up registering all those "RegisterPrimOp"'s
with an arity of zero on all but the 2nd instance of the EvalState.

Not moving the memory will add a tiny bit of memory overhead during the
eval since we need a copy of all the argument lists of all the primOp's.
The overhead shouldn't be too bad as it is static (based on the amonut
of registered operations) and only occurs once during the interpreter
startup.
2021-11-28 02:06:47 +01:00
Alex Kladov 1d5a881da5
Fix typo 2021-11-27 23:29:48 +03:00
regnat 3a0277305a Don’t try to complete flakes is the feature isn’t enabled
Fix #5661
2021-11-26 16:56:51 +01:00
Eelco Dolstra 3ef66cd23a nix repl: Reset the terminal on exceptional exits 2021-11-26 13:10:28 +01:00
Eelco Dolstra b653fb9ccf
Merge pull request #5662 from NixOS/5259-nix-shell-with-ca-bash
Make `nix-shell` work when nixpkgs is content-addressed
2021-11-26 12:47:30 +01:00
Alex Shabalin 6d166d19a6 Catch flake-related exception type in REPL
Closes https://github.com/NixOS/nix/issues/5656
2021-11-26 11:03:48 +01:00
regnat 05081bedc1 Make nix-shell work when nixpkgs is content-addressed
Fix #5259
2021-11-26 09:56:48 +01:00
Eelco Dolstra 5fcf7f04a9
Merge pull request #5384 from baloo/baloo/dns-timeout
preloadNSS / dns timeout
2021-11-25 17:37:53 +01:00
Eelco Dolstra d5d0621250
Merge pull request #5603 from obsidiansystems/recursive-nix-system-feature
"recursive-nix" system feature only with experimental feature
2021-11-25 17:20:58 +01:00
Eelco Dolstra 6f46434f32
Merge pull request #5648 from edolstra/list-iter
Support range-based for loop over list values
2021-11-25 17:13:25 +01:00
Eelco Dolstra b6c8e57056 Support range-based for loop over list values 2021-11-25 16:31:39 +01:00
regnat c47027f3a1 Fix the error when accessing a forbidden path in pure eval
If we’re in pure eval mode, then tell that in the error message rather
than (wrongly) speaking about restricted mode.

Fix https://github.com/NixOS/nix/issues/5611
2021-11-25 14:48:01 +01:00
Eelco Dolstra 1f7584d24c
Merge pull request #5644 from tweag/fix-interrupted-data-race
Fix a minor data race with _isInterrupted
2021-11-25 13:55:03 +01:00
Las Safin 8388d2c7c6
Make recursive-nix work even when not privileged
Before this, `setns` would fail when switching to the mount namespace,
since we did not have the privileges to do so when not root.

Closes #5360
2021-11-25 11:15:11 +00:00
Eelco Dolstra d58f149140
Merge pull request #5631 from Infinisil/list-compare
Make lists be comparable
2021-11-24 15:48:05 +01:00
Alex Shabalin 2970ca18bf Fix a minor data race with _isInterrupted 2021-11-24 14:50:08 +01:00
Eelco Dolstra 884674a8e2 nix flake check: Fix markdown 2021-11-24 13:53:09 +01:00
Silvan Mosberger 09471d2680 Make lists be comparable
Makes lists comparable using lexicographic comparison.

Increments builtins.langVersion in order for this change to be
detectable
2021-11-24 13:40:46 +01:00
John Ericson 286eb81143 "recursive-nix" system feature only with experimental feature 2021-11-23 20:19:04 +00:00
Farid Zakaria d41af23a6c Fix heap use after free in progress-bar.cc
Fix some heap-use-after-free in progress-bar.cc

These are somewhat tricky failures here due to temporary variable
creation and string_view
2021-11-23 08:34:55 -08:00
Alex Shabalin b26cb0c9ac Fix use after free in content-address.cc
Inspired by https://github.com/NixOS/nix/pull/5599
2021-11-23 16:15:34 +01:00
Eelco Dolstra dbfcaa607a
Merge pull request #5628 from matthewbauer/rosetta2-new-path
Set new rosetta 2 path in sandbox
2021-11-23 10:46:50 +01:00
Matthew Bauer 08b1ac3e38 Set new rosetta 2 path in sandbox
see:

https://github.com/NixOS/nix/pull/5388

and

https://github.com/NixOS/nix/pull/5251
2021-11-22 14:42:31 -06:00
Eelco Dolstra b367f1061c
Merge pull request #5624 from rofrol/typo-single-quote
Typo: change to normal single quote
2021-11-22 21:33:21 +01:00
Eelco Dolstra 720ed47678
Merge pull request #5613 from tomberek/allow_realpath
add real path to allowedPaths
2021-11-22 15:10:47 +01:00
Eelco Dolstra dcaa8dfd10
Merge pull request #5620 from alyssais/wayland
nix-shell --pure: let variables for Wayland through
2021-11-22 15:09:52 +01:00
Alex Shabalin db2e4489a5 Unify #if linux 2021-11-22 13:57:56 +01:00
Roman Frołow 0768c08d99 Typo: change to normal singlequote 2021-11-22 13:37:38 +01:00
Alyssa Ross 664ee49e0d
nix-shell --pure: let variables for Wayland through
We let DISPLAY (X11) through, so we should let the Wayland equivalents
through as well.  Similarly, we let HOME through, so it should be okay
to allow XDG_RUNTIME_DIR (which is needed for connecting to Wayland
with WAYLAND_DISPLAY) through as well.  Otherwise graphical
applications will either fall back to X11 (if they support it), or
just not work (if they don't).
2021-11-21 16:14:19 +00:00
Tom Bereknyei 4318ba2ec5 add real path to allowedPaths 2021-11-20 00:25:36 -05:00
Alex Shabalin 86b7962807 Use warn to print a warning 2021-11-19 16:29:55 +01:00
Alex Shabalin a18d9269a5 Fix build warnings on MacOS 2021-11-19 15:22:31 +01:00