Commit graph

1529 commits

Author SHA1 Message Date
Yorick fcb8af550f
Restore parent mount namespace in restoreProcessContext
This ensures any started processes can't write to /nix/store (except
during builds). This partially reverts 01d07b1e, which happened because
of #2646.

The problem was only happening after nix downloads anything, causing
me to suspect the download thread. The problem turns out to be:
"A  process  can't  join a new mount namespace if it is sharing
filesystem-related attributes with another process", in this case this
process is the curl thread.

Ideally, we might kill it before spawning the shell process, but it's
inside a static variable in the getFileTransfer() function. So
instead, stop it from sharing FS state using unshare(). A strategy
such as the one from #5057 (single-threaded chroot helper binary) is
also very much on the table.

Fixes #4337.
2021-10-15 16:25:49 +02:00
Eelco Dolstra eab934cb2a Make the canReachRoots() traversal non-recursive 2021-10-14 12:34:32 +02:00
kvtb eae29b0385
fix build with gcc11 2021-10-13 18:03:33 +00:00
Eelco Dolstra dced45f146 strcpy -> memcpy
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
2021-10-13 12:12:44 +02:00
Eelco Dolstra 262520fcfe Use a thread per connection 2021-10-13 12:12:44 +02:00
Eelco Dolstra 8614cf1334 Non-blocking garbage collector
The garbage collector no longer blocks other processes from
adding/building store paths or adding GC roots. To prevent the
collector from deleting store paths just added by another process,
processes need to connect to the garbage collector via a Unix domain
socket to register new temporary roots.
2021-10-13 12:12:44 +02:00
Eelco Dolstra 0fac86fd6f Style tweaks 2021-10-13 11:00:10 +02:00
Tom Bereknyei 03bb8f84e0 Add compression level for NARs
Based off on @dtzWill's #2276
2021-10-12 02:14:36 -04:00
Eelco Dolstra c6718a9d95 Don't reset the logger in a vfork
9c766a40cb broke logging from the
daemon, because commonChildInit is called when starting the build hook
in a vfork, so it ends up resetting the parent's logger. So don't
vfork.

It might be best to get rid of vfork altogether, but that may cause
problems, e.g. when we call an external program like git from the
evaluator.
2021-10-06 13:54:59 +02:00
Eelco Dolstra e4a5d64a81 Show failing PID 2021-10-05 13:19:55 +02:00
Eelco Dolstra b299560872 Typo 2021-10-05 11:24:09 +02:00
Eelco Dolstra 43d4d75e22 Connect/bind Unix domain sockets in a child process
In the child process, we can do a chdir() and avoid the problem of the
path not fitting into sockaddr_un.
2021-10-05 10:44:59 +02:00
kvtb 638c73776a
fix creation of NAR files >4GB on 32-bit platforms
`size_t` is 32-bit on 32-bit platforms, while the file size can be bigger
2021-10-02 21:04:01 +00:00
Eelco Dolstra 9c766a40cb Fix 'error: reading a line: Input/output error' in startBuilder()
With -vvvv, the ProgressBar was polluting the stderr of the child,
messing up its \2 message to the parent.
2021-09-27 14:44:21 +02:00
Eelco Dolstra d8c10028d9 Make setDefault() typed 2021-09-22 14:15:35 +02:00
Eelco Dolstra 817562e694 Add "nix profile rollback" command 2021-09-14 19:32:33 +02:00
Eelco Dolstra a4ae601b44
Merge pull request #5245 from edolstra/warnings
Change warning messages from yellow to magenta
2021-09-14 11:46:34 +02:00
Eelco Dolstra 16d4922dd2
Merge pull request #5240 from edolstra/builtin-help
nix --help: Display help using lowdown instead of man
2021-09-14 11:31:23 +02:00
Eelco Dolstra 58b5036c54 Change warnings from yellow to magenta
This matches gcc and clang.
2021-09-14 10:42:29 +02:00
Eelco Dolstra 4ffda0af7c ANSI_YELLOW -> ANSI_WARNING 2021-09-14 10:42:29 +02:00
Eelco Dolstra 76e368a3b4 Fix macOS build 2021-09-14 10:07:31 +02:00
Eelco Dolstra 1ba993d07c Fix clang warning 2021-09-14 08:15:33 +02:00
Eelco Dolstra 4ed66735b6 RunOptions: Use designated initializers
Also get rid of _killStderr because it wasn't actually checked
anywhere.
2021-09-13 23:31:04 +02:00
Ben Burdette 176911102c printEnvPosChain 2021-09-13 11:57:25 -06:00
Eelco Dolstra 49a932fb18 nix --help: Display help using lowdown instead of man
Fixes #4476.
Fixes #5231.
2021-09-13 14:45:21 +02:00
Naïm Favier 7f0d177ce7
Add missing include in util.cc 2021-09-08 12:20:08 +02:00
Eelco Dolstra 50a35860ee TarArchive: Small refactoring 2021-08-30 17:02:51 +02:00
Eelco Dolstra c7a7652725 Don't segfault if archive_entry_pathname() returns null
Issues #4499.
2021-08-30 17:02:39 +02:00
Eelco Dolstra eda0fee160
Merge pull request #5175 from Pamplemousse/make
Don't overwrite user provided `lib*_LDFLAGS`
2021-08-30 12:44:29 +02:00
Robert Hensing f10465774f Force all Pos* to be non-null
This fixes a class of crashes and introduces ptr<T> to make the
code robust against this failure mode going forward.

Thanks regnat for the idea of a ref<T> without overhead!

Closes #4895
Closes #4893
Closes #5127
Closes #5113
2021-08-29 18:11:58 +02:00
Pamplemousse a4c6d319a8 Don't overwrite user provided lib*_LDFLAGS
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
2021-08-25 08:59:19 -07:00
Yaroslav Bolyukin bbb3bcb165
Replace invalid characters in json logger
Fixes #5159

Signed-off-by: Yaroslav Bolyukin <iam@lach.pw>
2021-08-22 00:26:22 +03:00
Pamplemousse 0da416f820 15f4d4f follow up
* libstore: `bz2` should not be linked
  * libutil: `zlib.h` should not be included

Signed-off-by: Pamplemousse <xav.maso@gmail.com>
2021-08-10 16:04:53 -07:00
Andreas Rammhold b9c9c25766
libutil: initialize the base64 decode array only once
Previously, despite having a boolean that tracked initialization, the
decode characters have been "calculated" every single time a base64
string was being decoded.

With this change we only initialize the decode array once in a
thread-safe manner.
2021-07-30 21:07:32 +02:00
Alyssa Ross 369ed71858
libutil: use uniform initialization in _deletePath
Otherwise I get a compiler error when building for NetBSD:

src/libutil/util.cc: In function 'void nix::_deletePath(const Path&, uint64_t&)':
src/libutil/util.cc:438:17: error: base operand of '->' is not a pointer
  438 |     AutoCloseFD dirfd(open(dir.c_str(), O_RDONLY));
      |                 ^~~~~
src/libutil/util.cc:439:10: error: 'dirfd' was not declared in this scope
  439 |     if (!dirfd) {
      |          ^~~~~
src/libutil/util.cc:444:17: error: 'dirfd' was not declared in this scope
  444 |     _deletePath(dirfd.get(), path, bytesFreed);
      |                 ^~~~~
2021-07-24 09:19:48 +00:00
Eelco Dolstra 140ccf1368 deletePath(): Return ENFILE instead of EBADF when out of file descriptors
Also remove an erroneous comment.
2021-07-20 20:59:45 +02:00
regnat 9b1f3cbc13 Forward the whole Nix config to the post-build-hook
Fill `NIX_CONFIG` with the value of the current Nix configuration before
calling the post-build-hook.
That way the whole configuration (including the possible
`experimental-features`, a possibly `--store` option or whatever) will
be made available to the hook
2021-07-15 18:41:56 +02:00
Eelco Dolstra 07790fdddf ref: Add equality operators 2021-07-09 14:03:48 +02:00
Domen Kožar f9d72855ae
Merge pull request #4967 from Pamplemousse/specific_errors
Prefer to throw specific errors
2021-07-03 10:34:56 +02:00
Yestin L. Harrison 0fe84bef72 Add $NO_COLOR check to ANSI escape conditions 2021-07-02 09:33:54 -06:00
Yestin L. Harrison 20cce079f2 Respect TERM=dumb more consistently 2021-07-01 18:19:01 -06:00
Pamplemousse 4a7a8b87cd Prefer to throw specific errors
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
2021-07-01 11:09:31 -07:00
regnat a22755721b Recursively substitute the realisations
Make sure that whenever we substitute a realisation, we also substitute
its entire closure
2021-05-26 18:44:17 +02:00
regnat 184558834a Extract a generic computeClosure function
Move the `closure` logic of `computeFSClosure` to its own (templated) function.

This doesn’t bring much by itself (except for the ability to properly
test the “closure” functionality independently from the rest), but it
allows reusing it (in particular for the realisations which will require
a very similar closure computation)
2021-05-19 11:44:58 +02:00
Matthew Bauer 3d90ab9345 Fix extra slash in canonPath output
When you have a symlink like:

  /tmp -> ./private/tmp

you need to resolve ./private/tmp relative to /tmp’s dir: ‘/’. Unlike
any other path output by dirOf, / ends with a slash. We don’t want
trailing slashes here since we will append another slash in the next
comoponent, so clear s like we would if it was a symlink to an absoute
path.

This should fix at least part of the issue in
https://github.com/NixOS/nix/issues/4822, will need confirmation that
it actually fixes the problem to close though.

Introduced in f3f228700a.
2021-05-18 16:38:55 -05:00
Eelco Dolstra 5385755227 Remove makeDecompressionSource()
This function doesn't support all compression methods (i.e. 'none' and
'br') so it shouldn't be exposed.

Also restore the original decompress() as a wrapper around
makeDecompressionSink().
2021-04-22 10:23:20 +02:00
Luke Granger-Brown 97dde3cdd9 libutil: allow decompression with none/empty method
The S3 store relies on the ability to be able to decompress things with
an empty method, because it just passes the value of the Content-Encoding
directly to decompress.

If the file is not compressed, then this will cause the compression
routine to get confused.

This caused NixOS/nixpkgs#120120.
2021-04-22 02:33:59 +00:00
Eelco Dolstra 8d651a1f68 Fix brotli compression of files > 128 KiB
This has been broken since faa31f4084.
2021-04-20 22:54:49 +02:00
Eelco Dolstra 3ee0ecdda0
Merge pull request #4679 from ony/feature/one-pass-canon-path
Optimize canonPath to resolve relative symlinks in one pass
2021-04-15 14:11:51 +02:00
Eelco Dolstra 15f4d4fd43 Drop libbz2 / zlib / lzma dependency + style fixes 2021-04-15 13:55:22 +02:00
Eelco Dolstra 6fb7582413 Merge branch 'libarchive-decompress' of https://github.com/serokell/nix 2021-04-15 13:39:04 +02:00
Eelco Dolstra 906adadacd Restore stack size in child processes
Fixes #4673.
2021-04-07 13:40:13 +02:00
Eelco Dolstra 9b9e703df4 restoreSignals() + restoreAffinity() -> restoreProcessContext() 2021-04-07 13:10:02 +02:00
Eelco Dolstra 8a29052cb2 PathSubstitutionGoal: Clean up pipe
If there were many top-level goals (which are not destroyed until the
very end), commands like

  $ nix copy --to 'ssh://localhost?remote-store=/tmp/nix' \
    /run/current-system --no-check-sigs --substitute-on-destination

could fail with "Too many open files". So now we do some explicit
cleanup from amDone(). It would be cleaner to separate goals from
their temporary internal state, but that would be a bigger refactor.
2021-04-07 12:21:31 +02:00
Mykola Orliuk f3f228700a canonPath in one pass 2021-03-31 04:58:49 +02:00
Eelco Dolstra 4638bcfb2c Fix some typos
Fixes #4671.
2021-03-26 16:14:38 +01:00
Maximilian Bosch ddcef3bcbb
Fix Nix to properly work with stores using a scoped IPv6 address
According to RFC4007[1], IPv6 addresses can have a so-called zone_id
separated from the actual address with `%` as delimiter. In contrast to
Nix 2.3, the version on `master` doesn't recognize it as such:

    $ nix ping-store --store ssh://root@fe80::1%18 --experimental-features nix-command
    warning: 'ping-store' is a deprecated alias for 'store ping'
    error: --- Error ----------------------------------------------------------------- nix
    don't know how to open Nix store 'ssh://root@fe80::1%18'

I modified the IPv6 match-regex accordingly to optionally detect this
part of the address. As we don't seem to do anything special with it, I
decided to leave it as part of the URL for now.

Fixes #4490

[1] https://tools.ietf.org/html/rfc4007
2021-03-16 19:14:42 +01:00
Yorick 8a0c00b856
Use libarchive for all compression 2021-03-10 22:34:29 +01:00
Eelco Dolstra 453c3a603f nix flake update: Recreate the lock file
This is probably what most people expect it to do. Fixes #3781.

There is a new command 'nix flake lock' that has the old behaviour of
'nix flake update', i.e. it just adds missing lock file entries unless
overriden using --update-input.
2021-02-26 14:55:54 +01:00
Eelco Dolstra 5edab777d1
Merge pull request #4530 from alyssais/kill
libutil: EPERM from kill(-1, ...) is fine
2021-02-26 12:19:16 +01:00
Shea Levy f6c5b05488
Respect command registrations in plugins. 2021-02-24 08:25:45 -05:00
Shea Levy 98d1b64400
Initialize plugins after handling initial command line flags
This is technically a breaking change, since attempting to set plugin
files after the first non-flag argument will now throw an error. This
is acceptable given the relative lack of stability in a plugin
interface and the need to tie the knot somewhere once plugins can
actually define new subcommands.
2021-02-24 08:22:17 -05:00
Daniël de Kok 2de232d2b3 Add x86_64 compute levels as additional system types
When performing distributed builds of machine learning packages, it
would be nice if builders without the required SIMD instructions can
be excluded as build nodes.

Since x86_64 has accumulated a large number of different instruction
set extensions, listing all possible extensions would be unwieldy.
AMD, Intel, Red Hat, and SUSE have recently defined four different
microarchitecture levels that are now part of the x86-64 psABI
supplement and will be used in glibc 2.33:

https://gitlab.com/x86-psABIs/x86-64-ABI
https://lwn.net/Articles/844831/

This change uses libcpuid to detect CPU features and then uses them to
add the supported x86_64 levels to the additional system types. For
example on a Ryzen 3700X:

$ ~/aps/bin/nix -vv --version | grep "Additional system"
Additional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux
2021-02-22 09:11:15 +01:00
Eelco Dolstra 37352aa7e1 Support --no-net for backwards compatibility 2021-02-07 20:44:56 +01:00
Alyssa Ross 7c112351d9
libutil: EPERM from kill(-1, ...) is fine
I tested a trivial program that called kill(-1, SIGKILL), which was
run as the only process for an unpriveleged user, on Linux and
FreeBSD.  On Linux, kill reported success, while on FreeBSD it failed
with EPERM.

POSIX says:

> If pid is -1, sig shall be sent to all processes (excluding an
> unspecified set of system processes) for which the process has
> permission to send that signal.

and

> The kill() function is successful if the process has permission to
> send sig to any of the processes specified by pid.  If kill() fails,
> no signal shall be sent.

and

> [EPERM]
>     The process does not have permission to send the signal to any
>     receiving process.

My reading of this is that kill(-1, ...) may fail with EPERM when
there are no other processes to kill (since the current process is
ignored).  Since kill(-1, ...) only attempts to kill processes the
user has permission to kill, it can't mean that we tried to do
something we didn't have permission to kill, so it should be fine to
interpret EPERM the same as success here for any POSIX-compliant
system.

This fixes an issue that Mic92 encountered[1] when he tried to review a
Nixpkgs PR on FreeBSD.

[1]: https://github.com/NixOS/nixpkgs/pull/81459#issuecomment-606073668
2021-02-07 13:56:50 +00:00
regnat d2091af231 Move the GENERATE_CMP macro to its own file
Despite being an ugly hack, it can probably be useful in a couple extra
places
2021-02-05 11:42:33 +01:00
Eelco Dolstra 8e758d402b Remove mkFlag() 2021-01-27 12:06:03 +01:00
Eelco Dolstra 36c4d6f592 Group common options 2021-01-25 19:03:13 +01:00
Eelco Dolstra 807d963ee8 Group subcommands by category 2021-01-25 18:19:32 +01:00
Eelco Dolstra b159d23800 Make '--help' do the same as 'help' (i.e. show a manpage) 2021-01-25 14:38:15 +01:00
Eelco Dolstra 488a826842
Merge pull request #4467 from edolstra/error-formatting
Improve error formatting
2021-01-25 12:50:57 +01:00
Eelco Dolstra b7bfc7ee52
Add FIXME 2021-01-22 12:36:50 +01:00
Eelco Dolstra 55849e153e Change error position formatting
It's now

  at /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix:7:7:

instead of

  at: (7:7) in file: /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix

The new format is more standard and clickable.
2021-01-21 11:02:09 +01:00
Eelco Dolstra 40608342cb Remove trailing whitespace 2021-01-21 11:02:09 +01:00
Eelco Dolstra 8d4268d190 Improve error formatting
Changes:

* The divider lines are gone. These were in practice a bit confusing,
  in particular with --show-trace or --keep-going, since then there
  were multiple lines, suggesting a start/end which wasn't the case.

* Instead, multi-line error messages are now indented to align with
  the prefix (e.g. "error: ").

* The 'description' field is gone since we weren't really using it.

* 'hint' is renamed to 'msg' since it really wasn't a hint.

* The error is now printed *before* the location info.

* The 'name' field is no longer printed since most of the time it
  wasn't very useful since it was just the name of the exception (like
  EvalError). Ideally in the future this would be a unique, easily
  googleable error ID (like rustc).

* "trace:" is now just "…". This assumes error contexts start with
  something like "while doing X".

Example before:

  error: --- AssertionError ---------------------------------------------------------------------------------------- nix
  at: (7:7) in file: /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix

       6|
       7|   x = assert false; 1;
        |       ^
       8|

  assertion 'false' failed
  ----------------------------------------------------- show-trace -----------------------------------------------------
  trace: while evaluating the attribute 'x' of the derivation 'hello-2.10'
  at: (192:11) in file: /home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/make-derivation.nix

     191|         // (lib.optionalAttrs (!(attrs ? name) && attrs ? pname && attrs ? version)) {
     192|           name = "${attrs.pname}-${attrs.version}";
        |           ^
     193|         } // (lib.optionalAttrs (stdenv.hostPlatform != stdenv.buildPlatform && !dontAddHostSuffix && (attrs ? name || (attrs ? pname && attrs ? version)))) {

Example after:

  error: assertion 'false' failed

         at: (7:7) in file: /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix

              6|
              7|   x = assert false; 1;
               |       ^
              8|

         … while evaluating the attribute 'x' of the derivation 'hello-2.10'

         at: (192:11) in file: /home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/make-derivation.nix

            191|         // (lib.optionalAttrs (!(attrs ? name) && attrs ? pname && attrs ? version)) {
            192|           name = "${attrs.pname}-${attrs.version}";
               |           ^
            193|         } // (lib.optionalAttrs (stdenv.hostPlatform != stdenv.buildPlatform && !dontAddHostSuffix && (attrs ? name || (attrs ? pname && attrs ? version)))) {
2021-01-21 11:02:09 +01:00
Eelco Dolstra 1bbc66f865 Merge branch 'slashes-in-github-branches' of https://github.com/Ma27/nix 2021-01-18 12:55:06 +01:00
Domen Kožar 00f99fdfe6
Merge pull request #4240 from bburdette/2259-error-message
2259 error message - "auto-call" error
2021-01-15 18:26:21 +00:00
Eelco Dolstra e21aee58f6 Fix tests 2021-01-08 14:17:06 +01:00
Eelco Dolstra 17beae299d Support binary unit prefixes in command line arguments 2021-01-08 12:51:19 +01:00
Eelco Dolstra 6548b89cc4 string2Int(): Return std::optional 2021-01-08 12:22:21 +01:00
Eelco Dolstra 29a445840a Remove unused mkFlag1 2021-01-08 11:42:44 +01:00
Eelco Dolstra 1d4954e73e Remove mkFlag integer specialisation 2021-01-08 11:40:36 +01:00
Eelco Dolstra 48a9be2aab Remove mkIntFlag 2021-01-08 10:44:55 +01:00
Eelco Dolstra a93916b190
Merge pull request #4336 from NixOS/manpages
Documentation for nix subcommands
2020-12-23 21:10:32 +01:00
Eelco Dolstra 1047cb1e53
Command: Remove examples() 2020-12-23 18:26:40 +01:00
Eelco Dolstra e27044216b Fix tests 2020-12-22 16:23:57 +01:00
Eelco Dolstra 724b7f4fb6 Don't log from inside the logger
This deadlocks ProgressBar, e.g.

  # nix run --impure --no-substitute --store '/tmp/nix2?store=/foo' --expr 'derivation { builder = /nix/store/zi90rxslsm4mlr46l2xws1rm94g7pk8p-busybox-1.31.1-x86_64-unknown-linux-musl/bin/busybox; }'

leads to

  Thread 1 (Thread 0x7ffff6126e80 (LWP 12250)):
  #0  0x00007ffff7215d62 in __lll_lock_wait () from /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libpthread.so.0
  #1  0x00007ffff720e721 in pthread_mutex_lock () from /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libpthread.so.0
  #2  0x00007ffff7ad17fa in __gthread_mutex_lock (__mutex=0x6c5448) at /nix/store/h31cy7jm6g7cfqbhc5pm4rf9c53i3qfb-gcc-9.3.0/include/c++/9.3.0/x86_64-unknown-linux-gnu/bits/gthr-default.h:749
  #3  std::mutex::lock (this=0x6c5448) at /nix/store/h31cy7jm6g7cfqbhc5pm4rf9c53i3qfb-gcc-9.3.0/include/c++/9.3.0/bits/std_mutex.h:100
  #4  std::unique_lock<std::mutex>::lock (this=0x7fffffff09a8, this=0x7fffffff09a8) at /nix/store/h31cy7jm6g7cfqbhc5pm4rf9c53i3qfb-gcc-9.3.0/include/c++/9.3.0/bits/unique_lock.h:141
  #5  std::unique_lock<std::mutex>::unique_lock (__m=..., this=0x7fffffff09a8) at /nix/store/h31cy7jm6g7cfqbhc5pm4rf9c53i3qfb-gcc-9.3.0/include/c++/9.3.0/bits/unique_lock.h:71
  #6  nix::Sync<nix::ProgressBar::State, std::mutex>::Lock::Lock (s=0x6c5448, this=0x7fffffff09a0) at src/libutil/sync.hh:45
  #7  nix::Sync<nix::ProgressBar::State, std::mutex>::lock (this=0x6c5448) at src/libutil/sync.hh:85
  #8  nix::ProgressBar::logEI (this=0x6c5440, ei=...) at src/libmain/progress-bar.cc:131
  #9  0x00007ffff7608cfd in nix::Logger::logEI (ei=..., lvl=nix::lvlError, this=0x6c5440) at src/libutil/logging.hh:88
  #10 nix::getCodeLines (errPos=...) at src/libutil/error.cc:66
  #11 0x00007ffff76073f2 in nix::showErrorInfo (out=..., einfo=..., showTrace=<optimized out>) at /nix/store/h31cy7jm6g7cfqbhc5pm4rf9c53i3qfb-gcc-9.3.0/include/c++/9.3.0/optional:897
  #12 0x00007ffff7ad19e7 in nix::ProgressBar::logEI (this=0x6c5440, ei=...) at src/libmain/progress-bar.cc:134
  #13 0x00007ffff7ab9d10 in nix::Logger::logEI (ei=..., lvl=nix::lvlError, this=0x6c5440) at src/libutil/logging.hh:88
  #14 nix::handleExceptions(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void ()>) (programName="/home/eelco/Dev/nix/outputs/out/bin/nix", fun=...) at src/libmain/shared.cc:328
  #15 0x000000000046226b in main (argc=<optimized out>, argv=<optimized out>) at /nix/store/h31cy7jm6g7cfqbhc5pm4rf9c53i3qfb-gcc-9.3.0/include/c++/9.3.0/ext/new_allocator.h:80
2020-12-22 11:15:29 +01:00
Eelco Dolstra 346baec783
Move doc() to Args 2020-12-21 13:32:23 +01:00
Eelco Dolstra a8f533b664 Add lvlNotice log level
This is like syslog's LOG_NOTICE: "normal, but significant,
condition".
2020-12-10 16:41:24 +01:00
Maximilian Bosch 93a8a005de
libstore/openStore: fix stores with IPv6 addresses
In `nixStable` (2.3.7 to be precise) it's possible to connect to stores
using an IPv6 address:

  nix ping-store --store ssh://root@2001:db8::1

This is also useful for `nixops(1)` where you could specify an IPv6
address in `deployment.targetHost`.

However, this behavior is broken on `nixUnstable` and fails with the
following error:

  $ nix store ping --store ssh://root@2001:db8::1
  don't know how to open Nix store 'ssh://root@2001:db8::1'

This happened because `openStore` from `libstore` uses the `parseURL`
function from `libfetchers` which expects a valid URL as defined in
RFC2732. However, this is unsupported by `ssh(1)`:

  $ nix store ping --store 'ssh://root@[2001:db8::1]'
  cannot connect to 'root@[2001:db8::1]'

This patch now allows both ways of specifying a store (`root@2001:db8::1`) and
also `root@[2001:db8::1]` since the latter one is useful to pass query
parameters to the remote store.

In order to achieve this, the following changes were made:

* The URL regex from `url-parts.hh` now allows an IPv6 address in the
  form `2001:db8::1` and also `[2001:db8::1]`.

* In `libstore`, a new function named `extractConnStr` ensures that a
  proper URL is passed to e.g. `ssh(1)`:

  * If a URL looks like either `[2001:db8::1]` or `root@[2001:db8::1]`,
    the brackets will be removed using a regex. No additional validation
    is done here as only strings parsed by `parseURL` are expected.

  * In any other case, the string will be left untouched.

* The rules above only apply for `LegacySSHStore` and `SSHStore` (a.k.a
  `ssh://` and `ssh-ng://`).

Unresolved questions:

* I'm not really sure whether we want to allow both variants of IPv6
  addresses in the URL parser. However it should be noted that both seem
  to be possible according to RFC2732:

  > This document incudes an update to the generic syntax for Uniform
  > Resource Identifiers defined in RFC 2396 [URL].  It defines a syntax
  > for IPv6 addresses and allows the use of "[" and "]" within a URI
  > explicitly for this reserved purpose.

* Currently, it's not supported to specify a port number behind the
  hostname, however it seems as this is not really supported by the URL
  parser. Hence, this is probably out of scope here.
2020-12-09 12:23:29 +01:00
Eelco Dolstra af373c2ece Add deprecated aliases for renamed commands 2020-12-03 22:45:44 +01:00
regnat a8a96dbaf8 Add forgotten override annotation 2020-12-02 14:23:38 +01:00
Eelco Dolstra 1b79b5b983 read(): Use char * instead of unsigned char *
This gets rid of some pointless casts.
2020-12-02 14:17:27 +01:00
Eelco Dolstra faa31f4084 Sink: Use std::string_view 2020-12-02 14:17:27 +01:00
Eelco Dolstra aa68486112 writeFull/writeFile: Use std::string_view 2020-12-02 14:17:27 +01:00
Eelco Dolstra 88798613ee replaceStrings(): Use std::string_view 2020-12-01 13:45:43 +01:00
Eelco Dolstra c0d1354b7d Macro hygiene 2020-12-01 13:45:06 +01:00
Eelco Dolstra e224c16d28 Macro hygiene 2020-12-01 13:43:33 +01:00
Dominique Martinet 1fd13d67e8 archive: disable preallocate-contents by default
using fallocate() to preallocate files space does more harm than good:
 - breaks compression on btrfs
 - has been called "not the right thing to do" by xfs developers
(because delayed allocation that most filesystems implement leads to smarter
allocation than what the filesystem needs to do if we upfront fallocate files)
2020-11-26 14:26:57 +01:00
Eelco Dolstra 0287f83057 Ask for confirmation before allowing flake Nix configuration settings 2020-11-26 12:37:23 +01:00
Eelco Dolstra ef84c780bb
filterANSIEscapes(): Handle UTF-8 characters 2020-11-16 16:41:53 +01:00
Ben Burdette 7d9037035e usage example location 2020-11-11 09:21:26 -07:00
Ricardo M. Correia 108a2dab7e Fix stack overflow introduced in #4206 2020-11-10 04:25:24 +01:00
Eelco Dolstra b87f84cf55
Fix appending to Setting<StringSet>
Fixes: warning: unknown setting 'extra-sandbox-paths'
2020-11-09 15:04:34 +01:00
Robert Hensing c4d903ddb0 Fix memory corruption caused by GC-invisible coroutine stacks
Crucially this introduces BoehmGCStackAllocator, but it also
adds a bunch of wiring to avoid making libutil depend on bdw-gc.

Part of the solutions for #4178, #4200
2020-10-30 21:21:59 +01:00
Eelco Dolstra dc5696b84f Fix test 2020-10-30 12:00:53 +01:00
Eelco Dolstra 7f56cf67ba Fix assertion failure in tab completion for --option 2020-10-29 18:26:35 +01:00
Eelco Dolstra ff4dea63c9 Generalize extra-* settings
This removes the extra-substituters and extra-sandbox-paths settings
and instead makes every array setting extensible by setting
"extra-<name> = <value>" in the configuration file or passing
"--<name> <value>" on the command line.
2020-10-29 18:17:39 +01:00
Eelco Dolstra 662e67f8de
Merge pull request #4198 from mkenigs/capitalize-JSON
Capitalize JSON for consistency
2020-10-29 07:34:34 +01:00
Matthew Kenigsberg 6a4bf535d8 Capitalize JSON for consistency 2020-10-28 17:54:28 -05:00
Eelco Dolstra a5019f0508 Consistency 2020-10-28 20:45:57 +01:00
Eelco Dolstra 14aecbb288 BaseSetting<StringMap>::set(): Don't append to previous value 2020-10-26 20:36:46 +01:00
Eelco Dolstra 9d5e9ef0da Move Explicit 2020-10-26 17:01:20 +01:00
John Ericson 64be1c15c2 Add missing include for MAX_PATH
And remove one that we didn't actually need to add
2020-10-15 19:05:17 +00:00
John Ericson fccef6a7fa Merge remote-tracking branch 'upstream/master' into fix-and-ci-static-builds 2020-10-15 18:55:03 +00:00
John Ericson aef44cbaa9 Split out commonChildInit 2020-10-11 16:38:46 +00:00
Eelco Dolstra 59bd6e87a4 Completions::add(): Guard against newlines 2020-10-09 21:55:59 +02:00
Eelco Dolstra ea4b2b985f
Merge pull request #4128 from tweag/extended-completions
Add a zsh completion script
2020-10-09 21:53:12 +02:00
John Ericson 39de73550d Merge remote-tracking branch 'upstream/master' into fix-and-ci-static-builds 2020-10-09 18:26:47 +00:00
Eelco Dolstra e845d19ae3 Remove Lazy
This fixes a crash during startup when compiling Nix as a single
compilation unit.
2020-10-09 17:54:59 +02:00
Eelco Dolstra 87157b2bd3 writeFile(): Add error context to writeFull() failure
Issue #4092.
2020-10-09 16:02:53 +02:00
regnat 04e5d0e704 Add a description in the completion outputs
Make nix output completions in the form `completion\tdescription`.
This can't be used by bash (afaik), but other shells like zsh or fish
can display it along the completion choices
2020-10-09 09:39:51 +02:00
Eelco Dolstra be149acfda Serialize exceptions from the sandbox process to the parent
Fixes #4118.
2020-10-07 16:34:03 +02:00
Eelco Dolstra 27ca87c46a Formatting 2020-10-07 16:33:19 +02:00
Maximilian Bosch 59f2dd8e8d
libfetchers/github: allow slashes in refs
Refs #4061
2020-10-06 20:08:51 +02:00
Eelco Dolstra 85c8be6286 Remove static variable name clashes
This was useful for an experiment with building Nix as a single
compilation unit. It's not very useful otherwise but also doesn't
hurt...
2020-10-06 13:49:20 +02:00
Eelco Dolstra 6691256e79 Factor out common showBytes() 2020-10-06 10:40:49 +02:00
Eelco Dolstra d0bb544128 Add missing #pragma once 2020-10-06 10:40:07 +02:00
Eelco Dolstra 88a667e49e
Fix s3:// store
Fixes https://github.com/NixOS/nixos-org-configurations/issues/123.
2020-10-05 17:53:30 +02:00
Kevin Quick 66c3959e8c
Merge branch 'master' into access-tokens 2020-09-29 08:32:06 -07:00
Gregory Hale faa5607f54 Merge remote-tracking branch 'origin/master' into github-api-token 2020-09-25 12:10:58 -04:00
John Ericson cfe791a638 stdout_ -> cout
Better to get creative than just sprinkle arbitrary underscores.
2020-09-25 11:30:04 -04:00
Eelco Dolstra 7b2ae472ff
expectArg(): Respect the 'optional' flag 2020-09-25 10:27:40 +02:00
Kevin Quick a439e9488d
Support StringMap configuration settings.
Allows Configuration values that are space-separated key=value pairs.
2020-09-24 22:42:59 -07:00
Eelco Dolstra 8d9402f411
Merge pull request #4054 from edolstra/fix-4021
registerOutputs(): Don't call canonicalisePathMetaData() twice
2020-09-23 21:57:53 +02:00
Eelco Dolstra 236d9ee7f7 lstat() cleanup 2020-09-23 19:17:28 +02:00
Dominique Martinet 2548347bba libutil/archive: add preallocate-contents option
Make archive preallocation (fallocate) optional because some filesystems
like btrfs do not behave as expected with fallocate.

See #3550.
2020-09-23 18:49:11 +02:00
Eelco Dolstra 92ac8df0ec Merge branch 'add-ca-to-store' of https://github.com/hercules-ci/nix 2020-09-22 11:31:33 +02:00
regnat c1e79f870c Silence a compiler warning in serialise.hh
Explicitely cast to `uint64_t` in `readNum` to avoid a "comparison
between signed and unsigned" warning
2020-09-22 10:39:29 +02:00
Eelco Dolstra ecc8672aa0 fmt.hh: Don't include boost/algorithm/string/replace.hpp
This cuts compilation time by ~49s.

Issue #4045.
2020-09-21 19:07:55 +02:00
Eelco Dolstra 557d2427ee Random header cleanup 2020-09-21 18:59:02 +02:00
Eelco Dolstra 0716adaa8b abstractsettingtojson.hh -> abstract-setting-to-json.hh 2020-09-21 18:49:43 +02:00
Eelco Dolstra d51ba43047 Move Callback into its own header
This gets rid of the inclusion of <future> in util.hh, cutting
compilation time by ~20s (CPU time).

Issue #4045.
2020-09-21 18:42:21 +02:00
Eelco Dolstra e8e1d420f3 Don't include <regex> in header files
This reduces compilation time by ~15 seconds (CPU time).

Issue #4045.
2020-09-21 18:22:45 +02:00
Robert Hensing 8279178b07 Move FramedSink next to FramedSource 2020-09-21 07:55:47 +02:00
Robert Hensing 14b30b3f3d Move FramedSource and FramedSink, extract withFramedSink 2020-09-17 20:21:04 +02:00
Robert Hensing 29c82ccc77 Add Source.drainInto(Sink) 2020-09-17 20:21:04 +02:00
Robert Hensing 3f93bc0d39 Typo 2020-09-17 20:21:04 +02:00
Greg Hale a303c0b6dc Fetch commits from github/gitlab using Auth header
`nix flake info` calls the github 'commits' API, which requires
authorization when the repository is private. Currently this request
fails with a 404.

This commit adds an authorization header when calling the 'commits' API.
It also changes the way that the 'tarball' API authenticates, moving the
user's token from a query parameter into the Authorization header.

The query parameter method is recently deprecated and will be disallowed
in November 2020. Using them today triggers a warning email.
2020-09-16 13:46:48 -04:00
regnat e0817cbcdc Don't include nlohmann/json.hpp in config.hh
Instead make a separate header with the template implementation of
`BaseSetting<T>::toJSONObj` that can be included where needed
2020-09-16 13:53:09 +02:00
regnat 93c0e14a30 Include the full nlohmann/json header in config.hh
It is apparently required for using `toJSONObject()`, which we do inside
the header file (because it's in a template).

This was accidentally working when building Nix itself (presumably because
`config.hh` was always included after `nlohman/json.hpp`) but caused a
(pretty dirty) build failure in the perl bindings package.
2020-09-16 13:53:09 +02:00
regnat a1e82ba450 fixup! Add a default value for the settings 2020-09-16 13:53:09 +02:00
regnat d65962db4d Make uri schemes grammar more RFC-compliant
Allow `-` and `.` in the RFC schemes as stated by
[RFC3986](https://tools.ietf.org/html/rfc3986#section-3.1).

Practically, this is needed so that `ssh-ng` is a valid URI scheme
2020-09-16 13:53:09 +02:00
regnat 35042c9623 Add a default value for the settings
The default value is initialized when creating the setting and unchanged
after that
2020-09-16 13:53:08 +02:00
regnat 3c525d1590 Complete the toJSON instance for Setting<T>
Don't let it just contain the value, but also the other fields of the
setting (description, aliases, etc..)
2020-09-16 13:53:08 +02:00
John Ericson 25f7ff16fa Merge remote-tracking branch 'upstream/master' into fix-and-ci-static-builds 2020-09-04 02:40:36 +00:00
John Ericson ef278d00f9 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-09-01 18:01:48 +00:00
Eelco Dolstra 84f5cabbea Merge remote-tracking branch 'origin/master' into markdown 2020-08-31 14:24:26 +02:00
John Ericson 8017fe7487 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-08-28 19:59:14 +00:00
Eelco Dolstra 691a1bd717 Merge branch 'minimal-logger' of https://github.com/Ma27/nix 2020-08-28 10:50:52 +02:00
Eelco Dolstra e915fd6d2a Typo 2020-08-27 14:51:50 +02:00
Eelco Dolstra a0f19d9f3a RemoteStore::addToStore(): Fix race between stderrThread and NAR writer
As pointed out by @B4dM4n, the call to to.flush() on stderrThread is
unsafe because the NAR writer thread is also writing to 'to'.

Fixes #3943.
2020-08-27 14:50:51 +02:00
Eelco Dolstra 24b1c2c66b
Fix tests 2020-08-25 10:51:14 +02:00
Eelco Dolstra 88d5c9ec58
Fix tests 2020-08-24 10:37:10 +02:00
Eelco Dolstra dc2f278c95
Allow 'nix' subcommands to provide docs in Markdown format 2020-08-20 12:21:46 +02:00
Eelco Dolstra 3c4f8c9175
List deprecated option aliases in the docs 2020-08-20 11:13:17 +02:00
Eelco Dolstra acb99f03f9
Config: Use nlohmann/json 2020-08-20 11:02:16 +02:00
Eelco Dolstra c8fa39324a
Generate the nix.conf docs from the source code
This means we don't have two (divergent) sets of option descriptions
anymore.
2020-08-19 18:28:04 +02:00
Eelco Dolstra 6f19c776db
Start generation of the nix.1 manpage 2020-08-17 19:33:18 +02:00
Eelco Dolstra a72a20d68f
Add 'nix dump-args' to dump all commands/flags for manpage generation 2020-08-17 17:44:52 +02:00
John Ericson 3c8b5b6219 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-08-14 17:00:13 +00:00
Eelco Dolstra d81f13f7cb
Merge pull request #3899 from obsidiansystems/make-narHash-not-optional
Make narHash in ValidPathInfo not optional
2020-08-14 17:00:18 +02:00
John Ericson 85aacbee64 Use TeeSink and TeeSouce in a few more places 2020-08-13 14:51:17 +00:00
John Ericson e913a2989f Squashed get CA derivations building 2020-08-07 19:51:55 +00:00
John Ericson 39ae9a3d4a Merge remote-tracking branch 'upstream/master' into fix-and-ci-static-builds 2020-08-07 14:44:47 +00:00
Carlo Nucera 8241e660ba Remove Hash::operator bool ()
Since the hash is not optional anymore
2020-08-05 15:30:38 -04:00
Carlo Nucera be6e1c6457 Merge branch 'master' of github.com:NixOS/nix into make-narHash-not-optional 2020-08-05 15:14:47 -04:00
Carlo Nucera 1ad6394b33 Add Hash::dummy to signal default value
We did this in the same spirit of the dummy value that's present in
libstore/path.hh
2020-08-05 15:11:49 -04:00
John Ericson 839f0fe095 Merge remote-tracking branch 'upstream/master' into misc-ca 2020-08-05 14:40:01 +00:00
John Ericson 03f4fafc27 Merge remote-tracking branch 'upstream/master' into misc-ca 2020-08-05 14:36:25 +00:00
Eelco Dolstra b91dc7ebad
Merge pull request #3730 from obsidiansystems/better-ca-parse-errors
Improve hash parsing and errors
2020-08-05 16:33:07 +02:00
Eelco Dolstra 75f220a595
Merge pull request #3864 from obsidiansystems/more-topo-sort
Abstract out topo sorting logic
2020-08-05 16:07:29 +02:00
John Ericson d3452a5ed6 Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors 2020-08-04 16:13:20 +00:00
Eelco Dolstra 24e07c428f Delete compressed NARs
Fixes #3891.
2020-08-03 18:34:52 +02:00
John Ericson bc165e28ae Embelish documentation of new Hash functions 2020-08-01 15:32:20 +00:00
John Ericson 3cbee1e840 Convert to C-style comments 2020-08-01 15:26:57 +00:00
Carlo Nucera eee6ef86cd Merge branch 'master' of github.com:NixOS/nix into better-ca-parse-errors 2020-07-30 18:27:25 -04:00
Matthew Bauer d7ffe327ae Merge remote-tracking branch 'origin/master' into fix-and-ci-static-builds 2020-07-30 14:59:57 -05:00
Eelco Dolstra a785b3eddf Fix build 2020-07-30 15:27:28 +02:00
Eelco Dolstra 3f6e88a552 unsigned long long -> uint64_t 2020-07-30 13:34:04 +02:00
Eelco Dolstra ebee2b7852 receiveContents(): unsigned int -> size_t 2020-07-30 13:00:30 +02:00
John Ericson 8065c6d160 Abstract out topo sorting logic 2020-07-27 20:45:34 +00:00
John Ericson e6115ca004 Merge remote-tracking branch 'upstream/master' into ca-derivation-data-types 2020-07-27 17:41:43 +00:00
John Ericson 1d7d94ceea Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors 2020-07-27 16:17:50 +00:00
John Ericson 78466bcb2f Merge branch 'optional-derivation-output-storepath' into ca-derivation-data-types 2020-07-27 14:40:08 +00:00
John Ericson 699fc89b39 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-07-27 14:22:45 +00:00
Maximilian Bosch 6ccfdb79c7
libutil/logging: extend internal-json logger to make it more machine-readable
The new error-format is pretty nice from a UX point-of-view, however
it's fairly hard to parse the output e.g. for editor plugins such as
vim-ale[1] that use `nix-instantiate --parse` to determine syntax errors in
Nix expression files.

This patch extends the `internal-json` logger by adding the fields
`line`, `column` and `file` to easily locate an error in a file and the
field `raw_msg` which contains the error-message itself without
code-lines and additional helpers.

An exemplary output may look like this:

```
[nix-shell]$ ./inst/bin/nix-instantiate ~/test.nix --log-format minimal
{"action":"msg","column":1,"file":"/home/ma27/test.nix","level":0,"line":4,"raw_msg":"syntax error, unexpected IF, expecting $end","msg":"<full error-msg with code-lines etc>"}
```

[1] https://github.com/dense-analysis/ale
2020-07-26 02:29:57 +02:00
Eelco Dolstra 2292814049
createUnixDomainSocket(): Fix off-by-one error in copying the socket path
Reported by Kane York.
2020-07-24 11:19:17 +02:00
John Ericson cdb3f39b64 Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors 2020-07-21 13:43:20 +00:00
John Ericson 5055c595bd Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into ca-derivation-data-types 2020-07-21 01:20:53 +00:00
Carlo Nucera 362ae93851 Add UnimplementedError to ease grepping for these 2020-07-20 14:13:37 -04:00
John Ericson c58c6165c5 Remove period at the end of the exception message 2020-07-20 17:43:19 +00:00
John Ericson bf61871271 parser.hh -> split.hh 2020-07-20 17:42:34 +00:00
John Ericson eab945f591 Merge remote-tracking branch 'upstream/master' into from-dump-stream 2020-07-20 17:29:23 +00:00
Eelco Dolstra 1c5f8bbfb5
Merge pull request #3822 from obsidiansystems/dump-thrice-fixme
Optimize `addToStoreSlow` and remove `TeeParseSink`
2020-07-20 18:55:05 +02:00
Carlo Nucera 745a03cef5 Merge branch 'optional-derivation-output-storepath' of github.com:obsidiansystems/nix into ca-derivation-data-types 2020-07-16 13:36:01 -04:00
John Ericson cc0d77f8c9 Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors 2020-07-16 17:28:52 +00:00
John Ericson 5ea817dace Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-07-16 14:58:53 +00:00
John Ericson 3dcca18c30 Fix bug in TeeSource
We use this to simplify `LocalStore::addToStoreFromDump`.

Also, hope I fixed build error with old clang (used in Darwin CI).
2020-07-16 13:39:27 +00:00
John Ericson 68dfb8c6ae Optimize addToStoreSlow and remove TeeParseSink 2020-07-16 05:09:41 +00:00
John Ericson 5602637d9e Revert "LocalStore::addToStoreFromDump copy in chunks"
This reverts commit 592851fb67. We don't
need this extra feature anymore
2020-07-15 23:37:49 +00:00
John Ericson bc109648c4 Get rid of LocalStore::addToStoreCommon
I got it to just become `LocalStore::addToStoreFromDump`, cleanly taking
a store and then doing nothing too fancy with it.

`LocalStore::addToStore(...Path...)` is now just a simple wrapper with a
bare-bones sinkToSource of the right dump command.
2020-07-15 23:14:30 +00:00
Carlo Nucera d090562348 Merge branch 'master' of github.com:NixOS/nix into hash-always-has-type 2020-07-15 17:21:01 -04:00
John Ericson 592851fb67 LocalStore::addToStoreFromDump copy in chunks
Rather than copying byte-by-byte, we let the coroutine know how much
data we would like it to send back to us.
2020-07-14 13:56:36 +00:00
Eelco Dolstra 832e111494 Merge remote-tracking branch 'origin/master' into flakes 2020-07-14 13:56:18 +02:00
Eelco Dolstra 7c2fef0a81 Make 'nix copy' to s3:// binary caches run in constant memory 2020-07-13 20:07:19 +02:00
Eelco Dolstra 493961b689 Remove istringstream_nocopy 2020-07-13 18:31:19 +02:00
Eelco Dolstra 0a9da00a10 NarAccessor: Run in constant memory 2020-07-13 17:30:42 +02:00
Eelco Dolstra fc84c358d9 Make 'nix copy' to file:// binary caches run in constant memory 2020-07-13 16:28:45 +02:00
John Ericson 4415765385 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-07-13 03:01:44 +00:00
John Ericson ffc18583b1 Move C++17 "pattern matching" boilerplat to utils.hh 2020-07-12 22:15:14 +00:00
Maximilian Bosch 64f03635d7
Fix ANSI color constants
The `m` acts as termination-symbol when declaring graphics. Because
of this, the `;1m` doesn't have any effect and is directly printed to
the console:

```
$ nix repl
> builtins.fetchGit { /* ... */ }
{ outPath = "/nix/store/s0f0iz4a41cxx2h055lmh6p2d5k5bc6r-source"; rev = "e73e45b723a9a6eecb98bd5f3df395d9ab3633b6"; revCount = ;1m428; shortRev = "e73e45b"; submodules = ;1mfalse; }
```

Introduced by 6403508f5a.
2020-07-12 16:52:20 +02:00
John Ericson 323d51a7a0 Merge branch 'master' of github.com:NixOS/nix into better-ca-parse-errors 2020-07-09 19:07:06 +00:00
Eelco Dolstra 9223603908 Merge remote-tracking branch 'origin/master' into flakes 2020-07-08 15:55:19 +02:00
Eelco Dolstra 16ec7785ca Fix 'got unknown message type 1 from Nix daemon'
Example:

  $ nix-build -E 'with import <nixpkgs> {}; runCommand "foo" { x = runCommand "bar" {} "exit 1"; } "echo foo; exit 1"'
  warning: unknown setting 'auto-allocate-uids'
  these 2 derivations will be built:
    /nix/store/v4fbdbhcdi949929a67g8farwf72zgam-bar.drv
    /nix/store/k4fsvrjl7cp2xpz7927iv7g0dqj1zyhs-foo.drv
  warning: unknown setting 'auto-allocate-uids'
  building '/nix/store/v4fbdbhcdi949929a67g8farwf72zgam-bar.drv'...
  error: --- Error ----------------------------------------------------------------------------------------------------------------------------------------------------------------- nix-daemon
  builder for '/nix/store/v4fbdbhcdi949929a67g8farwf72zgam-bar.drv' failed with exit code 1
  error: --- Error ------------------------------------------------------------------------------------------------------------------------------------------------------------------ nix-build
  got unknown message type 1 from Nix daemon
2020-07-08 15:53:14 +02:00
Eelco Dolstra 1ab9da9154 Merge remote-tracking branch 'origin/master' into flakes 2020-07-07 14:38:57 +02:00
Eelco Dolstra 4055cfee36 Fix coverage build 2020-07-07 14:37:47 +02:00
Carlo Nucera 0f05a36e20 Merge branch 'master' of github.com:NixOS/nix into better-ca-parse-errors 2020-07-06 16:57:09 -04:00
Ben Burdette efd6a8b230 bump 2020-07-06 11:54:53 -06:00
Ben Burdette 75bfcf8d15 revamp trace code and test 2020-07-06 10:51:48 -06:00
Eelco Dolstra 54712aaf8a Merge remote-tracking branch 'origin/master' into flakes 2020-07-06 16:40:10 +02:00
Ben Burdette a168224464 spacing 2020-07-04 18:30:49 -06:00
John Ericson 3134db1a83 Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors 2020-07-03 14:12:38 +00:00
John Ericson dbffd309fe Merge branch 'master' of github.com:NixOS/nix into hash-always-has-type 2020-07-03 14:11:38 +00:00
Ben Burdette b29a4ea1dc Merge branch 'master' into add-trace 2020-07-03 07:57:36 -06:00
John Ericson 13796be78d Have splitPrefix and splitPrefixTo parser helpers 2020-07-02 23:18:22 +00:00
John Ericson a7cd7425d9 Move getParsedTypeAndSRI to a more suitable location
Also mark it static
2020-07-02 23:10:11 +00:00
Eelco Dolstra b5e4253697 Fix abort in 'nix develop' 2020-07-02 18:24:11 +02:00
Ben Burdette 5818271c6e spacing 2020-07-02 09:41:54 -06:00
Carlo Nucera b6b10b1d4c Write the implementation for parseNonSRIUnprefixed 2020-07-02 11:34:40 -04:00
Carlo Nucera ea48e3a5b5 Abstract common parsing functionality 2020-07-02 11:29:33 -04:00
Carlo Nucera 36cbc74689 Inline and simplify in parseAnyPrefixed 2020-07-02 11:21:00 -04:00
Carlo Nucera 9462d8a50b Rename fromSRI to parseSRI for constistency 2020-07-02 11:11:18 -04:00
Carlo Nucera f61bc45d19 Get rid of the std::pair 2020-07-02 11:09:04 -04:00
Ben Burdette bf2788e4c1 move showTrace to new loggerSettings 2020-07-02 09:04:31 -06:00
Carlo Nucera 27c8029573 Inline newFunction 2020-07-02 11:01:10 -04:00
Carlo Nucera 343d1569b1 Fix test suite 2020-07-02 10:48:47 -04:00
Carlo Nucera 263ccdd489 Rename two hash constructors to proper functions 2020-07-01 18:34:18 -04:00
Carlo Nucera c8c4bcf90e Inline Hash::init() 2020-07-01 18:03:35 -04:00
Carlo Nucera d63a5ded76 Remove unused import 2020-07-01 17:53:24 -04:00
Carlo Nucera 6faeec3b2a Keep the previous name, for diffing 2020-07-01 17:50:34 -04:00
Carlo Nucera 274a8136fb Correct FIXMEs in libfetchers 2020-07-01 17:47:15 -04:00
Carlo Nucera c2e7f7a712 Fixed build, we still have test errors 2020-07-01 17:32:06 -04:00
Ben Burdette 8497891b99 spacing 2020-07-01 13:50:18 -06:00
Eelco Dolstra 6ff9aa8df7 Don't process an option if any of its arguments need completion 2020-07-01 20:31:39 +02:00
Ben Burdette a295b2ea96 if no errLoc, no Loc. 2020-07-01 12:02:02 -06:00
Ben Burdette 3629b0585a don't include errpos for addErrorContext 2020-07-01 11:49:01 -06:00
Ben Burdette 2a39c083dc non-pos trace test 2020-07-01 10:37:31 -06:00
Ben Burdette a7d5d26443 fix tests with the 'from string' change 2020-06-30 22:05:21 -06:00
Ben Burdette 9159dfe3d8 comments and cleanup 2020-06-30 16:31:55 -06:00
Ben Burdette 70bcb39d3f double addtrace for 'called from' 2020-06-30 15:44:19 -06:00
Ben Burdette ddb81ca126 Merge branch 'master' into add-trace 2020-06-30 12:21:45 -06:00
Carlo Nucera b798efb829 WIP initial design 2020-06-30 14:10:30 -04:00
Ben Burdette a0705e0dd1 invalid pos check 2020-06-30 11:01:46 -06:00
Ben Burdette e72a16a339 check for a null symbol 2020-06-30 11:00:51 -06:00
Carlo Nucera a1f66d1d9e Factor the prefix splitting in hash 2020-06-30 12:49:00 -04:00
Carlo Nucera 7ba0fae0dd Create the spitPrefix function in parser.hh 2020-06-30 11:57:09 -04:00
Ben Burdette c484a67914 trace formatting 2020-06-29 15:46:21 -06:00
John Ericson 24da034bc3 Add possibly missing <string> include 2020-06-29 21:22:00 +00:00
Eelco Dolstra 2b834d48aa NAR parser: Fix missing name field check
Discovered by @Kloenk.
2020-06-29 22:45:41 +02:00
Matthew Bauer da77331cb7 Remove lazy lookup in getHome
this seems to break in Musl/Static with:

terminate called after throwing an instance of 'std::bad_function_call'
  what():  bad_function_call
2020-06-29 20:34:26 +00:00
Matthew Bauer 88cf6ffce3 Rename logging->stdout to logging->stdout_
musl doesn't like this identifier
2020-06-29 20:34:26 +00:00
John Ericson 987a4a0be9 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-06-29 18:03:21 +00:00
Ben Burdette 8f81fae116 showTrace flag in loggers 2020-06-29 10:20:51 -06:00
Ben Burdette ef24a0835d showtrace as function arg 2020-06-27 12:19:31 -06:00
Eelco Dolstra adf2fbbdc2 Merge remote-tracking branch 'origin/master' into flakes 2020-06-26 08:46:46 +02:00
Ben Burdette bc9e87412c 'string' makes more sense in nix repl 2020-06-25 09:56:32 -06:00
Ben Burdette 9ab808c926 showTrace flag for ErrorInfo; showTrace test. 2020-06-25 09:23:12 -06:00
Eelco Dolstra de2641ae99 Fix empty std::optional dereference in writeDerivation()
https://hydra.nixos.org/build/123017579
2020-06-25 15:50:30 +02:00
Ben Burdette 9c0e1fd4f1 add trace test; error formatting refinements 2020-06-24 18:31:28 -06:00
Ben Burdette 023912def3 convenience form of addTrace 2020-06-24 13:46:25 -06:00
Ben Burdette b18ed02b76 repl indenting 2020-06-24 13:10:41 -06:00
Ben Burdette 00fe653ea5 nixCode -> LinesOfCode 2020-06-24 08:33:53 -06:00
Ben Burdette 1d43a6e123 use plain errPos instead of nixCode; fix tests 2020-06-23 15:30:13 -06:00
John Ericson ec3a857307 Fix and clean up hash parser 2020-06-23 18:19:40 +00:00
John Ericson 98e5d1af03 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-06-23 17:03:37 +00:00
Ben Burdette d0e78fbb03 re-add Pos origin in tests 2020-06-23 10:51:58 -06:00
Ben Burdette abe0552504 Merge remote-tracking branch 'upstream/master' into add-trace 2020-06-23 09:40:28 -06:00
Ben Burdette 13e87535ff traces to bottom 2020-06-23 09:36:58 -06:00
Eelco Dolstra 015e1c2131
Merge pull request #3724 from bburdette/hintfmt-percent
Hintfmt percent test, and fix
2020-06-23 12:26:00 +02:00
Ben Burdette 9d1cb0c5e6 with normaltxt, elide yellow color code instead of canceling it; use normaltxt on plain_string hintfmt 2020-06-22 11:32:20 -06:00
Ben Burdette 28b079067f
Update src/libutil/fmt.hh
Co-authored-by: John Ericson <git@JohnEricson.me>
2020-06-22 10:00:37 -06:00
Ben Burdette be4f444175 tidying up 2020-06-19 16:58:12 -06:00
Ben Burdette 0309488a66 fmt -> hintfmt test 2020-06-19 16:46:49 -06:00
Ben Burdette 397dbe114e remove formathelper 2020-06-19 15:57:19 -06:00
John Ericson 507aa48739 WIP: Make Hash always store a valid hash type 2020-06-19 21:48:57 +00:00
Ben Burdette b193aca4ae escape percents 2020-06-19 15:29:19 -06:00
Ben Burdette db475f9e7e too few, too many args 2020-06-19 15:28:13 -06:00
Ben Burdette cdddf24f25 add hintfmt test 2020-06-19 14:54:41 -06:00
Ben Burdette 54e8f550c9 addErrorTrace 2020-06-19 13:44:08 -06:00
John Ericson e288c0987a Merge remote-tracking branch 'upstream/master' into validPathInfo-ca-proper-datatype 2020-06-19 18:44:24 +00:00
John Ericson c98081d270 Merge remote-tracking branch 'upstream/master' into no-hash-type-unknown 2020-06-19 17:50:05 +00:00
John Ericson c1892a5316 tabs -> spaces 2020-06-19 17:49:57 +00:00
John Ericson 2f0e395c99 Merge remote-tracking branch 'me/no-stringly-typed-derivation-output' into validPathInfo-ca-proper-datatype 2020-06-19 15:26:59 +00:00
John Ericson b90cac3bad Remove uneeded = default for Hash 2020-06-19 15:00:38 +00:00
John Ericson 01dc8b0bab Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output 2020-06-19 14:59:05 +00:00
Eelco Dolstra 2886c92aef
Merge pull request #3669 from gilligan/add-compression-tests
Add compression unit tests
2020-06-19 13:59:04 +02:00
John Ericson 3fc58a9638 Remove some Base:: that crept in 2020-06-19 00:24:47 +00:00
John Ericson 669c3992e8 Merge branch 'no-hash-type-unknown' into validPathInfo-temp 2020-06-18 22:33:07 +00:00
John Ericson 15abb2aa2b Revert the enum struct change
Not a regular git revert as there have been many merges and things.
2020-06-18 22:11:26 +00:00
John Ericson bbbf3602a3 Merge branch 'enum-class' into no-hash-type-unknown 2020-06-18 22:11:19 +00:00
John Ericson 40526fbea5 Merge remote-tracking branch 'upstream/master' into enum-class 2020-06-18 21:38:15 +00:00
Ben Burdette 4d1a4f0217 addTrace 2020-06-18 15:25:26 -06:00
Ben Burdette e6f93b94fc Merge branch 'master' into caveman-LOCs 2020-06-18 13:07:53 -06:00
Eelco Dolstra 4d5169bdd5
Merge pull request #3707 from p01arst0rm/outdated-function-fix
replaced uncaught_exception with uncaught_exceptions
2020-06-17 18:26:01 +02:00
John Ericson 6403508f5a Use ansicolor.hh in nix repl rather than duplicates 2020-06-17 15:13:00 +00:00
Eelco Dolstra 1524752c17 Merge remote-tracking branch 'origin/master' into flakes 2020-06-17 10:26:52 +02:00
John Ericson 517f5980e2 Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output 2020-06-17 04:58:43 +00:00
p01arst0rm e9970a34e8 appended ' __attribute__((weak)); ' to 'extern char * * environ ' 2020-06-17 03:25:34 +01:00
p01arst0rm c9d06558b6 replaced uncaught_exception with uncaught_exceptions 2020-06-17 03:15:47 +01:00
Eelco Dolstra df4da4f5da
Merge pull request #3702 from NixOS/store-path-cxx
Rewrite StorePath class in C++
2020-06-16 21:37:26 +02:00
John Ericson fbf90bd693 Add another missing #include 2020-06-16 14:19:49 +00:00
Eelco Dolstra 759947bf72 StorePath: Rewrite in C++
On nix-env -qa -f '<nixpkgs>', this reduces maximum RSS by 20970 KiB
and runtime by 0.8%. This is mostly because we're not parsing the hash
part as a hash anymore (just validating that it consists of base-32
characters).

Also, replace storePathToHash() by StorePath::hashPart().
2020-06-16 14:28:41 +02:00
Tobias Pflug cd8214c398 Fix logging unit tests 2020-06-16 10:23:15 +02:00
John Ericson 7e7e3b71f3 Add mising #include for strerror 2020-06-15 23:35:07 +00:00
Eelco Dolstra 8b099812ea Respect terminal width printing error messages 2020-06-15 19:35:31 +02:00
Eelco Dolstra f20bb983ca Cleanup 2020-06-15 18:16:03 +02:00
Eelco Dolstra 5ed5d7acbd Improve "waiting for locks" messages
These are now shown in the progress bar.

Closes #3577.
2020-06-15 16:03:29 +02:00
Eelco Dolstra e14e62fddd Remove trailing whitespace 2020-06-15 14:12:39 +02:00
Eelco Dolstra 1fb762d11f Get rid of explicit ErrorInfo constructors 2020-06-15 14:06:58 +02:00
Eelco Dolstra fd64e4fb96 Disambiguate BaseError(Args) constructor
This means that 'throw Error({ ... ErrorInfo ... })' now works.
2020-06-15 13:50:33 +02:00
Eelco Dolstra 7a77762961 Merge branch 'errors-phase-2' of https://github.com/bburdette/nix 2020-06-15 11:46:31 +02:00
John Ericson f6f01416b7 Use std::string_view in a few more places 2020-06-12 21:32:30 +00:00
Eelco Dolstra 2853ba4ab2 Fix build 2020-06-12 19:00:48 +02:00
Eelco Dolstra 00fa7e2205
Merge pull request #3674 from matthewbauer/allow-empty-hash2
Allow empty hash in derivations
2020-06-12 18:18:12 +02:00
Matthew Bauer ea0d29d99a Provide base argument to to_string 2020-06-12 10:18:27 -05:00
Matthew Bauer b260c9ee03 Add newHashAllowEmpty helper function
This replaces the copy&paste with a helper function in hash.hh.
2020-06-12 10:11:16 -05:00
Eelco Dolstra 4a4c063222
Merge pull request #3670 from gilligan/add-pool-tests
Add tests for pool.hh
2020-06-12 11:19:05 +02:00
Ben Burdette ef1b3f21b6 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-06-11 14:06:35 -06:00
Eelco Dolstra ac4d43a31b
Merge pull request #3073 from tweag/machine-logs
Add an option to print the logs in a machine-readable format
2020-06-11 15:45:18 +02:00
Eelco Dolstra dd9bb11d0d Move names.{cc,hh} to libstore 2020-06-11 15:42:18 +02:00
Tobias Pflug 4750d98bbd Add tests for pool.hh 2020-06-10 22:29:50 +02:00
Eelco Dolstra b9ae1bdd7a
Merge pull request #3655 from zimbatm/hash-encoding-prepare
libutils/hash: remove default encoding
2020-06-10 11:48:38 +02:00
Ben Burdette 2f19650768 add file origin to Pos in stests 2020-06-08 11:21:17 -06:00
Ben Burdette b1c53b034c Merge branch 'errors-phase-2' into caveman-LOCs 2020-06-08 11:10:13 -06:00
regnat 801112de1a Move progress-bar.cc to libmain
Needed so that we can include it as a logger in loggers.cc without
adding a dependency on nix

This also requires moving names.hh to libutil to prevent a circular
dependency between libmain and libexpr
2020-06-08 17:16:52 +02:00
Eelco Dolstra 6470450ab4 Add completion for --update-input 2020-06-08 16:20:00 +02:00
Tobias Pflug cd6dbf951a Add compression unit tests 2020-06-08 11:34:37 +02:00
regnat 4983401440 Unify the printing of the logs between bar-with-logs and raw
Make the printing of the build logs systematically go through the
logger, and replicate the behavior of `no-build-output` by having two
different loggers (one that prints the build logs and one that doesn't)
2020-06-08 09:31:15 +02:00
regnat 170e86dff5 Make the logger customisable
Add a new `--log-format` cli argument to change the format of the logs.
The possible values are
- raw (the default one for old-style commands)
- bar (the default one for new-style commands)
- bar-with-logs (equivalent to `--print-build-logs`)
- internal-json (the internal machine-readable json format)
2020-06-08 09:31:15 +02:00