6251 commits

Author	SHA1	Message	Date
John Ericson	d3452a5ed6	Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors	2020-08-04 16:13:20 +00:00
Eelco Dolstra	5b22a2c0d4	Merge pull request #3894 from obsidiansystems/json-test-order-agnostic ... Make JSON equality tests agnostic to ordering	2020-08-04 17:24:46 +02:00
Eelco Dolstra	4e7f1c7f11	S3BinaryCacheStore: Fix size determination	2020-08-04 16:01:13 +02:00
Eelco Dolstra	dfe66420e7	Revert "Remove putBytes" ... This reverts commit b8eea7e81a.	2020-08-04 15:56:10 +02:00
Eelco Dolstra	327b1bf378	BinaryCacheStore: Explicitly flush file sink ... The file sink is also flushed in its destructor, but we ignore any exceptions in the destructor. Issue #3886.	2020-08-04 14:54:07 +02:00
Eelco Dolstra	ed52cf632b	Merge pull request #3892 from obsidiansystems/path-info-header ... Move ValidPathInfo to its own header	2020-08-04 10:44:47 +02:00
John Ericson	fe7e57a80d	tab -> space	2020-08-04 03:47:09 +00:00
John Ericson	062533f7cd	Merge remote-tracking branch 'upstream/master' into path-info-header	2020-08-03 21:02:28 +00:00
Eelco Dolstra	24e07c428f	Delete compressed NARs ... Fixes #3891.	2020-08-03 18:34:52 +02:00
John Ericson	54281f3ac1	addToStore in terms of addToStoreFromDump is not local-store-specific	2020-08-03 04:13:45 +00:00
John Ericson	9357512d73	Merge remote-tracking branch 'upstream/master' into derivation-header-include-order	2020-08-01 19:38:35 +00:00
John Ericson	c4ada76e86	Fix error message and avoid recalculation	2020-08-01 16:22:50 +00:00
John Ericson	bc165e28ae	Embelish documentation of new Hash functions	2020-08-01 15:32:20 +00:00
John Ericson	3cbee1e840	Convert to C-style comments	2020-08-01 15:26:57 +00:00
Carlo Nucera	b6d97fdbf4	Merge branch 'master' of github.com:NixOS/nix into drv-outputs-map-allow-missing	2020-07-31 13:12:51 -04:00
Eelco Dolstra	a3f9625818	Tweak description	2020-07-31 17:32:40 +02:00
Eelco Dolstra	935723eb3b	Merge branch 'nix-command-build-check' of https://github.com/Ma27/nix	2020-07-31 17:30:55 +02:00
Eelco Dolstra	f56dfce734	nix bundle: Set category	2020-07-31 17:30:12 +02:00
Maximilian Bosch	5a09eb86f1	nix/build: add --rebuild option ... Occasionally, `nix-build --check` is fairly helpful and I'd like to be able to use this feature for flakes that need to be built with `nix build` as well.	2020-07-31 17:17:03 +02:00
Eelco Dolstra	bf290c2306	Merge remote-tracking branch 'origin/master' into markdown	2020-07-31 16:07:04 +02:00
Eelco Dolstra	1d0a7b54fa	Enable syntax highlighting	2020-07-31 15:43:25 +02:00
Eelco Dolstra	0604cfd0eb	Merge pull request #3880 from matthewbauer/nix-bundle ... Add "bundle" command to Nix	2020-07-31 11:45:53 +02:00
John Ericson	e3a2154f5a	Fix indentation	2020-07-31 01:07:59 +00:00
Carlo Nucera	eee6ef86cd	Merge branch 'master' of github.com:NixOS/nix into better-ca-parse-errors	2020-07-30 18:27:25 -04:00
Matthew Bauer	cdc2386644	Make expectedHash optional in prim_path ... This fixes an error found in builtins.path that looks like: store path mismatch in (possibly filtered) path added from '/private/tmp/nix-shell.CyXViH/nix-test/filter-source/filterin' when no hash is specified	2020-07-30 16:40:40 -05:00
Matthew Bauer	fa2d1fb36e	Pass system to bundler	2020-07-30 15:37:05 -05:00
Matthew Bauer	22fcfdf18a	Address misc review	2020-07-30 15:18:57 -05:00
Matthew Bauer	1a705637ce	Remove single file restriction for bundler	2020-07-30 15:16:29 -05:00
Matthew Bauer	52407f83a1	exporter -> bundler	2020-07-30 15:16:23 -05:00
Matthew Bauer	d7ffe327ae	Merge remote-tracking branch 'origin/master' into fix-and-ci-static-builds	2020-07-30 14:59:57 -05:00
Matthew Bauer	05ac4db39a	Merge remote-tracking branch 'origin/master' into substitute-other-storedir	2020-07-30 12:38:24 -05:00
Matthew Bauer	5d04a4db9b	Handle exporters checking correctly	2020-07-30 11:45:47 -05:00
Matthew Bauer	2f4250a416	Add "export" to Nix ... This adds a ‘nix export’ command which hooks into nix-bundle. It can be used in a similar way as nix-bundle, with the benefit of hooking into the new “app” functionality. For instance, $ nix export nixpkgs#jq $ ./jq --help jq - commandline JSON processor [version 1.6] ... $ scp jq machine-without-nix: $ ssh machine-without-nix ./jq jq - commandline JSON processor [version 1.6] ... Note that nix-bundle currently requires Linux to run. Other exporters might not have that requirement. “exporters” are meant to be reusable, so that, other repos can implement their own bundling. Fixes #3705	2020-07-30 11:33:22 -05:00
Eelco Dolstra	a785b3eddf	Fix build	2020-07-30 15:27:28 +02:00
Eelco Dolstra	3f6e88a552	unsigned long long -> uint64_t	2020-07-30 13:34:04 +02:00
Eelco Dolstra	ebee2b7852	receiveContents(): unsigned int -> size_t	2020-07-30 13:00:30 +02:00
Mat Marini	d9e23bfee2	Prevent nix-build from accepting --run/--command	2020-07-29 20:53:02 -04:00
Mat Marini	e4f6f8da77	Prevent nix-build from accepting --packages	2020-07-29 20:50:06 -04:00
Eelco Dolstra	0c94c17644	Merge remote-tracking branch 'origin/master' into markdown	2020-07-29 18:08:57 +02:00
Eelco Dolstra	f63839bfa4	Cleanup	2020-07-29 18:04:51 +02:00
Eelco Dolstra	39311e7009	Merge pull request #3549 from Ma27/fetchgit-hash ... Merge legacy `fetchGit`-builtin with the generic `fetchTree`-function	2020-07-29 11:33:27 +02:00
Eelco Dolstra	7c097275c4	Merge pull request #3877 from matthewbauer/develop-continuous-regex ... Set regex_constants::match_continuous for quicker search in nix develop	2020-07-29 10:07:52 +02:00
Matthew Bauer	fa8515d7ec	Set regex_constants::match_continuous for quicker search ... match_continuous limits the search to the current start position, instead of searching the entire file. On libc++, this improves performance dramatically: $ time /nix/store/70ai68dfm6xbzwn26j5n4li9di52ylia-nix-3.0pre20200728_c159f48/bin/nix print-dev-env >/dev/null /nix/store/70ai68dfm6xbzwn26j5n4li9di52ylia-nix-3.0pre20200728_c159f48/bin/ni 2.39s user 0.19s system 64% cpu 4.032 total $ time /nix/store/cwjfxxlp83zln4mfyy1d2dbsx7f6s962-nix-3.0pre20200728_dirty/bin/nix print-dev-env >/dev/null /nix/store/cwjfxxlp83zln4mfyy1d2dbsx7f6s962-nix-3.0pre20200728_dirty/bin/nix 0.09s user 0.05s system 65% cpu 0.204 total Fixes #3874	2020-07-29 00:57:20 -05:00
Eelco Dolstra	4c0077a07d	Fix RemoteStore::addToStore() latency ... Since 6185d25e52, this was very latency-bound since it required a round-trip for every 32 KiB. So for example copying a 514 MiB closure over a virtual ethernet device with a articial delay of just 1 ms took 343s. Now it takes 2.7s. Fixes #3372.	2020-07-29 00:48:39 +02:00
Eelco Dolstra	c159f48a39	Cleanup	2020-07-29 00:24:55 +02:00
John Ericson	df707d05d1	Merge branch 'ca-drv' of github.com:Ericson2314/nix into misc-ca	2020-07-28 21:12:36 +00:00
Maximilian Bosch	e4940e90f3	Restore backwards-compat for current builtins.fetchGit ... If a repo is dirty, it used to return a `rev` object with an "empty" sha1 (0000000000000000000000000000000000000000). Please note that this only applies for `builtins.fetchGit` and *not* for `builtins.fetchTree{ type = "git"; }`.	2020-07-28 22:46:39 +02:00
Carlo Nucera	2980b244b7	Use assert(false) instead of abort()	2020-07-28 15:39:45 -04:00
Carlo Nucera	cfe96bd7ce	Merge branch 'master' of github.com:NixOS/nix into ca-drv	2020-07-28 15:36:45 -04:00
Carlo Nucera	c318d398f3	Merge branch 'misc-ca' of github.com:obsidiansystems/nix into new-interface-for-path-pathOpt	2020-07-28 14:22:24 -04:00
Carlo Nucera	7ef1e3cd14	Use the new interface	2020-07-28 13:59:24 -04:00
Carlo Nucera	d564ac1c50	Offer a safer interface for pathOpt ... The new interface we offer provides a way of getting all the DerivationOutputs with the storePaths directly, based on the observation that it's the most common usecase.	2020-07-28 13:55:15 -04:00
Eelco Dolstra	b8eea7e81a	Remove putBytes ... istream->tellg() returns -1 so we can't get the number of bytes written. Fixes 'uploaded 's3://nix-cache/nar/00819r9lp5kajr6baxfw5dhhc0cx8ndxaz43qmd2f0gn1hk1ynlp.nar.xz' (-1 bytes) in 11620 ms' messages.	2020-07-28 13:27:56 +02:00
Maximilian Bosch	f742438465	Merge legacy fetchGit-builtin with the generic fetchTree-function ... The original idea was to implement a git-fetcher in Nix's core that supports content hashes[1]. In #3549[2] it has been suggested to actually use `fetchTree` for this since it's a fairly generic wrapper over the new fetcher-API[3] and already supports content-hashes. This patch implements a new git-fetcher based on `fetchTree` by incorporating the following changes: * Removed the original `fetchGit`-implementation and replaced it with an alias on the `fetchTree` implementation. * Ensured that the `git`-fetcher from `libfetchers` always computes a content-hash and returns an "empty" revision on dirty trees (the latter one is needed to retain backwards-compatibility). * The hash-mismatch error in the fetcher-API exits with code 102 as it usually happens whenever a hash-mismatch is detected by Nix. * Removed the `flakes`-feature-flag: I didn't see a reason why this API is so tightly coupled to the flakes-API and at least `fetchGit` should remain usable without any feature-flags. * It's only possible to specify a `narHash` for a `git`-tree if either a `ref` or a `rev` is given[4]. * It's now possible to specify an URL without a protocol. If it's missing, `file://` is automatically added as it was the case in the original `fetchGit`-implementation. [1] https://github.com/NixOS/nix/pull/3216 [2] https://github.com/NixOS/nix/pull/3549#issuecomment-625194383 [3] https://github.com/NixOS/nix/pull/3459 [4] https://github.com/NixOS/nix/pull/3216#issuecomment-553956703	2020-07-28 00:44:39 +02:00
John Ericson	8065c6d160	Abstract out topo sorting logic	2020-07-27 20:45:34 +00:00
John Ericson	951415b568	Require ca-derivations everywhere we create a CA derivation ... "create" as in read one in from a serialized form, or build one from scratch in memory.	2020-07-27 17:56:36 +00:00
John Ericson	e32a9e124b	Merge branch 'misc-ca' of github.com:obsidiansystems/nix into derivation-primop-floating-output	2020-07-27 17:50:06 +00:00
John Ericson	7cf978440c	Merge branch 'ca-derivation-data-types' of github.com:obsidiansystems/nix into misc-ca	2020-07-27 17:47:40 +00:00
John Ericson	e6115ca004	Merge remote-tracking branch 'upstream/master' into ca-derivation-data-types	2020-07-27 17:41:43 +00:00
John Ericson	1d7d94ceea	Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors	2020-07-27 16:17:50 +00:00
John Ericson	43f2bd8dc5	Merge remote-tracking branch 'upstream/master' into hash-always-has-type	2020-07-27 16:13:57 +00:00
John Ericson	387699fe7f	Merge branch 'ca-derivation-data-types' of github.com:obsidiansystems/nix into misc-ca	2020-07-27 15:00:28 +00:00
John Ericson	78466bcb2f	Merge branch 'optional-derivation-output-storepath' into ca-derivation-data-types	2020-07-27 14:40:08 +00:00
John Ericson	d5bb67cfa4	Merge remote-tracking branch 'upstream/master' into optional-derivation-output-storepath	2020-07-27 14:29:32 +00:00
John Ericson	699fc89b39	Merge remote-tracking branch 'upstream/master' into hash-always-has-type	2020-07-27 14:22:45 +00:00
Maximilian Bosch	6ccfdb79c7	libutil/logging: extend internal-json logger to make it more machine-readable ... The new error-format is pretty nice from a UX point-of-view, however it's fairly hard to parse the output e.g. for editor plugins such as vim-ale[1] that use `nix-instantiate --parse` to determine syntax errors in Nix expression files. This patch extends the `internal-json` logger by adding the fields `line`, `column` and `file` to easily locate an error in a file and the field `raw_msg` which contains the error-message itself without code-lines and additional helpers. An exemplary output may look like this: ``` [nix-shell]$ ./inst/bin/nix-instantiate ~/test.nix --log-format minimal {"action":"msg","column":1,"file":"/home/ma27/test.nix","level":0,"line":4,"raw_msg":"syntax error, unexpected IF, expecting $end","msg":"<full error-msg with code-lines etc>"} ``` [1] https://github.com/dense-analysis/ale	2020-07-26 02:29:57 +02:00
John Ericson	2c7557481b	queryDerivationOutputMap no longer assumes all outputs have a mapping ... This assumption is broken by CA derivations. Making a PR now to do the breaking daemon change as soon as possible (if it is already too late, we can bump protocol intead).	2020-07-24 21:14:06 +00:00
Eelco Dolstra	a71d1cedff	printVersion(): Show system types	2020-07-24 11:34:01 +02:00
Eelco Dolstra	2292814049	createUnixDomainSocket(): Fix off-by-one error in copying the socket path ... Reported by Kane York.	2020-07-24 11:19:17 +02:00
John Ericson	58e55c0923	Merge branch 'misc-ca' of github.com:obsidiansystems/nix into derivation-primop-floating-output	2020-07-23 19:17:52 +00:00
John Ericson	a2af25c55a	Merge remote-tracking branch 'upstream/master' into misc-ca	2020-07-23 19:14:17 +00:00
John Ericson	e1de1fe0d8	Make Buildable a std::variant ... I think this better captures the intent of what's going on: we either have an opaque store path, or a drv path with some outputs. Having this structure will also help us support CA derivations: we'll have to allow the outpath paths to be optional, so the structure we gain now makes up for the structure we loose then.	2020-07-23 19:02:57 +00:00
John Ericson	c36916eca0	Merge branch 'misc-ca' of github.com:obsidiansystems/nix into derivation-primop-floating-output	2020-07-23 15:06:56 +00:00
John Ericson	6fcd9966bf	Merge remote-tracking branch 'upstream/master' into misc-ca	2020-07-23 15:02:29 +00:00
John Ericson	66a2067288	Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors	2020-07-23 14:56:35 +00:00
John Ericson	2274f63453	Get rid of basicDerivation::findOutput ... It's a tiny function which is: - hardly worth abstrating over, and also only used once. - doesn't work once we get CA drvs I rewrote the one callsite to be forwards compatable with CA derivations, and also potentially more performant: instead of reading in the derivation it can ust consult the SQLite DB in the common case.	2020-07-23 14:39:02 +00:00
Fabian Möller	b9ead08ca8	Save changes made by "nix registry pin" to user registry	2020-07-23 14:21:27 +02:00
John Ericson	9423f64ee2	Parse CA derivations using new output variants ... We no longer need `ParsedDerivation` because everything libstore needs to know about is in the `BasicDerivation` proper.	2020-07-22 23:59:25 +00:00
John Ericson	c56356bacc	Separate concerns in scanForReferences with TeeSink ... This also will make it easier to use a `HashModuloSink` instead for CA derivations.	2020-07-22 22:13:44 +00:00
Eelco Dolstra	ff314f186e	Merge pull request #3736 from obsidiansystems/allow-relative-paths-in-store-option ... Allow relative paths in --store option	2020-07-21 19:43:07 +02:00
Eelco Dolstra	0951330680	Merge pull request #3714 from obsidiansystems/add-body-to-network-errors ... Add response body to network errors	2020-07-21 18:09:42 +02:00
John Ericson	02639716ea	Merge branch 'allow-relative-paths-in-store-option' into remove-storetype-delegate-regStore	2020-07-21 15:55:27 +00:00
John Ericson	6cce32c8e8	Change logic for deciding what is a relative path for the local store ... The was Eelco's prefered logic, and it looks good to me!	2020-07-21 15:39:47 +00:00
Carlo Nucera	922a845ffc	Update chunkSize to the suggested value ... This was a suggested course of action in a review in one of our earlier commits, https://github.com/NixOS/nix/pull/3801#discussion_r457557079	2020-07-21 10:24:19 -04:00
John Ericson	54e507a7aa	Merge remote-tracking branch 'upstream/master' into add-body-to-network-errors	2020-07-21 14:17:59 +00:00
John Ericson	5ce95b9529	Update src/libstore/build.cc	2020-07-21 09:47:40 -04:00
John Ericson	cdb3f39b64	Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors	2020-07-21 13:43:20 +00:00
John Ericson	5055c595bd	Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into ca-derivation-data-types	2020-07-21 01:20:53 +00:00
John Ericson	5a37413796	Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into from-dump-stream	2020-07-21 01:20:14 +00:00
John Ericson	bf0dca7790	Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into misc-ca	2020-07-21 01:18:33 +00:00
Carlo Nucera	9aae179f34	Correct bug, thoroughly document addToStoreSlow	2020-07-20 20:18:12 -04:00
Carlo Nucera	0ca9744694	Use heuristics to decide when to show the response ... Due to https://github.com/NixOS/nix/issues/3841 we don't know how print different messages for different verbosity levels.	2020-07-20 15:57:58 -04:00
Carlo Nucera	6357b1b0fb	Add another Unimplemented case	2020-07-20 14:17:25 -04:00
Carlo Nucera	007bf6d17a	Merge branch 'ca-derivation-data-types' of github.com:obsidiansystems/nix into misc-ca	2020-07-20 14:14:39 -04:00
Carlo Nucera	362ae93851	Add UnimplementedError to ease grepping for these	2020-07-20 14:13:37 -04:00
John Ericson	c58c6165c5	Remove period at the end of the exception message	2020-07-20 17:43:19 +00:00
John Ericson	bf61871271	parser.hh -> split.hh	2020-07-20 17:42:34 +00:00
John Ericson	eab945f591	Merge remote-tracking branch 'upstream/master' into from-dump-stream	2020-07-20 17:29:23 +00:00
Eelco Dolstra	1c5f8bbfb5	Merge pull request #3822 from obsidiansystems/dump-thrice-fixme ... Optimize `addToStoreSlow` and remove `TeeParseSink`	2020-07-20 18:55:05 +02:00
John Ericson	ac2fc7ba1f	Apply suggestions from code review ... Co-authored-by: Eelco Dolstra <edolstra@gmail.com>	2020-07-20 11:29:46 -04:00
Carlo Nucera	0aa79dcc6f	Remove StoreType abstraction and delegate regStore ... to each Store implementation. The generic regStore implementation will only be for the ambiguous shorthands, like "" and "auto". This also could get us close to simplifying the daemon command.	2020-07-17 17:24:02 -04:00
John Ericson	6756cecfcf	Add DerivationType::CAFloating	2020-07-17 19:55:41 +00:00
Carlo Nucera	fefd6c9e5f	Merge branch 'master' of github.com:NixOS/nix into allow-relative-paths-in-store-option	2020-07-17 14:34:20 -04:00
Carlo Nucera	bbc633c98c	Revert "Don't anticipate CA but not fixed outputs for now" ... This reverts commit 3a9e4c3262.	2020-07-17 13:10:32 -04:00
Carlo Nucera	205dcd140d	Revert "Don't anticipate multiple CA outputs for now" ... This reverts commit 74b251b2f3.	2020-07-17 12:43:46 -04:00
Carlo Nucera	1feb8981df	Revert "Don't anticipate hash algo without hash in derivation for now" ... This reverts commit 3804e3df9b.	2020-07-17 12:05:54 -04:00
Lars Mühmel	bc73590151	nix edit: call restoreSignals() before execvp-ing the $EDITOR ... Currently resizing of the terminal doesn't play nicely with nix edit when using kakoune as the editor, as it relies on the SIGWINCH signal which is trapped by nix. How this is not a problem with e.g. vim is beyond me. Virtually all other exec* calls are following a call to restoreSignals(). This commit adds this behavior to nix edit as well.	2020-07-17 17:35:59 +02:00
Carlo Nucera	487c5751c6	Merge branch 'ca-drv' of github.com:Ericson2314/nix into misc-ca	2020-07-17 11:32:59 -04:00
Carlo Nucera	a395c12830	Merge branch 'master' of github.com:NixOS/nix into misc-ca	2020-07-17 11:06:54 -04:00
Carlo Nucera	5cb840541b	Merge branch 'multi-output-hashDerivationModulo' of github.com:Ericson2314/nix into misc-ca	2020-07-17 10:28:33 -04:00
Eelco Dolstra	17f75f9cc4	parseFlakeRef(): Only search for the top-level directory for CLI flakerefs	2020-07-17 14:54:21 +02:00
Eelco Dolstra	52c8be38e0	nix profile diff-closures: Don't inherit EvalCommand	2020-07-17 12:36:12 +02:00
Carlo Nucera	745a03cef5	Merge branch 'optional-derivation-output-storepath' of github.com:obsidiansystems/nix into ca-derivation-data-types	2020-07-16 13:36:01 -04:00
Carlo Nucera	048e916f64	Merge branch 'master' of github.com:NixOS/nix into optional-derivation-output-storepath	2020-07-16 13:32:28 -04:00
John Ericson	cc0d77f8c9	Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors	2020-07-16 17:28:52 +00:00
Carlo Nucera	e3a3406db8	Merge branch 'master' of github.com:NixOS/nix into add-body-to-network-errors	2020-07-16 12:04:10 -04:00
Eelco Dolstra	16c9f6762d	Add command 'nix profile diff-closure' ... This shows all changes between generations of a profile. E.g. $ nix profile diff-closures --profile /nix/var/nix/profiles/system Generation 654 -> 655: nix: 2.4pre20200617_5d69bbf → 2.4pre20200701_6ff9aa8, +42.2 KiB Generation 655 -> 656: blender-bin: 2.83.0 → 2.83.1, -294.2 KiB Generation 656 -> 657: curl: 7.68.0 → 7.70.0, +19.1 KiB firmware-linux-nonfree: 2020-01-22 → 2020-05-19, +30827.7 KiB ibus: -21.8 KiB initrd-linux: 5.4.46 → 5.4.49 ...	2020-07-16 17:00:42 +02:00
John Ericson	5ea817dace	Merge remote-tracking branch 'upstream/master' into hash-always-has-type	2020-07-16 14:58:53 +00:00
John Ericson	3dcca18c30	Fix bug in TeeSource ... We use this to simplify `LocalStore::addToStoreFromDump`. Also, hope I fixed build error with old clang (used in Darwin CI).	2020-07-16 13:39:27 +00:00
Carlo Nucera	e3b394b6e8	Small namespace fix	2020-07-16 09:36:02 -04:00
Eelco Dolstra	5517eee17e	Generations API cleanup	2020-07-16 15:18:21 +02:00
Eelco Dolstra	8807ff902e	nix diff-closures: Fix build	2020-07-16 14:25:51 +02:00
Eelco Dolstra	716eafaca4	Merge branch 'diff-closures'	2020-07-16 14:19:35 +02:00
Eelco Dolstra	3f264916db	Merge remote-tracking branch 'origin/flakes'	2020-07-16 14:07:32 +02:00
John Ericson	68dfb8c6ae	Optimize addToStoreSlow and remove TeeParseSink	2020-07-16 05:09:41 +00:00
John Ericson	5602637d9e	Revert "LocalStore::addToStoreFromDump copy in chunks" ... This reverts commit 592851fb67. We don't need this extra feature anymore	2020-07-15 23:37:49 +00:00
John Ericson	bc109648c4	Get rid of LocalStore::addToStoreCommon ... I got it to just become `LocalStore::addToStoreFromDump`, cleanly taking a store and then doing nothing too fancy with it. `LocalStore::addToStore(...Path...)` is now just a simple wrapper with a bare-bones sinkToSource of the right dump command.	2020-07-15 23:14:30 +00:00
Carlo Nucera	455bdee205	Merge branch 'master' of github.com:NixOS/nix into derivation-header-include-order	2020-07-15 17:58:30 -04:00
John Ericson	64b7421741	Merge branch 'handle-flat-case' into from-dump-stream	2020-07-15 21:22:55 +00:00
Carlo Nucera	d090562348	Merge branch 'master' of github.com:NixOS/nix into hash-always-has-type	2020-07-15 17:21:01 -04:00
John Ericson	d087cf4855	Revert "Revert "LocalStore::addToStore(srcPath): Handle the flat case"" ... This reverts commit cff2157185.	2020-07-15 21:10:33 +00:00
John Ericson	aff69e229c	Merge remote-tracking branch 'obsidian/fix-add-to-star-narhash' into from-dump-stream	2020-07-15 21:05:29 +00:00
Eelco Dolstra	36a1242603	nix why-depends: Fix shortest path calculation ... This was completely broken since d8972317fc.	2020-07-15 21:11:07 +02:00
Eelco Dolstra	2d6d53bc87	nix: Fix examples	2020-07-15 20:28:16 +02:00
Eelco Dolstra	3624c042ac	nix: Add --derivation flag to operate on .drv paths ... For instance, 'nix why-depends --use-derivation nixpkgs#hello nixpkgs#glibc' shows why hello's .drv depends on glibc's .drv.	2020-07-15 20:25:10 +02:00
Eelco Dolstra	dfe8f3ebc6	nix why-depends: Fix misleading message	2020-07-15 20:09:50 +02:00
Eelco Dolstra	94eb5fad76	Clean up RealiseMode	2020-07-15 20:05:42 +02:00
Eelco Dolstra	e3c2b00237	Make InstallableStorePath behave consistently with InstallableValue ... That is, the commands 'nix path-info nixpkgs#hello' and 'nix path-info /nix/store/00ls0qi49qkqpqblmvz5s1ajl3gc63lr-hello-2.10.drv' now do the same thing (i.e. build the derivation and operate on the output store path, rather than the .drv path).	2020-07-15 19:50:32 +02:00
Carlo Nucera	2d2a10e79a	Merge branch 'master' of github.com:NixOS/nix into add-body-to-network-errors	2020-07-15 13:06:08 -04:00
Eelco Dolstra	0356f14459	Add 'nix diff-closures' command ... This command makes it easier to see what changed between two closures, i.e. what packages/versions got added or removed, and whether there were any notable changes in path size. For example: $ nix diff-closures /nix/var/nix/profiles/system-655-link /nix/var/nix/profiles/system-658-link blender-bin: 2.83.0 → 2.83.2, -294.2 KiB curl: 7.68.0 → 7.70.0, +19.1 KiB firmware-linux-nonfree: 2020-01-22 → 2020-05-19, +30827.7 KiB ibus: -21.8 KiB initrd-linux: 5.4.46 → 5.4.51, +16.9 KiB libexif: 0.6.21 → 0.6.22, +497.6 KiB linux: 5.4.46 → 5.4.51, +13.2 KiB mesa: 19.3.3 → 19.3.5, -183.9 KiB nix: 2.4pre20200701_6ff9aa8 → 2.4pre20200708_9223603, +9.7 KiB nix-bash-completions: 0.6.8 → ∅, -57.6 KiB nixos-system-hagbard: 20.03.20200615.a84b797 → 20.03.20200713.add5529 nvidia-persistenced: 440.82 → 440.100 nvidia-settings: 440.82 → 440.100 nvidia-x11: 440.82-5.4.46 → 440.100-5.4.51, +664.7 KiB pcre: 8.43 → 8.44 php: 7.3.16 → 7.3.20, -26.2 KiB python3.7-youtube-dl: 2020.06.06 → 2020.06.16.1, +8.4 KiB samba: 4.11.5 → 4.11.9, +30.1 KiB sane-backends: 1.0.28 → 1.0.30, +680.5 KiB source: -182.0 KiB zfs-kernel: 0.8.3-5.4.46 → 0.8.4-5.4.51, +9.9 KiB zfs-user: 0.8.3 → 0.8.4, +20.1 KiB	2020-07-15 16:24:25 +02:00
Eelco Dolstra	cff2157185	Revert "LocalStore::addToStore(srcPath): Handle the flat case" ... This reverts commit a2c27022e9. See addToStoreSlow(), we don't need to handle this case efficiently anymore. In fact, we can almost remove the method/hashAlgo arguments since the non-recursive and/or non-SHA256 are almost not used anymore.	2020-07-15 12:49:03 +02:00
John Ericson	650c2c6558	Rename variable nar -> dump according to TODO	2020-07-14 21:28:50 +00:00
John Ericson	8173e7bfef	Fix localhost::addToStore(...Path...) ... We were calculating the nar hash wrong when the file ingestion method was flat. I don't think there's anything we can do in that case but dump the file again, so that's what I do. As an optomization, we again could reuse the original dump for just the recursive and non-sha256 case, but I rather do that after this fix, and after my other PRs which deduplicate this code.	2020-07-14 21:12:11 +00:00
John Ericson	592851fb67	LocalStore::addToStoreFromDump copy in chunks ... Rather than copying byte-by-byte, we let the coroutine know how much data we would like it to send back to us.	2020-07-14 13:56:36 +00:00
John Ericson	9de96ef7d4	Dedup LocalStore::addToStore* ... The downsides is that the coroutine has byte-by-byte loop transfer. Will fix that next.	2020-07-14 13:56:36 +00:00
John Ericson	c86fc3a965	Crudely make addToStoreFromDump take Source not string ... I just as little beyond the type as possible, so the implementation changes this enables can be reviewed separately.	2020-07-14 13:56:36 +00:00
John Ericson	9ec10046e0	Narrow scope of temporary value	2020-07-14 13:54:29 +00:00
Eelco Dolstra	298ff6af8f	Merge pull request #3809 from Ma27/gitlab-refs ... Fix gitlab-fetcher to obtain tags and branches	2020-07-14 15:22:52 +02:00
Eelco Dolstra	da3aea291d	EvalCache: Ignore SQLite errors ... Fixes #3794.	2020-07-14 15:17:38 +02:00
Eelco Dolstra	832e111494	Merge remote-tracking branch 'origin/master' into flakes	2020-07-14 13:56:18 +02:00
Eelco Dolstra	926c3a6664	Doh	2020-07-14 11:55:54 +02:00
Eelco Dolstra	43b8e96d30	Fix 'nix verify --all' on a binary cache (cached case)	2020-07-13 20:17:00 +02:00
Eelco Dolstra	9502c0e2eb	nix verify: Show correct path when using --all on a binary cache	2020-07-13 20:12:44 +02:00
Eelco Dolstra	7c2fef0a81	Make 'nix copy' to s3:// binary caches run in constant memory	2020-07-13 20:07:19 +02:00
Maximilian Bosch	cf9f33995b	Fix gitlab-fetcher to obtain tags and branches ... Until now, the `gitlab`-fetcher determined the source's rev by checking the latest commit of the given `ref` using the `/repository/branches`-API. This breaks however when trying to fetch a gitlab-repo by its tag: ``` $ nix repl nix-repl> builtins.fetchTree gitlab:Ma27/nvim.nix/0.2.0 error: --- Error ------------------------------------------------------------------------------------- nix unable to download 'https://gitlab.com/api/v4/projects/Ma27%2Fnvim.nix/repository/branches/0.2.0': HTTP error 404 ('') ``` When using the `/commits?ref_name`-endpoint[1] you can pass any kind of valid ref to the `gitlab`-fetcher. Please note that this fetches the only first 20 commits on a ref, unfortunately there's currently no endpoint which only retrieves the latest commit of any kind of `ref`. [1] https://docs.gitlab.com/ee/api/commits.html#list-repository-commits	2020-07-13 19:22:59 +02:00
Eelco Dolstra	493961b689	Remove istringstream_nocopy	2020-07-13 18:31:19 +02:00
Eelco Dolstra	545bb2ed03	Remove 'accessor' from addToStore() ... This is only used by hydra-queue-runner and it's better to implement it there.	2020-07-13 18:31:19 +02:00
Eelco Dolstra	0a9da00a10	NarAccessor: Run in constant memory	2020-07-13 17:30:42 +02:00
Eelco Dolstra	fc84c358d9	Make 'nix copy' to file:// binary caches run in constant memory	2020-07-13 16:28:45 +02:00
Eelco Dolstra	400f1a9b59	Store::pathInfoToJSON(): Use consistent format for downloadHash	2020-07-13 16:26:09 +02:00
Eelco Dolstra	c0dd05131e	toStorePath(): Return a StorePath and the suffix	2020-07-13 16:25:48 +02:00
Eelco Dolstra	1d01ae816b	Fix 'nix verify --all' on a binary cache and add a test	2020-07-13 14:35:01 +02:00
John Ericson	c466cb2091	Merge branch 'hash-always-has-type' into better-ca-parse-errors	2020-07-13 03:02:09 +00:00
John Ericson	4415765385	Merge remote-tracking branch 'upstream/master' into hash-always-has-type	2020-07-13 03:01:44 +00:00
John Ericson	230c9b4329	Change types to prepare the way for CA derivations ... We've added the variant to `DerivationOutput` to support them, but made `DerivationOutput::path` partial to avoid actually implementing them. With this chage, we can all collaborate on "just" removing `DerivationOutput::path` calls to implement CA derivations.	2020-07-12 23:53:33 +00:00
John Ericson	fedfc913ad	Use more std::visit to prepare for new variant ... N.B. not using `std::visit` for fetchurl because there is no attempt to handle all the cases (e.g. no `else`) and lambda complicates early return.	2020-07-12 23:51:40 +00:00
John Ericson	ffc18583b1	Move C++17 "pattern matching" boilerplat to utils.hh	2020-07-12 22:15:14 +00:00
John Ericson	503b425690	DerivationOutputExtensional -> DerivationOutputInputAddressed ... Thanks @regnat for the great name.	2020-07-12 15:56:20 +00:00
John Ericson	a8d4707107	Undo erroneous indentation change	2020-07-12 15:54:12 +00:00
John Ericson	18152406ce	String .drv suffix to create derivation name	2020-07-12 15:40:14 +00:00
John Ericson	5d0b75e5b6	std::string_view for new derivation name parameters	2020-07-12 15:02:36 +00:00
Maximilian Bosch	64f03635d7	Fix ANSI color constants ... The `m` acts as termination-symbol when declaring graphics. Because of this, the `;1m` doesn't have any effect and is directly printed to the console: ``` $ nix repl > builtins.fetchGit { /* ... */ } { outPath = "/nix/store/s0f0iz4a41cxx2h055lmh6p2d5k5bc6r-source"; rev = "e73e45b723a9a6eecb98bd5f3df395d9ab3633b6"; revCount = ;1m428; shortRev = "e73e45b"; submodules = ;1mfalse; } ``` Introduced by 6403508f5a.	2020-07-12 16:52:20 +02:00
John Ericson	13ec627e0a	Set derivation name in dervationStrict	2020-07-12 03:03:12 +00:00
John Ericson	1c9bec226f	Don't improperly assume path is store path	2020-07-12 02:38:03 +00:00
John Ericson	abea26a968	BasicDerivation::findOutput cannot return reference anymore	2020-07-12 01:57:06 +00:00
John Ericson	442d43178b	Merge remote-tracking branch 'upstream/master' into optional-derivation-output-storepath	2020-07-11 02:27:45 +00:00
Matthew Bauer	acb74d4d94	Merge remote-tracking branch 'origin/master' into substitute-other-storedir	2020-07-10 18:13:45 -04:00
Eelco Dolstra	8efa23bb99	Avoid a redundant hash	2020-07-10 15:56:24 +02:00
Eelco Dolstra	5dff49f661	Factor out commonality between nix-prefetch-url and nix-store --add-fixed	2020-07-10 13:21:37 +02:00
Eelco Dolstra	7f1a86d57c	nix-store --add-fixed: Run in constant memory	2020-07-10 12:51:56 +02:00
Eelco Dolstra	06e3dd9005	nix-prefetch-url: Run in constant memory when using RemoteStore ... Fixes #3684.	2020-07-10 11:22:48 +02:00
John Ericson	323d51a7a0	Merge branch 'master' of github.com:NixOS/nix into better-ca-parse-errors	2020-07-09 19:07:06 +00:00
Matthew Bauer	a7884970c5	Fix DerivationOutputExtensional name	2020-07-09 11:37:18 -04:00
Eelco Dolstra	a2c27022e9	LocalStore::addToStore(srcPath): Handle the flat case ... This helps nix-prefetch-url when using a local store.	2020-07-09 15:54:32 +02:00
Matthew Bauer	06a4e15478	Fix build.cc on linux	2020-07-08 19:27:51 -04:00
Matthew Bauer	8e0d0689be	Only store hash of fixed derivation output ... we don’t need a full storepath for a fixedoutput derivation. So just putting the ingestion method + the hash is sufficient.	2020-07-08 19:11:39 -04:00
Eelco Dolstra	b981e5aacf	Cleanup	2020-07-08 22:07:21 +02:00
Eelco Dolstra	34f25124ba	Make LocalStore::addToStore(srcPath) run in constant memory ... This reduces memory consumption of nix-instantiate \ -E 'with import <nixpkgs> {}; runCommand "foo" { src = ./blender; } "echo foo"' \ --option nar-buffer-size 10000 (where ./blender is a 1.1 GiB tree) from 1716 to 36 MiB, while still ensuring that we don't do any write I/O for small source paths (up to 'nar-buffer-size' bytes). The downside is that large paths are now always written to a temporary location in the store, even if they produce an already valid store path. Thus, adding large paths might be slower and run out of disk space. ¯\_(ツ)_/¯ Of course, you can always restore the old behaviour by setting 'nar-buffer-size' to a very high value.	2020-07-08 22:07:21 +02:00
Matthew Bauer	af95a7c16b	Add name to BasicDerivation ... We always have a name for BasicDerivation, since we have a derivation store path that has a name.	2020-07-08 15:38:01 -04:00
Eelco Dolstra	9223603908	Merge remote-tracking branch 'origin/master' into flakes	2020-07-08 15:55:19 +02:00
Eelco Dolstra	16ec7785ca	Fix 'got unknown message type 1 from Nix daemon' ... Example: $ nix-build -E 'with import <nixpkgs> {}; runCommand "foo" { x = runCommand "bar" {} "exit 1"; } "echo foo; exit 1"' warning: unknown setting 'auto-allocate-uids' these 2 derivations will be built: /nix/store/v4fbdbhcdi949929a67g8farwf72zgam-bar.drv /nix/store/k4fsvrjl7cp2xpz7927iv7g0dqj1zyhs-foo.drv warning: unknown setting 'auto-allocate-uids' building '/nix/store/v4fbdbhcdi949929a67g8farwf72zgam-bar.drv'... error: --- Error ----------------------------------------------------------------------------------------------------------------------------------------------------------------- nix-daemon builder for '/nix/store/v4fbdbhcdi949929a67g8farwf72zgam-bar.drv' failed with exit code 1 error: --- Error ------------------------------------------------------------------------------------------------------------------------------------------------------------------ nix-build got unknown message type 1 from Nix daemon	2020-07-08 15:53:14 +02:00
Eelco Dolstra	1ab9da9154	Merge remote-tracking branch 'origin/master' into flakes	2020-07-07 14:38:57 +02:00
Eelco Dolstra	4055cfee36	Fix coverage build	2020-07-07 14:37:47 +02:00
Eelco Dolstra	7c9ece5dca	exportReferencesGraph: Fix support for non-top-level store paths ... Fixes #3471.	2020-07-07 14:25:43 +02:00
Carlo Nucera	0f05a36e20	Merge branch 'master' of github.com:NixOS/nix into better-ca-parse-errors	2020-07-06 16:57:09 -04:00
Ben Burdette	efd6a8b230	bump	2020-07-06 11:54:53 -06:00
Ben Burdette	75bfcf8d15	revamp trace code and test	2020-07-06 10:51:48 -06:00
Eelco Dolstra	68f524d717	nix develop: Support derivations with multiple outputs	2020-07-06 18:34:58 +02:00
Eelco Dolstra	cd8eb8a7d1	nix develop: Fall back to "bash" if nixpkgs#bashInteractive is unavailable	2020-07-06 17:08:54 +02:00
Eelco Dolstra	54712aaf8a	Merge remote-tracking branch 'origin/master' into flakes	2020-07-06 16:40:10 +02:00
John Ericson	a38ab99d57	Merge remote-tracking branch 'upstream/master' into derivation-header-include-order	2020-07-05 21:49:01 +00:00
Ben Burdette	a168224464	spacing	2020-07-04 18:30:49 -06:00
John Ericson	465daa9396	Merge remote-tracking branch 'upstream/master' into add-body-to-network-errors	2020-07-03 17:08:39 +00:00
John Ericson	3134db1a83	Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors	2020-07-03 14:12:38 +00:00
John Ericson	dbffd309fe	Merge branch 'master' of github.com:NixOS/nix into hash-always-has-type	2020-07-03 14:11:38 +00:00
Ben Burdette	b29a4ea1dc	Merge branch 'master' into add-trace	2020-07-03 07:57:36 -06:00
Eelco Dolstra	c3c7aedbb5	nix develop: Fix bad regex ... This was accepted by libstdc++ but not libc++. https://hydra.nixos.org/build/123569154	2020-07-03 14:58:58 +02:00
Eelco Dolstra	6f8fd3a3f2	Shut up a clang warning	2020-07-03 14:50:07 +02:00
regnat	223fbe644a	Shorten the path to the test root ... Fix a socket length failure on the OSX builders	2020-07-03 09:20:01 +02:00
John Ericson	13796be78d	Have splitPrefix and splitPrefixTo parser helpers	2020-07-02 23:18:22 +00:00
John Ericson	a7cd7425d9	Move getParsedTypeAndSRI to a more suitable location ... Also mark it static	2020-07-02 23:10:11 +00:00
John Ericson	2f93d9f2ba	Merge branch 'hash-always-has-type' into HEAD	2020-07-02 21:47:51 +00:00
John Ericson	1be279af26	Fix Narinfo corruption detection bug ... The aim of this check was just to ensure each key occurs once.	2020-07-02 21:46:10 +00:00
Eelco Dolstra	b5e4253697	Fix abort in 'nix develop'	2020-07-02 18:24:11 +02:00
Carlo Nucera	1fc835aa22	Tighten parsing for drv files and pathinfo	2020-07-02 11:57:21 -04:00
Ben Burdette	5818271c6e	spacing	2020-07-02 09:41:54 -06:00
Carlo Nucera	b6b10b1d4c	Write the implementation for parseNonSRIUnprefixed	2020-07-02 11:34:40 -04:00
Carlo Nucera	ea48e3a5b5	Abstract common parsing functionality	2020-07-02 11:29:33 -04:00
Carlo Nucera	36cbc74689	Inline and simplify in parseAnyPrefixed	2020-07-02 11:21:00 -04:00
Matthew Bauer	fc2ab42e86	Merge remote-tracking branch 'origin/master' into substitute-other-storedir	2020-07-02 11:14:04 -04:00
Matthew Bauer	d2e8b9ff0e	Store subPath in SubstitutionGoal	2020-07-02 11:12:05 -04:00
Carlo Nucera	9462d8a50b	Rename fromSRI to parseSRI for constistency	2020-07-02 11:11:18 -04:00
Carlo Nucera	f61bc45d19	Get rid of the std::pair	2020-07-02 11:09:04 -04:00
Ben Burdette	bf2788e4c1	move showTrace to new loggerSettings	2020-07-02 09:04:31 -06:00
Carlo Nucera	27c8029573	Inline newFunction	2020-07-02 11:01:10 -04:00
Matthew Bauer	1f9cb06db2	Try next when no ca exists and have different store dirs	2020-07-02 10:59:24 -04:00
Carlo Nucera	343d1569b1	Fix test suite	2020-07-02 10:48:47 -04:00
Ben Burdette	5ae498872a	assert for invalid fileorigin	2020-07-02 07:14:40 -06:00
Carlo Nucera	263ccdd489	Rename two hash constructors to proper functions	2020-07-01 18:34:18 -04:00
Carlo Nucera	c8c4bcf90e	Inline Hash::init()	2020-07-01 18:03:35 -04:00
Carlo Nucera	d63a5ded76	Remove unused import	2020-07-01 17:53:24 -04:00
Carlo Nucera	6faeec3b2a	Keep the previous name, for diffing	2020-07-01 17:50:34 -04:00
Carlo Nucera	274a8136fb	Correct FIXMEs in libfetchers	2020-07-01 17:47:15 -04:00
Carlo Nucera	c2e7f7a712	Fixed build, we still have test errors	2020-07-01 17:32:06 -04:00
Ben Burdette	8497891b99	spacing	2020-07-01 13:50:18 -06:00
Eelco Dolstra	6ff9aa8df7	Don't process an option if any of its arguments need completion	2020-07-01 20:31:39 +02:00
Eelco Dolstra	d746503e5c	Add --inputs-from to use flake inputs as registry entries ... This allows you to refer to an input from another flake. For example, $ nix run --inputs-from /path/to/hydra nixpkgs#hello runs 'hello' from the 'nixpkgs' inputs of the 'hydra' flake. Fixes #3769.	2020-07-01 20:25:13 +02:00
Ben Burdette	a295b2ea96	if no errLoc, no Loc.	2020-07-01 12:02:02 -06:00
Ben Burdette	3629b0585a	don't include errpos for addErrorContext	2020-07-01 11:49:01 -06:00
Ben Burdette	2a39c083dc	non-pos trace test	2020-07-01 10:37:31 -06:00
Eelco Dolstra	38ccf2e241	Cleanup	2020-07-01 15:31:34 +02:00
Eelco Dolstra	86a4aba6c4	Merge branch 'remote-query-outputs' of https://github.com/tweag/nix	2020-07-01 15:10:29 +02:00
Eelco Dolstra	7d554f295c	Support building flakes from a shallow Git repo ... Fixes #3756.	2020-07-01 14:57:59 +02:00
Ben Burdette	a7d5d26443	fix tests with the 'from string' change	2020-06-30 22:05:21 -06:00
Ben Burdette	9159dfe3d8	comments and cleanup	2020-06-30 16:31:55 -06:00
Ben Burdette	70bcb39d3f	double addtrace for 'called from'	2020-06-30 15:44:19 -06:00
Matthew Bauer	09833126f3	Merge remote-tracking branch 'origin/master' into substitute-other-storedir	2020-06-30 16:35:08 -04:00
Ben Burdette	ddb81ca126	Merge branch 'master' into add-trace	2020-06-30 12:21:45 -06:00
Carlo Nucera	b798efb829	WIP initial design	2020-06-30 14:10:30 -04:00
Ben Burdette	a0705e0dd1	invalid pos check	2020-06-30 11:01:46 -06:00
Ben Burdette	e72a16a339	check for a null symbol	2020-06-30 11:00:51 -06:00
Carlo Nucera	a1f66d1d9e	Factor the prefix splitting in hash	2020-06-30 12:49:00 -04:00
Carlo Nucera	77b51f4598	Factor the prefix splitting in content-address	2020-06-30 11:57:46 -04:00
Carlo Nucera	7ba0fae0dd	Create the spitPrefix function in parser.hh	2020-06-30 11:57:09 -04:00
Ben Burdette	c484a67914	trace formatting	2020-06-29 15:46:21 -06:00
John Ericson	24da034bc3	Add possibly missing <string> include	2020-06-29 21:22:00 +00:00
Eelco Dolstra	2b834d48aa	NAR parser: Fix missing name field check ... Discovered by @Kloenk.	2020-06-29 22:45:41 +02:00
Matthew Bauer	289558dffb	Add unordered_set to globals.cc header	2020-06-29 20:34:45 +00:00
Matthew Bauer	da77331cb7	Remove lazy lookup in getHome ... this seems to break in Musl/Static with: terminate called after throwing an instance of 'std::bad_function_call' what(): bad_function_call	2020-06-29 20:34:26 +00:00
Matthew Bauer	88cf6ffce3	Rename logging->stdout to logging->stdout_ ... musl doesn't like this identifier	2020-06-29 20:34:26 +00:00
John Ericson	a83566e5bc	Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors	2020-06-29 18:40:34 +00:00
John Ericson	987a4a0be9	Merge remote-tracking branch 'upstream/master' into hash-always-has-type	2020-06-29 18:03:21 +00:00
Eelco Dolstra	26cf0c674f	nix run: Use packages/legacyPackages as fallback if there is no app definition ... 'nix run' will try to run $out/bin/<name>, where <name> is the derivation name (excluding the version). This often works well: $ nix run nixpkgs#hello Hello, world! $ nix run nix -- --version nix (Nix) 2.4pre20200626_adf2fbb $ nix run patchelf -- --version patchelf 0.11.20200623.e61654b $ nix run nixpkgs#firefox -- --version Mozilla Firefox 77.0.1 $ nix run nixpkgs#gimp -- --version GNU Image Manipulation Program version 2.10.14 though not always: $ nix run nixpkgs#git error: unable to execute '/nix/store/kp7wp760l4gryq9s36x481b2x4rfklcy-git-2.25.4/bin/git-minimal': No such file or directory	2020-06-29 19:08:50 +02:00
Eelco Dolstra	50f13b06fb	EvalCache: Store string contexts	2020-06-29 19:08:37 +02:00
Ben Burdette	8f81fae116	showTrace flag in loggers	2020-06-29 10:20:51 -06:00
Eelco Dolstra	b681408879	Factor out EvalCache::forceDerivation()	2020-06-29 16:39:41 +02:00
Eelco Dolstra	ca946860ce	Fix bash completion	2020-06-29 14:37:22 +02:00
Eelco Dolstra	bc03c6f23d	Move App	2020-06-29 14:14:23 +02:00
Eelco Dolstra	58bc3b6578	Merge pull request #3729 from obsidiansystems/simpler-hased-mirror ... hashed-mirrors: Use parsed derivation output rather than reconstructing it	2020-06-29 14:04:12 +02:00
Eelco Dolstra	64232f3ea6	Merge pull request #3749 from rodarima/master ... Fall back to copyPath if link fails with EPERM	2020-06-29 13:31:24 +02:00
Ben Burdette	ef24a0835d	showtrace as function arg	2020-06-27 12:19:31 -06:00
Carlo Nucera	278a679782	Merge branch 'master' of github.com:NixOS/nix into allow-relative-paths-in-store-option	2020-06-26 10:40:13 -04:00
Eelco Dolstra	adf2fbbdc2	Merge remote-tracking branch 'origin/master' into flakes	2020-06-26 08:46:46 +02:00
John Ericson	d92d4f85a5	Move ValidPathInfo to its own header	2020-06-25 17:48:49 +00:00
Ben Burdette	bc9e87412c	'string' makes more sense in nix repl	2020-06-25 09:56:32 -06:00
Ben Burdette	9ab808c926	showTrace flag for ErrorInfo; showTrace test.	2020-06-25 09:23:12 -06:00
Eelco Dolstra	de2641ae99	Fix empty std::optional dereference in writeDerivation() ... https://hydra.nixos.org/build/123017579	2020-06-25 15:50:30 +02:00
Rodrigo	3a642187c3	Fall back to copyPath if link fails with EPERM ... BeeGFS doesn't allow hard-links and returns EPERM, so we fall back to copyPath. See https://github.com/NixOS/nix/issues/3748	2020-06-25 12:03:26 +02:00
Ben Burdette	9c0e1fd4f1	add trace test; error formatting refinements	2020-06-24 18:31:28 -06:00
Ben Burdette	6359d71d6b	re-enable --show-trace check	2020-06-24 18:28:20 -06:00
Ben Burdette	023912def3	convenience form of addTrace	2020-06-24 13:46:25 -06:00
Ben Burdette	93e9307329	repl indenting	2020-06-24 13:14:49 -06:00
Ben Burdette	b18ed02b76	repl indenting	2020-06-24 13:10:41 -06:00
regnat	d38f860c3e	Add a way to get all the outputs of a derivation with their label ... Generalize `queryDerivationOutputNames` and `queryDerivationOutputs` by adding a `queryDerivationOutputMap` that returns the map `outputName=>outputPath` (not that this is not equivalent to merging the results of `queryDerivationOutputs` and `queryDerivationOutputNames` as sets don't preserve the order, so we would end up with an incorrect mapping). squash! Add a way to get all the outputs of a derivation with their label Rename StorePathMap to OutputPathMap	2020-06-24 20:38:40 +02:00
Ben Burdette	6fe660acf9	re-remove	2020-06-24 12:33:05 -06:00
Matthew Bauer	bcf8582091	Merge remote-tracking branch 'origin/master' into substitute-other-storedir	2020-06-24 12:52:09 -04:00
John Ericson	d9fb9e9f26	Merge remote-tracking branch 'upstream/master' into ca-drv	2020-06-24 15:41:29 +00:00
Ben Burdette	00fe653ea5	nixCode -> LinesOfCode	2020-06-24 08:33:53 -06:00
Ben Burdette	1d43a6e123	use plain errPos instead of nixCode; fix tests	2020-06-23 15:30:13 -06:00
Carlo Nucera	8d51d38e4c	Fix test suite	2020-06-23 17:16:20 -04:00
John Ericson	ec3a857307	Fix and clean up hash parser	2020-06-23 18:19:40 +00:00
John Ericson	98e5d1af03	Merge remote-tracking branch 'upstream/master' into hash-always-has-type	2020-06-23 17:03:37 +00:00
Ben Burdette	d0e78fbb03	re-add Pos origin in tests	2020-06-23 10:51:58 -06:00
Ben Burdette	abe0552504	Merge remote-tracking branch 'upstream/master' into add-trace	2020-06-23 09:40:28 -06:00
Ben Burdette	13e87535ff	traces to bottom	2020-06-23 09:36:58 -06:00
Carlo Nucera	e197bc6229	Enable the --store option to take relative paths ... In nix commands which accept --store options, we can now specify a relative path, which will be canonicalized.	2020-06-23 11:13:18 -04:00
Eelco Dolstra	09fc06daab	nix flake init: Use git add --force	2020-06-23 16:25:32 +02:00
Eelco Dolstra	015e1c2131	Merge pull request #3724 from bburdette/hintfmt-percent ... Hintfmt percent test, and fix	2020-06-23 12:26:00 +02:00
John Ericson	55d4bd6e0e	Improve content address parsing ... - Ensure hash is in form <algo>-<prefix> and not SRI. - Better errors if something goes wrong - string_view for no coppying	2020-06-22 18:08:27 +00:00
Ben Burdette	9d1cb0c5e6	with normaltxt, elide yellow color code instead of canceling it; use normaltxt on plain_string hintfmt	2020-06-22 11:32:20 -06:00
Matthew Bauer	66a62b3189	Merge remote-tracking branch 'origin/master' into substitute-other-storedir	2020-06-22 13:08:11 -04:00
Ben Burdette	28b079067f	Update src/libutil/fmt.hh ... Co-authored-by: John Ericson <git@JohnEricson.me>	2020-06-22 10:00:37 -06:00
John Ericson	f4a5913125	hashed-mirrors: Use parsed derivation output rather than reconstructing it ... Now the derivation outputs are parsed up front, we can avoid a reparse by doing it. Also, this just feels a bit better as the `output*` env vars are more of a `libnixexpr` interface than `libnixstore` interface: ultimately, it's the derivation outputs that decide whether the derivation is fixed-output. Yes, hashed mirrors might go away with #3689, but this bit of code would be moved rather than deleted, so it's worth doing a cleanup anyways I think.	2020-06-22 15:17:20 +00:00
Eelco Dolstra	334e26bfc2	nix flake check: Don't build apps ... This was inconsistent since we're not building 'packages' or 'defaultPackage' either. Closes #3726.	2020-06-22 11:31:07 +02:00
John Ericson	93bbe6e8ab	Merge remote-tracking branch 'upstream/master' into ca-drv	2020-06-21 21:11:58 +00:00
John Ericson	3804e3df9b	Don't anticipate hash algo without hash in derivation for now ... When we merge with master, the new lack of string types make this case impossible (after parsing). Later, when we actually implemenent CA-derivations, we'll change the types to allow that.	2020-06-21 21:05:37 +00:00
John Ericson	8313f0e939	Merge remote-tracking branch 'upstream/master' into derivation-header-include-order	2020-06-21 20:39:10 +00:00
John Ericson	bf9f040112	Tweak declaration ... I think this is clearer	2020-06-21 16:51:39 +00:00
John Ericson	fdeabf7160	Merge remote-tracking branch 'upstream/master' into multi-output-hashDerivationModulo	2020-06-21 16:43:17 +00:00
Ben Burdette	be4f444175	tidying up	2020-06-19 16:58:12 -06:00
Ben Burdette	0309488a66	fmt -> hintfmt test	2020-06-19 16:46:49 -06:00
Matthew Bauer	f2a6cee334	Update worker protocol to support sending storepath maps ... We need to also send the ca to daemon in addition to the path.	2020-06-19 18:06:19 -04:00
Ben Burdette	397dbe114e	remove formathelper	2020-06-19 15:57:19 -06:00
John Ericson	507aa48739	WIP: Make Hash always store a valid hash type	2020-06-19 21:48:57 +00:00
Ben Burdette	b193aca4ae	escape percents	2020-06-19 15:29:19 -06:00
Ben Burdette	db475f9e7e	too few, too many args	2020-06-19 15:28:13 -06:00
Ben Burdette	cdddf24f25	add hintfmt test	2020-06-19 14:54:41 -06:00
Ben Burdette	54e8f550c9	addErrorTrace	2020-06-19 13:44:08 -06:00
Matthew Bauer	e8e1f5282f	Replace error message in getDerivationCA	2020-06-19 15:19:09 -04:00
Matthew Bauer	0c9c1b8826	Return map of StorePaths in copyPaths ... This allows the caller to know what values were actually added to the store.	2020-06-19 14:48:57 -04:00
John Ericson	e288c0987a	Merge remote-tracking branch 'upstream/master' into validPathInfo-ca-proper-datatype	2020-06-19 18:44:24 +00:00
Matthew Bauer	7e11cf3399	Merge remote-tracking branch 'origin/master' into substitute-other-storedir	2020-06-19 14:40:49 -04:00
Matthew Bauer	2796b01e86	Merge remote-tracking branch 'origin/master' into substitute-other-storedir	2020-06-19 14:22:06 -04:00
John Ericson	68294746ae	Merge remote-tracking branch 'upstream/master' into no-hash-type-unknown	2020-06-19 17:53:34 +00:00
John Ericson	c98081d270	Merge remote-tracking branch 'upstream/master' into no-hash-type-unknown	2020-06-19 17:50:05 +00:00
John Ericson	c1892a5316	tabs -> spaces	2020-06-19 17:49:57 +00:00
John Ericson	911fc88bcb	More designated initializers	2020-06-19 17:42:56 +00:00
John Ericson	2f0e395c99	Merge remote-tracking branch 'me/no-stringly-typed-derivation-output' into validPathInfo-ca-proper-datatype	2020-06-19 15:26:59 +00:00
John Ericson	fb39a5e00c	Remove unneeded constructor for DerivationOutputHash	2020-06-19 15:11:11 +00:00
John Ericson	b90cac3bad	Remove uneeded = default for Hash	2020-06-19 15:00:38 +00:00
John Ericson	01dc8b0bab	Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output	2020-06-19 14:59:05 +00:00
John Ericson	145d88cb2a	Use designated initializers for DerivationOutputHash	2020-06-19 14:58:30 +00:00
John Ericson	237d88c97e	FileSystemHash -> DerivationOutputHash	2020-06-19 14:47:10 +00:00
Eelco Dolstra	2886c92aef	Merge pull request #3669 from gilligan/add-compression-tests ... Add compression unit tests	2020-06-19 13:59:04 +02:00
John Ericson	3fc58a9638	Remove some Base:: that crept in	2020-06-19 00:24:47 +00:00
John Ericson	3f8dcfe3fd	Merge branch 'validPathInfo-temp' into validPathInfo-ca-proper-datatype	2020-06-18 23:01:58 +00:00
John Ericson	669c3992e8	Merge branch 'no-hash-type-unknown' into validPathInfo-temp	2020-06-18 22:33:07 +00:00
John Ericson	15abb2aa2b	Revert the enum struct change ... Not a regular git revert as there have been many merges and things.	2020-06-18 22:11:26 +00:00
John Ericson	bbbf3602a3	Merge branch 'enum-class' into no-hash-type-unknown	2020-06-18 22:11:19 +00:00
John Ericson	40526fbea5	Merge remote-tracking branch 'upstream/master' into enum-class	2020-06-18 21:38:15 +00:00
Ben Burdette	4d1a4f0217	addTrace	2020-06-18 15:25:26 -06:00
Ben Burdette	e6f93b94fc	Merge branch 'master' into caveman-LOCs	2020-06-18 13:07:53 -06:00
John Ericson	1b23fe4afb	Fix bugs ... - Bad dynamic cast target ...classic - std::shared_ptr need explicit deref	2020-06-18 19:03:10 +00:00
John Ericson	fb432b0bd2	Merge remote-tracking branch 'obsidian/fix-url-format' into add-body-to-network-errors	2020-06-18 18:08:02 +00:00
John Ericson	639e20dc3e	Prevent '%' in URL from causing crashes ... We have a larger problem that passsing computed strings to the first variable argument of many exception constructors is unsafe because that first variable argument is interpreted not as a plain string, but format string, and if it contains '%' boost::format will abort, since there are no arguments to the format string. In this particular instance '%' was used as part of an escape code in a URL, which, when the download failed, caused Nix to abort displaying the `FileTransferError`.	2020-06-18 17:54:16 +00:00
Eelco Dolstra	6c000eed80	Merge pull request #3709 from expipiplus1/master ... Mention number of derivations to be build/fetched in output	2020-06-18 19:03:05 +02:00
John Ericson	74b219ef6e	Adjust FileTransferError message to use opt response	2020-06-18 14:48:45 +00:00
Eelco Dolstra	5771c8bbf2	Don't provide 'getFlake' if the 'flakes' feature is not enabled ... (cherry picked from commit 0a1d3c1dd3)	2020-06-18 14:03:00 +02:00
Eelco Dolstra	2a61bbf77f	Some backports from the flakes branch	2020-06-18 14:03:00 +02:00
Eelco Dolstra	377345e26f	Remove unneeded #include	2020-06-18 13:47:05 +02:00
Eelco Dolstra	7083d33efe	Make constant primops lazy ... (cherry picked from commit aa0e2a2e70)	2020-06-18 13:42:47 +02:00
Eelco Dolstra	3d492199bb	github: Respect default branch	2020-06-18 13:25:08 +02:00
regnat	4fef2ba7e4	Rename content-addressed-paths into ca-derivations ... See <https://github.com/NixOS/nix/pull/3710#issuecomment-645480333>	2020-06-18 09:25:55 +02:00
Carlo Nucera	004570a377	Add HTTP responses to FileTransferErrors	2020-06-17 22:34:31 +00:00
Carlo Nucera	a835c740ca	Replace TransferItem::status with a local variable ... Everywhere seems to use `getHTTPStatus` now.	2020-06-17 22:16:24 +00:00
Carlo Nucera	289b9b8dcf	Create a new TeeSink abstraction ... This is a bit complex because we want to expose extra functionality the wrapped class has. Perhaps there is some inheritancy trickery to do this nicer, but I don't know it, and this is the first thing we tried after a series of attempts that did build. This design is kind of like that of Rust's Writer, Reader, or Iter adapters, which impliment more traits based on what the inner type implements.	2020-06-17 22:09:32 +00:00
Carlo Nucera	fb05a6adcf	Eliminate old TeeSink abstraction ... This was introduced in fa125b9b28, and then "reverted" in 1cf4801108, except that revert left the struct around doing nothing useful. We're removing it all the way now because we want to make a new `TeeSink` complementing the already-exiting `TeeSource`, that is actually a completely different concept as far as the class hierarchy is concerned.	2020-06-17 22:07:22 +00:00
Eelco Dolstra	2b8f33bf5f	Merge pull request #3713 from matthewbauer/cleanup-warnings ... Cleanup class StorePath warning	2020-06-17 21:19:30 +02:00
Matthew Bauer	5e631e3304	Add StorePathCAMap for querySubstitutablePathInfos ... I’m not 100% sure this is wanted since it kind of makes everything have to know about ca even if they don’t really want to. But it also make things easier in dealing with looking up ca.	2020-06-17 15:03:05 -04:00
Matthew Bauer	be50de1142	Make sure references are empty for store path replacing ... also copy info2 instead of casting	2020-06-17 14:15:08 -04:00
Matthew Bauer	8974755d19	Add assert for replaced storePath	2020-06-17 14:04:46 -04:00
Eelco Dolstra	3078404e35	Merge pull request #3712 from obsidiansystems/make-http-successful-states-coherent ... Make successful states coherent	2020-06-17 19:37:46 +02:00
Matthew Bauer	7bd88cc1dc	Merge remote-tracking branch 'origin/master' into substitute-other-storedir	2020-06-17 13:32:29 -04:00
Matthew Bauer	22d7d36703	Remove unused narInfoFile in binary-cache-store	2020-06-17 13:27:19 -04:00
Matthew Bauer	f767bedfac	Replace struct StorePath with class StorePath ... also a similar case with struct Goal	2020-06-17 13:26:37 -04:00
Carlo Nucera	4930cb48a2	Include review comments	2020-06-17 12:58:59 -04:00
Matthew Bauer	6438ba1e99	Update strings from review comment	2020-06-17 12:31:01 -04:00
Eelco Dolstra	2f51cd8dc9	Merge pull request #3710 from tweag/reserve_ca_derivations ... Reserve the `__contentAddressed` derivation parameter	2020-06-17 18:28:26 +02:00
Eelco Dolstra	4d5169bdd5	Merge pull request #3707 from p01arst0rm/outdated-function-fix ... replaced uncaught_exception with uncaught_exceptions	2020-06-17 18:26:01 +02:00
Eelco Dolstra	2e4bd78211	nix eval: Add --apply flag for post-processing the result	2020-06-17 18:12:24 +02:00
regnat	480b54e1c6	fixup! Reserve the __contentAddressed derivation parameter	2020-06-17 17:37:04 +02:00
Carlo Nucera	079c6e87de	Make successful states coherent ... The successful states used in these two places in the code were slightly different. Should they be the same list?	2020-06-17 11:16:16 -04:00
Eelco Dolstra	5332c439d0	InstallableFlake: Show all possible attribute names ... E.g. $ nix run nixpkgs#hello error: --- Error ---------- nix flake 'flake:nixpkgs' does not provide attribute 'apps.x86_64-linux.hello' or 'hello' instead of $ nix run nixpkgs#hello error: --- Error ---------- nix flake 'flake:nixpkgs' does not provide attribute 'hello'	2020-06-17 17:13:01 +02:00
John Ericson	6403508f5a	Use ansicolor.hh in nix repl rather than duplicates	2020-06-17 15:13:00 +00:00
Eelco Dolstra	ad66fb0a37	getFlake -> builtins.getFlake	2020-06-17 17:05:08 +02:00
Eelco Dolstra	0a1d3c1dd3	Don't provide 'getFlake' if the 'flakes' feature is not enabled	2020-06-17 16:54:32 +02:00
regnat	56d75bf4fc	Reserve the __contentAddressed derivation parameter ... Not implementing anything here, just throwing an error if a derivation sets `__contentAddressed = true` without `--experimental-features content-addressed-paths` (and also with it as there's nothing implemented yet)	2020-06-17 15:41:17 +02:00
Eelco Dolstra	fdff09e57c	Fix coverage build	2020-06-17 15:18:10 +02:00
Eelco Dolstra	ccbea8255c	Merge pull request #3657 from obsidiansystems/sligthly-improve-store-path-documentation ... Clarify the description of StorePath inputs	2020-06-17 14:54:37 +02:00
Joe Hermaszewski	da8aac6ce8	Mention number of derivations to be build/fetched in output ... Also correct grammar for the case of a single derivation.	2020-06-17 20:27:27 +08:00
Eelco Dolstra	1524752c17	Merge remote-tracking branch 'origin/master' into flakes	2020-06-17 10:26:52 +02:00
John Ericson	517f5980e2	Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output	2020-06-17 04:58:43 +00:00
John Ericson	18493fd9c4	Move some Store functions from derivations.cc to store-api.cc ... This further continues with the dependency inverstion. Also I just went ahead and exposed `parseDerivation`: it seems like the more proper building block, and not a bad thing to expose if we are trying to be less wedded to drv files on disk anywas.	2020-06-17 04:30:16 +00:00
John Ericson	7130f0a3a6	Don't need abstract struct Derivation in local-store	2020-06-17 04:05:38 +00:00
John Ericson	21ef342172	Merge remote-tracking branch 'upstream/master' into derivation-header-include-order	2020-06-17 03:52:01 +00:00
John Ericson	02928f76fd	Merge remote-tracking branch 'upstream/master' into multi-output-hashDerivationModulo	2020-06-17 03:51:02 +00:00
John Ericson	49308ef039	Merge remote-tracking branch 'upstream/master' into ca-drv	2020-06-17 03:39:39 +00:00
p01arst0rm	e9970a34e8	appended ' __attribute__((weak)); ' to 'extern char * * environ '	2020-06-17 03:25:34 +01:00
p01arst0rm	c9d06558b6	replaced uncaught_exception with uncaught_exceptions	2020-06-17 03:15:47 +01:00
Eelco Dolstra	29542865ce	Remove StorePath::clone() and related functions	2020-06-16 22:20:18 +02:00
Eelco Dolstra	df4da4f5da	Merge pull request #3702 from NixOS/store-path-cxx ... Rewrite StorePath class in C++	2020-06-16 21:37:26 +02:00
John Ericson	fbf90bd693	Add another missing #include	2020-06-16 14:19:49 +00:00
Eelco Dolstra	759947bf72	StorePath: Rewrite in C++ ... On nix-env -qa -f '<nixpkgs>', this reduces maximum RSS by 20970 KiB and runtime by 0.8%. This is mostly because we're not parsing the hash part as a hash anymore (just validating that it consists of base-32 characters). Also, replace storePathToHash() by StorePath::hashPart().	2020-06-16 14:28:41 +02:00
Eelco Dolstra	72e17290d4	Fix FTP support ... Fixes #3618.	2020-06-16 11:53:04 +02:00
Tobias Pflug	cd8214c398	Fix logging unit tests	2020-06-16 10:23:15 +02:00
John Ericson	7e7e3b71f3	Add mising #include for strerror	2020-06-15 23:35:07 +00:00
Eelco Dolstra	a588b6b19d	Print only one error message if a build fails ... E.g. instead of error: --- BuildError ----------------------------------------------- nix builder for '/nix/store/03nk0a3n8h2948k4lqfgnnmym7knkcma-foo.drv' failed with exit code 1 error: --- Error ---------------------------------------------------- nix build of '/nix/store/03nk0a3n8h2948k4lqfgnnmym7knkcma-foo.drv' failed we now get error: --- Error ---------------------------------------------------- nix builder for '/nix/store/03nk0a3n8h2948k4lqfgnnmym7knkcma-foo.drv' failed with exit code 1	2020-06-15 19:35:31 +02:00
Eelco Dolstra	24a3208247	Include only the base name of the program in error messages	2020-06-15 19:35:31 +02:00
Eelco Dolstra	8b099812ea	Respect terminal width printing error messages	2020-06-15 19:35:31 +02:00
Eelco Dolstra	f20bb983ca	Cleanup	2020-06-15 18:16:03 +02:00
Eelco Dolstra	4e995bc8a6	Always hide the progress bar on exit	2020-06-15 18:01:05 +02:00
Eelco Dolstra	31707735b6	Remove unnecessary amDone() overrides	2020-06-15 16:47:21 +02:00
Eelco Dolstra	ccfa6b3eee	Give better error message about <...> in pure eval mode	2020-06-15 16:12:27 +02:00
Eelco Dolstra	5ed5d7acbd	Improve "waiting for locks" messages ... These are now shown in the progress bar. Closes #3577.	2020-06-15 16:03:29 +02:00
Eelco Dolstra	e14e62fddd	Remove trailing whitespace	2020-06-15 14:12:39 +02:00
Eelco Dolstra	1fb762d11f	Get rid of explicit ErrorInfo constructors	2020-06-15 14:06:58 +02:00
Eelco Dolstra	fd64e4fb96	Disambiguate BaseError(Args) constructor ... This means that 'throw Error({ ... ErrorInfo ... })' now works.	2020-06-15 13:50:33 +02:00
Eelco Dolstra	7a77762961	Merge branch 'errors-phase-2' of https://github.com/bburdette/nix	2020-06-15 11:46:31 +02:00
Matthew Bauer	c214cda940	Correctly substitute from different storeDir ... Originally, the test was only checking for different “real” storeDir. That’s an easy case to handle, but the much harder one is if different virtual store dirs are used. To do this, we need the SubstitutionGoal to know about the ca, so it can recalculate the path to copy it over. An important note here is that the store path passed to copyStorePath needs to be one for srcStore - so that queryPathInfo works properly. This also adds an error message when the store path from queryPathInfo is different from the one we requested.	2020-06-13 00:07:42 -05:00
Matthew Bauer	b2cb288cdd	Add makeFixedOutputPathFromCA function ... This puts what we are already doing into a shared method. It just needs a path name and a ca and produces a store path.	2020-06-12 16:36:35 -05:00
John Ericson	f6f01416b7	Use std::string_view in a few more places	2020-06-12 21:32:30 +00:00
Matthew Bauer	88120442d2	Debug when storePath changes ... these rewrites should be transparent, but they are important to know about when debugging	2020-06-12 15:32:52 -05:00
Matthew Bauer	e3cb536f19	Fix add-to-store --flat to put in correct hash	2020-06-12 15:25:45 -05:00
Matthew Bauer	5b386b05f5	Recompute storePath based on isContentAddressed	2020-06-12 14:57:45 -05:00
Matthew Bauer	85d01e1f65	Don’t use makeStorePath	2020-06-12 14:53:30 -05:00
Matthew Bauer	9077b9e6b2	Separate dstStore path from srcStore path	2020-06-12 14:27:28 -05:00
Matthew Bauer	2f2ac850b5	Compute new store path correctly	2020-06-12 14:03:17 -05:00
Matthew Bauer	1c55f16a16	Add --flat to nix add-to-store ... This can be used to add flat hashes to the nix store.	2020-06-12 12:42:54 -05:00
Eelco Dolstra	2853ba4ab2	Fix build	2020-06-12 19:00:48 +02:00
Eelco Dolstra	00fa7e2205	Merge pull request #3674 from matthewbauer/allow-empty-hash2 ... Allow empty hash in derivations	2020-06-12 18:18:12 +02:00
Matthew Bauer	ea0d29d99a	Provide base argument to to_string	2020-06-12 10:18:27 -05:00
Matthew Bauer	11c97070f3	Fix storeDir != storeDir condition ... this needs to only continue if the path replacement fails.	2020-06-12 10:14:03 -05:00
Matthew Bauer	b260c9ee03	Add newHashAllowEmpty helper function ... This replaces the copy&paste with a helper function in hash.hh.	2020-06-12 10:11:16 -05:00
Matthew Bauer	3e3eaa90dd	Remove hashed-mirrors	2020-06-12 09:51:44 -05:00
Matthew Bauer	79c169d1c6	Allow substituting from different storeDir ... Substituters can substitute from one store dir to another with a little bit of help. The store api just needs to have a CA so it can recompute the store path based on the new store dir. We can only do this for fixed output derivations with no references, though.	2020-06-12 09:49:09 -05:00
Eelco Dolstra	9f736dd89d	Add Store::readDerivation() convenience function	2020-06-12 13:04:52 +02:00
Eelco Dolstra	045b07200c	Remove Store::queryDerivationOutputNames() ... This function was used in only one place, where it could easily be replaced by readDerivation() since it's not performance-critical. (This function appears to have been modelled after queryDerivationOutputs(), which exists only to make the garbage collector faster.)	2020-06-12 12:46:33 +02:00
Eelco Dolstra	4a4c063222	Merge pull request #3670 from gilligan/add-pool-tests ... Add tests for pool.hh	2020-06-12 11:19:05 +02:00
Eelco Dolstra	7db879e65e	Check 'follows' inputs	2020-06-12 00:52:56 +02:00
Ben Burdette	ef1b3f21b6	Merge remote-tracking branch 'upstream/master' into errors-phase-2	2020-06-11 14:06:35 -06:00
Eelco Dolstra	d15c20efd5	diffLockFiles(): Show 'follows' changes	2020-06-11 22:00:58 +02:00
Eelco Dolstra	ac4d43a31b	Merge pull request #3073 from tweag/machine-logs ... Add an option to print the logs in a machine-readable format	2020-06-11 15:45:18 +02:00
Eelco Dolstra	dd9bb11d0d	Move names.{cc,hh} to libstore	2020-06-11 15:42:18 +02:00
Eelco Dolstra	95eb064062	Shut up warning	2020-06-11 15:39:30 +02:00
Eelco Dolstra	8bd892117a	Style fixes	2020-06-11 15:39:08 +02:00
Eelco Dolstra	0c62b4ad0f	Represent 'follows' inputs explicitly in the lock file ... This fixes an issue where lockfile generation was not idempotent: after updating a lockfile, a "follows" node would end up pointing to a new copy of the node, rather than to the original node.	2020-06-11 14:40:21 +02:00
Tobias Pflug	4750d98bbd	Add tests for pool.hh	2020-06-10 22:29:50 +02:00
Eelco Dolstra	195ed43b60	Preserve 'isFlake' when not updating a lock file entry	2020-06-10 16:24:05 +02:00
Eelco Dolstra	2226e97ec2	Combine lock file update messages	2020-06-10 15:22:12 +02:00
Eelco Dolstra	fc6c7af424	Add helper function printInputPath()	2020-06-10 15:20:00 +02:00
Eelco Dolstra	b9ae1bdd7a	Merge pull request #3655 from zimbatm/hash-encoding-prepare ... libutils/hash: remove default encoding	2020-06-10 11:48:38 +02:00
Matthew Bauer	b2c8061b44	Disable extra-platforms = i686-linux on wsl1 (#3676) ... WSL1 doesn’t support i686-linux emulation, see https://github.com/microsoft/wsl/issues/2468	2020-06-09 21:53:53 +00:00
Matthew Bauer	19aa892f20	Support empty hash in fetchers ... fetchTarball, fetchTree, and fetchGit all have *optional* hash attrs. This means that we need to be careful with what we allow to avoid accidentally making these defaults. When ‘hash = ""’ we assume the empty hash is wanted.	2020-06-09 11:10:54 -05:00
Eelco Dolstra	29e0748847	Show HTTP status message ... For example: warning: unable to download 'https://api.github.com/repos/edolstra/dwarffs/commits/master': HTTP error 403 ('rate limit exceeded'); using cached version	2020-06-09 14:20:22 +02:00
Eelco Dolstra	447ea52b07	FileTransfer: Don't store status since curl already does that	2020-06-09 14:05:15 +02:00
Eelco Dolstra	6cfc2db494	Fix applyOverride() for github	2020-06-09 13:45:07 +02:00
Matthew Bauer	762273f1fd	Allow empty hash in derivations ... follow up of https://github.com/NixOS/nix/pull/3544 This allows hash="" so that it can be used for debugging purposes. For instance, this gives you an error message like: warning: found empty hash, assuming you wanted 'sha256:0000000000000000000000000000000000000000000000000000' hash mismatch in fixed-output derivation '/nix/store/asx6qw1r1xk6iak6y6jph4n58h4hdmbm-nix': wanted: sha256:0000000000000000000000000000000000000000000000000000 got: sha256:0fpfhipl9v1mfzw2ffmxiyyzqwlkvww22bh9wcy4qrfslb4jm429	2020-06-09 01:23:37 -05:00
Ben Burdette	2f19650768	add file origin to Pos in stests	2020-06-08 11:21:17 -06:00
Ben Burdette	b1c53b034c	Merge branch 'errors-phase-2' into caveman-LOCs	2020-06-08 11:10:13 -06:00
regnat	801112de1a	Move progress-bar.cc to libmain ... Needed so that we can include it as a logger in loggers.cc without adding a dependency on nix This also requires moving names.hh to libutil to prevent a circular dependency between libmain and libexpr	2020-06-08 17:16:52 +02:00
Eelco Dolstra	e073f2c584	nix flake: Require 'flakes' feature	2020-06-08 16:23:54 +02:00
Eelco Dolstra	6470450ab4	Add completion for --update-input	2020-06-08 16:20:00 +02:00
Eelco Dolstra	c27f92698b	Style fixes	2020-06-08 13:24:01 +02:00
Tobias Pflug	cd6dbf951a	Add compression unit tests	2020-06-08 11:34:37 +02:00
regnat	4983401440	Unify the printing of the logs between bar-with-logs and raw ... Make the printing of the build logs systematically go through the logger, and replicate the behavior of `no-build-output` by having two different loggers (one that prints the build logs and one that doesn't)	2020-06-08 09:31:15 +02:00
regnat	170e86dff5	Make the logger customisable ... Add a new `--log-format` cli argument to change the format of the logs. The possible values are - raw (the default one for old-style commands) - bar (the default one for new-style commands) - bar-with-logs (equivalent to `--print-build-logs`) - internal-json (the internal machine-readable json format)	2020-06-08 09:31:15 +02:00
Ben Burdette	94c347577e	set verbosity levels	2020-06-07 07:24:49 -06:00
Tobias Pflug	e60747b5fb	Remove error-demo/error-demo.cc ... The logging.hh superseeds the demo	2020-06-06 10:23:12 +02:00
Tobias Pflug	952e72c804	Add tests for logging.hh	2020-06-06 10:22:32 +02:00
Eelco Dolstra	9ef6048d78	diffLockFiles(): Fix assertion failure ... There are some cases where this inequality didn't hold, in particular due to the Input / TreeInfo merge, where we're not always showing narHash.	2020-06-05 20:52:23 +02:00
Eelco Dolstra	d558fb98f6	Merge pull request #3656 from obsidiansystems/handle-unknown-file-ingestion ... Add error message when FileIngestionMethod is out of bounds	2020-06-05 17:18:12 +02:00
Eelco Dolstra	488ff83e6b	Fix completion of --template	2020-06-05 14:09:12 +02:00
Eelco Dolstra	39e84c35d0	Fix log-prefix of nix build -L ... Alternative fix to #3661. The cause was that 'name' is a std::string_view into a temporary which could get overwritten.	2020-06-05 10:45:05 +02:00
Carlo Nucera	d614166cb6	Fix condition error and make test suite pass	2020-06-04 17:21:21 -04:00
John Ericson	e5cc1ebc5d	Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output	2020-06-04 21:04:35 +00:00
John Ericson	a7b82fd006	Remove file which shouldn't be committed	2020-06-04 21:04:20 +00:00
John Ericson	94ddea9e2f	Use readString rather than >> temporary ... Fixed the rest of these before, but this one slipped through.	2020-06-04 20:55:08 +00:00
John Ericson	744ce9ce16	Merge branch 'master' of github.com:NixOS/nix into validPathInfo-ca-proper-datatype	2020-06-04 20:46:58 +00:00
John Ericson	2041499b5e	Flip boolean ... Thanks Matt!	2020-06-04 20:42:25 +00:00
John Ericson	ed86acf02a	Use some std::optional::has_value for clarity	2020-06-04 20:42:02 +00:00
John Ericson	574d5460f0	Make sure info.ca tag bit is set in nix add-to-store	2020-06-04 20:33:28 +00:00
Eelco Dolstra	ab54031e04	getDefaultUrl() -> value_or()	2020-06-04 20:24:28 +02:00
Eelco Dolstra	810b2c6a48	nix flake init: Add a '--template' flag ... The initial contents of the flake is specified by the 'templates.<name>' or 'defaultTemplate' output of another flake. E.g. outputs = { self }: { templates = { nixos-container = { path = ./nixos-container; description = "An example of a NixOS container"; }; }; }; allows $ nix flake init -t templates#nixos-container Also add a command 'nix flake new', which is identical to 'nix flake init' except that it initializes a specified directory rather than the current directory.	2020-06-04 20:22:25 +02:00
Ben Burdette	94427ffee3	add some comments	2020-06-04 11:53:19 -06:00
Matthew Bauer	2299ef705c	Add error message when FileIngestionMethod is out of bounds ... bool coerces anything >0 to true, but in the future we may have other file ingestion methods. This shows a better error message when the “recursive” byte isn’t 1.	2020-06-04 11:32:39 -05:00
Eelco Dolstra	dc305500c3	Merge pull request #3660 from Kloenk/selfhost-gitlab ... add support for selfhosted gitlab/github	2020-06-04 14:59:33 +02:00
Eelco Dolstra	d746ef4a81	Disable eval cache with --impure ... Fixes $ nix build nixpkgs#zoom-us error: Package ‘zoom-us-5.0.399860.0429’ in /nix/store/m79v7h75b69fkk8d2qcwm555l3wq6fmv-source/pkgs/applications/networking/instant-messengers/zoom-us/default.nix:126 has an unfree license (‘unfree’), refusing to evaluate. $ nix build nixpkgs#zoom-us --impure error: cached failure of attribute 'legacyPackages.x86_64-linux.zoom-us.drvPath'	2020-06-04 14:53:51 +02:00
Finn Behrens	108debef6f	add support for selfhosted gitlab/github	2020-06-04 14:42:39 +02:00
Eelco Dolstra	f85606c431	Merge remote-tracking branch 'origin/master' into flakes	2020-06-04 13:16:28 +02:00
Eelco Dolstra	0f44b60e6d	Make 'nix dev-shell' a deprecated alias for 'nix develop'	2020-06-04 11:14:19 +02:00
Eelco Dolstra	61e3d598b6	Rename 'nix dev-shell' to 'nix develop' ... Fixes #3648.	2020-06-04 10:57:40 +02:00
John Ericson	53bc8ff152	No C++ designated initializers yet with Clang 7	2020-06-03 20:45:14 -04:00
Ben Burdette	4335ba999b	Merge remote-tracking branch 'upstream/master' into errors-phase-2	2020-06-03 17:00:00 -06:00
Ben Burdette	f97576c5d9	newline-as-prefix; no final newline in output.	2020-06-03 14:47:00 -06:00
Carlo Nucera	132d6f2c24	Clarify the description of StorePath construction	2020-06-03 16:08:32 -04:00
John Ericson	74b251b2f3	Don't anticipate multiple CA outputs for now	2020-06-03 18:53:04 +00:00
John Ericson	3a9e4c3262	Don't anticipate CA but not fixed outputs for now	2020-06-03 18:50:45 +00:00
John Ericson	2500403059	Use enum and predicates rather than bitfile for derivation type	2020-06-03 17:38:54 +00:00
John Ericson	6b7f4ec4ab	Merge remote-tracking branch 'upstream/master' into ca-drv	2020-06-03 16:36:06 +00:00
Eelco Dolstra	c20591ddc3	Merge remote-tracking branch 'origin/master' into flakes	2020-06-03 16:15:22 +02:00
Jonas Chevalier	6ee03b8444	libutils/hash: remove default encoding ... This will make it easier to reason about the hash encoding and switch to SRI everywhere where possible.	2020-06-03 13:49:51 +02:00
John Ericson	01572c2198	Missing #include <cassert> in lru-cache.hh (#3654) ... This was a latent bug that just appeared because of the tests that were added. Remember to wait for CI! :)	2020-06-03 10:15:22 +00:00
John Ericson	3c78ac348c	Merge remote-tracking branch 'obsidian/no-hash-type-unknown' into validPathInfo-ca-proper-datatype	2020-06-03 04:44:24 +00:00
John Ericson	fecff16a6e	Merge remote-tracking branch 'obsidian/missing-include-0' into validPathInfo-ca-proper-datatype	2020-06-02 23:23:30 +00:00
John Ericson	39ba87be9b	Missing #include <cassert> in lru-cache.hh ... This was a latent bug that just appeared because of the tests that were added. Remember to wait for CI! :)	2020-06-02 21:36:53 +00:00
John Ericson	406dbb7fce	outputHashAlgo can be blank so parse accordingly ... It is blank for SRI hashes.	2020-06-02 21:09:15 +00:00
John Ericson	1fcd3afc38	Fix hashes	2020-06-02 20:35:17 +00:00
Carlo Nucera	75d2581390	Typo	2020-06-02 16:21:18 -04:00
Carlo Nucera	78f137e931	Validate text version instead, throw Errors	2020-06-02 16:20:22 -04:00
Carlo Nucera	a5cdf1867e	Add assertions for SHA256 in fixed case	2020-06-02 16:13:08 -04:00
Carlo Nucera	fd2eb41e64	Move file-hash to content-address	2020-06-02 15:44:58 -04:00
Carlo Nucera	343c20a404	WIP Completed implementation	2020-06-02 15:23:21 -04:00
John Ericson	c664e68b87	Fix to-base --type handler to correctly set std::optional flag ... Now that we have a separate flag function, also describe why it is optional.	2020-06-02 18:25:32 +00:00
Carlo Nucera	390bf64858	WIP	2020-06-02 14:15:58 -04:00
John Ericson	c502119fd3	to-base supports parsing SRI hashes, so make type flag optional	2020-06-02 18:05:26 +00:00
John Ericson	a33270ce1d	Clean up ValidPathInfo::isContentAddressed with std::visit	2020-06-02 17:04:21 +00:00
John Ericson	25e61812f3	Apply suggestions from code review ... Co-authored-by: Matthew Bauer <mjbauer95@gmail.com>	2020-06-02 12:47:18 -04:00
John Ericson	d73dbc8e4c	Remove hashingWithUnknownAlgoExits ... A valid hash type must be provided now. The hash itself can still be invalid, but that doesn't cause an `abort()`.	2020-06-02 16:28:54 +00:00
John Ericson	64cffb804a	Merge remote-tracking branch 'upstream/master' into no-hash-type-unknown	2020-06-02 16:07:25 +00:00
John Ericson	450dcf2c1b	Remove HashType::Unknown ... Instead, `Hash` uses `std::optional<HashType>`. In the future, we may also make `Hash` itself require a known hash type, encoraging people to use `std::optional<Hash>` instead.	2020-06-02 15:52:13 +00:00
Carlo Nucera	a5d820a0a3	Change parseCa(Opt) to parseContentAddress(Opt)	2020-06-02 11:00:10 -04:00
Ben Burdette	156d4f8bc8	remove extra space in SysErrors	2020-06-02 08:45:37 -06:00
John Ericson	1b6461f671	Merge remote-tracking branch 'upstream/master' into validPathInfo-ca-proper-datatype	2020-06-02 14:31:18 +00:00
Ben Burdette	d82d230b40	elide the 'ErrorInfo' in logError and logWarning calls	2020-06-02 08:22:24 -06:00
Eelco Dolstra	bfa1acd85c	Merge pull request #3639 from obsidiansystems/do-fixme-store-removes ... Remove `addToStore` variant as requested by `FIXME`	2020-06-02 15:39:07 +02:00
Eelco Dolstra	c16fdda3a6	Merge branch 'lru-tests' of https://github.com/gilligan/nix	2020-06-02 12:07:48 +02:00
Eelco Dolstra	e9fee8e6a7	src/libutil/tests/lru-cache.cc: Check erase() ... Co-authored-by: James Lee <jbit@jbit.net>	2020-06-02 12:06:59 +02:00
John Ericson	efcd30da89	WIP	2020-06-02 00:37:43 +00:00
John Ericson	754c910953	WIP more progress	2020-06-01 19:26:40 -04:00
Carlo Nucera	da39092a39	WIP	2020-06-01 18:53:31 -04:00
Carlo Nucera	0e9438b6d3	Create new file-hash files	2020-06-01 17:32:40 -04:00
Carlo Nucera	0cb67ecbd3	Merge branch 'derivation-header-include-order' of github.com:Ericson2314/nix into validPathInfo-ca-proper-datatype	2020-06-01 17:13:11 -04:00
Carlo Nucera	f4b89e11a4	Merge branch 'no-stringly-typed-derivation-output' of github.com:Ericson2314/nix into validPathInfo-ca-proper-datatype	2020-06-01 17:12:50 -04:00
Matthew Kenigsberg	c254254a80	use Tree ctor	2020-06-01 12:32:17 -06:00
Matthew Kenigsberg	7680993506	Tree ctors	2020-06-01 09:01:37 -06:00
Matthew Kenigsberg	ff1320b850	fetchOrSubstituteTree improvements ... Caches tree in addition to lockedRef, and explicitly writes out the logic for different combinations of cached/uncached flakes and indirect/resolved/locked flakes. This eliminates uneccessary calls to lookupInFlakeCache, fetchTree, maybeLookupFlake, and flakeCache.push_back	2020-06-01 02:57:22 -06:00
Tobias Pflug	eca1ff7a9f	Add tests for lru-cache.hh	2020-05-31 01:05:05 +02:00
Nikola Knezevic	fb38459d6e	Ensure we restrict refspec interpretation while fetching ... As `git fetch` may chose to interpret refspec to it's liking, ensure that we only pass refs that begin with `refs/` as is, otherwise, prepend them with `refs/heads`. Otherwise, branches named `heads/foo` (I know it's bad, but it's allowed), would be fetched as `foo`, instead of `heads/foo`.	2020-05-30 12:33:38 +02:00
Nikola Knezevic	77007d4eab	Improve ref validity checking in fetchGit ... The previous regex was too strict and did not match what git was allowing. It could lead to `fetchGit` not accepting valid branch names, even though they exist in a repository (for example, branch names containing `/`, which are pretty standard, like `release/1.0` branches). The new regex defines what a branch name should **NOT** contain. It takes the definitions from `refs.c` in https://github.com/git/git and `git help check-ref-format` pages. This change also introduces a test for ref name validity checking, which compares the result from Nix with the result of `git check-ref-format --branch`.	2020-05-30 12:29:35 +02:00
Eelco Dolstra	89e0b3e2d6	Move substitution into Input::fetch() ... Closes #3520.	2020-05-30 01:16:53 +02:00
Eelco Dolstra	0e7f77a59a	Check revCount / lastModified input attributes if specified	2020-05-30 00:59:13 +02:00
Eelco Dolstra	950b46821f	Remove TreeInfo ... The attributes previously stored in TreeInfo (narHash, revCount, lastModified) are now stored in Input. This makes it less arbitrary what attributes are stored where. As a result, the lock file format has changed. An entry like "info": { "lastModified": 1585405475, "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE=" }, "locked": { "owner": "NixOS", "repo": "nixpkgs", "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be", "type": "github" }, is now stored as "locked": { "owner": "NixOS", "repo": "nixpkgs", "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be", "type": "github", "lastModified": 1585405475, "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE=" }, The 'Input' class is now a dumb set of attributes. All the fetcher implementations subclass InputScheme, not Input. This simplifies the API. Also, fix substitution of flake inputs. This was broken since lazy flake fetching started using fetchTree internally.	2020-05-30 00:44:11 +02:00
John Ericson	fac0c2d54a	Remove addToStore variant as requested by FIXME ... The idea is it's always more flexible to consumer a `Source` than a plain string, and it might even reduce memory consumption. I also looked at `addToStoreFromDump` with its `// FIXME: remove?`, but the worked needed for that is far more up for interpretation, so I punted for now.	2020-05-29 17:02:32 -04:00
Ben Burdette	734283d636	Merge remote-tracking branch 'upstream/master' into errors-phase-2	2020-05-29 09:51:37 -06:00
Eelco Dolstra	5633c0975b	Factor out GitHub / GitLab commonality	2020-05-29 14:23:32 +02:00
Finn Behrens	5256bc77ca	add gitlab libfetcher	2020-05-28 23:00:08 +02:00
Carlo Nucera	6dd471ebf6	Fixing the result of merge	2020-05-28 12:14:36 -04:00
Carlo Nucera	4f597fb901	Merge branch 'master' of github.com:NixOS/nix into enum-class	2020-05-28 10:58:22 -04:00
John Ericson	5b4cd84bc2	Merge remote-tracking branch 'me/more-rust-ffi' into no-stringly-typed-derivation-output	2020-05-28 10:35:53 -04:00
John Ericson	ef71caba29	Merge remote-tracking branch 'upstream/master' into more-rust-ffi	2020-05-28 10:31:46 -04:00
Eelco Dolstra	f60ce4fa20	Merge pull request #3631 from andir/libutil-config-tests ... Add unit tests for config.cc	2020-05-28 13:51:37 +02:00
Eelco Dolstra	de141fcb79	Merge pull request #3455 from Ericson2314/enum-FileIngestionMethod ... Replace some `bool recursive` with a new `FileIngestionMethod` enum	2020-05-28 13:50:06 +02:00
Eelco Dolstra	17ca997fc6	Merge remote-tracking branch 'origin/master' into flakes	2020-05-28 12:55:24 +02:00
Eelco Dolstra	c3eff22f46	Merge branch 'store-visited' of https://github.com/mkenigs/nix into flakes	2020-05-28 12:15:39 +02:00
Eelco Dolstra	6286272371	nixpkgsFlakeRef(): Use locked nixpkgs	2020-05-28 12:13:13 +02:00
John Ericson	0f96f45061	Use FileIngestionMethod for nix hash ... There was an enum there that matched in perfectly.	2020-05-27 23:50:11 -04:00
Matthew Bauer	c66441a646	Rename some variables named “recursive” to “method” ... This is much less confusing since recursive is no longer a boolean.	2020-05-27 13:21:26 -05:00
Matthew Bauer	7873fd175d	Don’t use FileIngestionMethod for StorePathsCommand ... This is a different recursive than used in makeFixedOutputPath.	2020-05-27 13:21:11 -05:00
Andreas Rammhold	fc137d2f00	config.hh: Add documentation ... Provides some general overview on the mechanics of Config/Setting and comments for the public methods of Config.	2020-05-27 17:47:18 +02:00
Andreas Rammhold	9df3d8ccd7	tests/config.cc: add tests for Config::applyConfig	2020-05-27 17:47:18 +02:00
Andreas Rammhold	e1b8c64c04	config.cc: extract parts of applyConfigFile into applyConfig ... This moves the actual parsing of configuration contents into applyConfig which applyConfigFile is then going to call. By changing this we can now test the configuration file parsing without actually create a file on disk.	2020-05-27 17:47:18 +02:00
Andreas Rammhold	93129cf1dd	Add unit tests for config.cc	2020-05-27 17:47:17 +02:00
Eelco Dolstra	66d3ac94c9	Merge pull request #3621 from gilligan/add-json-tests ... Add unit tests for "json.hh"	2020-05-27 11:08:12 +02:00
Eelco Dolstra	dae6a267a8	Merge pull request #3625 from gilligan/xml-writer-tests ... Add unit tests for xml-writer	2020-05-27 11:07:53 +02:00
Carlo Nucera	f3f520c14c	Change syntax for CI	2020-05-26 12:51:28 -04:00
Carlo Nucera	89a5ac9d3b	Merge remote-tracking branch 'john-ericson/more-rust-ffi' into no-stringly-typed-derivation-output	2020-05-26 12:31:26 -04:00
Carlo Nucera	d49e65ba9d	Merge remote-tracking branch 'john-ericson/enum-FileIngestionMethod' into no-stringly-typed-derivation-output	2020-05-26 12:30:48 -04:00
Carlo Nucera	0f3f901071	Merge remote-tracking branch 'origin/master' into more-rust-ffi	2020-05-26 11:46:42 -04:00
Carlo Nucera	c2f33edd1f	Update src/libutil/rust-ffi.hh ... Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>	2020-05-26 11:43:18 -04:00
Carlo Nucera	b90241ceb1	Change remaining bools with FileIngestionMethod	2020-05-26 11:32:41 -04:00
Carlo Nucera	6d73c10041	Merge remote-tracking branch 'origin/master' into enum-FileIngestionMethod	2020-05-26 11:14:08 -04:00
Lucas Franceschino	b2748c6e99	Make functionArgs primitive accept primops	2020-05-25 19:07:38 +02:00
Tobias Pflug	4b388e8431	Add unit tests for xml-writer	2020-05-25 18:34:55 +02:00
Tobias Pflug	c284700867	Add unit tests for "json.hh"	2020-05-25 11:57:45 +02:00
Tobias Pflug	ecc5c90dfc	Add unit tests for hashing functions	2020-05-25 11:50:41 +02:00
Matthew Kenigsberg	8d67794da1	handle circular flake dependencies in list-inputs	2020-05-21 17:06:11 -06:00
Ben Burdette	b7057fa627	remove error-demo from make; clean up comment	2020-05-21 16:04:18 -06:00
Ben Burdette	0e49de6a2b	position for stdin, string; (string) for trace; fix tests	2020-05-21 14:28:45 -06:00
Ben Burdette	6a420d672c	print LOC for stdin, string args	2020-05-20 22:18:26 -06:00
Ben Burdette	85ce455b85	get code lines from the nix file	2020-05-20 17:25:02 -06:00
Tobias Pflug	a73a820a5d	Add unit testes for url.cc ... This adds tests for - parseURL - percentDecode - decodeQuery	2020-05-20 16:37:35 +02:00
Eelco Dolstra	5ef64f05e6	Cleanup	2020-05-18 15:50:29 +02:00
Eelco Dolstra	0ed946aa61	Merge branch 'wait-for-builders' of https://github.com/serokell/nix	2020-05-18 13:48:45 +02:00
Matthew Kenigsberg	c4beded32e	rm includes	2020-05-16 11:19:41 -06:00
Matthew Kenigsberg	ba7d7ed2e3	Create bashInteractive InstallableFlake	2020-05-16 11:03:06 -06:00
Matthew Kenigsberg	0858793604	Call lockFlake once and store in _lockedFlake	2020-05-16 11:03:06 -06:00
Matthew Kenigsberg	04821bc171	use flake's nixpkgs to find bashInteractive	2020-05-16 11:03:06 -06:00
Matthew Kenigsberg	8fbc8540d3	use nixpkgs#bashInteractive for dev-shell	2020-05-16 11:03:06 -06:00
Ben Burdette	92123c6c79	Merge remote-tracking branch 'upstream/master' into errors-phase-2	2020-05-15 07:00:36 -06:00
Eelco Dolstra	5f64655ff4	Move registry-related commands from 'nix flake' to 'nix registry' ... This makes 'nix flake' less cluttered and more consistent (it's only subcommands that operator on a flake). Also, the registry is not inherently flake-related (e.g. fetchTree could also use it to remap inputs).	2020-05-15 14:38:10 +02:00
Ben Burdette	19694aa213	fix compile errors	2020-05-14 12:28:18 -06:00
Ben Burdette	4daccb279c	formatting	2020-05-14 10:28:17 -06:00
Alexander Bantyev	183dd28266	Don't lock a user while doing remote builds	2020-05-14 17:00:54 +03:00
Ben Burdette	ef9dd9f9bc	formatting and a few minor changes	2020-05-13 15:56:39 -06:00
Ben Burdette	c79d4addab	consistent capitalization	2020-05-13 10:02:18 -06:00
Ben Burdette	bfca5fc395	change status messages to info level	2020-05-13 09:52:36 -06:00
Eelco Dolstra	849d3968db	Update src/libfetchers/git.cc ... Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>	2020-05-13 10:41:21 +02:00
Ben Burdette	ecbb8e9c0a	no blank line if no LOC	2020-05-12 14:41:30 -06:00
Ben Burdette	960d4362ed	hint only	2020-05-12 13:54:18 -06:00
Ben Burdette	72ecccee57	convert to logWarning format	2020-05-12 12:19:34 -06:00
Ben Burdette	19cffc29c9	remove unused extra json fields	2020-05-12 12:09:12 -06:00
Ben Burdette	2a19bf8619	move pos to the first arg, to indicate its not used in a fmt template	2020-05-12 11:27:37 -06:00
Ben Burdette	ec870b9c85	new pos format for more errors	2020-05-12 10:52:26 -06:00
Eelco Dolstra	215f09d765	Merge pull request #3587 from NixOS/bash-completion ... Generic shell completion support for the 'nix' command	2020-05-12 18:26:13 +02:00
Eelco Dolstra	ebc024df22	Show hint how to enable experimental features	2020-05-12 15:47:09 +02:00
Eelco Dolstra	268ecf5b3f	nix: Don't require --experimental-features=nix-command for some subcommands	2020-05-12 15:47:09 +02:00
Eelco Dolstra	b8b2dbf272	Fix InstallableCommand	2020-05-12 11:53:32 +02:00
Eelco Dolstra	437614b479	Fix macOS build ... macOS doesn't have GLOB_ONLYDIR.	2020-05-12 11:09:09 +02:00
Ben Burdette	7c3138844c	more pos reporting	2020-05-11 17:34:57 -06:00
Ben Burdette	631642c5b4	new format for pos	2020-05-11 16:58:08 -06:00
Ben Burdette	b93c1bf3d6	fixes to merged code	2020-05-11 15:52:15 -06:00
Ben Burdette	59b1f5c701	Merge branch 'master' into errors-phase-2	2020-05-11 14:35:30 -06:00
Eelco Dolstra	649c2db308	nix flake: Add completion support	2020-05-11 22:10:33 +02:00
Eelco Dolstra	27d34ef770	When completing flakerefs, only return directories	2020-05-11 22:04:13 +02:00
Ben Burdette	536bbf53e1	comments and cleanup	2020-05-11 13:58:38 -06:00
Eelco Dolstra	259ff74bde	Add completion for installables ... This completes flakerefs using the registry (e.g. 'nix<TAB>' => 'nix nixpkgs') and flake output attributes by evaluating the flake (e.g. 'dwarffs#nix<TAB>' => 'dwarffs#nixosModules').	2020-05-11 21:49:02 +02:00
Eelco Dolstra	e917332d63	Shut up warnings while running completers	2020-05-11 21:38:17 +02:00
Ben Burdette	958e81987b	switch from printError warnings to logWarnings	2020-05-11 13:02:16 -06:00
Domen Kožar	5bdb67c843	Merge pull request #3568 from kolloch/outputHashModeError ... libstore/build.cc: more explicit error about form of output	2020-05-11 18:14:32 +02:00
Domen Kožar	1d8144e36b	Update src/libstore/build.cc	2020-05-11 18:14:23 +02:00
Eelco Dolstra	4c3c638a05	Cleanup	2020-05-11 15:57:45 +02:00
Eelco Dolstra	0884f180f5	Simplify	2020-05-10 21:50:32 +02:00
Eelco Dolstra	e0c19ee620	Add completion for paths	2020-05-10 21:35:07 +02:00
Eelco Dolstra	91ddee6bf0	nix: Implement basic bash completion	2020-05-10 20:32:21 +02:00
Matthew Kenigsberg	73ee1afffe	Reorder to build ... This reverts commit 883948d7a0add742ccae58e9845d769a8064371c.	2020-05-09 14:42:32 -06:00
Matthew Kenigsberg	9f4cfbb2e7	Refactor installables ... InstallableValue has children InstallableFlake and InstallableAttrPath, but InstallableFlake was overriding toDerivations, and usage was changed so that InstallableFlake didn't need cmd. So these changes were made: InstallableValue::toDerivations() -> InstalllableAttrPath::toDerivations() InstallableValue::cmd -> InstallableAttrPath::cmd InstallableValue uses state instead of cmd toBuildables() and toDerivations() were made abstract	2020-05-09 14:42:32 -06:00
Matthew Kenigsberg	bf81dd40e9	InstallableExpr unused	2020-05-09 10:16:00 -06:00
Benjamin Hipple	146f9c114f	doc: consistently refer to 'fixed-output' with a dash ... General cleanup that makes it easier to search for the term.	2020-05-09 10:58:43 -04:00
Ben Burdette	55eb717148	add pos to errorinfo, remove from hints	2020-05-08 18:18:28 -06:00
Eelco Dolstra	d3d8186c9c	Merge pull request #3571 from gilligan/nix-unit-testing ... Add unit tests	2020-05-08 17:02:25 +02:00
Tobias Pflug	181a47d884	Enable toLower umlauts test ... Update comment and enable the test	2020-05-08 15:13:55 +02:00
Tobias Pflug	2191141274	Enable baseNameOf test ... Add note about removal of trailing slashes in the doc comment of baseNameOf and enabled the test.	2020-05-08 15:07:40 +02:00
Tobias Pflug	e3df9c2a6e	Enable dirOf test ... Adjusted the doc comment for `dirOf` to reflect the implementation behavior.	2020-05-08 15:03:44 +02:00
Eelco Dolstra	5b8883faac	configure: Look for gtest	2020-05-08 12:09:37 +02:00
Eelco Dolstra	ca657525b8	Don't install unit tests	2020-05-08 12:03:27 +02:00
Eelco Dolstra	7898cdb75a	make check: Run unit tests	2020-05-08 11:49:40 +02:00
Eelco Dolstra	72b9d971bc	Fix warning	2020-05-08 11:35:57 +02:00
Eelco Dolstra	7cc7cef950	Move unit tests to sr/libutil/tests, use mk make rules	2020-05-08 11:34:09 +02:00
Alexander Bantyev	772e5db828	Mention build users in the 'waiting for' message	2020-05-08 12:29:00 +03:00
Alexander Bantyev	14073fb76b	Don't block while waiting for build users	2020-05-08 12:22:39 +03:00
Ben Burdette	1b801cec40	pretending to be const	2020-05-07 16:43:36 -06:00
Tobias Pflug	1f3602a2c9	Remove replaceInSet ... The function isn't being used anywhere so it seems safe to remove	2020-05-07 18:15:13 +02:00
Eelco Dolstra	14a3a62bfc	Update src/nix/search.cc ... Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>	2020-05-07 12:13:16 +02:00
Ben Burdette	e3901638b5	todo removal	2020-05-06 15:01:13 -06:00
Ben Burdette	e76ad2e48a	implement SysError errno handling	2020-05-06 14:07:20 -06:00
Eelco Dolstra	1ad71bc62c	Remove support for old lockfiles and the epoch/uri attributes	2020-05-06 17:48:18 +02:00
Eelco Dolstra	b0e9b07e80	Remove obsolete FIXME	2020-05-06 17:40:01 +02:00
Eelco Dolstra	ff394ff206	Remove the nixpkgs.<attr> compatibility hack ... Since we've changed a lot of things in the 'nix' command (e.g. rename 'nix run') there is not much point in keeping this around.	2020-05-06 17:20:23 +02:00
Domen Kožar	672985531c	Merge pull request #3570 from Mic92/nix-search ... nix/search: no error for empty search results if json is enabled	2020-05-06 16:14:58 +02:00
Jörg Thalheim	85c1932c94	nix/search: no error for empty search results if json is enabled ... - result list will be always empty if --json is passed - for scripts an empty search result is not really an error, we rather want to distinguish between evaluation errors and empty results	2020-05-06 14:43:54 +01:00
Eelco Dolstra	2f8ee4578f	Merge remote-tracking branch 'origin/master' into flakes	2020-05-06 12:01:40 +02:00
Eelco Dolstra	74a1bfdcab	Merge pull request #3546 from guibou/nix_readfile_on_0_sized_files ... builtins.readFile: do not truncate content	2020-05-06 11:33:55 +02:00
Eelco Dolstra	272c4ba36d	Merge pull request #3557 from Ma27/nix-repl-help ... Improve help-message for nix-repl	2020-05-06 11:25:47 +02:00
Peter Kolloch	9be46859a9	libstore/build.cc: more explicit about form of output ... Be more explicit about why we expect a regular file as output when outputHashMode=flat for a fixed output derivation.	2020-05-06 11:21:12 +02:00
Eelco Dolstra	02c5914ea4	Merge pull request #3562 from pikajude/master ... Use fragment size for autoGC capacity calculation	2020-05-06 11:19:30 +02:00
Eelco Dolstra	fd4911269f	Revert "Merge pull request #3558 from LnL7/ssh-ng-stderr" ... This reverts commit 3ebfbecdd1, reversing changes made to c089c52d5f. https://github.com/NixOS/nix/pull/3558	2020-05-06 10:54:18 +02:00
Eelco Dolstra	6f3244ce45	Merge remote-tracking branch 'origin/master' into flakes	2020-05-05 18:59:33 +02:00
Eelco Dolstra	909b4a8820	nix doctor: Consistency	2020-05-05 15:27:47 +02:00
Eelco Dolstra	f132d82a79	nix --help: Group commands	2020-05-05 15:18:23 +02:00
Alexander Bantyev	04967dee9d	Wait for build users when none are available	2020-05-05 13:04:36 +03:00
Ben Burdette	7ffb5efdbc	appending to hints; remove _printError	2020-05-04 16:19:57 -06:00
Ben Burdette	f30de61578	add normaltxt, yellowify->yellowtxt	2020-05-04 16:19:20 -06:00
Jude Taylor	e2fc575c61	nix auto-gc: use fragment size	2020-05-04 14:42:06 -07:00
Ben Burdette	8c8f2b74ec	log as warning	2020-05-04 14:44:42 -06:00
Ben Burdette	afaa541013	affinity operator<<	2020-05-04 14:44:00 -06:00
Eelco Dolstra	a721a0b114	Flag: Use designated initializers	2020-05-04 22:40:19 +02:00
Ben Burdette	9c5ece44a7	separate msgs instead of appending to what()	2020-05-04 13:46:15 -06:00
Eelco Dolstra	3ebfbecdd1	Merge pull request #3558 from LnL7/ssh-ng-stderr ... remote-store: don't log raw stderr by default	2020-05-04 13:02:33 +02:00
Ben Burdette	ab6f0b9641	convert some printError calls to logError	2020-05-03 08:01:25 -06:00
Daiderd Jordan	4769eea5e2	logging: handle build log lines in simple logger ... The raw stderr output isn't logged anymore so the build logs need to be printed by the default logger in order for the old commands like nix-build to still show build output.	2020-05-02 23:40:53 +02:00
Daiderd Jordan	f16e24f95e	remote-store: don't log raw stderr by default ... For remote stores the log messages are already forwarded as structured STDERR_RESULT messages so the old format is duplicate information. But still included with -vvv since it could be useful for debugging problems. $ nix build -L /nix/store/nl71b2niws857ffiaggyrkjwgx9jjzc0-foo.drv --store ssh-ng://localhost Hello World! foo> Hello World! [1/0/1 built] building foo Fixes #3556	2020-05-02 23:40:50 +02:00
Maximilian Bosch	2aeb874e83	Improve help-message for nix-repl ... * Remove obsolete `printHelp` function * Add an example to demonstrate how to list all available commands within the REPL	2020-05-01 23:32:01 +02:00
Ben Burdette	4b99c09f5c	convert some errors	2020-05-01 14:32:06 -06:00
Eelco Dolstra	c089c52d5f	Fix build	2020-05-01 12:42:39 +02:00
Matthew Kenigsberg	d6b4047c2f	rename nix run to nix shell and nix app to nix run ... (cherry picked from commit 5d8504b978)	2020-05-01 12:17:36 +02:00
Eelco Dolstra	941f95284a	Merge remote-tracking branch 'origin/master' into flakes	2020-05-01 11:59:56 +02:00
Ben Burdette	a3030e3c31	fix error calls	2020-04-30 17:56:26 -06:00
Ben Burdette	f5d3215c87	logError	2020-04-30 16:31:47 -06:00
Ben Burdette	171b4ce85c	typo	2020-04-30 09:57:01 -06:00
Eelco Dolstra	0135fd6ec4	nix dev-shell: Unset shellHook ... This avoids inheriting the caller's shellHook, which can happen when running a dev-shell inside a dev-shell.	2020-04-30 14:47:56 +02:00
Eelco Dolstra	efe6c186ea	nix dev-shell: Support structured attrs ... Tested against https://github.com/NixOS/nixpkgs/pull/72074. Fixes #3540.	2020-04-30 14:47:47 +02:00
Eelco Dolstra	2fcfc6c2c6	nix dev-shell: Refactor script for getting the environment	2020-04-30 13:05:29 +02:00
Domen Kožar	df8e9d691c	Merge pull request #3548 from Ma27/fetchtarball-pos ... Fix displaying error-position in `builtins.fetch{Tree,Tarball}`	2020-04-30 10:33:12 +02:00
Ben Burdette	39ff80d031	errorinfo constructor test	2020-04-29 18:57:05 -06:00
Matthew Kenigsberg	5d8504b978	rename nix run to nix shell and nix app to nix run	2020-04-29 15:45:10 -06:00
Maximilian Bosch	d1229859c2	Fix displaying error-position in builtins.fetch{Tree,Tarball} ... Without dereferencing this pointer, you'd get an error like this: ``` error: unsupported argument 'abc' to 'fetchTarball', at 0x13627e8 ```	2020-04-29 22:53:39 +02:00
Guillaume Bouchard	2e5be2a749	StringSink pre allocate ... When used with `readFile`, we have a pretty good heuristic of the file size, so `reserve` this in the `string`. This will save some allocation / copy when the string is growing.	2020-04-29 18:44:01 +02:00
Guillaume Bouchard	7afcb5af98	Remove the drain argument from readFile ... Now it is always `drain` (see previous commit).	2020-04-29 18:43:45 +02:00
Ben Burdette	e2f61263eb	uncrustify formatting	2020-04-29 10:14:32 -06:00
Eelco Dolstra	9570036146	nix copy: Build derivations ... Fixes $ nix copy .#hydraJobs.vendoredCrates --to /tmp/nix error: path '/nix/store/...' is not valid	2020-04-29 15:51:45 +02:00
Eelco Dolstra	9c4e05766b	nix copy: Move --from / --to check ... This means you now get an error message *before* stuff gets built: $ nix copy .#hydraJobs.vendoredCrates error: you must pass '--from' and/or '--to' Try 'nix --help' for more information.	2020-04-29 15:50:59 +02:00
Antoine Eiche	ca93b26db6	Only call grantpt on MacOS systems ... The commit 3cc1125595 adds a `grantpt` call on the builder pseudo terminal fd. This call is actually only required for MacOS, but it however requires a RW access to /dev/pts which is only RO bindmounted in the Bazel Linux sandbox. So, Nix can not be actually run in the Bazel Linux sandbox for unneeded reasons.	2020-04-29 15:43:20 +02:00
Eelco Dolstra	70bcd6a55c	Evaluation cache: Don't barf in read-only mode ... Fixes $ nix copy warning: Git tree '/home/eelco/Dev/nix-flake' is dirty nix: src/nix/installables.cc:348: std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, nix::FlakeRef, nix::InstallableValue::DerivationInfo> nix::InstallableFlake::toDerivation(): Assertion `state->store->isValidPath(drvPath)' failed. Aborted (core dumped)	2020-04-29 15:42:53 +02:00
Guillaume Bouchard	5a34a473dd	builtins.readFile: do not truncate content ... This closes #3026 by allowing `builtins.readFile` to read a file with a wrongly reported file size, for example, files in `/proc` may report a file size of 0. Reading file in `/proc` is not a good enough motivation, however I do think it just makes nix more robust by allowing more file to be read. Especially, I do considerer the previous behavior to be dangerous because nix was previously reading truncated files. Examples of file system which incorrectly report file size may be network file system or dynamic file system (for performance reason, a dynamic file system such as FUSE may generate the content of the file on demand). ``` nix-repl> builtins.readFile "/proc/version" "" ``` With this commit: ``` nix-repl> builtins.readFile "/proc/version" "Linux version 5.6.7 (nixbld@localhost) (gcc version 9.3.0 (GCC)) #1-NixOS SMP Thu Apr 23 08:38:27 UTC 2020\n" ``` Here is a summary of the behavior changes: - If the reported size is smaller, previous implementation was silently returning a truncated file content. The new implementation is returning the correct file content. - If a file had a bigger reported file size, previous implementation was failing with an exception, but the new implementation is returning the correct file content. This change of behavior is coherent with this pull request. Open questions - The behavior is unchanged for correctly reported file size, however performances may vary because it uses the more complex sink interface. Considering that sink is used a lot, I don't think this impacts the performance a lot. - `builtins.readFile` on an infinite file, such as `/dev/random` may fill the memory. - it does not support adding file to store, such as `${/proc/version}`.	2020-04-29 14:50:52 +02:00
Eelco Dolstra	5ada0831cf	Merge remote-tracking branch 'origin/master' into flakes	2020-04-29 13:32:27 +02:00
Ben Burdette	22e6490311	Error classname as name	2020-04-28 21:06:08 -06:00
Eelco Dolstra	06849c3090	Merge pull request #3542 from mkenigs/gcroots ... Set GCROOT to store path to prevent garbage collection	2020-04-28 21:04:06 +02:00
Matthew Kenigsberg	6d40fe573c	rename to NIX_GCROOT	2020-04-28 11:18:54 -06:00
Eelco Dolstra	52a3ca823d	Tweak warning message	2020-04-28 17:56:01 +02:00
Eelco Dolstra	6a8cba83bb	Merge branch 'nix-env-warn-unmatched' of https://github.com/lheckemann/nix	2020-04-28 17:45:25 +02:00
Ben Burdette	e51a757720	astyle format	2020-04-27 15:15:08 -06:00
Eelco Dolstra	6521c92ce8	Improve path:// handling ... In particular, doing 'nix build /path/to/dir' now works if /path/to/dir is not a Git tree (it only has to contain a flake.nix file). Also, 'nix flake init' no longer requires a Git tree (but it will do a 'git add flake.nix' if it's a Git tree)	2020-04-27 22:53:11 +02:00
Eelco Dolstra	829dcb35d5	flake-template.nix: Add defaultPackage	2020-04-27 22:52:49 +02:00
Alyssa Ross	c05e20daa1	Fix long paths permanently breaking GC ... Suppose I have a path /nix/store/[hash]-[name]/a/a/a/a/a/[...]/a, long enough that everything after "/nix/store/" is longer than 4096 (MAX_PATH) bytes. Nix will happily allow such a path to be inserted into the store, because it doesn't look at all the nested structure. It just cares about the /nix/store/[hash]-[name] part. But, when the path is deleted, we encounter a problem. Nix will move the path to /nix/store/trash, but then when it's trying to recursively delete the trash directory, it will at some point try to unlink /nix/store/trash/[hash]-[name]/a/a/a/a/a/[...]/a. This will fail, because the path is too long. After this has failed, any store deletion operation will never work again, because Nix needs to delete the trash directory before recreating it to move new things to it. (I assume this is because otherwise a path being deleted could already exist in the trash, and then moving it would fail.) This means that if I can trick somebody into just fetching a tarball containing a path of the right length, they won't be able to delete store paths or garbage collect ever again, until the offending path is manually removed from /nix/store/trash. (And even fixing this manually is quite difficult if you don't understand the issue, because the absolute path that Nix says it failed to remove is also too long for rm(1).) This patch fixes the issue by making Nix's recursive delete operation use unlinkat(2). This function takes a relative path and a directory file descriptor. We ensure that the relative path is always just the name of the directory entry, and therefore its length will never exceed 255 bytes. This means that it will never even come close to AX_PATH, and Nix will therefore be able to handle removing arbitrarily deep directory hierachies. Since the directory file descriptor is used for recursion after being used in readDirectory, I made a variant of readDirectory that takes an already open directory stream, to avoid the directory being opened multiple times. As we have seen from this issue, the less we have to interact with paths, the better, and so it's good to reuse file descriptors where possible. I left _deletePath as succeeding even if the parent directory doesn't exist, even though that feels wrong to me, because without that early return, the linux-sandbox test failed. Reported-by: Alyssa Ross <hi@alyssa.is> Thanks-to: Puck Meerburg <puck@puckipedia.com> Tested-by: Puck Meerburg <puck@puckipedia.com> Reviewed-by: Puck Meerburg <puck@puckipedia.com>	2020-04-27 20:50:17 +00:00
Matthew Kenigsberg	9e95b95a5d	comment	2020-04-27 13:18:26 -06:00
Matthew Kenigsberg	a3bc695e7d	Set GCROOT to store path to prevent garbage collection	2020-04-27 11:22:20 -06:00
Eelco Dolstra	b51dff431c	Improve error message when an argument is not a flake	2020-04-27 18:55:20 +02:00
Eelco Dolstra	b4e23dcd9e	nix search: Search legacyPackages recursively	2020-04-27 16:29:26 +02:00
Ben Burdette	1ff42722ce	error.hh	2020-04-26 14:47:41 -06:00
Ben Burdette	d4fd7b543e	print dashes instead of empty name string	2020-04-25 12:05:26 -06:00