Commit graph

6084 commits

Author SHA1 Message Date
Carlo Nucera ea48e3a5b5 Abstract common parsing functionality 2020-07-02 11:29:33 -04:00
Carlo Nucera 36cbc74689 Inline and simplify in parseAnyPrefixed 2020-07-02 11:21:00 -04:00
Matthew Bauer fc2ab42e86 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-07-02 11:14:04 -04:00
Matthew Bauer d2e8b9ff0e Store subPath in SubstitutionGoal 2020-07-02 11:12:05 -04:00
Carlo Nucera 9462d8a50b Rename fromSRI to parseSRI for constistency 2020-07-02 11:11:18 -04:00
Carlo Nucera f61bc45d19 Get rid of the std::pair 2020-07-02 11:09:04 -04:00
Ben Burdette bf2788e4c1 move showTrace to new loggerSettings 2020-07-02 09:04:31 -06:00
Carlo Nucera 27c8029573 Inline newFunction 2020-07-02 11:01:10 -04:00
Matthew Bauer 1f9cb06db2 Try next when no ca exists and have different store dirs 2020-07-02 10:59:24 -04:00
Carlo Nucera 343d1569b1 Fix test suite 2020-07-02 10:48:47 -04:00
Ben Burdette 5ae498872a assert for invalid fileorigin 2020-07-02 07:14:40 -06:00
Carlo Nucera 263ccdd489 Rename two hash constructors to proper functions 2020-07-01 18:34:18 -04:00
Carlo Nucera c8c4bcf90e Inline Hash::init() 2020-07-01 18:03:35 -04:00
Carlo Nucera d63a5ded76 Remove unused import 2020-07-01 17:53:24 -04:00
Carlo Nucera 6faeec3b2a Keep the previous name, for diffing 2020-07-01 17:50:34 -04:00
Carlo Nucera 274a8136fb Correct FIXMEs in libfetchers 2020-07-01 17:47:15 -04:00
Carlo Nucera c2e7f7a712 Fixed build, we still have test errors 2020-07-01 17:32:06 -04:00
Ben Burdette 8497891b99 spacing 2020-07-01 13:50:18 -06:00
Eelco Dolstra 6ff9aa8df7 Don't process an option if any of its arguments need completion 2020-07-01 20:31:39 +02:00
Eelco Dolstra d746503e5c Add --inputs-from to use flake inputs as registry entries
This allows you to refer to an input from another flake. For example,

  $ nix run --inputs-from /path/to/hydra nixpkgs#hello

runs 'hello' from the 'nixpkgs' inputs of the 'hydra' flake.

Fixes #3769.
2020-07-01 20:25:13 +02:00
Ben Burdette a295b2ea96 if no errLoc, no Loc. 2020-07-01 12:02:02 -06:00
Ben Burdette 3629b0585a don't include errpos for addErrorContext 2020-07-01 11:49:01 -06:00
Ben Burdette 2a39c083dc non-pos trace test 2020-07-01 10:37:31 -06:00
Eelco Dolstra 38ccf2e241 Cleanup 2020-07-01 15:31:34 +02:00
Eelco Dolstra 86a4aba6c4 Merge branch 'remote-query-outputs' of https://github.com/tweag/nix 2020-07-01 15:10:29 +02:00
Eelco Dolstra 7d554f295c Support building flakes from a shallow Git repo
Fixes #3756.
2020-07-01 14:57:59 +02:00
Ben Burdette a7d5d26443 fix tests with the 'from string' change 2020-06-30 22:05:21 -06:00
Ben Burdette 9159dfe3d8 comments and cleanup 2020-06-30 16:31:55 -06:00
Ben Burdette 70bcb39d3f double addtrace for 'called from' 2020-06-30 15:44:19 -06:00
Matthew Bauer 09833126f3 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-30 16:35:08 -04:00
Ben Burdette ddb81ca126 Merge branch 'master' into add-trace 2020-06-30 12:21:45 -06:00
Carlo Nucera b798efb829 WIP initial design 2020-06-30 14:10:30 -04:00
Ben Burdette a0705e0dd1 invalid pos check 2020-06-30 11:01:46 -06:00
Ben Burdette e72a16a339 check for a null symbol 2020-06-30 11:00:51 -06:00
Carlo Nucera a1f66d1d9e Factor the prefix splitting in hash 2020-06-30 12:49:00 -04:00
Carlo Nucera 77b51f4598 Factor the prefix splitting in content-address 2020-06-30 11:57:46 -04:00
Carlo Nucera 7ba0fae0dd Create the spitPrefix function in parser.hh 2020-06-30 11:57:09 -04:00
Ben Burdette c484a67914 trace formatting 2020-06-29 15:46:21 -06:00
John Ericson 24da034bc3 Add possibly missing <string> include 2020-06-29 21:22:00 +00:00
Eelco Dolstra 2b834d48aa NAR parser: Fix missing name field check
Discovered by @Kloenk.
2020-06-29 22:45:41 +02:00
Matthew Bauer 289558dffb Add unordered_set to globals.cc header 2020-06-29 20:34:45 +00:00
Matthew Bauer da77331cb7 Remove lazy lookup in getHome
this seems to break in Musl/Static with:

terminate called after throwing an instance of 'std::bad_function_call'
  what():  bad_function_call
2020-06-29 20:34:26 +00:00
Matthew Bauer 88cf6ffce3 Rename logging->stdout to logging->stdout_
musl doesn't like this identifier
2020-06-29 20:34:26 +00:00
John Ericson a83566e5bc Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors 2020-06-29 18:40:34 +00:00
John Ericson 987a4a0be9 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-06-29 18:03:21 +00:00
Eelco Dolstra 26cf0c674f nix run: Use packages/legacyPackages as fallback if there is no app definition
'nix run' will try to run $out/bin/<name>, where <name> is the
derivation name (excluding the version). This often works well:

  $ nix run nixpkgs#hello
  Hello, world!

  $ nix run nix -- --version
  nix (Nix) 2.4pre20200626_adf2fbb

  $ nix run patchelf -- --version
  patchelf 0.11.20200623.e61654b

  $ nix run nixpkgs#firefox -- --version
  Mozilla Firefox 77.0.1

  $ nix run nixpkgs#gimp -- --version
  GNU Image Manipulation Program version 2.10.14

though not always:

  $ nix run nixpkgs#git
  error: unable to execute '/nix/store/kp7wp760l4gryq9s36x481b2x4rfklcy-git-2.25.4/bin/git-minimal': No such file or directory
2020-06-29 19:08:50 +02:00
Eelco Dolstra 50f13b06fb EvalCache: Store string contexts 2020-06-29 19:08:37 +02:00
Ben Burdette 8f81fae116 showTrace flag in loggers 2020-06-29 10:20:51 -06:00
Eelco Dolstra b681408879 Factor out EvalCache::forceDerivation() 2020-06-29 16:39:41 +02:00
Eelco Dolstra ca946860ce Fix bash completion 2020-06-29 14:37:22 +02:00
Eelco Dolstra bc03c6f23d Move App 2020-06-29 14:14:23 +02:00
Eelco Dolstra 58bc3b6578
Merge pull request #3729 from obsidiansystems/simpler-hased-mirror
hashed-mirrors: Use parsed derivation output rather than reconstructing it
2020-06-29 14:04:12 +02:00
Eelco Dolstra 64232f3ea6
Merge pull request #3749 from rodarima/master
Fall back to copyPath if link fails with EPERM
2020-06-29 13:31:24 +02:00
Ben Burdette ef24a0835d showtrace as function arg 2020-06-27 12:19:31 -06:00
Carlo Nucera 278a679782 Merge branch 'master' of github.com:NixOS/nix into allow-relative-paths-in-store-option 2020-06-26 10:40:13 -04:00
Eelco Dolstra adf2fbbdc2 Merge remote-tracking branch 'origin/master' into flakes 2020-06-26 08:46:46 +02:00
John Ericson d92d4f85a5 Move ValidPathInfo to its own header 2020-06-25 17:48:49 +00:00
Ben Burdette bc9e87412c 'string' makes more sense in nix repl 2020-06-25 09:56:32 -06:00
Ben Burdette 9ab808c926 showTrace flag for ErrorInfo; showTrace test. 2020-06-25 09:23:12 -06:00
Eelco Dolstra de2641ae99 Fix empty std::optional dereference in writeDerivation()
https://hydra.nixos.org/build/123017579
2020-06-25 15:50:30 +02:00
Rodrigo 3a642187c3 Fall back to copyPath if link fails with EPERM
BeeGFS doesn't allow hard-links and returns EPERM, so we fall back
to copyPath. See https://github.com/NixOS/nix/issues/3748
2020-06-25 12:03:26 +02:00
Ben Burdette 9c0e1fd4f1 add trace test; error formatting refinements 2020-06-24 18:31:28 -06:00
Ben Burdette 6359d71d6b re-enable --show-trace check 2020-06-24 18:28:20 -06:00
Ben Burdette 023912def3 convenience form of addTrace 2020-06-24 13:46:25 -06:00
Ben Burdette 93e9307329 repl indenting 2020-06-24 13:14:49 -06:00
Ben Burdette b18ed02b76 repl indenting 2020-06-24 13:10:41 -06:00
regnat d38f860c3e Add a way to get all the outputs of a derivation with their label
Generalize `queryDerivationOutputNames` and `queryDerivationOutputs` by
adding a `queryDerivationOutputMap` that returns the map
`outputName=>outputPath`

(not that this is not equivalent to merging the results of
`queryDerivationOutputs` and `queryDerivationOutputNames` as sets don't
preserve the order, so we would end up with an incorrect mapping).

squash! Add a way to get all the outputs of a derivation with their label

Rename StorePathMap to OutputPathMap
2020-06-24 20:38:40 +02:00
Ben Burdette 6fe660acf9 re-remove 2020-06-24 12:33:05 -06:00
Matthew Bauer bcf8582091 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-24 12:52:09 -04:00
John Ericson d9fb9e9f26 Merge remote-tracking branch 'upstream/master' into ca-drv 2020-06-24 15:41:29 +00:00
Ben Burdette 00fe653ea5 nixCode -> LinesOfCode 2020-06-24 08:33:53 -06:00
Ben Burdette 1d43a6e123 use plain errPos instead of nixCode; fix tests 2020-06-23 15:30:13 -06:00
Carlo Nucera 8d51d38e4c Fix test suite 2020-06-23 17:16:20 -04:00
John Ericson ec3a857307 Fix and clean up hash parser 2020-06-23 18:19:40 +00:00
John Ericson 98e5d1af03 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-06-23 17:03:37 +00:00
Ben Burdette d0e78fbb03 re-add Pos origin in tests 2020-06-23 10:51:58 -06:00
Ben Burdette abe0552504 Merge remote-tracking branch 'upstream/master' into add-trace 2020-06-23 09:40:28 -06:00
Ben Burdette 13e87535ff traces to bottom 2020-06-23 09:36:58 -06:00
Carlo Nucera e197bc6229 Enable the --store option to take relative paths
In nix commands which accept --store options, we can now specify a
relative path, which will be canonicalized.
2020-06-23 11:13:18 -04:00
Eelco Dolstra 09fc06daab nix flake init: Use git add --force 2020-06-23 16:25:32 +02:00
Eelco Dolstra 015e1c2131
Merge pull request #3724 from bburdette/hintfmt-percent
Hintfmt percent test, and fix
2020-06-23 12:26:00 +02:00
John Ericson 55d4bd6e0e Improve content address parsing
- Ensure hash is in form <algo>-<prefix> and not SRI.

 - Better errors if something goes wrong

 - string_view for no coppying
2020-06-22 18:08:27 +00:00
Ben Burdette 9d1cb0c5e6 with normaltxt, elide yellow color code instead of canceling it; use normaltxt on plain_string hintfmt 2020-06-22 11:32:20 -06:00
Matthew Bauer 66a62b3189 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-22 13:08:11 -04:00
Ben Burdette 28b079067f
Update src/libutil/fmt.hh
Co-authored-by: John Ericson <git@JohnEricson.me>
2020-06-22 10:00:37 -06:00
John Ericson f4a5913125 hashed-mirrors: Use parsed derivation output rather than reconstructing it
Now the derivation outputs are parsed up front, we can avoid a reparse
by doing it. Also, this just feels a bit better as the `output*` env
vars are more of a `libnixexpr` interface than `libnixstore` interface:
ultimately, it's the derivation outputs that decide whether the
derivation is fixed-output.

Yes, hashed mirrors might go away with #3689, but this bit of code would
be moved rather than deleted, so it's worth doing a cleanup anyways I
think.
2020-06-22 15:17:20 +00:00
Eelco Dolstra 334e26bfc2 nix flake check: Don't build apps
This was inconsistent since we're not building 'packages' or
'defaultPackage' either.

Closes #3726.
2020-06-22 11:31:07 +02:00
John Ericson 93bbe6e8ab Merge remote-tracking branch 'upstream/master' into ca-drv 2020-06-21 21:11:58 +00:00
John Ericson 3804e3df9b Don't anticipate hash algo without hash in derivation for now
When we merge with master, the new lack of string types make this case
impossible (after parsing). Later, when we actually implemenent
CA-derivations, we'll change the types to allow that.
2020-06-21 21:05:37 +00:00
John Ericson 8313f0e939 Merge remote-tracking branch 'upstream/master' into derivation-header-include-order 2020-06-21 20:39:10 +00:00
John Ericson bf9f040112 Tweak declaration
I think this is clearer
2020-06-21 16:51:39 +00:00
John Ericson fdeabf7160 Merge remote-tracking branch 'upstream/master' into multi-output-hashDerivationModulo 2020-06-21 16:43:17 +00:00
Ben Burdette be4f444175 tidying up 2020-06-19 16:58:12 -06:00
Ben Burdette 0309488a66 fmt -> hintfmt test 2020-06-19 16:46:49 -06:00
Matthew Bauer f2a6cee334 Update worker protocol to support sending storepath maps
We need to also send the ca to daemon in addition to the path.
2020-06-19 18:06:19 -04:00
Ben Burdette 397dbe114e remove formathelper 2020-06-19 15:57:19 -06:00
John Ericson 507aa48739 WIP: Make Hash always store a valid hash type 2020-06-19 21:48:57 +00:00
Ben Burdette b193aca4ae escape percents 2020-06-19 15:29:19 -06:00
Ben Burdette db475f9e7e too few, too many args 2020-06-19 15:28:13 -06:00
Ben Burdette cdddf24f25 add hintfmt test 2020-06-19 14:54:41 -06:00
Ben Burdette 54e8f550c9 addErrorTrace 2020-06-19 13:44:08 -06:00
Matthew Bauer e8e1f5282f Replace error message in getDerivationCA 2020-06-19 15:19:09 -04:00
Matthew Bauer 0c9c1b8826 Return map of StorePaths in copyPaths
This allows the caller to know what values were actually added to the
store.
2020-06-19 14:48:57 -04:00
John Ericson e288c0987a Merge remote-tracking branch 'upstream/master' into validPathInfo-ca-proper-datatype 2020-06-19 18:44:24 +00:00
Matthew Bauer 7e11cf3399 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-19 14:40:49 -04:00
Matthew Bauer 2796b01e86 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-19 14:22:06 -04:00
John Ericson 68294746ae Merge remote-tracking branch 'upstream/master' into no-hash-type-unknown 2020-06-19 17:53:34 +00:00
John Ericson c98081d270 Merge remote-tracking branch 'upstream/master' into no-hash-type-unknown 2020-06-19 17:50:05 +00:00
John Ericson c1892a5316 tabs -> spaces 2020-06-19 17:49:57 +00:00
John Ericson 911fc88bcb More designated initializers 2020-06-19 17:42:56 +00:00
John Ericson 2f0e395c99 Merge remote-tracking branch 'me/no-stringly-typed-derivation-output' into validPathInfo-ca-proper-datatype 2020-06-19 15:26:59 +00:00
John Ericson fb39a5e00c Remove unneeded constructor for DerivationOutputHash 2020-06-19 15:11:11 +00:00
John Ericson b90cac3bad Remove uneeded = default for Hash 2020-06-19 15:00:38 +00:00
John Ericson 01dc8b0bab Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output 2020-06-19 14:59:05 +00:00
John Ericson 145d88cb2a Use designated initializers for DerivationOutputHash 2020-06-19 14:58:30 +00:00
John Ericson 237d88c97e FileSystemHash -> DerivationOutputHash 2020-06-19 14:47:10 +00:00
Eelco Dolstra 2886c92aef
Merge pull request #3669 from gilligan/add-compression-tests
Add compression unit tests
2020-06-19 13:59:04 +02:00
John Ericson 3fc58a9638 Remove some Base:: that crept in 2020-06-19 00:24:47 +00:00
John Ericson 3f8dcfe3fd Merge branch 'validPathInfo-temp' into validPathInfo-ca-proper-datatype 2020-06-18 23:01:58 +00:00
John Ericson 669c3992e8 Merge branch 'no-hash-type-unknown' into validPathInfo-temp 2020-06-18 22:33:07 +00:00
John Ericson 15abb2aa2b Revert the enum struct change
Not a regular git revert as there have been many merges and things.
2020-06-18 22:11:26 +00:00
John Ericson bbbf3602a3 Merge branch 'enum-class' into no-hash-type-unknown 2020-06-18 22:11:19 +00:00
John Ericson 40526fbea5 Merge remote-tracking branch 'upstream/master' into enum-class 2020-06-18 21:38:15 +00:00
Ben Burdette 4d1a4f0217 addTrace 2020-06-18 15:25:26 -06:00
Ben Burdette e6f93b94fc Merge branch 'master' into caveman-LOCs 2020-06-18 13:07:53 -06:00
John Ericson 1b23fe4afb Fix bugs
- Bad dynamic cast target ...classic

 - std::shared_ptr need explicit deref
2020-06-18 19:03:10 +00:00
John Ericson fb432b0bd2 Merge remote-tracking branch 'obsidian/fix-url-format' into add-body-to-network-errors 2020-06-18 18:08:02 +00:00
John Ericson 639e20dc3e Prevent '%' in URL from causing crashes
We have a larger problem that passsing computed strings to the first
variable argument of many exception constructors is unsafe because that
first variable argument is interpreted not as a plain string, but format
string, and if it contains '%' boost::format will abort, since there are
no arguments to the format string.

In this particular instance '%' was used as part of an escape code in a
URL, which, when the download failed, caused Nix to abort displaying the
`FileTransferError`.
2020-06-18 17:54:16 +00:00
Eelco Dolstra 6c000eed80
Merge pull request #3709 from expipiplus1/master
Mention number of derivations to be build/fetched in output
2020-06-18 19:03:05 +02:00
John Ericson 74b219ef6e Adjust FileTransferError message to use opt response 2020-06-18 14:48:45 +00:00
Eelco Dolstra 5771c8bbf2 Don't provide 'getFlake' if the 'flakes' feature is not enabled
(cherry picked from commit 0a1d3c1dd3)
2020-06-18 14:03:00 +02:00
Eelco Dolstra 2a61bbf77f Some backports from the flakes branch 2020-06-18 14:03:00 +02:00
Eelco Dolstra 377345e26f Remove unneeded #include 2020-06-18 13:47:05 +02:00
Eelco Dolstra 7083d33efe Make constant primops lazy
(cherry picked from commit aa0e2a2e70)
2020-06-18 13:42:47 +02:00
Eelco Dolstra 3d492199bb github: Respect default branch 2020-06-18 13:25:08 +02:00
regnat 4fef2ba7e4 Rename content-addressed-paths into ca-derivations
See <https://github.com/NixOS/nix/pull/3710#issuecomment-645480333>
2020-06-18 09:25:55 +02:00
Carlo Nucera 004570a377 Add HTTP responses to FileTransferErrors 2020-06-17 22:34:31 +00:00
Carlo Nucera a835c740ca Replace TransferItem::status with a local variable
Everywhere seems to use `getHTTPStatus` now.
2020-06-17 22:16:24 +00:00
Carlo Nucera 289b9b8dcf Create a new TeeSink abstraction
This is a bit complex because we want to expose extra functionality the
wrapped class has. Perhaps there is some inheritancy trickery to do this
nicer, but I don't know it, and this is the first thing we tried after a
series of attempts that did build.

This design is kind of like that of Rust's Writer, Reader, or Iter
adapters, which impliment more traits based on what the inner type
implements.
2020-06-17 22:09:32 +00:00
Carlo Nucera fb05a6adcf Eliminate old TeeSink abstraction
This was introduced in fa125b9b28, and
then "reverted" in 1cf4801108, except that
revert left the struct around doing nothing useful.

We're removing it all the way now because we want to make a new
`TeeSink` complementing the already-exiting `TeeSource`, that is
actually a completely different concept as far as the class hierarchy is
concerned.
2020-06-17 22:07:22 +00:00
Eelco Dolstra 2b8f33bf5f
Merge pull request #3713 from matthewbauer/cleanup-warnings
Cleanup class StorePath warning
2020-06-17 21:19:30 +02:00
Matthew Bauer 5e631e3304 Add StorePathCAMap for querySubstitutablePathInfos
I’m not 100% sure this is wanted since it kind of makes everything
have to know about ca even if they don’t really want to. But it also
make things easier in dealing with looking up ca.
2020-06-17 15:03:05 -04:00
Matthew Bauer be50de1142 Make sure references are empty for store path replacing
also copy info2 instead of casting
2020-06-17 14:15:08 -04:00
Matthew Bauer 8974755d19 Add assert for replaced storePath 2020-06-17 14:04:46 -04:00
Eelco Dolstra 3078404e35
Merge pull request #3712 from obsidiansystems/make-http-successful-states-coherent
Make successful states coherent
2020-06-17 19:37:46 +02:00
Matthew Bauer 7bd88cc1dc Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-17 13:32:29 -04:00
Matthew Bauer 22d7d36703 Remove unused narInfoFile in binary-cache-store 2020-06-17 13:27:19 -04:00
Matthew Bauer f767bedfac Replace struct StorePath with class StorePath
also a similar case with struct Goal
2020-06-17 13:26:37 -04:00
Carlo Nucera 4930cb48a2 Include review comments 2020-06-17 12:58:59 -04:00
Matthew Bauer 6438ba1e99 Update strings from review comment 2020-06-17 12:31:01 -04:00
Eelco Dolstra 2f51cd8dc9
Merge pull request #3710 from tweag/reserve_ca_derivations
Reserve the `__contentAddressed` derivation parameter
2020-06-17 18:28:26 +02:00
Eelco Dolstra 4d5169bdd5
Merge pull request #3707 from p01arst0rm/outdated-function-fix
replaced uncaught_exception with uncaught_exceptions
2020-06-17 18:26:01 +02:00
Eelco Dolstra 2e4bd78211 nix eval: Add --apply flag for post-processing the result 2020-06-17 18:12:24 +02:00
regnat 480b54e1c6 fixup! Reserve the __contentAddressed derivation parameter 2020-06-17 17:37:04 +02:00
Carlo Nucera 079c6e87de Make successful states coherent
The successful states used in these two places in the code were slightly
different. Should they be the same list?
2020-06-17 11:16:16 -04:00
Eelco Dolstra 5332c439d0 InstallableFlake: Show all possible attribute names
E.g.

  $ nix run nixpkgs#hello
  error: --- Error ---------- nix
  flake 'flake:nixpkgs' does not provide attribute 'apps.x86_64-linux.hello' or 'hello'

instead of

  $ nix run nixpkgs#hello
  error: --- Error ---------- nix
  flake 'flake:nixpkgs' does not provide attribute 'hello'
2020-06-17 17:13:01 +02:00
John Ericson 6403508f5a Use ansicolor.hh in nix repl rather than duplicates 2020-06-17 15:13:00 +00:00
Eelco Dolstra ad66fb0a37 getFlake -> builtins.getFlake 2020-06-17 17:05:08 +02:00
Eelco Dolstra 0a1d3c1dd3 Don't provide 'getFlake' if the 'flakes' feature is not enabled 2020-06-17 16:54:32 +02:00
regnat 56d75bf4fc Reserve the __contentAddressed derivation parameter
Not implementing anything here, just throwing an error if a derivation
sets `__contentAddressed = true` without
`--experimental-features content-addressed-paths`
(and also with it as there's nothing implemented yet)
2020-06-17 15:41:17 +02:00
Eelco Dolstra fdff09e57c Fix coverage build 2020-06-17 15:18:10 +02:00
Eelco Dolstra ccbea8255c
Merge pull request #3657 from obsidiansystems/sligthly-improve-store-path-documentation
Clarify the description of StorePath inputs
2020-06-17 14:54:37 +02:00
Joe Hermaszewski da8aac6ce8 Mention number of derivations to be build/fetched in output
Also correct grammar for the case of a single derivation.
2020-06-17 20:27:27 +08:00
Eelco Dolstra 1524752c17 Merge remote-tracking branch 'origin/master' into flakes 2020-06-17 10:26:52 +02:00
John Ericson 517f5980e2 Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output 2020-06-17 04:58:43 +00:00
John Ericson 18493fd9c4 Move some Store functions from derivations.cc to store-api.cc
This further continues with the dependency inverstion. Also I just went
ahead and exposed `parseDerivation`: it seems like the more proper
building block, and not a bad thing to expose if we are trying to be
less wedded to drv files on disk anywas.
2020-06-17 04:30:16 +00:00
John Ericson 7130f0a3a6 Don't need abstract struct Derivation in local-store 2020-06-17 04:05:38 +00:00
John Ericson 21ef342172 Merge remote-tracking branch 'upstream/master' into derivation-header-include-order 2020-06-17 03:52:01 +00:00
John Ericson 02928f76fd Merge remote-tracking branch 'upstream/master' into multi-output-hashDerivationModulo 2020-06-17 03:51:02 +00:00
John Ericson 49308ef039 Merge remote-tracking branch 'upstream/master' into ca-drv 2020-06-17 03:39:39 +00:00
p01arst0rm e9970a34e8 appended ' __attribute__((weak)); ' to 'extern char * * environ ' 2020-06-17 03:25:34 +01:00
p01arst0rm c9d06558b6 replaced uncaught_exception with uncaught_exceptions 2020-06-17 03:15:47 +01:00
Eelco Dolstra 29542865ce Remove StorePath::clone() and related functions 2020-06-16 22:20:18 +02:00
Eelco Dolstra df4da4f5da
Merge pull request #3702 from NixOS/store-path-cxx
Rewrite StorePath class in C++
2020-06-16 21:37:26 +02:00
John Ericson fbf90bd693 Add another missing #include 2020-06-16 14:19:49 +00:00
Eelco Dolstra 759947bf72 StorePath: Rewrite in C++
On nix-env -qa -f '<nixpkgs>', this reduces maximum RSS by 20970 KiB
and runtime by 0.8%. This is mostly because we're not parsing the hash
part as a hash anymore (just validating that it consists of base-32
characters).

Also, replace storePathToHash() by StorePath::hashPart().
2020-06-16 14:28:41 +02:00
Eelco Dolstra 72e17290d4 Fix FTP support
Fixes #3618.
2020-06-16 11:53:04 +02:00
Tobias Pflug cd8214c398 Fix logging unit tests 2020-06-16 10:23:15 +02:00
John Ericson 7e7e3b71f3 Add mising #include for strerror 2020-06-15 23:35:07 +00:00
Eelco Dolstra a588b6b19d Print only one error message if a build fails
E.g. instead of

  error: --- BuildError ----------------------------------------------- nix
  builder for '/nix/store/03nk0a3n8h2948k4lqfgnnmym7knkcma-foo.drv' failed with exit code 1
  error: --- Error ---------------------------------------------------- nix
  build of '/nix/store/03nk0a3n8h2948k4lqfgnnmym7knkcma-foo.drv' failed

we now get

  error: --- Error ---------------------------------------------------- nix
  builder for '/nix/store/03nk0a3n8h2948k4lqfgnnmym7knkcma-foo.drv' failed with exit code 1
2020-06-15 19:35:31 +02:00
Eelco Dolstra 24a3208247 Include only the base name of the program in error messages 2020-06-15 19:35:31 +02:00
Eelco Dolstra 8b099812ea Respect terminal width printing error messages 2020-06-15 19:35:31 +02:00
Eelco Dolstra f20bb983ca Cleanup 2020-06-15 18:16:03 +02:00
Eelco Dolstra 4e995bc8a6 Always hide the progress bar on exit 2020-06-15 18:01:05 +02:00
Eelco Dolstra 31707735b6 Remove unnecessary amDone() overrides 2020-06-15 16:47:21 +02:00
Eelco Dolstra ccfa6b3eee Give better error message about <...> in pure eval mode 2020-06-15 16:12:27 +02:00
Eelco Dolstra 5ed5d7acbd Improve "waiting for locks" messages
These are now shown in the progress bar.

Closes #3577.
2020-06-15 16:03:29 +02:00
Eelco Dolstra e14e62fddd Remove trailing whitespace 2020-06-15 14:12:39 +02:00
Eelco Dolstra 1fb762d11f Get rid of explicit ErrorInfo constructors 2020-06-15 14:06:58 +02:00
Eelco Dolstra fd64e4fb96 Disambiguate BaseError(Args) constructor
This means that 'throw Error({ ... ErrorInfo ... })' now works.
2020-06-15 13:50:33 +02:00
Eelco Dolstra 7a77762961 Merge branch 'errors-phase-2' of https://github.com/bburdette/nix 2020-06-15 11:46:31 +02:00
Matthew Bauer c214cda940 Correctly substitute from different storeDir
Originally, the test was only checking for different “real” storeDir.
That’s an easy case to handle, but the much harder one is if different
virtual store dirs are used. To do this, we need the SubstitutionGoal
to know about the ca, so it can recalculate the path to copy it over.
An important note here is that the store path passed to copyStorePath
needs to be one for srcStore - so that queryPathInfo works properly.

This also adds an error message when the store path from queryPathInfo
is different from the one we requested.
2020-06-13 00:07:42 -05:00
Matthew Bauer b2cb288cdd Add makeFixedOutputPathFromCA function
This puts what we are already doing into a shared method. It just
needs a path name and a ca and produces a store path.
2020-06-12 16:36:35 -05:00
John Ericson f6f01416b7 Use std::string_view in a few more places 2020-06-12 21:32:30 +00:00
Matthew Bauer 88120442d2 Debug when storePath changes
these rewrites should be transparent, but they are important to know
about when debugging
2020-06-12 15:32:52 -05:00
Matthew Bauer e3cb536f19 Fix add-to-store --flat to put in correct hash 2020-06-12 15:25:45 -05:00
Matthew Bauer 5b386b05f5 Recompute storePath based on isContentAddressed 2020-06-12 14:57:45 -05:00
Matthew Bauer 85d01e1f65 Don’t use makeStorePath 2020-06-12 14:53:30 -05:00
Matthew Bauer 9077b9e6b2 Separate dstStore path from srcStore path 2020-06-12 14:27:28 -05:00
Matthew Bauer 2f2ac850b5 Compute new store path correctly 2020-06-12 14:03:17 -05:00
Matthew Bauer 1c55f16a16 Add --flat to nix add-to-store
This can be used to add flat hashes to the nix store.
2020-06-12 12:42:54 -05:00
Eelco Dolstra 2853ba4ab2 Fix build 2020-06-12 19:00:48 +02:00
Eelco Dolstra 00fa7e2205
Merge pull request #3674 from matthewbauer/allow-empty-hash2
Allow empty hash in derivations
2020-06-12 18:18:12 +02:00
Matthew Bauer ea0d29d99a Provide base argument to to_string 2020-06-12 10:18:27 -05:00
Matthew Bauer 11c97070f3 Fix storeDir != storeDir condition
this needs to only continue if the path replacement fails.
2020-06-12 10:14:03 -05:00
Matthew Bauer b260c9ee03 Add newHashAllowEmpty helper function
This replaces the copy&paste with a helper function in hash.hh.
2020-06-12 10:11:16 -05:00
Matthew Bauer 3e3eaa90dd Remove hashed-mirrors 2020-06-12 09:51:44 -05:00
Matthew Bauer 79c169d1c6 Allow substituting from different storeDir
Substituters can substitute from one store dir to another with a
little bit of help. The store api just needs to have a CA so it can
recompute the store path based on the new store dir. We can only do
this for fixed output derivations with no references, though.
2020-06-12 09:49:09 -05:00
Eelco Dolstra 9f736dd89d Add Store::readDerivation() convenience function 2020-06-12 13:04:52 +02:00
Eelco Dolstra 045b07200c Remove Store::queryDerivationOutputNames()
This function was used in only one place, where it could easily be
replaced by readDerivation() since it's not
performance-critical. (This function appears to have been modelled
after queryDerivationOutputs(), which exists only to make the garbage
collector faster.)
2020-06-12 12:46:33 +02:00
Eelco Dolstra 4a4c063222
Merge pull request #3670 from gilligan/add-pool-tests
Add tests for pool.hh
2020-06-12 11:19:05 +02:00
Eelco Dolstra 7db879e65e Check 'follows' inputs 2020-06-12 00:52:56 +02:00
Ben Burdette ef1b3f21b6 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-06-11 14:06:35 -06:00
Eelco Dolstra d15c20efd5 diffLockFiles(): Show 'follows' changes 2020-06-11 22:00:58 +02:00
Eelco Dolstra ac4d43a31b
Merge pull request #3073 from tweag/machine-logs
Add an option to print the logs in a machine-readable format
2020-06-11 15:45:18 +02:00
Eelco Dolstra dd9bb11d0d Move names.{cc,hh} to libstore 2020-06-11 15:42:18 +02:00
Eelco Dolstra 95eb064062 Shut up warning 2020-06-11 15:39:30 +02:00
Eelco Dolstra 8bd892117a Style fixes 2020-06-11 15:39:08 +02:00
Eelco Dolstra 0c62b4ad0f Represent 'follows' inputs explicitly in the lock file
This fixes an issue where lockfile generation was not idempotent:
after updating a lockfile, a "follows" node would end up pointing to a
new copy of the node, rather than to the original node.
2020-06-11 14:40:21 +02:00
Tobias Pflug 4750d98bbd Add tests for pool.hh 2020-06-10 22:29:50 +02:00
Eelco Dolstra 195ed43b60 Preserve 'isFlake' when not updating a lock file entry 2020-06-10 16:24:05 +02:00
Eelco Dolstra 2226e97ec2 Combine lock file update messages 2020-06-10 15:22:12 +02:00
Eelco Dolstra fc6c7af424 Add helper function printInputPath() 2020-06-10 15:20:00 +02:00
Eelco Dolstra b9ae1bdd7a
Merge pull request #3655 from zimbatm/hash-encoding-prepare
libutils/hash: remove default encoding
2020-06-10 11:48:38 +02:00
Matthew Bauer b2c8061b44
Disable extra-platforms = i686-linux on wsl1 (#3676)
WSL1 doesn’t support i686-linux emulation, see https://github.com/microsoft/wsl/issues/2468
2020-06-09 21:53:53 +00:00
Matthew Bauer 19aa892f20 Support empty hash in fetchers
fetchTarball, fetchTree, and fetchGit all have *optional* hash attrs.
This means that we need to be careful with what we allow to avoid
accidentally making these defaults. When ‘hash = ""’ we assume the
empty hash is wanted.
2020-06-09 11:10:54 -05:00
Eelco Dolstra 29e0748847 Show HTTP status message
For example:

  warning: unable to download 'https://api.github.com/repos/edolstra/dwarffs/commits/master': HTTP error 403 ('rate limit exceeded'); using cached version
2020-06-09 14:20:22 +02:00
Eelco Dolstra 447ea52b07 FileTransfer: Don't store status since curl already does that 2020-06-09 14:05:15 +02:00
Eelco Dolstra 6cfc2db494 Fix applyOverride() for github 2020-06-09 13:45:07 +02:00
Matthew Bauer 762273f1fd Allow empty hash in derivations
follow up of https://github.com/NixOS/nix/pull/3544

This allows hash="" so that it can be used for debugging purposes. For
instance, this gives you an error message like:

  warning: found empty hash, assuming you wanted 'sha256:0000000000000000000000000000000000000000000000000000'
  hash mismatch in fixed-output derivation '/nix/store/asx6qw1r1xk6iak6y6jph4n58h4hdmbm-nix':
    wanted: sha256:0000000000000000000000000000000000000000000000000000
    got:    sha256:0fpfhipl9v1mfzw2ffmxiyyzqwlkvww22bh9wcy4qrfslb4jm429
2020-06-09 01:23:37 -05:00
Ben Burdette 2f19650768 add file origin to Pos in stests 2020-06-08 11:21:17 -06:00
Ben Burdette b1c53b034c Merge branch 'errors-phase-2' into caveman-LOCs 2020-06-08 11:10:13 -06:00
regnat 801112de1a Move progress-bar.cc to libmain
Needed so that we can include it as a logger in loggers.cc without
adding a dependency on nix

This also requires moving names.hh to libutil to prevent a circular
dependency between libmain and libexpr
2020-06-08 17:16:52 +02:00
Eelco Dolstra e073f2c584 nix flake: Require 'flakes' feature 2020-06-08 16:23:54 +02:00
Eelco Dolstra 6470450ab4 Add completion for --update-input 2020-06-08 16:20:00 +02:00
Eelco Dolstra c27f92698b Style fixes 2020-06-08 13:24:01 +02:00
Tobias Pflug cd6dbf951a Add compression unit tests 2020-06-08 11:34:37 +02:00
regnat 4983401440 Unify the printing of the logs between bar-with-logs and raw
Make the printing of the build logs systematically go through the
logger, and replicate the behavior of `no-build-output` by having two
different loggers (one that prints the build logs and one that doesn't)
2020-06-08 09:31:15 +02:00
regnat 170e86dff5 Make the logger customisable
Add a new `--log-format` cli argument to change the format of the logs.
The possible values are
- raw (the default one for old-style commands)
- bar (the default one for new-style commands)
- bar-with-logs (equivalent to `--print-build-logs`)
- internal-json (the internal machine-readable json format)
2020-06-08 09:31:15 +02:00
Ben Burdette 94c347577e set verbosity levels 2020-06-07 07:24:49 -06:00
Tobias Pflug e60747b5fb Remove error-demo/error-demo.cc
The logging.hh superseeds the demo
2020-06-06 10:23:12 +02:00
Tobias Pflug 952e72c804 Add tests for logging.hh 2020-06-06 10:22:32 +02:00
Eelco Dolstra 9ef6048d78 diffLockFiles(): Fix assertion failure
There are some cases where this inequality didn't hold, in particular
due to the Input / TreeInfo merge, where we're not always showing
narHash.
2020-06-05 20:52:23 +02:00
Eelco Dolstra d558fb98f6
Merge pull request #3656 from obsidiansystems/handle-unknown-file-ingestion
Add error message when FileIngestionMethod is out of bounds
2020-06-05 17:18:12 +02:00
Eelco Dolstra 488ff83e6b Fix completion of --template 2020-06-05 14:09:12 +02:00
Eelco Dolstra 39e84c35d0 Fix log-prefix of nix build -L
Alternative fix to #3661. The cause was that 'name' is a
std::string_view into a temporary which could get overwritten.
2020-06-05 10:45:05 +02:00
Carlo Nucera d614166cb6 Fix condition error and make test suite pass 2020-06-04 17:21:21 -04:00
John Ericson e5cc1ebc5d Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output 2020-06-04 21:04:35 +00:00
John Ericson a7b82fd006 Remove file which shouldn't be committed 2020-06-04 21:04:20 +00:00
John Ericson 94ddea9e2f Use readString rather than >> temporary
Fixed the rest of these before, but this one slipped through.
2020-06-04 20:55:08 +00:00
John Ericson 744ce9ce16 Merge branch 'master' of github.com:NixOS/nix into validPathInfo-ca-proper-datatype 2020-06-04 20:46:58 +00:00
John Ericson 2041499b5e Flip boolean
Thanks Matt!
2020-06-04 20:42:25 +00:00
John Ericson ed86acf02a Use some std::optional::has_value for clarity 2020-06-04 20:42:02 +00:00
John Ericson 574d5460f0 Make sure info.ca tag bit is set in nix add-to-store 2020-06-04 20:33:28 +00:00
Eelco Dolstra ab54031e04 getDefaultUrl() -> value_or() 2020-06-04 20:24:28 +02:00
Eelco Dolstra 810b2c6a48 nix flake init: Add a '--template' flag
The initial contents of the flake is specified by the
'templates.<name>' or 'defaultTemplate' output of another flake. E.g.

  outputs = { self }: {

    templates = {

      nixos-container = {
        path = ./nixos-container;
        description = "An example of a NixOS container";
      };

    };

  };

allows

  $ nix flake init -t templates#nixos-container

Also add a command 'nix flake new', which is identical to 'nix flake
init' except that it initializes a specified directory rather than the
current directory.
2020-06-04 20:22:25 +02:00
Ben Burdette 94427ffee3 add some comments 2020-06-04 11:53:19 -06:00
Matthew Bauer 2299ef705c Add error message when FileIngestionMethod is out of bounds
bool coerces anything >0 to true, but in the future we may have other
file ingestion methods. This shows a better error message when the
“recursive” byte isn’t 1.
2020-06-04 11:32:39 -05:00
Eelco Dolstra dc305500c3
Merge pull request #3660 from Kloenk/selfhost-gitlab
add support for selfhosted gitlab/github
2020-06-04 14:59:33 +02:00
Eelco Dolstra d746ef4a81 Disable eval cache with --impure
Fixes

  $ nix build nixpkgs#zoom-us
  error: Package ‘zoom-us-5.0.399860.0429’ in /nix/store/m79v7h75b69fkk8d2qcwm555l3wq6fmv-source/pkgs/applications/networking/instant-messengers/zoom-us/default.nix:126 has an unfree license (‘unfree’), refusing to evaluate.

  $ nix build nixpkgs#zoom-us --impure
  error: cached failure of attribute 'legacyPackages.x86_64-linux.zoom-us.drvPath'
2020-06-04 14:53:51 +02:00
Finn Behrens 108debef6f
add support for selfhosted gitlab/github 2020-06-04 14:42:39 +02:00
Eelco Dolstra f85606c431 Merge remote-tracking branch 'origin/master' into flakes 2020-06-04 13:16:28 +02:00
Eelco Dolstra 0f44b60e6d Make 'nix dev-shell' a deprecated alias for 'nix develop' 2020-06-04 11:14:19 +02:00
Eelco Dolstra 61e3d598b6 Rename 'nix dev-shell' to 'nix develop'
Fixes #3648.
2020-06-04 10:57:40 +02:00
John Ericson 53bc8ff152 No C++ designated initializers yet with Clang 7 2020-06-03 20:45:14 -04:00
Ben Burdette 4335ba999b Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-06-03 17:00:00 -06:00
Ben Burdette f97576c5d9 newline-as-prefix; no final newline in output. 2020-06-03 14:47:00 -06:00
Carlo Nucera 132d6f2c24 Clarify the description of StorePath construction 2020-06-03 16:08:32 -04:00
John Ericson 74b251b2f3 Don't anticipate multiple CA outputs for now 2020-06-03 18:53:04 +00:00
John Ericson 3a9e4c3262 Don't anticipate CA but not fixed outputs for now 2020-06-03 18:50:45 +00:00
John Ericson 2500403059 Use enum and predicates rather than bitfile for derivation type 2020-06-03 17:38:54 +00:00
John Ericson 6b7f4ec4ab Merge remote-tracking branch 'upstream/master' into ca-drv 2020-06-03 16:36:06 +00:00
Eelco Dolstra c20591ddc3 Merge remote-tracking branch 'origin/master' into flakes 2020-06-03 16:15:22 +02:00
Jonas Chevalier 6ee03b8444
libutils/hash: remove default encoding
This will make it easier to reason about the hash encoding and switch to
SRI everywhere where possible.
2020-06-03 13:49:51 +02:00
John Ericson 01572c2198
Missing #include <cassert> in lru-cache.hh (#3654)
This was a latent bug that just appeared because of the tests that were
added. Remember to wait for CI! :)
2020-06-03 10:15:22 +00:00
John Ericson 3c78ac348c Merge remote-tracking branch 'obsidian/no-hash-type-unknown' into validPathInfo-ca-proper-datatype 2020-06-03 04:44:24 +00:00
John Ericson fecff16a6e Merge remote-tracking branch 'obsidian/missing-include-0' into validPathInfo-ca-proper-datatype 2020-06-02 23:23:30 +00:00
John Ericson 39ba87be9b Missing #include <cassert> in lru-cache.hh
This was a latent bug that just appeared because of the tests that were
added. Remember to wait for CI! :)
2020-06-02 21:36:53 +00:00
John Ericson 406dbb7fce outputHashAlgo can be blank so parse accordingly
It is blank for SRI hashes.
2020-06-02 21:09:15 +00:00
John Ericson 1fcd3afc38 Fix hashes 2020-06-02 20:35:17 +00:00
Carlo Nucera 75d2581390 Typo 2020-06-02 16:21:18 -04:00
Carlo Nucera 78f137e931 Validate text version instead, throw Errors 2020-06-02 16:20:22 -04:00
Carlo Nucera a5cdf1867e Add assertions for SHA256 in fixed case 2020-06-02 16:13:08 -04:00
Carlo Nucera fd2eb41e64 Move file-hash to content-address 2020-06-02 15:44:58 -04:00
Carlo Nucera 343c20a404 WIP Completed implementation 2020-06-02 15:23:21 -04:00
John Ericson c664e68b87 Fix to-base --type handler to correctly set std::optional flag
Now that we have a separate flag function, also describe why it is
optional.
2020-06-02 18:25:32 +00:00
Carlo Nucera 390bf64858 WIP 2020-06-02 14:15:58 -04:00
John Ericson c502119fd3 to-base supports parsing SRI hashes, so make type flag optional 2020-06-02 18:05:26 +00:00
John Ericson a33270ce1d Clean up ValidPathInfo::isContentAddressed with std::visit 2020-06-02 17:04:21 +00:00
John Ericson 25e61812f3
Apply suggestions from code review
Co-authored-by: Matthew Bauer <mjbauer95@gmail.com>
2020-06-02 12:47:18 -04:00
John Ericson d73dbc8e4c Remove hashingWithUnknownAlgoExits
A valid hash type must be provided now. The hash itself can still be
invalid, but that doesn't cause an `abort()`.
2020-06-02 16:28:54 +00:00
John Ericson 64cffb804a Merge remote-tracking branch 'upstream/master' into no-hash-type-unknown 2020-06-02 16:07:25 +00:00
John Ericson 450dcf2c1b Remove HashType::Unknown
Instead, `Hash` uses `std::optional<HashType>`. In the future, we may
also make `Hash` itself require a known hash type, encoraging people to
use `std::optional<Hash>` instead.
2020-06-02 15:52:13 +00:00
Carlo Nucera a5d820a0a3 Change parseCa(Opt) to parseContentAddress(Opt) 2020-06-02 11:00:10 -04:00
Ben Burdette 156d4f8bc8 remove extra space in SysErrors 2020-06-02 08:45:37 -06:00
John Ericson 1b6461f671 Merge remote-tracking branch 'upstream/master' into validPathInfo-ca-proper-datatype 2020-06-02 14:31:18 +00:00
Ben Burdette d82d230b40 elide the 'ErrorInfo' in logError and logWarning calls 2020-06-02 08:22:24 -06:00
Eelco Dolstra bfa1acd85c
Merge pull request #3639 from obsidiansystems/do-fixme-store-removes
Remove `addToStore` variant as requested by `FIXME`
2020-06-02 15:39:07 +02:00
Eelco Dolstra c16fdda3a6 Merge branch 'lru-tests' of https://github.com/gilligan/nix 2020-06-02 12:07:48 +02:00
Eelco Dolstra e9fee8e6a7
src/libutil/tests/lru-cache.cc: Check erase()
Co-authored-by: James Lee <jbit@jbit.net>
2020-06-02 12:06:59 +02:00
John Ericson efcd30da89 WIP 2020-06-02 00:37:43 +00:00
John Ericson 754c910953 WIP more progress 2020-06-01 19:26:40 -04:00
Carlo Nucera da39092a39 WIP 2020-06-01 18:53:31 -04:00
Carlo Nucera 0e9438b6d3 Create new file-hash files 2020-06-01 17:32:40 -04:00
Carlo Nucera 0cb67ecbd3 Merge branch 'derivation-header-include-order' of github.com:Ericson2314/nix into validPathInfo-ca-proper-datatype 2020-06-01 17:13:11 -04:00
Carlo Nucera f4b89e11a4 Merge branch 'no-stringly-typed-derivation-output' of github.com:Ericson2314/nix into validPathInfo-ca-proper-datatype 2020-06-01 17:12:50 -04:00
Matthew Kenigsberg c254254a80 use Tree ctor 2020-06-01 12:32:17 -06:00
Matthew Kenigsberg 7680993506 Tree ctors 2020-06-01 09:01:37 -06:00
Matthew Kenigsberg ff1320b850 fetchOrSubstituteTree improvements
Caches tree in addition to lockedRef, and explicitly writes out the logic for different combinations of cached/uncached flakes and indirect/resolved/locked flakes. This eliminates uneccessary calls to lookupInFlakeCache, fetchTree, maybeLookupFlake, and flakeCache.push_back
2020-06-01 02:57:22 -06:00
Tobias Pflug eca1ff7a9f Add tests for lru-cache.hh 2020-05-31 01:05:05 +02:00
Nikola Knezevic fb38459d6e Ensure we restrict refspec interpretation while fetching
As `git fetch` may chose to interpret refspec to it's liking, ensure that we
only pass refs that begin with `refs/` as is, otherwise, prepend them with
`refs/heads`. Otherwise, branches named `heads/foo` (I know it's bad, but it's
allowed), would be fetched as `foo`, instead of `heads/foo`.
2020-05-30 12:33:38 +02:00
Nikola Knezevic 77007d4eab Improve ref validity checking in fetchGit
The previous regex was too strict and did not match what git was allowing. It
could lead to `fetchGit` not accepting valid branch names, even though they
exist in a repository (for example, branch names containing `/`, which are
pretty standard, like `release/1.0` branches).

The new regex defines what a branch name should **NOT** contain. It takes the
definitions from `refs.c` in https://github.com/git/git and `git help
check-ref-format` pages.

This change also introduces a test for ref name validity checking, which
compares the result from Nix with the result of `git check-ref-format --branch`.
2020-05-30 12:29:35 +02:00
Eelco Dolstra 89e0b3e2d6 Move substitution into Input::fetch()
Closes #3520.
2020-05-30 01:16:53 +02:00
Eelco Dolstra 0e7f77a59a Check revCount / lastModified input attributes if specified 2020-05-30 00:59:13 +02:00
Eelco Dolstra 950b46821f Remove TreeInfo
The attributes previously stored in TreeInfo (narHash, revCount,
lastModified) are now stored in Input. This makes it less arbitrary
what attributes are stored where.

As a result, the lock file format has changed. An entry like

    "info": {
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },
    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github"
    },

is now stored as

    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github",
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },

The 'Input' class is now a dumb set of attributes. All the fetcher
implementations subclass InputScheme, not Input. This simplifies the
API.

Also, fix substitution of flake inputs. This was broken since lazy
flake fetching started using fetchTree internally.
2020-05-30 00:44:11 +02:00
John Ericson fac0c2d54a Remove addToStore variant as requested by FIXME
The idea is it's always more flexible to consumer a `Source` than a
plain string, and it might even reduce memory consumption.

I also looked at `addToStoreFromDump` with its `// FIXME: remove?`, but
the worked needed for that is far more up for interpretation, so I
punted for now.
2020-05-29 17:02:32 -04:00
Ben Burdette 734283d636 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-05-29 09:51:37 -06:00
Eelco Dolstra 5633c0975b Factor out GitHub / GitLab commonality 2020-05-29 14:23:32 +02:00
Finn Behrens 5256bc77ca
add gitlab libfetcher 2020-05-28 23:00:08 +02:00
Carlo Nucera 6dd471ebf6 Fixing the result of merge 2020-05-28 12:14:36 -04:00
Carlo Nucera 4f597fb901 Merge branch 'master' of github.com:NixOS/nix into enum-class 2020-05-28 10:58:22 -04:00
John Ericson 5b4cd84bc2 Merge remote-tracking branch 'me/more-rust-ffi' into no-stringly-typed-derivation-output 2020-05-28 10:35:53 -04:00
John Ericson ef71caba29 Merge remote-tracking branch 'upstream/master' into more-rust-ffi 2020-05-28 10:31:46 -04:00
Eelco Dolstra f60ce4fa20
Merge pull request #3631 from andir/libutil-config-tests
Add unit tests for config.cc
2020-05-28 13:51:37 +02:00
Eelco Dolstra de141fcb79
Merge pull request #3455 from Ericson2314/enum-FileIngestionMethod
Replace some `bool recursive` with a new `FileIngestionMethod` enum
2020-05-28 13:50:06 +02:00
Eelco Dolstra 17ca997fc6 Merge remote-tracking branch 'origin/master' into flakes 2020-05-28 12:55:24 +02:00
Eelco Dolstra c3eff22f46 Merge branch 'store-visited' of https://github.com/mkenigs/nix into flakes 2020-05-28 12:15:39 +02:00
Eelco Dolstra 6286272371 nixpkgsFlakeRef(): Use locked nixpkgs 2020-05-28 12:13:13 +02:00
John Ericson 0f96f45061 Use FileIngestionMethod for nix hash
There was an enum there that matched in perfectly.
2020-05-27 23:50:11 -04:00
Matthew Bauer c66441a646 Rename some variables named “recursive” to “method”
This is much less confusing since recursive is no longer a boolean.
2020-05-27 13:21:26 -05:00
Matthew Bauer 7873fd175d Don’t use FileIngestionMethod for StorePathsCommand
This is a different recursive than used in makeFixedOutputPath.
2020-05-27 13:21:11 -05:00
Andreas Rammhold fc137d2f00
config.hh: Add documentation
Provides some general overview on the mechanics of Config/Setting and
comments for the public methods of Config.
2020-05-27 17:47:18 +02:00
Andreas Rammhold 9df3d8ccd7
tests/config.cc: add tests for Config::applyConfig 2020-05-27 17:47:18 +02:00
Andreas Rammhold e1b8c64c04
config.cc: extract parts of applyConfigFile into applyConfig
This moves the actual parsing of configuration contents into applyConfig
which applyConfigFile is then going to call. By changing this we can now
test the configuration file parsing without actually create a file on
disk.
2020-05-27 17:47:18 +02:00
Andreas Rammhold 93129cf1dd
Add unit tests for config.cc 2020-05-27 17:47:17 +02:00
Eelco Dolstra 66d3ac94c9 Merge pull request #3621 from gilligan/add-json-tests
Add unit tests for "json.hh"
2020-05-27 11:08:12 +02:00
Eelco Dolstra dae6a267a8
Merge pull request #3625 from gilligan/xml-writer-tests
Add unit tests for xml-writer
2020-05-27 11:07:53 +02:00
Carlo Nucera f3f520c14c Change syntax for CI 2020-05-26 12:51:28 -04:00
Carlo Nucera 89a5ac9d3b Merge remote-tracking branch 'john-ericson/more-rust-ffi' into no-stringly-typed-derivation-output 2020-05-26 12:31:26 -04:00
Carlo Nucera d49e65ba9d Merge remote-tracking branch 'john-ericson/enum-FileIngestionMethod' into no-stringly-typed-derivation-output 2020-05-26 12:30:48 -04:00
Carlo Nucera 0f3f901071 Merge remote-tracking branch 'origin/master' into more-rust-ffi 2020-05-26 11:46:42 -04:00
Carlo Nucera c2f33edd1f
Update src/libutil/rust-ffi.hh
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2020-05-26 11:43:18 -04:00
Carlo Nucera b90241ceb1 Change remaining bools with FileIngestionMethod 2020-05-26 11:32:41 -04:00
Carlo Nucera 6d73c10041 Merge remote-tracking branch 'origin/master' into enum-FileIngestionMethod 2020-05-26 11:14:08 -04:00
Lucas Franceschino b2748c6e99 Make functionArgs primitive accept primops 2020-05-25 19:07:38 +02:00
Tobias Pflug 4b388e8431 Add unit tests for xml-writer 2020-05-25 18:34:55 +02:00
Tobias Pflug c284700867 Add unit tests for "json.hh" 2020-05-25 11:57:45 +02:00
Tobias Pflug ecc5c90dfc Add unit tests for hashing functions 2020-05-25 11:50:41 +02:00
Matthew Kenigsberg 8d67794da1 handle circular flake dependencies in list-inputs 2020-05-21 17:06:11 -06:00
Ben Burdette b7057fa627 remove error-demo from make; clean up comment 2020-05-21 16:04:18 -06:00
Ben Burdette 0e49de6a2b position for stdin, string; (string) for trace; fix tests 2020-05-21 14:28:45 -06:00
Ben Burdette 6a420d672c print LOC for stdin, string args 2020-05-20 22:18:26 -06:00
Ben Burdette 85ce455b85 get code lines from the nix file 2020-05-20 17:25:02 -06:00
Tobias Pflug a73a820a5d Add unit testes for url.cc
This adds tests for

- parseURL
- percentDecode
- decodeQuery
2020-05-20 16:37:35 +02:00
Eelco Dolstra 5ef64f05e6 Cleanup 2020-05-18 15:50:29 +02:00
Eelco Dolstra 0ed946aa61 Merge branch 'wait-for-builders' of https://github.com/serokell/nix 2020-05-18 13:48:45 +02:00
Matthew Kenigsberg c4beded32e rm includes 2020-05-16 11:19:41 -06:00
Matthew Kenigsberg ba7d7ed2e3 Create bashInteractive InstallableFlake 2020-05-16 11:03:06 -06:00
Matthew Kenigsberg 0858793604 Call lockFlake once and store in _lockedFlake 2020-05-16 11:03:06 -06:00
Matthew Kenigsberg 04821bc171 use flake's nixpkgs to find bashInteractive 2020-05-16 11:03:06 -06:00
Matthew Kenigsberg 8fbc8540d3 use nixpkgs#bashInteractive for dev-shell 2020-05-16 11:03:06 -06:00
Ben Burdette 92123c6c79 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-05-15 07:00:36 -06:00
Eelco Dolstra 5f64655ff4 Move registry-related commands from 'nix flake' to 'nix registry'
This makes 'nix flake' less cluttered and more consistent (it's only
subcommands that operator on a flake). Also, the registry is not
inherently flake-related (e.g. fetchTree could also use it to remap
inputs).
2020-05-15 14:38:10 +02:00
Ben Burdette 19694aa213 fix compile errors 2020-05-14 12:28:18 -06:00
Ben Burdette 4daccb279c formatting 2020-05-14 10:28:17 -06:00
Alexander Bantyev 183dd28266
Don't lock a user while doing remote builds 2020-05-14 17:00:54 +03:00
Ben Burdette ef9dd9f9bc formatting and a few minor changes 2020-05-13 15:56:39 -06:00
Ben Burdette c79d4addab consistent capitalization 2020-05-13 10:02:18 -06:00
Ben Burdette bfca5fc395 change status messages to info level 2020-05-13 09:52:36 -06:00
Eelco Dolstra 849d3968db
Update src/libfetchers/git.cc
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
2020-05-13 10:41:21 +02:00
Ben Burdette ecbb8e9c0a no blank line if no LOC 2020-05-12 14:41:30 -06:00
Ben Burdette 960d4362ed hint only 2020-05-12 13:54:18 -06:00
Ben Burdette 72ecccee57 convert to logWarning format 2020-05-12 12:19:34 -06:00
Ben Burdette 19cffc29c9 remove unused extra json fields 2020-05-12 12:09:12 -06:00
Ben Burdette 2a19bf8619 move pos to the first arg, to indicate its not used in a fmt template 2020-05-12 11:27:37 -06:00
Ben Burdette ec870b9c85 new pos format for more errors 2020-05-12 10:52:26 -06:00
Eelco Dolstra 215f09d765
Merge pull request #3587 from NixOS/bash-completion
Generic shell completion support for the 'nix' command
2020-05-12 18:26:13 +02:00
Eelco Dolstra ebc024df22
Show hint how to enable experimental features 2020-05-12 15:47:09 +02:00
Eelco Dolstra 268ecf5b3f
nix: Don't require --experimental-features=nix-command for some subcommands 2020-05-12 15:47:09 +02:00
Eelco Dolstra b8b2dbf272 Fix InstallableCommand 2020-05-12 11:53:32 +02:00
Eelco Dolstra 437614b479 Fix macOS build
macOS doesn't have GLOB_ONLYDIR.
2020-05-12 11:09:09 +02:00
Ben Burdette 7c3138844c more pos reporting 2020-05-11 17:34:57 -06:00
Ben Burdette 631642c5b4 new format for pos 2020-05-11 16:58:08 -06:00
Ben Burdette b93c1bf3d6 fixes to merged code 2020-05-11 15:52:15 -06:00
Ben Burdette 59b1f5c701 Merge branch 'master' into errors-phase-2 2020-05-11 14:35:30 -06:00
Eelco Dolstra 649c2db308 nix flake: Add completion support 2020-05-11 22:10:33 +02:00
Eelco Dolstra 27d34ef770 When completing flakerefs, only return directories 2020-05-11 22:04:13 +02:00
Ben Burdette 536bbf53e1 comments and cleanup 2020-05-11 13:58:38 -06:00
Eelco Dolstra 259ff74bde Add completion for installables
This completes flakerefs using the registry (e.g. 'nix<TAB>' => 'nix
nixpkgs') and flake output attributes by evaluating the flake
(e.g. 'dwarffs#nix<TAB>' => 'dwarffs#nixosModules').
2020-05-11 21:49:02 +02:00
Eelco Dolstra e917332d63 Shut up warnings while running completers 2020-05-11 21:38:17 +02:00
Ben Burdette 958e81987b switch from printError warnings to logWarnings 2020-05-11 13:02:16 -06:00
Domen Kožar 5bdb67c843
Merge pull request #3568 from kolloch/outputHashModeError
libstore/build.cc: more explicit error about form of output
2020-05-11 18:14:32 +02:00
Domen Kožar 1d8144e36b
Update src/libstore/build.cc 2020-05-11 18:14:23 +02:00
Eelco Dolstra 4c3c638a05 Cleanup 2020-05-11 15:57:45 +02:00
Eelco Dolstra 0884f180f5 Simplify 2020-05-10 21:50:32 +02:00
Eelco Dolstra e0c19ee620 Add completion for paths 2020-05-10 21:35:07 +02:00
Eelco Dolstra 91ddee6bf0 nix: Implement basic bash completion 2020-05-10 20:32:21 +02:00
Matthew Kenigsberg 73ee1afffe Reorder to build
This reverts commit 883948d7a0add742ccae58e9845d769a8064371c.
2020-05-09 14:42:32 -06:00
Matthew Kenigsberg 9f4cfbb2e7 Refactor installables
InstallableValue has children InstallableFlake and InstallableAttrPath, but InstallableFlake was overriding toDerivations, and usage was changed so that InstallableFlake didn't need cmd. So these changes were made:
InstallableValue::toDerivations() -> InstalllableAttrPath::toDerivations()
InstallableValue::cmd -> InstallableAttrPath::cmd

InstallableValue uses state instead of cmd

toBuildables() and toDerivations() were made abstract
2020-05-09 14:42:32 -06:00
Matthew Kenigsberg bf81dd40e9 InstallableExpr unused 2020-05-09 10:16:00 -06:00
Benjamin Hipple 146f9c114f doc: consistently refer to 'fixed-output' with a dash
General cleanup that makes it easier to search for the term.
2020-05-09 10:58:43 -04:00
Ben Burdette 55eb717148 add pos to errorinfo, remove from hints 2020-05-08 18:18:28 -06:00
Eelco Dolstra d3d8186c9c
Merge pull request #3571 from gilligan/nix-unit-testing
Add unit tests
2020-05-08 17:02:25 +02:00
Tobias Pflug 181a47d884 Enable toLower umlauts test
Update comment and enable the test
2020-05-08 15:13:55 +02:00
Tobias Pflug 2191141274 Enable baseNameOf test
Add note about removal of trailing slashes in the doc comment of
baseNameOf and enabled the test.
2020-05-08 15:07:40 +02:00
Tobias Pflug e3df9c2a6e Enable dirOf test
Adjusted the doc comment for `dirOf` to reflect the implementation
behavior.
2020-05-08 15:03:44 +02:00
Eelco Dolstra 5b8883faac
configure: Look for gtest 2020-05-08 12:09:37 +02:00
Eelco Dolstra ca657525b8
Don't install unit tests 2020-05-08 12:03:27 +02:00
Eelco Dolstra 7898cdb75a
make check: Run unit tests 2020-05-08 11:49:40 +02:00
Eelco Dolstra 72b9d971bc
Fix warning 2020-05-08 11:35:57 +02:00
Eelco Dolstra 7cc7cef950
Move unit tests to sr/libutil/tests, use mk make rules 2020-05-08 11:34:09 +02:00
Alexander Bantyev 772e5db828
Mention build users in the 'waiting for' message 2020-05-08 12:29:00 +03:00
Alexander Bantyev 14073fb76b
Don't block while waiting for build users 2020-05-08 12:22:39 +03:00
Ben Burdette 1b801cec40 pretending to be const 2020-05-07 16:43:36 -06:00
Tobias Pflug 1f3602a2c9 Remove replaceInSet
The function isn't being used anywhere so it seems safe to remove
2020-05-07 18:15:13 +02:00
Eelco Dolstra 14a3a62bfc
Update src/nix/search.cc
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
2020-05-07 12:13:16 +02:00
Ben Burdette e3901638b5 todo removal 2020-05-06 15:01:13 -06:00
Ben Burdette e76ad2e48a implement SysError errno handling 2020-05-06 14:07:20 -06:00
Eelco Dolstra 1ad71bc62c Remove support for old lockfiles and the epoch/uri attributes 2020-05-06 17:48:18 +02:00
Eelco Dolstra b0e9b07e80 Remove obsolete FIXME 2020-05-06 17:40:01 +02:00
Eelco Dolstra ff394ff206 Remove the nixpkgs.<attr> compatibility hack
Since we've changed a lot of things in the 'nix' command (e.g. rename
'nix run') there is not much point in keeping this around.
2020-05-06 17:20:23 +02:00
Domen Kožar 672985531c
Merge pull request #3570 from Mic92/nix-search
nix/search: no error for empty search results if json is enabled
2020-05-06 16:14:58 +02:00
Jörg Thalheim 85c1932c94
nix/search: no error for empty search results if json is enabled
- result list will be always empty if --json is passed
- for scripts an empty search result is not really an error,
  we rather want to distinguish between evaluation errors and empty results
2020-05-06 14:43:54 +01:00
Eelco Dolstra 2f8ee4578f
Merge remote-tracking branch 'origin/master' into flakes 2020-05-06 12:01:40 +02:00
Eelco Dolstra 74a1bfdcab
Merge pull request #3546 from guibou/nix_readfile_on_0_sized_files
builtins.readFile: do not truncate content
2020-05-06 11:33:55 +02:00
Eelco Dolstra 272c4ba36d
Merge pull request #3557 from Ma27/nix-repl-help
Improve help-message for nix-repl
2020-05-06 11:25:47 +02:00
Peter Kolloch 9be46859a9 libstore/build.cc: more explicit about form of output
Be more explicit about why we expect a regular file as output
when outputHashMode=flat for a fixed output derivation.
2020-05-06 11:21:12 +02:00
Eelco Dolstra 02c5914ea4
Merge pull request #3562 from pikajude/master
Use fragment size for autoGC capacity calculation
2020-05-06 11:19:30 +02:00
Eelco Dolstra fd4911269f
Revert "Merge pull request #3558 from LnL7/ssh-ng-stderr"
This reverts commit 3ebfbecdd1, reversing
changes made to c089c52d5f.

https://github.com/NixOS/nix/pull/3558
2020-05-06 10:54:18 +02:00
Eelco Dolstra 6f3244ce45 Merge remote-tracking branch 'origin/master' into flakes 2020-05-05 18:59:33 +02:00
Eelco Dolstra 909b4a8820 nix doctor: Consistency 2020-05-05 15:27:47 +02:00
Eelco Dolstra f132d82a79 nix --help: Group commands 2020-05-05 15:18:23 +02:00
Alexander Bantyev 04967dee9d
Wait for build users when none are available 2020-05-05 13:04:36 +03:00
Ben Burdette 7ffb5efdbc appending to hints; remove _printError 2020-05-04 16:19:57 -06:00
Ben Burdette f30de61578 add normaltxt, yellowify->yellowtxt 2020-05-04 16:19:20 -06:00
Jude Taylor e2fc575c61 nix auto-gc: use fragment size 2020-05-04 14:42:06 -07:00
Ben Burdette 8c8f2b74ec log as warning 2020-05-04 14:44:42 -06:00
Ben Burdette afaa541013 affinity operator<< 2020-05-04 14:44:00 -06:00
Eelco Dolstra a721a0b114 Flag: Use designated initializers 2020-05-04 22:40:19 +02:00
Ben Burdette 9c5ece44a7 separate msgs instead of appending to what() 2020-05-04 13:46:15 -06:00
Eelco Dolstra 3ebfbecdd1
Merge pull request #3558 from LnL7/ssh-ng-stderr
remote-store: don't log raw stderr by default
2020-05-04 13:02:33 +02:00
Ben Burdette ab6f0b9641 convert some printError calls to logError 2020-05-03 08:01:25 -06:00
Daiderd Jordan 4769eea5e2
logging: handle build log lines in simple logger
The raw stderr output isn't logged anymore so the build logs need to be
printed by the default logger in order for the old commands like
nix-build to still show build output.
2020-05-02 23:40:53 +02:00
Daiderd Jordan f16e24f95e
remote-store: don't log raw stderr by default
For remote stores the log messages are already forwarded as structured
STDERR_RESULT messages so the old format is duplicate information.  But
still included with -vvv since it could be useful for debugging
problems.

    $ nix build -L /nix/store/nl71b2niws857ffiaggyrkjwgx9jjzc0-foo.drv --store ssh-ng://localhost
    Hello World!
    foo> Hello World!
    [1/0/1 built] building foo

Fixes #3556
2020-05-02 23:40:50 +02:00
Maximilian Bosch 2aeb874e83
Improve help-message for nix-repl
* Remove obsolete `printHelp` function
* Add an example to demonstrate how to list all available commands
  within the REPL
2020-05-01 23:32:01 +02:00
Ben Burdette 4b99c09f5c convert some errors 2020-05-01 14:32:06 -06:00
Eelco Dolstra c089c52d5f Fix build 2020-05-01 12:42:39 +02:00
Matthew Kenigsberg d6b4047c2f rename nix run to nix shell and nix app to nix run
(cherry picked from commit 5d8504b978)
2020-05-01 12:17:36 +02:00
Eelco Dolstra 941f95284a Merge remote-tracking branch 'origin/master' into flakes 2020-05-01 11:59:56 +02:00
Ben Burdette a3030e3c31 fix error calls 2020-04-30 17:56:26 -06:00
Ben Burdette f5d3215c87 logError 2020-04-30 16:31:47 -06:00
Ben Burdette 171b4ce85c typo 2020-04-30 09:57:01 -06:00
Eelco Dolstra 0135fd6ec4
nix dev-shell: Unset shellHook
This avoids inheriting the caller's shellHook, which can happen when
running a dev-shell inside a dev-shell.
2020-04-30 14:47:56 +02:00
Eelco Dolstra efe6c186ea
nix dev-shell: Support structured attrs
Tested against https://github.com/NixOS/nixpkgs/pull/72074.

Fixes #3540.
2020-04-30 14:47:47 +02:00
Eelco Dolstra 2fcfc6c2c6
nix dev-shell: Refactor script for getting the environment 2020-04-30 13:05:29 +02:00
Domen Kožar df8e9d691c
Merge pull request #3548 from Ma27/fetchtarball-pos
Fix displaying error-position in `builtins.fetch{Tree,Tarball}`
2020-04-30 10:33:12 +02:00
Ben Burdette 39ff80d031 errorinfo constructor test 2020-04-29 18:57:05 -06:00
Matthew Kenigsberg 5d8504b978 rename nix run to nix shell and nix app to nix run 2020-04-29 15:45:10 -06:00
Maximilian Bosch d1229859c2
Fix displaying error-position in builtins.fetch{Tree,Tarball}
Without dereferencing this pointer, you'd get an error like this:

```
error: unsupported argument 'abc' to 'fetchTarball', at 0x13627e8
```
2020-04-29 22:53:39 +02:00
Guillaume Bouchard 2e5be2a749 StringSink pre allocate
When used with `readFile`, we have a pretty good heuristic of the file
size, so `reserve` this in the `string`. This will save some allocation
/ copy when the string is growing.
2020-04-29 18:44:01 +02:00
Guillaume Bouchard 7afcb5af98 Remove the drain argument from readFile
Now it is always `drain` (see previous commit).
2020-04-29 18:43:45 +02:00
Ben Burdette e2f61263eb uncrustify formatting 2020-04-29 10:14:32 -06:00
Eelco Dolstra 9570036146 nix copy: Build derivations
Fixes

  $ nix copy .#hydraJobs.vendoredCrates --to /tmp/nix
  error: path '/nix/store/...' is not valid
2020-04-29 15:51:45 +02:00
Eelco Dolstra 9c4e05766b nix copy: Move --from / --to check
This means you now get an error message *before* stuff gets built:

  $ nix copy .#hydraJobs.vendoredCrates
  error: you must pass '--from' and/or '--to'
  Try 'nix --help' for more information.
2020-04-29 15:50:59 +02:00
Antoine Eiche ca93b26db6 Only call grantpt on MacOS systems
The commit 3cc1125595 adds a `grantpt`
call on the builder pseudo terminal fd. This call is actually only
required for MacOS, but it however requires a RW access to /dev/pts
which is only RO bindmounted in the Bazel Linux sandbox. So, Nix can
not be actually run in the Bazel Linux sandbox for unneeded reasons.
2020-04-29 15:43:20 +02:00
Eelco Dolstra 70bcd6a55c Evaluation cache: Don't barf in read-only mode
Fixes

  $ nix copy
  warning: Git tree '/home/eelco/Dev/nix-flake' is dirty
  nix: src/nix/installables.cc:348: std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, nix::FlakeRef, nix::InstallableValue::DerivationInfo> nix::InstallableFlake::toDerivation(): Assertion `state->store->isValidPath(drvPath)' failed.
  Aborted (core dumped)
2020-04-29 15:42:53 +02:00
Guillaume Bouchard 5a34a473dd builtins.readFile: do not truncate content
This closes #3026 by allowing `builtins.readFile` to read a file with a
wrongly reported file size, for example, files in `/proc` may report a
file size of 0. Reading file in `/proc` is not a good enough motivation,
however I do think it just makes nix more robust by allowing more file
to be read.  Especially, I do considerer the previous behavior to be
dangerous because nix was previously reading truncated files. Examples
of file system which incorrectly report file size may be network file
system or dynamic file system (for performance reason, a dynamic file
system such as FUSE may generate the content of the file on demand).

```
nix-repl> builtins.readFile "/proc/version"
""
```

With this commit:

```
nix-repl> builtins.readFile "/proc/version"
"Linux version 5.6.7 (nixbld@localhost) (gcc version 9.3.0 (GCC)) #1-NixOS SMP Thu Apr 23 08:38:27 UTC 2020\n"
```

Here is a summary of the behavior changes:

- If the reported size is smaller, previous implementation
was silently returning a truncated file content. The new implementation
is returning the correct file content.

- If a file had a bigger reported file size, previous implementation was
failing with an exception, but the new implementation is returning the
correct file content. This change of behavior is coherent with this pull
request.

Open questions

- The behavior is unchanged for correctly reported file size, however
performances may vary because it uses the more complex sink interface.
Considering that sink is used a lot, I don't think this impacts the
performance a lot.
- `builtins.readFile` on an infinite file, such as `/dev/random` may
fill the memory.
- it does not support adding file to store, such as `${/proc/version}`.
2020-04-29 14:50:52 +02:00
Eelco Dolstra 5ada0831cf Merge remote-tracking branch 'origin/master' into flakes 2020-04-29 13:32:27 +02:00
Ben Burdette 22e6490311 Error classname as name 2020-04-28 21:06:08 -06:00
Eelco Dolstra 06849c3090
Merge pull request #3542 from mkenigs/gcroots
Set GCROOT to store path to prevent garbage collection
2020-04-28 21:04:06 +02:00
Matthew Kenigsberg 6d40fe573c rename to NIX_GCROOT 2020-04-28 11:18:54 -06:00
Eelco Dolstra 52a3ca823d Tweak warning message 2020-04-28 17:56:01 +02:00
Eelco Dolstra 6a8cba83bb Merge branch 'nix-env-warn-unmatched' of https://github.com/lheckemann/nix 2020-04-28 17:45:25 +02:00
Ben Burdette e51a757720 astyle format 2020-04-27 15:15:08 -06:00
Eelco Dolstra 6521c92ce8 Improve path:// handling
In particular, doing 'nix build /path/to/dir' now works if
/path/to/dir is not a Git tree (it only has to contain a flake.nix
file).

Also, 'nix flake init' no longer requires a Git tree (but it will do a
'git add flake.nix' if it's a Git tree)
2020-04-27 22:53:11 +02:00
Eelco Dolstra 829dcb35d5 flake-template.nix: Add defaultPackage 2020-04-27 22:52:49 +02:00
Alyssa Ross c05e20daa1
Fix long paths permanently breaking GC
Suppose I have a path /nix/store/[hash]-[name]/a/a/a/a/a/[...]/a,
long enough that everything after "/nix/store/" is longer than 4096
(MAX_PATH) bytes.

Nix will happily allow such a path to be inserted into the store,
because it doesn't look at all the nested structure.  It just cares
about the /nix/store/[hash]-[name] part.  But, when the path is deleted,
we encounter a problem.  Nix will move the path to /nix/store/trash, but
then when it's trying to recursively delete the trash directory, it will
at some point try to unlink
/nix/store/trash/[hash]-[name]/a/a/a/a/a/[...]/a.  This will fail,
because the path is too long.  After this has failed, any store deletion
operation will never work again, because Nix needs to delete the trash
directory before recreating it to move new things to it.  (I assume this
is because otherwise a path being deleted could already exist in the
trash, and then moving it would fail.)

This means that if I can trick somebody into just fetching a tarball
containing a path of the right length, they won't be able to delete
store paths or garbage collect ever again, until the offending path is
manually removed from /nix/store/trash.  (And even fixing this manually
is quite difficult if you don't understand the issue, because the
absolute path that Nix says it failed to remove is also too long for
rm(1).)

This patch fixes the issue by making Nix's recursive delete operation
use unlinkat(2).  This function takes a relative path and a directory
file descriptor.  We ensure that the relative path is always just the
name of the directory entry, and therefore its length will never exceed
255 bytes.  This means that it will never even come close to AX_PATH,
and Nix will therefore be able to handle removing arbitrarily deep
directory hierachies.

Since the directory file descriptor is used for recursion after being
used in readDirectory, I made a variant of readDirectory that takes an
already open directory stream, to avoid the directory being opened
multiple times.  As we have seen from this issue, the less we have to
interact with paths, the better, and so it's good to reuse file
descriptors where possible.

I left _deletePath as succeeding even if the parent directory doesn't
exist, even though that feels wrong to me, because without that early
return, the linux-sandbox test failed.

Reported-by: Alyssa Ross <hi@alyssa.is>
Thanks-to: Puck Meerburg <puck@puckipedia.com>
Tested-by: Puck Meerburg <puck@puckipedia.com>
Reviewed-by: Puck Meerburg <puck@puckipedia.com>
2020-04-27 20:50:17 +00:00
Matthew Kenigsberg 9e95b95a5d comment 2020-04-27 13:18:26 -06:00
Matthew Kenigsberg a3bc695e7d Set GCROOT to store path to prevent garbage collection 2020-04-27 11:22:20 -06:00
Eelco Dolstra b51dff431c Improve error message when an argument is not a flake 2020-04-27 18:55:20 +02:00
Eelco Dolstra b4e23dcd9e nix search: Search legacyPackages recursively 2020-04-27 16:29:26 +02:00
Ben Burdette 1ff42722ce error.hh 2020-04-26 14:47:41 -06:00
Ben Burdette d4fd7b543e print dashes instead of empty name string 2020-04-25 12:05:26 -06:00
Linus Heckemann f59404e1a6 nix-env: refactor uninstallDerivations
Reduces the number of store queries it performs. Also prints a warning
if any of the selectors did not match any installed derivations.

UX Caveats:
- Will print a warning that nothing matched if a previous selector
  already removed the path
- Will not do anything if no selectors were provided (no change from
  before).

Fixes #3531
2020-04-25 16:30:42 +02:00
Ben Burdette cdac083dc5 don't print blank lines for blank description 2020-04-24 21:40:13 -06:00
Ben Burdette d8d4844b88 all things error to error.hh 2020-04-24 14:57:51 -06:00
Ben Burdette d9632765a8 add has_value check; remove obslete friend class 2020-04-24 12:44:23 -06:00
Eelco Dolstra 0469795978 nix search: Show version 2020-04-24 14:42:17 +02:00
Ben Burdette 833501f6f1 'what' string 2020-04-23 15:55:34 -06:00
Eelco Dolstra ef4d3fc111 Merge remote-tracking branch 'origin/master' into flakes 2020-04-23 15:16:18 +02:00
Ben Burdette 3bc9155dfc a few more 'format's rremoved 2020-04-22 15:00:11 -06:00
Eelco Dolstra c9d0cf7e02
Don't include error.hh in util.hh to prevent header bloat 2020-04-22 15:29:27 +02:00
Eelco Dolstra 16e3bf4537
Merge branch 'error-format' of https://github.com/bburdette/nix 2020-04-22 15:29:10 +02:00
Eelco Dolstra 2ea4d45449
Path fetcher: Fix store path name
(cherry picked from commit c7af247bea)
2020-04-22 15:27:06 +02:00
Eelco Dolstra c7af247bea Path fetcher: Fix store path name 2020-04-22 10:15:32 +02:00
Dustin DeWeese c0d940978a Replace select() with poll() to allow waiting on more than FD_SETSIZE fds 2020-04-21 16:21:28 -07:00
Ben Burdette e4fb9a3849 remove 'format' from Error constructor calls 2020-04-21 17:07:07 -06:00
Ben Burdette d3052197fe add ErrorInfo to BaseError 2020-04-21 13:25:41 -06:00
Eelco Dolstra 8c75621da6 Fix typo 2020-04-20 15:28:56 +02:00
Eelco Dolstra b69323f8c9 Revive 'nix search'
It uses the evaluation cache now rather than the ad hoc JSON cache.
2020-04-20 15:27:09 +02:00
Eelco Dolstra 42a12f9232 Move eval-cache.{cc,hh} 2020-04-20 13:14:59 +02:00
Eelco Dolstra 539a9c1c5f Get rid of the old eval cache 2020-04-20 13:13:52 +02:00
Ben Burdette 15e9564fd1 logEI for tunnelLogger and progressbar 2020-04-19 17:16:51 -06:00
Eelco Dolstra 0725ab2fd7 Store more stuff in the evaluation cache
In particular, we store whether an attribute failed to evaluate (threw
an exception) or was an unsupported type. This is to ensure that a
repeated 'nix flake show' never has to evaluate anything, so it can
execute without fetching the flake.

With this, 'nix flake show nixpkgs/nixos-20.03 --legacy' executes in
0.6s (was 3.4s).
2020-04-19 23:07:06 +02:00
Eelco Dolstra 3738bcb05e Eval cache: Don't replace real attributes with placeholders 2020-04-18 15:12:31 +02:00
Domen Kožar 25ed842725
Merge pull request #3502 from NixOS/more-pos
pass Pos to forceValue to improve infinite recursion error
2020-04-18 14:05:21 +02:00
Ben Burdette 4697552948 demoing other error levels than warn/error; rename line and file fields in errPos 2020-04-17 15:50:46 -06:00
Eelco Dolstra 69cb9f7eee Wrap eval cache creation in a giant transaction
This speeds up the creation of the cache for the nixpkgs flake from
21.2s to 10.2s. Oddly, it also speeds up querying the cache
(i.e. running 'nix flake show nixpkgs/nixos-20.03 --legacy') from 4.2s
to 3.4s.

(For comparison, running with --no-eval-cache takes 9.5s, so the
overhead of building the SQLite cache is only 0.7s.)
2020-04-17 23:17:21 +02:00
Ben Burdette 3d5b1032a1 logError, logWarning; Logger functions; switch to Verbosity enum 2020-04-17 15:07:44 -06:00
Eelco Dolstra aaa109565e Use a more space/time-efficient representation for the eval cache 2020-04-17 23:04:21 +02:00
Eelco Dolstra bdb3226607 Add flag to disable the eval cache 2020-04-17 14:30:04 +02:00
Eelco Dolstra aa34c0ef51 nix flake show: Speed up eval cache bigly
In the fully cached case for the 'nixpkgs' flake, it went from 101s to
4.6s. Populating the cache went from 132s to 17.4s (which could
probably be improved further by combining INSERTs).
2020-04-17 13:57:02 +02:00
Eelco Dolstra 9ea4f93f88 nix flake show: Support apps 2020-04-17 01:21:24 +02:00
Eelco Dolstra a6c4fd044c Hide progress bar on exit 2020-04-17 01:13:13 +02:00
Eelco Dolstra 12b7eefbc5 nix flake show: Use evaluation cache 2020-04-17 01:02:29 +02:00
Eelco Dolstra 7a9687ba30 SQLiteStmt: Use std::string_view 2020-04-17 01:00:56 +02:00
Eelco Dolstra 3b489e8843 Add 'nix flake show' command 2020-04-16 19:52:39 +02:00
Eelco Dolstra 29043e7e9e Fix 2020-04-16 19:01:49 +02:00
Ben Burdette 12814806ef iomanip no longer needed 2020-04-16 10:48:15 -06:00
Eelco Dolstra c277231b7d Use RootValue 2020-04-16 18:33:34 +02:00
Eelco Dolstra f89349f07e Merge remote-tracking branch 'origin/master' into flakes 2020-04-16 18:33:10 +02:00
Eelco Dolstra 0858738355 Merge remote-tracking branch 'origin/master' into flakes 2020-04-16 18:27:37 +02:00
Eelco Dolstra efaffaa9d1 Use Logger::stdout()
(cherry picked from commit 8f41847394)
2020-04-16 18:14:01 +02:00
Eelco Dolstra 67a5941472 Logger: Add method for writing to stdout
Usually this just writes to stdout, but for ProgressBar, we need to
clear the current line, write the line to stdout, and then redraw the
progress bar.

(cherry picked from commit 696c026006)
2020-04-16 18:03:38 +02:00
Eelco Dolstra fcd048a526 Use RootValue 2020-04-16 18:02:59 +02:00
Ben Burdette 96262e744e switch to structs, which don't need public: 2020-04-16 09:55:38 -06:00
Eelco Dolstra 9f46f54de4 JSONSax: Use a RootValue
More #3377.
2020-04-16 17:30:18 +02:00
Eelco Dolstra 10e17eaa58 ValueMap, VectorVector: Use traceable_allocator
We want to *trace* the 'Value *' arrays, not garbage-collect them!
Otherwise the vectors/maps can end up pointing to nowhere.

Fixes #3377. Closes #3384.
2020-04-16 17:30:13 +02:00
Eelco Dolstra b3e5eea4a9 Add function to allocate a Value in traceable memory 2020-04-16 17:30:05 +02:00
Eelco Dolstra 1290411c2d fetchMercurial: Use inputFromAttrs() 2020-04-16 17:29:30 +02:00
Eelco Dolstra 8f41847394 Use Logger::stdout() 2020-04-16 13:47:59 +02:00
Eelco Dolstra 696c026006 Logger: Add method for writing to stdout
Usually this just writes to stdout, but for ProgressBar, we need to
clear the current line, write the line to stdout, and then redraw the
progress bar.
2020-04-16 13:47:59 +02:00
Domen Kožar b865b5b40c
pass Pos to forceValue to improve infinite recursion error 2020-04-16 12:32:07 +02:00
Ben Burdette 057e5b6b2e move implementation to cc 2020-04-15 10:09:43 -06:00
Ben Burdette adf03b0b8e Merge branch 'initializer-style' into error-format 2020-04-15 10:06:20 -06:00
Eelco Dolstra 2f9789c2e6
Merge pull request #3492 from andir/nix-build-gc-free
SourceExprCommand: allocate the vSourceExpr via uncollectable memory
2020-04-15 13:01:04 +02:00
Eelco Dolstra a118293bd0
Merge pull request #3458 from zimbatm/nix-user-conf-dir
NIX_USER_CONF_FILES
2020-04-15 13:00:28 +02:00
Jonas Chevalier 895516cadf
add NIX_USER_CONF_FILES
Motivation: maintain project-level configuration files.

Document the whole situation a bit better so that it corresponds to the
implementation, and add NIX_USER_CONF_FILES that allows overriding
which user files Nix will load during startup.
2020-04-14 18:45:06 +02:00
Eelco Dolstra 3729df34da Make Registry::read() more robust 2020-04-14 17:25:39 +02:00
Eelco Dolstra c0c2cb871d Merge remote-tracking branch 'origin/master' into flakes 2020-04-14 13:02:55 +02:00
Andreas Rammhold d2c371927e SourceExprCommand: allocate the vSourceExpr via uncollectable memory
Previously the memory would occasionally be collected during eval since
the GC doesn't consider the member variable as alive / doesn't scan the
region of memory where the pointer lives.

By using the traceable_allocator<T> allocator provided by Boehm GC we
can ensure the memory isn't collected. It should be properly freed when
SourceExprCommand goes out of scope.
2020-04-13 21:23:54 +02:00
Daiderd Jordan 4d9db420ff
never use /var/folders for TMPDIR on darwin
This doesn't just cause problems for nix-store --serve but also results
in certain build failures. Builds that use unix domain sockets in their
tests often fail because the /var/folders prefix already consumes more
than half of the maximum length of socket paths.

    struct sockaddr_un {
       sa_family_t sun_family;               /* AF_UNIX */
       char        sun_path[108];            /* Pathname */
    };
2020-04-12 09:57:22 +02:00
Domen Kožar ea2148f47c
Merge pull request #2688 from tollb/fix/build_check_keep_failed_sandbox_perms
Fix nix-build --check -K in sandbox w/o root
2020-04-11 20:03:17 +02:00
Domen Kožar fc144242d5
Merge pull request #3447 from DavHau/improve-tofile-error-msg
improve toFile error message when containing potential drv path
2020-04-11 13:12:11 +02:00
DavHau fc14213d2d improve toFile error message when containing potential drv path 2020-04-11 10:54:26 +00:00
Bruce Toll 8132d0a12e Fix nix-build --check -K in sandbox w/o root
Temporarily add user-write permission to build directory so that it
can be moved out of the sandbox to the store with a .check suffix.

This is necessary because the build directory has already had its
permissions set read-only, but write permission is required
to update the directory's parent link to move it out of the sandbox.

Updated the related --check "derivation may not be deterministic"
messages to consistently use the real store paths.

Added test for non-root sandbox nix-build --check -K to demonstrate
issue and help prevent regressions.
2020-04-10 16:23:10 -04:00
Eelco Dolstra e5ea01c1a8
Remove flake 'edition' field
Future editions of flakes or the Nix language can be supported by
renaming flake.nix (e.g. flake-v2.nix). This avoids a bootstrap
problem where we don't know which grammar to use to parse
flake*.nix. It also allows a project to support multiple flake
editions, in theory.
2020-04-10 10:24:09 +02:00
Domen Kožar db25a6d7bb
Merge pull request #2689 from tollb/fix/delete_tmp_dir_when_build_check_ok
Delete temporary directory on successful build
2020-04-10 09:47:57 +02:00
Bruce Toll 16a4864759 Delete temporary directory on successful build
With --check and the --keep-failed (-K) flag, the temporary directory
was being retained regardless of whether the build was successful and
reproducible.  This removes the temporary directory, as expected, on
a reproducible check build.

Added tests to verify that temporary build directories are not
retained unnecessarily, particularly when using --check with
--keep-failed.
2020-04-09 16:37:41 -04:00
Philipp Middendorf 04bedda0b6 gc.cc: Ignore hidden files in temproots 2020-04-09 17:05:29 +02:00
Eelco Dolstra 3aaceeb7e2
Fix build 2020-04-09 16:00:43 +02:00
Eelco Dolstra d103c79144
Merge remote-tracking branch 'origin/master' into flakes 2020-04-09 15:54:21 +02:00
Domen Kožar a693a9fa4b
Attach pos to if expression errors 2020-04-09 09:45:15 +02:00
Nikola Knezevic 7867685dcd after flake rebase 2020-04-08 22:26:57 +02:00
Nikola Knezevic f5095594e7 datatransfer.{cc,hh} -> filetransfer.{cc,hh} 2020-04-08 22:26:57 +02:00
Nikola Knezevic c330109bfa DataTransfer -> FileTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic 7848372b0f Add upload method 2020-04-08 22:26:57 +02:00
Nikola Knezevic a0c5931208 actDownload -> actDataTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic c4c1ae0a00 DownloadError -> DataTransferError 2020-04-08 22:26:57 +02:00
Nikola Knezevic 213d124277 DownloadItem -> TransferItem 2020-04-08 22:26:57 +02:00
Nikola Knezevic cd391206e6 {get,make,new}Downloader -> DataTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic 2df2741ec6 enqueueDownload -> enqueueDataTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic 142ed7fe45 DownloadResult -> DataTransferResult 2020-04-08 22:26:57 +02:00
Nikola Knezevic e5cc53beec DownloadSettings -> DataTransferSettings 2020-04-08 22:26:57 +02:00
Nikola Knezevic 741e9012d3 Rename src/lib/download.* to src/lib/datatransfer.* 2020-04-08 22:26:57 +02:00
Nikola Knezevic 65ef57e0cb DownloadRequest -> DataTransferRequest 2020-04-08 22:26:57 +02:00
Eelco Dolstra 5449ff7d8a
Merge pull request #3477 from Ninlives/nix-run-using-env
`nix run` using $SHELL as default command
2020-04-08 19:35:39 +02:00
Ben Burdette 805ffe1bc9 indention 2020-04-08 11:33:46 -06:00
Ben Burdette 8c2bf15c4f format -> fmt 2020-04-08 11:17:02 -06:00
Domen Kožar bf81b31559
build.cc: improve message if home directory exists 2020-04-08 18:27:10 +02:00
Ben Burdette 555baa8fb0 comments 2020-04-08 09:56:10 -06:00
Ben Burdette 54f91923c8 return of NixCode 2020-04-08 09:48:21 -06:00
Eelco Dolstra 96f3c36709
Merge pull request #3478 from edolstra/ignore-failed-data
Downloader: Only write data to the sink on a 200 response
2020-04-08 17:15:53 +02:00
Ben Burdette 47ed067d45 initializer style 2020-04-08 09:07:58 -06:00
Eelco Dolstra 9ed097db7b
Merge pull request #3468 from Infinisil/functionArgsPositions
Make function arguments retain position info
2020-04-08 15:29:39 +02:00
Eelco Dolstra 1ab8d6ac18
Downloader: Only write data to the sink on a 200 response
Hopefully fixes #3278.
2020-04-08 15:27:09 +02:00
Ben Burdette 00c507cc52 columnRange -> column 2020-04-07 14:36:32 -06:00
Ben Burdette 20c0984a46 remove columnrange; switch to fmt in error.cc 2020-04-07 10:14:15 -06:00
Eelco Dolstra 03a4a3c95c Merge branch 'flakes' of github.com:NixOS/nix into flakes 2020-04-07 14:30:25 +02:00
Eelco Dolstra 54955867a6 Merge remote-tracking branch 'origin/master' into flakes 2020-04-07 14:29:45 +02:00
mlatus 9d04b5da17 nix run using $SHELL as default command 2020-04-07 20:29:40 +08:00
Eelco Dolstra 55cefd41d6 Merge branch 'fetchgit-recursive' of https://github.com/blitz/nix 2020-04-07 13:45:17 +02:00
Eelco Dolstra cd39709003 Cleanup 2020-04-07 09:27:17 +02:00
Eelco Dolstra 26aeeb7653 Add FIXME
(cherry picked from commit 2f494531b7)
2020-04-07 09:09:43 +02:00
Eelco Dolstra a6dfa3cb85 PathInput: Add some methods
(cherry picked from commit 78ad5b3d91)
2020-04-07 09:09:43 +02:00
Eelco Dolstra f58a9b0e62 Respect the narHash attribute in more input types
(cherry picked from commit a6ff66b658)
2020-04-07 09:09:18 +02:00
Eelco Dolstra 670feb000a Add 'path' fetcher
This fetchers copies a plain directory (i.e. not a Git/Mercurial
repository) to the store (or does nothing if the path is already a
store path).

One use case is to pin the 'nixpkgs' flake used to build the current
NixOS system, and prevent it from being garbage-collected, via a
system registry entry like this:

  {
      "from": {
          "id": "nixpkgs",
          "type": "indirect"
      },
      "to": {
          "type": "path",
          "path": "/nix/store/rralhl3wj4rdwzjn16g7d93mibvlr521-source",
          "lastModified": 1585388205,
          "rev": "b0c285807d6a9f1b7562ec417c24fa1a30ecc31a"
      },
      "exact": true
  }

Note the fake "lastModified" and "rev" attributes that ensure that the
flake gives the same evaluation results as the corresponding
Git/GitHub inputs.

(cherry picked from commit 12f9379123)
2020-04-07 09:08:51 +02:00
Eelco Dolstra 462421d345 Backport libfetchers from the flakes branch
This provides a pluggable mechanism for defining new fetchers. It adds
a builtin function 'fetchTree' that generalizes existing fetchers like
'fetchGit', 'fetchMercurial' and 'fetchTarball'. 'fetchTree' takes a
set of attributes, e.g.

  fetchTree {
    type = "git";
    url = "https://example.org/repo.git";
    ref = "some-branch";
    rev = "abcdef...";
  }

The existing fetchers are just wrappers around this. Note that the
input attributes to fetchTree are the same as flake input
specifications and flake lock file entries.

All fetchers share a common cache stored in
~/.cache/nix/fetcher-cache-v1.sqlite. This replaces the ad hoc caching
mechanisms in fetchGit and download.cc (e.g. ~/.cache/nix/{tarballs,git-revs*}).

This also adds support for Git worktrees (c169ea5904).
2020-04-07 09:03:14 +02:00
Ben Burdette 55c96b64e4 comment cleanup 2020-04-06 20:14:48 -06:00
Ben Burdette ec449c8450 constructor style basically working 2020-04-06 19:43:22 -06:00
Ben Burdette 85f14c4582 add libutil, libexpr include dirs 2020-04-06 11:15:01 -06:00
Ben Burdette 216263c36f Merge branch 'master' into error-format 2020-04-06 10:00:00 -06:00
Eelco Dolstra 68b43e01dd
nix flake info: Show resolved URL
This is useful for finding out what a registry lookup resolves to, e.g

  $ nix flake info patchelf
  Resolved URL:  github:NixOS/patchelf
  Locked URL:    github:NixOS/patchelf/cd7955af31698c571c30b7a0f78e59fd624d0229
2020-04-06 14:56:13 +02:00
Eelco Dolstra ce3173edc1
nix flake info --json: Don't evaluate
This makes its behaviour consistent with the non-json
variant. Querying the outputs should be done by another command
(e.g. 'nix search')
2020-04-06 14:39:47 +02:00
Eelco Dolstra 2f494531b7
Add FIXME 2020-04-06 14:28:37 +02:00
Cole Helbling c976cb0b8a
Don't retry on "unsupported protocol" error
When encountering an unsupported protocol, there's no need to retry.
Chances are, it won't suddenly be supported between retry attempts;
error instead. Otherwise, you see something like the following:

    $ nix-env -i -f git://git@github.com/foo/bar
    warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 335 ms
    warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 604 ms
    warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 1340 ms
    warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 2685 ms

With this change, you now see:

    $ nix-env -i -f git://git@github.com/foo/bar
    error: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1)
2020-04-05 09:00:34 -07:00
Ben Burdette 1221ae3dd0 libexpr 2020-04-05 07:12:16 -06:00
Ben Burdette 9a8b3e9747 move out of tests/ 2020-04-03 14:55:26 -06:00
Ben Burdette 9bb528d392 handle Pos instead of individual file/line/columnrange args 2020-04-03 13:15:59 -06:00
Eelco Dolstra 63fa92605b nix-env: Refuse to operate on a new-style profile
This prevents users from accidentally nuking their profile via
nix-env.

(cherry picked from commit 021634e3e3)
2020-04-03 20:14:34 +02:00
Ben Burdette 7b7801d3f0 variadic args for hint format 2020-04-03 08:48:20 -06:00
Eelco Dolstra 6e7f252ea6 Make --override-input sticky
When we do something like 'nix flake update --override-input nixpkgs
...', the override is now kept on subsequent calls. (If you don't want
this behaviour, you can use --no-write-lock-file.)
2020-04-03 14:06:29 +02:00
Eelco Dolstra 9c78f7f196 getFlake: In pure mode, check that the argument is an immutable flakeref 2020-04-03 13:07:05 +02:00
Ben Burdette c6b3fcddb0 formatted with astyle 2020-04-02 16:02:40 -06:00
Ben Burdette 1c329ca433 indenting 2020-04-02 14:25:43 -06:00
Eelco Dolstra 485a87f22f Don't barf on registry parse errors 2020-04-02 19:09:17 +02:00
Eelco Dolstra a6ff66b658 Respect the narHash attribute in more input types
call-flake.nix now passes node.info.narHash to fetchTree. This ensures
that dirty Git trees work even in pure mode.
2020-04-02 19:04:33 +02:00
Eelco Dolstra 78ad5b3d91 PathInput: Add some methods 2020-04-02 19:04:27 +02:00
Eelco Dolstra e1fc9f6690 Improve error message 2020-04-02 19:03:32 +02:00
Eelco Dolstra ab47868639 Change lastModified to the number of seconds in the epoch
'lastModifiedDate' is now a string representing the equivalent
date/time.
2020-04-02 18:39:41 +02:00
Eelco Dolstra 6d6467d376 Move parseTreeInfo() 2020-04-02 18:26:39 +02:00
Andreas Rammhold 4fc4eb6c93 libexpr: remove unused attrError
The attrError variable is no longer used but still allocated on every
call to the findAlongAttrPath function.
2020-04-02 17:04:00 +02:00
Eelco Dolstra 6cf91d6fbd fetchTree: Support integer attributes 2020-04-02 16:51:56 +02:00
Eelco Dolstra 12f9379123 Add 'path' fetcher
This fetchers copies a plain directory (i.e. not a Git/Mercurial
repository) to the store (or does nothing if the path is already a
store path).

One use case is to pin the 'nixpkgs' flake used to build the current
NixOS system, and prevent it from being garbage-collected, via a
system registry entry like this:

  {
      "from": {
          "id": "nixpkgs",
          "type": "indirect"
      },
      "to": {
          "type": "path",
          "path": "/nix/store/rralhl3wj4rdwzjn16g7d93mibvlr521-source",
          "lastModified": 1585388205,
          "rev": "b0c285807d6a9f1b7562ec417c24fa1a30ecc31a"
      },
      "exact": true
  }

Note the fake "lastModified" and "rev" attributes that ensure that the
flake gives the same evaluation results as the corresponding
Git/GitHub inputs.
2020-04-02 14:56:20 +02:00
Eelco Dolstra 00e1400eb7 Doh 2020-04-02 11:55:41 +02:00
Eelco Dolstra 021634e3e3 nix-env: Refuse to operate on a new-style profile
This prevents users from accidentally nuking their profile via
nix-env.
2020-04-02 11:54:48 +02:00
Eelco Dolstra ed13457dbf nix flake info --json: Show TreeInfo 2020-04-02 11:51:34 +02:00
Silvan Mosberger c34e96f7e0
Make function arguments retain position info
This allows querying the location of function arguments. E.g.

  builtins.unsafeGetAttrPos "x" (builtins.functionArgs ({ x }: null))

  => { column = 57; file = "/home/infinisil/src/nix/inst/test.nix"; line = 1; }
2020-04-02 05:52:52 +02:00
Ben Burdette e697884f65 using std:: everywhere; fix a formatting error; add exception flags 2020-04-01 21:30:19 -06:00
Ben Burdette dd7b8183a5 indenting 2020-04-01 16:20:20 -06:00
Ben Burdette 8713aeac5e remove using std::*, switch to include guard 2020-04-01 15:51:14 -06:00
Eelco Dolstra 74024515a3 Support registry entries that must match exactly
An example use is for pinning the "nixpkgs" entry the system-wide
registry to a particular store path. Inexact matches
(e.g. "nixpkgs/master") should still use the global registry.
2020-04-01 23:12:45 +02:00
Eelco Dolstra bd10a07d17 Registry: Use a struct instead of a tuple for entries 2020-04-01 23:03:27 +02:00
Eelco Dolstra 77ffaea4fa Add a system-wide flake registry /etc/nix/registry.json
One application for this is pinning the 'nixpkgs' flake to the exact
revision used to build the NixOS system, e.g.

  {
      "flakes": [
          {
              "from": {
                  "id": "nixpkgs",
                  "type": "indirect"
              },
              "to": {
                  "owner": "NixOS",
                  "repo": "nixpkgs",
                  "type": "github",
                  "rev": "b0c285807d6a9f1b7562ec417c24fa1a30ecc31a"
              }
          }
      ],
      "version": 2
  }
2020-04-01 22:56:50 +02:00
Eelco Dolstra 4fdec5f61d Merge remote-tracking branch 'origin/master' into flakes 2020-04-01 00:03:50 +02:00
Eelco Dolstra 03b56e96bf Typo 2020-03-31 23:55:07 +02:00
Ben Burdette 5b3aefff85 add some explanatory comments 2020-03-31 12:42:41 -06:00
Ben Burdette 9e7b89bf10 rename errors/warnings 2020-03-31 11:56:37 -06:00
Ben Burdette 09652f597c enum style 2020-03-31 09:36:20 -06:00
Eelco Dolstra a7540294cf
Merge pull request #3460 from NixOS/dev-shell
Backport 'nix dev-shell' from the flakes branch
2020-03-31 14:46:15 +02:00
Eelco Dolstra 3166b97174 nix shell -> nix dev-shell 2020-03-31 13:45:28 +02:00
mlatus 12556e5709 fix placeholder not substituted in passAsFile 2020-03-31 19:40:16 +08:00
John Ericson 7e9a2718f0 s/outputHashRecursive/ingestionMethod/c 2020-03-30 22:36:15 +00:00
John Ericson 51afea3af2 Never cast FileIngestionMethod to or from boolean 2020-03-30 22:31:51 +00:00
John Ericson c251b011cd Merge remote-tracking branch 'upstream/master' into enum-FileIngestionMethod 2020-03-30 18:16:44 -04:00
John Ericson bbbb7c1bc7 Use auto with some FileIngestionMethod local variables 2020-03-30 18:15:55 -04:00
Eelco Dolstra e1a94ad852 Backport 'nix dev-shell' from the flakes branch
This also adds a '--profile' option to 'nix build' (replacing 'nix-env
--set').
2020-03-30 19:16:45 +02:00
John Ericson 832bd534dc Store parsed hashes in DerivationOutput
It's best to detect invalid data as soon as possible, with data types
that make storing it impossible.
2020-03-30 11:33:35 -04:00
Ben Burdette 35c7bab09a build with make 2020-03-30 09:14:29 -06:00
John Ericson f5494d9442 Merge remote-tracking branch 'me/enum-FileIngestionMethod' into HEAD 2020-03-30 11:08:13 -04:00
Eelco Dolstra 367577d9a6 Fix macOS build 2020-03-30 17:00:40 +02:00
Eelco Dolstra d15d91cad1 Makefile cleanup 2020-03-30 16:50:13 +02:00
Eelco Dolstra e322a16523 Remove global -I flags
(cherry picked from commit 2c692a3b14)
2020-03-30 15:30:19 +02:00
Eelco Dolstra 2c692a3b14 Remove global -I flags 2020-03-30 14:39:33 +02:00
Eelco Dolstra e0a0ae0467 Move fetchers from libstore to libfetchers 2020-03-30 14:04:53 +02:00
Eelco Dolstra 2287cc6486 Fix segfault 2020-03-30 13:31:55 +02:00
Julian Stecklina 40c023ecfe fetchGit: don't use std::filesystem to filter git repos
Using std::filesystem means also having to link with -lstdc++fs on
some platforms and it's hard to discover for what platforms this is
needed. As all the functionality is already implemented as utilities,
use those instead.
2020-03-30 00:32:42 +02:00
Bjørn Forsman f686efeed4 fetchGit: fix submodule corner case by fetching all refs from cacheDir
Due to fetchGit not checking if rev is an ancestor of ref (there is even
a FIXME comment about it in the code), the cache repo might not have the
ref even though it has the rev. This doesn't matter when submodule =
false, but the submodule = true code blows up because it tries to fetch
the (missing) ref from the cache repo.

Fix this in the simplest way possible: fetch all refs from the local
cache repo when submodules = true.

TODO: Add tests.
2020-03-29 22:29:58 +02:00
Bjørn Forsman cc522d0d23 fetchGit: fix submodules = true for dirty trees 2020-03-29 22:29:58 +02:00
Bjørn Forsman b306b7039e fetchGit: checkout rev instead of latest ref
Major bugfix for the submodules = true code path.

TODO: Add tests.
2020-03-29 22:29:58 +02:00
Bjørn Forsman 369fffd6f1 fetchGit: add submodules attribute to the .link file
The .link file is used as a lock, so I think we should put the
"submodule" attribute in there since turning on submodules creates a new
.link file path.
2020-03-29 22:29:58 +02:00
Bjørn Forsman 6864ad7cf5 fetchGit: fix submodule output attribute
Before this change it would be false for all evaluations but the first.
Now it follows the input argument (as it should).
2020-03-29 22:29:58 +02:00
Bjørn Forsman 002a3a95dc fetchGit: fix "fatal: couldn't find remote ref refs/heads/master" issue with submodules 2020-03-29 22:29:58 +02:00
Julian Stecklina cc4fe977e5 Link to stdc++fs
Some platforms seem to still require linking with stdc++fs to enable
STL std::filesystem support.
2020-03-29 22:29:58 +02:00
Julian Stecklina 435366ed3c Rename fetchGit fetchSubmodules to just submodules 2020-03-29 22:29:58 +02:00
Julian Stecklina c8d33de777 Add git submodule fixes from @bjornfor
This fixes fetching repositories with no submodules and also cleans up
.git files in checkouts.
2020-03-29 22:29:57 +02:00
Julian Stecklina f58604ac32 Add fetchSubmodules to builtins.fetchGit
There are some downsides to this features:

 - Submodules are not cached (unlike the root repo),
 - Full checkouts are created in a temporary directory.
2020-03-29 22:29:57 +02:00
John Ericson 225e62a56a Replace some bool recursive with a new FileIngestionMethod enum 2020-03-29 15:16:20 -04:00
John Ericson 87b32bab05 Use enum struct and drop prefixes
This does a few enums; the rest will be gotten in subsequent commits.
2020-03-29 11:23:15 -04:00
Eelco Dolstra 4989c04dd2 nix flake info --json: Dump attr-style flakeref 2020-03-28 22:59:38 +01:00
Eelco Dolstra 2fccef0c59 Warn about --override-input / --update-input flags that don't match an input 2020-03-28 19:09:36 +01:00
Eelco Dolstra 2287e2f279 nix flake info: Show flake subdirectory 2020-03-28 18:05:50 +01:00
Eelco Dolstra 015f8f1c13 Improve lock file generation
This is now done in a single pass. Also fixes some issues when
updating flakes with circular dependencies. Finally, when using
'--recreate-lock-file --commit-lock-file', the commit message now
correctly shows the differences.
2020-03-27 21:08:41 +01:00
Ben Burdette 759f39800b remove util.hh from deps 2020-03-27 10:55:09 -06:00
Ben Burdette 00eb3fcb7a more cleanup 2020-03-27 10:13:46 -06:00
Ben Burdette a3ef00be6c camelcase; optional hint 2020-03-27 10:03:02 -06:00
Eelco Dolstra 3fa1e7dace Fix diffLockFiles() 2020-03-27 16:15:50 +01:00
John Ericson e433d4af4c Extend Rust FFI
Do idiomatic C++ copy and move constructors for a few things, so
wrapping structs' defaults can work.
2020-03-25 16:12:14 -04:00
Ben Burdette d44c9c5581 some colors 2020-03-25 11:20:44 -06:00
Ben Burdette 3582dc3c88 programName as static member var 2020-03-25 10:52:03 -06:00
John Ericson bcde5456cc Flip dependency so store-api.hh includes derivations.hh
I think it makes more sense to define the data model (derivations),
before the operations (store api).
2020-03-24 20:39:45 +00:00
Ben Burdette fc310eda3a switch to one level of builder function, not subobject functions 2020-03-24 14:24:57 -06:00
Ben Burdette 0166e7ab6d MkNixCode, MkErrLine approach 2020-03-24 11:21:35 -06:00
Ben Burdette 4171ab4bbd renaming 2020-03-24 09:18:23 -06:00
Eelco Dolstra 0a10854f85 Misc changes from the flakes branch 2020-03-24 14:34:47 +01:00
Eelco Dolstra c85097da7c Fix --refresh with --no-net
https://hydra.nixos.org/build/110879699
(cherry picked from commit 5bbe793abf)
2020-03-24 14:26:23 +01:00
Eelco Dolstra 6b824c78f1 nix: Add --refresh as an alias for --tarball-ttl 0
(cherry picked from commit e721f99817)
2020-03-24 14:26:23 +01:00
Eelco Dolstra 777e21e596 nix path-info --json: Print hash in SRI format
(cherry picked from commit 442e665d6d)
2020-03-24 14:26:23 +01:00
Eelco Dolstra 7a8de57d3e Pretty-print 'nix why-depends' / 'nix-store -q --tree' output
Extracted from 678301072f.
2020-03-24 14:26:23 +01:00
Eelco Dolstra 4260a22a55 absPath(): Use std::optional
(cherry picked from commit 1bf9eb21b7)
2020-03-24 14:25:28 +01:00
Eelco Dolstra f9611c7ae4 buildenv: Eliminate global variables, other cleanup
(cherry picked from commit b82f75464d)
2020-03-24 14:06:47 +01:00
Eelco Dolstra 76e7d958ed Fix coverage build
https://hydra.nixos.org/build/110757285
(cherry picked from commit b430a81a1f)
2020-03-24 14:06:47 +01:00
Eelco Dolstra 231a8aa2c2 nix edit: Support non-derivation attributes
E.g.

  $ nix edit .#nixosConfigurations.bla

now works.

(cherry picked from commit d2032edb2f)
2020-03-24 14:06:47 +01:00
Eelco Dolstra c1ca4f0acc findAlongAttrPath(): Return position
(cherry picked from commit 0b013a54dc)
2020-03-24 14:06:47 +01:00
Eelco Dolstra 1eb952d27a findAlongAttrPath(): Throw AttrPathNotFound
(cherry picked from commit 6b0ca8e803)
2020-03-24 14:06:47 +01:00
Eelco Dolstra edc34cc1a2 Add function for quoting strings
(cherry picked from commit 7dcf5b011a)
2020-03-24 13:44:04 +01:00
Eelco Dolstra 5a7e7fc35f Use std::string_view
(cherry picked from commit 6529490cc1)
2020-03-24 13:26:37 +01:00
Eelco Dolstra c34a20e1f6 EvalState::allocAttr(): Add convenience method
(cherry picked from commit c02da99757)
2020-03-24 13:26:37 +01:00
Ben Burdette aadd59d005 error test 2020-03-23 15:29:49 -06:00
Eelco Dolstra 7abe3bde8a Merge remote-tracking branch 'origin/master' into flakes 2020-03-23 13:27:54 +01:00
Eelco Dolstra 5885e20404
Merge pull request #3429 from LnL7/darwin-sandbox
darwin sandbox
2020-03-23 09:13:36 +01:00
Ben Burdette f694f43d7d straightforward port of rust mockup code 2020-03-22 12:25:47 -06:00
Daiderd Jordan 2e9bc1245c
sandbox: fix /bin/sh on catalina
Sadly 10.15 changed /bin/sh to a shim which executes bash, this means it
can't be used anymore without also opening up the sandbox to allow bash.

    Failed to exec /bin/bash as variant for /bin/sh (1: Operation not permitted).
2020-03-20 22:12:30 +01:00
Daiderd Jordan f6c122aaeb
sandbox: allow pty devices
Nix now runs builds with a pseudo-terminal to enable colored build
output.
2020-03-20 21:58:45 +01:00
Daiderd Jordan 7f2df903d9
libstore: relax default sandbox-paths on darwin 2020-03-20 21:31:20 +01:00
Daiderd Jordan afb78ebd34
libstore: disable resolve-system-dependencies hook
This is used to determine the dependency tree of impure libraries so nix
knows what paths to open in the sandbox.  With the less restrictive
defaults it isn't needed anymore.
2020-03-20 21:21:56 +01:00
Eelco Dolstra 1537e270fb Merge remote-tracking branch 'origin/master' into flakes 2020-03-20 13:15:11 +01:00
John Ericson e317324236
Apply suggestions from code review 2020-03-19 23:38:51 -04:00
John Ericson d5b3328dd1
Apply suggestions from code review
Co-Authored-By: Cole Helbling <cole.e.helbling@outlook.com>
2020-03-19 23:37:52 -04:00
jakobrs c5a488afc0 Remove the --delete option for --gc
Running `nix-store --gc --delete` will, as of Nix 2.3.3, simply fail
because the --delete option conflicts with the --delete operation.

  $ nix-store --gc --delete
  error: only one operation may be specified
  Try 'nix-store --help' for more information.

Furthermore, it has been broken since at least Nix 0.16 (which was
released sometime in 2010), which means that any scripts which depend
on it should have been broken at least nine years ago. This commit
simply formally removes the option. There should be no actual difference
in behaviour as far as the user is concerned: it errors with the exact
same error message. The manual has been edited to remove any references
to the (now gone) --delete option.

Other information:
* Path for Nix 0.16 used:
  /nix/store/rp3sgmskn0p0pj1ia2qwd5al6f6pinz4-nix-0.16
2020-03-19 19:41:23 +01:00
John Ericson f1cf3ab870 hashDerivationModulo: Generalize for multiple fixed ouputs per drv
See documentattion in header and comments in implementation for details.

This is actually done in preparation for floating ca derivations, not
multi-output fixed ca derivations, but the distinction doesn't yet
mattter.

Thanks @cole-h for finding and fixing a bunch of typos.
2020-03-19 10:30:49 -04:00
Eelco Dolstra ef74fafc03
nix repl: Put EvalState on the heap
See 0629601da1.
2020-03-19 13:52:28 +01:00
Eelco Dolstra b244e65cdb
nix repl: Scan NixRepl for GC roots
Fixes #3175.
2020-03-19 13:50:01 +01:00
Eelco Dolstra 8aa354fdfd
Register flake-registry.json as a GC root again 2020-03-19 11:45:34 +01:00
Eelco Dolstra 1c127e6a82
downloadFile(): Use expired file if the download fails 2020-03-19 11:42:50 +01:00
John Ericson 049179ba07
Fix typos
Thanks @asymmetric

I failed to do them all in one batch

Co-Authored-By: asymmetric <lorenzo@mailbox.org>
2020-03-18 19:07:05 -04:00
Eelco Dolstra f6ddf48882
Get rid of downloadCached()
Everything uses the generic caching system now.
2020-03-18 17:24:19 +01:00
Eelco Dolstra c5ec95e2c7
tarball.cc: Use ETags 2020-03-18 15:14:23 +01:00
Eelco Dolstra 1b49479836
Remove flake closure caching
This is not compatible with lazy flake input fetching.
2020-03-18 14:11:58 +01:00
Eelco Dolstra 1e7ce1d6da
tarball / github fetchers: Use generic caching system 2020-03-18 14:08:25 +01:00
Eelco Dolstra 38e360154d
Git: Use unified caching system 2020-03-17 22:35:29 +01:00
Eelco Dolstra d1165d8791
Require shallow clones to be requested explicitly
If you do a fetchTree on a Git repository, whether the result contains
a revCount attribute should not depend on whether that repository
happens to be a shallow clone or not. That would complicate caching a
lot and would be semantically messy. So applying fetchTree/fetchGit to
a shallow repository is now an error unless you pass the attribute
'shallow = true'. If 'shallow = true', we don't return revCount, even
if the repository is not actually shallow.

Note that Nix itself is not doing shallow clones at the moment. But it
could do so as an optimisation if the user specifies 'shallow = true'.

Issue #2988.
2020-03-17 22:35:29 +01:00
Eelco Dolstra 2a4e4f6a6e
Unified fetcher caching system 2020-03-17 22:35:29 +01:00
John Ericson e5178fd22d
Fix typos
Thanks @asymmetric!

Co-Authored-By: asymmetric <lorenzo@mailbox.org>
2020-03-16 16:40:13 -04:00
Eelco Dolstra fbcb897e21
Add a test for shallow Git clones
Also, don't return a revCount anymore for shallow or dirty Git trees,
since it's incorrect.

Closes #2988.
2020-03-16 13:20:32 +01:00
John Ericson 2be64efb02 Generalize isFixedOutput in preparation for CA drvs
Today's fixed output derivations and regular derivations differ in a few
ways which are largely orthogonal. This replaces `isFixedOutput` with a
`type` that returns an enum of possible combinations.
2020-03-15 11:05:37 -04:00
John Ericson 68fe0d9809 Add missing #include <regex> 2020-03-13 21:24:35 -04:00
Eelco Dolstra 779ef8f5ef
Merge pull request #3380 from contrun/no-attr-path-for-installed
display attr-path only when queried available
2020-03-13 19:26:20 +01:00
YI b6d794fb8d display attr-path only when queried available 2020-03-14 00:36:26 +08:00
Eelco Dolstra 0c2088d438 Merge remote-tracking branch 'origin/master' into flakes 2020-03-13 17:03:30 +01:00
Eelco Dolstra b816515f61
Fix ca-references feature check
Fixes #3406.
2020-03-13 13:15:51 +01:00
Eelco Dolstra ae9119167e Change the lock file to a graph
This enables support for cycles between flakes.
2020-03-12 22:06:57 +01:00
Eelco Dolstra d048577909
Merge pull request #3403 from hercules-ci/issue-3398-path-info-cache-ttls
pathInfoCache: Respect disk cache TTLs #3398
2020-03-12 11:43:31 +01:00
Robert Hensing 3f55f8a8fb pathInfoCache: Respect disk cache TTLs #3398 2020-03-12 10:30:28 +01:00
Will Dietz 15edd2349e local.mk: fix user-env.cc dep on buildenv.nix.gen.hh, resolve occasional build failure 2020-03-12 00:51:56 +01:00
Eelco Dolstra e188fe7c6d Move call-flake.nix into libexpr 2020-03-11 17:04:51 +01:00
Eelco Dolstra 35f6651735 Merge remote-tracking branch 'origin/master' into flakes 2020-03-11 17:03:38 +01:00
Eelco Dolstra 9950cdec35 Move some corepkgs into the nix binary 2020-03-11 16:57:48 +01:00
Eelco Dolstra e02481ded2 parseExprFromString(): Use std::string_view 2020-03-11 16:56:29 +01:00
Eelco Dolstra 97d1c7f932 Merge remote-tracking branch 'origin/master' into flakes 2020-03-11 15:17:21 +01:00
Eelco Dolstra cfc38257cf Fix flake subdirectory handling 2020-03-10 19:21:47 +01:00
Eelco Dolstra 8a1d8701f6
nix-store -q --graph: Fix edges
Fixes #3389.
2020-03-10 11:11:46 +01:00
Eelco Dolstra 983fab7ea9
dotgraph.cc: Remove dead code 2020-03-10 11:06:55 +01:00
Eelco Dolstra 73769b28e3 Move calling flakes into a Nix helper function (call-flake.nix) 2020-03-09 15:28:41 +01:00
Eelco Dolstra 5a1514adb8 Add 'flake' attribute to lock files to denote non-flakes 2020-03-09 15:27:49 +01:00
Eelco Dolstra 73b6d87e17
Merge remote-tracking branch 'origin/master' into flakes 2020-03-04 13:58:42 +01:00
Eelco Dolstra d37dc71e3c
nix-build: Fix !<output> handling
This was broken by 22a754c091.

https://hydra.nixos.org/eval/1573669
2020-03-04 13:56:17 +01:00
Eelco Dolstra 75db069f92
Optimise Derivation::unparse()
In

  nix-instantiate --dry-run '<nixpkgs/nixos/release-combined.nix>' -A nixos.tests.simple.x86_64-linux

this reduces time spent in unparse() from 9.15% to 4.31%. The main
culprit was appending characters one at a time to the destination
string. Even though the string has enough capacity, push_back() still
needs to check this on every call.
2020-03-04 11:44:45 +01:00
Eelco Dolstra 401b5bc541
builtins.cache: Cache regular expressions
The evaluator was spending about 1% of its time compiling a small
number of regexes over and over again.
2020-03-04 11:44:33 +01:00
Eelco Dolstra 22a754c091
Fix GC failures on bad store path names
It failed on names like '/nix/store/9ip48nkc9rfy0a4yaw98lp6gipqlib1a-'.
2020-02-28 18:07:10 +01:00
Eelco Dolstra f443d5ca19 Merge remote-tracking branch 'origin/master' into flakes 2020-02-27 15:19:58 +01:00
Eelco Dolstra 2672a28bb4 nix dev-shell: Add --command option
Note: like 'nix run', and unlike 'nix-shell', this takes an argv
vector rather than a shell command. So

  nix dev-shell -c 'echo $PATH'

doesn't work. Instead you need to do

  nix dev-shell -c bash -c 'echo $PATH'
2020-02-27 15:17:37 +01:00
Eelco Dolstra 4a4521f462 Fix nlohmann::json exception 2020-02-20 23:47:02 +01:00
Eelco Dolstra 73c9840569 Restore subdir -> dir
Got this mixed up somewhere.
2020-02-20 23:44:06 +01:00
Eelco Dolstra d068f9ffff Restore subdir support in registries
Hacky...
2020-02-20 22:14:44 +01:00
Eelco Dolstra 890df325c7
fetchTree: Use a feature flag 2020-02-20 13:36:16 +01:00
Eelco Dolstra 2e953b567e
Merge pull request #3325 from xzfc/clean-tmpdir
nix-shell: clean up the tmpDir and escape variables
2020-02-19 21:29:18 +01:00
Eelco Dolstra 4ad5826a18 nix eval-hydra-jobs: Remove
On second thought, let's move this back to Hydra.
2020-02-19 16:12:49 +01:00
Albert Safin f2a03acf3f nix-shell: clean up the tmpDir and escape variables
The problem fixed: each nix-shell invocation creates a new temporary
directory (`/tmp/nix-shell-*`) and never cleans up.

And while I'm here, shellescape all variables inlined into the rcfile.
See what might happen without escaping:

    $ export TZ="';echo pwned'"
    $ nix-shell -p hello --run hello
    pwned
    Hello, world!
2020-02-19 14:28:49 +00:00
Eelco Dolstra 1d99c4ab25
Merge pull request #3229 from Ma27/flakes-fetchgit-worktree-support
builtins.fetchGit: Fix build when fetching a git worktree
2020-02-19 14:11:18 +01:00