puck/lix
0
0
Fork 0
forked from lix-project/lix
Code Pull requests Activity
9460 commits 19 branches 0 tags 68 MiB
Commit graph

5930 commits

Author SHA1 Message Date
Eelco Dolstra 4bf5faf416 Merge remote-tracking branch 'origin/master' into markdown 2020-08-25 19:47:34 +02:00
Eelco Dolstra 7a02865b94
Move import docs 2020-08-25 14:06:01 +02:00
Eelco Dolstra f53b5f1058
Add getDoc() function 2020-08-25 13:31:11 +02:00
Eelco Dolstra 2a2121d264
Use RegisterPrimOp for some undocumented primops 2020-08-25 11:25:01 +02:00
Eelco Dolstra b8416779e3
Document some primops 2020-08-25 11:16:45 +02:00
Eelco Dolstra 24b1c2c66b
Fix tests 2020-08-25 10:51:14 +02:00
Eelco Dolstra b42789f013
Fix clang build 2020-08-24 21:13:39 +02:00
John Ericson 8eb73a8724 CA derivations that depend on other CA derivations 
Co-authored-by: Théophane Hufschmitt <regnat@users.noreply.github.com>
 2020-08-24 19:01:15 +00:00
John Ericson e0b0e18905 Add constructor for BasicDerivation -> Derivation 2020-08-24 19:01:07 +00:00
John Ericson 59979e7053 Fix bad debug format string 2020-08-24 19:01:07 +00:00
Eelco Dolstra d9a8619762
Don't barf if corepkgs is in the store but not a valid path 
This can happen when using a dummy store (or indeed any non-local store).
 2020-08-24 19:15:17 +02:00
Eelco Dolstra 6a67e57019
Add DummyStore (dummy://) 
DummyStore does not allow building or adding paths. This is useful for
evaluation tests when you don't want to initialize a "proper" store.
 2020-08-24 18:54:16 +02:00
Eelco Dolstra d0690bc311
nix repl ':doc': Render using lowdown 2020-08-24 18:10:33 +02:00
Eelco Dolstra 0f314f3c25
Generate builtins section of the manual 2020-08-24 14:49:30 +02:00
Eelco Dolstra a990f063ff
Move primop docs inline 
This makes them available to 'nix repl'.
 2020-08-24 14:31:10 +02:00
Eelco Dolstra 33b1679d75
Allow primops to have Markdown documentation 2020-08-24 13:16:02 +02:00
Eelco Dolstra 88d5c9ec58
Fix tests 2020-08-24 10:37:10 +02:00
Eelco Dolstra 2530cc6827
Merge pull request #3950 from obsidiansystems/tabs-spaces 
tabs -> spaces
 2020-08-24 10:20:28 +02:00
John Ericson b0b59fd05a Merge remote-tracking branch 'obsidian/write-derivation-borrow' into HEAD 2020-08-23 15:11:10 +00:00
John Ericson 35e6288be1 writeDerivation just needs a plain store reference 2020-08-23 15:01:11 +00:00
John Ericson 3a7b330b64 "Downstream placeholders" should not be store paths 
Insead they should be opaque `/<hash>` like the placeholders we already
have.
 2020-08-21 19:35:35 +00:00
John Ericson 422affe102 tabs -> spaces 
Sorry I let the tab sneak in there in the first place.
 2020-08-21 19:19:24 +00:00
John Ericson c265e0ea82 Merge remote-tracking branch 'upstream/master' into templated-daemon-protocol 2020-08-20 20:24:22 +00:00
John Ericson 27a3f82c0b Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-08-20 18:28:17 +00:00
John Ericson 45a2f1baab Rename drv output querying functions, like master 
- `queryDerivationOutputMapAssumeTotal` -> `queryPartialDerivationOutputMap`
 - `queryDerivationOutputMapAssumeTotal` -> `queryDerivationOutputMap
 2020-08-20 18:14:12 +00:00
Eelco Dolstra 4d77513d97
Merge pull request #3859 from obsidiansystems/drv-outputs-map-allow-missing 
`queryDerivationOutputMap` no longer assumes all outputs have a mapping
 2020-08-20 16:49:23 +02:00
John Ericson 9a9d834dc7 Rename drv output querying functions 
- `queryDerivationOutputMapAssumeTotal` -> `queryPartialDerivationOutputMap`
 - `queryDerivationOutputMapAssumeTotal` -> `queryDerivationOutputMap`
 2020-08-20 14:12:51 +00:00
Eelco Dolstra 25ecfffdc3
Remove PrimOp constructor 2020-08-20 12:34:04 +02:00
Eelco Dolstra dc2f278c95
Allow 'nix' subcommands to provide docs in Markdown format 2020-08-20 12:21:46 +02:00
Eelco Dolstra 3c4f8c9175
List deprecated option aliases in the docs 2020-08-20 11:13:17 +02:00
Eelco Dolstra acb99f03f9
Config: Use nlohmann/json 2020-08-20 11:02:16 +02:00
John Ericson 3df78858f2 Fix max fd calc and add test 2020-08-20 05:08:50 +00:00
Ryan Mulligan f36793c7b9
fix spelling 2020-08-19 20:31:01 -07:00
John Ericson a83694c7a1 Use RemoteStore to open connection for proxying daemon 
Removes duplicate websocket opening code, and also means we should be
able to to ssh-ssh-... daemon relays, not just uds-uds-... ones.
 2020-08-19 19:34:47 +00:00
Eelco Dolstra c8fa39324a
Generate the nix.conf docs from the source code 
This means we don't have two (divergent) sets of option descriptions
anymore.
 2020-08-19 18:28:04 +02:00
John Ericson d5af5763cf Merge branch 'master' of github.com:NixOS/nix into remove-storetype-delegate-regStore 2020-08-19 15:54:50 +00:00
Eelco Dolstra 34b22e0123
Change option descriptions to Markdown 2020-08-19 14:21:27 +02:00
John Ericson be0d429b95 Merge branch 'master' of github.com:NixOS/nix into templated-daemon-protocol 2020-08-19 03:17:41 +00:00
John Ericson 950ddfdb82 Merge remote-tracking branch 'upstream/master' into derivation-header-include-order 2020-08-18 14:36:44 +00:00
Eelco Dolstra 1c8b550e34
Merge pull request #3917 from obsidiansystems/output-env-var-unconditional 
Simplify code as output env vars are unconditional
 2020-08-18 16:21:17 +02:00
Eelco Dolstra dfeb76dbf9
Merge pull request #3930 from obsidiansystems/legacy-ssh-build-paths 
Define `LegacySSHStore::buildPaths` using `cmdBuildPaths`
 2020-08-18 16:07:40 +02:00
Eelco Dolstra 069340179e
Improve nix.1 manpage generator 2020-08-18 15:15:35 +02:00
Eelco Dolstra 6f19c776db
Start generation of the nix.1 manpage 2020-08-17 19:33:18 +02:00
Eelco Dolstra a72a20d68f
Add 'nix dump-args' to dump all commands/flags for manpage generation 2020-08-17 17:44:52 +02:00
Eelco Dolstra 7cdc739ece
Merge remote-tracking branch 'origin/master' into markdown 2020-08-17 13:43:39 +02:00
Eelco Dolstra e849b19872
Merge pull request #3932 from chkno/no-show-signature 
Don't try to parse signature check as commit timestamp
 2020-08-17 11:00:24 +02:00
Eelco Dolstra 847a5392f4 Merge branch 'ca-no-need-trust' of https://github.com/obsidiansystems/nix 2020-08-17 10:57:02 +02:00
Eelco Dolstra 3c619f6290 Merge branch 'test-RemoteStore-buildDerivation' of https://github.com/obsidiansystems/nix 2020-08-17 10:53:18 +02:00
John Ericson 6f7ac5e865 Remove extra closing paren 2020-08-14 21:59:31 +00:00
John Ericson f899a7c6d7 Work around clang bug 2020-08-14 18:51:31 +00:00
John Ericson 3c8b5b6219 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-08-14 17:00:13 +00:00
John Ericson 4b571ea321
Update src/libstore/daemon.cc 
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
 2020-08-14 11:52:37 -04:00
Eelco Dolstra 13e49be660
Merge pull request #3875 from obsidiansystems/new-interface-for-path-pathOpt 
Offer a safer interface for path and pathOpt
 2020-08-14 17:19:19 +02:00
Eelco Dolstra 7714d9a943
Merge pull request #3924 from obsidiansystems/features-per-store 
Make `system-features` a store setting
 2020-08-14 17:13:07 +02:00
Eelco Dolstra 9b9d5297ba
Merge pull request #3909 from matthewbauer/readd-hashed-mirrors 
Add hashed-mirrors back
 2020-08-14 17:05:47 +02:00
Eelco Dolstra d81f13f7cb
Merge pull request #3899 from obsidiansystems/make-narHash-not-optional 
Make narHash in ValidPathInfo not optional
 2020-08-14 17:00:18 +02:00
Chuck ed026f7206 Don't try to parse signature check as commit timestamp 
When the log.showSignature git setting is enabled, the output of
"git log" contains signature verification information in addition to the
timestamp GitInputScheme::fetch wants:

  $ git log -1 --format=%ct
  gpg: Signature made Sat 07 Sep 2019 02:02:03 PM PDT
  gpg:                using RSA key 0123456789ABCDEF0123456789ABCDEF01234567
  gpg:                issuer "user@example.com"
  gpg: Good signature from "User <user@example.com>" [ultimate] 1567890123
  1567890123

For folks that had log.showSignature set, this caused all nix operations
on flakes to fail:

  $ nix build
  error: stoull
 2020-08-13 17:44:42 -07:00
John Ericson e1308b1211 Define LegacySSHStore::buildPaths using cmdBuildPaths 
Evidentally this was never implemented because Nix switched to using
`buildDerivation` exclusively before `build-remote.pl` was rewritten.

The `nix-copy-ssh` test (already) tests this.
 2020-08-13 21:27:55 +00:00
John Ericson 5ccd94501d Allow trustless building of CA derivations 
Include a long comment explaining the policy. Perhaps this can be moved
to the manual at some point in the future.

Also bump the daemon protocol minor version, so clients can tell whether
`wopBuildDerivation` supports trustless CA derivation building. I hope
to take advantage of this in a follow-up PR to support trustless remote
building with the minimal sending of derivation closures.
 2020-08-13 18:15:57 +00:00
John Ericson 85aacbee64 Use TeeSink and TeeSouce in a few more places 2020-08-13 14:51:17 +00:00
John Ericson d2f2be0f70 Test RemoteStore::buildDerivation 
Fix `wopNarFromPath` which needed a `toRealPath`.
 2020-08-13 04:07:14 +00:00
John Ericson 5d67f18c86 Merge branch 'daemon-auth-cleanup' of github.com:obsidiansystems/nix into HEAD 2020-08-12 18:22:31 +00:00
John Ericson 4720853129 Make system-features a store setting 
This seems more correct. It also means one can specify the features a
store should support with --store and remote-store=..., which is useful.
I use this to clean up the build remotes test.
 2020-08-12 18:13:00 +00:00
John Ericson 8d4162ff9e Separate auth and logic for the daemon 
Before, processConnection wanted to know a user name and user id, and
`nix-daemon --stdio`, when it isn't proxying to an underlying daemon,
would just assume "root" and 0. But `nix-daemon --stdio` (no proxying)
shouldn't make guesses about who holds the other end of its standard
streams.

Now processConnection takes an "auth hook", so `nix-daemon` can provide
the appropriate policy and daemon.cc doesn't need to know or care what
it is.
 2020-08-12 15:22:33 +00:00
John Ericson 5f80aea795 Break out lambda so output can be matched just once 
This is much better.
 2020-08-12 02:23:31 +00:00
John Ericson 18834f7764 Recheck path validity after acquiring lock 
It might have changed, and in any event this is how the cod used to work
so let's just keep it.
 2020-08-11 23:44:02 +00:00
John Ericson 2de201254e Don't assume a total output map in two places in build.cc 
Thanks @regnat for catching one of them. The other follows for many of
the same reasons. I'm find fixing others on a need-to-fix basis,
provided their are no regressions.
 2020-08-11 23:07:50 +00:00
John Ericson 4c6aac8fdf Clarify comment on sandbox and temp fresh paths 2020-08-11 22:46:05 +00:00
John Ericson 6d57139050 Clarify outputReferences variable with self-describing type 
Thanks for the idea, @Regnat!
 2020-08-11 22:34:09 +00:00
John Ericson 8a068bd025 Remove redundant equality check 2020-08-11 21:25:40 +00:00
John Ericson 07e3466eb4 Float comment to out describe gaveUpOnSubstitution in general 2020-08-11 21:16:14 +00:00
John Ericson d0f6e338dd
Apply suggestions from code review 
Thanks!!

Co-authored-by: Théophane Hufschmitt <regnat@users.noreply.github.com>
 2020-08-11 16:49:10 -04:00
Ben Burdette 1a281ec07f demote remote build message to Info 2020-08-11 10:29:43 -06:00
Eelco Dolstra 7a911b6783
Merge pull request #3914 from obsidiansystems/small-drv-serialize-cleanup 
Two small derivation serialization cleanups
 2020-08-11 07:14:38 +02:00
John Ericson d3fa8c04c6 Simplify code as output env vars are unconditional 
Since the jsonObject unique ptr is reset to flush the string to make
`__json`, all these `!jsonObject` conditions will always be true.
 2020-08-11 01:13:26 +00:00
John Ericson 2a0902634e Fix error in merge breaking floating CA drvs 
Forgot to add this hunk!
 2020-08-11 00:13:19 +00:00
Maximilian Bosch 5f8ae16c8b
Always reset ANSI colors in progress-bar line 
When having a message like `waiting for a machine to build X` and
building with `nix build -L`, the log-prefix is always colored yellow[1]
on a small terminal-width as everything (including the ANSI color-reset) is
stripped away.

To work around that problem, this patch explicitly adds an `ANSI_NORMAL`
to the end of the line.

[1] https://imgur.com/a/FjtJOk3
 2020-08-10 17:44:17 +02:00
John Ericson 1b5c24662b Merge branch 'small-drv-serialize-cleanup' of github.com:obsidiansystems/nix into single-ca-drv-build 2020-08-10 01:57:54 +00:00
John Ericson 581183d4d5 Deduplicate parsing and reading derivations 2020-08-10 01:40:50 +00:00
John Ericson bcd0629c2e Remove name parameter from writeDerivation 
The name is now stored with the derivation itself.
 2020-08-10 01:35:59 +00:00
John Ericson e913a2989f Squashed get CA derivations building 2020-08-07 19:51:55 +00:00
Eelco Dolstra edfd676e05 Fix .ls file names in binary caches 
These are not supposed to include the 'name' part of the store
path. This was broken by 759947bf72.
 2020-08-07 21:18:29 +02:00
John Ericson 8f92bb5ad9 Merge branch 'drv-outputs-map-allow-missing' of github.com:obsidiansystems/nix into templated-daemon-protocol 2020-08-07 18:51:01 +00:00
John Ericson f7ba16f9cb Merge remote-tracking branch 'upstream/master' into drv-outputs-map-allow-missing 2020-08-07 17:46:39 +00:00
John Ericson 47644e49ca Specialize std::optional<StorePath> so this is backwards compatible 
While I am cautious to break parametricity, I think it's OK in this
cases---we're not about to try to do some crazy polymorphic protocol
anytime soon.
 2020-08-07 17:05:14 +00:00
Eelco Dolstra 3c75ddc16b nix build (and others): Force re-evaluation of cached errors 
Fixes #3872.

This is a bit hacky. Ideally we would automatically re-evaluate the
failed attribute iff we need to print the error message (so in
commands like 'nix search' we wouldn't re-evaluate because we're
suppressing errors).
 2020-08-07 14:47:23 +02:00
Eelco Dolstra 2ffc058950 Make --no-eval-cache a global setting 2020-08-07 14:13:24 +02:00
Matthew Bauer 96c158d6e1 Fix build 2020-08-06 21:04:31 -05:00
Matthew Bauer 8abc577cc2 Merge remote-tracking branch 'origin/master' into readd-hashed-mirrors 2020-08-06 18:58:59 -05:00
Carlo Nucera 46f9dd56da Fix bug due to non-deterministic arg eval order 2020-08-06 19:30:05 -04:00
Matthew Bauer 641c950701 Add hashed-mirrors back 
Some users have their own hashed-mirrors setup, that is used to mirror
things in addition to what’s available on tarballs.nixos.org. Although
this should be feasable to do with a Binary Cache, it’s not always
easy, since you have to remember what "name" each of the tarballs has.
Continuing to support hashed-mirrors is cheap, so it’s best to leave
support in Nix. Note that NIX_HASHED_MIRRORS is also supported in
Nixpkgs through fetchurl.nix.

Note that this excludes tarballs.nixos.org from the default, as in
\#3689. All of these are available on cache.nixos.org.
 2020-08-06 18:19:09 -05:00
Carlo Nucera 9ab07e99f5 Use template structs instead of phantoms 2020-08-06 18:04:13 -04:00
Carlo Nucera 3d8240c32e Remove leftover commented code 2020-08-06 16:04:18 -04:00
Carlo Nucera f795f0fabc Merge branch 'drv-outputs-map-allow-missing-namespace' of github.com:obsidiansystems/nix into templated-daemon-protocol 2020-08-06 15:53:09 -04:00
John Ericson e89b5bd0bf Minimize the usage of Hash::dummy 2020-08-06 18:31:48 +00:00
John Ericson 5e59b25a23 Merge remote-tracking branch 'upstream/master' into make-narHash-not-optional 2020-08-06 15:34:15 +00:00
Eelco Dolstra 6146447842
Merge pull request #3856 from obsidiansystems/buildable-variant 
Make `Buildable` a `std::variant`
 2020-08-06 14:14:06 +02:00
Eelco Dolstra 59067f0f58 repl.cc: Check for HAVE_BOEHMGC 
Fixes #3906.
 2020-08-06 11:40:41 +02:00
Carlo Nucera 8b175f58d1 Simplify the namespace 2020-08-05 17:57:07 -04:00
Carlo Nucera 0739d428e0 Solve template deduction problem 
We had to predeclare our template functions
 2020-08-05 17:49:45 -04:00
John Ericson 6c66331d5c WIP: Put the worker protocol read and write in a namespace to disambig 2020-08-05 20:37:48 +00:00
Carlo Nucera 1d2e80ddd6 Merge branch 'master' of github.com:NixOS/nix into new-interface-for-path-pathOpt 2020-08-05 15:45:33 -04:00
John Ericson ed96e603e1 Proxy -> Phantom to match Rust 
Sorry, Haskell.
 2020-08-05 19:44:08 +00:00
Carlo Nucera 8241e660ba Remove Hash::operator bool () 
Since the hash is not optional anymore
 2020-08-05 15:30:38 -04:00
Carlo Nucera be6e1c6457 Merge branch 'master' of github.com:NixOS/nix into make-narHash-not-optional 2020-08-05 15:14:47 -04:00
Carlo Nucera 1ad6394b33 Add Hash::dummy to signal default value 
We did this in the same spirit of the dummy value that's present in
libstore/path.hh
 2020-08-05 15:11:49 -04:00
Carlo Nucera 1d71028f4d Remove optionality in ValidPathInfo::narInfo 2020-08-05 14:42:48 -04:00
John Ericson 7302761f64 Merge remote-tracking branch 'obsidian/drv-outputs-map-allow-missing' into templated-daemon-protocol 2020-08-05 17:53:24 +00:00
Ben Burdette 31f1af0cab don't crash if there's no drvPath 2020-08-05 11:26:06 -06:00
Ben Burdette e4eae078a5 add derivation path to hint 2020-08-05 11:21:36 -06:00
Ben Burdette f1a47a96b6 error messages for issue 2238 2020-08-05 10:58:00 -06:00
John Ericson cf939055c8 Merge remote-tracking branch 'upstream/master' into drv-outputs-map-allow-missing 2020-08-05 16:43:30 +00:00
John Ericson a9bbfaa851 Fix --profile with multiple opaque paths 2020-08-05 16:27:15 +00:00
John Ericson d89472a912 Merge remote-tracking branch 'upstream/master' into buildable-variant 2020-08-05 15:41:57 +00:00
John Ericson b3e73547a0
Update src/libexpr/primops.cc 
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
 2020-08-05 11:05:46 -04:00
John Ericson 0559ff3d8b Merge branch 'master' of github.com:NixOS/nix into derivation-primop-floating-output 2020-08-05 14:56:40 +00:00
John Ericson 92ad550e96 Merge remote-tracking branch 'obsidian/misc-ca' into derivation-primop-floating-output 2020-08-05 14:51:41 +00:00
Eelco Dolstra 790b694be7 Style fix 2020-08-05 16:51:06 +02:00
Eelco Dolstra e48f944e9d Merge branch 'misc-ca' of https://github.com/obsidiansystems/nix 2020-08-05 16:50:05 +02:00
John Ericson b9ebe373bb Sed some names to perhaps avoid conflicts 2020-08-05 14:49:25 +00:00
Eelco Dolstra 25f7912156 Style fix 2020-08-05 16:47:48 +02:00
John Ericson e561a13a58 Reanme DerivationType::Regular defintion too 
This is the one non-prefixed occurence
 2020-08-05 14:45:56 +00:00
John Ericson e7b0847f2d Make names more consistent 2020-08-05 14:44:39 +00:00
John Ericson 839f0fe095 Merge remote-tracking branch 'upstream/master' into misc-ca 2020-08-05 14:40:01 +00:00
John Ericson 03f4fafc27 Merge remote-tracking branch 'upstream/master' into misc-ca 2020-08-05 14:36:25 +00:00
Eelco Dolstra b91dc7ebad
Merge pull request #3730 from obsidiansystems/better-ca-parse-errors 
Improve hash parsing and errors
 2020-08-05 16:33:07 +02:00
Eelco Dolstra 75f220a595
Merge pull request #3864 from obsidiansystems/more-topo-sort 
Abstract out topo sorting logic
 2020-08-05 16:07:29 +02:00
Eelco Dolstra 088dcea0e8 Typo 2020-08-05 15:41:51 +02:00
John Ericson 16c98bf57c Get rid of some unneeded temporaries 2020-08-04 22:36:31 +00:00
John Ericson 1dfcbebc95 Organize and format code a bit 2020-08-04 22:28:10 +00:00
John Ericson 45b6fdb22b Remove unused functions 2020-08-04 22:10:13 +00:00
John Ericson 1bab8a321f Remove unneeded definition 
Template instantiations will cover this case fine.
 2020-08-04 21:56:42 +00:00
John Ericson 2f2ae993dc WIP systematize more of the worker protocol 
This refactor should *not* change the wire protocol.
 2020-08-04 19:02:05 +00:00
John Ericson fbeb8695fb Merge remote-tracking branch 'upstream/master' into drv-outputs-map-allow-missing 2020-08-04 18:35:33 +00:00
John Ericson d3452a5ed6 Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors 2020-08-04 16:13:20 +00:00
Eelco Dolstra 5b22a2c0d4
Merge pull request #3894 from obsidiansystems/json-test-order-agnostic 
Make JSON equality tests agnostic to ordering
 2020-08-04 17:24:46 +02:00
Eelco Dolstra 4e7f1c7f11 S3BinaryCacheStore: Fix size determination 2020-08-04 16:01:13 +02:00
Eelco Dolstra dfe66420e7 Revert "Remove putBytes" 
This reverts commit b8eea7e81a.
 2020-08-04 15:56:10 +02:00
Eelco Dolstra 327b1bf378 BinaryCacheStore: Explicitly flush file sink 
The file sink is also flushed in its destructor, but we ignore any
exceptions in the destructor.

Issue #3886.
 2020-08-04 14:54:07 +02:00
Eelco Dolstra ed52cf632b
Merge pull request #3892 from obsidiansystems/path-info-header 
Move ValidPathInfo to its own header
 2020-08-04 10:44:47 +02:00
John Ericson fe7e57a80d tab -> space 2020-08-04 03:47:09 +00:00
John Ericson 062533f7cd Merge remote-tracking branch 'upstream/master' into path-info-header 2020-08-03 21:02:28 +00:00
Eelco Dolstra 24e07c428f Delete compressed NARs 
Fixes #3891.
 2020-08-03 18:34:52 +02:00
John Ericson 54281f3ac1 addToStore in terms of addToStoreFromDump is not local-store-specific 2020-08-03 04:13:45 +00:00
John Ericson 9357512d73 Merge remote-tracking branch 'upstream/master' into derivation-header-include-order 2020-08-01 19:38:35 +00:00
John Ericson c4ada76e86 Fix error message and avoid recalculation 2020-08-01 16:22:50 +00:00
John Ericson bc165e28ae Embelish documentation of new Hash functions 2020-08-01 15:32:20 +00:00
John Ericson 3cbee1e840 Convert to C-style comments 2020-08-01 15:26:57 +00:00
Carlo Nucera b6d97fdbf4 Merge branch 'master' of github.com:NixOS/nix into drv-outputs-map-allow-missing 2020-07-31 13:12:51 -04:00
Eelco Dolstra a3f9625818 Tweak description 2020-07-31 17:32:40 +02:00
Eelco Dolstra 935723eb3b Merge branch 'nix-command-build-check' of https://github.com/Ma27/nix 2020-07-31 17:30:55 +02:00
Eelco Dolstra f56dfce734 nix bundle: Set category 2020-07-31 17:30:12 +02:00
Maximilian Bosch 5a09eb86f1
nix/build: add --rebuild option 
Occasionally, `nix-build --check` is fairly helpful and I'd like to be
able to use this feature for flakes that need to be built with `nix
build` as well.
 2020-07-31 17:17:03 +02:00
Eelco Dolstra bf290c2306
Merge remote-tracking branch 'origin/master' into markdown 2020-07-31 16:07:04 +02:00
Eelco Dolstra 1d0a7b54fa
Enable syntax highlighting 2020-07-31 15:43:25 +02:00
Eelco Dolstra 0604cfd0eb
Merge pull request #3880 from matthewbauer/nix-bundle 
Add "bundle" command to Nix
 2020-07-31 11:45:53 +02:00
John Ericson e3a2154f5a Fix indentation 2020-07-31 01:07:59 +00:00
Carlo Nucera eee6ef86cd Merge branch 'master' of github.com:NixOS/nix into better-ca-parse-errors 2020-07-30 18:27:25 -04:00
Matthew Bauer cdc2386644 Make expectedHash optional in prim_path 
This fixes an error found in builtins.path that looks like:

store path mismatch in (possibly filtered) path added from '/private/tmp/nix-shell.CyXViH/nix-test/filter-source/filterin'

when no hash is specified
 2020-07-30 16:40:40 -05:00
Matthew Bauer fa2d1fb36e Pass system to bundler 2020-07-30 15:37:05 -05:00
Matthew Bauer 22fcfdf18a Address misc review 2020-07-30 15:18:57 -05:00
Matthew Bauer 1a705637ce Remove single file restriction for bundler 2020-07-30 15:16:29 -05:00
Matthew Bauer 52407f83a1 exporter -> bundler 2020-07-30 15:16:23 -05:00
Matthew Bauer 05ac4db39a Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-07-30 12:38:24 -05:00
Matthew Bauer 5d04a4db9b Handle exporters checking correctly 2020-07-30 11:45:47 -05:00
Matthew Bauer 2f4250a416 Add "export" to Nix 
This adds a ‘nix export’ command which hooks into nix-bundle. It can
be used in a similar way as nix-bundle, with the benefit of hooking
into the new “app” functionality. For instance,

$ nix export nixpkgs#jq
$ ./jq --help
jq - commandline JSON processor [version 1.6]
...
$ scp jq machine-without-nix:
$ ssh machine-without-nix ./jq
jq - commandline JSON processor [version 1.6]
...

Note that nix-bundle currently requires Linux to run. Other exporters
might not have that requirement.

“exporters” are meant to be reusable, so that, other repos can
implement their own bundling.

Fixes #3705
 2020-07-30 11:33:22 -05:00
Eelco Dolstra a785b3eddf Fix build 2020-07-30 15:27:28 +02:00
Eelco Dolstra 3f6e88a552 unsigned long long -> uint64_t 2020-07-30 13:34:04 +02:00
Eelco Dolstra ebee2b7852 receiveContents(): unsigned int -> size_t 2020-07-30 13:00:30 +02:00
Mat Marini d9e23bfee2 Prevent nix-build from accepting --run/--command 2020-07-29 20:53:02 -04:00
Mat Marini e4f6f8da77 Prevent nix-build from accepting --packages 2020-07-29 20:50:06 -04:00
Eelco Dolstra 0c94c17644 Merge remote-tracking branch 'origin/master' into markdown 2020-07-29 18:08:57 +02:00
Eelco Dolstra f63839bfa4 Cleanup 2020-07-29 18:04:51 +02:00
Eelco Dolstra 39311e7009
Merge pull request #3549 from Ma27/fetchgit-hash 
Merge legacy `fetchGit`-builtin with the generic `fetchTree`-function
 2020-07-29 11:33:27 +02:00
Eelco Dolstra 7c097275c4
Merge pull request #3877 from matthewbauer/develop-continuous-regex 
Set regex_constants::match_continuous for quicker search in nix develop
 2020-07-29 10:07:52 +02:00
Matthew Bauer fa8515d7ec Set regex_constants::match_continuous for quicker search 
match_continuous limits the search to the current start position,
instead of searching the entire file.

On libc++, this improves performance dramatically:

$ time /nix/store/70ai68dfm6xbzwn26j5n4li9di52ylia-nix-3.0pre20200728_c159f48/bin/nix print-dev-env >/dev/null
/nix/store/70ai68dfm6xbzwn26j5n4li9di52ylia-nix-3.0pre20200728_c159f48/bin/ni  2.39s user 0.19s system 64% cpu 4.032 total
$ time /nix/store/cwjfxxlp83zln4mfyy1d2dbsx7f6s962-nix-3.0pre20200728_dirty/bin/nix print-dev-env >/dev/null
/nix/store/cwjfxxlp83zln4mfyy1d2dbsx7f6s962-nix-3.0pre20200728_dirty/bin/nix   0.09s user 0.05s system 65% cpu 0.204 total

Fixes #3874
 2020-07-29 00:57:20 -05:00
Eelco Dolstra 4c0077a07d Fix RemoteStore::addToStore() latency 
Since 6185d25e52, this was very
latency-bound since it required a round-trip for every 32 KiB. So for
example copying a 514 MiB closure over a virtual ethernet device with
a articial delay of just 1 ms took 343s. Now it takes 2.7s.

Fixes #3372.
 2020-07-29 00:48:39 +02:00
Eelco Dolstra c159f48a39 Cleanup 2020-07-29 00:24:55 +02:00
John Ericson df707d05d1 Merge branch 'ca-drv' of github.com:Ericson2314/nix into misc-ca 2020-07-28 21:12:36 +00:00
Maximilian Bosch e4940e90f3
Restore backwards-compat for current builtins.fetchGit 
If a repo is dirty, it used to return a `rev` object with an "empty"
sha1 (0000000000000000000000000000000000000000). Please note that this
only applies for `builtins.fetchGit` and *not* for `builtins.fetchTree{
type = "git"; }`.
 2020-07-28 22:46:39 +02:00
Carlo Nucera 2980b244b7 Use assert(false) instead of abort() 2020-07-28 15:39:45 -04:00
Carlo Nucera cfe96bd7ce Merge branch 'master' of github.com:NixOS/nix into ca-drv 2020-07-28 15:36:45 -04:00
Carlo Nucera c318d398f3 Merge branch 'misc-ca' of github.com:obsidiansystems/nix into new-interface-for-path-pathOpt 2020-07-28 14:22:24 -04:00
Carlo Nucera 7ef1e3cd14 Use the new interface 2020-07-28 13:59:24 -04:00
Carlo Nucera d564ac1c50 Offer a safer interface for pathOpt 
The new interface we offer provides a way of getting all the
DerivationOutputs with the storePaths directly, based on the observation
that it's the most common usecase.
 2020-07-28 13:55:15 -04:00
Eelco Dolstra b8eea7e81a Remove putBytes 
istream->tellg() returns -1 so we can't get the number of bytes
written.

Fixes 'uploaded 's3://nix-cache/nar/00819r9lp5kajr6baxfw5dhhc0cx8ndxaz43qmd2f0gn1hk1ynlp.nar.xz' (-1 bytes) in 11620 ms' messages.
 2020-07-28 13:27:56 +02:00
Maximilian Bosch f742438465
Merge legacy fetchGit-builtin with the generic fetchTree-function 
The original idea was to implement a git-fetcher in Nix's core that
supports content hashes[1]. In #3549[2] it has been suggested to
actually use `fetchTree` for this since it's a fairly generic wrapper
over the new fetcher-API[3] and already supports content-hashes.

This patch implements a new git-fetcher based on `fetchTree` by
incorporating the following changes:

* Removed the original `fetchGit`-implementation and replaced it with an
  alias on the `fetchTree` implementation.

* Ensured that the `git`-fetcher from `libfetchers` always computes a
  content-hash and returns an "empty" revision on dirty trees (the
  latter one is needed to retain backwards-compatibility).

* The hash-mismatch error in the fetcher-API exits with code 102 as it
  usually happens whenever a hash-mismatch is detected by Nix.

* Removed the `flakes`-feature-flag: I didn't see a reason why this API
  is so tightly coupled to the flakes-API and at least `fetchGit` should
  remain usable without any feature-flags.

* It's only possible to specify a `narHash` for a `git`-tree if either a
  `ref` or a `rev` is given[4].

* It's now possible to specify an URL without a protocol. If it's missing,
  `file://` is automatically added as it was the case in the original
  `fetchGit`-implementation.

[1] https://github.com/NixOS/nix/pull/3216
[2] https://github.com/NixOS/nix/pull/3549#issuecomment-625194383
[3] https://github.com/NixOS/nix/pull/3459
[4] https://github.com/NixOS/nix/pull/3216#issuecomment-553956703
 2020-07-28 00:44:39 +02:00
John Ericson 8065c6d160 Abstract out topo sorting logic 2020-07-27 20:45:34 +00:00
John Ericson 951415b568 Require ca-derivations everywhere we create a CA derivation 
"create" as in read one in from a serialized form, or build one from
scratch in memory.
 2020-07-27 17:56:36 +00:00
John Ericson e32a9e124b Merge branch 'misc-ca' of github.com:obsidiansystems/nix into derivation-primop-floating-output 2020-07-27 17:50:06 +00:00
John Ericson 7cf978440c Merge branch 'ca-derivation-data-types' of github.com:obsidiansystems/nix into misc-ca 2020-07-27 17:47:40 +00:00
John Ericson e6115ca004 Merge remote-tracking branch 'upstream/master' into ca-derivation-data-types 2020-07-27 17:41:43 +00:00
John Ericson 1d7d94ceea Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors 2020-07-27 16:17:50 +00:00
John Ericson 43f2bd8dc5 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-07-27 16:13:57 +00:00
John Ericson 387699fe7f Merge branch 'ca-derivation-data-types' of github.com:obsidiansystems/nix into misc-ca 2020-07-27 15:00:28 +00:00
John Ericson 78466bcb2f Merge branch 'optional-derivation-output-storepath' into ca-derivation-data-types 2020-07-27 14:40:08 +00:00
John Ericson d5bb67cfa4 Merge remote-tracking branch 'upstream/master' into optional-derivation-output-storepath 2020-07-27 14:29:32 +00:00
John Ericson 699fc89b39 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-07-27 14:22:45 +00:00
Maximilian Bosch 6ccfdb79c7
libutil/logging: extend internal-json logger to make it more machine-readable 
The new error-format is pretty nice from a UX point-of-view, however
it's fairly hard to parse the output e.g. for editor plugins such as
vim-ale[1] that use `nix-instantiate --parse` to determine syntax errors in
Nix expression files.

This patch extends the `internal-json` logger by adding the fields
`line`, `column` and `file` to easily locate an error in a file and the
field `raw_msg` which contains the error-message itself without
code-lines and additional helpers.

An exemplary output may look like this:

```
[nix-shell]$ ./inst/bin/nix-instantiate ~/test.nix --log-format minimal
{"action":"msg","column":1,"file":"/home/ma27/test.nix","level":0,"line":4,"raw_msg":"syntax error, unexpected IF, expecting $end","msg":"<full error-msg with code-lines etc>"}
```

[1] https://github.com/dense-analysis/ale
 2020-07-26 02:29:57 +02:00
John Ericson 2c7557481b queryDerivationOutputMap no longer assumes all outputs have a mapping 
This assumption is broken by CA derivations. Making a PR now to do the
breaking daemon change as soon as possible (if it is already too late,
we can bump protocol intead).
 2020-07-24 21:14:06 +00:00
Eelco Dolstra a71d1cedff
printVersion(): Show system types 2020-07-24 11:34:01 +02:00
Eelco Dolstra 2292814049
createUnixDomainSocket(): Fix off-by-one error in copying the socket path 
Reported by Kane York.
 2020-07-24 11:19:17 +02:00
John Ericson 58e55c0923 Merge branch 'misc-ca' of github.com:obsidiansystems/nix into derivation-primop-floating-output 2020-07-23 19:17:52 +00:00
John Ericson a2af25c55a Merge remote-tracking branch 'upstream/master' into misc-ca 2020-07-23 19:14:17 +00:00
John Ericson e1de1fe0d8 Make Buildable a std::variant 
I think this better captures the intent of what's going on: we either
have an opaque store path, or a drv path with some outputs.

Having this structure will also help us support CA derivations: we'll
have to allow the outpath paths to be optional, so the structure we gain
now makes up for the structure we loose then.
 2020-07-23 19:02:57 +00:00
John Ericson c36916eca0 Merge branch 'misc-ca' of github.com:obsidiansystems/nix into derivation-primop-floating-output 2020-07-23 15:06:56 +00:00
John Ericson 6fcd9966bf Merge remote-tracking branch 'upstream/master' into misc-ca 2020-07-23 15:02:29 +00:00
John Ericson 66a2067288 Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors 2020-07-23 14:56:35 +00:00
John Ericson 2274f63453 Get rid of basicDerivation::findOutput 
It's a tiny function which is:

 - hardly worth abstrating over, and also only used once.

 - doesn't work once we get CA drvs

I rewrote the one callsite to be forwards compatable with CA
derivations, and also potentially more performant: instead of reading in
the derivation it can ust consult the SQLite DB in the common case.
 2020-07-23 14:39:02 +00:00
Fabian Möller b9ead08ca8
Save changes made by "nix registry pin" to user registry 2020-07-23 14:21:27 +02:00
John Ericson 9423f64ee2 Parse CA derivations using new output variants 
We no longer need `ParsedDerivation` because everything libstore needs
to know about is in the `BasicDerivation` proper.
 2020-07-22 23:59:25 +00:00
John Ericson c56356bacc Separate concerns in scanForReferences with TeeSink 
This also will make it easier to use a `HashModuloSink` instead for CA
derivations.
 2020-07-22 22:13:44 +00:00
Eelco Dolstra ff314f186e
Merge pull request #3736 from obsidiansystems/allow-relative-paths-in-store-option 
Allow relative paths in --store option
 2020-07-21 19:43:07 +02:00
Eelco Dolstra 0951330680
Merge pull request #3714 from obsidiansystems/add-body-to-network-errors 
Add response body to network errors
 2020-07-21 18:09:42 +02:00
John Ericson 02639716ea Merge branch 'allow-relative-paths-in-store-option' into remove-storetype-delegate-regStore 2020-07-21 15:55:27 +00:00
John Ericson 6cce32c8e8 Change logic for deciding what is a relative path for the local store 
The was Eelco's prefered logic, and it looks good to me!
 2020-07-21 15:39:47 +00:00
Carlo Nucera 922a845ffc Update chunkSize to the suggested value 
This was a suggested course of action in a review in one of our earlier
commits, https://github.com/NixOS/nix/pull/3801#discussion_r457557079
 2020-07-21 10:24:19 -04:00
John Ericson 54e507a7aa Merge remote-tracking branch 'upstream/master' into add-body-to-network-errors 2020-07-21 14:17:59 +00:00
John Ericson 5ce95b9529
Update src/libstore/build.cc 2020-07-21 09:47:40 -04:00
John Ericson cdb3f39b64 Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors 2020-07-21 13:43:20 +00:00
John Ericson 5055c595bd Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into ca-derivation-data-types 2020-07-21 01:20:53 +00:00
John Ericson 5a37413796 Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into from-dump-stream 2020-07-21 01:20:14 +00:00
John Ericson bf0dca7790 Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into misc-ca 2020-07-21 01:18:33 +00:00
Carlo Nucera 9aae179f34 Correct bug, thoroughly document addToStoreSlow 2020-07-20 20:18:12 -04:00
Carlo Nucera 0ca9744694 Use heuristics to decide when to show the response 
Due to https://github.com/NixOS/nix/issues/3841 we don't know how print
different messages for different verbosity levels.
 2020-07-20 15:57:58 -04:00
Carlo Nucera 6357b1b0fb Add another Unimplemented case 2020-07-20 14:17:25 -04:00
Carlo Nucera 007bf6d17a Merge branch 'ca-derivation-data-types' of github.com:obsidiansystems/nix into misc-ca 2020-07-20 14:14:39 -04:00
Carlo Nucera 362ae93851 Add UnimplementedError to ease grepping for these 2020-07-20 14:13:37 -04:00
John Ericson c58c6165c5 Remove period at the end of the exception message 2020-07-20 17:43:19 +00:00
John Ericson bf61871271 parser.hh -> split.hh 2020-07-20 17:42:34 +00:00
John Ericson eab945f591 Merge remote-tracking branch 'upstream/master' into from-dump-stream 2020-07-20 17:29:23 +00:00
Eelco Dolstra 1c5f8bbfb5
Merge pull request #3822 from obsidiansystems/dump-thrice-fixme 
Optimize `addToStoreSlow` and remove `TeeParseSink`
 2020-07-20 18:55:05 +02:00
John Ericson ac2fc7ba1f
Apply suggestions from code review 
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
 2020-07-20 11:29:46 -04:00
Carlo Nucera 0aa79dcc6f Remove StoreType abstraction and delegate regStore 
to each Store implementation. The generic regStore implementation will
only be for the ambiguous shorthands, like "" and "auto".

This also could get us close to simplifying the daemon command.
 2020-07-17 17:24:02 -04:00
John Ericson 6756cecfcf Add DerivationType::CAFloating 2020-07-17 19:55:41 +00:00
Carlo Nucera fefd6c9e5f Merge branch 'master' of github.com:NixOS/nix into allow-relative-paths-in-store-option 2020-07-17 14:34:20 -04:00
Carlo Nucera bbc633c98c Revert "Don't anticipate CA but not fixed outputs for now" 
This reverts commit 3a9e4c3262.
 2020-07-17 13:10:32 -04:00
Carlo Nucera 205dcd140d Revert "Don't anticipate multiple CA outputs for now" 
This reverts commit 74b251b2f3.
 2020-07-17 12:43:46 -04:00
Carlo Nucera 1feb8981df Revert "Don't anticipate hash algo without hash in derivation for now" 
This reverts commit 3804e3df9b.
 2020-07-17 12:05:54 -04:00
Lars Mühmel bc73590151 nix edit: call restoreSignals() before execvp-ing the $EDITOR 
Currently resizing of the terminal doesn't play nicely with
nix edit when using kakoune as the editor, as it relies on the
SIGWINCH signal which is trapped by nix. How this is not a problem
with e.g. vim is beyond me.

Virtually all other exec* calls are following a call to
restoreSignals(). This commit adds this behavior to nix edit
as well.
 2020-07-17 17:35:59 +02:00
Carlo Nucera 487c5751c6 Merge branch 'ca-drv' of github.com:Ericson2314/nix into misc-ca 2020-07-17 11:32:59 -04:00
Carlo Nucera a395c12830 Merge branch 'master' of github.com:NixOS/nix into misc-ca 2020-07-17 11:06:54 -04:00
Carlo Nucera 5cb840541b Merge branch 'multi-output-hashDerivationModulo' of github.com:Ericson2314/nix into misc-ca 2020-07-17 10:28:33 -04:00
Eelco Dolstra 17f75f9cc4 parseFlakeRef(): Only search for the top-level directory for CLI flakerefs 2020-07-17 14:54:21 +02:00
Eelco Dolstra 52c8be38e0 nix profile diff-closures: Don't inherit EvalCommand 2020-07-17 12:36:12 +02:00
Carlo Nucera 745a03cef5 Merge branch 'optional-derivation-output-storepath' of github.com:obsidiansystems/nix into ca-derivation-data-types 2020-07-16 13:36:01 -04:00
Carlo Nucera 048e916f64 Merge branch 'master' of github.com:NixOS/nix into optional-derivation-output-storepath 2020-07-16 13:32:28 -04:00
John Ericson cc0d77f8c9 Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors 2020-07-16 17:28:52 +00:00
Carlo Nucera e3a3406db8 Merge branch 'master' of github.com:NixOS/nix into add-body-to-network-errors 2020-07-16 12:04:10 -04:00
Eelco Dolstra 16c9f6762d Add command 'nix profile diff-closure' 
This shows all changes between generations of a profile. E.g.

  $ nix profile diff-closures --profile /nix/var/nix/profiles/system
  Generation 654 -> 655:
    nix: 2.4pre20200617_5d69bbf → 2.4pre20200701_6ff9aa8, +42.2 KiB

  Generation 655 -> 656:
    blender-bin: 2.83.0 → 2.83.1, -294.2 KiB

  Generation 656 -> 657:
    curl: 7.68.0 → 7.70.0, +19.1 KiB
    firmware-linux-nonfree: 2020-01-22 → 2020-05-19, +30827.7 KiB
    ibus: -21.8 KiB
    initrd-linux: 5.4.46 → 5.4.49
    ...
 2020-07-16 17:00:42 +02:00