John Ericson
a83694c7a1
Use RemoteStore to open connection for proxying daemon
...
Removes duplicate websocket opening code, and also means we should be
able to to ssh-ssh-... daemon relays, not just uds-uds-... ones.
2020-08-19 19:34:47 +00:00
Eelco Dolstra
c8fa39324a
Generate the nix.conf docs from the source code
...
This means we don't have two (divergent) sets of option descriptions
anymore.
2020-08-19 18:28:04 +02:00
John Ericson
d5af5763cf
Merge branch 'master' of github.com:NixOS/nix into remove-storetype-delegate-regStore
2020-08-19 15:54:50 +00:00
John Ericson
be0d429b95
Merge branch 'master' of github.com:NixOS/nix into templated-daemon-protocol
2020-08-19 03:17:41 +00:00
John Ericson
950ddfdb82
Merge remote-tracking branch 'upstream/master' into derivation-header-include-order
2020-08-18 14:36:44 +00:00
Eelco Dolstra
dfeb76dbf9
Merge pull request #3930 from obsidiansystems/legacy-ssh-build-paths
...
Define `LegacySSHStore::buildPaths` using `cmdBuildPaths`
2020-08-18 16:07:40 +02:00
Eelco Dolstra
7cdc739ece
Merge remote-tracking branch 'origin/master' into markdown
2020-08-17 13:43:39 +02:00
Eelco Dolstra
847a5392f4
Merge branch 'ca-no-need-trust' of https://github.com/obsidiansystems/nix
2020-08-17 10:57:02 +02:00
Eelco Dolstra
3c619f6290
Merge branch 'test-RemoteStore-buildDerivation' of https://github.com/obsidiansystems/nix
2020-08-17 10:53:18 +02:00
John Ericson
6f7ac5e865
Remove extra closing paren
2020-08-14 21:59:31 +00:00
John Ericson
3c8b5b6219
Merge remote-tracking branch 'upstream/master' into single-ca-drv-build
2020-08-14 17:00:13 +00:00
John Ericson
4b571ea321
Update src/libstore/daemon.cc
...
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2020-08-14 11:52:37 -04:00
Eelco Dolstra
13e49be660
Merge pull request #3875 from obsidiansystems/new-interface-for-path-pathOpt
...
Offer a safer interface for path and pathOpt
2020-08-14 17:19:19 +02:00
Eelco Dolstra
7714d9a943
Merge pull request #3924 from obsidiansystems/features-per-store
...
Make `system-features` a store setting
2020-08-14 17:13:07 +02:00
Eelco Dolstra
9b9d5297ba
Merge pull request #3909 from matthewbauer/readd-hashed-mirrors
...
Add hashed-mirrors back
2020-08-14 17:05:47 +02:00
Eelco Dolstra
d81f13f7cb
Merge pull request #3899 from obsidiansystems/make-narHash-not-optional
...
Make narHash in ValidPathInfo not optional
2020-08-14 17:00:18 +02:00
John Ericson
e1308b1211
Define LegacySSHStore::buildPaths using cmdBuildPaths
...
Evidentally this was never implemented because Nix switched to using
`buildDerivation` exclusively before `build-remote.pl` was rewritten.
The `nix-copy-ssh` test (already) tests this.
2020-08-13 21:27:55 +00:00
John Ericson
5ccd94501d
Allow trustless building of CA derivations
...
Include a long comment explaining the policy. Perhaps this can be moved
to the manual at some point in the future.
Also bump the daemon protocol minor version, so clients can tell whether
`wopBuildDerivation` supports trustless CA derivation building. I hope
to take advantage of this in a follow-up PR to support trustless remote
building with the minimal sending of derivation closures.
2020-08-13 18:15:57 +00:00
John Ericson
85aacbee64
Use TeeSink and TeeSouce in a few more places
2020-08-13 14:51:17 +00:00
John Ericson
d2f2be0f70
Test RemoteStore::buildDerivation
...
Fix `wopNarFromPath` which needed a `toRealPath`.
2020-08-13 04:07:14 +00:00
John Ericson
5d67f18c86
Merge branch 'daemon-auth-cleanup' of github.com:obsidiansystems/nix into HEAD
2020-08-12 18:22:31 +00:00
John Ericson
4720853129
Make system-features a store setting
...
This seems more correct. It also means one can specify the features a
store should support with --store and remote-store=..., which is useful.
I use this to clean up the build remotes test.
2020-08-12 18:13:00 +00:00
John Ericson
8d4162ff9e
Separate auth and logic for the daemon
...
Before, processConnection wanted to know a user name and user id, and
`nix-daemon --stdio`, when it isn't proxying to an underlying daemon,
would just assume "root" and 0. But `nix-daemon --stdio` (no proxying)
shouldn't make guesses about who holds the other end of its standard
streams.
Now processConnection takes an "auth hook", so `nix-daemon` can provide
the appropriate policy and daemon.cc doesn't need to know or care what
it is.
2020-08-12 15:22:33 +00:00
John Ericson
5f80aea795
Break out lambda so output can be matched just once
...
This is much better.
2020-08-12 02:23:31 +00:00
John Ericson
18834f7764
Recheck path validity after acquiring lock
...
It might have changed, and in any event this is how the cod used to work
so let's just keep it.
2020-08-11 23:44:02 +00:00
John Ericson
2de201254e
Don't assume a total output map in two places in build.cc
...
Thanks @regnat for catching one of them. The other follows for many of
the same reasons. I'm find fixing others on a need-to-fix basis,
provided their are no regressions.
2020-08-11 23:07:50 +00:00
John Ericson
4c6aac8fdf
Clarify comment on sandbox and temp fresh paths
2020-08-11 22:46:05 +00:00
John Ericson
6d57139050
Clarify outputReferences variable with self-describing type
...
Thanks for the idea, @Regnat!
2020-08-11 22:34:09 +00:00
John Ericson
8a068bd025
Remove redundant equality check
2020-08-11 21:25:40 +00:00
John Ericson
07e3466eb4
Float comment to out describe gaveUpOnSubstitution in general
2020-08-11 21:16:14 +00:00
John Ericson
d0f6e338dd
Apply suggestions from code review
...
Thanks!!
Co-authored-by: Théophane Hufschmitt <regnat@users.noreply.github.com>
2020-08-11 16:49:10 -04:00
John Ericson
2a0902634e
Fix error in merge breaking floating CA drvs
...
Forgot to add this hunk!
2020-08-11 00:13:19 +00:00
John Ericson
1b5c24662b
Merge branch 'small-drv-serialize-cleanup' of github.com:obsidiansystems/nix into single-ca-drv-build
2020-08-10 01:57:54 +00:00
John Ericson
581183d4d5
Deduplicate parsing and reading derivations
2020-08-10 01:40:50 +00:00
John Ericson
bcd0629c2e
Remove name parameter from writeDerivation
...
The name is now stored with the derivation itself.
2020-08-10 01:35:59 +00:00
John Ericson
e913a2989f
Squashed get CA derivations building
2020-08-07 19:51:55 +00:00
Eelco Dolstra
edfd676e05
Fix .ls file names in binary caches
...
These are not supposed to include the 'name' part of the store
path. This was broken by 759947bf72
2020-08-07 21:18:29 +02:00
John Ericson
8f92bb5ad9
Merge branch 'drv-outputs-map-allow-missing' of github.com:obsidiansystems/nix into templated-daemon-protocol
2020-08-07 18:51:01 +00:00
John Ericson
f7ba16f9cb
Merge remote-tracking branch 'upstream/master' into drv-outputs-map-allow-missing
2020-08-07 17:46:39 +00:00
John Ericson
47644e49ca
Specialize std::optional<StorePath> so this is backwards compatible
...
While I am cautious to break parametricity, I think it's OK in this
cases---we're not about to try to do some crazy polymorphic protocol
anytime soon.
2020-08-07 17:05:14 +00:00
Matthew Bauer
96c158d6e1
Fix build
2020-08-06 21:04:31 -05:00
Matthew Bauer
8abc577cc2
Merge remote-tracking branch 'origin/master' into readd-hashed-mirrors
2020-08-06 18:58:59 -05:00
Carlo Nucera
46f9dd56da
Fix bug due to non-deterministic arg eval order
2020-08-06 19:30:05 -04:00
Matthew Bauer
641c950701
Add hashed-mirrors back
...
Some users have their own hashed-mirrors setup, that is used to mirror
things in addition to what’s available on tarballs.nixos.org. Although
this should be feasable to do with a Binary Cache, it’s not always
easy, since you have to remember what "name" each of the tarballs has.
Continuing to support hashed-mirrors is cheap, so it’s best to leave
support in Nix. Note that NIX_HASHED_MIRRORS is also supported in
Nixpkgs through fetchurl.nix.
Note that this excludes tarballs.nixos.org from the default, as in
\#3689. All of these are available on cache.nixos.org.
2020-08-06 18:19:09 -05:00
Carlo Nucera
9ab07e99f5
Use template structs instead of phantoms
2020-08-06 18:04:13 -04:00
Carlo Nucera
3d8240c32e
Remove leftover commented code
2020-08-06 16:04:18 -04:00
Carlo Nucera
f795f0fabc
Merge branch 'drv-outputs-map-allow-missing-namespace' of github.com:obsidiansystems/nix into templated-daemon-protocol
2020-08-06 15:53:09 -04:00
John Ericson
e89b5bd0bf
Minimize the usage of Hash::dummy
2020-08-06 18:31:48 +00:00
John Ericson
5e59b25a23
Merge remote-tracking branch 'upstream/master' into make-narHash-not-optional
2020-08-06 15:34:15 +00:00
Carlo Nucera
8b175f58d1
Simplify the namespace
2020-08-05 17:57:07 -04:00
Carlo Nucera
0739d428e0
Solve template deduction problem
...
We had to predeclare our template functions
2020-08-05 17:49:45 -04:00
John Ericson
6c66331d5c
WIP: Put the worker protocol read and write in a namespace to disambig
2020-08-05 20:37:48 +00:00
Carlo Nucera
1d2e80ddd6
Merge branch 'master' of github.com:NixOS/nix into new-interface-for-path-pathOpt
2020-08-05 15:45:33 -04:00
John Ericson
ed96e603e1
Proxy -> Phantom to match Rust
...
Sorry, Haskell.
2020-08-05 19:44:08 +00:00
Carlo Nucera
8241e660ba
Remove Hash::operator bool ()
...
Since the hash is not optional anymore
2020-08-05 15:30:38 -04:00
Carlo Nucera
be6e1c6457
Merge branch 'master' of github.com:NixOS/nix into make-narHash-not-optional
2020-08-05 15:14:47 -04:00
Carlo Nucera
1ad6394b33
Add Hash::dummy to signal default value
...
We did this in the same spirit of the dummy value that's present in
libstore/path.hh
2020-08-05 15:11:49 -04:00
Carlo Nucera
1d71028f4d
Remove optionality in ValidPathInfo::narInfo
2020-08-05 14:42:48 -04:00
John Ericson
7302761f64
Merge remote-tracking branch 'obsidian/drv-outputs-map-allow-missing' into templated-daemon-protocol
2020-08-05 17:53:24 +00:00
Ben Burdette
f1a47a96b6
error messages for issue 2238
2020-08-05 10:58:00 -06:00
John Ericson
cf939055c8
Merge remote-tracking branch 'upstream/master' into drv-outputs-map-allow-missing
2020-08-05 16:43:30 +00:00
John Ericson
0559ff3d8b
Merge branch 'master' of github.com:NixOS/nix into derivation-primop-floating-output
2020-08-05 14:56:40 +00:00
John Ericson
92ad550e96
Merge remote-tracking branch 'obsidian/misc-ca' into derivation-primop-floating-output
2020-08-05 14:51:41 +00:00
Eelco Dolstra
790b694be7
Style fix
2020-08-05 16:51:06 +02:00
Eelco Dolstra
e48f944e9d
Merge branch 'misc-ca' of https://github.com/obsidiansystems/nix
2020-08-05 16:50:05 +02:00
John Ericson
b9ebe373bb
Sed some names to perhaps avoid conflicts
2020-08-05 14:49:25 +00:00
Eelco Dolstra
25f7912156
Style fix
2020-08-05 16:47:48 +02:00
John Ericson
e561a13a58
Reanme DerivationType::Regular defintion too
...
This is the one non-prefixed occurence
2020-08-05 14:45:56 +00:00
John Ericson
e7b0847f2d
Make names more consistent
2020-08-05 14:44:39 +00:00
John Ericson
839f0fe095
Merge remote-tracking branch 'upstream/master' into misc-ca
2020-08-05 14:40:01 +00:00
John Ericson
03f4fafc27
Merge remote-tracking branch 'upstream/master' into misc-ca
2020-08-05 14:36:25 +00:00
Eelco Dolstra
b91dc7ebad
Merge pull request #3730 from obsidiansystems/better-ca-parse-errors
...
Improve hash parsing and errors
2020-08-05 16:33:07 +02:00
Eelco Dolstra
75f220a595
Merge pull request #3864 from obsidiansystems/more-topo-sort
...
Abstract out topo sorting logic
2020-08-05 16:07:29 +02:00
John Ericson
16c98bf57c
Get rid of some unneeded temporaries
2020-08-04 22:36:31 +00:00
John Ericson
1dfcbebc95
Organize and format code a bit
2020-08-04 22:28:10 +00:00
John Ericson
45b6fdb22b
Remove unused functions
2020-08-04 22:10:13 +00:00
John Ericson
1bab8a321f
Remove unneeded definition
...
Template instantiations will cover this case fine.
2020-08-04 21:56:42 +00:00
John Ericson
2f2ae993dc
WIP systematize more of the worker protocol
...
This refactor should *not* change the wire protocol.
2020-08-04 19:02:05 +00:00
John Ericson
fbeb8695fb
Merge remote-tracking branch 'upstream/master' into drv-outputs-map-allow-missing
2020-08-04 18:35:33 +00:00
John Ericson
d3452a5ed6
Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors
2020-08-04 16:13:20 +00:00
Eelco Dolstra
4e7f1c7f11
S3BinaryCacheStore: Fix size determination
2020-08-04 16:01:13 +02:00
Eelco Dolstra
dfe66420e7
Revert "Remove putBytes"
...
This reverts commit b8eea7e81a
2020-08-04 15:56:10 +02:00
Eelco Dolstra
327b1bf378
BinaryCacheStore: Explicitly flush file sink
...
The file sink is also flushed in its destructor, but we ignore any
exceptions in the destructor.
Issue #3886 .
2020-08-04 14:54:07 +02:00
Eelco Dolstra
ed52cf632b
Merge pull request #3892 from obsidiansystems/path-info-header
...
Move ValidPathInfo to its own header
2020-08-04 10:44:47 +02:00
John Ericson
062533f7cd
Merge remote-tracking branch 'upstream/master' into path-info-header
2020-08-03 21:02:28 +00:00
Eelco Dolstra
24e07c428f
Delete compressed NARs
...
Fixes #3891 .
2020-08-03 18:34:52 +02:00
John Ericson
54281f3ac1
addToStore in terms of addToStoreFromDump is not local-store-specific
2020-08-03 04:13:45 +00:00
John Ericson
9357512d73
Merge remote-tracking branch 'upstream/master' into derivation-header-include-order
2020-08-01 19:38:35 +00:00
Carlo Nucera
b6d97fdbf4
Merge branch 'master' of github.com:NixOS/nix into drv-outputs-map-allow-missing
2020-07-31 13:12:51 -04:00
Eelco Dolstra
bf290c2306
Merge remote-tracking branch 'origin/master' into markdown
2020-07-31 16:07:04 +02:00
Eelco Dolstra
1d0a7b54fa
Enable syntax highlighting
2020-07-31 15:43:25 +02:00
Carlo Nucera
eee6ef86cd
Merge branch 'master' of github.com:NixOS/nix into better-ca-parse-errors
2020-07-30 18:27:25 -04:00
Matthew Bauer
05ac4db39a
Merge remote-tracking branch 'origin/master' into substitute-other-storedir
2020-07-30 12:38:24 -05:00
Eelco Dolstra
3f6e88a552
unsigned long long -> uint64_t
2020-07-30 13:34:04 +02:00
Eelco Dolstra
ebee2b7852
receiveContents(): unsigned int -> size_t
2020-07-30 13:00:30 +02:00
Eelco Dolstra
f63839bfa4
Cleanup
2020-07-29 18:04:51 +02:00
Eelco Dolstra
4c0077a07d
Fix RemoteStore::addToStore() latency
...
Since 6185d25e52Fixes #3372 .
2020-07-29 00:48:39 +02:00
Eelco Dolstra
c159f48a39
Cleanup
2020-07-29 00:24:55 +02:00
John Ericson
df707d05d1
Merge branch 'ca-drv' of github.com:Ericson2314/nix into misc-ca
2020-07-28 21:12:36 +00:00
Carlo Nucera
2980b244b7
Use assert(false) instead of abort()
2020-07-28 15:39:45 -04:00
Carlo Nucera
cfe96bd7ce
Merge branch 'master' of github.com:NixOS/nix into ca-drv
2020-07-28 15:36:45 -04:00
Carlo Nucera
c318d398f3
Merge branch 'misc-ca' of github.com:obsidiansystems/nix into new-interface-for-path-pathOpt
2020-07-28 14:22:24 -04:00
Carlo Nucera
7ef1e3cd14
Use the new interface
2020-07-28 13:59:24 -04:00
Carlo Nucera
d564ac1c50
Offer a safer interface for pathOpt
...
The new interface we offer provides a way of getting all the
DerivationOutputs with the storePaths directly, based on the observation
that it's the most common usecase.
2020-07-28 13:55:15 -04:00
Eelco Dolstra
b8eea7e81a
Remove putBytes
...
istream->tellg() returns -1 so we can't get the number of bytes
written.
Fixes 'uploaded 's3://nix-cache/nar/00819r9lp5kajr6baxfw5dhhc0cx8ndxaz43qmd2f0gn1hk1ynlp.nar.xz' (-1 bytes) in 11620 ms' messages.
2020-07-28 13:27:56 +02:00
John Ericson
8065c6d160
Abstract out topo sorting logic
2020-07-27 20:45:34 +00:00
John Ericson
951415b568
Require ca-derivations everywhere we create a CA derivation
...
"create" as in read one in from a serialized form, or build one from
scratch in memory.
2020-07-27 17:56:36 +00:00
John Ericson
e32a9e124b
Merge branch 'misc-ca' of github.com:obsidiansystems/nix into derivation-primop-floating-output
2020-07-27 17:50:06 +00:00
John Ericson
7cf978440c
Merge branch 'ca-derivation-data-types' of github.com:obsidiansystems/nix into misc-ca
2020-07-27 17:47:40 +00:00
John Ericson
e6115ca004
Merge remote-tracking branch 'upstream/master' into ca-derivation-data-types
2020-07-27 17:41:43 +00:00
John Ericson
1d7d94ceea
Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors
2020-07-27 16:17:50 +00:00
John Ericson
43f2bd8dc5
Merge remote-tracking branch 'upstream/master' into hash-always-has-type
2020-07-27 16:13:57 +00:00
John Ericson
387699fe7f
Merge branch 'ca-derivation-data-types' of github.com:obsidiansystems/nix into misc-ca
2020-07-27 15:00:28 +00:00
John Ericson
78466bcb2f
Merge branch 'optional-derivation-output-storepath' into ca-derivation-data-types
2020-07-27 14:40:08 +00:00
John Ericson
d5bb67cfa4
Merge remote-tracking branch 'upstream/master' into optional-derivation-output-storepath
2020-07-27 14:29:32 +00:00
John Ericson
699fc89b39
Merge remote-tracking branch 'upstream/master' into hash-always-has-type
2020-07-27 14:22:45 +00:00
John Ericson
2c7557481b
queryDerivationOutputMap no longer assumes all outputs have a mapping
...
This assumption is broken by CA derivations. Making a PR now to do the
breaking daemon change as soon as possible (if it is already too late,
we can bump protocol intead).
2020-07-24 21:14:06 +00:00
John Ericson
58e55c0923
Merge branch 'misc-ca' of github.com:obsidiansystems/nix into derivation-primop-floating-output
2020-07-23 19:17:52 +00:00
John Ericson
a2af25c55a
Merge remote-tracking branch 'upstream/master' into misc-ca
2020-07-23 19:14:17 +00:00
John Ericson
c36916eca0
Merge branch 'misc-ca' of github.com:obsidiansystems/nix into derivation-primop-floating-output
2020-07-23 15:06:56 +00:00
John Ericson
6fcd9966bf
Merge remote-tracking branch 'upstream/master' into misc-ca
2020-07-23 15:02:29 +00:00
John Ericson
66a2067288
Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors
2020-07-23 14:56:35 +00:00
John Ericson
2274f63453
Get rid of basicDerivation::findOutput
...
It's a tiny function which is:
- hardly worth abstrating over, and also only used once.
- doesn't work once we get CA drvs
I rewrote the one callsite to be forwards compatable with CA
derivations, and also potentially more performant: instead of reading in
the derivation it can ust consult the SQLite DB in the common case.
2020-07-23 14:39:02 +00:00
John Ericson
9423f64ee2
Parse CA derivations using new output variants
...
We no longer need `ParsedDerivation` because everything libstore needs
to know about is in the `BasicDerivation` proper.
2020-07-22 23:59:25 +00:00
John Ericson
c56356bacc
Separate concerns in scanForReferences with TeeSink
...
This also will make it easier to use a `HashModuloSink` instead for CA
derivations.
2020-07-22 22:13:44 +00:00
Eelco Dolstra
ff314f186e
Merge pull request #3736 from obsidiansystems/allow-relative-paths-in-store-option
...
Allow relative paths in --store option
2020-07-21 19:43:07 +02:00
Eelco Dolstra
0951330680
Merge pull request #3714 from obsidiansystems/add-body-to-network-errors
...
Add response body to network errors
2020-07-21 18:09:42 +02:00
John Ericson
02639716ea
Merge branch 'allow-relative-paths-in-store-option' into remove-storetype-delegate-regStore
2020-07-21 15:55:27 +00:00
John Ericson
6cce32c8e8
Change logic for deciding what is a relative path for the local store
...
The was Eelco's prefered logic, and it looks good to me!
2020-07-21 15:39:47 +00:00
Carlo Nucera
922a845ffc
Update chunkSize to the suggested value
...
This was a suggested course of action in a review in one of our earlier
commits, https://github.com/NixOS/nix/pull/3801#discussion_r457557079
2020-07-21 10:24:19 -04:00
John Ericson
54e507a7aa
Merge remote-tracking branch 'upstream/master' into add-body-to-network-errors
2020-07-21 14:17:59 +00:00
John Ericson
5ce95b9529
Update src/libstore/build.cc
2020-07-21 09:47:40 -04:00
John Ericson
cdb3f39b64
Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors
2020-07-21 13:43:20 +00:00
John Ericson
5055c595bd
Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into ca-derivation-data-types
2020-07-21 01:20:53 +00:00
John Ericson
5a37413796
Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into from-dump-stream
2020-07-21 01:20:14 +00:00
John Ericson
bf0dca7790
Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into misc-ca
2020-07-21 01:18:33 +00:00
Carlo Nucera
9aae179f34
Correct bug, thoroughly document addToStoreSlow
2020-07-20 20:18:12 -04:00
Carlo Nucera
0ca9744694
Use heuristics to decide when to show the response
...
Due to https://github.com/NixOS/nix/issues/3841 we don't know how print
different messages for different verbosity levels.
2020-07-20 15:57:58 -04:00
Carlo Nucera
6357b1b0fb
Add another Unimplemented case
2020-07-20 14:17:25 -04:00
Carlo Nucera
007bf6d17a
Merge branch 'ca-derivation-data-types' of github.com:obsidiansystems/nix into misc-ca
2020-07-20 14:14:39 -04:00
Carlo Nucera
362ae93851
Add UnimplementedError to ease grepping for these
2020-07-20 14:13:37 -04:00
John Ericson
bf61871271
parser.hh -> split.hh
2020-07-20 17:42:34 +00:00
John Ericson
eab945f591
Merge remote-tracking branch 'upstream/master' into from-dump-stream
2020-07-20 17:29:23 +00:00
Eelco Dolstra
1c5f8bbfb5
Merge pull request #3822 from obsidiansystems/dump-thrice-fixme
...
Optimize `addToStoreSlow` and remove `TeeParseSink`
2020-07-20 18:55:05 +02:00
John Ericson
ac2fc7ba1f
Apply suggestions from code review
...
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2020-07-20 11:29:46 -04:00
Carlo Nucera
0aa79dcc6f
Remove StoreType abstraction and delegate regStore
...
to each Store implementation. The generic regStore implementation will
only be for the ambiguous shorthands, like "" and "auto".
This also could get us close to simplifying the daemon command.
2020-07-17 17:24:02 -04:00
John Ericson
6756cecfcf
Add DerivationType::CAFloating
2020-07-17 19:55:41 +00:00
Carlo Nucera
fefd6c9e5f
Merge branch 'master' of github.com:NixOS/nix into allow-relative-paths-in-store-option
2020-07-17 14:34:20 -04:00
Carlo Nucera
bbc633c98c
Revert "Don't anticipate CA but not fixed outputs for now"
...
This reverts commit 3a9e4c3262
2020-07-17 13:10:32 -04:00
Carlo Nucera
205dcd140d
Revert "Don't anticipate multiple CA outputs for now"
...
This reverts commit 74b251b2f3
2020-07-17 12:43:46 -04:00
Carlo Nucera
1feb8981df
Revert "Don't anticipate hash algo without hash in derivation for now"
...
This reverts commit 3804e3df9b
2020-07-17 12:05:54 -04:00
Carlo Nucera
487c5751c6
Merge branch 'ca-drv' of github.com:Ericson2314/nix into misc-ca
2020-07-17 11:32:59 -04:00
Carlo Nucera
5cb840541b
Merge branch 'multi-output-hashDerivationModulo' of github.com:Ericson2314/nix into misc-ca
2020-07-17 10:28:33 -04:00
Carlo Nucera
745a03cef5
Merge branch 'optional-derivation-output-storepath' of github.com:obsidiansystems/nix into ca-derivation-data-types
2020-07-16 13:36:01 -04:00
Carlo Nucera
048e916f64
Merge branch 'master' of github.com:NixOS/nix into optional-derivation-output-storepath
2020-07-16 13:32:28 -04:00
John Ericson
cc0d77f8c9
Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors
2020-07-16 17:28:52 +00:00
Carlo Nucera
e3a3406db8
Merge branch 'master' of github.com:NixOS/nix into add-body-to-network-errors
2020-07-16 12:04:10 -04:00
John Ericson
5ea817dace
Merge remote-tracking branch 'upstream/master' into hash-always-has-type
2020-07-16 14:58:53 +00:00
John Ericson
3dcca18c30
Fix bug in TeeSource
...
We use this to simplify `LocalStore::addToStoreFromDump`.
Also, hope I fixed build error with old clang (used in Darwin CI).
2020-07-16 13:39:27 +00:00
Carlo Nucera
e3b394b6e8
Small namespace fix
2020-07-16 09:36:02 -04:00
Eelco Dolstra
5517eee17e
Generations API cleanup
2020-07-16 15:18:21 +02:00
Eelco Dolstra
3f264916db
Merge remote-tracking branch 'origin/flakes'
2020-07-16 14:07:32 +02:00
John Ericson
68dfb8c6ae
Optimize addToStoreSlow and remove TeeParseSink
2020-07-16 05:09:41 +00:00
John Ericson
5602637d9e
Revert "LocalStore::addToStoreFromDump copy in chunks"
...
This reverts commit 592851fb67
2020-07-15 23:37:49 +00:00
John Ericson
bc109648c4
Get rid of LocalStore::addToStoreCommon
...
I got it to just become `LocalStore::addToStoreFromDump`, cleanly taking
a store and then doing nothing too fancy with it.
`LocalStore::addToStore(...Path...)` is now just a simple wrapper with a
bare-bones sinkToSource of the right dump command.
2020-07-15 23:14:30 +00:00
Carlo Nucera
455bdee205
Merge branch 'master' of github.com:NixOS/nix into derivation-header-include-order
2020-07-15 17:58:30 -04:00
Carlo Nucera
d090562348
Merge branch 'master' of github.com:NixOS/nix into hash-always-has-type
2020-07-15 17:21:01 -04:00
John Ericson
aff69e229c
Merge remote-tracking branch 'obsidian/fix-add-to-star-narhash' into from-dump-stream
2020-07-15 21:05:29 +00:00
Carlo Nucera
2d2a10e79a
Merge branch 'master' of github.com:NixOS/nix into add-body-to-network-errors
2020-07-15 13:06:08 -04:00
Eelco Dolstra
cff2157185
Revert "LocalStore::addToStore(srcPath): Handle the flat case"
...
This reverts commit a2c27022e9
2020-07-15 12:49:03 +02:00
John Ericson
650c2c6558
Rename variable nar -> dump according to TODO
2020-07-14 21:28:50 +00:00
John Ericson
8173e7bfef
Fix localhost::addToStore(...Path...)
...
We were calculating the nar hash wrong when the file ingestion method
was flat. I don't think there's anything we can do in that case but dump
the file again, so that's what I do.
As an optomization, we again could reuse the original dump for just the
recursive and non-sha256 case, but I rather do that after this fix, and
after my other PRs which deduplicate this code.
2020-07-14 21:12:11 +00:00
John Ericson
592851fb67
LocalStore::addToStoreFromDump copy in chunks
...
Rather than copying byte-by-byte, we let the coroutine know how much
data we would like it to send back to us.
2020-07-14 13:56:36 +00:00
John Ericson
9de96ef7d4
Dedup LocalStore::addToStore*
...
The downsides is that the coroutine has byte-by-byte loop transfer. Will
fix that next.
2020-07-14 13:56:36 +00:00
John Ericson
c86fc3a965
Crudely make addToStoreFromDump take Source not string
...
I just as little beyond the type as possible, so the implementation
changes this enables can be reviewed separately.
2020-07-14 13:56:36 +00:00
John Ericson
9ec10046e0
Narrow scope of temporary value
2020-07-14 13:54:29 +00:00
Eelco Dolstra
832e111494
Merge remote-tracking branch 'origin/master' into flakes
2020-07-14 13:56:18 +02:00
Eelco Dolstra
926c3a6664
Doh
2020-07-14 11:55:54 +02:00
Eelco Dolstra
43b8e96d30
Fix 'nix verify --all' on a binary cache (cached case)
2020-07-13 20:17:00 +02:00
Eelco Dolstra
7c2fef0a81
Make 'nix copy' to s3:// binary caches run in constant memory
2020-07-13 20:07:19 +02:00
Eelco Dolstra
493961b689
Remove istringstream_nocopy
2020-07-13 18:31:19 +02:00
Eelco Dolstra
545bb2ed03
Remove 'accessor' from addToStore()
...
This is only used by hydra-queue-runner and it's better to implement
it there.
2020-07-13 18:31:19 +02:00
Eelco Dolstra
0a9da00a10
NarAccessor: Run in constant memory
2020-07-13 17:30:42 +02:00
Eelco Dolstra
fc84c358d9
Make 'nix copy' to file:// binary caches run in constant memory
2020-07-13 16:28:45 +02:00
Eelco Dolstra
400f1a9b59
Store::pathInfoToJSON(): Use consistent format for downloadHash
2020-07-13 16:26:09 +02:00
Eelco Dolstra
c0dd05131e
toStorePath(): Return a StorePath and the suffix
2020-07-13 16:25:48 +02:00
Eelco Dolstra
1d01ae816b
Fix 'nix verify --all' on a binary cache and add a test
2020-07-13 14:35:01 +02:00
John Ericson
c466cb2091
Merge branch 'hash-always-has-type' into better-ca-parse-errors
2020-07-13 03:02:09 +00:00
John Ericson
4415765385
Merge remote-tracking branch 'upstream/master' into hash-always-has-type
2020-07-13 03:01:44 +00:00
John Ericson
230c9b4329
Change types to prepare the way for CA derivations
...
We've added the variant to `DerivationOutput` to support them, but made
`DerivationOutput::path` partial to avoid actually implementing them.
With this chage, we can all collaborate on "just" removing
`DerivationOutput::path` calls to implement CA derivations.
2020-07-12 23:53:33 +00:00
John Ericson
fedfc913ad
Use more std::visit to prepare for new variant
...
N.B. not using `std::visit` for fetchurl because there is no attempt to
handle all the cases (e.g. no `else`) and lambda complicates early
return.
2020-07-12 23:51:40 +00:00
John Ericson
ffc18583b1
Move C++17 "pattern matching" boilerplat to utils.hh
2020-07-12 22:15:14 +00:00
John Ericson
503b425690
DerivationOutputExtensional -> DerivationOutputInputAddressed
...
Thanks @regnat for the great name.
2020-07-12 15:56:20 +00:00
John Ericson
a8d4707107
Undo erroneous indentation change
2020-07-12 15:54:12 +00:00
John Ericson
18152406ce
String .drv suffix to create derivation name
2020-07-12 15:40:14 +00:00
John Ericson
5d0b75e5b6
std::string_view for new derivation name parameters
2020-07-12 15:02:36 +00:00
John Ericson
abea26a968
BasicDerivation::findOutput cannot return reference anymore
2020-07-12 01:57:06 +00:00
John Ericson
442d43178b
Merge remote-tracking branch 'upstream/master' into optional-derivation-output-storepath
2020-07-11 02:27:45 +00:00
Matthew Bauer
acb74d4d94
Merge remote-tracking branch 'origin/master' into substitute-other-storedir
2020-07-10 18:13:45 -04:00
Eelco Dolstra
8efa23bb99
Avoid a redundant hash
2020-07-10 15:56:24 +02:00
