Matthew Bauer
f2a6cee334
Update worker protocol to support sending storepath maps
...
We need to also send the ca to daemon in addition to the path.
2020-06-19 18:06:19 -04:00
John Ericson
507aa48739
WIP: Make Hash always store a valid hash type
2020-06-19 21:48:57 +00:00
Matthew Bauer
e8e1f5282f
Replace error message in getDerivationCA
2020-06-19 15:19:09 -04:00
Matthew Bauer
0c9c1b8826
Return map of StorePaths in copyPaths
...
This allows the caller to know what values were actually added to the
store.
2020-06-19 14:48:57 -04:00
Matthew Bauer
7e11cf3399
Merge remote-tracking branch 'origin/master' into substitute-other-storedir
2020-06-19 14:40:49 -04:00
Matthew Bauer
2796b01e86
Merge remote-tracking branch 'origin/master' into substitute-other-storedir
2020-06-19 14:22:06 -04:00
John Ericson
68294746ae
Merge remote-tracking branch 'upstream/master' into no-hash-type-unknown
2020-06-19 17:53:34 +00:00
John Ericson
911fc88bcb
More designated initializers
2020-06-19 17:42:56 +00:00
John Ericson
2f0e395c99
Merge remote-tracking branch 'me/no-stringly-typed-derivation-output' into validPathInfo-ca-proper-datatype
2020-06-19 15:26:59 +00:00
John Ericson
fb39a5e00c
Remove unneeded constructor for DerivationOutputHash
2020-06-19 15:11:11 +00:00
John Ericson
01dc8b0bab
Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output
2020-06-19 14:59:05 +00:00
John Ericson
145d88cb2a
Use designated initializers for DerivationOutputHash
2020-06-19 14:58:30 +00:00
John Ericson
237d88c97e
FileSystemHash -> DerivationOutputHash
2020-06-19 14:47:10 +00:00
John Ericson
3fc58a9638
Remove some Base::
that crept in
2020-06-19 00:24:47 +00:00
John Ericson
3f8dcfe3fd
Merge branch 'validPathInfo-temp' into validPathInfo-ca-proper-datatype
2020-06-18 23:01:58 +00:00
John Ericson
669c3992e8
Merge branch 'no-hash-type-unknown' into validPathInfo-temp
2020-06-18 22:33:07 +00:00
John Ericson
15abb2aa2b
Revert the enum struct
change
...
Not a regular git revert as there have been many merges and things.
2020-06-18 22:11:26 +00:00
John Ericson
bbbf3602a3
Merge branch 'enum-class' into no-hash-type-unknown
2020-06-18 22:11:19 +00:00
John Ericson
40526fbea5
Merge remote-tracking branch 'upstream/master' into enum-class
2020-06-18 21:38:15 +00:00
John Ericson
1b23fe4afb
Fix bugs
...
- Bad dynamic cast target ...classic
- std::shared_ptr need explicit deref
2020-06-18 19:03:10 +00:00
John Ericson
fb432b0bd2
Merge remote-tracking branch 'obsidian/fix-url-format' into add-body-to-network-errors
2020-06-18 18:08:02 +00:00
John Ericson
639e20dc3e
Prevent '%' in URL from causing crashes
...
We have a larger problem that passsing computed strings to the first
variable argument of many exception constructors is unsafe because that
first variable argument is interpreted not as a plain string, but format
string, and if it contains '%' boost::format will abort, since there are
no arguments to the format string.
In this particular instance '%' was used as part of an escape code in a
URL, which, when the download failed, caused Nix to abort displaying the
`FileTransferError`.
2020-06-18 17:54:16 +00:00
John Ericson
74b219ef6e
Adjust FileTransferError message to use opt response
2020-06-18 14:48:45 +00:00
regnat
4fef2ba7e4
Rename content-addressed-paths into ca-derivations
...
See <https://github.com/NixOS/nix/pull/3710#issuecomment-645480333 >
2020-06-18 09:25:55 +02:00
Carlo Nucera
004570a377
Add HTTP responses to FileTransferErrors
2020-06-17 22:34:31 +00:00
Carlo Nucera
a835c740ca
Replace TransferItem::status
with a local variable
...
Everywhere seems to use `getHTTPStatus` now.
2020-06-17 22:16:24 +00:00
Carlo Nucera
fb05a6adcf
Eliminate old TeeSink abstraction
...
This was introduced in fa125b9b28
, and
then "reverted" in 1cf4801108
, except that
revert left the struct around doing nothing useful.
We're removing it all the way now because we want to make a new
`TeeSink` complementing the already-exiting `TeeSource`, that is
actually a completely different concept as far as the class hierarchy is
concerned.
2020-06-17 22:07:22 +00:00
Eelco Dolstra
2b8f33bf5f
Merge pull request #3713 from matthewbauer/cleanup-warnings
...
Cleanup class StorePath warning
2020-06-17 21:19:30 +02:00
Matthew Bauer
5e631e3304
Add StorePathCAMap for querySubstitutablePathInfos
...
I’m not 100% sure this is wanted since it kind of makes everything
have to know about ca even if they don’t really want to. But it also
make things easier in dealing with looking up ca.
2020-06-17 15:03:05 -04:00
Matthew Bauer
be50de1142
Make sure references are empty for store path replacing
...
also copy info2 instead of casting
2020-06-17 14:15:08 -04:00
Matthew Bauer
8974755d19
Add assert for replaced storePath
2020-06-17 14:04:46 -04:00
Eelco Dolstra
3078404e35
Merge pull request #3712 from obsidiansystems/make-http-successful-states-coherent
...
Make successful states coherent
2020-06-17 19:37:46 +02:00
Matthew Bauer
7bd88cc1dc
Merge remote-tracking branch 'origin/master' into substitute-other-storedir
2020-06-17 13:32:29 -04:00
Matthew Bauer
22d7d36703
Remove unused narInfoFile in binary-cache-store
2020-06-17 13:27:19 -04:00
Matthew Bauer
f767bedfac
Replace struct StorePath with class StorePath
...
also a similar case with struct Goal
2020-06-17 13:26:37 -04:00
Carlo Nucera
4930cb48a2
Include review comments
2020-06-17 12:58:59 -04:00
Matthew Bauer
6438ba1e99
Update strings from review comment
2020-06-17 12:31:01 -04:00
Eelco Dolstra
2f51cd8dc9
Merge pull request #3710 from tweag/reserve_ca_derivations
...
Reserve the `__contentAddressed` derivation parameter
2020-06-17 18:28:26 +02:00
Eelco Dolstra
4d5169bdd5
Merge pull request #3707 from p01arst0rm/outdated-function-fix
...
replaced uncaught_exception with uncaught_exceptions
2020-06-17 18:26:01 +02:00
regnat
480b54e1c6
fixup! Reserve the __contentAddressed
derivation parameter
2020-06-17 17:37:04 +02:00
Carlo Nucera
079c6e87de
Make successful states coherent
...
The successful states used in these two places in the code were slightly
different. Should they be the same list?
2020-06-17 11:16:16 -04:00
regnat
56d75bf4fc
Reserve the __contentAddressed
derivation parameter
...
Not implementing anything here, just throwing an error if a derivation
sets `__contentAddressed = true` without
`--experimental-features content-addressed-paths`
(and also with it as there's nothing implemented yet)
2020-06-17 15:41:17 +02:00
Eelco Dolstra
fdff09e57c
Fix coverage build
2020-06-17 15:18:10 +02:00
Eelco Dolstra
ccbea8255c
Merge pull request #3657 from obsidiansystems/sligthly-improve-store-path-documentation
...
Clarify the description of StorePath inputs
2020-06-17 14:54:37 +02:00
Eelco Dolstra
1524752c17
Merge remote-tracking branch 'origin/master' into flakes
2020-06-17 10:26:52 +02:00
John Ericson
517f5980e2
Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output
2020-06-17 04:58:43 +00:00
John Ericson
02928f76fd
Merge remote-tracking branch 'upstream/master' into multi-output-hashDerivationModulo
2020-06-17 03:51:02 +00:00
John Ericson
49308ef039
Merge remote-tracking branch 'upstream/master' into ca-drv
2020-06-17 03:39:39 +00:00
p01arst0rm
c9d06558b6
replaced uncaught_exception with uncaught_exceptions
2020-06-17 03:15:47 +01:00
Eelco Dolstra
29542865ce
Remove StorePath::clone() and related functions
2020-06-16 22:20:18 +02:00
Eelco Dolstra
759947bf72
StorePath: Rewrite in C++
...
On nix-env -qa -f '<nixpkgs>', this reduces maximum RSS by 20970 KiB
and runtime by 0.8%. This is mostly because we're not parsing the hash
part as a hash anymore (just validating that it consists of base-32
characters).
Also, replace storePathToHash() by StorePath::hashPart().
2020-06-16 14:28:41 +02:00
Eelco Dolstra
72e17290d4
Fix FTP support
...
Fixes #3618 .
2020-06-16 11:53:04 +02:00
Eelco Dolstra
a588b6b19d
Print only one error message if a build fails
...
E.g. instead of
error: --- BuildError ----------------------------------------------- nix
builder for '/nix/store/03nk0a3n8h2948k4lqfgnnmym7knkcma-foo.drv' failed with exit code 1
error: --- Error ---------------------------------------------------- nix
build of '/nix/store/03nk0a3n8h2948k4lqfgnnmym7knkcma-foo.drv' failed
we now get
error: --- Error ---------------------------------------------------- nix
builder for '/nix/store/03nk0a3n8h2948k4lqfgnnmym7knkcma-foo.drv' failed with exit code 1
2020-06-15 19:35:31 +02:00
Eelco Dolstra
31707735b6
Remove unnecessary amDone() overrides
2020-06-15 16:47:21 +02:00
Eelco Dolstra
5ed5d7acbd
Improve "waiting for locks" messages
...
These are now shown in the progress bar.
Closes #3577 .
2020-06-15 16:03:29 +02:00
Eelco Dolstra
e14e62fddd
Remove trailing whitespace
2020-06-15 14:12:39 +02:00
Eelco Dolstra
1fb762d11f
Get rid of explicit ErrorInfo constructors
2020-06-15 14:06:58 +02:00
Eelco Dolstra
7a77762961
Merge branch 'errors-phase-2' of https://github.com/bburdette/nix
2020-06-15 11:46:31 +02:00
Matthew Bauer
c214cda940
Correctly substitute from different storeDir
...
Originally, the test was only checking for different “real” storeDir.
That’s an easy case to handle, but the much harder one is if different
virtual store dirs are used. To do this, we need the SubstitutionGoal
to know about the ca, so it can recalculate the path to copy it over.
An important note here is that the store path passed to copyStorePath
needs to be one for srcStore - so that queryPathInfo works properly.
This also adds an error message when the store path from queryPathInfo
is different from the one we requested.
2020-06-13 00:07:42 -05:00
Matthew Bauer
b2cb288cdd
Add makeFixedOutputPathFromCA function
...
This puts what we are already doing into a shared method. It just
needs a path name and a ca and produces a store path.
2020-06-12 16:36:35 -05:00
John Ericson
f6f01416b7
Use std::string_view
in a few more places
2020-06-12 21:32:30 +00:00
Matthew Bauer
88120442d2
Debug when storePath changes
...
these rewrites should be transparent, but they are important to know
about when debugging
2020-06-12 15:32:52 -05:00
Matthew Bauer
5b386b05f5
Recompute storePath based on isContentAddressed
2020-06-12 14:57:45 -05:00
Matthew Bauer
85d01e1f65
Don’t use makeStorePath
2020-06-12 14:53:30 -05:00
Matthew Bauer
9077b9e6b2
Separate dstStore path from srcStore path
2020-06-12 14:27:28 -05:00
Matthew Bauer
2f2ac850b5
Compute new store path correctly
2020-06-12 14:03:17 -05:00
Matthew Bauer
11c97070f3
Fix storeDir != storeDir condition
...
this needs to only continue if the path replacement fails.
2020-06-12 10:14:03 -05:00
Matthew Bauer
3e3eaa90dd
Remove hashed-mirrors
2020-06-12 09:51:44 -05:00
Matthew Bauer
79c169d1c6
Allow substituting from different storeDir
...
Substituters can substitute from one store dir to another with a
little bit of help. The store api just needs to have a CA so it can
recompute the store path based on the new store dir. We can only do
this for fixed output derivations with no references, though.
2020-06-12 09:49:09 -05:00
Eelco Dolstra
9f736dd89d
Add Store::readDerivation() convenience function
2020-06-12 13:04:52 +02:00
Eelco Dolstra
045b07200c
Remove Store::queryDerivationOutputNames()
...
This function was used in only one place, where it could easily be
replaced by readDerivation() since it's not
performance-critical. (This function appears to have been modelled
after queryDerivationOutputs(), which exists only to make the garbage
collector faster.)
2020-06-12 12:46:33 +02:00
Ben Burdette
ef1b3f21b6
Merge remote-tracking branch 'upstream/master' into errors-phase-2
2020-06-11 14:06:35 -06:00
Eelco Dolstra
ac4d43a31b
Merge pull request #3073 from tweag/machine-logs
...
Add an option to print the logs in a machine-readable format
2020-06-11 15:45:18 +02:00
Eelco Dolstra
dd9bb11d0d
Move names.{cc,hh} to libstore
2020-06-11 15:42:18 +02:00
Eelco Dolstra
b9ae1bdd7a
Merge pull request #3655 from zimbatm/hash-encoding-prepare
...
libutils/hash: remove default encoding
2020-06-10 11:48:38 +02:00
Matthew Bauer
b2c8061b44
Disable extra-platforms = i686-linux on wsl1 ( #3676 )
...
WSL1 doesn’t support i686-linux emulation, see https://github.com/microsoft/wsl/issues/2468
2020-06-09 21:53:53 +00:00
Eelco Dolstra
29e0748847
Show HTTP status message
...
For example:
warning: unable to download 'https://api.github.com/repos/edolstra/dwarffs/commits/master ': HTTP error 403 ('rate limit exceeded'); using cached version
2020-06-09 14:20:22 +02:00
Eelco Dolstra
447ea52b07
FileTransfer: Don't store status since curl already does that
2020-06-09 14:05:15 +02:00
regnat
4983401440
Unify the printing of the logs between bar-with-logs and raw
...
Make the printing of the build logs systematically go through the
logger, and replicate the behavior of `no-build-output` by having two
different loggers (one that prints the build logs and one that doesn't)
2020-06-08 09:31:15 +02:00
Carlo Nucera
d614166cb6
Fix condition error and make test suite pass
2020-06-04 17:21:21 -04:00
John Ericson
e5cc1ebc5d
Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output
2020-06-04 21:04:35 +00:00
John Ericson
94ddea9e2f
Use readString
rather than >> temporary
...
Fixed the rest of these before, but this one slipped through.
2020-06-04 20:55:08 +00:00
John Ericson
2041499b5e
Flip boolean
...
Thanks Matt!
2020-06-04 20:42:25 +00:00
John Ericson
ed86acf02a
Use some std::optional::has_value
for clarity
2020-06-04 20:42:02 +00:00
Matthew Bauer
2299ef705c
Add error message when FileIngestionMethod is out of bounds
...
bool coerces anything >0 to true, but in the future we may have other
file ingestion methods. This shows a better error message when the
“recursive” byte isn’t 1.
2020-06-04 11:32:39 -05:00
John Ericson
53bc8ff152
No C++ designated initializers yet with Clang 7
2020-06-03 20:45:14 -04:00
Ben Burdette
4335ba999b
Merge remote-tracking branch 'upstream/master' into errors-phase-2
2020-06-03 17:00:00 -06:00
Carlo Nucera
132d6f2c24
Clarify the description of StorePath construction
2020-06-03 16:08:32 -04:00
John Ericson
74b251b2f3
Don't anticipate multiple CA outputs for now
2020-06-03 18:53:04 +00:00
John Ericson
3a9e4c3262
Don't anticipate CA but not fixed outputs for now
2020-06-03 18:50:45 +00:00
John Ericson
2500403059
Use enum and predicates rather than bitfile for derivation type
2020-06-03 17:38:54 +00:00
John Ericson
6b7f4ec4ab
Merge remote-tracking branch 'upstream/master' into ca-drv
2020-06-03 16:36:06 +00:00
Eelco Dolstra
c20591ddc3
Merge remote-tracking branch 'origin/master' into flakes
2020-06-03 16:15:22 +02:00
Jonas Chevalier
6ee03b8444
libutils/hash: remove default encoding
...
This will make it easier to reason about the hash encoding and switch to
SRI everywhere where possible.
2020-06-03 13:49:51 +02:00
John Ericson
3c78ac348c
Merge remote-tracking branch 'obsidian/no-hash-type-unknown' into validPathInfo-ca-proper-datatype
2020-06-03 04:44:24 +00:00
John Ericson
406dbb7fce
outputHashAlgo
can be blank so parse accordingly
...
It is blank for SRI hashes.
2020-06-02 21:09:15 +00:00
Carlo Nucera
75d2581390
Typo
2020-06-02 16:21:18 -04:00
Carlo Nucera
78f137e931
Validate text version instead, throw Errors
2020-06-02 16:20:22 -04:00
Carlo Nucera
a5cdf1867e
Add assertions for SHA256 in fixed case
2020-06-02 16:13:08 -04:00
Carlo Nucera
fd2eb41e64
Move file-hash to content-address
2020-06-02 15:44:58 -04:00
Carlo Nucera
343c20a404
WIP Completed implementation
2020-06-02 15:23:21 -04:00
Carlo Nucera
390bf64858
WIP
2020-06-02 14:15:58 -04:00
John Ericson
a33270ce1d
Clean up ValidPathInfo::isContentAddressed
with std::visit
2020-06-02 17:04:21 +00:00
John Ericson
25e61812f3
Apply suggestions from code review
...
Co-authored-by: Matthew Bauer <mjbauer95@gmail.com>
2020-06-02 12:47:18 -04:00
John Ericson
64cffb804a
Merge remote-tracking branch 'upstream/master' into no-hash-type-unknown
2020-06-02 16:07:25 +00:00
John Ericson
450dcf2c1b
Remove HashType::Unknown
...
Instead, `Hash` uses `std::optional<HashType>`. In the future, we may
also make `Hash` itself require a known hash type, encoraging people to
use `std::optional<Hash>` instead.
2020-06-02 15:52:13 +00:00
Carlo Nucera
a5d820a0a3
Change parseCa(Opt) to parseContentAddress(Opt)
2020-06-02 11:00:10 -04:00
John Ericson
1b6461f671
Merge remote-tracking branch 'upstream/master' into validPathInfo-ca-proper-datatype
2020-06-02 14:31:18 +00:00
Ben Burdette
d82d230b40
elide the 'ErrorInfo' in logError and logWarning calls
2020-06-02 08:22:24 -06:00
John Ericson
efcd30da89
WIP
2020-06-02 00:37:43 +00:00
John Ericson
754c910953
WIP more progress
2020-06-01 19:26:40 -04:00
Carlo Nucera
da39092a39
WIP
2020-06-01 18:53:31 -04:00
Carlo Nucera
0e9438b6d3
Create new file-hash files
2020-06-01 17:32:40 -04:00
Carlo Nucera
0cb67ecbd3
Merge branch 'derivation-header-include-order' of github.com:Ericson2314/nix into validPathInfo-ca-proper-datatype
2020-06-01 17:13:11 -04:00
Carlo Nucera
f4b89e11a4
Merge branch 'no-stringly-typed-derivation-output' of github.com:Ericson2314/nix into validPathInfo-ca-proper-datatype
2020-06-01 17:12:50 -04:00
John Ericson
fac0c2d54a
Remove addToStore variant as requested by FIXME
...
The idea is it's always more flexible to consumer a `Source` than a
plain string, and it might even reduce memory consumption.
I also looked at `addToStoreFromDump` with its `// FIXME: remove?`, but
the worked needed for that is far more up for interpretation, so I
punted for now.
2020-05-29 17:02:32 -04:00
Ben Burdette
734283d636
Merge remote-tracking branch 'upstream/master' into errors-phase-2
2020-05-29 09:51:37 -06:00
Carlo Nucera
6dd471ebf6
Fixing the result of merge
2020-05-28 12:14:36 -04:00
Carlo Nucera
4f597fb901
Merge branch 'master' of github.com:NixOS/nix into enum-class
2020-05-28 10:58:22 -04:00
John Ericson
5b4cd84bc2
Merge remote-tracking branch 'me/more-rust-ffi' into no-stringly-typed-derivation-output
2020-05-28 10:35:53 -04:00
John Ericson
ef71caba29
Merge remote-tracking branch 'upstream/master' into more-rust-ffi
2020-05-28 10:31:46 -04:00
Eelco Dolstra
17ca997fc6
Merge remote-tracking branch 'origin/master' into flakes
2020-05-28 12:55:24 +02:00
Matthew Bauer
c66441a646
Rename some variables named “recursive” to “method”
...
This is much less confusing since recursive is no longer a boolean.
2020-05-27 13:21:26 -05:00
Carlo Nucera
d49e65ba9d
Merge remote-tracking branch 'john-ericson/enum-FileIngestionMethod' into no-stringly-typed-derivation-output
2020-05-26 12:30:48 -04:00
Carlo Nucera
0f3f901071
Merge remote-tracking branch 'origin/master' into more-rust-ffi
2020-05-26 11:46:42 -04:00
Carlo Nucera
6d73c10041
Merge remote-tracking branch 'origin/master' into enum-FileIngestionMethod
2020-05-26 11:14:08 -04:00
Eelco Dolstra
5ef64f05e6
Cleanup
2020-05-18 15:50:29 +02:00
Eelco Dolstra
0ed946aa61
Merge branch 'wait-for-builders' of https://github.com/serokell/nix
2020-05-18 13:48:45 +02:00
Ben Burdette
92123c6c79
Merge remote-tracking branch 'upstream/master' into errors-phase-2
2020-05-15 07:00:36 -06:00
Alexander Bantyev
183dd28266
Don't lock a user while doing remote builds
2020-05-14 17:00:54 +03:00
Ben Burdette
ef9dd9f9bc
formatting and a few minor changes
2020-05-13 15:56:39 -06:00
Ben Burdette
c79d4addab
consistent capitalization
2020-05-13 10:02:18 -06:00
Ben Burdette
bfca5fc395
change status messages to info level
2020-05-13 09:52:36 -06:00
Ben Burdette
19cffc29c9
remove unused extra json fields
2020-05-12 12:09:12 -06:00
Ben Burdette
ec870b9c85
new pos format for more errors
2020-05-12 10:52:26 -06:00
Eelco Dolstra
ebc024df22
Show hint how to enable experimental features
2020-05-12 15:47:09 +02:00
Ben Burdette
b93c1bf3d6
fixes to merged code
2020-05-11 15:52:15 -06:00
Ben Burdette
59b1f5c701
Merge branch 'master' into errors-phase-2
2020-05-11 14:35:30 -06:00
Ben Burdette
958e81987b
switch from printError warnings to logWarnings
2020-05-11 13:02:16 -06:00
Domen Kožar
5bdb67c843
Merge pull request #3568 from kolloch/outputHashModeError
...
libstore/build.cc: more explicit error about form of output
2020-05-11 18:14:32 +02:00
Domen Kožar
1d8144e36b
Update src/libstore/build.cc
2020-05-11 18:14:23 +02:00
Benjamin Hipple
146f9c114f
doc: consistently refer to 'fixed-output' with a dash
...
General cleanup that makes it easier to search for the term.
2020-05-09 10:58:43 -04:00
Alexander Bantyev
772e5db828
Mention build users in the 'waiting for' message
2020-05-08 12:29:00 +03:00
Alexander Bantyev
14073fb76b
Don't block while waiting for build users
2020-05-08 12:22:39 +03:00
Ben Burdette
e3901638b5
todo removal
2020-05-06 15:01:13 -06:00
Eelco Dolstra
2f8ee4578f
Merge remote-tracking branch 'origin/master' into flakes
2020-05-06 12:01:40 +02:00
Eelco Dolstra
74a1bfdcab
Merge pull request #3546 from guibou/nix_readfile_on_0_sized_files
...
builtins.readFile: do not truncate content
2020-05-06 11:33:55 +02:00
Peter Kolloch
9be46859a9
libstore/build.cc: more explicit about form of output
...
Be more explicit about why we expect a regular file as output
when outputHashMode=flat for a fixed output derivation.
2020-05-06 11:21:12 +02:00
Eelco Dolstra
02c5914ea4
Merge pull request #3562 from pikajude/master
...
Use fragment size for autoGC capacity calculation
2020-05-06 11:19:30 +02:00
Eelco Dolstra
fd4911269f
Revert "Merge pull request #3558 from LnL7/ssh-ng-stderr"
...
This reverts commit 3ebfbecdd1
, reversing
changes made to c089c52d5f
.
https://github.com/NixOS/nix/pull/3558
2020-05-06 10:54:18 +02:00
Eelco Dolstra
6f3244ce45
Merge remote-tracking branch 'origin/master' into flakes
2020-05-05 18:59:33 +02:00
Alexander Bantyev
04967dee9d
Wait for build users when none are available
2020-05-05 13:04:36 +03:00
Ben Burdette
7ffb5efdbc
appending to hints; remove _printError
2020-05-04 16:19:57 -06:00
Jude Taylor
e2fc575c61
nix auto-gc: use fragment size
2020-05-04 14:42:06 -07:00
Eelco Dolstra
a721a0b114
Flag: Use designated initializers
2020-05-04 22:40:19 +02:00
Ben Burdette
ab6f0b9641
convert some printError calls to logError
2020-05-03 08:01:25 -06:00
Daiderd Jordan
f16e24f95e
remote-store: don't log raw stderr by default
...
For remote stores the log messages are already forwarded as structured
STDERR_RESULT messages so the old format is duplicate information. But
still included with -vvv since it could be useful for debugging
problems.
$ nix build -L /nix/store/nl71b2niws857ffiaggyrkjwgx9jjzc0-foo.drv --store ssh-ng://localhost
Hello World!
foo> Hello World!
[1/0/1 built] building foo
Fixes #3556
2020-05-02 23:40:50 +02:00
Ben Burdette
4b99c09f5c
convert some errors
2020-05-01 14:32:06 -06:00
Eelco Dolstra
941f95284a
Merge remote-tracking branch 'origin/master' into flakes
2020-05-01 11:59:56 +02:00
Ben Burdette
39ff80d031
errorinfo constructor test
2020-04-29 18:57:05 -06:00
Guillaume Bouchard
7afcb5af98
Remove the drain
argument from readFile
...
Now it is always `drain` (see previous commit).
2020-04-29 18:43:45 +02:00
Antoine Eiche
ca93b26db6
Only call grantpt on MacOS systems
...
The commit 3cc1125595
adds a `grantpt`
call on the builder pseudo terminal fd. This call is actually only
required for MacOS, but it however requires a RW access to /dev/pts
which is only RO bindmounted in the Bazel Linux sandbox. So, Nix can
not be actually run in the Bazel Linux sandbox for unneeded reasons.
2020-04-29 15:43:20 +02:00
Ben Burdette
22e6490311
Error classname as name
2020-04-28 21:06:08 -06:00
Eelco Dolstra
ef4d3fc111
Merge remote-tracking branch 'origin/master' into flakes
2020-04-23 15:16:18 +02:00
Dustin DeWeese
c0d940978a
Replace select() with poll() to allow waiting on more than FD_SETSIZE fds
2020-04-21 16:21:28 -07:00
Ben Burdette
e4fb9a3849
remove 'format' from Error constructor calls
2020-04-21 17:07:07 -06:00
Ben Burdette
15e9564fd1
logEI for tunnelLogger and progressbar
2020-04-19 17:16:51 -06:00
Ben Burdette
3d5b1032a1
logError, logWarning; Logger functions; switch to Verbosity enum
2020-04-17 15:07:44 -06:00
Eelco Dolstra
aaa109565e
Use a more space/time-efficient representation for the eval cache
2020-04-17 23:04:21 +02:00
Eelco Dolstra
7a9687ba30
SQLiteStmt: Use std::string_view
2020-04-17 01:00:56 +02:00
Eelco Dolstra
0858738355
Merge remote-tracking branch 'origin/master' into flakes
2020-04-16 18:27:37 +02:00
Eelco Dolstra
a118293bd0
Merge pull request #3458 from zimbatm/nix-user-conf-dir
...
NIX_USER_CONF_FILES
2020-04-15 13:00:28 +02:00
Jonas Chevalier
895516cadf
add NIX_USER_CONF_FILES
...
Motivation: maintain project-level configuration files.
Document the whole situation a bit better so that it corresponds to the
implementation, and add NIX_USER_CONF_FILES that allows overriding
which user files Nix will load during startup.
2020-04-14 18:45:06 +02:00
Eelco Dolstra
c0c2cb871d
Merge remote-tracking branch 'origin/master' into flakes
2020-04-14 13:02:55 +02:00
Bruce Toll
8132d0a12e
Fix nix-build --check -K in sandbox w/o root
...
Temporarily add user-write permission to build directory so that it
can be moved out of the sandbox to the store with a .check suffix.
This is necessary because the build directory has already had its
permissions set read-only, but write permission is required
to update the directory's parent link to move it out of the sandbox.
Updated the related --check "derivation may not be deterministic"
messages to consistently use the real store paths.
Added test for non-root sandbox nix-build --check -K to demonstrate
issue and help prevent regressions.
2020-04-10 16:23:10 -04:00
Domen Kožar
db25a6d7bb
Merge pull request #2689 from tollb/fix/delete_tmp_dir_when_build_check_ok
...
Delete temporary directory on successful build
2020-04-10 09:47:57 +02:00
Bruce Toll
16a4864759
Delete temporary directory on successful build
...
With --check and the --keep-failed (-K) flag, the temporary directory
was being retained regardless of whether the build was successful and
reproducible. This removes the temporary directory, as expected, on
a reproducible check build.
Added tests to verify that temporary build directories are not
retained unnecessarily, particularly when using --check with
--keep-failed.
2020-04-09 16:37:41 -04:00
Philipp Middendorf
04bedda0b6
gc.cc: Ignore hidden files in temproots
2020-04-09 17:05:29 +02:00
Eelco Dolstra
d103c79144
Merge remote-tracking branch 'origin/master' into flakes
2020-04-09 15:54:21 +02:00
Nikola Knezevic
f5095594e7
datatransfer.{cc,hh} -> filetransfer.{cc,hh}
2020-04-08 22:26:57 +02:00
Nikola Knezevic
c330109bfa
DataTransfer -> FileTransfer
2020-04-08 22:26:57 +02:00
Nikola Knezevic
7848372b0f
Add upload method
2020-04-08 22:26:57 +02:00
Nikola Knezevic
a0c5931208
actDownload -> actDataTransfer
2020-04-08 22:26:57 +02:00
Nikola Knezevic
c4c1ae0a00
DownloadError -> DataTransferError
2020-04-08 22:26:57 +02:00
Nikola Knezevic
213d124277
DownloadItem -> TransferItem
2020-04-08 22:26:57 +02:00
Nikola Knezevic
cd391206e6
{get,make,new}Downloader -> DataTransfer
2020-04-08 22:26:57 +02:00
Nikola Knezevic
2df2741ec6
enqueueDownload -> enqueueDataTransfer
2020-04-08 22:26:57 +02:00
Nikola Knezevic
142ed7fe45
DownloadResult -> DataTransferResult
2020-04-08 22:26:57 +02:00
Nikola Knezevic
e5cc53beec
DownloadSettings -> DataTransferSettings
2020-04-08 22:26:57 +02:00
Nikola Knezevic
741e9012d3
Rename src/lib/download.* to src/lib/datatransfer.*
2020-04-08 22:26:57 +02:00
Nikola Knezevic
65ef57e0cb
DownloadRequest -> DataTransferRequest
2020-04-08 22:26:57 +02:00
Domen Kožar
bf81b31559
build.cc: improve message if home directory exists
2020-04-08 18:27:10 +02:00
Eelco Dolstra
1ab8d6ac18
Downloader: Only write data to the sink on a 200 response
...
Hopefully fixes #3278 .
2020-04-08 15:27:09 +02:00
Eelco Dolstra
54955867a6
Merge remote-tracking branch 'origin/master' into flakes
2020-04-07 14:29:45 +02:00
Eelco Dolstra
cd39709003
Cleanup
2020-04-07 09:27:17 +02:00
Eelco Dolstra
462421d345
Backport libfetchers from the flakes branch
...
This provides a pluggable mechanism for defining new fetchers. It adds
a builtin function 'fetchTree' that generalizes existing fetchers like
'fetchGit', 'fetchMercurial' and 'fetchTarball'. 'fetchTree' takes a
set of attributes, e.g.
fetchTree {
type = "git";
url = "https://example.org/repo.git ";
ref = "some-branch";
rev = "abcdef...";
}
The existing fetchers are just wrappers around this. Note that the
input attributes to fetchTree are the same as flake input
specifications and flake lock file entries.
All fetchers share a common cache stored in
~/.cache/nix/fetcher-cache-v1.sqlite. This replaces the ad hoc caching
mechanisms in fetchGit and download.cc (e.g. ~/.cache/nix/{tarballs,git-revs*}).
This also adds support for Git worktrees (c169ea5904
).
2020-04-07 09:03:14 +02:00
Cole Helbling
c976cb0b8a
Don't retry on "unsupported protocol" error
...
When encountering an unsupported protocol, there's no need to retry.
Chances are, it won't suddenly be supported between retry attempts;
error instead. Otherwise, you see something like the following:
$ nix-env -i -f git://git@github.com/foo/bar
warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 335 ms
warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 604 ms
warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 1340 ms
warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 2685 ms
With this change, you now see:
$ nix-env -i -f git://git@github.com/foo/bar
error: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1)
2020-04-05 09:00:34 -07:00
Eelco Dolstra
4fdec5f61d
Merge remote-tracking branch 'origin/master' into flakes
2020-04-01 00:03:50 +02:00
mlatus
12556e5709
fix placeholder not substituted in passAsFile
2020-03-31 19:40:16 +08:00
John Ericson
51afea3af2
Never cast FileIngestionMethod
to or from boolean
2020-03-30 22:31:51 +00:00
John Ericson
c251b011cd
Merge remote-tracking branch 'upstream/master' into enum-FileIngestionMethod
2020-03-30 18:16:44 -04:00
John Ericson
832bd534dc
Store parsed hashes in DerivationOutput
...
It's best to detect invalid data as soon as possible, with data types
that make storing it impossible.
2020-03-30 11:33:35 -04:00
John Ericson
f5494d9442
Merge remote-tracking branch 'me/enum-FileIngestionMethod' into HEAD
2020-03-30 11:08:13 -04:00
Eelco Dolstra
d15d91cad1
Makefile cleanup
2020-03-30 16:50:13 +02:00
Eelco Dolstra
e322a16523
Remove global -I flags
...
(cherry picked from commit 2c692a3b14
)
2020-03-30 15:30:19 +02:00
Eelco Dolstra
2c692a3b14
Remove global -I flags
2020-03-30 14:39:33 +02:00
Eelco Dolstra
e0a0ae0467
Move fetchers from libstore to libfetchers
2020-03-30 14:04:53 +02:00
Eelco Dolstra
2287cc6486
Fix segfault
2020-03-30 13:31:55 +02:00
John Ericson
225e62a56a
Replace some bool recursive
with a new FileIngestionMethod
enum
2020-03-29 15:16:20 -04:00
John Ericson
87b32bab05
Use enum struct
and drop prefixes
...
This does a few enums; the rest will be gotten in subsequent commits.
2020-03-29 11:23:15 -04:00
Eelco Dolstra
2287e2f279
nix flake info: Show flake subdirectory
2020-03-28 18:05:50 +01:00
John Ericson
e433d4af4c
Extend Rust FFI
...
Do idiomatic C++ copy and move constructors for a few things, so
wrapping structs' defaults can work.
2020-03-25 16:12:14 -04:00
John Ericson
bcde5456cc
Flip dependency so store-api.hh includes derivations.hh
...
I think it makes more sense to define the data model (derivations),
before the operations (store api).
2020-03-24 20:39:45 +00:00
Eelco Dolstra
0a10854f85
Misc changes from the flakes branch
2020-03-24 14:34:47 +01:00
Eelco Dolstra
777e21e596
nix path-info --json: Print hash in SRI format
...
(cherry picked from commit 442e665d6d
)
2020-03-24 14:26:23 +01:00
Eelco Dolstra
f9611c7ae4
buildenv: Eliminate global variables, other cleanup
...
(cherry picked from commit b82f75464d
)
2020-03-24 14:06:47 +01:00
Eelco Dolstra
edc34cc1a2
Add function for quoting strings
...
(cherry picked from commit 7dcf5b011a
)
2020-03-24 13:44:04 +01:00
Eelco Dolstra
5a7e7fc35f
Use std::string_view
...
(cherry picked from commit 6529490cc1
)
2020-03-24 13:26:37 +01:00
Eelco Dolstra
7abe3bde8a
Merge remote-tracking branch 'origin/master' into flakes
2020-03-23 13:27:54 +01:00
Eelco Dolstra
5885e20404
Merge pull request #3429 from LnL7/darwin-sandbox
...
darwin sandbox
2020-03-23 09:13:36 +01:00
Daiderd Jordan
2e9bc1245c
sandbox: fix /bin/sh on catalina
...
Sadly 10.15 changed /bin/sh to a shim which executes bash, this means it
can't be used anymore without also opening up the sandbox to allow bash.
Failed to exec /bin/bash as variant for /bin/sh (1: Operation not permitted).
2020-03-20 22:12:30 +01:00
Daiderd Jordan
f6c122aaeb
sandbox: allow pty devices
...
Nix now runs builds with a pseudo-terminal to enable colored build
output.
2020-03-20 21:58:45 +01:00
Daiderd Jordan
7f2df903d9
libstore: relax default sandbox-paths on darwin
2020-03-20 21:31:20 +01:00
Daiderd Jordan
afb78ebd34
libstore: disable resolve-system-dependencies hook
...
This is used to determine the dependency tree of impure libraries so nix
knows what paths to open in the sandbox. With the less restrictive
defaults it isn't needed anymore.
2020-03-20 21:21:56 +01:00
John Ericson
e317324236
Apply suggestions from code review
2020-03-19 23:38:51 -04:00
John Ericson
d5b3328dd1
Apply suggestions from code review
...
Co-Authored-By: Cole Helbling <cole.e.helbling@outlook.com>
2020-03-19 23:37:52 -04:00
John Ericson
f1cf3ab870
hashDerivationModulo: Generalize for multiple fixed ouputs per drv
...
See documentattion in header and comments in implementation for details.
This is actually done in preparation for floating ca derivations, not
multi-output fixed ca derivations, but the distinction doesn't yet
mattter.
Thanks @cole-h for finding and fixing a bunch of typos.
2020-03-19 10:30:49 -04:00
Eelco Dolstra
8aa354fdfd
Register flake-registry.json as a GC root again
2020-03-19 11:45:34 +01:00
Eelco Dolstra
1c127e6a82
downloadFile(): Use expired file if the download fails
2020-03-19 11:42:50 +01:00
John Ericson
049179ba07
Fix typos
...
Thanks @asymmetric
I failed to do them all in one batch
Co-Authored-By: asymmetric <lorenzo@mailbox.org>
2020-03-18 19:07:05 -04:00
Eelco Dolstra
f6ddf48882
Get rid of downloadCached()
...
Everything uses the generic caching system now.
2020-03-18 17:24:19 +01:00
Eelco Dolstra
c5ec95e2c7
tarball.cc: Use ETags
2020-03-18 15:14:23 +01:00
Eelco Dolstra
1e7ce1d6da
tarball / github fetchers: Use generic caching system
2020-03-18 14:08:25 +01:00
Eelco Dolstra
38e360154d
Git: Use unified caching system
2020-03-17 22:35:29 +01:00
Eelco Dolstra
d1165d8791
Require shallow clones to be requested explicitly
...
If you do a fetchTree on a Git repository, whether the result contains
a revCount attribute should not depend on whether that repository
happens to be a shallow clone or not. That would complicate caching a
lot and would be semantically messy. So applying fetchTree/fetchGit to
a shallow repository is now an error unless you pass the attribute
'shallow = true'. If 'shallow = true', we don't return revCount, even
if the repository is not actually shallow.
Note that Nix itself is not doing shallow clones at the moment. But it
could do so as an optimisation if the user specifies 'shallow = true'.
Issue #2988 .
2020-03-17 22:35:29 +01:00
Eelco Dolstra
2a4e4f6a6e
Unified fetcher caching system
2020-03-17 22:35:29 +01:00
John Ericson
e5178fd22d
Fix typos
...
Thanks @asymmetric!
Co-Authored-By: asymmetric <lorenzo@mailbox.org>
2020-03-16 16:40:13 -04:00
Eelco Dolstra
fbcb897e21
Add a test for shallow Git clones
...
Also, don't return a revCount anymore for shallow or dirty Git trees,
since it's incorrect.
Closes #2988 .
2020-03-16 13:20:32 +01:00
John Ericson
2be64efb02
Generalize isFixedOutput
in preparation for CA drvs
...
Today's fixed output derivations and regular derivations differ in a few
ways which are largely orthogonal. This replaces `isFixedOutput` with a
`type` that returns an enum of possible combinations.
2020-03-15 11:05:37 -04:00
Eelco Dolstra
0c2088d438
Merge remote-tracking branch 'origin/master' into flakes
2020-03-13 17:03:30 +01:00
Eelco Dolstra
b816515f61
Fix ca-references feature check
...
Fixes #3406 .
2020-03-13 13:15:51 +01:00
Robert Hensing
3f55f8a8fb
pathInfoCache: Respect disk cache TTLs #3398
2020-03-12 10:30:28 +01:00
Eelco Dolstra
e188fe7c6d
Move call-flake.nix into libexpr
2020-03-11 17:04:51 +01:00
Eelco Dolstra
73b6d87e17
Merge remote-tracking branch 'origin/master' into flakes
2020-03-04 13:58:42 +01:00
Eelco Dolstra
d37dc71e3c
nix-build: Fix !<output> handling
...
This was broken by 22a754c091
.
https://hydra.nixos.org/eval/1573669
2020-03-04 13:56:17 +01:00
Eelco Dolstra
75db069f92
Optimise Derivation::unparse()
...
In
nix-instantiate --dry-run '<nixpkgs/nixos/release-combined.nix>' -A nixos.tests.simple.x86_64-linux
this reduces time spent in unparse() from 9.15% to 4.31%. The main
culprit was appending characters one at a time to the destination
string. Even though the string has enough capacity, push_back() still
needs to check this on every call.
2020-03-04 11:44:45 +01:00
Eelco Dolstra
22a754c091
Fix GC failures on bad store path names
...
It failed on names like '/nix/store/9ip48nkc9rfy0a4yaw98lp6gipqlib1a-'.
2020-02-28 18:07:10 +01:00
Eelco Dolstra
4a4521f462
Fix nlohmann::json exception
2020-02-20 23:47:02 +01:00
Eelco Dolstra
73c9840569
Restore subdir -> dir
...
Got this mixed up somewhere.
2020-02-20 23:44:06 +01:00
Eelco Dolstra
d068f9ffff
Restore subdir support in registries
...
Hacky...
2020-02-20 22:14:44 +01:00