moraxyc/lix
0
0
Fork 0
forked from lix-project/lix
Code Pull requests Activity
9036 commits 1 branch 2 tags 68 MiB
Commit graph

848 commits

Author SHA1 Message Date
Rodrigo 3a642187c3 Fall back to copyPath if link fails with EPERM 
BeeGFS doesn't allow hard-links and returns EPERM, so we fall back
to copyPath. See https://github.com/NixOS/nix/issues/3748
 2020-06-25 12:03:26 +02:00
regnat d38f860c3e Add a way to get all the outputs of a derivation with their label 
Generalize `queryDerivationOutputNames` and `queryDerivationOutputs` by
adding a `queryDerivationOutputMap` that returns the map
`outputName=>outputPath`

(not that this is not equivalent to merging the results of
`queryDerivationOutputs` and `queryDerivationOutputNames` as sets don't
preserve the order, so we would end up with an incorrect mapping).

squash! Add a way to get all the outputs of a derivation with their label

Rename StorePathMap to OutputPathMap
 2020-06-24 20:38:40 +02:00
John Ericson d9fb9e9f26 Merge remote-tracking branch 'upstream/master' into ca-drv 2020-06-24 15:41:29 +00:00
John Ericson 98e5d1af03 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-06-23 17:03:37 +00:00
Matthew Bauer 66a62b3189 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-22 13:08:11 -04:00
John Ericson 93bbe6e8ab Merge remote-tracking branch 'upstream/master' into ca-drv 2020-06-21 21:11:58 +00:00
Carlo Nucera e7a14118df WIP bug fixing 2020-06-19 16:50:28 -04:00
John Ericson 20799a5151 WIP: Make Hash always store a valid hash type 2020-06-19 19:32:29 +00:00
Matthew Bauer 7e11cf3399 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-19 14:40:49 -04:00
Matthew Bauer 2796b01e86 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-19 14:22:06 -04:00
John Ericson 68294746ae Merge remote-tracking branch 'upstream/master' into no-hash-type-unknown 2020-06-19 17:53:34 +00:00
John Ericson 911fc88bcb More designated initializers 2020-06-19 17:42:56 +00:00
John Ericson 2f0e395c99 Merge remote-tracking branch 'me/no-stringly-typed-derivation-output' into validPathInfo-ca-proper-datatype 2020-06-19 15:26:59 +00:00
John Ericson 01dc8b0bab Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output 2020-06-19 14:59:05 +00:00
John Ericson 3f8dcfe3fd Merge branch 'validPathInfo-temp' into validPathInfo-ca-proper-datatype 2020-06-18 23:01:58 +00:00
John Ericson 669c3992e8 Merge branch 'no-hash-type-unknown' into validPathInfo-temp 2020-06-18 22:33:07 +00:00
John Ericson 15abb2aa2b Revert the enum struct change 
Not a regular git revert as there have been many merges and things.
 2020-06-18 22:11:26 +00:00
John Ericson bbbf3602a3 Merge branch 'enum-class' into no-hash-type-unknown 2020-06-18 22:11:19 +00:00
John Ericson 40526fbea5 Merge remote-tracking branch 'upstream/master' into enum-class 2020-06-18 21:38:15 +00:00
regnat 4fef2ba7e4 Rename content-addressed-paths into ca-derivations 
See <https://github.com/NixOS/nix/pull/3710#issuecomment-645480333>
 2020-06-18 09:25:55 +02:00
Matthew Bauer be50de1142 Make sure references are empty for store path replacing 
also copy info2 instead of casting
 2020-06-17 14:15:08 -04:00
Matthew Bauer 8974755d19 Add assert for replaced storePath 2020-06-17 14:04:46 -04:00
Matthew Bauer 7bd88cc1dc Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-17 13:32:29 -04:00
Matthew Bauer f767bedfac Replace struct StorePath with class StorePath 
also a similar case with struct Goal
 2020-06-17 13:26:37 -04:00
regnat 480b54e1c6 fixup! Reserve the __contentAddressed derivation parameter 2020-06-17 17:37:04 +02:00
regnat 56d75bf4fc Reserve the __contentAddressed derivation parameter 
Not implementing anything here, just throwing an error if a derivation
sets `__contentAddressed = true` without
`--experimental-features content-addressed-paths`
(and also with it as there's nothing implemented yet)
 2020-06-17 15:41:17 +02:00
John Ericson 517f5980e2 Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output 2020-06-17 04:58:43 +00:00
John Ericson 49308ef039 Merge remote-tracking branch 'upstream/master' into ca-drv 2020-06-17 03:39:39 +00:00
Eelco Dolstra 29542865ce Remove StorePath::clone() and related functions 2020-06-16 22:20:18 +02:00
Eelco Dolstra a588b6b19d Print only one error message if a build fails 
E.g. instead of

  error: --- BuildError ----------------------------------------------- nix
  builder for '/nix/store/03nk0a3n8h2948k4lqfgnnmym7knkcma-foo.drv' failed with exit code 1
  error: --- Error ---------------------------------------------------- nix
  build of '/nix/store/03nk0a3n8h2948k4lqfgnnmym7knkcma-foo.drv' failed

we now get

  error: --- Error ---------------------------------------------------- nix
  builder for '/nix/store/03nk0a3n8h2948k4lqfgnnmym7knkcma-foo.drv' failed with exit code 1
 2020-06-15 19:35:31 +02:00
Eelco Dolstra 31707735b6 Remove unnecessary amDone() overrides 2020-06-15 16:47:21 +02:00
Eelco Dolstra 5ed5d7acbd Improve "waiting for locks" messages 
These are now shown in the progress bar.

Closes #3577.
 2020-06-15 16:03:29 +02:00
Eelco Dolstra 7a77762961 Merge branch 'errors-phase-2' of https://github.com/bburdette/nix 2020-06-15 11:46:31 +02:00
Matthew Bauer c214cda940 Correctly substitute from different storeDir 
Originally, the test was only checking for different “real” storeDir.
That’s an easy case to handle, but the much harder one is if different
virtual store dirs are used. To do this, we need the SubstitutionGoal
to know about the ca, so it can recalculate the path to copy it over.
An important note here is that the store path passed to copyStorePath
needs to be one for srcStore - so that queryPathInfo works properly.

This also adds an error message when the store path from queryPathInfo
is different from the one we requested.
 2020-06-13 00:07:42 -05:00
Eelco Dolstra 045b07200c Remove Store::queryDerivationOutputNames() 
This function was used in only one place, where it could easily be
replaced by readDerivation() since it's not
performance-critical. (This function appears to have been modelled
after queryDerivationOutputs(), which exists only to make the garbage
collector faster.)
 2020-06-12 12:46:33 +02:00
Ben Burdette ef1b3f21b6 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-06-11 14:06:35 -06:00
Eelco Dolstra ac4d43a31b
Merge pull request #3073 from tweag/machine-logs 
Add an option to print the logs in a machine-readable format
 2020-06-11 15:45:18 +02:00
regnat 4983401440 Unify the printing of the logs between bar-with-logs and raw 
Make the printing of the build logs systematically go through the
logger, and replicate the behavior of `no-build-output` by having two
different loggers (one that prints the build logs and one that doesn't)
 2020-06-08 09:31:15 +02:00
John Ericson 53bc8ff152 No C++ designated initializers yet with Clang 7 2020-06-03 20:45:14 -04:00
John Ericson 3a9e4c3262 Don't anticipate CA but not fixed outputs for now 2020-06-03 18:50:45 +00:00
John Ericson 2500403059 Use enum and predicates rather than bitfile for derivation type 2020-06-03 17:38:54 +00:00
John Ericson 6b7f4ec4ab Merge remote-tracking branch 'upstream/master' into ca-drv 2020-06-03 16:36:06 +00:00
Jonas Chevalier 6ee03b8444
libutils/hash: remove default encoding 
This will make it easier to reason about the hash encoding and switch to
SRI everywhere where possible.
 2020-06-03 13:49:51 +02:00
John Ericson 3c78ac348c Merge remote-tracking branch 'obsidian/no-hash-type-unknown' into validPathInfo-ca-proper-datatype 2020-06-03 04:44:24 +00:00
John Ericson 450dcf2c1b Remove HashType::Unknown 
Instead, `Hash` uses `std::optional<HashType>`. In the future, we may
also make `Hash` itself require a known hash type, encoraging people to
use `std::optional<Hash>` instead.
 2020-06-02 15:52:13 +00:00
Ben Burdette d82d230b40 elide the 'ErrorInfo' in logError and logWarning calls 2020-06-02 08:22:24 -06:00
John Ericson efcd30da89 WIP 2020-06-02 00:37:43 +00:00
Carlo Nucera f4b89e11a4 Merge branch 'no-stringly-typed-derivation-output' of github.com:Ericson2314/nix into validPathInfo-ca-proper-datatype 2020-06-01 17:12:50 -04:00
Ben Burdette 734283d636 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-05-29 09:51:37 -06:00
Carlo Nucera 6dd471ebf6 Fixing the result of merge 2020-05-28 12:14:36 -04:00
Carlo Nucera 4f597fb901 Merge branch 'master' of github.com:NixOS/nix into enum-class 2020-05-28 10:58:22 -04:00
John Ericson 5b4cd84bc2 Merge remote-tracking branch 'me/more-rust-ffi' into no-stringly-typed-derivation-output 2020-05-28 10:35:53 -04:00
Matthew Bauer c66441a646 Rename some variables named “recursive” to “method” 
This is much less confusing since recursive is no longer a boolean.
 2020-05-27 13:21:26 -05:00
Carlo Nucera d49e65ba9d Merge remote-tracking branch 'john-ericson/enum-FileIngestionMethod' into no-stringly-typed-derivation-output 2020-05-26 12:30:48 -04:00
Carlo Nucera 6d73c10041 Merge remote-tracking branch 'origin/master' into enum-FileIngestionMethod 2020-05-26 11:14:08 -04:00
Eelco Dolstra 5ef64f05e6 Cleanup 2020-05-18 15:50:29 +02:00
Eelco Dolstra 0ed946aa61 Merge branch 'wait-for-builders' of https://github.com/serokell/nix 2020-05-18 13:48:45 +02:00
Alexander Bantyev 183dd28266
Don't lock a user while doing remote builds 2020-05-14 17:00:54 +03:00
Ben Burdette ef9dd9f9bc formatting and a few minor changes 2020-05-13 15:56:39 -06:00
Ben Burdette bfca5fc395 change status messages to info level 2020-05-13 09:52:36 -06:00
Ben Burdette b93c1bf3d6 fixes to merged code 2020-05-11 15:52:15 -06:00
Ben Burdette 59b1f5c701 Merge branch 'master' into errors-phase-2 2020-05-11 14:35:30 -06:00
Ben Burdette 958e81987b switch from printError warnings to logWarnings 2020-05-11 13:02:16 -06:00
Domen Kožar 5bdb67c843
Merge pull request #3568 from kolloch/outputHashModeError 
libstore/build.cc: more explicit error about form of output
 2020-05-11 18:14:32 +02:00
Domen Kožar 1d8144e36b
Update src/libstore/build.cc 2020-05-11 18:14:23 +02:00
Benjamin Hipple 146f9c114f doc: consistently refer to 'fixed-output' with a dash 
General cleanup that makes it easier to search for the term.
 2020-05-09 10:58:43 -04:00
Alexander Bantyev 772e5db828
Mention build users in the 'waiting for' message 2020-05-08 12:29:00 +03:00
Alexander Bantyev 14073fb76b
Don't block while waiting for build users 2020-05-08 12:22:39 +03:00
Peter Kolloch 9be46859a9 libstore/build.cc: more explicit about form of output 
Be more explicit about why we expect a regular file as output
when outputHashMode=flat for a fixed output derivation.
 2020-05-06 11:21:12 +02:00
Alexander Bantyev 04967dee9d
Wait for build users when none are available 2020-05-05 13:04:36 +03:00
Ben Burdette 7ffb5efdbc appending to hints; remove _printError 2020-05-04 16:19:57 -06:00
Ben Burdette ab6f0b9641 convert some printError calls to logError 2020-05-03 08:01:25 -06:00
Antoine Eiche ca93b26db6 Only call grantpt on MacOS systems 
The commit 3cc1125595 adds a `grantpt`
call on the builder pseudo terminal fd. This call is actually only
required for MacOS, but it however requires a RW access to /dev/pts
which is only RO bindmounted in the Bazel Linux sandbox. So, Nix can
not be actually run in the Bazel Linux sandbox for unneeded reasons.
 2020-04-29 15:43:20 +02:00
Dustin DeWeese c0d940978a Replace select() with poll() to allow waiting on more than FD_SETSIZE fds 2020-04-21 16:21:28 -07:00
Ben Burdette e4fb9a3849 remove 'format' from Error constructor calls 2020-04-21 17:07:07 -06:00
Bruce Toll 8132d0a12e Fix nix-build --check -K in sandbox w/o root 
Temporarily add user-write permission to build directory so that it
can be moved out of the sandbox to the store with a .check suffix.

This is necessary because the build directory has already had its
permissions set read-only, but write permission is required
to update the directory's parent link to move it out of the sandbox.

Updated the related --check "derivation may not be deterministic"
messages to consistently use the real store paths.

Added test for non-root sandbox nix-build --check -K to demonstrate
issue and help prevent regressions.
 2020-04-10 16:23:10 -04:00
Bruce Toll 16a4864759 Delete temporary directory on successful build 
With --check and the --keep-failed (-K) flag, the temporary directory
was being retained regardless of whether the build was successful and
reproducible.  This removes the temporary directory, as expected, on
a reproducible check build.

Added tests to verify that temporary build directories are not
retained unnecessarily, particularly when using --check with
--keep-failed.
 2020-04-09 16:37:41 -04:00
Nikola Knezevic f5095594e7 datatransfer.{cc,hh} -> filetransfer.{cc,hh} 2020-04-08 22:26:57 +02:00
Nikola Knezevic c330109bfa DataTransfer -> FileTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic a0c5931208 actDownload -> actDataTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic 741e9012d3 Rename src/lib/download.* to src/lib/datatransfer.* 2020-04-08 22:26:57 +02:00
Domen Kožar bf81b31559
build.cc: improve message if home directory exists 2020-04-08 18:27:10 +02:00
mlatus 12556e5709 fix placeholder not substituted in passAsFile 2020-03-31 19:40:16 +08:00
John Ericson 51afea3af2 Never cast FileIngestionMethod to or from boolean 2020-03-30 22:31:51 +00:00
John Ericson 832bd534dc Store parsed hashes in DerivationOutput 
It's best to detect invalid data as soon as possible, with data types
that make storing it impossible.
 2020-03-30 11:33:35 -04:00
John Ericson 225e62a56a Replace some bool recursive with a new FileIngestionMethod enum 2020-03-29 15:16:20 -04:00
John Ericson 87b32bab05 Use enum struct and drop prefixes 
This does a few enums; the rest will be gotten in subsequent commits.
 2020-03-29 11:23:15 -04:00
Eelco Dolstra 0a10854f85 Misc changes from the flakes branch 2020-03-24 14:34:47 +01:00
Eelco Dolstra f9611c7ae4 buildenv: Eliminate global variables, other cleanup 
(cherry picked from commit b82f75464d)
 2020-03-24 14:06:47 +01:00
Eelco Dolstra edc34cc1a2 Add function for quoting strings 
(cherry picked from commit 7dcf5b011a)
 2020-03-24 13:44:04 +01:00
John Ericson 049179ba07
Fix typos 
Thanks @asymmetric

I failed to do them all in one batch

Co-Authored-By: asymmetric <lorenzo@mailbox.org>
 2020-03-18 19:07:05 -04:00
John Ericson 2be64efb02 Generalize isFixedOutput in preparation for CA drvs 
Today's fixed output derivations and regular derivations differ in a few
ways which are largely orthogonal. This replaces `isFixedOutput` with a
`type` that returns an enum of possible combinations.
 2020-03-15 11:05:37 -04:00
Robin Gloster f8dbde0813
structured-attrs: chown .attrs.* files to builder 
Otherwise `chmod .`'ing the build directory doesn't work anymore, which
is done in nixpkgs if sourceRoot is set to '.'.
 2020-01-23 17:38:07 +01:00
Daiderd Jordan 8b3217f832
build: remove warning when in sandboxing test mode 
Introduced in 66fccd5832, but somehow
breaks the secure-drv-outputs test.
 2020-01-13 22:09:18 +01:00
Eelco Dolstra bc22a7ee6a Fix use of uninitialized store path 
Fixes 'building of '/nix/store/00000000000000000000000000000000-': ...'.
 2020-01-06 22:20:10 +01:00
Eelco Dolstra e2988f48a1
Merge pull request #3303 from LnL7/darwin-sandbox 
build: fix sandboxing on darwin
 2020-01-06 20:56:35 +01:00
Daiderd Jordan 66fccd5832
build: fix sandboxing on darwin 
Starting ba87b08f85 getEnv now returns an
std::optional which means these getEnv() != "" conditions no longer happen
if the variables are not defined.
 2020-01-05 20:23:52 +01:00
Daiderd Jordan 7d448bc966
build: fix path repairing when hash rewriting is required 
Handle store path repairing on darwin when sandboxing is enabled. Unlike
on linux sandboxing on darwin still requires hash rewriting.
 2020-01-04 20:25:25 +01:00
Daiderd Jordan b33fefcb92
build: recover store path when replacing fails 
This shouldn't happen in normal circumstances, but just in case
attempt to move the temporary path back if possible.
 2020-01-04 20:24:27 +01:00
edef c65a6fa86a passAsFile: leave out the hash prefix 
Having a colon in the path may cause issues, and having the hash
function indicated isn't actually necessary. We now verify the path 
format in the tests to prevent regressions.
 2020-01-02 23:56:06 +00:00