Commit graph

2050 commits

Author SHA1 Message Date
Matthew Bauer c214cda940 Correctly substitute from different storeDir
Originally, the test was only checking for different “real” storeDir.
That’s an easy case to handle, but the much harder one is if different
virtual store dirs are used. To do this, we need the SubstitutionGoal
to know about the ca, so it can recalculate the path to copy it over.
An important note here is that the store path passed to copyStorePath
needs to be one for srcStore - so that queryPathInfo works properly.

This also adds an error message when the store path from queryPathInfo
is different from the one we requested.
2020-06-13 00:07:42 -05:00
Matthew Bauer b2cb288cdd Add makeFixedOutputPathFromCA function
This puts what we are already doing into a shared method. It just
needs a path name and a ca and produces a store path.
2020-06-12 16:36:35 -05:00
Matthew Bauer 88120442d2 Debug when storePath changes
these rewrites should be transparent, but they are important to know
about when debugging
2020-06-12 15:32:52 -05:00
Matthew Bauer 5b386b05f5 Recompute storePath based on isContentAddressed 2020-06-12 14:57:45 -05:00
Matthew Bauer 85d01e1f65 Don’t use makeStorePath 2020-06-12 14:53:30 -05:00
Matthew Bauer 9077b9e6b2 Separate dstStore path from srcStore path 2020-06-12 14:27:28 -05:00
Matthew Bauer 2f2ac850b5 Compute new store path correctly 2020-06-12 14:03:17 -05:00
Matthew Bauer 11c97070f3 Fix storeDir != storeDir condition
this needs to only continue if the path replacement fails.
2020-06-12 10:14:03 -05:00
Matthew Bauer 3e3eaa90dd Remove hashed-mirrors 2020-06-12 09:51:44 -05:00
Matthew Bauer 79c169d1c6 Allow substituting from different storeDir
Substituters can substitute from one store dir to another with a
little bit of help. The store api just needs to have a CA so it can
recompute the store path based on the new store dir. We can only do
this for fixed output derivations with no references, though.
2020-06-12 09:49:09 -05:00
Matthew Bauer 2299ef705c Add error message when FileIngestionMethod is out of bounds
bool coerces anything >0 to true, but in the future we may have other
file ingestion methods. This shows a better error message when the
“recursive” byte isn’t 1.
2020-06-04 11:32:39 -05:00
John Ericson fac0c2d54a Remove addToStore variant as requested by FIXME
The idea is it's always more flexible to consumer a `Source` than a
plain string, and it might even reduce memory consumption.

I also looked at `addToStoreFromDump` with its `// FIXME: remove?`, but
the worked needed for that is far more up for interpretation, so I
punted for now.
2020-05-29 17:02:32 -04:00
Matthew Bauer c66441a646 Rename some variables named “recursive” to “method”
This is much less confusing since recursive is no longer a boolean.
2020-05-27 13:21:26 -05:00
Carlo Nucera 6d73c10041 Merge remote-tracking branch 'origin/master' into enum-FileIngestionMethod 2020-05-26 11:14:08 -04:00
Eelco Dolstra 5ef64f05e6 Cleanup 2020-05-18 15:50:29 +02:00
Eelco Dolstra 0ed946aa61 Merge branch 'wait-for-builders' of https://github.com/serokell/nix 2020-05-18 13:48:45 +02:00
Alexander Bantyev 183dd28266
Don't lock a user while doing remote builds 2020-05-14 17:00:54 +03:00
Eelco Dolstra ebc024df22
Show hint how to enable experimental features 2020-05-12 15:47:09 +02:00
Domen Kožar 5bdb67c843
Merge pull request #3568 from kolloch/outputHashModeError
libstore/build.cc: more explicit error about form of output
2020-05-11 18:14:32 +02:00
Domen Kožar 1d8144e36b
Update src/libstore/build.cc 2020-05-11 18:14:23 +02:00
Benjamin Hipple 146f9c114f doc: consistently refer to 'fixed-output' with a dash
General cleanup that makes it easier to search for the term.
2020-05-09 10:58:43 -04:00
Alexander Bantyev 772e5db828
Mention build users in the 'waiting for' message 2020-05-08 12:29:00 +03:00
Alexander Bantyev 14073fb76b
Don't block while waiting for build users 2020-05-08 12:22:39 +03:00
Eelco Dolstra 74a1bfdcab
Merge pull request #3546 from guibou/nix_readfile_on_0_sized_files
builtins.readFile: do not truncate content
2020-05-06 11:33:55 +02:00
Peter Kolloch 9be46859a9 libstore/build.cc: more explicit about form of output
Be more explicit about why we expect a regular file as output
when outputHashMode=flat for a fixed output derivation.
2020-05-06 11:21:12 +02:00
Eelco Dolstra 02c5914ea4
Merge pull request #3562 from pikajude/master
Use fragment size for autoGC capacity calculation
2020-05-06 11:19:30 +02:00
Eelco Dolstra fd4911269f
Revert "Merge pull request #3558 from LnL7/ssh-ng-stderr"
This reverts commit 3ebfbecdd1, reversing
changes made to c089c52d5f.

https://github.com/NixOS/nix/pull/3558
2020-05-06 10:54:18 +02:00
Alexander Bantyev 04967dee9d
Wait for build users when none are available 2020-05-05 13:04:36 +03:00
Jude Taylor e2fc575c61 nix auto-gc: use fragment size 2020-05-04 14:42:06 -07:00
Eelco Dolstra a721a0b114 Flag: Use designated initializers 2020-05-04 22:40:19 +02:00
Daiderd Jordan f16e24f95e
remote-store: don't log raw stderr by default
For remote stores the log messages are already forwarded as structured
STDERR_RESULT messages so the old format is duplicate information.  But
still included with -vvv since it could be useful for debugging
problems.

    $ nix build -L /nix/store/nl71b2niws857ffiaggyrkjwgx9jjzc0-foo.drv --store ssh-ng://localhost
    Hello World!
    foo> Hello World!
    [1/0/1 built] building foo

Fixes #3556
2020-05-02 23:40:50 +02:00
Guillaume Bouchard 7afcb5af98 Remove the drain argument from readFile
Now it is always `drain` (see previous commit).
2020-04-29 18:43:45 +02:00
Antoine Eiche ca93b26db6 Only call grantpt on MacOS systems
The commit 3cc1125595 adds a `grantpt`
call on the builder pseudo terminal fd. This call is actually only
required for MacOS, but it however requires a RW access to /dev/pts
which is only RO bindmounted in the Bazel Linux sandbox. So, Nix can
not be actually run in the Bazel Linux sandbox for unneeded reasons.
2020-04-29 15:43:20 +02:00
Dustin DeWeese c0d940978a Replace select() with poll() to allow waiting on more than FD_SETSIZE fds 2020-04-21 16:21:28 -07:00
Eelco Dolstra a118293bd0
Merge pull request #3458 from zimbatm/nix-user-conf-dir
NIX_USER_CONF_FILES
2020-04-15 13:00:28 +02:00
Jonas Chevalier 895516cadf
add NIX_USER_CONF_FILES
Motivation: maintain project-level configuration files.

Document the whole situation a bit better so that it corresponds to the
implementation, and add NIX_USER_CONF_FILES that allows overriding
which user files Nix will load during startup.
2020-04-14 18:45:06 +02:00
Bruce Toll 8132d0a12e Fix nix-build --check -K in sandbox w/o root
Temporarily add user-write permission to build directory so that it
can be moved out of the sandbox to the store with a .check suffix.

This is necessary because the build directory has already had its
permissions set read-only, but write permission is required
to update the directory's parent link to move it out of the sandbox.

Updated the related --check "derivation may not be deterministic"
messages to consistently use the real store paths.

Added test for non-root sandbox nix-build --check -K to demonstrate
issue and help prevent regressions.
2020-04-10 16:23:10 -04:00
Domen Kožar db25a6d7bb
Merge pull request #2689 from tollb/fix/delete_tmp_dir_when_build_check_ok
Delete temporary directory on successful build
2020-04-10 09:47:57 +02:00
Bruce Toll 16a4864759 Delete temporary directory on successful build
With --check and the --keep-failed (-K) flag, the temporary directory
was being retained regardless of whether the build was successful and
reproducible.  This removes the temporary directory, as expected, on
a reproducible check build.

Added tests to verify that temporary build directories are not
retained unnecessarily, particularly when using --check with
--keep-failed.
2020-04-09 16:37:41 -04:00
Philipp Middendorf 04bedda0b6 gc.cc: Ignore hidden files in temproots 2020-04-09 17:05:29 +02:00
Nikola Knezevic f5095594e7 datatransfer.{cc,hh} -> filetransfer.{cc,hh} 2020-04-08 22:26:57 +02:00
Nikola Knezevic c330109bfa DataTransfer -> FileTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic 7848372b0f Add upload method 2020-04-08 22:26:57 +02:00
Nikola Knezevic a0c5931208 actDownload -> actDataTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic c4c1ae0a00 DownloadError -> DataTransferError 2020-04-08 22:26:57 +02:00
Nikola Knezevic 213d124277 DownloadItem -> TransferItem 2020-04-08 22:26:57 +02:00
Nikola Knezevic cd391206e6 {get,make,new}Downloader -> DataTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic 2df2741ec6 enqueueDownload -> enqueueDataTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic 142ed7fe45 DownloadResult -> DataTransferResult 2020-04-08 22:26:57 +02:00
Nikola Knezevic e5cc53beec DownloadSettings -> DataTransferSettings 2020-04-08 22:26:57 +02:00