puck/lix
0
0
Fork 0
forked from lix-project/lix
Code Pull requests Activity
lix/tests/functional/fetchurl.sh

87 lines
3.1 KiB
Bash
Raw Normal View History

Add a test for the fetchurl function
2012-07-09 19:41:43 +00:00
source common.sh
clearStore
<nix/fetchurl.nix>: Support xz-compressed NARs
2015-10-30 11:33:40 +00:00
# Test fetching a flat file.
<nix/fetchurl.nix>: Support downloading and unpacking NARs This removes the need to have multiple downloads in the stdenv bootstrap process (like a separate busybox binary for Linux, or curl/mkdir/sh/bzip2 for Darwin). Now all those files can be combined into a single NAR.
2015-10-30 10:27:47 +00:00
hash=$(nix-hash --flat --type sha256 ./fetchurl.sh)
Add a test for the fetchurl function
2012-07-09 19:41:43 +00:00
tests/fetchurl.sh: Be verbose to help debug darwin eof bug #2794
2021-08-31 11:17:01 +00:00
outPath=$(nix-build -vvvvv --expr 'import <nix/fetchurl.nix>' --argstr url file://$(pwd)/fetchurl.sh --argstr sha256 $hash --no-out-link)
Add a test for the fetchurl function
2012-07-09 19:41:43 +00:00
<nix/fetchurl.nix>: Support downloading and unpacking NARs This removes the need to have multiple downloads in the stdenv bootstrap process (like a separate busybox binary for Linux, or curl/mkdir/sh/bzip2 for Darwin). Now all those files can be combined into a single NAR.
2015-10-30 10:27:47 +00:00
cmp $outPath fetchurl.sh
fetchTree: Do not re-fetch paths already present
2022-02-27 14:25:22 +00:00
# Do not re-fetch paths already present.
outPath2=$(nix-build -vvvvv --expr 'import <nix/fetchurl.nix>' --argstr url file:///does-not-exist/must-remain-unused/fetchurl.sh --argstr sha256 $hash --no-out-link)
test "$outPath" == "$outPath2"
Support base-64 hashes Also simplify the Hash API. Fixes #1437.
2017-07-04 12:47:59 +00:00
# Now using a base-64 hash.
clearStore
Move 'nix hash-*' and 'nix to-*' to 'nix hash' From the 'nix' UX review.
2020-07-24 16:44:43 +00:00
hash=$(nix hash file --type sha512 --base64 ./fetchurl.sh)
Support base-64 hashes Also simplify the Hash API. Fixes #1437.
2017-07-04 12:47:59 +00:00
tests/fetchurl.sh: Be verbose to help debug darwin eof bug #2794
2021-08-31 11:17:01 +00:00
outPath=$(nix-build -vvvvv --expr 'import <nix/fetchurl.nix>' --argstr url file://$(pwd)/fetchurl.sh --argstr sha512 $hash --no-out-link)
Support base-64 hashes Also simplify the Hash API. Fixes #1437.
2017-07-04 12:47:59 +00:00
cmp $outPath fetchurl.sh
Support SRI hashes SRI hashes (https://www.w3.org/TR/SRI/) combine the hash algorithm and a base-64 hash. This allows more concise and standard hash specifications. For example, instead of import <nix/fetchurl.nl> { url = https://nixos.org/releases/nix/nix-2.1.3/nix-2.1.3.tar.xz; sha256 = "5d22dad058d5c800d65a115f919da22938c50dd6ba98c5e3a183172d149840a4"; }; you can write import <nix/fetchurl.nl> { url = https://nixos.org/releases/nix/nix-2.1.3/nix-2.1.3.tar.xz; hash = "sha256-XSLa0FjVyADWWhFfkZ2iKTjFDda6mMXjoYMXLRSYQKQ="; }; In fixed-output derivations, the outputHashAlgo is no longer mandatory if outputHash specifies the hash (either as an SRI or in the old "<type>:<hash>" format). 'nix hash-{file,path}' now print hashes in SRI format by default. I also reverted them to use SHA-256 by default because that's what we're using most of the time in Nixpkgs. Suggested by @zimbatm.
2018-12-13 13:30:52 +00:00
# Now using an SRI hash.
clearStore
Move 'nix hash-*' and 'nix to-*' to 'nix hash' From the 'nix' UX review.
2020-07-24 16:44:43 +00:00
hash=$(nix hash file ./fetchurl.sh)
Support SRI hashes SRI hashes (https://www.w3.org/TR/SRI/) combine the hash algorithm and a base-64 hash. This allows more concise and standard hash specifications. For example, instead of import <nix/fetchurl.nl> { url = https://nixos.org/releases/nix/nix-2.1.3/nix-2.1.3.tar.xz; sha256 = "5d22dad058d5c800d65a115f919da22938c50dd6ba98c5e3a183172d149840a4"; }; you can write import <nix/fetchurl.nl> { url = https://nixos.org/releases/nix/nix-2.1.3/nix-2.1.3.tar.xz; hash = "sha256-XSLa0FjVyADWWhFfkZ2iKTjFDda6mMXjoYMXLRSYQKQ="; }; In fixed-output derivations, the outputHashAlgo is no longer mandatory if outputHash specifies the hash (either as an SRI or in the old "<type>:<hash>" format). 'nix hash-{file,path}' now print hashes in SRI format by default. I also reverted them to use SHA-256 by default because that's what we're using most of the time in Nixpkgs. Suggested by @zimbatm.
2018-12-13 13:30:52 +00:00
tests/fetchurl: fix after changing default hash from 512 to 256
2018-12-15 04:37:20 +00:00
[[ $hash =~ ^sha256- ]]
Support SRI hashes SRI hashes (https://www.w3.org/TR/SRI/) combine the hash algorithm and a base-64 hash. This allows more concise and standard hash specifications. For example, instead of import <nix/fetchurl.nl> { url = https://nixos.org/releases/nix/nix-2.1.3/nix-2.1.3.tar.xz; sha256 = "5d22dad058d5c800d65a115f919da22938c50dd6ba98c5e3a183172d149840a4"; }; you can write import <nix/fetchurl.nl> { url = https://nixos.org/releases/nix/nix-2.1.3/nix-2.1.3.tar.xz; hash = "sha256-XSLa0FjVyADWWhFfkZ2iKTjFDda6mMXjoYMXLRSYQKQ="; }; In fixed-output derivations, the outputHashAlgo is no longer mandatory if outputHash specifies the hash (either as an SRI or in the old "<type>:<hash>" format). 'nix hash-{file,path}' now print hashes in SRI format by default. I also reverted them to use SHA-256 by default because that's what we're using most of the time in Nixpkgs. Suggested by @zimbatm.
2018-12-13 13:30:52 +00:00
tests/fetchurl.sh: Be verbose to help debug darwin eof bug #2794
2021-08-31 11:17:01 +00:00
outPath=$(nix-build -vvvvv --expr 'import <nix/fetchurl.nix>' --argstr url file://$(pwd)/fetchurl.sh --argstr hash $hash --no-out-link)
Support SRI hashes SRI hashes (https://www.w3.org/TR/SRI/) combine the hash algorithm and a base-64 hash. This allows more concise and standard hash specifications. For example, instead of import <nix/fetchurl.nl> { url = https://nixos.org/releases/nix/nix-2.1.3/nix-2.1.3.tar.xz; sha256 = "5d22dad058d5c800d65a115f919da22938c50dd6ba98c5e3a183172d149840a4"; }; you can write import <nix/fetchurl.nl> { url = https://nixos.org/releases/nix/nix-2.1.3/nix-2.1.3.tar.xz; hash = "sha256-XSLa0FjVyADWWhFfkZ2iKTjFDda6mMXjoYMXLRSYQKQ="; }; In fixed-output derivations, the outputHashAlgo is no longer mandatory if outputHash specifies the hash (either as an SRI or in the old "<type>:<hash>" format). 'nix hash-{file,path}' now print hashes in SRI format by default. I also reverted them to use SHA-256 by default because that's what we're using most of the time in Nixpkgs. Suggested by @zimbatm.
2018-12-13 13:30:52 +00:00
cmp $outPath fetchurl.sh
Replace --hashed-mirrors with substituters test
2020-06-12 19:39:44 +00:00
# Test that we can substitute from a different store dir.
Always use base-16 for hashed mirror lookups In particular, don't use base-64, which we don't support. (We do have base-32 redirects for hysterical reasons.) Also, add a test for the hashed mirror feature.
2017-07-17 11:13:18 +00:00
clearStore
Correctly substitute from different storeDir Originally, the test was only checking for different “real” storeDir. That’s an easy case to handle, but the much harder one is if different virtual store dirs are used. To do this, we need the SubstitutionGoal to know about the ca, so it can recalculate the path to copy it over. An important note here is that the store path passed to copyStorePath needs to be one for srcStore - so that queryPathInfo works properly. This also adds an error message when the store path from queryPathInfo is different from the one we requested.
2020-06-13 05:07:42 +00:00
other_store=file://$TEST_ROOT/other_store?store=/fnord/store
Always use base-16 for hashed mirror lookups In particular, don't use base-64, which we don't support. (We do have base-32 redirects for hysterical reasons.) Also, add a test for the hashed mirror feature.
2017-07-17 11:13:18 +00:00
Move 'nix hash-*' and 'nix to-*' to 'nix hash' From the 'nix' UX review.
2020-07-24 16:44:43 +00:00
hash=$(nix hash file --type sha256 --base16 ./fetchurl.sh)
Always use base-16 for hashed mirror lookups In particular, don't use base-64, which we don't support. (We do have base-32 redirects for hysterical reasons.) Also, add a test for the hashed mirror feature.
2017-07-17 11:13:18 +00:00
Split 'nix store add-to-store' into 'add-path' and 'add-file' This makes it consistent with 'nix hash <path|file>'.
2020-12-03 23:58:09 +00:00
storePath=$(nix --store $other_store store add-file ./fetchurl.sh)
Debug when storePath changes these rewrites should be transparent, but they are important to know about when debugging
2020-06-12 20:32:52 +00:00
tests/fetchurl.sh: Be verbose to help debug darwin eof bug #2794
2021-08-31 11:17:01 +00:00
outPath=$(nix-build -vvvvv --expr 'import <nix/fetchurl.nix>' --argstr url file:///no-such-dir/fetchurl.sh --argstr sha256 $hash --no-out-link --substituters $other_store)
Always use base-16 for hashed mirror lookups In particular, don't use base-64, which we don't support. (We do have base-32 redirects for hysterical reasons.) Also, add a test for the hashed mirror feature.
2017-07-17 11:13:18 +00:00
Fix segfault in builtin fetchurl with hashed mirrors + SRI hashes
2019-06-01 13:27:43 +00:00
# Test hashed mirrors with an SRI hash.
tests/fetchurl.sh: Be verbose to help debug darwin eof bug #2794
2021-08-31 11:17:01 +00:00
nix-build -vvvvv --expr 'import <nix/fetchurl.nix>' --argstr url file:///no-such-dir/fetchurl.sh --argstr hash $(nix hash to-sri --type sha256 $hash) \
Fix SRI test We can’t use custom name here because different names will have different store paths. This is a limitation of the Store API’s reliance on store paths. We might be able to get around the above in the future by using a dummy name for certain fixed output paths.
2020-06-12 20:33:24 +00:00
--no-out-link --substituters $other_store
Fix segfault in builtin fetchurl with hashed mirrors + SRI hashes
2019-06-01 13:27:43 +00:00
<nix/fetchurl.nix>: Support xz-compressed NARs
2015-10-30 11:33:40 +00:00
# Test unpacking a NAR.
<nix/fetchurl.nix>: Support downloading and unpacking NARs This removes the need to have multiple downloads in the stdenv bootstrap process (like a separate busybox binary for Linux, or curl/mkdir/sh/bzip2 for Darwin). Now all those files can be combined into a single NAR.
2015-10-30 10:27:47 +00:00
rm -rf $TEST_ROOT/archive
mkdir -p $TEST_ROOT/archive
cp ./fetchurl.sh $TEST_ROOT/archive
chmod +x $TEST_ROOT/archive/fetchurl.sh
ln -s foo $TEST_ROOT/archive/symlink
nar=$TEST_ROOT/archive.nar
nix-store --dump $TEST_ROOT/archive > $nar
hash=$(nix-hash --flat --type sha256 $nar)
tests/fetchurl.sh: Be verbose to help debug darwin eof bug #2794
2021-08-31 11:17:01 +00:00
outPath=$(nix-build -vvvvv --expr 'import <nix/fetchurl.nix>' --argstr url file://$nar --argstr sha256 $hash \
<nix/fetchurl.nix>: Support downloading and unpacking NARs This removes the need to have multiple downloads in the stdenv bootstrap process (like a separate busybox binary for Linux, or curl/mkdir/sh/bzip2 for Darwin). Now all those files can be combined into a single NAR.
2015-10-30 10:27:47 +00:00
--arg unpack true --argstr name xyzzy --no-out-link)
Harden tests' bash Use `set -u` and `set -o pipefail` to catch accidental mistakes and failures more strongly. - `set -u` catches the use of undefined variables - `set -o pipefail` catches failures (like `set -e`) earlier in the pipeline. This makes the tests a bit more robust. It is nice to read code not worrying about these spurious success paths (via uncaught) errors undermining the tests. Indeed, I caught some bugs doing this. There are a few tests where we run a command that should fail, and then search its output to make sure the failure message is one that we expect. Before, since the `grep` was the last command in the pipeline the exit code of those failing programs was silently ignored. Now with `set -o pipefail` it won't be, and we have to do something so the expected failure doesn't accidentally fail the test. To do that we use `expect` and a new `expectStderr` to check for the exact failing exit code. See the comments on each for why. `grep -q` is replaced with `grepQuiet`, see the comments on that function for why. `grep -v` when we just want the exit code is replaced with `grepInverse, see the comments on that function for why. `grep -q -v` together is, surprise surprise, replaced with `grepQuietInverse`, which is both combined. Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2021-12-09 15:26:46 +00:00
echo $outPath | grepQuiet 'xyzzy'
<nix/fetchurl.nix>: Support downloading and unpacking NARs This removes the need to have multiple downloads in the stdenv bootstrap process (like a separate busybox binary for Linux, or curl/mkdir/sh/bzip2 for Darwin). Now all those files can be combined into a single NAR.
2015-10-30 10:27:47 +00:00
test -x $outPath/fetchurl.sh
test -L $outPath/symlink
<nix/fetchurl.nix>: Support xz-compressed NARs
2015-10-30 11:33:40 +00:00
nix-store --delete $outPath
# Test unpacking a compressed NAR.
narxz=$TEST_ROOT/archive.nar.xz
rm -f $narxz
xz --keep $nar
tests/fetchurl.sh: Be verbose to help debug darwin eof bug #2794
2021-08-31 11:17:01 +00:00
outPath=$(nix-build -vvvvv --expr 'import <nix/fetchurl.nix>' --argstr url file://$narxz --argstr sha256 $hash \
<nix/fetchurl.nix>: Support xz-compressed NARs
2015-10-30 11:33:40 +00:00
--arg unpack true --argstr name xyzzy --no-out-link)
test -x $outPath/fetchurl.sh
test -L $outPath/symlink
builtin:fetchurl: Ensure a fixed-output derivation Previously we didn't check that the derivation was fixed-output, so you could use builtin:fetchurl to impurely fetch a file.
2024-02-01 20:46:01 +00:00
# Make sure that *not* passing a outputHash fails.
Better test fix
2024-02-02 12:22:18 +00:00
requireDaemonNewerThan "2.20"
Fix test
2024-02-02 12:07:08 +00:00
expected=100
if [[ -v NIX_DAEMON_PACKAGE ]]; then expected=1; fi # work around the daemon not returning a 100 status correctly
expectStderr $expected nix-build --expr '{ url }: builtins.derivation { name = "nix-cache-info"; system = "x86_64-linux"; builder = "builtin:fetchurl"; inherit url; outputHashMode = "flat"; }' --argstr url file://$narxz 2>&1 | grep 'must be a fixed-output derivation'
Copy permalink