emilytrau/lix
1
0
Fork 0
forked from lix-project/lix
Code Pull requests Activity
lix/tests
History
John Ericson c11836126b Harden tests' bash 
Use `set -u` and `set -o pipefail` to catch accidental mistakes and
failures more strongly.

 - `set -u` catches the use of undefined variables
 - `set -o pipefail` catches failures (like `set -e`) earlier in the
   pipeline.

This makes the tests a bit more robust. It is nice to read code not
worrying about these spurious success paths (via uncaught) errors
undermining the tests. Indeed, I caught some bugs doing this.

There are a few tests where we run a command that should fail, and then
search its output to make sure the failure message is one that we
expect. Before, since the `grep` was the last command in the pipeline
the exit code of those failing programs was silently ignored. Now with
`set -o pipefail` it won't be, and we have to do something so the
expected failure doesn't accidentally fail the test.

To do that we use `expect` and a new `expectStderr` to check for the
exact failing exit code. See the comments on each for why.

`grep -q` is replaced with `grepQuiet`, see the comments on that
function for why.

`grep -v` when we just want the exit code is replaced with `grepInverse,
see the comments on that function for why.

`grep -q -v` together is, surprise surprise, replaced with
`grepQuietInverse`, which is both combined.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-03-08 10:26:30 -05:00
..
ca Harden tests' bash 2023-03-08 10:26:30 -05:00
common Harden tests' bash 2023-03-08 10:26:30 -05:00
config add NIX_USER_CONF_FILES 2020-04-14 18:45:06 +02:00
flakes Harden tests' bash 2023-03-08 10:26:30 -05:00
installer Nix's own flake: Dedup and memoize more 2023-02-20 11:35:51 +01:00
lang Merge pull request #7158 from sternenseemann/foldl-strict-accumulation-value 2023-02-19 23:54:14 +01:00
nixos Make /etc writability conditional on uid-range feature 2023-02-14 13:55:41 +01:00
plugins Parse string context elements properly 2023-01-10 13:10:49 -05:00
add.sh tests/check.sh: Fix a race 2021-07-07 10:48:47 +02:00
bad.tar.xz Validate tarball components 2019-12-13 19:05:26 +01:00
bash-profile.sh tests: Rename nix-profile.sh -> bash-profile.sh 2022-03-02 11:21:00 +01:00
big-derivation-attr.nix Fix segfault or stack overflow caused by large derivation fields 2022-01-19 15:21:56 +01:00
binary-cache-build-remote.sh Improve documentation and test and requested 2021-01-22 15:58:58 +00:00
binary-cache.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
brotli.sh Move 'nix hash-*' and 'nix to-*' to 'nix hash' 2020-12-03 17:55:55 +01:00
build-delete.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
build-dry.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
build-hook-ca-fixed.nix tests: grep for string in nix log for remote-builds 2022-03-04 16:58:27 +01:00
build-hook-ca-floating.nix Fix nix log with CA derivations 2022-12-08 16:03:20 -05:00
build-hook.nix Fix nix log with CA derivations 2022-12-08 16:03:20 -05:00
build-remote-content-addressed-fixed.sh Restore now-working build-remote-content-addressed-fixed test 2021-02-26 16:32:52 +00:00
build-remote-content-addressed-floating.sh tests: remove 'ca-references' feature 2022-04-21 10:06:39 +02:00
build-remote-input-addressed.sh Test the post-build-hook with remote builders 2020-12-09 10:45:12 +01:00
build-remote.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
build.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
ca-shell.nix nix-shell: restore backwards-compat with old nixpkgs 2022-06-13 23:29:28 +02:00
case-hack.sh
case.nar
check-refs.nix move unsafeDiscardReferences out of outputChecks 2023-01-03 18:53:01 +01:00
check-refs.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
check-reqs.nix
check-reqs.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
check.nix tests: don't refer to TMPDIR 2022-12-06 16:36:42 +01:00
check.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
common.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
completions.sh SourceExprCommand: swallow EvalError, add tests for this 2022-11-03 10:11:28 +01:00
compression-levels.sh Add compression level for NARs 2021-10-12 02:14:36 -04:00
compute-levels.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
config.nix.in Make the CA tests actuall test CA derivations 2021-06-23 17:36:50 +02:00
config.sh tests/config: test retrieving a single setting's value with nix show-config <setting> 2023-01-13 07:57:55 -08:00
db-migration.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
dependencies.builder0.sh Make nix why-depends quieter by default 2022-01-19 14:24:14 +01:00
dependencies.nix Add a simple test for nix why-depends 2022-01-18 16:54:53 +01:00
dependencies.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
describe-stores.sh Add a test for nix describe-stores 2020-09-16 13:53:28 +02:00
dummy Add tests/dummy 2021-07-07 11:30:35 +02:00
dump-db.sh Allow running all the tests with the daemon 2021-07-27 17:06:11 +02:00
eval-store.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
eval.nix add tests for nix eval and nix-instantiate 2022-03-16 21:01:51 +02:00
eval.sh Trivial changes from the lazy-trees branch 2022-12-07 14:06:34 +01:00
export-graph.nix
export-graph.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
export.sh
failing.nix nix-store --serve: pass on settings.keepFailed from SSH store 2021-08-31 13:11:46 +02:00
fetchClosure.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
fetchGit.sh tests/fetchGit: add regression test for #7707 2023-02-27 15:33:54 +01:00
fetchGitRefs.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
fetchGitSubmodules.sh Add some tests 2023-02-07 22:46:25 +01:00
fetchMercurial.sh Test fetchMercurial with path containing a . segment 2022-04-09 23:39:00 +02:00
fetchPath.sh Don’t create a file in the worktree in the fetchPath test 2022-03-30 11:42:47 +02:00
fetchTree-file.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
fetchurl.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
filter-source.nix
filter-source.sh Add test for builtins.path 2020-07-30 16:38:04 -05:00
fixed.builder1.sh
fixed.builder2.sh
fixed.nix build: also throw hash-mismatch errors if buildMode == bmCheck 2021-09-27 15:44:39 +02:00
fixed.sh build: also throw hash-mismatch errors if buildMode == bmCheck 2021-09-27 15:44:39 +02:00
fmt.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
fmt.simple.sh nix-fmt: add command 2022-03-11 10:00:19 -05:00
function-trace.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
gc-auto.sh Allow running all the tests with the daemon 2021-07-27 17:06:11 +02:00
gc-concurrent.builder.sh Make the gc-concurrent test more reliable 2020-07-02 16:13:36 +02:00
gc-concurrent.nix Make the gc-concurrent test more reliable 2020-07-02 16:13:36 +02:00
gc-concurrent.sh Make the gc-concurrent test more reliable 2020-07-02 16:13:36 +02:00
gc-concurrent2.builder.sh Make the gc-concurrent test more reliable 2020-07-02 16:13:36 +02:00
gc-non-blocking.sh gc-non-blocking.sh: Don't hang indefinitely if the GC roots server crashes 2021-12-13 19:01:38 +01:00
gc-runtime.nix
gc-runtime.sh tests/*: show when tests are skipped 2021-03-31 08:30:12 +02:00
gc.sh Fix nix-collect-garbage -d with the new profile location 2023-03-07 13:08:04 +01:00
hash-check.nix
hash.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
import-derivation.nix Allow builtins.pathExists to check the existence of /nix/store paths 2019-07-30 11:27:35 +02:00
import-derivation.sh
impure-derivations.nix Provide default values for outputHashAlgo and outputHashMode 2022-03-31 16:56:44 +02:00
impure-derivations.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
init.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
install-darwin.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
lang.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
linux-sandbox.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
local-store.sh Fix disabled case in local store test 2021-08-06 12:36:03 -04:00
local.mk Harden tests' bash 2023-03-08 10:26:30 -05:00
logging.sh Implement regression test for empty logs loaded via nix log 2022-03-24 22:31:52 +01:00
misc.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
multiple-outputs.nix Add some tests for illegal output names 2023-01-18 14:21:17 +01:00
multiple-outputs.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
nar-access.nix
nar-access.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
nix-build-examples.nix tests: Test #4197 nix-build output order regression 2020-11-13 17:50:04 +01:00
nix-build.sh tests: Test #4197 nix-build output order regression 2020-11-13 17:50:04 +01:00
nix-channel.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
nix-copy-ssh.sh Fix test failures when $TMPDIR changes 2019-06-15 16:36:33 +02:00
nix-profile.sh fixup! tests: nix-profile: test install error message upon conflicting files 2023-02-28 10:53:42 +01:00
nix-shell.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
nix_path.sh Revert "getDefaultNixPath: actually respect {restrict,pure}-eval" 2023-02-27 15:11:36 +01:00
optimise-store.sh Allow running all the tests with the daemon 2021-07-27 17:06:11 +02:00
output-normalization.sh Test nix build --json return output paths in floating CA case 2023-02-10 18:04:13 -05:00
parallel.builder.sh
parallel.nix
parallel.sh
pass-as-file.sh passAsFile: leave out the hash prefix 2020-01-02 23:56:06 +00:00
path-from-hash-part.sh Add command 'nix store path-from-hash-part' 2022-10-18 16:51:12 +02:00
path.nix Add test for builtins.path 2020-07-30 16:38:04 -05:00
placeholders.sh Fix test failures when $TMPDIR changes 2019-06-15 16:36:33 +02:00
plugins.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
post-hook.sh Get rid of .drv special-casing for store path installables 2023-02-28 17:07:05 -05:00
pure-eval.nix
pure-eval.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
push-to-store-old.sh Get rid of .drv special-casing for store path installables 2023-02-28 17:07:05 -05:00
push-to-store.sh Get rid of .drv special-casing for store path installables 2023-02-28 17:07:05 -05:00
readfile-context.nix Test that the result of readFile gets ref-scanned 2022-11-14 16:13:26 +01:00
readfile-context.sh Add test for readFile keeping context 2022-01-09 13:42:36 +03:00
recursive.sh tests: Fix the recursive test with the daemon 2021-09-13 14:29:42 +02:00
referrers.sh Revert "Fix referrers test" 2021-10-15 16:58:21 +02:00
remote-store.sh Move the default profiles to the user’s home 2023-01-17 14:17:28 +01:00
repair.sh Make --repair-path also repair corrupt optimised links 2022-01-11 11:57:45 +01:00
repl.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
restricted.nix
restricted.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
search.nix
search.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
secure-drv-outputs.nix
secure-drv-outputs.sh
selfref-gc.sh Disable the selfref-gc test when the daemon is too old 2022-04-21 10:06:39 +02:00
shell-hello.nix Allow selecting derivation outputs using 'installable!outputs' 2022-05-03 13:43:52 +02:00
shell.nix Make nix-shell work when nixpkgs is content-addressed 2021-11-26 09:56:48 +01:00
shell.sh Allow selecting derivation outputs using 'installable!outputs' 2022-05-03 13:43:52 +02:00
shell.shebang.rb
shell.shebang.sh
signing.sh Dodge "trusted" vs "trustworthy" by being explicit 2022-09-22 14:37:52 -04:00
simple-failing.nix Inline file 2021-12-07 21:45:09 +01:00
simple.builder.sh
simple.nix
simple.sh Fix segfault or stack overflow caused by large derivation fields 2022-01-19 15:21:56 +01:00
ssh-relay.sh Move most store-related commands to 'nix store' 2020-12-03 23:22:22 +01:00
store-ping.sh tests/store-ping: test nix store ping --json 2023-01-31 15:10:54 +01:00
structured-attrs-shell.nix Add testcase for nix develop with __structuredAttrs 2021-06-22 19:15:57 +02:00
structured-attrs.nix Prefix env vars for attrs.* files with NIX_ 2021-06-22 19:45:08 +02:00
structured-attrs.sh Fix the min bound for the structured-attrs test 2021-10-14 16:00:59 +02:00
substitute-with-invalid-ca.sh Check the CA hash when importing stuff in the local store 2021-06-01 15:09:24 +02:00
suggestions.sh Also display some suggestions for invalid formal arguments 2022-03-08 16:40:22 +01:00
tarball.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
test-infra.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
timeout.nix
timeout.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
toString-path.sh Trivial changes from the lazy-trees branch 2022-12-07 14:06:34 +01:00
undefined-variable.nix Add a test that nix repl --show-trace actually shows the trace 2021-12-28 15:54:46 +03:00
user-envs-migration.sh Test the migration of the user profiles 2023-01-17 14:17:28 +01:00
user-envs.builder.sh
user-envs.nix Don't hide repeated values while generating manifest.nix 2022-03-22 13:18:56 +01:00
user-envs.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
why-depends.sh Harden tests' bash 2023-03-08 10:26:30 -05:00
zstd.sh add tests for zstd compression 2021-04-09 23:13:08 +02:00