Commit graph

10 commits

Author SHA1 Message Date
John Ericson 30dcc19d1f Put functional tests in tests/functional
I think it is bad for these reasons when `tests/` contains a mix of
functional and integration tests

 - Concepts is harder to understand, the documentation makes a good
   unit vs functional vs integration distinction, but when the
   integration tests are just two subdirs within `tests/` this is not
   clear.

 - Source filtering in the `flake.nix` is more complex. We need to
   filter out some of the dirs from `tests/`, rather than simply pick
   the dirs we want and take all of them. This is a good sign the
   structure of what we are trying to do is not matching the structure
   of the files.

With this change we have a clean:
```shell-session
$ git show 'HEAD:tests'
tree HEAD:tests

functional/
installer/
nixos/
```

(cherry picked from commit 68c81c737571794f7246db53fb4774e94fcf4b7e)
2023-12-01 12:06:43 -05:00
John Ericson c11836126b Harden tests' bash
Use `set -u` and `set -o pipefail` to catch accidental mistakes and
failures more strongly.

 - `set -u` catches the use of undefined variables
 - `set -o pipefail` catches failures (like `set -e`) earlier in the
   pipeline.

This makes the tests a bit more robust. It is nice to read code not
worrying about these spurious success paths (via uncaught) errors
undermining the tests. Indeed, I caught some bugs doing this.

There are a few tests where we run a command that should fail, and then
search its output to make sure the failure message is one that we
expect. Before, since the `grep` was the last command in the pipeline
the exit code of those failing programs was silently ignored. Now with
`set -o pipefail` it won't be, and we have to do something so the
expected failure doesn't accidentally fail the test.

To do that we use `expect` and a new `expectStderr` to check for the
exact failing exit code. See the comments on each for why.

`grep -q` is replaced with `grepQuiet`, see the comments on that
function for why.

`grep -v` when we just want the exit code is replaced with `grepInverse,
see the comments on that function for why.

`grep -q -v` together is, surprise surprise, replaced with
`grepQuietInverse`, which is both combined.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-03-08 10:26:30 -05:00
Andrea Ciceri f58759816d Tighten up the exportReferencesGraph tests
Add an `$` at the end of the `grep` regex. Without it, `checkRef foo`
would always imply `checkRef foo.drv`. We want to tell these situations
apart to more precisely test what is going on.
2023-01-23 20:41:32 -05:00
Eelco Dolstra 5722f9690c tests/binary-cache.sh: Improve incomplete closure test
Issue #3373.
2020-05-12 13:56:00 +02:00
Eelco Dolstra 90b5e69284 Support quoted attribute names in -A
This is requires if you have attribute names with dots in them.  So
you can now say:

  $ nix-instantiate '<nixos>' -A 'config.systemd.units."postgresql.service".text' --eval-only

Fixes #151.
2013-11-18 10:21:12 +00:00
Eelco Dolstra d4c3b6327f Don't put results symlinks in the tests directory 2012-09-11 19:14:15 -04:00
Eelco Dolstra 8af7d766f0 * Refactoring: remove unnecessary variables from the tests. 2011-10-10 21:32:34 +00:00
Eelco Dolstra e530e0a350 * Improve the test. 2009-03-18 16:36:13 +00:00
Eelco Dolstra 93b6926054 * Regression test for exportBuildReferencesGraph. It currently fails. 2009-03-17 17:38:32 +00:00
Eelco Dolstra 2d5114452d * Regression test for the `exportReferencesGraph'
derivation attribute.
2009-03-17 16:33:48 +00:00