Commit graph

36 commits

Author SHA1 Message Date
John Ericson 30dcc19d1f Put functional tests in tests/functional
I think it is bad for these reasons when `tests/` contains a mix of
functional and integration tests

 - Concepts is harder to understand, the documentation makes a good
   unit vs functional vs integration distinction, but when the
   integration tests are just two subdirs within `tests/` this is not
   clear.

 - Source filtering in the `flake.nix` is more complex. We need to
   filter out some of the dirs from `tests/`, rather than simply pick
   the dirs we want and take all of them. This is a good sign the
   structure of what we are trying to do is not matching the structure
   of the files.

With this change we have a clean:
```shell-session
$ git show 'HEAD:tests'
tree HEAD:tests

functional/
installer/
nixos/
```

(cherry picked from commit 68c81c737571794f7246db53fb4774e94fcf4b7e)
2023-12-01 12:06:43 -05:00
John Ericson e072e18475 Fix race condition in the language tests
When we pipe to `>(...)` like that, we unfortunately don't wait for the
process to finish. Better to just substitute the file.

Also, use the "unified" diff output that people (including myself) are
more familiar with, thanks to Git.
2023-07-13 08:09:03 -04:00
Mathnerd314 c70484454f Expanded test suite
* Lang now verifies errors and parse output

* Some new miscellaneous tests

* Easy way to update the tests

* Document workflow in manual

* Use `!` not `~` as separater char for sed

  It is confusing to use `~` when we are talking about paths and home
  directories!

* Test test suite itself (`test/lang-test/infra.sh`)

Additionally, run shellcheck on `tests/lang.sh` to help ensure it is
correct, now that is is more complex.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-07-11 21:43:09 -04:00
Robert Hensing 1c55544a42 eval: Fix crash on missing printValue tBlackhole case
Fixes #8119
2023-04-03 15:32:42 +02:00
John Ericson c11836126b Harden tests' bash
Use `set -u` and `set -o pipefail` to catch accidental mistakes and
failures more strongly.

 - `set -u` catches the use of undefined variables
 - `set -o pipefail` catches failures (like `set -e`) earlier in the
   pipeline.

This makes the tests a bit more robust. It is nice to read code not
worrying about these spurious success paths (via uncaught) errors
undermining the tests. Indeed, I caught some bugs doing this.

There are a few tests where we run a command that should fail, and then
search its output to make sure the failure message is one that we
expect. Before, since the `grep` was the last command in the pipeline
the exit code of those failing programs was silently ignored. Now with
`set -o pipefail` it won't be, and we have to do something so the
expected failure doesn't accidentally fail the test.

To do that we use `expect` and a new `expectStderr` to check for the
exact failing exit code. See the comments on each for why.

`grep -q` is replaced with `grepQuiet`, see the comments on that
function for why.

`grep -v` when we just want the exit code is replaced with `grepInverse,
see the comments on that function for why.

`grep -q -v` together is, surprise surprise, replaced with
`grepQuietInverse`, which is both combined.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-03-08 10:26:30 -05:00
Eelco Dolstra e86530ee46 Fix reference to test directory path 2022-12-12 16:55:42 +01:00
Eelco Dolstra 8e8a511aa0 Enable some language tests that were accidentally disabled
This didn't run because the corresponding .exp file didn't exist.
2022-12-12 16:39:06 +01:00
Gytis Ivaskevicius ba1fe85b65 Add builtins.traceVerbose
Co-Authored-By: Silvan Mosberger <contact@infinisil.com>

Add builtins.traceVerbose tests
2022-07-05 19:44:26 +03:00
Guillaume Maudoux 275f8561eb tests/lang: Distinguish crashes from expected failures 2022-05-02 15:12:50 +02:00
Guillaume Maudoux 4845886bed Revert "tests: Distinguish crashes from expected failures"
This reverts commit 143b73f52d.
2022-05-02 14:22:00 +02:00
Guillaume Maudoux 143b73f52d tests: Distinguish crashes from expected failures 2022-04-29 11:14:08 +02:00
Eelco Dolstra 6a67e57019
Add DummyStore (dummy://)
DummyStore does not allow building or adding paths. This is useful for
evaluation tests when you don't want to initialize a "proper" store.
2020-08-24 18:54:16 +02:00
Eelco Dolstra 3f80060500 Fix tests
So all these years I was totally deluded about the meaning of "set
-e". You might think that it causes statements like "false && true" or
"! true" to fail, but it doesn't...
2014-08-21 21:50:19 +02:00
Eelco Dolstra a8edf185a9 Add constant ‘nixPath’
It contains the Nix expression search path as a list of { prefix, path
} sets, e.g.

  [ { path = "/nix/var/nix/profiles/per-user/root/channels/nixos"; prefix = ""; }
    { path = "/etc/nixos/configuration.nix"; prefix = "nixos-config"; }
    { path = "/home/eelco/Dev/nix/inst/share/nix/corepkgs"; prefix = "nix"; }
  ]
2014-05-26 14:55:47 +02:00
Eelco Dolstra 7880973827 Test trace and addErrorContext 2014-02-26 19:12:31 +01:00
Eelco Dolstra 5ad263c26b Test some more primops 2014-02-26 19:08:44 +01:00
Eelco Dolstra e1cf40fa95 nix-instantiate: Rename --eval-only to --eval, --parse-only to --parse 2014-02-19 16:34:24 +01:00
Eelco Dolstra 7df4ef983e Test the delayed with a bit more 2013-07-31 13:12:35 +02:00
Eelco Dolstra 48cea0d01e * Refactoring: Get rid of a few subdirectories in corepkgs/, and some
other simplifications.
* Use <nix/...> to locate the corepkgs.  This allows them to be
  overriden through $NIX_PATH.
* Use bash's pipefail option in the NAR builder so that we don't need
  to create a temporary file.
2012-01-03 00:16:29 +00:00
Eelco Dolstra 8af7d766f0 * Refactoring: remove unnecessary variables from the tests. 2011-10-10 21:32:34 +00:00
Eelco Dolstra 1ecc97b6bd * Add a Nix expression search path feature. Paths between angle
brackets, e.g.

    import <nixpkgs/pkgs/lib>

  are resolved by looking them up relative to the elements listed in
  the search path.  This allows us to get rid of hacks like

    import "${builtins.getEnv "NIXPKGS_ALL"}/pkgs/lib"

  The search path can be specified through the ‘-I’ command-line flag
  and through the colon-separated ‘NIX_PATH’ environment variable,
  e.g.,

    $ nix-build -I /etc/nixos ...

  If a file is not found in the search path, an error message is
  lazily thrown.
2011-08-06 16:05:24 +00:00
Eelco Dolstra 83dfa89870 * Sync with the trunk. 2010-05-07 14:46:47 +00:00
Eelco Dolstra 7148df7971 * Update the expected test output (no longer an ATerm). 2010-04-21 16:22:03 +00:00
Eelco Dolstra cae4efdca3 * Because --parse-only no longer produces an ATerm, don't check the
output.  Whether it parses at all should be enough.
2010-04-21 16:02:12 +00:00
Eelco Dolstra f3b8833a48 * Drop the dependency on the ATerm library. 2010-04-19 14:51:58 +00:00
Ludovic Courtès 09381cccff Make source location info in the XML output optional.
* src/libexpr/expr-to-xml.cc (nix::showAttrs): Add `location'
  parameter.  Provide location XML attributes when it's true.  Update
  callers.
  (nix::printTermAsXML): Likewise.

* src/libexpr/expr-to-xml.hh (nix::printTermAsXML): Update prototype;
  have `location' default to `false'.

* src/nix-instantiate/nix-instantiate.cc (printResult, processExpr): Add
  `location' parameter; update callers.
  (run): Add support for `--no-location'.

* src/nix-instantiate/help.txt: Update accordingly.

* tests/lang.sh: Invoke `nix-instantiate' with `--no-location' for the
  XML tests.

* tests/lang/eval-okay-toxml.exp, tests/lang/eval-okay-to-xml.nix: New
  files.
2010-03-31 12:38:31 +00:00
Eelco Dolstra e418976107 * Option --argstr for passing string arguments easily. (NIX-75) 2007-01-14 12:32:44 +00:00
Eelco Dolstra 9e30694f98 * Fix the tests wrt the AST changes, i.e., Str(s) -> Str(s, []), and
the semantic changes.
2006-10-17 11:08:59 +00:00
Eelco Dolstra e47e0c2dbe * Builtin function `getEnv' for getting environment variables. 2006-09-24 17:48:41 +00:00
Eelco Dolstra 2ab4bc44c7 * Builtin function `add' to add integers.
* Put common test functions in tests/lang/lib.nix.
2006-09-22 15:29:21 +00:00
Eelco Dolstra 93d9797eda * Urgh (see NIX-56). 2006-08-22 09:34:38 +00:00
Eelco Dolstra 4874fd2d9a * Test for `nix-instantiate --eval-only --xml'. 2006-08-17 11:28:29 +00:00
Eelco Dolstra 84c4631221 * Simplification. 2006-03-01 12:51:18 +00:00
Eelco Dolstra a4c63c6e8e * Make it easy to run individual tests from the command line. 2006-03-01 12:15:33 +00:00
Eelco Dolstra ed09821859 * Use atdiff' instead of cmp' for checking test output.
* Don't use local file names in tests since they will produce
  different parse trees depending on the current directory.
2004-10-27 13:12:58 +00:00
Eelco Dolstra 463e2817c5 * Remove ancient Fix tests.
* Add automated Nix expression language tests.
2004-10-27 12:41:53 +00:00