Commit graph

34 commits

Author SHA1 Message Date
Mathnerd314 c70484454f Expanded test suite
* Lang now verifies errors and parse output

* Some new miscellaneous tests

* Easy way to update the tests

* Document workflow in manual

* Use `!` not `~` as separater char for sed

  It is confusing to use `~` when we are talking about paths and home
  directories!

* Test test suite itself (`test/lang-test/infra.sh`)

Additionally, run shellcheck on `tests/lang.sh` to help ensure it is
correct, now that is is more complex.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-07-11 21:43:09 -04:00
Robert Hensing 1c55544a42 eval: Fix crash on missing printValue tBlackhole case
Fixes #8119
2023-04-03 15:32:42 +02:00
John Ericson c11836126b Harden tests' bash
Use `set -u` and `set -o pipefail` to catch accidental mistakes and
failures more strongly.

 - `set -u` catches the use of undefined variables
 - `set -o pipefail` catches failures (like `set -e`) earlier in the
   pipeline.

This makes the tests a bit more robust. It is nice to read code not
worrying about these spurious success paths (via uncaught) errors
undermining the tests. Indeed, I caught some bugs doing this.

There are a few tests where we run a command that should fail, and then
search its output to make sure the failure message is one that we
expect. Before, since the `grep` was the last command in the pipeline
the exit code of those failing programs was silently ignored. Now with
`set -o pipefail` it won't be, and we have to do something so the
expected failure doesn't accidentally fail the test.

To do that we use `expect` and a new `expectStderr` to check for the
exact failing exit code. See the comments on each for why.

`grep -q` is replaced with `grepQuiet`, see the comments on that
function for why.

`grep -v` when we just want the exit code is replaced with `grepInverse,
see the comments on that function for why.

`grep -q -v` together is, surprise surprise, replaced with
`grepQuietInverse`, which is both combined.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-03-08 10:26:30 -05:00
Eelco Dolstra e86530ee46 Fix reference to test directory path 2022-12-12 16:55:42 +01:00
Eelco Dolstra 8e8a511aa0 Enable some language tests that were accidentally disabled
This didn't run because the corresponding .exp file didn't exist.
2022-12-12 16:39:06 +01:00
Gytis Ivaskevicius ba1fe85b65 Add builtins.traceVerbose
Co-Authored-By: Silvan Mosberger <contact@infinisil.com>

Add builtins.traceVerbose tests
2022-07-05 19:44:26 +03:00
Guillaume Maudoux 275f8561eb tests/lang: Distinguish crashes from expected failures 2022-05-02 15:12:50 +02:00
Guillaume Maudoux 4845886bed Revert "tests: Distinguish crashes from expected failures"
This reverts commit 143b73f52d.
2022-05-02 14:22:00 +02:00
Guillaume Maudoux 143b73f52d tests: Distinguish crashes from expected failures 2022-04-29 11:14:08 +02:00
Eelco Dolstra 6a67e57019
Add DummyStore (dummy://)
DummyStore does not allow building or adding paths. This is useful for
evaluation tests when you don't want to initialize a "proper" store.
2020-08-24 18:54:16 +02:00
Eelco Dolstra 3f80060500 Fix tests
So all these years I was totally deluded about the meaning of "set
-e". You might think that it causes statements like "false && true" or
"! true" to fail, but it doesn't...
2014-08-21 21:50:19 +02:00
Eelco Dolstra a8edf185a9 Add constant ‘nixPath’
It contains the Nix expression search path as a list of { prefix, path
} sets, e.g.

  [ { path = "/nix/var/nix/profiles/per-user/root/channels/nixos"; prefix = ""; }
    { path = "/etc/nixos/configuration.nix"; prefix = "nixos-config"; }
    { path = "/home/eelco/Dev/nix/inst/share/nix/corepkgs"; prefix = "nix"; }
  ]
2014-05-26 14:55:47 +02:00
Eelco Dolstra 7880973827 Test trace and addErrorContext 2014-02-26 19:12:31 +01:00
Eelco Dolstra 5ad263c26b Test some more primops 2014-02-26 19:08:44 +01:00
Eelco Dolstra e1cf40fa95 nix-instantiate: Rename --eval-only to --eval, --parse-only to --parse 2014-02-19 16:34:24 +01:00
Eelco Dolstra 7df4ef983e Test the delayed with a bit more 2013-07-31 13:12:35 +02:00
Eelco Dolstra 48cea0d01e * Refactoring: Get rid of a few subdirectories in corepkgs/, and some
other simplifications.
* Use <nix/...> to locate the corepkgs.  This allows them to be
  overriden through $NIX_PATH.
* Use bash's pipefail option in the NAR builder so that we don't need
  to create a temporary file.
2012-01-03 00:16:29 +00:00
Eelco Dolstra 8af7d766f0 * Refactoring: remove unnecessary variables from the tests. 2011-10-10 21:32:34 +00:00
Eelco Dolstra 1ecc97b6bd * Add a Nix expression search path feature. Paths between angle
brackets, e.g.

    import <nixpkgs/pkgs/lib>

  are resolved by looking them up relative to the elements listed in
  the search path.  This allows us to get rid of hacks like

    import "${builtins.getEnv "NIXPKGS_ALL"}/pkgs/lib"

  The search path can be specified through the ‘-I’ command-line flag
  and through the colon-separated ‘NIX_PATH’ environment variable,
  e.g.,

    $ nix-build -I /etc/nixos ...

  If a file is not found in the search path, an error message is
  lazily thrown.
2011-08-06 16:05:24 +00:00
Eelco Dolstra 83dfa89870 * Sync with the trunk. 2010-05-07 14:46:47 +00:00
Eelco Dolstra 7148df7971 * Update the expected test output (no longer an ATerm). 2010-04-21 16:22:03 +00:00
Eelco Dolstra cae4efdca3 * Because --parse-only no longer produces an ATerm, don't check the
output.  Whether it parses at all should be enough.
2010-04-21 16:02:12 +00:00
Eelco Dolstra f3b8833a48 * Drop the dependency on the ATerm library. 2010-04-19 14:51:58 +00:00
Ludovic Courtès 09381cccff Make source location info in the XML output optional.
* src/libexpr/expr-to-xml.cc (nix::showAttrs): Add `location'
  parameter.  Provide location XML attributes when it's true.  Update
  callers.
  (nix::printTermAsXML): Likewise.

* src/libexpr/expr-to-xml.hh (nix::printTermAsXML): Update prototype;
  have `location' default to `false'.

* src/nix-instantiate/nix-instantiate.cc (printResult, processExpr): Add
  `location' parameter; update callers.
  (run): Add support for `--no-location'.

* src/nix-instantiate/help.txt: Update accordingly.

* tests/lang.sh: Invoke `nix-instantiate' with `--no-location' for the
  XML tests.

* tests/lang/eval-okay-toxml.exp, tests/lang/eval-okay-to-xml.nix: New
  files.
2010-03-31 12:38:31 +00:00
Eelco Dolstra e418976107 * Option --argstr for passing string arguments easily. (NIX-75) 2007-01-14 12:32:44 +00:00
Eelco Dolstra 9e30694f98 * Fix the tests wrt the AST changes, i.e., Str(s) -> Str(s, []), and
the semantic changes.
2006-10-17 11:08:59 +00:00
Eelco Dolstra e47e0c2dbe * Builtin function `getEnv' for getting environment variables. 2006-09-24 17:48:41 +00:00
Eelco Dolstra 2ab4bc44c7 * Builtin function `add' to add integers.
* Put common test functions in tests/lang/lib.nix.
2006-09-22 15:29:21 +00:00
Eelco Dolstra 93d9797eda * Urgh (see NIX-56). 2006-08-22 09:34:38 +00:00
Eelco Dolstra 4874fd2d9a * Test for `nix-instantiate --eval-only --xml'. 2006-08-17 11:28:29 +00:00
Eelco Dolstra 84c4631221 * Simplification. 2006-03-01 12:51:18 +00:00
Eelco Dolstra a4c63c6e8e * Make it easy to run individual tests from the command line. 2006-03-01 12:15:33 +00:00
Eelco Dolstra ed09821859 * Use atdiff' instead of cmp' for checking test output.
* Don't use local file names in tests since they will produce
  different parse trees depending on the current directory.
2004-10-27 13:12:58 +00:00
Eelco Dolstra 463e2817c5 * Remove ancient Fix tests.
* Add automated Nix expression language tests.
2004-10-27 12:41:53 +00:00