lix-project/lix
20
57
Fork 31
Code Issues 347 Code Review (Gerrit) Projects 3 Releases Wiki Activity
lix/tests/functional2
History
jade 3571817e3a testsuite: add a NAR generator with some evil NARs 
This also rewrites a lot of the command handling in the fixtures
library, since we want to more precisely control which way that the nix
store is set up in the tests, rather than the previous method of
renaming /nix/store to some temp dir (which allows builds but does not
allow any /nix/store paths or stability across runs, which is a
significant issue for snapshot testing).

It uses a builder to reduce the amount of state carelessly thrown
around.

The evil NARs are inspired by CVE-2024-45593
(https://github.com/NixOS/nix/security/advisories/GHSA-h4vv-h3jq-v493).

No bugs were found in this endeavor.

Change-Id: Iee41b055fa96529c5a3c761f680ed1d0667ba5da
2024-10-09 14:47:39 -07:00
..
store testsuite: add a NAR generator with some evil NARs 2024-10-09 14:47:39 -07:00
testlib testsuite: add a NAR generator with some evil NARs 2024-10-09 14:47:39 -07:00
__init__.py
conftest.py
meson.build testsuite: use xdist for parallel test running 2024-10-09 14:47:39 -07:00
README.md testsuite: use xdist for parallel test running 2024-10-09 14:47:39 -07:00
test_eval_trivial.py

README.md

functional2 tests

This uncreatively named test suite is a Pytest based replacement for the shell framework used to write traditional Nix integration tests. Its primary goal is to make tests more concise, more self-contained, easier to write, and to produce better errors.

Goals

  • Eliminate implicit dependencies on files in the test directory as well as the requirement to copy the test files to the build directory as is currently hacked in the other functional test suite.
    • You should be able to write a DirectoryTree of files for your test declaratively.
  • Reduce the amount of global environment state being thrown around in the test suite.
  • Make tests very concise and easy to reuse code for, and hopefully turn more of what is currently code into data.
    • Provide rich ways of calling nix with pleasant syntax.

TODO: Intended features

  • Expect tests (pytest-expect-test) or snapshot tests (pytest-insta) or, likely, both! We::jade prefer to have short output written in-line as it makes it greatly easier to read the tests, but pytest-expect doesn't allow for putting larger stuff in external files, so something else is necessary for those.
  • Web server fixture: we don't test our network functionality because background processes are hard and this is simply goofy. We could just test it.
  • Nix daemon fixture.
  • Parallelism via pytest-xdist.