From 7c75f1d52b3078608be29cbe0b009875829cc03f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9ophane=20Hufschmitt?= Date: Tue, 10 May 2022 11:56:47 +0200 Subject: [PATCH 1/2] Expand the testing section of the hacking docs - Make it clear what the different kind of tests are, where they live and how they can be ran - Ask people to primarily write unit tests --- doc/manual/src/contributing/hacking.md | 41 ++++++++++++++++++-------- 1 file changed, 29 insertions(+), 12 deletions(-) diff --git a/doc/manual/src/contributing/hacking.md b/doc/manual/src/contributing/hacking.md index 90a8f1f94..7ce8d8de6 100644 --- a/doc/manual/src/contributing/hacking.md +++ b/doc/manual/src/contributing/hacking.md @@ -71,18 +71,6 @@ To install it in `$(pwd)/outputs` and test it: nix (Nix) 3.0 ``` -To run a functional test: - -```console -make tests/test-name-should-auto-complete.sh.test -``` - -To run the unit-tests for C++ code: - -``` -make check -``` - If you have a flakes-enabled Nix you can replace: ```console @@ -94,3 +82,32 @@ by: ```console $ nix develop ``` + +## Testing + +Nix comes with three different flavors of tests: unit, functional and integration. + +Most tests are currently written as functional tests. +**However**, it is preferable (as much as it makes sense) to primarily test new code with unit tests. + +### Unit-tests + +The unit-tests for each Nix library (`libexpr`, `libstore`, etc..) are defined +under `src/{library_name}/tests` using the +[googletest](https://google.github.io/googletest/) framework. + +You can run the whole testsuite with `make check`, or the tests for a specific component with `make libfoo-tests_RUN`. Finer-grained filtering is also possible using the [--gtest_filter](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) command-line option. + +### Functional tests + +The functional tests reside under the `tests` directory and are listed in `tests/local.mk`. +The whole testsuite can be run with `make install && make installcheck`. +Individual tests can be run with `make tests/{testName}.sh.test`. + +### Integration tests + +The integration tests are defined in the Nix flake under the `hydraJobs.tests` attribute. +These tests include everything that needs to interact with external services or run Nix in a non-trivial distributed setup. +Because these tests are expensive and require more than what the standard github-actions setup provides, they only run on the master branch (on ). + +You can run them manually with `nix build .#hydraJobs.tests.{testName}` or `nix-build -A hydraJobs.tests.{testName}` From 65a913d29be7305b2c743fb92c93f0e6bb12d610 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9ophane=20Hufschmitt?= Date: Thu, 12 May 2022 12:02:31 +0200 Subject: [PATCH 2/2] =?UTF-8?q?Don=E2=80=99t=20recommend=20writing=20unit?= =?UTF-8?q?=20tests?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit As asked in --- doc/manual/src/contributing/hacking.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/doc/manual/src/contributing/hacking.md b/doc/manual/src/contributing/hacking.md index 7ce8d8de6..59ce5cac7 100644 --- a/doc/manual/src/contributing/hacking.md +++ b/doc/manual/src/contributing/hacking.md @@ -87,9 +87,6 @@ $ nix develop Nix comes with three different flavors of tests: unit, functional and integration. -Most tests are currently written as functional tests. -**However**, it is preferable (as much as it makes sense) to primarily test new code with unit tests. - ### Unit-tests The unit-tests for each Nix library (`libexpr`, `libstore`, etc..) are defined