Commit graph

211 commits

Author SHA1 Message Date
Eelco Dolstra 420957e149 Move flakes tests to a subdirectory 2022-07-13 15:06:54 +02:00
Théophane Hufschmitt 260fb837de Fix the name of the completions test 2022-07-12 09:24:31 +02:00
Théophane Hufschmitt 32effccb51 Add some tests for the CLI completion 2022-07-12 09:24:31 +02:00
Eelco Dolstra 8bbbb6e737 Enable/fix tests in nix-static
pkgsStatic is apparently considered a cross environment, so checkPhase
and installCheckPhase are disabled even when we ask for them.
2022-06-22 23:33:15 +02:00
Tony Olagbaiye 5b8c1deb18 fetchTree: Allow fetching plain files
Add a new `file` fetcher type, which will fetch a plain file over
http(s), or from the local file.

Because plain `http(s)://` or `file://` urls can already correspond to
`tarball` inputs (if the path ends-up with a know archive extension),
the URL parsing logic is a bit convuluted in that:

- {http,https,file}:// urls will be interpreted as either a tarball or a
  file input, depending on the extensions of the path part (so
  `https://foo.com/bar` will be a `file` input and
  `https://foo.com/bar.tar.gz` as a `tarball` input)
- `file+{something}://` urls will be interpreted as `file` urls (with
  the `file+` part removed)
- `tarball+{something}://` urls will be interpreted as `tarball` urls (with
  the `tarball+` part removed)

Fix #3785

Co-Authored-By: Tony Olagbaiye <me@fron.io>
2022-05-19 18:24:49 +02:00
Sergei Trofimovich 6ada496311 nix: add (failing) selfreference test for multiple realizations
The test illustrates failure in issue #5320. Here derivation and
it's built input have identical CA sotre path. As a result we generate
extraneout reference to build input:

    $ make installcheck
    ...
    ran test tests/selfref-gc.sh... [PASS]
    ran test tests/ca/selfref-gc.sh... [FAIL]
    ...
        deleting '/tmp/.../tests/ca/selfref-gc/store/iqciq1mpg5hc7p6a52fp2bjxbyc9av0v-selfref-gc'
        deleting '/tmp/...tests/ca/selfref-gc/store/zh0kwpnirw3qbv6dl1ckr1y0kd5aw6ax-selfref-gc.drv'
        error: executing SQLite statement
          'delete from ValidPaths where path = '/tmp/.../tests/ca/selfref-gc/store/fsjq0k146r85lsh01l0icl30rnhv7z72-selfref-gc';':
            constraint failed (in '/tmp/.../tests/ca/selfref-gc/var/nix/db/db.sqlite')
2022-04-21 10:06:39 +02:00
John Ericson 08b8657978 Merge branch 'path-info' into ca-drv-exotic 2022-04-19 22:39:57 +00:00
Eelco Dolstra 48a467f2b9 Merge branch 'issue-6075' of https://github.com/kamadorueda/nix 2022-04-19 20:21:32 +02:00
Tom Bereknyei 9b41239d8f fix: ensure apps are apps and packages are packages 2022-04-14 23:57:52 -04:00
Eelco Dolstra 5cd72598fe Add support for impure derivations
Impure derivations are derivations that can produce a different result
every time they're built. Example:

  stdenv.mkDerivation {
    name = "impure";
    __impure = true; # marks this derivation as impure
    outputHashAlgo = "sha256";
    outputHashMode = "recursive";
    buildCommand = "date > $out";
  };

Some important characteristics:

* This requires the 'impure-derivations' experimental feature.

* Impure derivations are not "cached". Thus, running "nix-build" on
  the example above multiple times will cause a rebuild every time.

* They are implemented similar to CA derivations, i.e. the output is
  moved to a content-addressed path in the store. The difference is
  that we don't register a realisation in the Nix database.

* Pure derivations are not allowed to depend on impure derivations. In
  the future fixed-output derivations will be allowed to depend on
  impure derivations, thus forming an "impurity barrier" in the
  dependency graph.

* When sandboxing is enabled, impure derivations can access the
  network in the same way as fixed-output derivations. In relaxed
  sandboxing mode, they can access the local filesystem.
2022-03-31 13:43:20 +02:00
John Ericson ff2a8ccfe1 Merge branch 'path-info' into ca-drv-exotic 2022-03-25 19:40:52 +00:00
Eelco Dolstra 28186b7044 Add a test for fetchClosure and 'nix store make-content-addressed' 2022-03-24 21:33:33 +01:00
Théophane Hufschmitt 7117053457
Merge pull request #6270 from Artturin/stdineval
nix: allow using --file - to read from stdin
2022-03-17 14:35:01 +01:00
Artturin 4f8ad41d4e add tests for nix eval and nix-instantiate 2022-03-16 21:01:51 +02:00
Maximilian Bosch 975bade7f0
Implement simple test for path-fetcher setting a correct lastModifiedDate
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-03-15 12:55:32 +01:00
Kevin Amado 2191dab657
nix-fmt: add command 2022-03-11 10:00:19 -05:00
John Ericson 938650700f Merge branch 'path-info' into ca-drv-exotic 2022-03-10 16:20:01 +00:00
regnat 91635206c0 Add some end-to-end tests for the suggestions 2022-03-07 10:09:10 +01:00
Eelco Dolstra 5850fd17b4 Add basic tests for 'nix profile'
Fixes #6193.
2022-03-02 14:40:46 +01:00
Eelco Dolstra c10865a46e tests: Rename nix-profile.sh -> bash-profile.sh 2022-03-02 11:21:00 +01:00
John Ericson c863e5f338 Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2022-02-28 18:29:33 +00:00
regnat 42766f8924 Sort the tests by wall time
The tests are scheduled in the order they appear, so running the long
ones first slightly improves the scheduling.

On my machine, this decreases the time of `make install` from 40s to 36s
2022-02-24 14:32:31 +01:00
tomberek 2bf96bd9f2
Merge branch 'master' into bundler_drv 2022-01-28 10:18:29 -05:00
regnat fcdc60ed22 Don’t require NIX_PATH entries to be valid paths
It’s totally valid to have entries in `NIX_PATH` that aren’t valid paths
(they can even be arbitrary urls or `channel:<channel-name>`).

Fix #5998 and #5980
2022-01-27 16:26:39 +01:00
regnat d139474f48 Add some tests for nix store ping
Always good to have :)
2022-01-26 11:01:25 +01:00
Tom Bereknyei 93299efc7c bundler: add tests and change defaults to use a derivation 2022-01-25 03:39:18 -05:00
regnat 2ad2678c0b Add a simple test for nix why-depends 2022-01-18 16:54:53 +01:00
Théophane Hufschmitt dc4730ee94
Merge pull request #5720 from tomberek/flake_search
flakes: search up to git or filesystem boundary
2022-01-18 10:59:12 +01:00
Nikolay Amiantov 120ca245d1 Add test for readFile keeping context 2022-01-09 13:42:36 +03:00
tomberek f404189368
Merge branch 'master' into flake_search 2021-12-23 15:22:52 -05:00
regnat d90f9d4b99 Fix IFD with CA derivations
Rewrite the string taken by the IFD-like primops to contain the actual
output paths of the derivations rather than the placeholders

Fix #5805
2021-12-21 09:36:50 +01:00
Tom Bereknyei b6cc0a704d flakes: search up to git or filesystem boundary
While parsing a flakeref, upon not finding a flake.nix, search upwards
until git or filesystem boundary.
2021-12-03 10:55:30 -05:00
Alexander Sosedkin a9bd06d0ea Make libcpuid dependency optional with --disable-cpuid 2021-11-28 00:52:35 +01:00
Eelco Dolstra 736d6ab721
Merge pull request #5504 from NixOS/flake-options-and-daemon
Make the flake options work when using the daemon
2021-11-08 13:54:55 +01:00
regnat 1f3c3a3785 Make the flake options work when using the daemon
When setting flake-local options (with the `nixConfig` field), forward
these options to the daemon in case we’re using one.

This is necessary in particular for options like `binary-caches` or
`post-build-hook` to make sense.

Fix <343239fc8a (r44356843)>
2021-11-05 16:19:16 +01:00
regnat 1a4c9ba50b Fix nix repl’s building of CA derivations
When running a `:b` command in the repl, after building the derivations
query the store for its outputs rather than just assuming that they are
known in the derivation itself (which isn’t true for CA derivations)

Fix #5328
2021-11-05 11:04:14 +01:00
Eelco Dolstra 10f9a8e77d Add a test for the non-blocking GC 2021-10-15 16:52:37 +02:00
John Ericson 7869be49c2 Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2021-10-14 20:07:20 +00:00
Tom Bereknyei 03bb8f84e0 Add compression level for NARs
Based off on @dtzWill's #2276
2021-10-12 02:14:36 -04:00
John Ericson 9af9ab4212 Merge branch 'path-info' into ca-drv-exotic 2021-09-30 22:42:15 +00:00
Eelco Dolstra 29e4913f79 Add --eval-store test 2021-07-27 11:17:56 +02:00
regnat 8707773965 Properly lock the builds of CA derivations
Make sure that we can’t build the same derivation twice at the same
time.

Fix https://github.com/NixOS/nix/issues/5029
2021-07-20 06:57:56 +02:00
regnat c05bdef020 Forward the whole Nix config to the repl subprocesses
Fill `NIX_CONFIG` with the value of the current Nix configuration before
calling the nix subprocesses in the repl
That way the whole configuration (including the possible
`experimental-features`, a possibly `--store` option or whatever) will
be made available.

This is required for example to make `nix repl` work with a custom
`--store`
2021-07-16 09:48:45 +02:00
Eelco Dolstra f5320299dd
Merge pull request #4937 from NixOS/ca/make-the-tests-useful
Make the CA tests actually test something
2021-06-28 16:06:49 +02:00
regnat 7746cb13dc Make CA derivations compatible with recursive Nix
Add an access-control list to the realisations in recursive-nix (similar
to the already existing one for store paths), so that we can build
content-addressed derivations in the restricted store.

Fix #4353
2021-06-24 14:53:10 +02:00
regnat be7a4a6a13 Make the post-build-hook also run for unresolved CA derivations
Fix #4837
2021-06-24 11:41:57 +02:00
Eelco Dolstra 0a535dd5ac
Merge pull request #4839 from NixOS/ca/gracefully-handle-duplicate-realisations
Gracefully handle duplicate realisations
2021-06-23 11:50:18 +02:00
Eelco Dolstra 7945055c63
Merge pull request #4842 from NixOS/ca/fix-nix-shell
Make `nix-shell` support content-addressed derivations
2021-06-23 11:26:22 +02:00
regnat a5df669bc6 Add a test for the “two glibc” issue 2021-06-23 11:18:31 +02:00
Eelco Dolstra 26d2c62225
Merge pull request #4906 from NixOS/collect-garbage-ca
Make `computeFSClosure` ca-aware
2021-06-23 10:07:08 +02:00
regnat 3784c66a46 Remove a possible existing store path when building CA derivations
In case a previous interrupted build left a garbage path laying around,
remove it before trying to move the path to its final location.

Fix #4858
2021-06-21 15:29:15 +02:00
regnat a3ce88725b Add a test for the gc with CA derivations
Also add a small architecture to easily run CA-enabled tests
2021-06-15 12:11:31 +02:00
regnat 56605b4688 Make nix-shell support content-addressed derivations
Resolve the derivation before trying to load its environment −
essentially reproducing what the build loop does − so that we can
effectively access our dependencies (and not just their placeholders).

Fix #4821
2021-06-11 13:32:49 +02:00
regnat 5985b8b527 Check the CA hash when importing stuff in the local store
When adding a path to the local store (via `LocalStore::addToStore`),
ensure that the `ca` field of the provided `ValidPathInfo` does indeed
correspond to the content of the path.
Otherwise any untrusted user (or any binary cache) can add arbitrary
content-addressed paths to the store (as content-addressed paths don’t
need a signature).
2021-06-01 15:09:24 +02:00
regnat f46adb783c Add a test for nix run with CA derivations 2021-05-17 15:10:48 +02:00
Eelco Dolstra 6fb7582413 Merge branch 'libarchive-decompress' of https://github.com/serokell/nix 2021-04-15 13:39:04 +02:00
Domen Kožar c2dfda007e
add tests for zstd compression 2021-04-09 23:13:08 +02:00
John Ericson cdc9f34a44 Merge commit 'e12308dd63f0ad27b22dcdb3da89c411eebcad2b' into ca-drv-exotic 2021-04-05 19:06:30 -04:00
regnat 81df1b5c68 Remove the remote-store-old-daemon test
Doesn't make sense anymore with the new setup
2021-03-16 14:21:40 +01:00
regnat eab9cdbd75 Add a test for the migration of the db between versions 2021-03-16 14:20:41 +01:00
regnat 5716345adf Add a test ensuring compatibility with an old daemon
This requires adding `nix` to its own closure which is a bit unfortunate,
but as it is optional (the test will be disabled if `OUTER_NIX` is unset) it
shouldn't be too much of an issue.

(Ideally this should go in another derivation so that we can build Nix and run
the test independently, but as the tests are running in the same derivation
as the build it's a bit complicated to do so).
2021-03-16 14:20:41 +01:00
sternenseemann 5869b3025d tests/local.mk: fix missing newline escape
Fixes syntax error introduced in 54ced9072b.
2021-03-16 08:25:04 +01:00
Eelco Dolstra c0073f6268
Merge pull request #4580 from obsidiansystems/restore-test-build-remote-ca-fixed
Restore now-working build-remote-content-addressed-fixed test
2021-03-15 16:48:28 +01:00
regnat 54ced9072b Check the signatures when copying store paths around
Broken atm
2021-03-15 16:35:14 +01:00
regnat 93b5a59b67 Add a test for the remote caching of CA derivations 2021-03-01 14:00:17 +01:00
regnat 259d6778ef Move the CA tests to a sub-directory
Requires a slight update to the test infra to work properly, but
having the possibility to group tests that way makes the whole thing
quite cleaner imho
2021-03-01 11:08:01 +01:00
John Ericson 7863036634 Merge remote-tracking branch 'obsidian/path-info' into ca-drv-exotic 2021-02-27 05:46:59 +00:00
John Ericson 2dd11f0780 Reenable previously failing trustless remote builder tests 2021-02-27 05:33:47 +00:00
John Ericson e547fe12d1 Merge branch 'restore-test-build-remote-ca-fixed' into trustless-remote-builder-simple 2021-02-27 03:53:22 +00:00
John Ericson 5b42e5b177 Restore now-working build-remote-content-addressed-fixed test
This was

 - Added in dbf96e10ec.

 - Commented out in 07975979aa, which I
   believe only reached master by mistake.

 - Deleted in c32168c9bc, when
   `tests/build-hook-ca.nix` was reused for a new test.

But the test works, and we ought to have it.
2021-02-26 16:32:52 +00:00
Eelco Dolstra 94637cd7e5
Merge pull request #4477 from NixOS/ca/build-remote
Build ca derivations remotely
2021-02-26 16:54:44 +01:00
John Ericson 90d76fa399 Merge remote-tracking branch 'obsidian/path-info' into ca-drv-exotic 2021-02-25 21:58:41 +00:00
regnat aead35531a Add a test for the copy of CA paths 2021-02-25 17:19:18 +01:00
John Ericson 7d703f2106 Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2021-02-24 17:58:43 +00:00
regnat c32168c9bc Test the remote building of ca derivations 2021-02-23 08:04:03 +01:00
Daniël de Kok 2de232d2b3 Add x86_64 compute levels as additional system types
When performing distributed builds of machine learning packages, it
would be nice if builders without the required SIMD instructions can
be excluded as build nodes.

Since x86_64 has accumulated a large number of different instruction
set extensions, listing all possible extensions would be unwieldy.
AMD, Intel, Red Hat, and SUSE have recently defined four different
microarchitecture levels that are now part of the x86-64 psABI
supplement and will be used in glibc 2.33:

https://gitlab.com/x86-psABIs/x86-64-ABI
https://lwn.net/Articles/844831/

This change uses libcpuid to detect CPU features and then uses them to
add the supported x86_64 levels to the additional system types. For
example on a Ryzen 3700X:

$ ~/aps/bin/nix -vv --version | grep "Additional system"
Additional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux
2021-02-22 09:11:15 +01:00
John Ericson b0d52855dc Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2021-02-12 05:51:53 +00:00
John Ericson fed1237246 Test nix-build with non-local-store --store
Just a few small things needed fixing!
2020-12-23 22:42:06 +00:00
John Ericson bd96403da6 Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2020-12-20 14:02:12 +00:00
Matthew Kenigsberg d52b12c0a5 Test nix build --json 2020-11-11 10:27:02 -06:00
John Ericson a4e5de1b9d Derivations can output "text-hashed" data
In particular, this means that derivations can output derivations. But
that ramification isn't (yet!) useful as we would want, since there is
no way to have a dependent derivation that is itself a dependent
derivation.
2020-10-13 02:15:48 +00:00
John Ericson 6a0e0db349 Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2020-09-22 00:21:37 +00:00
John Ericson f60b380a7f Merge remote-tracking branch 'upstream/master' into remove-storetype-delegate-regStore 2020-09-16 22:35:24 +00:00
regnat c29624bf7d Add a test for nix describe-stores
Doesn't test much, but at least ensures that the command runs properly
2020-09-16 13:53:28 +02:00
John Ericson 27a3f82c0b Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-08-20 18:28:17 +00:00
John Ericson 3df78858f2 Fix max fd calc and add test 2020-08-20 05:08:50 +00:00
John Ericson 7c4f383b37 Merge branch 'fixed-output-remote-builder-test' of github.com:obsidiansystems/nix into trustless-remote-builder-simple 2020-08-18 13:56:00 +00:00
Carlo Nucera 07975979aa Comment out fixed content address test 2020-08-17 15:04:54 -04:00
John Ericson 36758a1a09 But back check.sh
Whether it fails or not, it is no a new test so we have to leave it.
2020-08-17 18:01:38 +00:00
Carlo Nucera 1bf31bad5f Disable failing tests 2020-08-17 13:15:08 -04:00
John Ericson dbf96e10ec Test remote building with fixed output derivations 2020-08-16 17:38:12 +00:00
John Ericson 9dd28a65c8 Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2020-08-16 16:05:54 +00:00
John Ericson 3c8b5b6219 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-08-14 17:00:13 +00:00
John Ericson cbc4344297 Trustless remote building
Co-authored-by: Matthew Bauer <mjbauer95@gmail.com>
2020-08-14 04:53:58 +00:00
John Ericson 4720853129 Make system-features a store setting
This seems more correct. It also means one can specify the features a
store should support with --store and remote-store=..., which is useful.
I use this to clean up the build remotes test.
2020-08-12 18:13:00 +00:00
John Ericson e913a2989f Squashed get CA derivations building 2020-08-07 19:51:55 +00:00
Carlo Nucera 4178f36a1d Test relative store paths 2020-07-17 15:50:53 -04:00
Eelco Dolstra 54712aaf8a Merge remote-tracking branch 'origin/master' into flakes 2020-07-06 16:40:10 +02:00
regnat 5101ed18bc Fix the test dependencies
Reuse the pre-existing list rather than the one written as part of #3777
2020-07-03 09:20:01 +02:00
Eelco Dolstra c20591ddc3 Merge remote-tracking branch 'origin/master' into flakes 2020-06-03 16:15:22 +02:00