Commit graph

1024 commits

Author SHA1 Message Date
John Ericson 3df78858f2 Fix max fd calc and add test 2020-08-20 05:08:50 +00:00
Eelco Dolstra 0c9365c6ba
Merge pull request #3940 from obsidiansystems/fixed-output-remote-builder-test
Add commented-out test for remote building with fixed output derivations
2020-08-18 16:12:35 +02:00
Carlo Nucera 07975979aa Comment out fixed content address test 2020-08-17 15:04:54 -04:00
Eelco Dolstra 7cdc739ece
Merge remote-tracking branch 'origin/master' into markdown 2020-08-17 13:43:39 +02:00
John Ericson dbf96e10ec Test remote building with fixed output derivations 2020-08-16 17:38:12 +00:00
John Ericson 3c8b5b6219 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-08-14 17:00:13 +00:00
Eelco Dolstra 7714d9a943
Merge pull request #3924 from obsidiansystems/features-per-store
Make `system-features` a store setting
2020-08-14 17:13:07 +02:00
John Ericson d2f2be0f70 Test RemoteStore::buildDerivation
Fix `wopNarFromPath` which needed a `toRealPath`.
2020-08-13 04:07:14 +00:00
John Ericson 5d67f18c86 Merge branch 'daemon-auth-cleanup' of github.com:obsidiansystems/nix into HEAD 2020-08-12 18:22:31 +00:00
John Ericson 4720853129 Make system-features a store setting
This seems more correct. It also means one can specify the features a
store should support with --store and remote-store=..., which is useful.
I use this to clean up the build remotes test.
2020-08-12 18:13:00 +00:00
John Ericson 8d4162ff9e Separate auth and logic for the daemon
Before, processConnection wanted to know a user name and user id, and
`nix-daemon --stdio`, when it isn't proxying to an underlying daemon,
would just assume "root" and 0. But `nix-daemon --stdio` (no proxying)
shouldn't make guesses about who holds the other end of its standard
streams.

Now processConnection takes an "auth hook", so `nix-daemon` can provide
the appropriate policy and daemon.cc doesn't need to know or care what
it is.
2020-08-12 15:22:33 +00:00
John Ericson 1b5c24662b Merge branch 'small-drv-serialize-cleanup' of github.com:obsidiansystems/nix into single-ca-drv-build 2020-08-10 01:57:54 +00:00
John Ericson e913a2989f Squashed get CA derivations building 2020-08-07 19:51:55 +00:00
Eelco Dolstra edfd676e05 Fix .ls file names in binary caches
These are not supposed to include the 'name' part of the store
path. This was broken by 759947bf72.
2020-08-07 21:18:29 +02:00
John Ericson 6d9ccde20d Make JSON equality tests agnostic to ordering
It is in fact more sorted than before, but I don't think we want to
guarantee anything about the ordering.
2020-08-04 14:20:13 +00:00
Eelco Dolstra bf290c2306
Merge remote-tracking branch 'origin/master' into markdown 2020-07-31 16:07:04 +02:00
Matthew Bauer 0bdd6cf6ea Add test for builtins.path 2020-07-30 16:38:04 -05:00
Matthew Bauer 05ac4db39a Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-07-30 12:38:24 -05:00
Eelco Dolstra 0c94c17644 Merge remote-tracking branch 'origin/master' into markdown 2020-07-29 18:08:57 +02:00
Maximilian Bosch e4940e90f3
Restore backwards-compat for current builtins.fetchGit
If a repo is dirty, it used to return a `rev` object with an "empty"
sha1 (0000000000000000000000000000000000000000). Please note that this
only applies for `builtins.fetchGit` and *not* for `builtins.fetchTree{
type = "git"; }`.
2020-07-28 22:46:39 +02:00
Maximilian Bosch f742438465
Merge legacy fetchGit-builtin with the generic fetchTree-function
The original idea was to implement a git-fetcher in Nix's core that
supports content hashes[1]. In #3549[2] it has been suggested to
actually use `fetchTree` for this since it's a fairly generic wrapper
over the new fetcher-API[3] and already supports content-hashes.

This patch implements a new git-fetcher based on `fetchTree` by
incorporating the following changes:

* Removed the original `fetchGit`-implementation and replaced it with an
  alias on the `fetchTree` implementation.

* Ensured that the `git`-fetcher from `libfetchers` always computes a
  content-hash and returns an "empty" revision on dirty trees (the
  latter one is needed to retain backwards-compatibility).

* The hash-mismatch error in the fetcher-API exits with code 102 as it
  usually happens whenever a hash-mismatch is detected by Nix.

* Removed the `flakes`-feature-flag: I didn't see a reason why this API
  is so tightly coupled to the flakes-API and at least `fetchGit` should
  remain usable without any feature-flags.

* It's only possible to specify a `narHash` for a `git`-tree if either a
  `ref` or a `rev` is given[4].

* It's now possible to specify an URL without a protocol. If it's missing,
  `file://` is automatically added as it was the case in the original
  `fetchGit`-implementation.

[1] https://github.com/NixOS/nix/pull/3216
[2] https://github.com/NixOS/nix/pull/3549#issuecomment-625194383
[3] https://github.com/NixOS/nix/pull/3459
[4] https://github.com/NixOS/nix/pull/3216#issuecomment-553956703
2020-07-28 00:44:39 +02:00
Eelco Dolstra 8e41c38867
Remove references to xmllint 2020-07-23 18:26:48 +02:00
Carlo Nucera 4178f36a1d Test relative store paths 2020-07-17 15:50:53 -04:00
Eelco Dolstra 17f75f9cc4 parseFlakeRef(): Only search for the top-level directory for CLI flakerefs 2020-07-17 14:54:21 +02:00
Eelco Dolstra 832e111494 Merge remote-tracking branch 'origin/master' into flakes 2020-07-14 13:56:18 +02:00
Eelco Dolstra 143a5f32ed Add a test for local NAR caching 2020-07-13 16:25:48 +02:00
Eelco Dolstra 1d01ae816b Fix 'nix verify --all' on a binary cache and add a test 2020-07-13 14:35:01 +02:00
Eelco Dolstra 2900a441f5 Add a test for DWARF debug info index generation 2020-07-13 13:38:01 +02:00
Eelco Dolstra 41bdf429ec Add a test for NAR listing generation 2020-07-13 13:32:33 +02:00
Matthew Bauer acb74d4d94 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-07-10 18:13:45 -04:00
Eelco Dolstra 68f524d717 nix develop: Support derivations with multiple outputs 2020-07-06 18:34:58 +02:00
Eelco Dolstra 54712aaf8a Merge remote-tracking branch 'origin/master' into flakes 2020-07-06 16:40:10 +02:00
Ben Burdette b29a4ea1dc Merge branch 'master' into add-trace 2020-07-03 07:57:36 -06:00
Eelco Dolstra dfaad374ff
Merge pull request #3778 from tweag/parallel-tests
Parallel tests fixes
2020-07-03 13:17:10 +02:00
Eelco Dolstra 017efae01f Hopefully fix macOS test failure 2020-07-03 13:16:22 +02:00
regnat 223fbe644a Shorten the path to the test root
Fix a socket length failure on the OSX builders
2020-07-03 09:20:01 +02:00
regnat 5101ed18bc Fix the test dependencies
Reuse the pre-existing list rather than the one written as part of #3777
2020-07-03 09:20:01 +02:00
Eelco Dolstra 5596f879b4 Add test for nix develop 2020-07-02 18:32:45 +02:00
Matthew Bauer fc2ab42e86 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-07-02 11:14:04 -04:00
Eelco Dolstra a5b6e870fe Set gc-reserved-space to 0 in tests
This reduces the amount of disk space needed to run the tests from
half a gigabyte to 10 megabytes.
2020-07-02 16:38:42 +02:00
regnat 11ba4ec795 Make the gc-auto test more reliable
Use a fifo pipe to handle the synchronisation between the different
threads rather than relying on delays
2020-07-02 16:13:36 +02:00
regnat c762385457 Make the gc-concurrent test more reliable
Use a fifo pipe to handle the synchronisation between the different
threads rather than relying on delays
2020-07-02 16:13:36 +02:00
regnat 1b5aa60767 Run the tests in parallel
Cause the time needed to run the testsuite to drop from ~4mins to ~40s
2020-07-02 16:13:36 +02:00
Ben Burdette dabbb4538f 'from string' 2020-06-30 16:43:01 -06:00
Matthew Bauer 09833126f3 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-30 16:35:08 -04:00
Ben Burdette ddb81ca126 Merge branch 'master' into add-trace 2020-06-30 12:21:45 -06:00
Eelco Dolstra adf2fbbdc2 Merge remote-tracking branch 'origin/master' into flakes 2020-06-26 08:46:46 +02:00
Eelco Dolstra b7ccf7ae2a build-remote.sh: Test LegacySSHStore 2020-06-25 18:42:55 +02:00
Ben Burdette 00fe653ea5 nixCode -> LinesOfCode 2020-06-24 08:33:53 -06:00
Ben Burdette e6f93b94fc Merge branch 'master' into caveman-LOCs 2020-06-18 13:07:53 -06:00
Eelco Dolstra 3d492199bb github: Respect default branch 2020-06-18 13:25:08 +02:00
Matthew Bauer 7bd88cc1dc Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-17 13:32:29 -04:00
Eelco Dolstra 1524752c17 Merge remote-tracking branch 'origin/master' into flakes 2020-06-17 10:26:52 +02:00
Matthew Bauer 9ac0d98a59 Remove -vvvvv from tests/fetchurl.sh nix-build call 2020-06-13 00:14:30 -05:00
Matthew Bauer c214cda940 Correctly substitute from different storeDir
Originally, the test was only checking for different “real” storeDir.
That’s an easy case to handle, but the much harder one is if different
virtual store dirs are used. To do this, we need the SubstitutionGoal
to know about the ca, so it can recalculate the path to copy it over.
An important note here is that the store path passed to copyStorePath
needs to be one for srcStore - so that queryPathInfo works properly.

This also adds an error message when the store path from queryPathInfo
is different from the one we requested.
2020-06-13 00:07:42 -05:00
Matthew Bauer 006f1252d2 Fix SRI test
We can’t use custom name here because different names will have
different store paths. This is a limitation of the Store API’s
reliance on store paths.

We might be able to get around the above in the future by using a
dummy name for certain fixed output paths.
2020-06-12 15:33:24 -05:00
Matthew Bauer 88120442d2 Debug when storePath changes
these rewrites should be transparent, but they are important to know
about when debugging
2020-06-12 15:32:52 -05:00
Matthew Bauer 1c148a80fe Replace --hashed-mirrors with substituters test 2020-06-12 14:39:44 -05:00
Ben Burdette ef1b3f21b6 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-06-11 14:06:35 -06:00
Eelco Dolstra 0c62b4ad0f Represent 'follows' inputs explicitly in the lock file
This fixes an issue where lockfile generation was not idempotent:
after updating a lockfile, a "follows" node would end up pointing to a
new copy of the node, rather than to the original node.
2020-06-11 14:40:21 +02:00
Ben Burdette b1c53b034c Merge branch 'errors-phase-2' into caveman-LOCs 2020-06-08 11:10:13 -06:00
regnat f6ac888d3e Actually test nix-env with a remote store
The `remote-store` test loads the `user-env` one to test nix-env when
using the daemon, but actually does it incorrectly because every test
starts (in `common.sh`) by resetting the value of `NIX_REMOTE`, meaning
that the `user-env` test will never use the daemon.

Fix this by setting `NIX_REMOTE_` before sourcing `user-env.sh` in the
`remote-store` test, so that `NIX_REMOTE` is correctly set inside the
test
2020-06-08 10:01:14 +02:00
Eelco Dolstra 810b2c6a48 nix flake init: Add a '--template' flag
The initial contents of the flake is specified by the
'templates.<name>' or 'defaultTemplate' output of another flake. E.g.

  outputs = { self }: {

    templates = {

      nixos-container = {
        path = ./nixos-container;
        description = "An example of a NixOS container";
      };

    };

  };

allows

  $ nix flake init -t templates#nixos-container

Also add a command 'nix flake new', which is identical to 'nix flake
init' except that it initializes a specified directory rather than the
current directory.
2020-06-04 20:22:25 +02:00
Ben Burdette 721943e1d4 update error grep 2020-06-03 17:32:57 -06:00
Ben Burdette 4335ba999b Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-06-03 17:00:00 -06:00
Eelco Dolstra 81cafda306 Fix GitHub test 2020-06-03 16:29:04 +02:00
Eelco Dolstra c20591ddc3 Merge remote-tracking branch 'origin/master' into flakes 2020-06-03 16:15:22 +02:00
Nikola Knezevic 77007d4eab Improve ref validity checking in fetchGit
The previous regex was too strict and did not match what git was allowing. It
could lead to `fetchGit` not accepting valid branch names, even though they
exist in a repository (for example, branch names containing `/`, which are
pretty standard, like `release/1.0` branches).

The new regex defines what a branch name should **NOT** contain. It takes the
definitions from `refs.c` in https://github.com/git/git and `git help
check-ref-format` pages.

This change also introduces a test for ref name validity checking, which
compares the result from Nix with the result of `git check-ref-format --branch`.
2020-05-30 12:29:35 +02:00
Eelco Dolstra 17ca997fc6 Merge remote-tracking branch 'origin/master' into flakes 2020-05-28 12:55:24 +02:00
Eelco Dolstra c3eff22f46 Merge branch 'store-visited' of https://github.com/mkenigs/nix into flakes 2020-05-28 12:15:39 +02:00
Matthew Kenigsberg 934cc802f3 circular test 2020-05-21 17:06:19 -06:00
Ben Burdette 0e49de6a2b position for stdin, string; (string) for trace; fix tests 2020-05-21 14:28:45 -06:00
Eelco Dolstra 00b562c87e Fix GitHub test 2020-05-21 22:02:34 +02:00
Ben Burdette 92123c6c79 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-05-15 07:00:36 -06:00
Eelco Dolstra 5f64655ff4 Move registry-related commands from 'nix flake' to 'nix registry'
This makes 'nix flake' less cluttered and more consistent (it's only
subcommands that operator on a flake). Also, the registry is not
inherently flake-related (e.g. fetchTree could also use it to remap
inputs).
2020-05-15 14:38:10 +02:00
Eelco Dolstra 5722f9690c tests/binary-cache.sh: Improve incomplete closure test
Issue #3373.
2020-05-12 13:56:00 +02:00
Ben Burdette 59b1f5c701 Merge branch 'master' into errors-phase-2 2020-05-11 14:35:30 -06:00
Ben Burdette 55eb717148 add pos to errorinfo, remove from hints 2020-05-08 18:18:28 -06:00
Matthew Kenigsberg 2852a486f8 rename run to shell in tests
(cherry picked from commit f459ca547f)
2020-05-01 12:17:53 +02:00
Matthew Kenigsberg f459ca547f rename run to shell in tests 2020-04-30 08:31:37 -06:00
Eelco Dolstra b69323f8c9 Revive 'nix search'
It uses the evaluation cache now rather than the ad hoc JSON cache.
2020-04-20 15:27:09 +02:00
Eelco Dolstra 0858738355 Merge remote-tracking branch 'origin/master' into flakes 2020-04-16 18:27:37 +02:00
Eelco Dolstra a118293bd0
Merge pull request #3458 from zimbatm/nix-user-conf-dir
NIX_USER_CONF_FILES
2020-04-15 13:00:28 +02:00
Jonas Chevalier 895516cadf
add NIX_USER_CONF_FILES
Motivation: maintain project-level configuration files.

Document the whole situation a bit better so that it corresponds to the
implementation, and add NIX_USER_CONF_FILES that allows overriding
which user files Nix will load during startup.
2020-04-14 18:45:06 +02:00
Eelco Dolstra c0c2cb871d Merge remote-tracking branch 'origin/master' into flakes 2020-04-14 13:02:55 +02:00
Bruce Toll e8bd1bc732 Add test case for temporary directories on darwin
A test case for correct handling of temporary directory deletion that
was added to check.sh as part of PR #2689 was initially disabled for
Darwin because of a directory permission issue in PR #2688.

Now that the issue in PR #2688 is fixed, this commit enables the test
case for Darwin.
2020-04-10 18:20:12 -04:00
Bruce Toll 8132d0a12e Fix nix-build --check -K in sandbox w/o root
Temporarily add user-write permission to build directory so that it
can be moved out of the sandbox to the store with a .check suffix.

This is necessary because the build directory has already had its
permissions set read-only, but write permission is required
to update the directory's parent link to move it out of the sandbox.

Updated the related --check "derivation may not be deterministic"
messages to consistently use the real store paths.

Added test for non-root sandbox nix-build --check -K to demonstrate
issue and help prevent regressions.
2020-04-10 16:23:10 -04:00
Eelco Dolstra e5ea01c1a8
Remove flake 'edition' field
Future editions of flakes or the Nix language can be supported by
renaming flake.nix (e.g. flake-v2.nix). This avoids a bootstrap
problem where we don't know which grammar to use to parse
flake*.nix. It also allows a project to support multiple flake
editions, in theory.
2020-04-10 10:24:09 +02:00
Bruce Toll 16a4864759 Delete temporary directory on successful build
With --check and the --keep-failed (-K) flag, the temporary directory
was being retained regardless of whether the build was successful and
reproducible.  This removes the temporary directory, as expected, on
a reproducible check build.

Added tests to verify that temporary build directories are not
retained unnecessarily, particularly when using --check with
--keep-failed.
2020-04-09 16:37:41 -04:00
Eelco Dolstra d103c79144
Merge remote-tracking branch 'origin/master' into flakes 2020-04-09 15:54:21 +02:00
Eelco Dolstra 9ed097db7b
Merge pull request #3468 from Infinisil/functionArgsPositions
Make function arguments retain position info
2020-04-08 15:29:39 +02:00
Eelco Dolstra 03a4a3c95c Merge branch 'flakes' of github.com:NixOS/nix into flakes 2020-04-07 14:30:25 +02:00
Eelco Dolstra 54955867a6 Merge remote-tracking branch 'origin/master' into flakes 2020-04-07 14:29:45 +02:00
Eelco Dolstra 55cefd41d6 Merge branch 'fetchgit-recursive' of https://github.com/blitz/nix 2020-04-07 13:45:17 +02:00
Eelco Dolstra 462421d345 Backport libfetchers from the flakes branch
This provides a pluggable mechanism for defining new fetchers. It adds
a builtin function 'fetchTree' that generalizes existing fetchers like
'fetchGit', 'fetchMercurial' and 'fetchTarball'. 'fetchTree' takes a
set of attributes, e.g.

  fetchTree {
    type = "git";
    url = "https://example.org/repo.git";
    ref = "some-branch";
    rev = "abcdef...";
  }

The existing fetchers are just wrappers around this. Note that the
input attributes to fetchTree are the same as flake input
specifications and flake lock file entries.

All fetchers share a common cache stored in
~/.cache/nix/fetcher-cache-v1.sqlite. This replaces the ad hoc caching
mechanisms in fetchGit and download.cc (e.g. ~/.cache/nix/{tarballs,git-revs*}).

This also adds support for Git worktrees (c169ea5904).
2020-04-07 09:03:14 +02:00
Eelco Dolstra 3473b1950a
Fix tests.githubFlakes evaluation 2020-04-03 21:36:21 +02:00
Eelco Dolstra 9c78f7f196 getFlake: In pure mode, check that the argument is an immutable flakeref 2020-04-03 13:07:05 +02:00
Silvan Mosberger c34e96f7e0
Make function arguments retain position info
This allows querying the location of function arguments. E.g.

  builtins.unsafeGetAttrPos "x" (builtins.functionArgs ({ x }: null))

  => { column = 57; file = "/home/infinisil/src/nix/inst/test.nix"; line = 1; }
2020-04-02 05:52:52 +02:00
Eelco Dolstra d15d91cad1 Makefile cleanup 2020-03-30 16:50:13 +02:00
Eelco Dolstra e322a16523 Remove global -I flags
(cherry picked from commit 2c692a3b14)
2020-03-30 15:30:19 +02:00