Commit graph

1983 commits

Author SHA1 Message Date
Maximilian Bosch c1c37f3200
flakes: throw an error if follows-declaration for an input is invalid
I recently got fairly confused why the following expression didn't have
any effect

    {
      description = "Foobar";
      inputs.sops-nix = {
        url = github:mic92/sops-nix;
        inputs.nixpkgs_22_05.follows = "nixpkgs";
      };
    }

until I found out that the input was called `nixpkgs-22_05` (please note
the dash vs. underscore).

IMHO it's not a good idea to not throw an error in that case and
probably leave end-users rather confused, so I implemented a small check
for that which basically checks whether `follows`-declaration from
overrides actually have corresponding inputs in the transitive flake.

In fact this was done by accident already in our own test-suite where
the removal of a `follows` was apparently forgotten[1].

Since the key of the `std::map` that holds the `overrides` is a vector
and we have to find the last element of each vector (i.e. the override)
this has to be done with a for loop in O(n) complexity with `n` being
the total amount of overrides (which shouldn't be that large though).

Please note that this doesn't work with nested expressions, i.e.

    inputs.fenix.inputs.nixpkgs.follows = "...";

which is a known problem[2].

For the expression demonstrated above, an error like this will be
thrown:

    error: sops-nix has a `follows'-declaration for a non-existant input nixpkgs_22_05!

[1] 2664a216e5
[2] https://github.com/NixOS/nix/issues/5790
2022-07-12 10:18:38 +02:00
Théophane Hufschmitt 21c443d4fd Test the tilde expansion for the flake completion
Also add a disabled test for when the `--override-input` flag comes
*before* the flake ref
2022-07-12 09:48:25 +02:00
Théophane Hufschmitt 07e14d3ef0 Harden the comparisons in the completion test
- Don't use `printf` for the expected result, but just use bash's `$' '`
  litteral strings
- Quote the `nix` call result
- Invert the order in the comparisons (just because it feels more
  natural)
2022-07-12 09:24:31 +02:00
Théophane Hufschmitt 260fb837de Fix the name of the completions test 2022-07-12 09:24:31 +02:00
Théophane Hufschmitt 32effccb51 Add some tests for the CLI completion 2022-07-12 09:24:31 +02:00
Gytis Ivaskevicius ba1fe85b65 Add builtins.traceVerbose
Co-Authored-By: Silvan Mosberger <contact@infinisil.com>

Add builtins.traceVerbose tests
2022-07-05 19:44:26 +03:00
Théophane Hufschmitt d63cd77549
Merge pull request #6664 from Ma27/innixshell-backwards-compat
nix-shell: restore backwards-compat with old nixpkgs
2022-07-05 15:57:20 +02:00
Théophane Hufschmitt b7eb4ac169
Merge pull request #6699 from tennox/better-flake-new-error-message
flakes: apply templates partially on conflicts
2022-06-29 18:21:07 +02:00
Théophane Hufschmitt 4c8210095e
Merge pull request #6233 from flox/nix-repl-flakes
Nix repl flakes
2022-06-29 17:59:22 +02:00
Théophane Hufschmitt 83f96e61a4 Add some test for nix flake init with conflicts 2022-06-29 17:08:03 +02:00
Eelco Dolstra 49c4a37efc
Merge pull request #6717 from edolstra/move-flake-registry
Fetch flake-registry.json from channels.nixos.org
2022-06-25 21:17:57 +02:00
Tom Bereknyei f801d70ba7 tests: enable ca-derivations for simple.nix in repl tests 2022-06-24 11:17:29 -04:00
Théophane Hufschmitt f06b6fa4fa
Merge pull request #6698 from tweag/forbid-tilde-paths-in-pure-mode
Forbid the tilde expansion in pure eval mode
2022-06-24 10:02:40 +02:00
Eelco Dolstra 4b6cc3da62 Fetch flake-registry.json from channels.nixos.org
Using fastly is slightly faster, provides some resilience due to a
high stale TTL, and allows some usage metrics.
2022-06-23 23:56:26 +02:00
Eelco Dolstra 8bbbb6e737 Enable/fix tests in nix-static
pkgsStatic is apparently considered a cross environment, so checkPhase
and installCheckPhase are disabled even when we ask for them.
2022-06-22 23:33:15 +02:00
Maximilian Bosch e94aa1f647
tests/nix-shell: more meaningful testcase 2022-06-22 22:35:48 +02:00
Théophane Hufschmitt 983efdbde4 Forbid the tilde expansion in pure eval mode
Fix #6684
2022-06-21 14:08:18 +02:00
Fishhh df21173b70
Add another test for --exclude in nix search 2022-06-20 19:21:45 +02:00
Tom Bereknyei 51268ceb79 Merge branch 'master' into nix-repl-flakes 2022-06-15 09:17:03 -04:00
Maximilian Bosch 98946e2d9c
nix-shell: restore backwards-compat with old nixpkgs
Basically an attempt to resume fixing #5543 for a breakage introduced
earlier[1]. Basically, when evaluating an older `nixpkgs` with
`nix-shell` the following error occurs:

    λ ma27 [~] → nix-shell -I nixpkgs=channel:nixos-18.03 -p nix
    error: anonymous function at /nix/store/zakqwc529rb6xcj8pwixjsxscvlx9fbi-source/pkgs/top-level/default.nix:20:1 called with unexpected argument 'inNixShell'

           at /nix/store/zakqwc529rb6xcj8pwixjsxscvlx9fbi-source/pkgs/top-level/impure.nix:82:1:

               81|
               82| import ./. (builtins.removeAttrs args [ "system" "platform" ] // {
                 | ^
               83|   inherit config overlays crossSystem;

This is a problem because one of the main selling points of Nix is that
you can evaluate any old Nix expression and still get the same result
(which also means that it *still evaluates*). In fact we're deprecating,
but not removing a lot of stuff for that reason such as unquoted URLs[2]
or `builtins.toPath`. However this property was essentially thrown away
here.

The change is rather simple: check if `inNixShell` is specified in the
formals of an auto-called function. This means that

    { inNixShell ? false }:
    builtins.trace inNixShell
      (with import <nixpkgs> { }; makeShell { name = "foo"; })

will show `trace: true` while

    args@{ ... }:
    builtins.trace args.inNixShell
      (with import <nixpkgs> { }; makeShell { name = "foo"; })

will throw the following error:

    error: attribute 'inNixShell' missing

This is explicitly needed because the function in
`pkgs/top-level/impure.nix` of e.g. NixOS 18.03 has an ellipsis[3], but
passes the attribute-set on to another lambda with formals that doesn't
have an ellipsis anymore (hence the error from above). This was perhaps
a mistake, but we can't fix it anymore. This also means that there's
AFAICS no proper way to check if the attr-set that's passed to the Nix
code via `EvalState::autoCallFunction` is eventually passed to a lambda
with formals where `inNixShell` is missing.

However, this fix comes with a certain price. Essentially every
`shell.nix` that assumes `inNixShell` to be passed to the formals even
without explicitly specifying it would break with this[4]. However I think
that this is ugly, but preferable:

* Nix 2.3 was declared stable by NixOS up until recently (well, it still
  is as long as 21.11 is alive), so most people might not have even
  noticed that feature.

* We're talking about a way shorter time-span with this change being
  in the wild, so the fallout should be smaller IMHO.

[1] 9d612c393a
[2] https://github.com/NixOS/rfcs/pull/45#issuecomment-488232537
[3] https://github.com/NixOS/nixpkgs/blob/release-18.03/pkgs/top-level/impure.nix#L75
[4] See e.g. the second expression in this commit-message or the changes
    for `tests/ca/nix-shell.sh`.
2022-06-13 23:29:28 +02:00
Eelco Dolstra 798efdf10b
Merge pull request #6027 from Ma27/pure-repl
repl: `--option pure-eval true` actually enables pure eval mode
2022-06-13 16:11:46 +02:00
Gabriel Fontes 9f6b4639c2
fix sourcehut brach/tag resolving regression
nixos/nix#6290 introduced a regex pattern to account for tags when
resolving sourcehut refs. nixos/nix#4638 reafactored the code,
accidentally treating the pattern as a regular string, causing all
non-HEAD ref resolving to break.

This fixes the regression and adds more test cases to avoid future
breakage.
2022-06-11 16:52:20 -03:00
Eelco Dolstra b2dea231cf
Merge pull request #6618 from afishhh/search-exclude
Add `-e`/`--exclude` flag to `nix search`
2022-06-07 19:01:36 +02:00
Fishhh 9ae22b1fde
Use grep -c instead of grep|wc -l in some nix search tests 2022-06-07 18:25:59 +02:00
Fishhh 0338cf5539
Add tests for --exclude flag in nix search 2022-06-07 18:25:52 +02:00
Eelco Dolstra 8977cdee6a
Merge pull request #6598 from cole-h/update-flake-for-real
flake: update to 22.05
2022-06-07 15:13:16 +02:00
Eelco Dolstra 0f8754cd30
Fix 22.05 eval warnings 2022-06-07 14:27:10 +02:00
Tom Bereknyei ffd41d1757 Merge branch 'master' into nix-repl-flakes 2022-06-02 16:58:35 -04:00
John Ericson b585548dfe Merge remote-tracking branch 'upstream/master' into indexed-store-path-outputs 2022-06-02 19:53:11 +00:00
Eelco Dolstra 24b3a500a7
Typo 2022-06-02 21:21:10 +02:00
Eelco Dolstra 2868acb4a5
tests/flakes.sh: Fix some ignored breakage 2022-06-02 21:19:16 +02:00
Maximilian Bosch 159b5815b5
repl: --option pure-eval true actually enables pure eval mode
To quote Eelco in #5867:

> Unfortunately we can't do
>
>     evalSettings.pureEval.setDefault(false);
>
> because then we have to do the same in main.cc (where
> pureEval is set to true), and that would allow pure-eval
> to be disabled globally from nix.conf.

Instead, a command should specify that it should be impure by
default. Then, `evalSettings.pureEval` will be set to `false;` unless
it's overridden by e.g. a CLI flag.

In that case it's IMHO OK to be (theoretically) able to override
`pure-eval` via `nix.conf` because it doesn't have an effect on commands
where `forceImpureByDefault` returns `false` (i.e. everything where pure
eval actually matters).

Closes #5867
2022-05-31 22:20:11 +02:00
Cole Helbling dbf0d3a150 tests/nss-preload: move nix-fetch binding 2022-05-31 12:14:34 -07:00
Eelco Dolstra 27ebb97d0a
Handle EOFs in string literals correctly
We can't return a STR token without setting a valid StringToken,
otherwise the parser will crash.

Fixes #6562.
2022-05-25 17:58:13 +02:00
tomberek 8c3939af14
Apply suggestions from code review
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-05-20 12:09:41 -04:00
Tom Bereknyei 7a04fb1c56 repl: add repl-flake experimental feature for gating 2022-05-20 08:20:00 -04:00
Tom Bereknyei 0053dab43f repl: fix tests to run on any testing store 2022-05-20 08:03:41 -04:00
Théophane Hufschmitt 938150472d Add some tests for the new REPL cli
- Test that without the XP feature things work as before
- Test that with or without the XP feature `--file file` works
- Test that with XP feature passing a flakeref works
- Test `:reload` with a flake
2022-05-20 07:42:06 -04:00
Tony Olagbaiye 5b8c1deb18 fetchTree: Allow fetching plain files
Add a new `file` fetcher type, which will fetch a plain file over
http(s), or from the local file.

Because plain `http(s)://` or `file://` urls can already correspond to
`tarball` inputs (if the path ends-up with a know archive extension),
the URL parsing logic is a bit convuluted in that:

- {http,https,file}:// urls will be interpreted as either a tarball or a
  file input, depending on the extensions of the path part (so
  `https://foo.com/bar` will be a `file` input and
  `https://foo.com/bar.tar.gz` as a `tarball` input)
- `file+{something}://` urls will be interpreted as `file` urls (with
  the `file+` part removed)
- `tarball+{something}://` urls will be interpreted as `tarball` urls (with
  the `tarball+` part removed)

Fix #3785

Co-Authored-By: Tony Olagbaiye <me@fron.io>
2022-05-19 18:24:49 +02:00
Théophane Hufschmitt 78dc64ec1e
Merge pull request #6544 from ncfavier/getFlake-no-write-lock-file
Do not attempt to write a lock file in builtins.getFlake
2022-05-18 16:41:13 +02:00
Naïm Favier 169384abb2
Do not attempt to write a lock file in builtins.getFlake
Fixes https://github.com/NixOS/nix/issues/6541
2022-05-18 15:45:06 +02:00
Eli Kogan-Wang be2b19041e Integrate review changes 2022-05-13 22:02:28 +02:00
John Ericson 49ad315c03 Use ^ not ! in indexed store derivations installable syntax
Match the other syntax that was recently added
2022-05-12 20:10:02 +00:00
John Ericson b18720ee17 Merge remote-tracking branch 'upstream/master' into indexed-store-path-outputs 2022-05-12 19:13:33 +00:00
Eli Kogan-Wang aefc6c4f41 Add priority for nix profile install 2022-05-11 12:16:35 +02:00
Eelco Dolstra f4102de84b
Merge pull request #6489 from alexshpilkin/fix-6488
nix: Add forgotten null check in AttrCursor::getListOfStrings()
2022-05-05 16:16:56 +02:00
Théophane Hufschmitt 4c5aa8520c Make sure that nix build works in --impure mode
Regression test for <https://github.com/NixOS/nix/issues/6488>
2022-05-05 14:53:59 +02:00
thkoch2001 7388cd55bf Change json example to be original
Closes: #3391
2022-05-05 12:39:23 +03:00
Eelco Dolstra a3c6c5b1c7 nix profile: Support overriding outputs 2022-05-03 15:00:34 +02:00
Eelco Dolstra 4a79cba511 Allow selecting derivation outputs using 'installable!outputs'
E.g. 'nixpkgs#glibc^dev,static' or 'nixpkgs#glibc^*'.
2022-05-03 13:43:52 +02:00
Eelco Dolstra 404c222444
Merge pull request #6426 from edolstra/respect-outputs-to-install
nix: Respect meta.outputsToInstall, and use all outputs by default
2022-05-03 13:43:22 +02:00
Théophane Hufschmitt cae7473884
Merge pull request #6465 from layus/fix-test-crashes
tests: Distinguish crashes from expected failures
2022-05-03 11:11:37 +02:00
Guillaume Maudoux 275f8561eb tests/lang: Distinguish crashes from expected failures 2022-05-02 15:12:50 +02:00
Guillaume Maudoux 4845886bed Revert "tests: Distinguish crashes from expected failures"
This reverts commit 143b73f52d.
2022-05-02 14:22:00 +02:00
Eelco Dolstra a26be9f3b8
Merge pull request #4638 from orbekk/read_head
Resolve reference for remote git repositories (makes fetchGit work with non-'master' branch)
2022-05-02 13:31:30 +02:00
Eelco Dolstra dde71899dd tests: Don't create tests/result
https://hydra.nixos.org/log/lns780srkka4dv7r69mn4zfy6fdij4yr-nix-2.9.0pre20220428_4bb111c.drv
2022-05-02 09:38:51 +02:00
Kjetil Orbekk 05a3fbac5a Test fetchGit with non-'master' remote repo 2022-04-29 18:42:31 -04:00
Kjetil Orbekk 401e60f289 Resolve reference for remote repository
Resolves the HEAD reference from the remote repository instead
of assuming "master".
2022-04-29 18:42:28 -04:00
Guillaume Maudoux 143b73f52d tests: Distinguish crashes from expected failures 2022-04-29 11:14:08 +02:00
Eelco Dolstra 4a9623b129 Fix passing $OUT_PATHS to the post-build hook
Fixes #6446.
2022-04-28 13:36:01 +02:00
Eelco Dolstra 6e0a2b971b Add a test for outputsToInstall 2022-04-26 17:17:27 +02:00
Eelco Dolstra 1ddabe1a01 nix: Respect meta.outputsToInstall, and use all outputs by default
'nix profile install' will now install all outputs listed in the
package's meta.outputsToInstall attribute, or all outputs if that
attribute doesn't exist. This makes it behave consistently with
nix-env. Fixes #6385.

Furthermore, for consistency, all other 'nix' commands do this as
well. E.g. 'nix build' will build and symlink the outputs in
meta.outputsToInstall, defaulting to all outputs. Previously, it only
built/symlinked the first output. Note that this means that selecting
a specific output using attrpath selection (e.g. 'nix build
nixpkgs#libxml2.dev') no longer works. A subsequent PR will add a way
to specify the desired outputs explicitly.
2022-04-26 17:17:27 +02:00
Théophane Hufschmitt be28603dca Merge remote-tracking branch 'origin/master' into nixbuildaddprintstorepaths 2022-04-22 11:11:01 +02:00
Théophane Hufschmitt c4ffc8e2f8
Merge pull request #6218 from pennae/pos-symbol-tables
reduce the size of Attr from 3 pointers to 2 on 64 bit machines
2022-04-22 10:28:06 +02:00
Théophane Hufschmitt 35ca5fdf91
Merge pull request #6436 from flox/tofile_allow
fix: builtins.toFile adds path to allowedPaths
2022-04-22 08:50:54 +02:00
Tom Bereknyei f25112d383 fix: builtins.toFile adds path to allowedPaths
The produced path is then allowed be imported or utilized elsewhere:
```
assert (43 == import (builtins.toFile "source" "43")); "good"
```

This will still fail on write-only stores.
2022-04-21 16:41:37 -04:00
pennae 6526d1676b replace most Pos objects/ptrs with indexes into a position table
Pos objects are somewhat wasteful as they duplicate the origin file name and
input type for each object. on files that produce more than one Pos when parsed
this a sizeable waste of memory (one pointer per Pos). the same goes for
ptr<Pos> on 64 bit machines: parsing enough source to require 8 bytes to locate
a position would need at least 8GB of input and 64GB of expression memory. it's
not likely that we'll hit that any time soon, so we can use a uint32_t index to
locate positions instead.
2022-04-21 21:46:06 +02:00
Sergei Trofimovich b6e59d7137 tests: remove 'ca-references' feature
The feature was ctabilized in d589a6aa8a.
2022-04-21 10:06:39 +02:00
regnat 74d6782a6a Disable the selfref-gc test when the daemon is too old 2022-04-21 10:06:39 +02:00
Sergei Trofimovich 6ada496311 nix: add (failing) selfreference test for multiple realizations
The test illustrates failure in issue #5320. Here derivation and
it's built input have identical CA sotre path. As a result we generate
extraneout reference to build input:

    $ make installcheck
    ...
    ran test tests/selfref-gc.sh... [PASS]
    ran test tests/ca/selfref-gc.sh... [FAIL]
    ...
        deleting '/tmp/.../tests/ca/selfref-gc/store/iqciq1mpg5hc7p6a52fp2bjxbyc9av0v-selfref-gc'
        deleting '/tmp/...tests/ca/selfref-gc/store/zh0kwpnirw3qbv6dl1ckr1y0kd5aw6ax-selfref-gc.drv'
        error: executing SQLite statement
          'delete from ValidPaths where path = '/tmp/.../tests/ca/selfref-gc/store/fsjq0k146r85lsh01l0icl30rnhv7z72-selfref-gc';':
            constraint failed (in '/tmp/.../tests/ca/selfref-gc/var/nix/db/db.sqlite')
2022-04-21 10:06:39 +02:00
Artturin 51cfea8bb0 nix build: add --print-out-paths flag
has the same functionality as default nix-build

$ nix-build . -A "bash" -A "bash.dev" -A "tinycc"
/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12
/nix/store/c49i1ggnr5cc8gxmk9xm0cn961z104dn-bash-5.1-p12-dev
/nix/store/dbapb08862ajgaax3621fz8hly9fdah3-tcc-0.9.27+date=2022-01-11

$ nix-build . -A "bash"
/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12

$ $HOME/nixgits/nix/result/bin/nix build "nixpkgs#bash" "nixpkgs#bash.dev" "nixpkgs#tinycc" --print-out-paths
/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12
/nix/store/c49i1ggnr5cc8gxmk9xm0cn961z104dn-bash-5.1-p12-dev
/nix/store/dbapb08862ajgaax3621fz8hly9fdah3-tcc-0.9.27+date=2022-01-11

$ $HOME/nixgits/nix/result/bin/nix build "nixpkgs#bash" --print-out-paths
/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12
2022-04-20 19:35:46 +03:00
John Ericson 08b8657978 Merge branch 'path-info' into ca-drv-exotic 2022-04-19 22:39:57 +00:00
mei (ckie) 0e2b01b14e
nix repl: make symlinks with the :bl command
Requested by ppepino on the Matrix:
https://matrix.to/#/!KqkRjyTEzAGRiZFBYT:nixos.org/$Tb32BS3rVE2BSULAX4sPm0h6CDewX2hClOTGzTC7gwM?via=nixos.org&via=matrix.org&via=nixos.dev

This adds a new command, :bl, which works like :b but also creates
a GC root symlink to the various derivation outputs.

ckie@cookiemonster ~/git/nix -> ./outputs/out/bin/nix repl
Welcome to Nix 2.6.0. Type :? for help.

nix-repl> :l <nixpkgs>
Added 16118 variables.

nix-repl> :b runCommand "hello" {} "echo hi > $out"

This derivation produced the following outputs:
  ./repl-result-out -> /nix/store/kidqq2acdpi05c4a9mlbg2baikmzik44-hello
[1 built, 0.0 MiB DL]
ckie@cookiemonster ~/git/nix -> cat ./repl-result-out
hi
2022-04-20 00:20:29 +03:00
John Ericson 6b61d7722d Merge remote-tracking branch 'upstream/master' into indexed-store-path-outputs 2022-04-19 20:30:12 +00:00
Eelco Dolstra a3c843e635 Fix 'nix fmt' test 2022-04-19 21:47:13 +02:00
Eelco Dolstra c9e58aa5ff Require formatters to be packages
Because of 9b41239d8f, a formatter can
no longer be a package *or* an app. So let's require it to be a
package for now.
2022-04-19 20:48:13 +02:00
Eelco Dolstra 48a467f2b9 Merge branch 'issue-6075' of https://github.com/kamadorueda/nix 2022-04-19 20:21:32 +02:00
John Ericson 41e755bee4 Merge remote-tracking branch 'upstream/master' into indexed-store-path-outputs 2022-04-19 15:14:06 +00:00
Eelco Dolstra 2016b7142a Fix compilation, style fixes 2022-04-19 13:41:32 +02:00
Eelco Dolstra b529a41814 Merge branch 'make-flake-show-more-lenient-on-apps' of https://github.com/flox/nix 2022-04-19 13:41:24 +02:00
Tom Bereknyei 9b41239d8f fix: ensure apps are apps and packages are packages 2022-04-14 23:57:52 -04:00
Daniel Pauls 38125a47ab Test fetchMercurial with path containing a . segment 2022-04-09 23:39:00 +02:00
John Ericson 5c1f2e0fb1 Merge remote-tracking branch 'upstream/master' into indexed-store-path-outputs 2022-04-07 19:20:27 +00:00
Théophane Hufschmitt 305d3a0ec3 Test fetchgit with path containing a . segment 2022-04-07 17:31:12 +02:00
Eelco Dolstra 589f6f267b fetchClosure: Don't allow URL query parameters
Allowing this is a potential security hole, since it allows the user
to specify parameters like 'local-nar-cache'.
2022-04-06 11:52:51 +02:00
Théophane Hufschmitt 5abe3f4aa6 Allow welcomeText when checking a flake template
Fix https://github.com/NixOS/nix/issues/6321
2022-04-05 11:03:43 +02:00
Eelco Dolstra fdfe737867 Fix handling of outputHash when outputHashAlgo is not specified
https://hydra.nixos.org/build/171351131
2022-04-01 12:40:49 +02:00
John Ericson 6951b26ed0 Require (new) computed-derivations experimental feature for ! installable 2022-04-01 03:57:51 +00:00
John Ericson 9c6be01d5f Merge remote-tracking branch 'upstream/master' into indexed-store-path-outputs 2022-04-01 03:48:22 +00:00
Eelco Dolstra 6377442c98 tests/impure-derivations.sh: Ensure that inputAddressed build fails 2022-03-31 17:38:15 +02:00
Eelco Dolstra 7537097284 Provide default values for outputHashAlgo and outputHashMode 2022-03-31 16:56:44 +02:00
Eelco Dolstra 162beb2595 Fix test 2022-03-31 13:43:20 +02:00
Eelco Dolstra b2ae922747 tests/impure-derivations.sh: Restart daemon 2022-03-31 13:43:20 +02:00
Eelco Dolstra 18935e8b9f Support fixed-output derivations depending on impure derivations 2022-03-31 13:43:20 +02:00
Eelco Dolstra 5cd72598fe Add support for impure derivations
Impure derivations are derivations that can produce a different result
every time they're built. Example:

  stdenv.mkDerivation {
    name = "impure";
    __impure = true; # marks this derivation as impure
    outputHashAlgo = "sha256";
    outputHashMode = "recursive";
    buildCommand = "date > $out";
  };

Some important characteristics:

* This requires the 'impure-derivations' experimental feature.

* Impure derivations are not "cached". Thus, running "nix-build" on
  the example above multiple times will cause a rebuild every time.

* They are implemented similar to CA derivations, i.e. the output is
  moved to a content-addressed path in the store. The difference is
  that we don't register a realisation in the Nix database.

* Pure derivations are not allowed to depend on impure derivations. In
  the future fixed-output derivations will be allowed to depend on
  impure derivations, thus forming an "impurity barrier" in the
  dependency graph.

* When sandboxing is enabled, impure derivations can access the
  network in the same way as fixed-output derivations. In relaxed
  sandboxing mode, they can access the local filesystem.
2022-03-31 13:43:20 +02:00
Théophane Hufschmitt 8dee15cd31 Don’t create a file in the worktree in the fetchPath test 2022-03-30 11:42:47 +02:00
John Ericson ff2a8ccfe1 Merge branch 'path-info' into ca-drv-exotic 2022-03-25 19:40:52 +00:00
John Ericson 0966532dc1 Merge remote-tracking branch 'upstream' into indexed-store-path-outputs
Co-Authored-By: Tom Bereknyei <tomberek@gmail.com>
2022-03-25 16:48:50 +00:00
Eelco Dolstra 16cf1e6089
Merge pull request #6311 from edolstra/return-wanted-paths
Make buildPathsWithResults() only return info on wanted outputs
2022-03-25 15:44:39 +01:00
Eelco Dolstra 187dc080a2 tests/build.sh: Test that 'nix build' only prints wanted outputs 2022-03-24 23:36:14 +01:00
Maximilian Bosch d02e34ef06
Implement regression test for empty logs loaded via nix log 2022-03-24 22:31:52 +01:00
Eelco Dolstra f902f3c2cb Add experimental feature 'fetch-closure' 2022-03-24 21:33:33 +01:00
Eelco Dolstra 28186b7044 Add a test for fetchClosure and 'nix store make-content-addressed' 2022-03-24 21:33:33 +01:00
Eelco Dolstra a0259a21a4 Don't hide repeated values while generating manifest.nix
Fixes #6243.
2022-03-22 13:18:56 +01:00
Gabriel Fontes 31544b93ff
Fix sourcehut integration test
The new implementation relies on tab separting the hash and ref (this is how sourcehut does it). This fixes the integration test to use a tab instead of a space.
2022-03-19 11:38:45 -03:00
Eelco Dolstra 6afc361798
Merge pull request #6277 from thufschmitt/ca/nix-build-dry-run
Fix `nix build --dry-run` with CA derivations
2022-03-17 22:23:15 +01:00
Théophane Hufschmitt 7117053457
Merge pull request #6270 from Artturin/stdineval
nix: allow using --file - to read from stdin
2022-03-17 14:35:01 +01:00
Théophane Hufschmitt 3fc4c612fb Fix nix build --dry-run with CA derivations
Don’t try and assume that we know the output paths when we’ve just built
with `--dry-run`. Instead make `--dry-run` follow a different code path
that won’t assume the knowledge of the output paths at all.

Fix #6275
2022-03-17 11:55:19 +01:00
Théophane Hufschmitt a0b517de57
Merge pull request #6242 from ncfavier/print-output-names
nix-env: always print output names in JSON and XML
2022-03-17 10:55:22 +01:00
Théophane Hufschmitt c20e07763d
Add some tests for nix-env -q --json 2022-03-16 21:33:38 +01:00
Artturin 4f8ad41d4e add tests for nix eval and nix-instantiate 2022-03-16 21:01:51 +02:00
Théophane Hufschmitt 3cea6f569e
Fix the date in the comment of fetchPath’s test
Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>
2022-03-16 08:56:01 +01:00
regnat 2d5c43f210 Fix the tests on 32bits machines
year 2222 is too much for a 32 bit timestamp. So replace it by something
smaller
2022-03-15 21:05:01 +01:00
Maximilian Bosch 975bade7f0
Implement simple test for path-fetcher setting a correct lastModifiedDate
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-03-15 12:55:32 +01:00
Kevin Amado 2191dab657
nix-fmt: add command 2022-03-11 10:00:19 -05:00
Eelco Dolstra aee56e0f89 Merge remote-tracking branch 'origin/eval-suggestions' 2022-03-11 12:02:26 +01:00
Théophane Hufschmitt 7ebd6f1093
Merge pull request #5758 from mschwaig/fix-git-workspace-dirty-detection
git fetcher: fix detection of dirty git workspaces
2022-03-11 09:35:01 +01:00
John Ericson e5c42bba9b Merge remote-tracking branch 'upstream/master' into indexed-store-path-outputs 2022-03-10 17:10:15 +00:00
John Ericson 938650700f Merge branch 'path-info' into ca-drv-exotic 2022-03-10 16:20:01 +00:00
regnat f6078e474d Also display some suggestions for invalid formal arguments
```console
$ nix eval --expr '({ foo ? 1 }: foo) { fob = 2; }'
error: anonymous function at (string):1:2 called with unexpected argument 'fob'

       at «string»:1:1:

            1| ({ foo ? 1 }: foo) { fob = 2; }
             | ^
       Did you mean foo?
```

Not that because Nix will first check for _missing_ arguments before
checking for extra arguments, `({ foo }: foo) { fob = 1; }` will
complain about the missing `foo` argument (rather than extra `fob`) and
so won’t display a suggestion.
2022-03-08 16:40:22 +01:00
regnat 0c6e46e349 Add some suggestions to the evaluator
Make the evaluator show some suggestions when trying to access an
invalid field from an attrset.

```console
$ nix eval --expr '{ foo = 1; }.foa'
error: attribute 'foa' missing

       at «string»:1:1:

            1| { foo = 1; }.foa
             | ^
       Did you mean foo?
```
2022-03-08 06:21:45 +01:00
Eelco Dolstra 30ddd37873 Merge branch 'cli-suggestions' of https://github.com/thufschmitt/nix 2022-03-07 19:47:45 +01:00
regnat 91635206c0 Add some end-to-end tests for the suggestions 2022-03-07 10:09:10 +01:00
Théophane Hufschmitt a2ace54fe4
Merge pull request #6029 from Ma27/nix-log-ssh-ng
ssh-ng: also store build logs to make them accessible by `nix log`
2022-03-07 09:51:40 +01:00
Théophane Hufschmitt 860016bcbf
Explain why the log tests are disabled for CA derivations 2022-03-07 09:15:34 +01:00
Maximilian Bosch 697d1dac01
tests: grep for string in nix log for remote-builds 2022-03-04 16:58:27 +01:00
Eelco Dolstra d4538034b7 nix profile test: Restart daemon
Fixes

  nix-daemon: src/libstore/sqlite.cc:97: nix::SQLiteStmt::Use::Use(nix::SQLiteStmt&): Assertion `stmt.stmt' failed.

which happens because the daemon doesn't properly handle the case
where ca-derivations isn't enabled at daemon startup.
2022-03-02 23:08:49 +01:00
Eelco Dolstra 3a3821bcd7 Remove obsolete todo 2022-03-02 22:22:55 +01:00
Eelco Dolstra b0d65b3d11 Silence kill output 2022-03-02 22:22:23 +01:00
Eelco Dolstra d2586188fe tests/common.sh.in: Add enableFeatures helper 2022-03-02 21:48:25 +01:00
Eelco Dolstra f9375778ae nix profile: Add a test for non-flake packages 2022-03-02 20:56:40 +01:00
Eelco Dolstra 161f798aa1 nix profile: Support CA derivations 2022-03-02 20:38:51 +01:00
Eelco Dolstra 5850fd17b4 Add basic tests for 'nix profile'
Fixes #6193.
2022-03-02 14:40:46 +01:00
Eelco Dolstra c10865a46e tests: Rename nix-profile.sh -> bash-profile.sh 2022-03-02 11:21:00 +01:00
Robert Hensing 158280d8e9 fetchTree: Do not re-fetch paths already present 2022-02-28 21:37:49 +01:00
John Ericson c863e5f338 Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2022-02-28 18:29:33 +00:00
Maximilian Bosch 287642f132
tests: implement test for nix log with ssh-ng:// builds
A few notes:

* The `echo hi` is needed to make sure that a file that can be read by
  `nix log` is properly created (i.e. some output is needed). This is
  known and to be fixed in #6051.
* We explicitly ignore the floating-CA case here: the `$out` of `input3`
  depends on `$out` of `input2`. This means that there are actually two
  derivations - I assume that this is because at eval time (i.e.
  `nix-instantiate -A`) the hash of `input2` isn't known yet and the
  other .drv is created as soon as `input2` was built. This is another
  issue on its own, so we ignore the case here explicitly.
2022-02-28 17:27:52 +01:00
regnat b8f8aef9d3 tests: Fix the start of the daemon
- Make sure that it starts even without the `nix-command` xp feature
- Fail if it doesn’t manage to start

This fixes a 30s wait for every test in `init.sh` as the daemon couldn’t
start, but the code was just waiting 30s and continuing as if everything
was all right.
2022-02-28 17:00:31 +01:00
Eelco Dolstra 59683733f5
Merge pull request #6089 from edolstra/dot-default
Replace defaultBla.$system with bla.$system.default
2022-02-24 21:16:58 +01:00
Eelco Dolstra 7ec244aec2
Merge pull request #6159 from NixOS/more-eager-daemon-polling-in-tests
testS: poll more eagerly for the daemon start/stop
2022-02-24 17:54:09 +01:00
regnat 9c470cb969 testS: poll more eagerly for the daemon start/stop
Polling every 1 second means that even the simplest test takes at least
2 seconds. We can reasonably poll 1/10 of that to make things much
  quicker (esp. given that most of the time 0.1s is enough for the
  daemon to be started or stopped)
2022-02-24 15:00:23 +01:00
regnat 42766f8924 Sort the tests by wall time
The tests are scheduled in the order they appear, so running the long
ones first slightly improves the scheduling.

On my machine, this decreases the time of `make install` from 40s to 36s
2022-02-24 14:32:31 +01:00
Gabriel Fontes 770f3af31d
add sourcehut integration test 2022-02-23 11:58:09 -03:00
Eelco Dolstra 162fbe31ff Replace defaultBla.$system with bla.$system.default
This also simplifies some InstallableFlake logic and fixes 'nix
bundle' parsing its installable twice.

Fixes #5532.
2022-02-22 11:47:41 +01:00
Martin Schwaighofer 9504445cab git fetcher: distinguish errors more precisely 2022-02-19 22:58:23 +01:00
Martin Schwaighofer c7e527b82b git fetcher: invoke diff instead of diff-index
diff-index operates on the view that git has of the working tree,
which might be outdated. The higher-level diff command does this
automatically. This change also adds handling for submodules.

fixes #4140

Alternative fixes would be invoking update-index before diff-index or
matching more closely what require_clean_work_tree from git-sh-setup.sh
does, but both those options make it more difficult to reason about
correctness.
2022-02-19 22:58:23 +01:00
Martin Schwaighofer 0bfa0cdea1 git fetcher: improve check for valid repository
The .git/refs/heads directory might be empty for a valid
usable git repository. This often happens in CI environments,
which might only fetch commits, not branches.
Therefore instead we let git itself check if HEAD points to
something that looks like a commit.

fixes #5302
2022-02-19 22:58:22 +01:00
Eelco Dolstra 93293fc66b
Merge pull request #6042 from pennae/fix-repl-a
fix nix repl not overriding existing bindings in :a
2022-02-06 13:17:58 +01:00
pennae 1daf1babf9 fix nix repl not overriding existing bindings in :a
previously :a would override old bindings of a name with new values if the added
set contained names that were already bound. in nix 2.6 this doesn't happen any
more, which is potentially confusing.

fixes #6041
2022-02-04 15:27:59 +01:00
tomberek 12ff354d01
Merge branch 'master' into bundler_drv 2022-02-03 02:39:18 -05:00
Eelco Dolstra fcb33440b6
Merge pull request #6036 from tweag/balsoft/and-yet-another-follows-bugfix
Flake follows: resolve all follows to absolute
2022-02-02 22:52:08 +01:00
Alexander Bantyev 169ea0b83f
Flake follows: resolve all follows to absolute
It's not possible in general to know in computeLocks, relative to
which path the follows was intended to be. So, we always resolve
follows to their absolute states when we encounter them (which can
either be in parseFlakeInput or computeLocks' fake input population).

Fixes https://github.com/NixOS/nix/issues/6013
Fixes https://github.com/NixOS/nix/issues/5609
Fixes https://github.com/NixOS/nix/issues/5697 (again)
2022-02-02 21:41:45 +03:00
Eelco Dolstra 17e3f353df Merge branch 'parser-improvements' of https://github.com/pennae/nix 2022-02-02 12:45:44 +01:00
tomberek 2bf96bd9f2
Merge branch 'master' into bundler_drv 2022-01-28 10:18:29 -05:00
Tom Bereknyei 73e82ae954 bundler: tests various combinations of referring to installables 2022-01-28 10:17:51 -05:00
Tom Bereknyei 4ebc50d92e bundler: revert default behavior to use defaultApp
Bundlers are now responsible for correctly handling their inputs which
are no longer constrained to be (Drv->Drv)->Drv->Drv, but can be of
type (attrset->Drv)->attrset->Drv.
2022-01-28 09:56:58 -05:00
regnat fcdc60ed22 Don’t require NIX_PATH entries to be valid paths
It’s totally valid to have entries in `NIX_PATH` that aren’t valid paths
(they can even be arbitrary urls or `channel:<channel-name>`).

Fix #5998 and #5980
2022-01-27 16:26:39 +01:00
regnat 3dc1418216 Fix the store ping test 2022-01-26 14:15:03 +01:00
regnat d139474f48 Add some tests for nix store ping
Always good to have :)
2022-01-26 11:01:25 +01:00
regnat f113ea6c73 Fix parsing of variable names that are a suffix of '__curPos'
Follow-up from #5969
Fix #5982
2022-01-25 10:49:27 +01:00
Tom Bereknyei 93299efc7c bundler: add tests and change defaults to use a derivation 2022-01-25 03:39:18 -05:00
Eelco Dolstra bed8270c0c Fix parsing of variable names that are a prefix of '__curPos'
Fixes

  $ nix-instantiate --parse -E 'x: with x; _'
  (x: (with x; __curPos))
2022-01-24 15:18:18 +01:00
Eelco Dolstra 3ff6186af1 Add a test for nix-assertion failures in nix-env 2022-01-24 14:49:32 +01:00
Théophane Hufschmitt 4530574363
Merge pull request #5945 from afishhh/master
Make `nix search` highlight all regexes and matches
2022-01-24 13:02:51 +01:00
Eelco Dolstra 5c3b6fbe00
Merge pull request #5963 from FlorianFranzen/test-flake-git
tests: skip flake search test if no git is present
2022-01-24 12:45:49 +01:00
Eelco Dolstra 076945c808
Merge pull request #5875 from hercules-ci/fix-large-drv-field-stack-overflow
Fix segfault or stack overflow caused by large derivation fields
2022-01-24 12:05:30 +01:00
Florian Franzen 12578ba672 tests: skip flake search test if no git is present 2022-01-22 20:13:16 +01:00
Eelco Dolstra c7223db871
Merge pull request #5942 from NixOS/5912-quieter-nix-why-depends
Make `nix why-depends` quieter by default
2022-01-21 10:18:28 +01:00
Hubert Głuchowski b03fe13b5b
Add some tests for multiple regex/match highlighting in nix search 2022-01-20 18:18:07 +01:00
pennae 7d4cc5515c defer formals duplicate check for incresed efficiency all round
if we defer the duplicate argument check for lambda formals we can use more
efficient data structures for the formals set, and we can get rid of the
duplication of formals names to boot. instead of a list of formals we've seen
and a set of names we'll keep a vector instead and run a sort+dupcheck step
before moving the parsed formals into a newly created lambda. this improves
performance on search and rebuild by ~1%, pure parsing gains more (about 4%).

this does reorder lambda arguments in the xml output, but the output is still
stable. this shouldn't be a problem since argument order is not semantically
important anyway.

 before

  nix search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      8.550 s ±  0.060 s    [User: 6.470 s, System: 1.664 s]
    Range (min … max):    8.435 s …  8.666 s    20 runs

  nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     346.7 ms ±   2.1 ms    [User: 312.4 ms, System: 34.2 ms]
    Range (min … max):   343.8 ms … 353.4 ms    20 runs

  nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.720 s ±  0.031 s    [User: 2.415 s, System: 0.231 s]
    Range (min … max):    2.662 s …  2.780 s    20 runs

 after

  nix search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      8.462 s ±  0.063 s    [User: 6.398 s, System: 1.661 s]
    Range (min … max):    8.339 s …  8.542 s    20 runs

  nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     329.1 ms ±   1.4 ms    [User: 296.8 ms, System: 32.3 ms]
    Range (min … max):   326.1 ms … 330.8 ms    20 runs

  nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.687 s ±  0.035 s    [User: 2.392 s, System: 0.228 s]
    Range (min … max):    2.626 s …  2.754 s    20 runs
2022-01-19 17:07:29 +01:00
Robert Hensing d038a67bd3 Fix segfault or stack overflow caused by large derivation fields
This removes a dynamic stack allocation, making the derivation
unparsing logic robust against overflows when large strings are
added to a derivation.
Overflow behavior depends on the platform and stack configuration.

For instance, x86_64-linux/glibc behaves as (somewhat) expected:

$ (ulimit -s 20000; nix-instantiate tests/lang/eval-okay-big-derivation-attr.nix)
error: stack overflow (possible infinite recursion)

$ (ulimit -s 40000; nix-instantiate tests/lang/eval-okay-big-derivation-attr.nix)
error: expression does not evaluate to a derivation (or a set or list of those)

However, on aarch64-darwin:

$ nix-instantiate big-attr.nix                                                                                                                                                                                                                                                       ~
zsh: segmentation fault  nix-instantiate big-attr.nix

This indicates a slight flaw in the single stack protection page
approach that is not encountered with normal stack frames.
2022-01-19 15:21:56 +01:00
regnat dd7c2e0695 Make nix why-depends quieter by default
Unless `--precise` is passed, make `nix why-depends` only show the
dependencies between the store paths, without introspecting them to
find the actual references.

This also makes it ~3x faster
2022-01-19 14:24:14 +01:00
Eelco Dolstra 5fe1ec8a05 Add a test for 'nix store copy-log' and 'nix log' 2022-01-18 17:30:50 +01:00
regnat 2ad2678c0b Add a simple test for nix why-depends 2022-01-18 16:54:53 +01:00
Théophane Hufschmitt dc4730ee94
Merge pull request #5720 from tomberek/flake_search
flakes: search up to git or filesystem boundary
2022-01-18 10:59:12 +01:00
Alexander Bantyev b9f5dccdbe
Check that we don't search past a git repo 2022-01-14 18:03:47 +03:00
Alexander Bantyev e3690ab393
Add more tests for flake upward searching 2022-01-14 17:21:18 +03:00
Eelco Dolstra 8a446aff75
Merge pull request #5898 from layus/repair-path-links
Make --repair-path also repair corrupt optimised links
2022-01-11 14:14:44 +01:00
Guillaume Maudoux e9a4abdb5d Make --repair-path also repair corrupt optimised links
There already existed a smoke test for the link content length,
but it appears that there exists some corruptions pernicious enough
to replace the file content with zeros, and keeping the same length.

--repair-path now goes as far as checking the content of the link,
making it true to its name and actually repairing the path for such
coruption cases.
2022-01-11 11:57:45 +01:00
Eelco Dolstra f39d94a55b Merge branch 'readfile' of https://github.com/abbradar/nix 2022-01-10 16:55:28 +01:00
Nikolay Amiantov 120ca245d1 Add test for readFile keeping context 2022-01-09 13:42:36 +03:00
Eelco Dolstra 3ec02c6743
Merge pull request #5546 from NixOS/allow-paths-in-flake-local-options
allow paths in flake local options
2022-01-06 21:55:20 +01:00
Eelco Dolstra 263a8d293c Remove non-method mk<X> functions 2022-01-04 18:40:39 +01:00
pennae 00c993f48b add zipAttrsWith primop
nixpkgs can save a good bit of eval memory with this primop. zipAttrsWith is
used quite a bit around nixpkgs (eg in the form of recursiveUpdate), but the
most costly application for this primop is in the module system. it improves
the implementation of zipAttrsWith from nixpkgs by not checking an attribute
multiple times if it occurs more than once in the input list, allocates less
values and set elements, and just avoids many a temporary object in general.

nixpkgs has a more generic version of this operation, zipAttrsWithNames, but
this version is only used once so isn't suitable for being the base of a new
primop. if it were to be used more we should add a second primop instead.
2022-01-03 21:05:53 +01:00
Eelco Dolstra 96d08fcd66
Merge pull request #5839 from tweag/balsoft/yet-another-follows-bugfix
flake.cc: computeLocks: Only verify overrides when they could change
2022-01-03 20:51:58 +01:00
Alexander Bantyev 2dc29e0d93
Add a test that nix repl --show-trace actually shows the trace 2021-12-28 15:54:46 +03:00
Alexander Bantyev 2664a216e5
flake.cc: computeLocks: Only verify overrides when they could change
When we check for disappeared overrides, we can get "false positives"
for follows and overrides which are defined in the dependencies of the
flake we are locking, since they are not parsed by
parseFlakeInputs. However, at that point we already know that the
overrides couldn't have possible been changed if the input itself
hasn't changed (since we check that oldLock->originalRef == *input.ref
for the input's parent). So, to prevent this, only perform this check
when it was possible that the flake changed (e.g. the flake we're
locking, or a new input, or the input has changed and mustRefetch ==
true).
2021-12-28 15:30:06 +03:00
tomberek f404189368
Merge branch 'master' into flake_search 2021-12-23 15:22:52 -05:00
regnat dc89dfa7b3 Properly return false on builtins.pathExists /someNonAllowedPath
Follow-up from https://github.com/NixOS/nix/pull/5807 to fix https://github.com/NixOS/nix/pull/5807#issuecomment-1000135394
2021-12-23 10:49:33 +01:00
regnat d90f9d4b99 Fix IFD with CA derivations
Rewrite the string taken by the IFD-like primops to contain the actual
output paths of the derivations rather than the placeholders

Fix #5805
2021-12-21 09:36:50 +01:00
Théophane Hufschmitt 23ea1e46cc
Merge pull request #5047 from symphorien/fix-nix-channel
nix-channel: use nix-env -i --remove-all to upgrade
2021-12-17 10:05:48 +01:00
regnat ab902521b1 Allow paths in flake local settings
Fix #5505
2021-12-17 09:31:13 +01:00
regnat 6f1b3d711e Test the update of multiple channels
Make sure that `nix-channel --update` won’t accidentaly remove one
2021-12-17 09:17:33 +01:00
Silvan Mosberger d9fbe3e208 Add a test case for builtins.groupBy 2021-12-16 19:59:13 +01:00
Théophane Hufschmitt 8868da45a8
Merge pull request #5623 from yorickvP/fix-5621
flakes: fix boolean and int nixConfig values
2021-12-14 10:35:37 +01:00
regnat bb8a53ab08 Add a test for boolean config options in flakes 2021-12-14 09:15:24 +01:00
Eelco Dolstra 32a62b0d25 Re-allow inputs.x.url = "/path/to/file" 2021-12-13 20:59:33 +01:00
Eelco Dolstra df1edd143f gc-non-blocking.sh: Don't hang indefinitely if the GC roots server crashes 2021-12-13 19:01:38 +01:00
regnat 55e3a1108c Properly take the last assignment in the REPL
When a variable is assigned in the REPL, make sure to remove any possible reference to the old one so that we correctly pick the new one afterwards

Fix #5706
2021-12-09 16:35:27 +01:00
Eelco Dolstra 853ef1304c Inline file 2021-12-07 21:45:09 +01:00
Alexander Bantyev 33926ed1e7
nix repl: add :log
Add a :log command that shows logs for a derivation.

Closes https://github.com/NixOS/nix/issues/3504

Co-authored-by: Taeer Bar-Yam <taeer@bar-yam.me>
2021-12-06 16:52:50 +03:00
Tom Bereknyei b6cc0a704d flakes: search up to git or filesystem boundary
While parsing a flakeref, upon not finding a flake.nix, search upwards
until git or filesystem boundary.
2021-12-03 10:55:30 -05:00
Alexander Bantyev 8cbf862e6f
Flakes: computeLocks: pass correct LockParent when reusing oldLock
Previously, when we were attempting to reuse the old lockfile
information in the computeLocks function, we have passed the parent of
the current input to the next computeLocks call. This was incorrect,
since the follows are resolved relative to the parent. This caused
issues when we tried to reuse oldLock but couldn't for some
reason (read: mustRefetch is true), in that case the follows were
resolved incorrectly.

Fix this by passing the correct parent, and adding some tests to
prevent this particular regression from happening again.

Closes https://github.com/NixOS/nix/issues/5697
2021-12-02 12:02:34 +03:00
Alexander Sosedkin a9bd06d0ea Make libcpuid dependency optional with --disable-cpuid 2021-11-28 00:52:35 +01:00
regnat 05081bedc1 Make nix-shell work when nixpkgs is content-addressed
Fix #5259
2021-11-26 09:56:48 +01:00
regnat c47027f3a1 Fix the error when accessing a forbidden path in pure eval
If we’re in pure eval mode, then tell that in the error message rather
than (wrongly) speaking about restricted mode.

Fix https://github.com/NixOS/nix/issues/5611
2021-11-25 14:48:01 +01:00
Silvan Mosberger 09471d2680 Make lists be comparable
Makes lists comparable using lexicographic comparison.

Increments builtins.langVersion in order for this change to be
detectable
2021-11-24 13:40:46 +01:00
John Ericson 06fb6aecea Fix testing the other daemon
The eventual PATH entry needs the `.../bin` or we will not use the right
daemon.
2021-11-18 22:23:36 +00:00
Eelco Dolstra 79f27500a4 Test that untrusted config is ignored without --accept-flake-config 2021-11-18 13:32:52 +01:00
Eelco Dolstra 0961c1068a Merge branch 'accept_conf' of https://github.com/tomberek/nix 2021-11-18 12:56:09 +01:00
regnat 83af9550a1 Add a test for the --accept-flake-config option 2021-11-12 16:02:32 +01:00
Alexander Bantyev 07bffe7998
Flakes: refetch the input when a follows disappears
When an input follows disappears, we can't just reuse the old lock
file entries since we may be missing some required ones. Refetch the
input when this happens.

Closes https://github.com/NixOS/nix/issues/5289
2021-11-11 12:05:18 +03:00
Eelco Dolstra 736d6ab721
Merge pull request #5504 from NixOS/flake-options-and-daemon
Make the flake options work when using the daemon
2021-11-08 13:54:55 +01:00
Eelco Dolstra c9ecc0948b
Merge pull request #5506 from NixOS/fix-post-hook-test-with-different-daemon
Make the post-build-hook use the daemon Nix package
2021-11-08 13:53:30 +01:00
Théophane Hufschmitt 3d6ee223d6
Merge pull request #5476 from NixOS/ca/fix-build-in-nix-repl
Fix `nix repl`’s building of CA derivations
2021-11-05 17:35:18 +01:00
regnat 1f3c3a3785 Make the flake options work when using the daemon
When setting flake-local options (with the `nixConfig` field), forward
these options to the daemon in case we’re using one.

This is necessary in particular for options like `binary-caches` or
`post-build-hook` to make sense.

Fix <343239fc8a (r44356843)>
2021-11-05 16:19:16 +01:00
regnat 93eadd5803 Make the post-build-hook use the daemon Nix package
Having the `post-build-hook` use `nix` from the client package can lead
to a deadlock in case there’s a db migration to do between both, as a
`nix` command running inside the hook will run as root (and as such will
bypass the daemon), so might trigger a db migration, which will get
stuck trying to get a global lock on the DB (as the daemon that ran the
hook already has a lock on it).
2021-11-05 11:11:33 +01:00
regnat 1a4c9ba50b Fix nix repl’s building of CA derivations
When running a `:b` command in the repl, after building the derivations
query the store for its outputs rather than just assuming that they are
known in the derivation itself (which isn’t true for CA derivations)

Fix #5328
2021-11-05 11:04:14 +01:00
Eelco Dolstra 05560f6350 Fix function-trace test case 2021-11-04 15:04:04 +01:00
Eelco Dolstra a7d4f3411e Merge remote-tracking branch 'origin/master' into non-blocking-gc 2021-10-28 14:56:55 +02:00
regnat af99941279 Make experimental-features a proper type
Rather than having them plain strings scattered through the whole
codebase, create an enum containing all the known experimental features.

This means that
- Nix can now `warn` when an unkwown experimental feature is passed
  (making it much nicer to spot typos and spot deprecated features)
- It’s now easy to remove a feature altogether (once the feature isn’t
  experimental anymore or is dropped) by just removing the field for the
  enum and letting the compiler point us to all the now invalid usages
  of it.
2021-10-26 07:02:31 +02:00
Eelco Dolstra a594d1afd5 Revert "Fix referrers test"
This reverts commit e31a48366f. Unnecessary after 0be8cc1466.
2021-10-15 16:58:21 +02:00
Eelco Dolstra 10f9a8e77d Add a test for the non-blocking GC 2021-10-15 16:52:37 +02:00
regnat 0b55c8767d Disable the eval-store test when using the daemon
Using the daemon will break most of the assumptions of this test, so
it’s as simple to just disable it
2021-10-15 14:15:43 +02:00
Eelco Dolstra 4d014221d4 Fix test against old daemon 2021-10-15 12:52:21 +02:00
John Ericson 7869be49c2 Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2021-10-14 20:07:20 +00:00
regnat b598e5c47c Fix the min bound for the structured-attrs test
The min bound written corresponds to the date of the commit that
introduced the change, but it only got merged on master some weeks
later. Since the version is essentially the commit date, that means that
there’s a whole range of commits on master (including the current
`nixUnstable`) that have a higher version but don’t contain the required
change.
2021-10-14 16:00:59 +02:00
regnat 3a2fc9ce1d Fix the isDaemonNewer check
- Don’t hardcode the “newer” version
- Remove an ill-placed `return`
2021-10-14 16:00:45 +02:00
Eelco Dolstra 0be8cc1466 pathInfoCache: Use the entire base name as the cache key
This fixes a bug in the garbage collector where if a path
/nix/store/abcd-foo is valid, but we do a
isValidPath("/nix/store/abcd-foo.lock") first, then a negative entry
for /nix/store/abcd is added to pathInfoCache, so /nix/store/abcd-foo
is subsequently considered invalid and deleted.
2021-10-14 13:28:22 +02:00
Eelco Dolstra e31a48366f Fix referrers test
This test broke the assumption that the hash parts of store paths are
unique.
2021-10-13 12:12:44 +02:00
Eelco Dolstra c24b9d68c5 tests/multiple-outputs.sh: Assert empty store 2021-10-13 12:12:44 +02:00
Eelco Dolstra 8614cf1334 Non-blocking garbage collector
The garbage collector no longer blocks other processes from
adding/building store paths or adding GC roots. To prevent the
collector from deleting store paths just added by another process,
processes need to connect to the garbage collector via a Unix domain
socket to register new temporary roots.
2021-10-13 12:12:44 +02:00
Tom Bereknyei 03bb8f84e0 Add compression level for NARs
Based off on @dtzWill's #2276
2021-10-12 02:14:36 -04:00
John Ericson 195daa8299 Merge remote-tracking branch 'upstream/master' into ca-drv-exotic 2021-10-08 23:59:15 +00:00
Arthur Gautier e33f74495b Adds a test for nss preload mechanism
This tests for the fix implemented in #5224

Signed-off-by: Arthur Gautier <baloo@superbaloo.net>
2021-10-07 17:25:41 +00:00
Eelco Dolstra 4806f2f6b0 Allow builtins.{path,filterSource} on paths with a context
We now build the context (so this has the side-effect of making
builtins.{path,filterSource} work on derivations outputs, if IFD is
enabled) and then check that the path has no references (which is what
we really care about).
2021-10-07 13:43:17 +02:00
Eelco Dolstra dcb3bc614c Add a test for builtins.path in a flake 2021-10-06 20:53:29 +02:00
Eelco Dolstra faeab0d5d5 Make Mercurial optional for the flakes tests 2021-10-06 18:29:20 +02:00
John Ericson edf67e1508 Merge branch 'path-info' into ca-drv-exotic 2021-10-01 17:25:22 +00:00
Eelco Dolstra 7cc220825d
Merge pull request #5167 from Ma27/keep-failed-on-ssh-remote-build
nix-store --serve: pass on `settings.keepFailed` from SSH store
2021-10-01 16:35:02 +02:00
John Ericson d6e0c511ec Fix texted hash output test to work when testing daemon
Need to get experiment features to daemon like with the other tests.
2021-09-30 23:57:07 +00:00
John Ericson 1ef88da350 Merge remote-tracking branch 'upstream/master' into indexed-store-path-outputs 2021-09-30 23:47:53 +00:00
John Ericson 9af9ab4212 Merge branch 'path-info' into ca-drv-exotic 2021-09-30 22:42:15 +00:00
Eelco Dolstra 6b5efeafd1
Merge pull request #5295 from Ma27/bmcheck-fod-mismatch
build: also throw hash-mismatch errors if `buildMode == bmCheck`
2021-09-29 10:13:45 +02:00
Maximilian Bosch bb1a851bcf
build: also throw hash-mismatch errors if buildMode == bmCheck
This actually bit me quite recently in `nixpkgs` because I assumed that
`nix-build --check` would also error out if hashes don't match anymore[1]
and so I wrongly assumed that I couldn't reproduce the mismatch error.

The fix is rather simple, during the output registration a so-called
`delayedException` is instantiated e.g. if a FOD hash-mismatch occurs.
However, in case of `nix-build --check` (or `--rebuild` in case of `nix
build`), the code-path where this exception is thrown will never be
reached.

By adding that check to the if-clause that causes an early exit in case
of `bmCheck`, the issue is gone. Also added a (previously failing)
test-case to demonstrate the problem.

[1] https://github.com/NixOS/nixpkgs/pull/139238, the underlying issue
    was that `nix-prefetch-git` returns different hashes than `fetchgit`
    because the latter one fetches submodules by default.
2021-09-27 15:44:39 +02:00
Eelco Dolstra 9c766a40cb Fix 'error: reading a line: Input/output error' in startBuilder()
With -vvvv, the ProgressBar was polluting the stderr of the child,
messing up its \2 message to the parent.
2021-09-27 14:44:21 +02:00
Eelco Dolstra d1bf7431bb Revert "Merge pull request #4922 from nrdxp/default-submodules"
This reverts commit 6678e98411, reversing
changes made to 90b2dd570c.
2021-09-22 17:25:25 +02:00
Eelco Dolstra 5cbb9c5406 path fetcher: Fix relative path check 2021-09-21 13:19:26 +02:00
Eelco Dolstra c17f3c5e69 Merge branch 'mh/fix-chroot-eval' of https://github.com/obsidiansystems/nix 2021-09-15 18:37:58 +02:00
Eelco Dolstra f3259af73e Merge branch 'tomberek/show_json' of https://github.com/tomberek/nix 2021-09-14 16:58:35 +02:00
regnat 7c8c42dfdc tests: Fix the recursive test with the daemon
Add the `recursive-nix` experimental-feature to the daemon, as the test
will otherwise fail
2021-09-13 14:29:42 +02:00
Tom Bereknyei dc25856d74 Ensure nix flake show produces valid json 2021-09-08 19:38:22 -04:00
Shea Levy 6678e98411
Merge pull request #4922 from nrdxp/default-submodules
libfetchers/git: fetch submodules by default
2021-09-07 05:48:23 -04:00
Eelco Dolstra b2e8120d25 parseInstallables(): Parse store paths as store paths
If the store path contains a flake, this means that a command like
"nix path-info /path" will show info about /path, not about the
default output of the flake in /path. If you want the latter, you can
explicitly ask for it by doing "nix path-info path:/path".

Fixes #4568.
2021-09-02 14:18:04 +02:00
Stéphan Kochen b2f966f487 Failing test case for flake in store 2021-09-02 13:53:09 +02:00
Eelco Dolstra 7ee639f9db
Merge pull request #5066 from Radvendii/master
add antiquotations to paths
2021-09-01 12:55:04 +02:00
Robert Hensing ecf9a3e128 tests/fetchurl.sh: Be verbose to help debug darwin eof bug #2794 2021-08-31 13:17:01 +02:00
Maximilian Bosch 50edbc4ddf
nix-store --serve: pass on settings.keepFailed from SSH store
When doing e.g.

    nix-build -A package --keep-failed --option \
      builders \
      'ssh://mfhydra?remote-store=/home/bosch/store x86_64-linux - 10 4 big-parallel'

this doesn't work properly because this build-setting is ignored.

I changed this behavior by passing the `settings.keepFailed` through the
serve-protocol to remote machines to make sure that I can introspect the
build-directory (which is particularly helpful when I have to look at a
`config.log` from a failed build for instance).
2021-08-31 13:11:46 +02:00
Alex Zero 57b9ba0ad0
Add tests for flake follow paths 2021-08-17 15:25:55 +01:00
Madeline Haraj d56ddbb999 Fix disabled case in local store test 2021-08-06 12:36:03 -04:00
Taeer Bar-Yam a6bfda7d95 path antiquotations: rename confusing test 2021-08-06 07:38:52 -04:00
Taeer Bar-Yam 624162c729 add path antiqutations test 2021-08-06 07:06:52 -04:00
ldesgoui 6eeb6f9c84 nix-shell -p: pass --arg values as nixpkgs params 2021-07-31 17:25:10 +02:00
regnat c2c0dba792 Remove the right socket before starting the daemon
For some reason, an old socket occasionally stays here on OSX, causing
the subsequent tests to fail
2021-07-27 17:06:11 +02:00
regnat addacfce4a Allow running all the tests with the daemon
When `NIX_DAEMON_PACKAGE` is set, make all the tests use the Nix daemon.
That way we can test every piece of Nix functionality both with and
without the daemon.

Tests for which using the daemon isn’t possible or doesn’t make sens can
selectively be disabled with `needLocalStore`
2021-07-27 17:06:11 +02:00
Eelco Dolstra 29e4913f79 Add --eval-store test 2021-07-27 11:17:56 +02:00
Eelco Dolstra fc248cf59e Merge branch 'balsoft/no-registries' of https://github.com/serokell/nix 2021-07-21 10:11:40 +02:00
regnat 8707773965 Properly lock the builds of CA derivations
Make sure that we can’t build the same derivation twice at the same
time.

Fix https://github.com/NixOS/nix/issues/5029
2021-07-20 06:57:56 +02:00
Alexander Bantyev 3e57e3480b
Add use-registries config option (and deprecate --no-registries flag)
Some people want to avoid using registries at all on their system; Instead
of having to add --no-registries to every command, this commit allows to
set use-registries = false in the config. --no-registries is still allowed
everywhere it was allowed previously, but is now deprecated.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-07-16 23:26:04 +03:00
regnat c05bdef020 Forward the whole Nix config to the repl subprocesses
Fill `NIX_CONFIG` with the value of the current Nix configuration before
calling the nix subprocesses in the repl
That way the whole configuration (including the possible
`experimental-features`, a possibly `--store` option or whatever) will
be made available.

This is required for example to make `nix repl` work with a custom
`--store`
2021-07-16 09:48:45 +02:00
Maximilian Bosch 04cd2da84c
Merge branch 'master' into structured-attrs-shell
Conflicts:
        src/nix/develop.cc
        src/nix/get-env.sh
        tests/shell.nix
2021-07-12 15:49:39 +02:00
Eelco Dolstra 86fb01c4be nix print-dev-env: Add --json flag 2021-07-09 12:10:48 +02:00
Eelco Dolstra e50408bd31 nix develop: Add a test for bash functions 2021-07-09 01:18:44 +02:00
Eelco Dolstra 9fc7da1e08 Add test for #4992 2021-07-09 01:18:44 +02:00
Eelco Dolstra b1cfe8f984 nix develop: Add a test for arrays 2021-07-09 00:32:55 +02:00
Eelco Dolstra 8648143120
Merge pull request #4988 from NixOS/fetchgit-name-attribute
Add a name attribute to the fetchers
2021-07-08 14:33:49 +02:00
regnat 7e5c79a2d2 Forbid the name attribute for fetchTree
We need to support it for the “old” fetch* functions for backwards
compatibility, but we don’t need it for fetchTree (as it’s a new
function).
Given that changing the `name` messes-up the content hashing, we can
just forbid passing a custom `name` argument to it
2021-07-08 13:53:52 +02:00
Eelco Dolstra 6060ea1b0e Add tests/dummy 2021-07-07 11:30:35 +02:00
Eelco Dolstra 02dd6bb610 tests/check.sh: Fix a race
Fixes this random failure:

    error: hash mismatch in fixed-output derivation '/tmp/nix-shell.EUgAVU/nix-test/tests/check/store/sfps3l3c5n7dabpx34kigxnfhmrwk2h6-dummy.drv':
             specified: sha256-47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=
                got:    sha256-0qhPS4tlCTfsj3PNi+LHSt1akRumTfJ0WO2CKdqASiY=

which happens because multiple tests were writing to ./dummy.
2021-07-07 10:48:47 +02:00
Eelco Dolstra e37ecd1282 Merge branch 'balsoft/registry' of https://github.com/serokell/nix 2021-07-07 09:57:15 +02:00
regnat e4b082a52b allow fetchMercurial to take a name argument 2021-07-06 08:43:06 +02:00
regnat a487a652ed allow fetchTarball to take a name argument 2021-07-06 08:42:47 +02:00
regnat 83615fcf8f Allow fetchGit to take a name argument
Fix #3388
2021-07-06 08:41:52 +02:00
Michael Fellinger 087c5f5325
Fix devShell handling of env values including @ and % 2021-07-05 12:05:06 +02:00
Alexander Bantyev d8ad6f1c10
Add tests for --registry and second arg for nix registry pin 2021-07-02 16:10:57 +03:00
Eelco Dolstra f5320299dd
Merge pull request #4937 from NixOS/ca/make-the-tests-useful
Make the CA tests actually test something
2021-06-28 16:06:49 +02:00
Eelco Dolstra bf68c693dc tests: Get rid of some result symlinks
Fixes

  error: cannot create symlink '/home/eelco/Dev/nix/tests/result'; already exists
2021-06-25 11:17:19 +02:00
regnat 7746cb13dc Make CA derivations compatible with recursive Nix
Add an access-control list to the realisations in recursive-nix (similar
to the already existing one for store paths), so that we can build
content-addressed derivations in the restricted store.

Fix #4353
2021-06-24 14:53:10 +02:00
regnat be7a4a6a13 Make the post-build-hook also run for unresolved CA derivations
Fix #4837
2021-06-24 11:41:57 +02:00
regnat 01a3f4d7ec Fix the CA gc test
Broken by https://github.com/NixOS/nix/issues/4936
2021-06-23 17:37:29 +02:00
regnat fd3f5e9085 Make the CA tests actuall test CA derivations
Fix a mistake in config.nix that was preventing
`NIX_TESTS_CA_BY_DEFAULT` from having any meaningful effect
2021-06-23 17:36:50 +02:00
Eelco Dolstra 0a535dd5ac
Merge pull request #4839 from NixOS/ca/gracefully-handle-duplicate-realisations
Gracefully handle duplicate realisations
2021-06-23 11:50:18 +02:00
Eelco Dolstra f9f773b332
Merge pull request #4908 from NixOS/ca/fix-nix-develop
Make `nix develop` work with CA derivations
2021-06-23 11:27:58 +02:00
Eelco Dolstra 7945055c63
Merge pull request #4842 from NixOS/ca/fix-nix-shell
Make `nix-shell` support content-addressed derivations
2021-06-23 11:26:22 +02:00
regnat a5df669bc6 Add a test for the “two glibc” issue 2021-06-23 11:18:31 +02:00
Eelco Dolstra 4a5aa1dbf6
Merge pull request #4838 from NixOS/ca/recursively-substitute-realisations
Recursively substitute the realisations
2021-06-23 10:33:25 +02:00
Eelco Dolstra 26d2c62225
Merge pull request #4906 from NixOS/collect-garbage-ca
Make `computeFSClosure` ca-aware
2021-06-23 10:07:08 +02:00
Eelco Dolstra d9a43d3137
Merge pull request #4905 from NixOS/ca-derivations-machine-feature
Add a ca-derivations required machine feature
2021-06-23 10:05:53 +02:00
Maximilian Bosch 27ce722638
Prefix env vars for attrs.* files with NIX_ 2021-06-22 19:45:08 +02:00
Maximilian Bosch 3504c811a5
Add testcase for nix develop with __structuredAttrs 2021-06-22 19:15:57 +02:00
Maximilian Bosch 3944a120ec
Set environment variables for .attrs.json & .attrs.sh
This way no derivation has to expect that these files are in the `cwd`
during the build. This is problematic for `nix-shell` where these files
would have to be inserted into the nix-shell's `cwd` which can become
problematic with e.g. recursive `nix-shell`.

To remain backwards-compatible, the location inside the build sandbox
will be kept, however using these files directly should be deprecated
from now on.
2021-06-22 19:15:57 +02:00
Maximilian Bosch 3b5429aec1
Source complete env in nix-shell with __structuredAttrs = true;
This is needed to push the adoption of structured attrs[1] forward. It's
now checked if a `__json` exists in the environment-map of the derivation
to be openend in a `nix-shell`.

Derivations with structured attributes enabled also make use of a file
named `.attrs.json` containing every environment variable represented as
JSON which is useful for e.g. `exportReferencesGraph`[2]. To
provide an environment similar to the build sandbox, `nix-shell` now
adds a `.attrs.json` to `cwd` (which is mostly equal to the one in the
build sandbox) and removes it using an exit hook when closing the shell.

To avoid leaking internals of the build-process to the `nix-shell`, the
entire logic to generate JSON and shell code for structured attrs was
moved into the `ParsedDerivation` class.

[1] https://nixos.mayflower.consulting/blog/2020/01/20/structured-attrs/
[2] https://nixos.org/manual/nix/unstable/expressions/advanced-attributes.html#advanced-attributes
2021-06-22 19:15:57 +02:00
regnat 3784c66a46 Remove a possible existing store path when building CA derivations
In case a previous interrupted build left a garbage path laying around,
remove it before trying to move the path to its final location.

Fix #4858
2021-06-21 15:29:15 +02:00
Timothy DeHerrera 769ca4e26d
libfetchers/git: fetch submodules by default 2021-06-17 12:00:26 -06:00
regnat a3ce88725b Add a test for the gc with CA derivations
Also add a small architecture to easily run CA-enabled tests
2021-06-15 12:11:31 +02:00
regnat 2cf591a134 Make nix develop work with CA derivations
Fix #4823
2021-06-11 13:35:13 +02:00
regnat 56605b4688 Make nix-shell support content-addressed derivations
Resolve the derivation before trying to load its environment −
essentially reproducing what the build loop does − so that we can
effectively access our dependencies (and not just their placeholders).

Fix #4821
2021-06-11 13:32:49 +02:00
regnat 7c077d2a0f Add a ca-derivations required machine feature
Make ca-derivations require a `ca-derivations` machine feature, and
ca-aware builders expose it.

That way, a network of builders can mix ca-aware and non-ca-aware
machines, and the scheduler will send them in the right place.
2021-06-11 09:12:53 +02:00
regnat d12b12a15b Let nix flake check keep going when keep-going is set
When the `keep-going` option is set to `true`, make `nix flake check`
continue as much as it can before failing.

The UI isn’t perfect as-it-is as all the lines currently start with a
mostly useless `error (ignored): error:` prefix, but I’m not sure what
the best output would be, so I’ll leave it as-it-is for the time being

(This is a bit hijacking the `keep-going` flag as it’s supposed to be a
build-time only thing. But I think it’s faire to reuse it here).

Fix https://github.com/NixOS/nix/issues/4450
2021-06-02 11:13:12 +02:00
regnat 5985b8b527 Check the CA hash when importing stuff in the local store
When adding a path to the local store (via `LocalStore::addToStore`),
ensure that the `ca` field of the provided `ValidPathInfo` does indeed
correspond to the content of the path.
Otherwise any untrusted user (or any binary cache) can add arbitrary
content-addressed paths to the store (as content-addressed paths don’t
need a signature).
2021-06-01 15:09:24 +02:00
Timothy Klim 4da9ec772c Add .tar.zst support for TarballInputScheme 2021-05-29 16:03:26 +07:00
regnat a22755721b Recursively substitute the realisations
Make sure that whenever we substitute a realisation, we also substitute
its entire closure
2021-05-26 18:44:17 +02:00
regnat f46adb783c Add a test for nix run with CA derivations 2021-05-17 15:10:48 +02:00
Eelco Dolstra 7f9759b18d
Merge pull request #4781 from NixOS/locally_cache_the_remote_realisations
Add a realisations disk cache
2021-05-10 20:37:57 +02:00
Eelco Dolstra 8768398d5f
Merge pull request #4787 from Ma27/builtins-ceil-floor
Implement `builtins.floor` and `builtins.ceil` using the C library functions internally
2021-05-10 20:37:27 +02:00
Maximilian Bosch 7f7f99f350
Implement builtins.floor and builtins.ceil using the C library functions internally
Closes #4782

Note: even though the type is internally called `NixFloat`, it's
actually a `double`.
2021-05-10 12:19:32 +02:00
regnat b66234134f Add a realisations disk cache
Similar to the nar-info disk cache (and using the same db).
This makes rebuilds muuch faster.

- This works regardless of the ca-derivations experimental feature.
  I could modify the logic to not touch the db if the flag isn’t there,
  but given that this is a trash-able local cache, it doesn’t seem to be
  really worth it.
- We could unify the `NARs` and `Realisation` tables to only have one
  generic kv table. This is left as an exercise to the reader.
- I didn’t update the cache db version number as the new schema just
  adds a new table to the previous one, so the db will be transparently
  migrated and is backwards-compatible.

Fix #4746
2021-05-06 17:38:20 +02:00
regnat bf485dcf46 Properly normalize the content-addressed paths
Make sure that their timestamp are always normalized.
Otherwise, strange − and non-deterministic − things might happen, like
https://github.com/NixOS/nixpkgs/issues/121813

Fix #4775
2021-05-05 21:00:08 +02:00
regnat 6ea9c65aec fixup! Add a regression test for #4725 2021-04-23 09:34:16 +02:00
regnat b1711071d1 Add a regression test for #4725 2021-04-22 20:07:02 +02:00
Eelco Dolstra 76980a1f3d Merge branch 'build-with-strictDeps' of https://github.com/hercules-ci/nix 2021-04-15 14:15:21 +02:00
Eelco Dolstra 6fb7582413 Merge branch 'libarchive-decompress' of https://github.com/serokell/nix 2021-04-15 13:39:04 +02:00
Domen Kožar c2dfda007e
add tests for zstd compression 2021-04-09 23:13:08 +02:00
John Ericson 8499f32fb2 New "indexed" installable syntax: <drvPath>!<outputName>
Being conservative and only doing a single output name for now.
2021-04-06 10:25:09 -04:00
John Ericson cdc9f34a44 Merge commit 'e12308dd63f0ad27b22dcdb3da89c411eebcad2b' into ca-drv-exotic 2021-04-05 19:06:30 -04:00
Robert Hensing c3090bc6fd tests/*: show when tests are skipped 2021-03-31 08:30:12 +02:00
Eelco Dolstra e7810665a7
Merge pull request #4676 from bjornfor/fix-fetchgit-ref-head
fetchGit: don't prefix "refs/heads/" on ref = "HEAD"
2021-03-30 12:07:05 +02:00
Bjørn Forsman f2a799b16d tests: check that builtins.fetchGit { ..., ref = "HEAD"; } works 2021-03-30 11:39:37 +02:00
Eelco Dolstra 3ab5e8a391
Merge pull request #4239 from tweag/test-against-old-daemon
Add a CI check to ensure compatibility with an old daemon
2021-03-29 16:15:36 +02:00
Eelco Dolstra dd77f71afe LocalBinaryCacheStore::upsertFile(): Fix race
When multiple threads try to upsert the same file, this could fail.

Fixes #4667.
2021-03-26 17:10:15 +01:00
Samuel Dionne-Riel bf07581497 tests: Test .config stays clean with XDG_CONFIG_HOME set 2021-03-19 15:20:47 -04:00
Samuel Dionne-Riel 1765711b68 tests/config: Fix config test configuration
First, "XDG_CONFIG_HOME" shouldn't be named "home", as it may be
confusing compared with `$HOME`, which an upcoming test will be using.

Then, using a fixed location for the test is problematic. Use
`$TEST_ROOT` instead.
2021-03-19 15:20:47 -04:00
Eelco Dolstra 66fa1c7375 Merge 'nix flake {info,list-inputs}' into 'nix flake metadata'
Fixes #4613.
2021-03-16 17:19:04 +01:00
regnat be60c9ef50 Fix the db-migration test 2021-03-16 14:21:41 +01:00
regnat 81df1b5c68 Remove the remote-store-old-daemon test
Doesn't make sense anymore with the new setup
2021-03-16 14:21:40 +01:00
regnat a0866c8ea4 Make the tests (optionnally) run in another derivation
That way we can run them without rebuilding Nix
2021-03-16 14:21:39 +01:00
regnat eab9cdbd75 Add a test for the migration of the db between versions 2021-03-16 14:20:41 +01:00
regnat 5716345adf Add a test ensuring compatibility with an old daemon
This requires adding `nix` to its own closure which is a bit unfortunate,
but as it is optional (the test will be disabled if `OUTER_NIX` is unset) it
shouldn't be too much of an issue.

(Ideally this should go in another derivation so that we can build Nix and run
the test independently, but as the tests are running in the same derivation
as the build it's a bit complicated to do so).
2021-03-16 14:20:41 +01:00
sternenseemann 5869b3025d tests/local.mk: fix missing newline escape
Fixes syntax error introduced in 54ced9072b.
2021-03-16 08:25:04 +01:00
Eelco Dolstra c0073f6268
Merge pull request #4580 from obsidiansystems/restore-test-build-remote-ca-fixed
Restore now-working build-remote-content-addressed-fixed test
2021-03-15 16:48:28 +01:00
regnat 54ced9072b Check the signatures when copying store paths around
Broken atm
2021-03-15 16:35:14 +01:00
Eelco Dolstra 306c154632
Merge pull request #4592 from NixOS/ca/remote-cache
Substitute content-addressed derivations
2021-03-15 16:22:42 +01:00
Kjetil Orbekk 92a234322f Add test for git+file with bare repository 2021-03-01 09:03:25 -05:00
regnat 93b5a59b67 Add a test for the remote caching of CA derivations 2021-03-01 14:00:17 +01:00
regnat 259d6778ef Move the CA tests to a sub-directory
Requires a slight update to the test infra to work properly, but
having the possibility to group tests that way makes the whole thing
quite cleaner imho
2021-03-01 11:08:01 +01:00
John Ericson 7863036634 Merge remote-tracking branch 'obsidian/path-info' into ca-drv-exotic 2021-02-27 05:46:59 +00:00
John Ericson 2dd11f0780 Reenable previously failing trustless remote builder tests 2021-02-27 05:33:47 +00:00
John Ericson e547fe12d1 Merge branch 'restore-test-build-remote-ca-fixed' into trustless-remote-builder-simple 2021-02-27 03:53:22 +00:00
John Ericson 5b42e5b177 Restore now-working build-remote-content-addressed-fixed test
This was

 - Added in dbf96e10ec.

 - Commented out in 07975979aa, which I
   believe only reached master by mistake.

 - Deleted in c32168c9bc, when
   `tests/build-hook-ca.nix` was reused for a new test.

But the test works, and we ought to have it.
2021-02-26 16:32:52 +00:00
Eelco Dolstra 94637cd7e5
Merge pull request #4477 from NixOS/ca/build-remote
Build ca derivations remotely
2021-02-26 16:54:44 +01:00
Eelco Dolstra 453c3a603f nix flake update: Recreate the lock file
This is probably what most people expect it to do. Fixes #3781.

There is a new command 'nix flake lock' that has the old behaviour of
'nix flake update', i.e. it just adds missing lock file entries unless
overriden using --update-input.
2021-02-26 14:55:54 +01:00
John Ericson 90d76fa399 Merge remote-tracking branch 'obsidian/path-info' into ca-drv-exotic 2021-02-25 21:58:41 +00:00
regnat aead35531a Add a test for the copy of CA paths 2021-02-25 17:19:18 +01:00
John Ericson 7d703f2106 Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2021-02-24 17:58:43 +00:00
Shea Levy 98d1b64400
Initialize plugins after handling initial command line flags
This is technically a breaking change, since attempting to set plugin
files after the first non-flag argument will now throw an error. This
is acceptable given the relative lack of stability in a plugin
interface and the need to tie the knot somewhere once plugins can
actually define new subcommands.
2021-02-24 08:22:17 -05:00
Eelco Dolstra a878c448d8
Merge pull request #4551 from danieldk/system-features-compute-level
Add x86_64 compute levels as system features
2021-02-23 14:30:10 +01:00
regnat c32168c9bc Test the remote building of ca derivations 2021-02-23 08:04:03 +01:00
Daniël de Kok 2de232d2b3 Add x86_64 compute levels as additional system types
When performing distributed builds of machine learning packages, it
would be nice if builders without the required SIMD instructions can
be excluded as build nodes.

Since x86_64 has accumulated a large number of different instruction
set extensions, listing all possible extensions would be unwieldy.
AMD, Intel, Red Hat, and SUSE have recently defined four different
microarchitecture levels that are now part of the x86-64 psABI
supplement and will be used in glibc 2.33:

https://gitlab.com/x86-psABIs/x86-64-ABI
https://lwn.net/Articles/844831/

This change uses libcpuid to detect CPU features and then uses them to
add the supported x86_64 levels to the additional system types. For
example on a Ryzen 3700X:

$ ~/aps/bin/nix -vv --version | grep "Additional system"
Additional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux
2021-02-22 09:11:15 +01:00
regnat be1b5c4e59 Test the garbage collection of CA derivations
Simple test to ensure that `nix-build && nix-collect-garbage &&
nix-build -j0` works as it should
2021-02-19 15:48:31 +01:00
John Ericson b0d52855dc Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2021-02-12 05:51:53 +00:00
Graham Christensen f2245091d0
Revert "narinfo: Change NAR URLs to be addressed on the NAR hash instead of the compressed hash" 2021-02-09 12:26:41 -05:00
Eelco Dolstra ee3846b587
Merge pull request #4464 from tweag/nar-narhash-addressed
narinfo: Change NAR URLs to be addressed on the NAR hash instead of the compressed hash
2021-02-09 14:47:39 +01:00
Eelco Dolstra 0187838e2e Add a trace to readLine() failures
Hopefully this helps to diagnose 'error: unexpected EOF reading a
line' on macOS.
2021-02-05 12:18:11 +01:00
Eelco Dolstra 488a826842
Merge pull request #4467 from edolstra/error-formatting
Improve error formatting
2021-01-25 12:50:57 +01:00
Eelco Dolstra 680d8a5b86
Merge pull request #4387 from obsidiansystems/non-local-store-build
Make `nix-build --store whatever` work
2021-01-25 12:24:23 +01:00
John Ericson 5738b08233 Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2021-01-22 16:22:00 +00:00
John Ericson 8c07ed1dda Improve documentation and test and requested 2021-01-22 15:58:58 +00:00
Eelco Dolstra 55849e153e Change error position formatting
It's now

  at /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix:7:7:

instead of

  at: (7:7) in file: /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix

The new format is more standard and clickable.
2021-01-21 11:02:09 +01:00
adisbladis 144cad9069
narinfo: Change NAR URLs to be addressed on the NAR hash instead of the compressed hash
This change is to simplify [Trustix](https://github.com/tweag/trustix) indexing and makes it possible to reconstruct this URL regardless of the compression used.

In particular this means that 7c2e9ca597/contrib/nix/nar/nar.go (L61-L71) can be removed and only the bits that are required to establish trust needs to be published in the Trustix build logs.
2021-01-21 10:32:56 +01:00
Eelco Dolstra d04d8463fa
Merge pull request #4281 from lilyball/shebang
Escape filename given to nix-shell in shebang mode
2021-01-20 10:00:10 +01:00
John Ericson 0027b05a15 Merge remote-tracking branch 'upstream/master' into non-local-store-build 2021-01-15 02:01:24 +00:00
Eelco Dolstra 7a472a76d4 Add 'nix daemon' command 2021-01-14 00:05:04 +01:00
Eelco Dolstra d33eca8539 Rename 'nix store sign-paths' to 'nix store sign' 2021-01-13 23:32:37 +01:00
Eelco Dolstra 6254b1f5d2 Add 'nix store delete' command 2021-01-11 19:46:59 +01:00
Eelco Dolstra fdcd62eec5 Add 'nix store gc' command 2021-01-10 23:29:14 +01:00
Eelco Dolstra 0df69d96e0 Make sodium a required dependency 2021-01-06 17:56:53 +01:00
Eelco Dolstra 9374c2baea Add commands for generating secret/public keys 2021-01-06 17:49:31 +01:00
John Ericson fed1237246 Test nix-build with non-local-store --store
Just a few small things needed fixing!
2020-12-23 22:42:06 +00:00
John Ericson 29bd63e990 Test nix-instantiate with binary cache store
Trying to make sure it work with obscurers stores.
2020-12-23 22:42:06 +00:00
Eelco Dolstra f4a9fb67da Merge branch 'git-rev-error' of https://github.com/Ma27/nix into master 2020-12-22 16:39:32 +01:00
Eelco Dolstra 75efa42134 Move <nix/fetchurl.nix> into the nix binary
This makes the statically linked nix binary just work, without needing
any additional files.
2020-12-22 14:43:20 +01:00
Maximilian Bosch 897ae235fc
tests/fetchGit: test behavior of allRefs = true; 2020-12-22 12:18:10 +01:00
John Ericson bd96403da6 Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2020-12-20 14:02:12 +00:00
regnat 58cdab64ac Store metadata about drv outputs realisations
For each known realisation, store:
- its output
- its output path

This comes with a set of needed changes:

- New `realisations` module declaring the types needed for describing
  these mappings
- New `Store::registerDrvOutput` method registering all the needed informations
  about a derivation output (also replaces `LocalStore::linkDeriverToPath`)
- new `Store::queryRealisation` method to retrieve the informations for a
  derivations

This introcudes some redundancy on the remote-store side between
`wopQueryDerivationOutputMap` and `wopQueryRealisation`.
However we might need to keep both (regardless of backwards compat)
because we sometimes need to get some infos for all the outputs of a
derivation (where `wopQueryDerivationOutputMap` is handy), but all the
stores can't implement it − because listing all the outputs of a
derivation isn't really possible for binary caches where the server
doesn't allow to list a directory.
2020-12-11 20:41:32 +01:00
Eelco Dolstra 9c143c411b
Merge pull request #4350 from NixOS/ca/fix-build-with-nix-command
Fix the `nix` command with CA derivations
2020-12-11 15:01:49 +01:00
regnat eb45308109 Fix the nix command with CA derivations
Prevents a crash because most `nix` subcommands assumed that derivations
know their output path, which isn't the case for CA derivations
2020-12-11 10:28:09 +01:00
regnat 5286310e59 Use no substituers by default in the tests
Otherwise https://cache.nixos.org is chosen by default, causing the OSX
testsuite to hang inside the sandbox.

(In a way, this is probably rugging an actual bug under the carpet as
Nix should be able to gracefully timeout in such a case, but that's
beyond mac OSX-fu)
2020-12-09 14:53:45 +01:00
regnat ee7c94fa1b Test the post-build-hook with remote builders
Regression test for #4245
2020-12-09 10:45:12 +01:00
Eelco Dolstra f337aa7099 Split 'nix store add-to-store' into 'add-path' and 'add-file'
This makes it consistent with 'nix hash <path|file>'.
2020-12-04 00:59:24 +01:00
Eelco Dolstra ea2062a2d9 Move most store-related commands to 'nix store' 2020-12-03 23:22:22 +01:00
Eelco Dolstra ef583303f0 Move NAR-related commands to 'nix nar' 2020-12-03 18:09:02 +01:00
Eelco Dolstra b2d6c6161e Move 'nix hash-*' and 'nix to-*' to 'nix hash'
From the 'nix' UX review.
2020-12-03 17:55:55 +01:00
Eelco Dolstra 0bd060f23a
Merge pull request #4308 from tweag/properly-test-early-cutoff
Properly test the early cutoff for CA derivations
2020-12-03 14:45:29 +01:00
regnat 8ad72b1f1c Properly test early cutoff with CA derivations
Build things with a different seed each time to make sure that it works
despite the different drvs
2020-12-03 13:31:07 +01:00
Eelco Dolstra df552a2645 nix eval: Add option to write a directory
This is useful for generating the nix manpages, but it may have other
applications (like generating configuration files without a Nix store).
2020-12-02 23:23:23 +01:00
Eelco Dolstra e5cf501c77
Merge pull request #4284 from tweag/fixed-output-depending-on-ca
Allow fixed-output derivations to depend on (floating) content-addressed ones
2020-12-01 20:25:41 +01:00
regnat 9bd8184f1f Allow fixed-output derivations to depend on (floating) content-addressed ones
Fix an overlook of https://github.com/NixOS/nix/pull/4056
2020-11-27 15:39:24 +01:00
Eelco Dolstra 05d9442f68 builtins.fetchGit: Fix shortRev attribute for dirty trees 2020-11-26 21:45:28 +01:00
regnat 13c557fe82 fix the hash rewriting for ca-derivations 2020-11-25 11:33:00 +01:00
Lily Ballard 437189e446 Escape filename given to nix-shell in shebang mode
This prevents spaces or other metacharacters from causing nix-shell to
execute the wrong path.

Fixes #4229.
2020-11-24 15:08:37 -08:00
Luke Granger-Brown 226116f482 fetchMercurial: set HGPLAIN when invoking hg
Without setting HGPLAIN, the user's environment leaks into
hg invocations, which means that the output may not be in the
expected format.

HGPLAIN is the Mercurial-recommended solution for this in that
it's intended for uses by scripts and programs which are looking
to parse Mercurial's output in a consistent manner.
2020-11-23 16:12:33 +00:00
Eelco Dolstra f89fd0bde7 Remove stray debug statement
This was causing a failure on macOS.

https://hydra.nixos.org/build/130354318
2020-11-17 15:36:20 +01:00
Eelco Dolstra f4e790cc85
Merge pull request #4182 from mkenigs/fix-1930
Print built derivations as json for build
2020-11-17 14:59:49 +01:00
Eelco Dolstra bccff827dc Fix deadlock in IFD through the daemon
Fixes #4235.
2020-11-17 13:50:36 +01:00
Robert Hensing d264da8d96 tests: Test #4197 nix-build output order regression 2020-11-13 17:50:04 +01:00
Matthew Kenigsberg d52b12c0a5 Test nix build --json 2020-11-11 10:27:02 -06:00
Sebastian Ullrich 3f24a417da Add test case for incidentally fixed #4228 2020-11-09 22:43:14 +01:00
Sebastian Ullrich 579b953231 Make test case more precise
Co-authored-by: Théophane Hufschmitt <regnat@users.noreply.github.com>
2020-11-09 22:43:14 +01:00
Sebastian Ullrich fb7735e4cf nix develop: Preserve stdin with -c 2020-11-09 22:43:14 +01:00
Graham Christensen 8cd2ff69c3
nix-copy-closure: verify it works with drvs
Creates test coverage for #4210 and 7cf874c17d
2020-11-02 15:50:14 -05:00
Eelco Dolstra 02a1facbdc
Merge pull request #4056 from tweag/non-ca-depending-on-ca
Allow non-CA derivations to depend on CA ones
2020-10-27 17:38:29 +01:00
regnat ab21ab6501 Test the remote caching of non-ca-depending-on-ca derivations
Although the non-resolved derivation will never get a cache-hit (it
doesn't have an output path to query the cache for anyways), we might
get one on the resolved derivation.
2020-10-27 07:29:25 +01:00
regnat c092fa4702 Allow non-CA derivations to depend on CA derivations 2020-10-27 07:29:23 +01:00
Eelco Dolstra 1e66d146a3 Fix test 2020-10-26 17:59:32 +01:00
Christian Kampka 461cf2b856
Add NIX_CONFIG env var for applying nix.conf overrides 2020-10-21 13:41:26 +02:00
Robert Hensing ea8d32020e Tests for #3964 2020-10-18 14:26:37 +02:00
aszlig cfa26cf181
tests: Add names to VM tests
Having vm-test-run-unnamed for all the test derivation doesn't look very
nice, so in order to better distinguish them from their store path,
let's actually give them proper names.

Signed-off-by: aszlig <aszlig@nix.build>
2020-10-17 23:34:38 +02:00
aszlig 5cfdf16dd6
Convert VM tests to Python
Perl-based tests are deprecated since NixOS 20.03 and subsequently got
removed in NixOS 20.09, which effectively means that tests are going to
fail as soon as we build it with NixOS 20.09 or anything newer.

I've put "# fmt: off" at the start of every testScript, because
formatting with Black really messes up indentation and I don't think it
really adds anything in value or readability for inlined Python scripts.

Signed-off-by: aszlig <aszlig@nix.build>
2020-10-17 23:32:03 +02:00
John Ericson a4e5de1b9d Derivations can output "text-hashed" data
In particular, this means that derivations can output derivations. But
that ramification isn't (yet!) useful as we would want, since there is
no way to have a dependent derivation that is itself a dependent
derivation.
2020-10-13 02:15:48 +00:00
John Ericson bdc7720227 Merge branch 'master' of github.com:NixOS/nix into trustless-remote-builder-simple 2020-10-12 23:00:18 +00:00
Eelco Dolstra 1e8855a7f7
Merge pull request #3958 from obsidiansystems/ca-floating-upstream
CA derivations that depend on other CA derivations
2020-09-29 11:37:29 +02:00
John Ericson 10202bbf29 Merge remote-tracking branch 'upstream/master' into ca-floating-upstream 2020-09-28 15:39:11 +00:00
Mateusz Piotrowski ed66d01065
Fix tar invocation on FreeBSD
tar(1) on FreeBSD does not use standard output or input when the -f flag
is not provided. Instead, it defaults to /dev/sa0 on FreeBSD.

Make this tar invocation a bit more robust and explicitly tell tar(1) to
use standard output.

This is one of the issues discovered while porting Nix to FreeBSD. It has
been tested and committed locally to FreeBSD ports:
https://svnweb.freebsd.org/ports/head/sysutils/nix/Makefile?revision=550026&view=markup#l108
2020-09-28 15:23:21 +02:00
Eelco Dolstra 31ab4c3816 Test whether build/repair results are read-only 2020-09-23 19:09:58 +02:00
John Ericson 6a0e0db349 Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2020-09-22 00:21:37 +00:00
John Ericson b7df353f27 Merge remote-tracking branch 'upstream/master' into ca-floating-upstream 2020-09-17 16:33:10 +00:00
Eelco Dolstra 649d3aaf24
Merge pull request #3829 from obsidiansystems/remove-storetype-delegate-regStore
Remove storetype delegate reg store -- contains #3736
2020-09-17 13:55:01 +02:00
Eelco Dolstra b94a35ef40
Merge pull request #4027 from tweag/fix-gc-of-ca-derivations
Fix garbage collection of CA derivations
2020-09-17 13:46:26 +02:00
regnat 520895b1da Fix garbage collection of CA derivations
Fix #4026
2020-09-17 13:36:58 +02:00
Eelco Dolstra c9f51e8705 Remove corepkgs/config.nix
This isn't used anywhere except in the configure script of the Perl
bindings. I've changed the latter to use the C++ API's Settings object
at runtime.
2020-09-17 10:42:51 +02:00
John Ericson f60b380a7f Merge remote-tracking branch 'upstream/master' into remove-storetype-delegate-regStore 2020-09-16 22:35:24 +00:00
John Ericson c5ccebae00 Merge remote-tracking branch 'upstream/master' into ca-floating-upstream 2020-09-16 17:50:40 +00:00
regnat c29624bf7d Add a test for nix describe-stores
Doesn't test much, but at least ensures that the command runs properly
2020-09-16 13:53:28 +02:00
John Ericson 3ba552b245 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-09-15 14:17:06 +00:00
John Ericson c08c9f08c7 Merge remote-tracking branch 'upstream/master' into remove-storetype-delegate-regStore 2020-09-15 14:08:35 +00:00
regnat 250f8a4bba Escape ${ in strings when printing Nix expressions
Otherwise the result of the printing can't be parsed back correctly by
Nix (because the unescaped `${` will be parsed as the begining of an
anti-quotation).

Fix #3989
2020-09-14 17:19:25 +02:00
Jade Lovelace 7cb5f643a6
docs+test: fix remaining installer downloads without -L (#4006)
Co-authored-by: lf- <lf-@users.noreply.github.com>
2020-09-12 22:08:40 +02:00
John Ericson 075d399e3f Merge remote-tracking branch 'obsidian/single-ca-drv-build' into ca-floating-upstream 2020-09-04 16:04:35 +00:00
John Ericson c9f1ed912c Don't chmod symlink before moving outputs around
Co-authored-by: Théophane Hufschmitt <regnat@users.noreply.github.com>
2020-09-04 14:41:53 +00:00
John Ericson b99062b023
Update tests/content-addressed.nix
Co-authored-by: Théophane Hufschmitt <regnat@users.noreply.github.com>
2020-09-04 10:29:28 -04:00
John Ericson aad4abcc9c Fix floating CA tests
We will sometimes try to query the outputs of derivations we can't
resolve. That's fine; it just means we don't know what those outputs are
yet.
2020-09-04 01:17:38 +00:00
John Ericson b836662f50 Merge remote-tracking branch 'obsidian/single-ca-drv-build' into ca-floating-upstream
Tests also now fail as they should
2020-09-03 22:45:00 +00:00
John Ericson c224a5e5c1 Rename derivation in floating CA test 2020-09-03 22:35:13 +00:00
John Ericson 145915eb39 Beef up floating CA derivations test a bit 2020-09-03 22:14:45 +00:00
John Ericson 975a47f7fe Merge remote-tracking branch 'obsidian/single-ca-drv-build' into ca-floating-upstream 2020-09-03 22:09:04 +00:00
John Ericson e7d93e7ece Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-09-03 15:43:17 +00:00
Eelco Dolstra 00d25e8457
Remove the --indirect flag
All GC roots are now indirect.
2020-09-03 11:22:00 +02:00
John Ericson ef278d00f9 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-09-01 18:01:48 +00:00
John Ericson 4db0010a93 Test CA derivation input caching 2020-08-28 22:03:54 +00:00
Eelco Dolstra 4bf5faf416 Merge remote-tracking branch 'origin/master' into markdown 2020-08-25 19:47:34 +02:00
Eelco Dolstra 7a02865b94
Move import docs 2020-08-25 14:06:01 +02:00
John Ericson 8eb73a8724 CA derivations that depend on other CA derivations
Co-authored-by: Théophane Hufschmitt <regnat@users.noreply.github.com>
2020-08-24 19:01:15 +00:00
Eelco Dolstra 6a67e57019
Add DummyStore (dummy://)
DummyStore does not allow building or adding paths. This is useful for
evaluation tests when you don't want to initialize a "proper" store.
2020-08-24 18:54:16 +02:00
John Ericson 27a3f82c0b Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-08-20 18:28:17 +00:00
John Ericson 3df78858f2 Fix max fd calc and add test 2020-08-20 05:08:50 +00:00
Eelco Dolstra 0c9365c6ba
Merge pull request #3940 from obsidiansystems/fixed-output-remote-builder-test
Add commented-out test for remote building with fixed output derivations
2020-08-18 16:12:35 +02:00
John Ericson 7c4f383b37 Merge branch 'fixed-output-remote-builder-test' of github.com:obsidiansystems/nix into trustless-remote-builder-simple 2020-08-18 13:56:00 +00:00
Carlo Nucera 07975979aa Comment out fixed content address test 2020-08-17 15:04:54 -04:00
John Ericson 36758a1a09 But back check.sh
Whether it fails or not, it is no a new test so we have to leave it.
2020-08-17 18:01:38 +00:00
Carlo Nucera 1bf31bad5f Disable failing tests 2020-08-17 13:15:08 -04:00
Eelco Dolstra 7cdc739ece
Merge remote-tracking branch 'origin/master' into markdown 2020-08-17 13:43:39 +02:00
John Ericson dbf96e10ec Test remote building with fixed output derivations 2020-08-16 17:38:12 +00:00
John Ericson 9dd28a65c8 Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2020-08-16 16:05:54 +00:00
John Ericson 3c8b5b6219 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-08-14 17:00:13 +00:00
Eelco Dolstra 7714d9a943
Merge pull request #3924 from obsidiansystems/features-per-store
Make `system-features` a store setting
2020-08-14 17:13:07 +02:00
John Ericson cbc4344297 Trustless remote building
Co-authored-by: Matthew Bauer <mjbauer95@gmail.com>
2020-08-14 04:53:58 +00:00
John Ericson d2f2be0f70 Test RemoteStore::buildDerivation
Fix `wopNarFromPath` which needed a `toRealPath`.
2020-08-13 04:07:14 +00:00
John Ericson 5d67f18c86 Merge branch 'daemon-auth-cleanup' of github.com:obsidiansystems/nix into HEAD 2020-08-12 18:22:31 +00:00
John Ericson 4720853129 Make system-features a store setting
This seems more correct. It also means one can specify the features a
store should support with --store and remote-store=..., which is useful.
I use this to clean up the build remotes test.
2020-08-12 18:13:00 +00:00
John Ericson 8d4162ff9e Separate auth and logic for the daemon
Before, processConnection wanted to know a user name and user id, and
`nix-daemon --stdio`, when it isn't proxying to an underlying daemon,
would just assume "root" and 0. But `nix-daemon --stdio` (no proxying)
shouldn't make guesses about who holds the other end of its standard
streams.

Now processConnection takes an "auth hook", so `nix-daemon` can provide
the appropriate policy and daemon.cc doesn't need to know or care what
it is.
2020-08-12 15:22:33 +00:00
John Ericson 1b5c24662b Merge branch 'small-drv-serialize-cleanup' of github.com:obsidiansystems/nix into single-ca-drv-build 2020-08-10 01:57:54 +00:00
John Ericson e913a2989f Squashed get CA derivations building 2020-08-07 19:51:55 +00:00
Eelco Dolstra edfd676e05 Fix .ls file names in binary caches
These are not supposed to include the 'name' part of the store
path. This was broken by 759947bf72.
2020-08-07 21:18:29 +02:00
John Ericson 6d9ccde20d Make JSON equality tests agnostic to ordering
It is in fact more sorted than before, but I don't think we want to
guarantee anything about the ordering.
2020-08-04 14:20:13 +00:00
Eelco Dolstra bf290c2306
Merge remote-tracking branch 'origin/master' into markdown 2020-07-31 16:07:04 +02:00
Matthew Bauer 0bdd6cf6ea Add test for builtins.path 2020-07-30 16:38:04 -05:00
Matthew Bauer 05ac4db39a Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-07-30 12:38:24 -05:00
Eelco Dolstra 0c94c17644 Merge remote-tracking branch 'origin/master' into markdown 2020-07-29 18:08:57 +02:00
Maximilian Bosch e4940e90f3
Restore backwards-compat for current builtins.fetchGit
If a repo is dirty, it used to return a `rev` object with an "empty"
sha1 (0000000000000000000000000000000000000000). Please note that this
only applies for `builtins.fetchGit` and *not* for `builtins.fetchTree{
type = "git"; }`.
2020-07-28 22:46:39 +02:00
Maximilian Bosch f742438465
Merge legacy fetchGit-builtin with the generic fetchTree-function
The original idea was to implement a git-fetcher in Nix's core that
supports content hashes[1]. In #3549[2] it has been suggested to
actually use `fetchTree` for this since it's a fairly generic wrapper
over the new fetcher-API[3] and already supports content-hashes.

This patch implements a new git-fetcher based on `fetchTree` by
incorporating the following changes:

* Removed the original `fetchGit`-implementation and replaced it with an
  alias on the `fetchTree` implementation.

* Ensured that the `git`-fetcher from `libfetchers` always computes a
  content-hash and returns an "empty" revision on dirty trees (the
  latter one is needed to retain backwards-compatibility).

* The hash-mismatch error in the fetcher-API exits with code 102 as it
  usually happens whenever a hash-mismatch is detected by Nix.

* Removed the `flakes`-feature-flag: I didn't see a reason why this API
  is so tightly coupled to the flakes-API and at least `fetchGit` should
  remain usable without any feature-flags.

* It's only possible to specify a `narHash` for a `git`-tree if either a
  `ref` or a `rev` is given[4].

* It's now possible to specify an URL without a protocol. If it's missing,
  `file://` is automatically added as it was the case in the original
  `fetchGit`-implementation.

[1] https://github.com/NixOS/nix/pull/3216
[2] https://github.com/NixOS/nix/pull/3549#issuecomment-625194383
[3] https://github.com/NixOS/nix/pull/3459
[4] https://github.com/NixOS/nix/pull/3216#issuecomment-553956703
2020-07-28 00:44:39 +02:00
Eelco Dolstra 8e41c38867
Remove references to xmllint 2020-07-23 18:26:48 +02:00
Carlo Nucera 4178f36a1d Test relative store paths 2020-07-17 15:50:53 -04:00
Eelco Dolstra 17f75f9cc4 parseFlakeRef(): Only search for the top-level directory for CLI flakerefs 2020-07-17 14:54:21 +02:00
Eelco Dolstra 832e111494 Merge remote-tracking branch 'origin/master' into flakes 2020-07-14 13:56:18 +02:00
Eelco Dolstra 143a5f32ed Add a test for local NAR caching 2020-07-13 16:25:48 +02:00
Eelco Dolstra 1d01ae816b Fix 'nix verify --all' on a binary cache and add a test 2020-07-13 14:35:01 +02:00
Eelco Dolstra 2900a441f5 Add a test for DWARF debug info index generation 2020-07-13 13:38:01 +02:00
Eelco Dolstra 41bdf429ec Add a test for NAR listing generation 2020-07-13 13:32:33 +02:00
Matthew Bauer acb74d4d94 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-07-10 18:13:45 -04:00
Eelco Dolstra 68f524d717 nix develop: Support derivations with multiple outputs 2020-07-06 18:34:58 +02:00
Eelco Dolstra 54712aaf8a Merge remote-tracking branch 'origin/master' into flakes 2020-07-06 16:40:10 +02:00
Ben Burdette b29a4ea1dc Merge branch 'master' into add-trace 2020-07-03 07:57:36 -06:00
Eelco Dolstra dfaad374ff
Merge pull request #3778 from tweag/parallel-tests
Parallel tests fixes
2020-07-03 13:17:10 +02:00
Eelco Dolstra 017efae01f Hopefully fix macOS test failure 2020-07-03 13:16:22 +02:00
regnat 223fbe644a Shorten the path to the test root
Fix a socket length failure on the OSX builders
2020-07-03 09:20:01 +02:00
regnat 5101ed18bc Fix the test dependencies
Reuse the pre-existing list rather than the one written as part of #3777
2020-07-03 09:20:01 +02:00
Eelco Dolstra 5596f879b4 Add test for nix develop 2020-07-02 18:32:45 +02:00
Matthew Bauer fc2ab42e86 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-07-02 11:14:04 -04:00
Eelco Dolstra a5b6e870fe Set gc-reserved-space to 0 in tests
This reduces the amount of disk space needed to run the tests from
half a gigabyte to 10 megabytes.
2020-07-02 16:38:42 +02:00
regnat 11ba4ec795 Make the gc-auto test more reliable
Use a fifo pipe to handle the synchronisation between the different
threads rather than relying on delays
2020-07-02 16:13:36 +02:00
regnat c762385457 Make the gc-concurrent test more reliable
Use a fifo pipe to handle the synchronisation between the different
threads rather than relying on delays
2020-07-02 16:13:36 +02:00
regnat 1b5aa60767 Run the tests in parallel
Cause the time needed to run the testsuite to drop from ~4mins to ~40s
2020-07-02 16:13:36 +02:00
Ben Burdette dabbb4538f 'from string' 2020-06-30 16:43:01 -06:00
Matthew Bauer 09833126f3 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-30 16:35:08 -04:00
Ben Burdette ddb81ca126 Merge branch 'master' into add-trace 2020-06-30 12:21:45 -06:00
Eelco Dolstra adf2fbbdc2 Merge remote-tracking branch 'origin/master' into flakes 2020-06-26 08:46:46 +02:00
Eelco Dolstra b7ccf7ae2a build-remote.sh: Test LegacySSHStore 2020-06-25 18:42:55 +02:00
Ben Burdette 00fe653ea5 nixCode -> LinesOfCode 2020-06-24 08:33:53 -06:00
Ben Burdette e6f93b94fc Merge branch 'master' into caveman-LOCs 2020-06-18 13:07:53 -06:00
Eelco Dolstra 3d492199bb github: Respect default branch 2020-06-18 13:25:08 +02:00
Matthew Bauer 7bd88cc1dc Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-17 13:32:29 -04:00
Eelco Dolstra 1524752c17 Merge remote-tracking branch 'origin/master' into flakes 2020-06-17 10:26:52 +02:00
Matthew Bauer 9ac0d98a59 Remove -vvvvv from tests/fetchurl.sh nix-build call 2020-06-13 00:14:30 -05:00
Matthew Bauer c214cda940 Correctly substitute from different storeDir
Originally, the test was only checking for different “real” storeDir.
That’s an easy case to handle, but the much harder one is if different
virtual store dirs are used. To do this, we need the SubstitutionGoal
to know about the ca, so it can recalculate the path to copy it over.
An important note here is that the store path passed to copyStorePath
needs to be one for srcStore - so that queryPathInfo works properly.

This also adds an error message when the store path from queryPathInfo
is different from the one we requested.
2020-06-13 00:07:42 -05:00
Matthew Bauer 006f1252d2 Fix SRI test
We can’t use custom name here because different names will have
different store paths. This is a limitation of the Store API’s
reliance on store paths.

We might be able to get around the above in the future by using a
dummy name for certain fixed output paths.
2020-06-12 15:33:24 -05:00
Matthew Bauer 88120442d2 Debug when storePath changes
these rewrites should be transparent, but they are important to know
about when debugging
2020-06-12 15:32:52 -05:00
Matthew Bauer 1c148a80fe Replace --hashed-mirrors with substituters test 2020-06-12 14:39:44 -05:00
Ben Burdette ef1b3f21b6 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-06-11 14:06:35 -06:00
Eelco Dolstra 0c62b4ad0f Represent 'follows' inputs explicitly in the lock file
This fixes an issue where lockfile generation was not idempotent:
after updating a lockfile, a "follows" node would end up pointing to a
new copy of the node, rather than to the original node.
2020-06-11 14:40:21 +02:00
Ben Burdette b1c53b034c Merge branch 'errors-phase-2' into caveman-LOCs 2020-06-08 11:10:13 -06:00
regnat f6ac888d3e Actually test nix-env with a remote store
The `remote-store` test loads the `user-env` one to test nix-env when
using the daemon, but actually does it incorrectly because every test
starts (in `common.sh`) by resetting the value of `NIX_REMOTE`, meaning
that the `user-env` test will never use the daemon.

Fix this by setting `NIX_REMOTE_` before sourcing `user-env.sh` in the
`remote-store` test, so that `NIX_REMOTE` is correctly set inside the
test
2020-06-08 10:01:14 +02:00
Eelco Dolstra 810b2c6a48 nix flake init: Add a '--template' flag
The initial contents of the flake is specified by the
'templates.<name>' or 'defaultTemplate' output of another flake. E.g.

  outputs = { self }: {

    templates = {

      nixos-container = {
        path = ./nixos-container;
        description = "An example of a NixOS container";
      };

    };

  };

allows

  $ nix flake init -t templates#nixos-container

Also add a command 'nix flake new', which is identical to 'nix flake
init' except that it initializes a specified directory rather than the
current directory.
2020-06-04 20:22:25 +02:00
Ben Burdette 721943e1d4 update error grep 2020-06-03 17:32:57 -06:00
Ben Burdette 4335ba999b Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-06-03 17:00:00 -06:00
Eelco Dolstra 81cafda306 Fix GitHub test 2020-06-03 16:29:04 +02:00
Eelco Dolstra c20591ddc3 Merge remote-tracking branch 'origin/master' into flakes 2020-06-03 16:15:22 +02:00
Nikola Knezevic 77007d4eab Improve ref validity checking in fetchGit
The previous regex was too strict and did not match what git was allowing. It
could lead to `fetchGit` not accepting valid branch names, even though they
exist in a repository (for example, branch names containing `/`, which are
pretty standard, like `release/1.0` branches).

The new regex defines what a branch name should **NOT** contain. It takes the
definitions from `refs.c` in https://github.com/git/git and `git help
check-ref-format` pages.

This change also introduces a test for ref name validity checking, which
compares the result from Nix with the result of `git check-ref-format --branch`.
2020-05-30 12:29:35 +02:00
Eelco Dolstra 17ca997fc6 Merge remote-tracking branch 'origin/master' into flakes 2020-05-28 12:55:24 +02:00
Eelco Dolstra c3eff22f46 Merge branch 'store-visited' of https://github.com/mkenigs/nix into flakes 2020-05-28 12:15:39 +02:00
Matthew Kenigsberg 934cc802f3 circular test 2020-05-21 17:06:19 -06:00
Ben Burdette 0e49de6a2b position for stdin, string; (string) for trace; fix tests 2020-05-21 14:28:45 -06:00
Eelco Dolstra 00b562c87e Fix GitHub test 2020-05-21 22:02:34 +02:00
Ben Burdette 92123c6c79 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-05-15 07:00:36 -06:00
Eelco Dolstra 5f64655ff4 Move registry-related commands from 'nix flake' to 'nix registry'
This makes 'nix flake' less cluttered and more consistent (it's only
subcommands that operator on a flake). Also, the registry is not
inherently flake-related (e.g. fetchTree could also use it to remap
inputs).
2020-05-15 14:38:10 +02:00
Eelco Dolstra 5722f9690c tests/binary-cache.sh: Improve incomplete closure test
Issue #3373.
2020-05-12 13:56:00 +02:00
Ben Burdette 59b1f5c701 Merge branch 'master' into errors-phase-2 2020-05-11 14:35:30 -06:00
Ben Burdette 55eb717148 add pos to errorinfo, remove from hints 2020-05-08 18:18:28 -06:00
Matthew Kenigsberg 2852a486f8 rename run to shell in tests
(cherry picked from commit f459ca547f)
2020-05-01 12:17:53 +02:00
Matthew Kenigsberg f459ca547f rename run to shell in tests 2020-04-30 08:31:37 -06:00
Eelco Dolstra b69323f8c9 Revive 'nix search'
It uses the evaluation cache now rather than the ad hoc JSON cache.
2020-04-20 15:27:09 +02:00
Eelco Dolstra 0858738355 Merge remote-tracking branch 'origin/master' into flakes 2020-04-16 18:27:37 +02:00
Eelco Dolstra a118293bd0
Merge pull request #3458 from zimbatm/nix-user-conf-dir
NIX_USER_CONF_FILES
2020-04-15 13:00:28 +02:00
Jonas Chevalier 895516cadf
add NIX_USER_CONF_FILES
Motivation: maintain project-level configuration files.

Document the whole situation a bit better so that it corresponds to the
implementation, and add NIX_USER_CONF_FILES that allows overriding
which user files Nix will load during startup.
2020-04-14 18:45:06 +02:00
Eelco Dolstra c0c2cb871d Merge remote-tracking branch 'origin/master' into flakes 2020-04-14 13:02:55 +02:00
Bruce Toll e8bd1bc732 Add test case for temporary directories on darwin
A test case for correct handling of temporary directory deletion that
was added to check.sh as part of PR #2689 was initially disabled for
Darwin because of a directory permission issue in PR #2688.

Now that the issue in PR #2688 is fixed, this commit enables the test
case for Darwin.
2020-04-10 18:20:12 -04:00
Bruce Toll 8132d0a12e Fix nix-build --check -K in sandbox w/o root
Temporarily add user-write permission to build directory so that it
can be moved out of the sandbox to the store with a .check suffix.

This is necessary because the build directory has already had its
permissions set read-only, but write permission is required
to update the directory's parent link to move it out of the sandbox.

Updated the related --check "derivation may not be deterministic"
messages to consistently use the real store paths.

Added test for non-root sandbox nix-build --check -K to demonstrate
issue and help prevent regressions.
2020-04-10 16:23:10 -04:00
Eelco Dolstra e5ea01c1a8
Remove flake 'edition' field
Future editions of flakes or the Nix language can be supported by
renaming flake.nix (e.g. flake-v2.nix). This avoids a bootstrap
problem where we don't know which grammar to use to parse
flake*.nix. It also allows a project to support multiple flake
editions, in theory.
2020-04-10 10:24:09 +02:00
Bruce Toll 16a4864759 Delete temporary directory on successful build
With --check and the --keep-failed (-K) flag, the temporary directory
was being retained regardless of whether the build was successful and
reproducible.  This removes the temporary directory, as expected, on
a reproducible check build.

Added tests to verify that temporary build directories are not
retained unnecessarily, particularly when using --check with
--keep-failed.
2020-04-09 16:37:41 -04:00
Eelco Dolstra d103c79144
Merge remote-tracking branch 'origin/master' into flakes 2020-04-09 15:54:21 +02:00
Eelco Dolstra 9ed097db7b
Merge pull request #3468 from Infinisil/functionArgsPositions
Make function arguments retain position info
2020-04-08 15:29:39 +02:00
Eelco Dolstra 03a4a3c95c Merge branch 'flakes' of github.com:NixOS/nix into flakes 2020-04-07 14:30:25 +02:00
Eelco Dolstra 54955867a6 Merge remote-tracking branch 'origin/master' into flakes 2020-04-07 14:29:45 +02:00
Eelco Dolstra 55cefd41d6 Merge branch 'fetchgit-recursive' of https://github.com/blitz/nix 2020-04-07 13:45:17 +02:00
Eelco Dolstra 462421d345 Backport libfetchers from the flakes branch
This provides a pluggable mechanism for defining new fetchers. It adds
a builtin function 'fetchTree' that generalizes existing fetchers like
'fetchGit', 'fetchMercurial' and 'fetchTarball'. 'fetchTree' takes a
set of attributes, e.g.

  fetchTree {
    type = "git";
    url = "https://example.org/repo.git";
    ref = "some-branch";
    rev = "abcdef...";
  }

The existing fetchers are just wrappers around this. Note that the
input attributes to fetchTree are the same as flake input
specifications and flake lock file entries.

All fetchers share a common cache stored in
~/.cache/nix/fetcher-cache-v1.sqlite. This replaces the ad hoc caching
mechanisms in fetchGit and download.cc (e.g. ~/.cache/nix/{tarballs,git-revs*}).

This also adds support for Git worktrees (c169ea5904).
2020-04-07 09:03:14 +02:00
Eelco Dolstra 3473b1950a
Fix tests.githubFlakes evaluation 2020-04-03 21:36:21 +02:00
Eelco Dolstra 9c78f7f196 getFlake: In pure mode, check that the argument is an immutable flakeref 2020-04-03 13:07:05 +02:00
Silvan Mosberger c34e96f7e0
Make function arguments retain position info
This allows querying the location of function arguments. E.g.

  builtins.unsafeGetAttrPos "x" (builtins.functionArgs ({ x }: null))

  => { column = 57; file = "/home/infinisil/src/nix/inst/test.nix"; line = 1; }
2020-04-02 05:52:52 +02:00
Eelco Dolstra d15d91cad1 Makefile cleanup 2020-03-30 16:50:13 +02:00
Eelco Dolstra e322a16523 Remove global -I flags
(cherry picked from commit 2c692a3b14)
2020-03-30 15:30:19 +02:00
Eelco Dolstra 2c692a3b14 Remove global -I flags 2020-03-30 14:39:33 +02:00
Bjørn Forsman be84049baf tests/fetchGitSubmodules.sh: more checks 2020-03-29 22:29:58 +02:00
Bjørn Forsman 6864ad7cf5 fetchGit: fix submodule output attribute
Before this change it would be false for all evaluations but the first.
Now it follows the input argument (as it should).
2020-03-29 22:29:58 +02:00
Bjørn Forsman 587e259bfd tests/fetchGitSubmodules: add more tests 2020-03-29 22:29:58 +02:00
Julian Stecklina 6c00a9545f Fix typo in submodule test 2020-03-29 22:29:58 +02:00
Julian Stecklina 435366ed3c Rename fetchGit fetchSubmodules to just submodules 2020-03-29 22:29:58 +02:00
Julian Stecklina c846abb5cc Add more test for git submodule functionality 2020-03-29 22:29:58 +02:00
Julian Stecklina c8d33de777 Add git submodule fixes from @bjornfor
This fixes fetching repositories with no submodules and also cleans up
.git files in checkouts.
2020-03-29 22:29:57 +02:00
Julian Stecklina c2a24c2b88 Add test for fetchGit submodule support 2020-03-29 22:29:57 +02:00
Eelco Dolstra bf0b7e5423 Add test for circular flake dependencies 2020-03-27 22:03:40 +01:00
Eelco Dolstra 015f8f1c13 Improve lock file generation
This is now done in a single pass. Also fixes some issues when
updating flakes with circular dependencies. Finally, when using
'--recreate-lock-file --commit-lock-file', the commit message now
correctly shows the differences.
2020-03-27 21:08:41 +01:00
Eelco Dolstra 0a10854f85 Misc changes from the flakes branch 2020-03-24 14:34:47 +01:00
Eelco Dolstra 7a8de57d3e Pretty-print 'nix why-depends' / 'nix-store -q --tree' output
Extracted from 678301072f.
2020-03-24 14:26:23 +01:00
Eelco Dolstra 4e67f89f38 Fix GitHub VM test 2020-03-20 15:03:18 +01:00
Eelco Dolstra 8aa354fdfd
Register flake-registry.json as a GC root again 2020-03-19 11:45:34 +01:00
Eelco Dolstra 1c127e6a82
downloadFile(): Use expired file if the download fails 2020-03-19 11:42:50 +01:00
Eelco Dolstra c5ec95e2c7
tarball.cc: Use ETags 2020-03-18 15:14:23 +01:00
Eelco Dolstra 38e360154d
Git: Use unified caching system 2020-03-17 22:35:29 +01:00
Eelco Dolstra d1165d8791
Require shallow clones to be requested explicitly
If you do a fetchTree on a Git repository, whether the result contains
a revCount attribute should not depend on whether that repository
happens to be a shallow clone or not. That would complicate caching a
lot and would be semantically messy. So applying fetchTree/fetchGit to
a shallow repository is now an error unless you pass the attribute
'shallow = true'. If 'shallow = true', we don't return revCount, even
if the repository is not actually shallow.

Note that Nix itself is not doing shallow clones at the moment. But it
could do so as an optimisation if the user specifies 'shallow = true'.

Issue #2988.
2020-03-17 22:35:29 +01:00
Eelco Dolstra 2a4e4f6a6e
Unified fetcher caching system 2020-03-17 22:35:29 +01:00
Eelco Dolstra fbcb897e21
Add a test for shallow Git clones
Also, don't return a revCount anymore for shallow or dirty Git trees,
since it's incorrect.

Closes #2988.
2020-03-16 13:20:32 +01:00
Eelco Dolstra 34c7645a58
Fix re-running the fetchGit.sh test 2020-03-16 12:30:11 +01:00
Eelco Dolstra ae9119167e Change the lock file to a graph
This enables support for cycles between flakes.
2020-03-12 22:06:57 +01:00
Eelco Dolstra 35f6651735 Merge remote-tracking branch 'origin/master' into flakes 2020-03-11 17:03:38 +01:00
Eelco Dolstra 9950cdec35 Move some corepkgs into the nix binary 2020-03-11 16:57:48 +01:00
Eelco Dolstra 73769b28e3 Move calling flakes into a Nix helper function (call-flake.nix) 2020-03-09 15:28:41 +01:00
Eelco Dolstra 73b6d87e17
Merge remote-tracking branch 'origin/master' into flakes 2020-03-04 13:58:42 +01:00
Eelco Dolstra d700eecea9
Add test for foldl' 2020-03-04 11:43:48 +01:00
Eelco Dolstra 890df325c7
fetchTree: Use a feature flag 2020-02-20 13:36:16 +01:00
Maximilian Bosch c169ea5904
builtins.fetchGit: Fix build when fetching a git worktree
Worktrees[1] are a feature of git which allow you to check out a ref in
a different directory.

While playing around with flakes I realized that git repositories in a
worktree checkout break when trying to build a flake:

```
$ git worktree add ../nixpkgs-flakes nixpkgs-flakes
$ cd ../nixpkgs-flakes
$ nix build .#hello
error: opening directory '/home/ma27/Projects/nixpkgs-flakes/.git/refs/heads': Not a directory
```

This issue has been fixed by determining with `git rev-parse --git-common-dir`
where the actual `.git` directory is.

Please note that this issue only exists on the `flakes` branch, fetching
worktree checkouts with Nix master seems to work fine.

[1] https://git-scm.com/docs/git-worktree
2020-02-19 14:00:36 +01:00
Eelco Dolstra 9659940659 Test narHash mismatch 2020-02-12 10:33:52 +01:00
Eelco Dolstra 26dacc0983 Add fetchTree builtin function
This allows all supported fetchers to be used, e.g.

  builtins.fetchTree {
    type = "github";
    owner = "NixOS";
    repo = "nix";
    rev = "d4df99a3349cf2228a8ee78dea320afef86eb3ba";
  }
2020-02-11 23:53:46 +01:00
Eelco Dolstra 379852a152 Registry: Use attr notation instead of URLs 2020-02-06 14:27:31 +01:00
Eelco Dolstra 9d7fb62db6 Add option --commit-lock-file 2020-02-05 14:48:49 +01:00
Eelco Dolstra d5334c466b
Automatically do git/hg add on flake.lock 2020-02-02 16:33:22 +01:00
Eelco Dolstra a9ebc3ea5d
Remove the git+ and hg+ prefixes from structured input refs 2020-02-02 13:06:00 +01:00
Eelco Dolstra 90ada8e31a
--tarball-ttl 0 -> --refresh 2020-02-02 12:47:21 +01:00
Eelco Dolstra 94a94da075
Substitute flake inputs
This improves reproducibility and may be faster than fetching from the
original source (especially for git/hg inputs, but probably also for
github inputs - our binary cache is probably faster than GitHub's
dynamically generated tarballs).

Unfortunately this doesn't work for the top-level flake since even if
we know the NAR hash of the tree, we don't know the other tree
attributes such as revCount and lastModified.

Fixes #3253.
2020-02-02 12:41:23 +01:00
Eelco Dolstra b270869466
Renamed ref / resolvedRef -> lockedRef 2020-02-02 00:06:59 +01:00
Eelco Dolstra 5d70b454be
nix flake update: Imply --refresh 2020-02-01 12:30:49 +01:00
Eelco Dolstra 9640208d00
Fix test
https://hydra.nixos.org/build/111166467
2020-02-01 09:30:55 +01:00