Commit graph

1465 commits

Author SHA1 Message Date
Maximilian Bosch 159b5815b5
repl: --option pure-eval true actually enables pure eval mode
To quote Eelco in #5867:

> Unfortunately we can't do
>
>     evalSettings.pureEval.setDefault(false);
>
> because then we have to do the same in main.cc (where
> pureEval is set to true), and that would allow pure-eval
> to be disabled globally from nix.conf.

Instead, a command should specify that it should be impure by
default. Then, `evalSettings.pureEval` will be set to `false;` unless
it's overridden by e.g. a CLI flag.

In that case it's IMHO OK to be (theoretically) able to override
`pure-eval` via `nix.conf` because it doesn't have an effect on commands
where `forceImpureByDefault` returns `false` (i.e. everything where pure
eval actually matters).

Closes #5867
2022-05-31 22:20:11 +02:00
Cole Helbling dbf0d3a150 tests/nss-preload: move nix-fetch binding 2022-05-31 12:14:34 -07:00
Eelco Dolstra 27ebb97d0a
Handle EOFs in string literals correctly
We can't return a STR token without setting a valid StringToken,
otherwise the parser will crash.

Fixes #6562.
2022-05-25 17:58:13 +02:00
tomberek 8c3939af14
Apply suggestions from code review
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-05-20 12:09:41 -04:00
Tom Bereknyei 7a04fb1c56 repl: add repl-flake experimental feature for gating 2022-05-20 08:20:00 -04:00
Tom Bereknyei 0053dab43f repl: fix tests to run on any testing store 2022-05-20 08:03:41 -04:00
Théophane Hufschmitt 938150472d Add some tests for the new REPL cli
- Test that without the XP feature things work as before
- Test that with or without the XP feature `--file file` works
- Test that with XP feature passing a flakeref works
- Test `:reload` with a flake
2022-05-20 07:42:06 -04:00
Tony Olagbaiye 5b8c1deb18 fetchTree: Allow fetching plain files
Add a new `file` fetcher type, which will fetch a plain file over
http(s), or from the local file.

Because plain `http(s)://` or `file://` urls can already correspond to
`tarball` inputs (if the path ends-up with a know archive extension),
the URL parsing logic is a bit convuluted in that:

- {http,https,file}:// urls will be interpreted as either a tarball or a
  file input, depending on the extensions of the path part (so
  `https://foo.com/bar` will be a `file` input and
  `https://foo.com/bar.tar.gz` as a `tarball` input)
- `file+{something}://` urls will be interpreted as `file` urls (with
  the `file+` part removed)
- `tarball+{something}://` urls will be interpreted as `tarball` urls (with
  the `tarball+` part removed)

Fix #3785

Co-Authored-By: Tony Olagbaiye <me@fron.io>
2022-05-19 18:24:49 +02:00
Théophane Hufschmitt 78dc64ec1e
Merge pull request #6544 from ncfavier/getFlake-no-write-lock-file
Do not attempt to write a lock file in builtins.getFlake
2022-05-18 16:41:13 +02:00
Naïm Favier 169384abb2
Do not attempt to write a lock file in builtins.getFlake
Fixes https://github.com/NixOS/nix/issues/6541
2022-05-18 15:45:06 +02:00
Eli Kogan-Wang be2b19041e Integrate review changes 2022-05-13 22:02:28 +02:00
Eli Kogan-Wang aefc6c4f41 Add priority for nix profile install 2022-05-11 12:16:35 +02:00
Eelco Dolstra f4102de84b
Merge pull request #6489 from alexshpilkin/fix-6488
nix: Add forgotten null check in AttrCursor::getListOfStrings()
2022-05-05 16:16:56 +02:00
Théophane Hufschmitt 4c5aa8520c Make sure that nix build works in --impure mode
Regression test for <https://github.com/NixOS/nix/issues/6488>
2022-05-05 14:53:59 +02:00
thkoch2001 7388cd55bf Change json example to be original
Closes: #3391
2022-05-05 12:39:23 +03:00
Eelco Dolstra a3c6c5b1c7 nix profile: Support overriding outputs 2022-05-03 15:00:34 +02:00
Eelco Dolstra 4a79cba511 Allow selecting derivation outputs using 'installable!outputs'
E.g. 'nixpkgs#glibc^dev,static' or 'nixpkgs#glibc^*'.
2022-05-03 13:43:52 +02:00
Eelco Dolstra 404c222444
Merge pull request #6426 from edolstra/respect-outputs-to-install
nix: Respect meta.outputsToInstall, and use all outputs by default
2022-05-03 13:43:22 +02:00
Théophane Hufschmitt cae7473884
Merge pull request #6465 from layus/fix-test-crashes
tests: Distinguish crashes from expected failures
2022-05-03 11:11:37 +02:00
Guillaume Maudoux 275f8561eb tests/lang: Distinguish crashes from expected failures 2022-05-02 15:12:50 +02:00
Guillaume Maudoux 4845886bed Revert "tests: Distinguish crashes from expected failures"
This reverts commit 143b73f52d.
2022-05-02 14:22:00 +02:00
Eelco Dolstra a26be9f3b8
Merge pull request #4638 from orbekk/read_head
Resolve reference for remote git repositories (makes fetchGit work with non-'master' branch)
2022-05-02 13:31:30 +02:00
Eelco Dolstra dde71899dd tests: Don't create tests/result
https://hydra.nixos.org/log/lns780srkka4dv7r69mn4zfy6fdij4yr-nix-2.9.0pre20220428_4bb111c.drv
2022-05-02 09:38:51 +02:00
Kjetil Orbekk 05a3fbac5a Test fetchGit with non-'master' remote repo 2022-04-29 18:42:31 -04:00
Kjetil Orbekk 401e60f289 Resolve reference for remote repository
Resolves the HEAD reference from the remote repository instead
of assuming "master".
2022-04-29 18:42:28 -04:00
Guillaume Maudoux 143b73f52d tests: Distinguish crashes from expected failures 2022-04-29 11:14:08 +02:00
Eelco Dolstra 4a9623b129 Fix passing $OUT_PATHS to the post-build hook
Fixes #6446.
2022-04-28 13:36:01 +02:00
Eelco Dolstra 6e0a2b971b Add a test for outputsToInstall 2022-04-26 17:17:27 +02:00
Eelco Dolstra 1ddabe1a01 nix: Respect meta.outputsToInstall, and use all outputs by default
'nix profile install' will now install all outputs listed in the
package's meta.outputsToInstall attribute, or all outputs if that
attribute doesn't exist. This makes it behave consistently with
nix-env. Fixes #6385.

Furthermore, for consistency, all other 'nix' commands do this as
well. E.g. 'nix build' will build and symlink the outputs in
meta.outputsToInstall, defaulting to all outputs. Previously, it only
built/symlinked the first output. Note that this means that selecting
a specific output using attrpath selection (e.g. 'nix build
nixpkgs#libxml2.dev') no longer works. A subsequent PR will add a way
to specify the desired outputs explicitly.
2022-04-26 17:17:27 +02:00
Théophane Hufschmitt be28603dca Merge remote-tracking branch 'origin/master' into nixbuildaddprintstorepaths 2022-04-22 11:11:01 +02:00
Théophane Hufschmitt c4ffc8e2f8
Merge pull request #6218 from pennae/pos-symbol-tables
reduce the size of Attr from 3 pointers to 2 on 64 bit machines
2022-04-22 10:28:06 +02:00
Théophane Hufschmitt 35ca5fdf91
Merge pull request #6436 from flox/tofile_allow
fix: builtins.toFile adds path to allowedPaths
2022-04-22 08:50:54 +02:00
Tom Bereknyei f25112d383 fix: builtins.toFile adds path to allowedPaths
The produced path is then allowed be imported or utilized elsewhere:
```
assert (43 == import (builtins.toFile "source" "43")); "good"
```

This will still fail on write-only stores.
2022-04-21 16:41:37 -04:00
pennae 6526d1676b replace most Pos objects/ptrs with indexes into a position table
Pos objects are somewhat wasteful as they duplicate the origin file name and
input type for each object. on files that produce more than one Pos when parsed
this a sizeable waste of memory (one pointer per Pos). the same goes for
ptr<Pos> on 64 bit machines: parsing enough source to require 8 bytes to locate
a position would need at least 8GB of input and 64GB of expression memory. it's
not likely that we'll hit that any time soon, so we can use a uint32_t index to
locate positions instead.
2022-04-21 21:46:06 +02:00
Sergei Trofimovich b6e59d7137 tests: remove 'ca-references' feature
The feature was ctabilized in d589a6aa8a.
2022-04-21 10:06:39 +02:00
regnat 74d6782a6a Disable the selfref-gc test when the daemon is too old 2022-04-21 10:06:39 +02:00
Sergei Trofimovich 6ada496311 nix: add (failing) selfreference test for multiple realizations
The test illustrates failure in issue #5320. Here derivation and
it's built input have identical CA sotre path. As a result we generate
extraneout reference to build input:

    $ make installcheck
    ...
    ran test tests/selfref-gc.sh... [PASS]
    ran test tests/ca/selfref-gc.sh... [FAIL]
    ...
        deleting '/tmp/.../tests/ca/selfref-gc/store/iqciq1mpg5hc7p6a52fp2bjxbyc9av0v-selfref-gc'
        deleting '/tmp/...tests/ca/selfref-gc/store/zh0kwpnirw3qbv6dl1ckr1y0kd5aw6ax-selfref-gc.drv'
        error: executing SQLite statement
          'delete from ValidPaths where path = '/tmp/.../tests/ca/selfref-gc/store/fsjq0k146r85lsh01l0icl30rnhv7z72-selfref-gc';':
            constraint failed (in '/tmp/.../tests/ca/selfref-gc/var/nix/db/db.sqlite')
2022-04-21 10:06:39 +02:00
Artturin 51cfea8bb0 nix build: add --print-out-paths flag
has the same functionality as default nix-build

$ nix-build . -A "bash" -A "bash.dev" -A "tinycc"
/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12
/nix/store/c49i1ggnr5cc8gxmk9xm0cn961z104dn-bash-5.1-p12-dev
/nix/store/dbapb08862ajgaax3621fz8hly9fdah3-tcc-0.9.27+date=2022-01-11

$ nix-build . -A "bash"
/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12

$ $HOME/nixgits/nix/result/bin/nix build "nixpkgs#bash" "nixpkgs#bash.dev" "nixpkgs#tinycc" --print-out-paths
/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12
/nix/store/c49i1ggnr5cc8gxmk9xm0cn961z104dn-bash-5.1-p12-dev
/nix/store/dbapb08862ajgaax3621fz8hly9fdah3-tcc-0.9.27+date=2022-01-11

$ $HOME/nixgits/nix/result/bin/nix build "nixpkgs#bash" --print-out-paths
/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12
2022-04-20 19:35:46 +03:00
mei (ckie) 0e2b01b14e
nix repl: make symlinks with the :bl command
Requested by ppepino on the Matrix:
https://matrix.to/#/!KqkRjyTEzAGRiZFBYT:nixos.org/$Tb32BS3rVE2BSULAX4sPm0h6CDewX2hClOTGzTC7gwM?via=nixos.org&via=matrix.org&via=nixos.dev

This adds a new command, :bl, which works like :b but also creates
a GC root symlink to the various derivation outputs.

ckie@cookiemonster ~/git/nix -> ./outputs/out/bin/nix repl
Welcome to Nix 2.6.0. Type :? for help.

nix-repl> :l <nixpkgs>
Added 16118 variables.

nix-repl> :b runCommand "hello" {} "echo hi > $out"

This derivation produced the following outputs:
  ./repl-result-out -> /nix/store/kidqq2acdpi05c4a9mlbg2baikmzik44-hello
[1 built, 0.0 MiB DL]
ckie@cookiemonster ~/git/nix -> cat ./repl-result-out
hi
2022-04-20 00:20:29 +03:00
Eelco Dolstra a3c843e635 Fix 'nix fmt' test 2022-04-19 21:47:13 +02:00
Eelco Dolstra c9e58aa5ff Require formatters to be packages
Because of 9b41239d8f, a formatter can
no longer be a package *or* an app. So let's require it to be a
package for now.
2022-04-19 20:48:13 +02:00
Eelco Dolstra 48a467f2b9 Merge branch 'issue-6075' of https://github.com/kamadorueda/nix 2022-04-19 20:21:32 +02:00
Eelco Dolstra 2016b7142a Fix compilation, style fixes 2022-04-19 13:41:32 +02:00
Eelco Dolstra b529a41814 Merge branch 'make-flake-show-more-lenient-on-apps' of https://github.com/flox/nix 2022-04-19 13:41:24 +02:00
Tom Bereknyei 9b41239d8f fix: ensure apps are apps and packages are packages 2022-04-14 23:57:52 -04:00
Daniel Pauls 38125a47ab Test fetchMercurial with path containing a . segment 2022-04-09 23:39:00 +02:00
Théophane Hufschmitt 305d3a0ec3 Test fetchgit with path containing a . segment 2022-04-07 17:31:12 +02:00
Eelco Dolstra 589f6f267b fetchClosure: Don't allow URL query parameters
Allowing this is a potential security hole, since it allows the user
to specify parameters like 'local-nar-cache'.
2022-04-06 11:52:51 +02:00
Théophane Hufschmitt 5abe3f4aa6 Allow welcomeText when checking a flake template
Fix https://github.com/NixOS/nix/issues/6321
2022-04-05 11:03:43 +02:00
Eelco Dolstra fdfe737867 Fix handling of outputHash when outputHashAlgo is not specified
https://hydra.nixos.org/build/171351131
2022-04-01 12:40:49 +02:00
Eelco Dolstra 6377442c98 tests/impure-derivations.sh: Ensure that inputAddressed build fails 2022-03-31 17:38:15 +02:00
Eelco Dolstra 7537097284 Provide default values for outputHashAlgo and outputHashMode 2022-03-31 16:56:44 +02:00
Eelco Dolstra 162beb2595 Fix test 2022-03-31 13:43:20 +02:00
Eelco Dolstra b2ae922747 tests/impure-derivations.sh: Restart daemon 2022-03-31 13:43:20 +02:00
Eelco Dolstra 18935e8b9f Support fixed-output derivations depending on impure derivations 2022-03-31 13:43:20 +02:00
Eelco Dolstra 5cd72598fe Add support for impure derivations
Impure derivations are derivations that can produce a different result
every time they're built. Example:

  stdenv.mkDerivation {
    name = "impure";
    __impure = true; # marks this derivation as impure
    outputHashAlgo = "sha256";
    outputHashMode = "recursive";
    buildCommand = "date > $out";
  };

Some important characteristics:

* This requires the 'impure-derivations' experimental feature.

* Impure derivations are not "cached". Thus, running "nix-build" on
  the example above multiple times will cause a rebuild every time.

* They are implemented similar to CA derivations, i.e. the output is
  moved to a content-addressed path in the store. The difference is
  that we don't register a realisation in the Nix database.

* Pure derivations are not allowed to depend on impure derivations. In
  the future fixed-output derivations will be allowed to depend on
  impure derivations, thus forming an "impurity barrier" in the
  dependency graph.

* When sandboxing is enabled, impure derivations can access the
  network in the same way as fixed-output derivations. In relaxed
  sandboxing mode, they can access the local filesystem.
2022-03-31 13:43:20 +02:00
Théophane Hufschmitt 8dee15cd31 Don’t create a file in the worktree in the fetchPath test 2022-03-30 11:42:47 +02:00
Eelco Dolstra 16cf1e6089
Merge pull request #6311 from edolstra/return-wanted-paths
Make buildPathsWithResults() only return info on wanted outputs
2022-03-25 15:44:39 +01:00
Eelco Dolstra 187dc080a2 tests/build.sh: Test that 'nix build' only prints wanted outputs 2022-03-24 23:36:14 +01:00
Maximilian Bosch d02e34ef06
Implement regression test for empty logs loaded via nix log 2022-03-24 22:31:52 +01:00
Eelco Dolstra f902f3c2cb Add experimental feature 'fetch-closure' 2022-03-24 21:33:33 +01:00
Eelco Dolstra 28186b7044 Add a test for fetchClosure and 'nix store make-content-addressed' 2022-03-24 21:33:33 +01:00
Eelco Dolstra a0259a21a4 Don't hide repeated values while generating manifest.nix
Fixes #6243.
2022-03-22 13:18:56 +01:00
Gabriel Fontes 31544b93ff
Fix sourcehut integration test
The new implementation relies on tab separting the hash and ref (this is how sourcehut does it). This fixes the integration test to use a tab instead of a space.
2022-03-19 11:38:45 -03:00
Eelco Dolstra 6afc361798
Merge pull request #6277 from thufschmitt/ca/nix-build-dry-run
Fix `nix build --dry-run` with CA derivations
2022-03-17 22:23:15 +01:00
Théophane Hufschmitt 7117053457
Merge pull request #6270 from Artturin/stdineval
nix: allow using --file - to read from stdin
2022-03-17 14:35:01 +01:00
Théophane Hufschmitt 3fc4c612fb Fix nix build --dry-run with CA derivations
Don’t try and assume that we know the output paths when we’ve just built
with `--dry-run`. Instead make `--dry-run` follow a different code path
that won’t assume the knowledge of the output paths at all.

Fix #6275
2022-03-17 11:55:19 +01:00
Théophane Hufschmitt a0b517de57
Merge pull request #6242 from ncfavier/print-output-names
nix-env: always print output names in JSON and XML
2022-03-17 10:55:22 +01:00
Théophane Hufschmitt c20e07763d
Add some tests for nix-env -q --json 2022-03-16 21:33:38 +01:00
Artturin 4f8ad41d4e add tests for nix eval and nix-instantiate 2022-03-16 21:01:51 +02:00
Théophane Hufschmitt 3cea6f569e
Fix the date in the comment of fetchPath’s test
Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>
2022-03-16 08:56:01 +01:00
regnat 2d5c43f210 Fix the tests on 32bits machines
year 2222 is too much for a 32 bit timestamp. So replace it by something
smaller
2022-03-15 21:05:01 +01:00
Maximilian Bosch 975bade7f0
Implement simple test for path-fetcher setting a correct lastModifiedDate
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-03-15 12:55:32 +01:00
Kevin Amado 2191dab657
nix-fmt: add command 2022-03-11 10:00:19 -05:00
Eelco Dolstra aee56e0f89 Merge remote-tracking branch 'origin/eval-suggestions' 2022-03-11 12:02:26 +01:00
Théophane Hufschmitt 7ebd6f1093
Merge pull request #5758 from mschwaig/fix-git-workspace-dirty-detection
git fetcher: fix detection of dirty git workspaces
2022-03-11 09:35:01 +01:00
regnat f6078e474d Also display some suggestions for invalid formal arguments
```console
$ nix eval --expr '({ foo ? 1 }: foo) { fob = 2; }'
error: anonymous function at (string):1:2 called with unexpected argument 'fob'

       at «string»:1:1:

            1| ({ foo ? 1 }: foo) { fob = 2; }
             | ^
       Did you mean foo?
```

Not that because Nix will first check for _missing_ arguments before
checking for extra arguments, `({ foo }: foo) { fob = 1; }` will
complain about the missing `foo` argument (rather than extra `fob`) and
so won’t display a suggestion.
2022-03-08 16:40:22 +01:00
regnat 0c6e46e349 Add some suggestions to the evaluator
Make the evaluator show some suggestions when trying to access an
invalid field from an attrset.

```console
$ nix eval --expr '{ foo = 1; }.foa'
error: attribute 'foa' missing

       at «string»:1:1:

            1| { foo = 1; }.foa
             | ^
       Did you mean foo?
```
2022-03-08 06:21:45 +01:00
Eelco Dolstra 30ddd37873 Merge branch 'cli-suggestions' of https://github.com/thufschmitt/nix 2022-03-07 19:47:45 +01:00
regnat 91635206c0 Add some end-to-end tests for the suggestions 2022-03-07 10:09:10 +01:00
Théophane Hufschmitt a2ace54fe4
Merge pull request #6029 from Ma27/nix-log-ssh-ng
ssh-ng: also store build logs to make them accessible by `nix log`
2022-03-07 09:51:40 +01:00
Théophane Hufschmitt 860016bcbf
Explain why the log tests are disabled for CA derivations 2022-03-07 09:15:34 +01:00
Maximilian Bosch 697d1dac01
tests: grep for string in nix log for remote-builds 2022-03-04 16:58:27 +01:00
Eelco Dolstra d4538034b7 nix profile test: Restart daemon
Fixes

  nix-daemon: src/libstore/sqlite.cc:97: nix::SQLiteStmt::Use::Use(nix::SQLiteStmt&): Assertion `stmt.stmt' failed.

which happens because the daemon doesn't properly handle the case
where ca-derivations isn't enabled at daemon startup.
2022-03-02 23:08:49 +01:00
Eelco Dolstra 3a3821bcd7 Remove obsolete todo 2022-03-02 22:22:55 +01:00
Eelco Dolstra b0d65b3d11 Silence kill output 2022-03-02 22:22:23 +01:00
Eelco Dolstra d2586188fe tests/common.sh.in: Add enableFeatures helper 2022-03-02 21:48:25 +01:00
Eelco Dolstra f9375778ae nix profile: Add a test for non-flake packages 2022-03-02 20:56:40 +01:00
Eelco Dolstra 161f798aa1 nix profile: Support CA derivations 2022-03-02 20:38:51 +01:00
Eelco Dolstra 5850fd17b4 Add basic tests for 'nix profile'
Fixes #6193.
2022-03-02 14:40:46 +01:00
Eelco Dolstra c10865a46e tests: Rename nix-profile.sh -> bash-profile.sh 2022-03-02 11:21:00 +01:00
Robert Hensing 158280d8e9 fetchTree: Do not re-fetch paths already present 2022-02-28 21:37:49 +01:00
Maximilian Bosch 287642f132
tests: implement test for nix log with ssh-ng:// builds
A few notes:

* The `echo hi` is needed to make sure that a file that can be read by
  `nix log` is properly created (i.e. some output is needed). This is
  known and to be fixed in #6051.
* We explicitly ignore the floating-CA case here: the `$out` of `input3`
  depends on `$out` of `input2`. This means that there are actually two
  derivations - I assume that this is because at eval time (i.e.
  `nix-instantiate -A`) the hash of `input2` isn't known yet and the
  other .drv is created as soon as `input2` was built. This is another
  issue on its own, so we ignore the case here explicitly.
2022-02-28 17:27:52 +01:00
regnat b8f8aef9d3 tests: Fix the start of the daemon
- Make sure that it starts even without the `nix-command` xp feature
- Fail if it doesn’t manage to start

This fixes a 30s wait for every test in `init.sh` as the daemon couldn’t
start, but the code was just waiting 30s and continuing as if everything
was all right.
2022-02-28 17:00:31 +01:00
Eelco Dolstra 59683733f5
Merge pull request #6089 from edolstra/dot-default
Replace defaultBla.$system with bla.$system.default
2022-02-24 21:16:58 +01:00
Eelco Dolstra 7ec244aec2
Merge pull request #6159 from NixOS/more-eager-daemon-polling-in-tests
testS: poll more eagerly for the daemon start/stop
2022-02-24 17:54:09 +01:00
regnat 9c470cb969 testS: poll more eagerly for the daemon start/stop
Polling every 1 second means that even the simplest test takes at least
2 seconds. We can reasonably poll 1/10 of that to make things much
  quicker (esp. given that most of the time 0.1s is enough for the
  daemon to be started or stopped)
2022-02-24 15:00:23 +01:00
regnat 42766f8924 Sort the tests by wall time
The tests are scheduled in the order they appear, so running the long
ones first slightly improves the scheduling.

On my machine, this decreases the time of `make install` from 40s to 36s
2022-02-24 14:32:31 +01:00
Gabriel Fontes 770f3af31d
add sourcehut integration test 2022-02-23 11:58:09 -03:00
Eelco Dolstra 162fbe31ff Replace defaultBla.$system with bla.$system.default
This also simplifies some InstallableFlake logic and fixes 'nix
bundle' parsing its installable twice.

Fixes #5532.
2022-02-22 11:47:41 +01:00
Martin Schwaighofer 9504445cab git fetcher: distinguish errors more precisely 2022-02-19 22:58:23 +01:00
Martin Schwaighofer c7e527b82b git fetcher: invoke diff instead of diff-index
diff-index operates on the view that git has of the working tree,
which might be outdated. The higher-level diff command does this
automatically. This change also adds handling for submodules.

fixes #4140

Alternative fixes would be invoking update-index before diff-index or
matching more closely what require_clean_work_tree from git-sh-setup.sh
does, but both those options make it more difficult to reason about
correctness.
2022-02-19 22:58:23 +01:00
Martin Schwaighofer 0bfa0cdea1 git fetcher: improve check for valid repository
The .git/refs/heads directory might be empty for a valid
usable git repository. This often happens in CI environments,
which might only fetch commits, not branches.
Therefore instead we let git itself check if HEAD points to
something that looks like a commit.

fixes #5302
2022-02-19 22:58:22 +01:00
Eelco Dolstra 93293fc66b
Merge pull request #6042 from pennae/fix-repl-a
fix nix repl not overriding existing bindings in :a
2022-02-06 13:17:58 +01:00
pennae 1daf1babf9 fix nix repl not overriding existing bindings in :a
previously :a would override old bindings of a name with new values if the added
set contained names that were already bound. in nix 2.6 this doesn't happen any
more, which is potentially confusing.

fixes #6041
2022-02-04 15:27:59 +01:00
tomberek 12ff354d01
Merge branch 'master' into bundler_drv 2022-02-03 02:39:18 -05:00
Eelco Dolstra fcb33440b6
Merge pull request #6036 from tweag/balsoft/and-yet-another-follows-bugfix
Flake follows: resolve all follows to absolute
2022-02-02 22:52:08 +01:00
Alexander Bantyev 169ea0b83f
Flake follows: resolve all follows to absolute
It's not possible in general to know in computeLocks, relative to
which path the follows was intended to be. So, we always resolve
follows to their absolute states when we encounter them (which can
either be in parseFlakeInput or computeLocks' fake input population).

Fixes https://github.com/NixOS/nix/issues/6013
Fixes https://github.com/NixOS/nix/issues/5609
Fixes https://github.com/NixOS/nix/issues/5697 (again)
2022-02-02 21:41:45 +03:00
Eelco Dolstra 17e3f353df Merge branch 'parser-improvements' of https://github.com/pennae/nix 2022-02-02 12:45:44 +01:00
tomberek 2bf96bd9f2
Merge branch 'master' into bundler_drv 2022-01-28 10:18:29 -05:00
Tom Bereknyei 73e82ae954 bundler: tests various combinations of referring to installables 2022-01-28 10:17:51 -05:00
Tom Bereknyei 4ebc50d92e bundler: revert default behavior to use defaultApp
Bundlers are now responsible for correctly handling their inputs which
are no longer constrained to be (Drv->Drv)->Drv->Drv, but can be of
type (attrset->Drv)->attrset->Drv.
2022-01-28 09:56:58 -05:00
regnat fcdc60ed22 Don’t require NIX_PATH entries to be valid paths
It’s totally valid to have entries in `NIX_PATH` that aren’t valid paths
(they can even be arbitrary urls or `channel:<channel-name>`).

Fix #5998 and #5980
2022-01-27 16:26:39 +01:00
regnat 3dc1418216 Fix the store ping test 2022-01-26 14:15:03 +01:00
regnat d139474f48 Add some tests for nix store ping
Always good to have :)
2022-01-26 11:01:25 +01:00
regnat f113ea6c73 Fix parsing of variable names that are a suffix of '__curPos'
Follow-up from #5969
Fix #5982
2022-01-25 10:49:27 +01:00
Tom Bereknyei 93299efc7c bundler: add tests and change defaults to use a derivation 2022-01-25 03:39:18 -05:00
Eelco Dolstra bed8270c0c Fix parsing of variable names that are a prefix of '__curPos'
Fixes

  $ nix-instantiate --parse -E 'x: with x; _'
  (x: (with x; __curPos))
2022-01-24 15:18:18 +01:00
Eelco Dolstra 3ff6186af1 Add a test for nix-assertion failures in nix-env 2022-01-24 14:49:32 +01:00
Théophane Hufschmitt 4530574363
Merge pull request #5945 from afishhh/master
Make `nix search` highlight all regexes and matches
2022-01-24 13:02:51 +01:00
Eelco Dolstra 5c3b6fbe00
Merge pull request #5963 from FlorianFranzen/test-flake-git
tests: skip flake search test if no git is present
2022-01-24 12:45:49 +01:00
Eelco Dolstra 076945c808
Merge pull request #5875 from hercules-ci/fix-large-drv-field-stack-overflow
Fix segfault or stack overflow caused by large derivation fields
2022-01-24 12:05:30 +01:00
Florian Franzen 12578ba672 tests: skip flake search test if no git is present 2022-01-22 20:13:16 +01:00
Eelco Dolstra c7223db871
Merge pull request #5942 from NixOS/5912-quieter-nix-why-depends
Make `nix why-depends` quieter by default
2022-01-21 10:18:28 +01:00
Hubert Głuchowski b03fe13b5b
Add some tests for multiple regex/match highlighting in nix search 2022-01-20 18:18:07 +01:00
pennae 7d4cc5515c defer formals duplicate check for incresed efficiency all round
if we defer the duplicate argument check for lambda formals we can use more
efficient data structures for the formals set, and we can get rid of the
duplication of formals names to boot. instead of a list of formals we've seen
and a set of names we'll keep a vector instead and run a sort+dupcheck step
before moving the parsed formals into a newly created lambda. this improves
performance on search and rebuild by ~1%, pure parsing gains more (about 4%).

this does reorder lambda arguments in the xml output, but the output is still
stable. this shouldn't be a problem since argument order is not semantically
important anyway.

 before

  nix search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      8.550 s ±  0.060 s    [User: 6.470 s, System: 1.664 s]
    Range (min … max):    8.435 s …  8.666 s    20 runs

  nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     346.7 ms ±   2.1 ms    [User: 312.4 ms, System: 34.2 ms]
    Range (min … max):   343.8 ms … 353.4 ms    20 runs

  nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.720 s ±  0.031 s    [User: 2.415 s, System: 0.231 s]
    Range (min … max):    2.662 s …  2.780 s    20 runs

 after

  nix search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      8.462 s ±  0.063 s    [User: 6.398 s, System: 1.661 s]
    Range (min … max):    8.339 s …  8.542 s    20 runs

  nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     329.1 ms ±   1.4 ms    [User: 296.8 ms, System: 32.3 ms]
    Range (min … max):   326.1 ms … 330.8 ms    20 runs

  nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.687 s ±  0.035 s    [User: 2.392 s, System: 0.228 s]
    Range (min … max):    2.626 s …  2.754 s    20 runs
2022-01-19 17:07:29 +01:00
Robert Hensing d038a67bd3 Fix segfault or stack overflow caused by large derivation fields
This removes a dynamic stack allocation, making the derivation
unparsing logic robust against overflows when large strings are
added to a derivation.
Overflow behavior depends on the platform and stack configuration.

For instance, x86_64-linux/glibc behaves as (somewhat) expected:

$ (ulimit -s 20000; nix-instantiate tests/lang/eval-okay-big-derivation-attr.nix)
error: stack overflow (possible infinite recursion)

$ (ulimit -s 40000; nix-instantiate tests/lang/eval-okay-big-derivation-attr.nix)
error: expression does not evaluate to a derivation (or a set or list of those)

However, on aarch64-darwin:

$ nix-instantiate big-attr.nix                                                                                                                                                                                                                                                       ~
zsh: segmentation fault  nix-instantiate big-attr.nix

This indicates a slight flaw in the single stack protection page
approach that is not encountered with normal stack frames.
2022-01-19 15:21:56 +01:00
regnat dd7c2e0695 Make nix why-depends quieter by default
Unless `--precise` is passed, make `nix why-depends` only show the
dependencies between the store paths, without introspecting them to
find the actual references.

This also makes it ~3x faster
2022-01-19 14:24:14 +01:00
Eelco Dolstra 5fe1ec8a05 Add a test for 'nix store copy-log' and 'nix log' 2022-01-18 17:30:50 +01:00
regnat 2ad2678c0b Add a simple test for nix why-depends 2022-01-18 16:54:53 +01:00
Théophane Hufschmitt dc4730ee94
Merge pull request #5720 from tomberek/flake_search
flakes: search up to git or filesystem boundary
2022-01-18 10:59:12 +01:00
Alexander Bantyev b9f5dccdbe
Check that we don't search past a git repo 2022-01-14 18:03:47 +03:00
Alexander Bantyev e3690ab393
Add more tests for flake upward searching 2022-01-14 17:21:18 +03:00
Eelco Dolstra 8a446aff75
Merge pull request #5898 from layus/repair-path-links
Make --repair-path also repair corrupt optimised links
2022-01-11 14:14:44 +01:00
Guillaume Maudoux e9a4abdb5d Make --repair-path also repair corrupt optimised links
There already existed a smoke test for the link content length,
but it appears that there exists some corruptions pernicious enough
to replace the file content with zeros, and keeping the same length.

--repair-path now goes as far as checking the content of the link,
making it true to its name and actually repairing the path for such
coruption cases.
2022-01-11 11:57:45 +01:00
Eelco Dolstra f39d94a55b Merge branch 'readfile' of https://github.com/abbradar/nix 2022-01-10 16:55:28 +01:00
Nikolay Amiantov 120ca245d1 Add test for readFile keeping context 2022-01-09 13:42:36 +03:00
Eelco Dolstra 3ec02c6743
Merge pull request #5546 from NixOS/allow-paths-in-flake-local-options
allow paths in flake local options
2022-01-06 21:55:20 +01:00
Eelco Dolstra 263a8d293c Remove non-method mk<X> functions 2022-01-04 18:40:39 +01:00
pennae 00c993f48b add zipAttrsWith primop
nixpkgs can save a good bit of eval memory with this primop. zipAttrsWith is
used quite a bit around nixpkgs (eg in the form of recursiveUpdate), but the
most costly application for this primop is in the module system. it improves
the implementation of zipAttrsWith from nixpkgs by not checking an attribute
multiple times if it occurs more than once in the input list, allocates less
values and set elements, and just avoids many a temporary object in general.

nixpkgs has a more generic version of this operation, zipAttrsWithNames, but
this version is only used once so isn't suitable for being the base of a new
primop. if it were to be used more we should add a second primop instead.
2022-01-03 21:05:53 +01:00
Eelco Dolstra 96d08fcd66
Merge pull request #5839 from tweag/balsoft/yet-another-follows-bugfix
flake.cc: computeLocks: Only verify overrides when they could change
2022-01-03 20:51:58 +01:00
Alexander Bantyev 2dc29e0d93
Add a test that nix repl --show-trace actually shows the trace 2021-12-28 15:54:46 +03:00
Alexander Bantyev 2664a216e5
flake.cc: computeLocks: Only verify overrides when they could change
When we check for disappeared overrides, we can get "false positives"
for follows and overrides which are defined in the dependencies of the
flake we are locking, since they are not parsed by
parseFlakeInputs. However, at that point we already know that the
overrides couldn't have possible been changed if the input itself
hasn't changed (since we check that oldLock->originalRef == *input.ref
for the input's parent). So, to prevent this, only perform this check
when it was possible that the flake changed (e.g. the flake we're
locking, or a new input, or the input has changed and mustRefetch ==
true).
2021-12-28 15:30:06 +03:00
tomberek f404189368
Merge branch 'master' into flake_search 2021-12-23 15:22:52 -05:00
regnat dc89dfa7b3 Properly return false on builtins.pathExists /someNonAllowedPath
Follow-up from https://github.com/NixOS/nix/pull/5807 to fix https://github.com/NixOS/nix/pull/5807#issuecomment-1000135394
2021-12-23 10:49:33 +01:00
regnat d90f9d4b99 Fix IFD with CA derivations
Rewrite the string taken by the IFD-like primops to contain the actual
output paths of the derivations rather than the placeholders

Fix #5805
2021-12-21 09:36:50 +01:00
Théophane Hufschmitt 23ea1e46cc
Merge pull request #5047 from symphorien/fix-nix-channel
nix-channel: use nix-env -i --remove-all to upgrade
2021-12-17 10:05:48 +01:00
regnat ab902521b1 Allow paths in flake local settings
Fix #5505
2021-12-17 09:31:13 +01:00
regnat 6f1b3d711e Test the update of multiple channels
Make sure that `nix-channel --update` won’t accidentaly remove one
2021-12-17 09:17:33 +01:00
Silvan Mosberger d9fbe3e208 Add a test case for builtins.groupBy 2021-12-16 19:59:13 +01:00