Commit graph

1906 commits

Author SHA1 Message Date
Roman Frołow 0768c08d99 Typo: change to normal singlequote 2021-11-22 13:37:38 +01:00
Tom Bereknyei 4318ba2ec5 add real path to allowedPaths 2021-11-20 00:25:36 -05:00
Eelco Dolstra 0961c1068a Merge branch 'accept_conf' of https://github.com/tomberek/nix 2021-11-18 12:56:09 +01:00
Eelco Dolstra 0adced4b9e
Merge pull request #5580 from ksonj/fix/non-standard-ssh
Fix detection of scp-style URIs to support non-standard SSH ports
2021-11-18 12:30:29 +01:00
Kalle Jepsen 46d2a5a10b Simplify fix by disallowing / in front of @ to match scp style 2021-11-17 13:49:10 +01:00
Eelco Dolstra d03e89e5d1 Parse '(f x) y' the same as 'f x y'
(cherry picked from commit 5253cb4b68ad248f37b27849c0ebf3614e4f2777)
2021-11-17 09:53:57 +01:00
Kalle Jepsen 6d46b5b609 Fix detection of scp-style URIs to support non-standard SSH ports for git 2021-11-17 08:41:26 +01:00
Eelco Dolstra d7bae52b9d Call functors with both arguments at once
This is not really useful on its own, but it does recover the
'infinite recursion' error message for '{ __functor = x: x; } 1', and
is more efficient in conjunction with #3718.

Fixes #5515.
2021-11-16 22:34:17 +01:00
Eelco Dolstra e41cf8511f Don't hang when calling an attrset
Fixes #5565.
2021-11-16 17:44:19 +01:00
Eelco Dolstra 6463eaca14
Merge pull request #5472 from NixOS/async-realisation-substitution
async realisation substitution
2021-11-16 12:54:20 +01:00
Eelco Dolstra 9fe0343bfd Merge branch 'master' of github.com:NixOS/nix 2021-11-15 18:45:20 +01:00
Eelco Dolstra 671817a858 Simplify lockFlake() a bit 2021-11-15 18:44:27 +01:00
Eelco Dolstra d1d223838b Merge branch 'balsoft/fix-subinputs-not-getting-updated' of https://github.com/tweag/nix 2021-11-15 18:37:13 +01:00
Domen Kožar 164179983e
Merge pull request #5428 from kreisys/add-pos-to-json-type-error
toJSON: report error position for fancier output
2021-11-15 07:57:46 -06:00
Kevin Amado d0e9e18489
toXML: display errors position
- This change applies to builtins.toXML and inner workings
- Proof of concept:
  ```nix
  let e = builtins.toXML e; in e
  ```
- Before:
  ```
  $ nix-instantiate --eval poc.nix
  error: infinite recursion encountered
  ```
- After:
  ```
  $ nix-instantiate --eval poc.nix
  error: infinite recursion encountered

       at /data/github/kamadorueda/nix/poc.nix:1:9:

            1| let e = builtins.toXML e; in e
             |
  ```
2021-11-13 20:33:34 -05:00
Tom Bereknyei 30496af598 Adds an accept-flake-config flag 2021-11-12 09:50:07 -05:00
Alexander Bantyev 07bffe7998
Flakes: refetch the input when a follows disappears
When an input follows disappears, we can't just reuse the old lock
file entries since we may be missing some required ones. Refetch the
input when this happens.

Closes https://github.com/NixOS/nix/issues/5289
2021-11-11 12:05:18 +03:00
Ben Burdette 7e2a3db4eb cleanup 2021-11-09 13:14:49 -07:00
Ben Burdette 885f819922 remove dead code 2021-11-09 11:20:14 -07:00
Eelco Dolstra 67179472df
Merge pull request #5494 from tweag/balsoft/allow-references-in-addPath
Allow references in addPath
2021-11-09 15:57:39 +01:00
Alexander Bantyev 0b005bc9d6
addToStore, addToStoreFromDump: refactor: pass refs by const reference
Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
2021-11-09 12:24:49 +03:00
Eelco Dolstra 736d6ab721
Merge pull request #5504 from NixOS/flake-options-and-daemon
Make the flake options work when using the daemon
2021-11-08 13:54:55 +01:00
Andreas Rammhold 8e7359db64
Remove unused "<let-body>" symbol
The requirement for the symbol has been removed since at least 7d47498.
2021-11-07 18:26:43 +01:00
Alexander Bantyev 9d4dcff37a
addPath: allow paths with references
Since 4806f2f6b0, we can't have paths with
references passed to builtins.{path,filterSource}. This prevents many cases
of those functions called on IFD outputs from working. Resolve this by
passing the references found in the original path to the added path.
2021-11-05 22:41:30 +03:00
regnat 1f3c3a3785 Make the flake options work when using the daemon
When setting flake-local options (with the `nixConfig` field), forward
these options to the daemon in case we’re using one.

This is necessary in particular for options like `binary-caches` or
`post-build-hook` to make sense.

Fix <343239fc8a (r44356843)>
2021-11-05 16:19:16 +01:00
Eelco Dolstra 40925337a9 Remove maxPrimOpArity 2021-11-04 15:04:07 +01:00
Eelco Dolstra acd6bddec7 Fix derivation primop 2021-11-04 15:04:00 +01:00
Eelco Dolstra cbfbf71e08 Use callFunction() with an array for some calls with arity > 1 2021-11-04 15:03:57 +01:00
Eelco Dolstra bcf4780006 Add level / displacement types 2021-11-04 15:03:45 +01:00
Eelco Dolstra 81e7c40264 Optimize primop calls
We now parse function applications as a vector of arguments rather
than as a chain of binary applications, e.g. 'substring 1 2 "foo"' is
parsed as

  ExprCall { .fun = <substring>, .args = [ <1>, <2>, <"foo"> ] }

rather than

  ExprApp (ExprApp (ExprApp <substring> <1>) <2>) <"foo">

This allows primops to be called immediately (if enough arguments are
supplied) without having to allocate intermediate tPrimOpApp values.

On

  $ nix-instantiate --dry-run '<nixpkgs/nixos/release-combined.nix>' -A nixos.tests.simple.x86_64-linux

this gives a substantial performance improvement:

  user CPU time:      median =      0.9209  mean =      0.9218  stddev =      0.0073  min =      0.9086  max =      0.9340  [rejected, p=0.00000, Δ=-0.21433±0.00677]
  elapsed time:       median =      1.0585  mean =      1.0584  stddev =      0.0024  min =      1.0523  max =      1.0623  [rejected, p=0.00000, Δ=-0.20594±0.00236]

because it reduces the number of tPrimOpApp allocations from 551990 to
42534 (i.e. only small minority of primop calls are partially
applied) which in turn reduces time spent in the garbage collector.
2021-11-04 15:03:40 +01:00
Eelco Dolstra ab35cbd675 StaticEnv: Use std::vector instead of std::map 2021-11-04 15:03:34 +01:00
regnat 5b2aa61f1b Don’t require ca-derivations when __contentAddressed = false
If we explicitely opt-out of it, there’s no need to require the
experimental feature
2021-11-03 06:51:32 +01:00
Shay Bergmann a50c027ece
toJSON: improve pos accuracy, add trace 2021-10-27 19:48:48 +00:00
Eelco Dolstra 5a160171d0 Remove redundant 'warning:' 2021-10-27 18:14:12 +02:00
Shay Bergmann 769de259f0
toJSON: pass pos in case of a list as well 2021-10-26 14:43:15 +00:00
Eelco Dolstra 9ce84c64c5 Tweak fetchTree docs 2021-10-26 14:21:24 +02:00
regnat af99941279 Make experimental-features a proper type
Rather than having them plain strings scattered through the whole
codebase, create an enum containing all the known experimental features.

This means that
- Nix can now `warn` when an unkwown experimental feature is passed
  (making it much nicer to spot typos and spot deprecated features)
- It’s now easy to remove a feature altogether (once the feature isn’t
  experimental anymore or is dropped) by just removing the field for the
  enum and letting the compiler point us to all the now invalid usages
  of it.
2021-10-26 07:02:31 +02:00
Shay Bergmann ba81e871b2
toJSON: report error position for fancier output
Given flake:

```nix
{ description = "nix json error provenance";
  inputs = {};
  outputs = { self }: {
    jsonFunction = _: "function";
    json = builtins.toJSON (_: "function");
  };
}

```
- Before:

```console
❯ nix eval --json .#jsonFunction
error: cannot convert a function to JSON
```

- After:

```console
❯ nix eval --json .#jsonFunction
error: cannot convert a function to JSON

       at /nix/store/b7imf1c2j4jnkg3ys7fsfbj02s5j0i4f-source/testflake/flake.nix:4:5:

            3|   outputs = { self }: {
            4|     jsonFunction = _: "function";
             |     ^
            5|     json = builtins.toJSON (_: "function");
```
2021-10-25 21:17:52 +00:00
Ben Burdette fb8377547b more code cleanup 2021-10-22 14:49:58 -06:00
Ben Burdette 71da988d47 more debug removal 2021-10-22 14:34:50 -06:00
Ben Burdette e54f17eb46 remove more debug code 2021-10-22 14:27:04 -06:00
Ben Burdette cbc2f0fe31 remove dead code 2021-10-22 14:02:47 -06:00
Kevin Amado 823dce945a
fetch: nicer infinite recursion errors
- This change applies to builtins.fetchurl and builtins.fetchTarball
- PoC: `let x = builtins.fetchurl x; in x`
- Before:
  ```bash
  $ nix-instantiate --extra-experimental-features flakes --strict
  error: infinite recursion encountered
  ```
- After:
  ```bash
  $ nix-instantiate --extra-experimental-features flakes --strict
  error: infinite recursion encountered

       at /data/github/kamadorueda/nix/test.nix:1:9:

            1| let x = builtins.fetchurl x; in x
             |         ^
  ```

Mentions: #3505
2021-10-17 12:54:53 -05:00
Kevin Amado e5a27a3b4e
fetchTree: add pos to EvalState::forceValue
- This way we improve error messages
  on infinite recursion
- Demo:
  ```nix
  let x = builtins.fetchTree {
    type = "git";
    inherit x;
  };
  in x
  ```
- Before:
  ```bash
  $ nix-instantiate --extra-experimental-features flakes --strict
  error: infinite recursion encountered
  ```
- After:
  ```bash
  $ nix-instantiate --extra-experimental-features flakes --strict
  error: infinite recursion encountered

       at /data/github/kamadorueda/nix/test.nix:3:10:

            2|   type = "git";
            3|   inherit x;
             |          ^
            4| };
  ```

Mentions: #3505
2021-10-15 19:25:19 -05:00
Kevin Amado 18e3d63341
fetchTree: add pos to EvalState::forceValue
- This way we improve error messages
  on infinite recursion
- Demo:
  ```nix
  let x = builtins.fetchTree x;
  in x
  ```
- Before:
  ```bash
  $ nix-instantiate --extra-experimental-features flakes --strict
  error: infinite recursion encountered
  ```
- After:
  ```bash
  $ nix-instantiate --extra-experimental-features flakes --strict
  error: infinite recursion encountered

       at /data/github/kamadorueda/nix/test.nix:1:9:

            1| let x = builtins.fetchTree x;
             |         ^
            2| in x
  ```

Mentions: #3505
2021-10-15 19:25:19 -05:00
Kevin Amado 1bdeef8395
add pos to EvalState::forceValue
- This way we improve error messages
  on infinite recursion
- Demo:
  ```nix
  let
    x = builtins.fetchMercurial x;
  in
  x
  ```
- Before:
  ```bash
  $ nix-instantiate --show-trace --strict
  error: infinite recursion encountered
  ```
- After:
  ```bash
  nix-instantiate --show-trace --strict
  error: infinite recursion encountered

       at /data/github/kamadorueda/test/default.nix:2:7:

            1| let
            2|   x = builtins.fetchMercurial x;
             |       ^
            3| in
  ```

Mentions: #3505
2021-10-14 23:23:05 -05:00
Ben Burdette 427fb8d158 comment out debugs 2021-10-11 16:48:10 -06:00
Ben Burdette 98eb13691a print staticenv bindings 2021-10-11 16:32:43 -06:00
Ben Burdette 2ee1fa4afd add nullable Expr argument 2021-10-11 14:42:29 -06:00
regnat 7466048d39 (partially) Revert "Don't copy in rethrow"
This reverts some parts of commit
8430a8f086 which was trying to rethrow
some exceptions while we weren’t in the context of a `catch` block,
causing some weird “terminate called without an active exception”
errors.

Fix #5368
2021-10-11 10:51:22 +02:00
Eelco Dolstra d39692e6b3 Make builtins.{path,filterSource} work with chroot stores 2021-10-07 14:22:39 +02:00
Eelco Dolstra c4dcf3cf25 Add a trace to all errors in addPath() 2021-10-07 13:47:15 +02:00
Eelco Dolstra 4806f2f6b0 Allow builtins.{path,filterSource} on paths with a context
We now build the context (so this has the side-effect of making
builtins.{path,filterSource} work on derivations outputs, if IFD is
enabled) and then check that the path has no references (which is what
we really care about).
2021-10-07 13:43:17 +02:00
Eelco Dolstra 7c50568788 Remove unnecessary call to queryMissing()
Worker::run() already does this.
2021-10-07 13:15:01 +02:00
Eelco Dolstra 972405edf5 Allow access to path copied to the store
Fixes https://github.com/NixOS/nix/pull/5163#issuecomment-931733912.
2021-10-07 12:15:22 +02:00
Eelco Dolstra cfaad7168e Refactoring: Add allowPath() method 2021-10-07 12:11:00 +02:00
Eelco Dolstra c9ee634f75
Merge pull request #5341 from andir/libexpr-formals
libexpr: remove matchAttrs boolean from ExprLambda
2021-10-07 11:58:56 +02:00
Eelco Dolstra 53e4794289
Merge pull request #5286 from ilkecan/add-a-warning-to-filterSource
Warn about the usage of filterSource with Nix store paths
2021-10-06 21:02:39 +02:00
ilkecan a4a6ef4fb2 Add a warning to filterSource
Warn about the usage of `filterSource` with Nix store paths
2021-10-06 19:25:33 +03:00
Eelco Dolstra 0dc8172458 Remove no-op call to realiseContext() 2021-10-06 18:08:37 +02:00
Eelco Dolstra c497fce011 Merge branch 'flakes_filterSource' of https://github.com/tomberek/nix 2021-10-06 18:08:18 +02:00
Eelco Dolstra 57a8eb4c01 fetchTree(): Parse type attribute first
The 'url' attribute depends on whether type == 'git', so this is needed for

  builtins.fetchTree {url = "git@github.com:NixOS/nix.git"; type = "git";}
2021-10-06 17:39:02 +02:00
Eelco Dolstra 83d86cc1b0 Cleanup 2021-10-06 17:30:10 +02:00
Andreas Rammhold cae41eebff libexpr: remove matchAttrs boolean from ExprLambda
The boolean is only used to determine if the formals are set to a
non-null pointer in all our cases. We can get rid of that allocation and
instead just compare the pointer value with NULL. Saving up to
sizeof(bool) + platform specific alignment per ExprLambda instace.
Probably not a lot of memory but perhaps a few kilobyte with nixpkgs?

This also gets rid of a potential issue with dereferencing formals based on
the value of the boolean that didn't have to be aligned with the formals
pointer but was in all our cases.
2021-10-06 17:24:06 +02:00
Eelco Dolstra 46753b5e9c Merge branch 'considerate/scp-like-urls' of https://github.com/considerate/nix 2021-10-06 17:19:27 +02:00
Ben Burdette aad27143c6 storing staticenv bindings 2021-10-02 13:47:36 -06:00
John Ericson 242f9bf3dc std::visit by reference
I had started the trend of doing `std::visit` by value (because a type
error once mislead me into thinking that was the only form that
existed). While the optomizer in principle should be able to deal with
extra coppying or extra indirection once the lambdas inlined, sticking
with by reference is the conventional default. I hope this might even
improve performance.
2021-09-30 21:35:09 +00:00
Taeer Bar-Yam f14660d5e2 reset yylloc when yyless(0) is called 2021-09-29 19:47:01 -04:00
Eelco Dolstra fd01c48d34
Merge pull request #5301 from Ma27/builtins-missing-feature-error
libexpr: throw a more helpful eval-error if a builtin is not available due to a missing feature-flag
2021-09-29 12:53:29 +02:00
Maximilian Bosch 2b02ce0e48
libexpr: throw a more helpful eval-error if a builtin is not available due to a missing feature-flag
I found it somewhat confusing to have an error like

    error: attribute 'getFlake' missing

if the required experimental-feature (`flakes`) is not enabled. Instead,
I'd expect Nix to throw an error just like it's the case when using e.g. `nix
flake` without `flakes` being enabled.

With this change, the error looks like this:

    $ nix-instantiate -E 'builtins.getFlake "nixpkgs"'
    error: Cannot call 'builtins.getFlake' because experimental Nix feature 'flakes' is disabled. You can enable it via '--extra-experimental-features flakes'.

           at «string»:1:1:

                1| builtins.getFlake "nixpkgs"
                 | ^

I didn't use `settings.requireExperimentalFeature` here on purpose
because this doesn't contain a position. Also, it doesn't seem as if we
need to catch the error and check for the missing feature here since
this already happens at evaluation time.
2021-09-29 11:57:15 +02:00
Eelco Dolstra 8430a8f086 Don't copy in rethrow 2021-09-27 14:38:10 +02:00
Eelco Dolstra 362d8f925e
Merge pull request #5253 from edolstra/flake-ifd
Don't allow IFD in flakes by default
2021-09-24 10:48:23 +02:00
Ben Burdette b9d08b98da ok was unconditoinally throwing on any With var 2021-09-23 13:02:39 -06:00
Geoff Reedy 9d67332e4b Better eval error locations for interpolation and +
Previously, type or coercion errors for string interpolation, path
interpolation, and plus expressions were always reported at the
beginning of the outer expression. This leads to confusing evaluation
error messages making it hard to accurately diagnose and then fix the
error.

For example, errors were reported as follows.

```
cannot coerce an integer to a string
1| let foo = 7; in "bar" + foo
 |                 ^

cannot add a string to an integer
1| let foo = "bar"; in 4 + foo
 |                     ^

cannot coerce an integer to a string
1| let foo = 7; in "x${foo}"
 |                  ^
```

This commit changes the ExprConcatStrings expression vector to store a
sequence of expressions *and* their expansion locations so that error
locations can be reported accurately. For interpolation, the error is
reported at the beginning of the entire `${foo}`, not at the beginning
of `foo` because I thought this was slightly clearer. The previous
errors are now reported as:

```
cannot coerce an integer to a string
1| let foo = 7; in "bar" + foo
 |                         ^

cannot add a string to an integer
1| let foo = "bar"; in 4 + foo
 |                         ^

cannot coerce an integer to a string
1| let foo = 7; in "x${foo}"
 |                   ^
```

The error is reported at this kind of precise location even for
multi-line indented strings.

This probably helps with at least some of the cases mentioned in #561
2021-09-22 20:57:34 -06:00
Ben Burdette c07edb1932 staticenv should be With 2021-09-22 18:14:57 -06:00
Ben Burdette c7e3d830c1 more debug stuff 2021-09-22 16:22:53 -06:00
Eelco Dolstra d1bf7431bb Revert "Merge pull request #4922 from nrdxp/default-submodules"
This reverts commit 6678e98411, reversing
changes made to 90b2dd570c.
2021-09-22 17:25:25 +02:00
Eelco Dolstra ff28fffce2 Don't cache realiseContext() errors
Errors that depend on the configuration (such as whether
allow-import-from-derivation is set) should not be cached.
2021-09-22 14:00:56 +02:00
Eelco Dolstra 60cc975d22 Set input parent at construction time 2021-09-21 14:07:16 +02:00
Geoff Reedy cbe9ddfd1a Include subpath in flake fingerprint
Without this, flakes within the same tree and same lock data will have
the same fingerprint and the eval cache for one flake will be
incorrectly used for another.
2021-09-16 15:58:21 -06:00
Ben Burdette cd8c232b55 add cout debugging 2021-09-15 16:16:53 -06:00
Eelco Dolstra 1ec4efa6c8
Merge pull request #5257 from edolstra/dirty-lock-file
If we can't write a lock file, pretend the top-level flake is dirty
2021-09-15 20:18:23 +02:00
Eelco Dolstra c17f3c5e69 Merge branch 'mh/fix-chroot-eval' of https://github.com/obsidiansystems/nix 2021-09-15 18:37:58 +02:00
Eelco Dolstra 027344ce7e If we can't write a lock file, pretend the top-level flake is dirty
Alternative to #4639. You can still read flake.lock, but at least in
reproducible workflows like NixOS configurations where you require a
non-dirty tree, evaluation will fail because there is no rev.
2021-09-15 18:31:42 +02:00
Eelco Dolstra 991cc53386 Revert "Disallow reading flake.lock"
This reverts commit e5596113f7.
2021-09-15 18:30:37 +02:00
Eelco Dolstra e5596113f7 Disallow reading flake.lock
With --no-write-lock-file, it's possible that flake.lock is out of
sync with the actual inputs used by the evaluation. So doing fromJSON
(readFile ./flake.lock) will give wrong results.

Fixes #4639.
2021-09-14 21:09:11 +02:00
Ben Burdette 21071bfdeb shared_ptr for StaticEnv 2021-09-14 10:49:22 -06:00
Eelco Dolstra 16d4922dd2
Merge pull request #5240 from edolstra/builtin-help
nix --help: Display help using lowdown instead of man
2021-09-14 11:31:23 +02:00
kvtb c6fa7775de hashFile, hashString: realize context before calculation, and discard afterwards 2021-09-13 22:34:58 +02:00
Ben Burdette 176911102c printEnvPosChain 2021-09-13 11:57:25 -06:00
Eelco Dolstra 49a932fb18 nix --help: Display help using lowdown instead of man
Fixes #4476.
Fixes #5231.
2021-09-13 14:45:21 +02:00
Eelco Dolstra 9180239081
Merge pull request #5215 from ncfavier/patch-1
Fix use-registries logic in builtins.getFlake
2021-09-08 11:19:39 +02:00
Shea Levy 6678e98411
Merge pull request #4922 from nrdxp/default-submodules
libfetchers/git: fetch submodules by default
2021-09-07 05:48:23 -04:00
Naïm Favier b0d4190f19
Fix use-registries logic in builtins.getFlake 2021-09-05 15:28:44 +02:00
Eelco Dolstra 7ee639f9db
Merge pull request #5066 from Radvendii/master
add antiquotations to paths
2021-09-01 12:55:04 +02:00
Taeer Bar-Yam 1ffb9f1970 fix parse of /${foo}. was // + foo
I don't think this changes the way any program would behave, but it's a
cleaner internal representation.
2021-08-31 15:55:55 -04:00
Taeer Bar-Yam 9da8f5e25d path antiquotations: canonizePath -> canonicalizePath 2021-08-31 08:02:04 -04:00
Eelco Dolstra eda0fee160
Merge pull request #5175 from Pamplemousse/make
Don't overwrite user provided `lib*_LDFLAGS`
2021-08-30 12:44:29 +02:00
Eelco Dolstra 323cafcb4e
Merge pull request #5191 from hercules-ci/evalstate-lifetime-hygiene
EvalState lifetime hygiene
2021-08-30 12:23:09 +02:00
Robert Hensing 8656b130ea Fix use after free with vImportedDrvToDerivation 2021-08-29 20:42:57 +02:00