Commit graph

8871 commits

Author SHA1 Message Date
John Ericson bbd7d5de09 Fix some bounds in rapid check instances
`inRange` is exclusive not inclusive:
https://github.com/emil-e/rapidcheck/blob/master/doc/generators.md#usage

Furthermore, use `std::variant_size_v` so we use the right number
automatically.

Finally, make the `switch` assert the discriminant is in bounds as
expected.
2023-05-15 10:41:44 -04:00
John Ericson d2162e7acd Make more string values work as installables
As discussed in #7417, it would be good to make more string values work
as installables. That is to say, if an installable refers to a value,
and the value is a string, it used to not work at all, since #7484, it
works somewhat, and this PR make it work some more.

The new cases that are added for `BuiltPath` contexts:

- Fixed input- or content-addressed derivation:

  ```
  nix-repl> hello.out.outPath
  "/nix/store/jppfl2bp1zhx8sgs2mgifmsx6dv16mv2-hello-2.12"

  nix-repl> :p builtins.getContext hello.out.outPath
  { "/nix/store/c7jrxqjhdda93lhbkanqfs07x2bzazbm-hello-2.12.drv" = { outputs = [ "out" ]; }; }

  The string matches the specified single output of that derivation, so
  it should also be valid.

- Floating content-addressed derivation:

  ```
  nix-repl> (hello.overrideAttrs (_: { __contentAddressed = true; })).out.outPath
  "/1a08j26xqc0zm8agps8anxpjji410yvsx4pcgyn4bfan1ddkx2g0"

  nix-repl> :p builtins.getContext (hello.overrideAttrs (_: { __contentAddressed = true; })).out.outPath
  { "/nix/store/qc645pyf9wl37c6qvqzaqkwsm1gp48al-hello-2.12.drv" = { outputs = [ "out" ]; }; }
  ```

  The string is not a path but a placeholder, however it also matches
  the context, and because it is a CA derivation we have no better
  option. This should also be valid.

We may also want to think about richer attrset based values (also
discussed in that issue and #6507), but this change "completes" our
string-based building blocks, from which the others can be desugared
into or at least described/document/taught in terms of.

Progress towards #7417

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-05-15 09:03:38 -04:00
John Ericson 5a23b80b0a Create EvalState::coerceToDerivedPath
This gives us some round trips to test.

`EvalState::coerceToDerivedPathUnchecked` is a factored out helper just
for unit testing.
2023-05-15 09:03:37 -04:00
John Ericson 8e1a990268 Expose mkOutputString as method of EvalState 2023-05-15 09:03:37 -04:00
John Ericson 0a9afce3b9 Split mkOutputString in two
This well help us with some unit testing
2023-05-15 09:03:37 -04:00
John Ericson 9550c3862f Remove unneeded argument for mkOutputString 2023-05-15 09:03:37 -04:00
John Ericson 914672dc4f
Merge pull request #8141 from tweag/user-files-doc
Document user files of nix
2023-05-15 07:11:47 -04:00
Matej Urbas c66a7af0c6 max-substitution-jobs release note entry 2023-05-14 09:51:11 +01:00
Eelco Dolstra 643b8d2126
Merge pull request #8299 from urbas/max-substitution-jobs
`max-substitution-jobs` setting
2023-05-12 12:24:27 +02:00
John Ericson a93110ab19 Fix nix print-dev-env & nix develop with drv paths
Fixes #8309

This regression was because both `CmdDevelop` and `CmdPrintDevEnv` were
switched to be `InstallableValueCommand` subclasses, but actually
neither should have been.

The `nixpkgsFlakeRef` method should indeed not be on the base
installable class, because "flake refs" and "nixpkgs" are not
installable-wide notions, but that doesn't mean these commands should
only accept installable values.
2023-05-10 11:29:45 -04:00
John Ericson 53a1354acf
Merge pull request #3959 from obsidiansystems/ca-drv-exotic
Derivations can output "text-hashed" data
2023-05-10 10:41:59 -04:00
Théophane Hufschmitt 85ff212051
Merge pull request #7721 from yorickvP/post-build-hook
Also pass unwanted outputs to post-build-hook
2023-05-10 14:30:42 +02:00
John Ericson 6a3a87a714 Improve error message for self reference with text hashing
The `ContentAddressWithReferences` method is made total, with error
handling now squarely the caller's job. This is better.
2023-05-09 14:44:08 -04:00
John Ericson e514b3939a Add name to some error messages 2023-05-09 13:24:53 -04:00
John Ericson 753fc1661d Cleanups to content address types 2023-05-09 13:05:38 -04:00
John Ericson d3c125e5a8
Apply suggestions from code review
Thanks!

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-05-09 12:45:51 -04:00
John Ericson 6513f4fe92 Fix bug, newInfo -> newInfo0
It appears we were checking a variable in the process of definining it.
2023-05-09 12:31:36 -04:00
John Ericson 35dcbe1c21 Fix spurious change
Didn't mean to use the private name that shouldn't be exposed.
2023-05-09 12:19:03 -04:00
Alex Ameen 82296f8113
prevent double quotation 2023-05-09 09:59:18 -05:00
Alex Ameen b72bc4a972
libexpr: quote reserved keys when printing
This fixes a bug in commands like `nix eval' which would emit invalid attribute
sets if they contained reserved keywords such as "assert", "let", etc.

These keywords will not be quoted when printed, making them valid expressions.
All keywords recognized by the lexer are quoted except "or", which does not
require quotation.
2023-05-09 09:45:12 -05:00
Matej Urbas 13185133bc introduces Goal::jobCategory 2023-05-08 19:45:46 +01:00
Matej Urbas 1ea1e378de removes MaxSubstitutionJobsSetting 2023-05-08 19:21:57 +01:00
John Ericson b5d9ef0a4c
Merge pull request #3921 from obsidiansystems/trustless-remote-builder-simple
Trustless remote building for input-addressed drvs
2023-05-08 10:43:37 -04:00
John Ericson df53a7d268 Split comment, match with each variable 2023-05-08 10:08:01 -04:00
John Ericson cd0d8e0bd5
Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-05-08 09:57:05 -04:00
John Ericson 278c94d607 Rename a few things in new tests
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-05-08 08:01:58 -04:00
Yorick 12685ef45f
CA: rewrite hashes for all outputs, not just the wanted ones 2023-05-08 12:58:59 +02:00
Yorick 2ca2c80c4e
libstore: also pass unwanted outputs to the post-build-hook 2023-05-08 12:58:59 +02:00
Robert Hensing cf8effdae2
Merge pull request #8263 from frederictobiasc/improve-doc-genericclosure
Documentation: Improve builtins.genericClosure
2023-05-08 12:23:19 +02:00
Matej Urbas 613bc699bb max-substitution-jobs setting 2023-05-07 20:22:18 +01:00
figsoda 0662fd8599 Fix hostRegex to accept hosts with a - 2023-05-03 18:59:44 -04:00
Eelco Dolstra 17e6b85d05 nix: Support the --repair flag 2023-04-28 17:03:04 +02:00
Valentin Gagarin f8620758aa display documentation on manifest files separately
it's probably better not to show the manifest file documentation in the
command-specific pages, because these are implementation details that are not really practically useful.

this means no additional hassle for building the manual, but clutters
the table of contents a bit.
2023-04-28 12:10:36 +02:00
Valentin Gagarin be7c236565 move compatibility info to nix profile docs 2023-04-26 15:39:35 +02:00
Alexander Bantyev aee5e82b42 Include user-profiles.md into nix profile --help 2023-04-26 15:38:19 +02:00
Alexander Bantyev 8a93b5a551 Document user files of nix 2023-04-26 15:38:19 +02:00
Frédéric Christ f0d2b7eef3 Doc: Improve builtins.genericClosure 2023-04-26 09:37:32 +02:00
Eelco Dolstra 87f676b3a0 Formatting 2023-04-25 16:52:02 +02:00
Eelco Dolstra a74d397549 nix build --json: Only show non-zero startTime / stopTime 2023-04-25 16:43:47 +02:00
Eelco Dolstra 880e7b8ed6 TarArchive: Remove a duplicate constant and increase the buffer size 2023-04-25 16:43:10 +02:00
Eelco Dolstra 5d3f6dbf59 Add some more SourcePath docs 2023-04-24 13:37:51 +02:00
Eelco Dolstra ad57cff9bc
Document tMisc
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-04-24 13:34:46 +02:00
Eelco Dolstra 01232358ff Merge remote-tracking branch 'origin/master' into source-path 2023-04-24 13:20:36 +02:00
John Ericson 8eeaf591db
Add more docs to TextIngestionMethod
Thanks so much!

Co-authored-by: Adam Joseph <54836058+amjoseph-nixpkgs@users.noreply.github.com>
2023-04-21 01:30:55 -04:00
John Ericson 85f0cdc370 Use std::set<StringContextElem> not PathSet for string contexts
Motivation

`PathSet` is not correct because string contexts have other forms
(`Built` and `DrvDeep`) that are not rendered as plain store paths.
Instead of wrongly using `PathSet`, or "stringly typed" using
`StringSet`, use `std::std<StringContextElem>`.

-----

In support of this change, `NixStringContext` is now defined as
`std::std<StringContextElem>` not `std:vector<StringContextElem>`. The
old definition was just used by a `getContext` method which was only
used by the eval cache. It can be deleted altogether since the types are
now unified and the preexisting `copyContext` function already suffices.

Summarizing the previous paragraph:

Old:

  - `value/context.hh`: `NixStringContext = std::vector<StringContextElem>`
  - `value.hh`: `NixStringContext Value::getContext(...)`
  - `value.hh`: `copyContext(...)`

New:

  - `value/context.hh`: `NixStringContext = std::set<StringContextElem>`
  - `value.hh`: `copyContext(...)`
----

The string representation of string context elements no longer contains
the store dir. The diff of `src/libexpr/tests/value/context.cc` should
make clear what the new representation is, so we recommend reviewing
that file first. This was done for two reasons:

Less API churn:

`Value::mkString` and friends did not take a `Store` before. But if
`NixStringContextElem::{parse, to_string}` *do* take a store (as they
did before), then we cannot have the `Value` functions use them (in
order to work with the fully-structured `NixStringContext`) without
adding that argument.

That would have been a lot of churn of threading the store, and this
diff is already large enough, so the easier and less invasive thing to
do was simply make the element `parse` and `to_string` functions not
take the `Store` reference, and the easiest way to do that was to simply
drop the store dir.

Space usage:

Dropping the `/nix/store/` (or similar) from the internal representation
will safe space in the heap of the Nix programming being interpreted. If
the heap contains many strings with non-trivial contexts, the saving
could add up to something significant.

----

The eval cache version is bumped.

The eval cache serialization uses `NixStringContextElem::{parse,
to_string}`, and since those functions are changed per the above, that
means the on-disk representation is also changed.

This is simply done by changing the name of the used for the eval cache
from `eval-cache-v4` to eval-cache-v5`.

----

To avoid some duplication `EvalCache::mkPathString` is added to abstract
over the simple case of turning a store path to a string with just that
string in the context.

Context

This PR picks up where #7543 left off. That one introduced the fully
structured `NixStringContextElem` data type, but kept `PathSet context`
as an awkward middle ground between internal `char[][]` interpreter heap
string contexts and `NixStringContext` fully parsed string contexts.

The infelicity of `PathSet context` was specifically called out during
Nix team group review, but it was agreeing that fixing it could be left
as future work. This is that future work.

A possible follow-up step would be to get rid of the `char[][]`
evaluator heap representation, too, but it is not yet clear how to do
that. To use `NixStringContextElem` there we would need to get the STL
containers to GC pointers in the GC build, and I am not sure how to do
that.

----

PR #7543 effectively is writing the inverse of a `mkPathString`,
`mkOutputString`, and one more such function for the `DrvDeep` case. I
would like that PR to have property tests ensuring it is actually the
inverse as expected.

This PR sets things up nicely so that reworking that PR to be in that
more elegant and better tested way is possible.

Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2023-04-21 01:05:49 -04:00
John Ericson 7103c6da70 Remove references from fixed output derivation ab syntax
In other words, use a plain `ContentAddress` not
`ContentAddressWithReferences` for `DerivationOutput::CAFixed`.

Supporting fixed output derivations with (fixed) references would be a
cool feature, but it is out of scope at this moment.
2023-04-19 15:00:04 -04:00
John Ericson aba8a8a83a Add a few more content addressing methods
Good to round out the library interface.
2023-04-19 14:13:30 -04:00
John Ericson 20decfd302 Gate dynamic-derivations with drv fromJSON too
Don't want `nix derivation add` to be a way to sneak by experimental
feature checks!
2023-04-19 12:44:38 -04:00
John Ericson 76baaeb341 Merge remote-tracking branch 'upstream/master' into ca-drv-exotic 2023-04-19 11:32:14 -04:00
Théophane Hufschmitt d3e2394e91
Merge pull request #8233 from wentasah/narinfo-corrupt-reason
Make "NAR info file is corrupt" messages more informative
2023-04-19 06:57:08 +02:00