Commit graph

2707 commits

Author SHA1 Message Date
eldritch horrors 0cf06c5ab5 Update src/libexpr/eval.cc
Co-authored-by: John Ericson <git@JohnEricson.me>
(cherry picked from commit 80b84710b8c676620ed1e8bf8ff3bb1d5bc19b80)
Change-Id: I128555f1ae13cf0e202f565ee439f698efe12431
2024-03-09 00:25:54 -07:00
eldritch horrors f7b2476415 don't repeatedly look up ast internal symbols
these symbols are used a *lot*, so it makes sense to cache them. this
mostly increases clarity of the code (however clear one may wish to call
the parser desugaring here), but it also provides a small performance
benefit.

(cherry picked from commit 09a1128d9e2ff0ae6176784938047350d6f8a782)
Change-Id: I73d9f66be4555168e048cb2d542277251580c2d1
2024-03-09 00:25:54 -07:00
eldritch horrors 92693973b6 decouple parser and EvalState
there's no reason the parser itself should be doing semantic analysis
like bindVars. split this bit apart (retaining the previous name in
EvalState) and have the parser really do *only* parsing, decoupled from
EvalState.

(cherry picked from commit b596cc9e7960b9256bcd557334d81e9d555be5a2)
Change-Id: I481a7623afc783e9d28a6eb4627552cf8a780986
2024-03-09 00:25:54 -07:00
eldritch horrors faaae9d045 slim down parser.y
most EvalState and Expr members defined here could be elsewhere, where
they'd be easier to maintain (not being embedded in a file with arcane
syntax) and *somewhat* more faithfully placed according to the path of
the file they're defined in.

(cherry picked from commit e1aa585964c3d864ebff0030584f3349a539d615)
Change-Id: Ibc704567462bb40f37cda05d8fadd465519db5f5
2024-03-09 00:25:54 -07:00
eldritch horrors f9f8664879 rename ParserState::{makeCurPos -> at}
most instances of this being used do not refer to the "current"
position, sometimes not even to one reasonably close by. it could also
be called `makePos` instead, but `at` seems clear in context.

(cherry picked from commit 835a6c7bcfd0b22acc16f31de5fc7bb650d52017)
Change-Id: I17cab8a6cc14cac5b64624431957bfcf04140809
2024-03-09 00:25:54 -07:00
eldritch horrors e1cd0077f3 move ParseData to own header, rename to ParserState
ParserState better describes what this struct really is. the parser
really does modify its state (most notably position and symbol tables),
so calling it that rather than obliquely "data" (which implies being
input only) makes sense.

(cherry picked from commit 007605616477f4f0d8a0064c375b1d3cf6188ac5)
Change-Id: I92feaec796530e1d4d0f7d4fba924229591cea95
2024-03-09 00:25:54 -07:00
eldritch horrors 60e3a05175 make parser utility functions members of ParseData
all of them need access to parser state in some way. make them members
to allow this without fussing so much.

(cherry picked from commit 1b09b80afac27c67157d4b315c237fa7bb9b8d08)
Change-Id: I3145c95666a5617b735eff7cb403c54c0fe86347
2024-03-09 00:25:54 -07:00
eldritch horrors ba16d99124 simplify parse error reporting
since nix doesn't use the bison `error` terminal anywhere any invocation
of yyerror will immediately cause a failure. since we're *already*
leaking tons of memory whatever little bit bison allocates internally
doesn't much matter any more, and we'll be replacing the parser soon anyway.

coincidentally this now also matches the error behavior of URIs when
they are disabled or ~/ paths in pure eval mode, duplicate attr
detection etc.

(cherry picked from commit e8d9de967fe47a7f9324b0022a2ef50df59f419d)
Change-Id: I560c50d11dceddc2d7cf9ed2c6c631a309ce574e
2024-03-09 00:25:54 -07:00
eldritch horrors 2cea973e29 remove ParserFormals
this is a proper subset of Formals anyway, so let's just use those and
avoid the extra allocations and moves.

(cherry picked from commit f07388bf985c2440413f398cf93d5f5840d1ec8c)
Change-Id: I4508c9c9c918cbaaed649dc753eb86f5cafc7ab6
2024-03-09 00:25:54 -07:00
eldritch horrors 609a8e0d94 Merge pull request #9754 from 9999years/print-value-when-coercion-fails
Print the value in `error: cannot coerce` messages

(cherry picked from commit 5b7bfd2d6b89d7dd5f54c1ca6c8072358d31a84e)

===

test taken from 6e8d5983143ae576e3f4b1d2954a5267f2943a49; it was added
previously (and not backported because its pr was a mostly-revert), but
it's useful to have around.

Change-Id: Icbd14b55e3610ce7b774667bf14b82e6dc717982
2024-03-09 00:05:41 -07:00
eldritch horrors 2f7c3fa251 Merge pull request #9818 from Ma27/print-value-on-function-call-type-error
libexpr: print value of what is attempted to be called as function
(cherry picked from commit 50e5d7b883042852538371237e32a66bb22f0485)
Change-Id: I7cb6290bd8f244e83bfce3b2eed2a4c8b4f16a83
2024-03-09 00:05:41 -07:00
eldritch horrors 87e6ac5eb7 Merge pull request #9753 from 9999years/print-value-on-type-error
Print the value in `value is X while a Y is expected` error

(cherry picked from commit 5f72a97092da6af28a7d2b2a50d74e9d34fae7e1)
Change-Id: Idb4bc903ae59a0f5b6fb3b1da4d47970fe0a6efe
2024-03-09 00:05:41 -07:00
eldritch horrors 512c1f05c3 Unify and refactor value printing
Previously, there were two mostly-identical value printers -- one in
`libexpr/eval.cc` (which didn't force values) and one in
`libcmd/repl.cc` (which did force values and also printed ANSI color
codes).

This PR unifies both of these printers into `print.cc` and provides a
`PrintOptions` struct for controlling the output, which allows for
toggling whether values are forced, whether repeated values are tracked,
and whether ANSI color codes are displayed.

Additionally, `PrintOptions` allows tuning the maximum number of
attributes, list items, and bytes in a string that will be displayed;
this makes it ideal for contexts where printing too much output (e.g.
all of Nixpkgs) is distracting. (As requested by @roberth in
https://github.com/NixOS/nix/pull/9554#issuecomment-1845095735)

Please read the tests for example output.

Future work:
- It would be nice to provide this function as a builtin, perhaps
  `builtins.toStringDebug` -- a printing function that never fails would
  be useful when debugging Nix code.
- It would be nice to support customizing `PrintOptions` members on the
  command line, e.g. `--option to-string-max-attrs 1000`.

(cherry picked from commit 0fa08b451682fb3311fe58112ff05c4fe5bee3a4, )

===

Restore ambiguous value printer for `nix-instantiate`

The Nix team has requested that this output format remain unchanged.
I've added a warning to the man page explaining that `nix-instantiate
--eval` output will not parse correctly in many situations.

(cherry picked from commit df84dd4d8dd3fd6381ac2ca3064432ab31a16b79)

Change-Id: I7cca6b4b53cd0642f2d49af657d5676a8554c9f8
2024-03-09 03:50:06 +01:00
eldritch horrors 0e8f505f66 Unindent print.hh declarations
(cherry picked from commit c9125603a535f82cc9a53f47533f0a3d174e7008)
Change-Id: I137a272feee7eb42a017f373cbe4705b0e79b7c1
2024-03-08 18:54:10 -07:00
eldritch horrors 852bc55c87 Merge pull request #9890 from obsidiansystems/remove-unneeded-toRealPath
Change an `allowPath` call to take a store path again

(cherry picked from commit 7c9ed1b1a325fe64a186e9d454607eaa0a7e8951)
Change-Id: Ia5ec924315a1f2640a0438cfb4b1ee0689cd3558
2024-03-07 13:04:25 +01:00
eldritch horrors b14f88e0d4 Merge pull request #9985 from alois31/symlink-resolution
Restore `builtins.pathExists` behavior on broken symlinks

(cherry picked from commit d53c8901ef7f2033855dd99063522e3d56a19dab)

===

note that this variant differs markedly from the source commit because
we haven't endured quite as much lazy trees.

Change-Id: I0facf282f21fe0db4134be5c65a8368c1b3a06fc
2024-03-07 00:43:51 -07:00
eldritch horrors 1342c8f18e Merge pull request #10074 from lf-/jade/ban-implicit-fallthrough
Warn on implicit switch case fallthrough

(cherry picked from commit 21282c3c204597641402c6bcff8fc9ee7bc31fa1)
Change-Id: I5ebbdfb6c037d2c55254f37dd391c07c2ce7443e
2024-03-07 00:11:12 -07:00
eldritch horrors 89e99d94e4 Merge pull request #9634 from 9999years/combine-abstract-pos-and-pos
Combine `AbstractPos`, `PosAdapter`, and `Pos`

(cherry picked from commit 113499d16fc87d53b73fb62fe6242154909756ed)

===

this is a bit cursed because originally it was based on InputAccessor
code that we don't have and moved/patched features we likewise don't
have (fetchToStore caching, all the individual accessors,
ContentAddressMethod). the commit is adjusted accordingly to
match (remove caching, ignore accessors, use FileIngestionMethod).

note that `state.rootPath . CanonPath == abs` and
computeStorePathForPath works relative to cwd, so the slight rewrite in
the moved fetchToStore is legal.

Change-Id: I05fd340c273f0bcc8ffabfebdc4a88b98083bce5
2024-03-05 23:46:18 -07:00
eldritch horrors 5bda6d9dc8 Merge pull request #9915 from 9999years/evaluating-attribute-position
Add position information to `while evaluating the attribute` errors in the debugger

(cherry picked from commit ffe67c86a8ef3695e5c8b9c9800c192ac633dded)
Change-Id: I177ea5ec60898abe09fb9d80d9602b2a32ff8f44
2024-03-04 09:25:17 +01:00
eldritch horrors 523250d18b Merge pull request #10055 from GrahamDennis/gdennis/faster-flake-lock-parsing
Faster flake.lock parsing

(cherry picked from commit ff4fa4dbd307fb155155421a7ec0625383dcedeb)
Change-Id: I34325834770db66fa56f0d8d02d3d2322dea3c65
2024-03-04 09:25:17 +01:00
eldritch horrors 647d762ab5 Merge pull request #9933 from pennae/debugger-fix
fix debugger crashing while printing envs

(cherry picked from commit 9b8b4860913afdb7f9cb1e1e81f9a9dd192cd1c1)
Change-Id: Ica6a17e4b3e3b8b8093719f442c28be7f13be09d
2024-03-04 08:57:44 +01:00
eldritch horrors 542a19104e Merge pull request #9918 from 9999years/debugger-locals-for-let-expressions
Expose locals from `let` expressions to the debugger

(cherry picked from commit acef4f17a2daab4ccdf656bdf229792db2f779e4)
Change-Id: Ib3623254f67ac762f4e7230d625e9f87dff38a84
2024-03-04 08:54:52 +01:00
eldritch horrors 43cedec6c5 Merge pull request #9919 from 9999years/reduce-debugger-clutter
Reduce visual clutter in the debugger

(cherry picked from commit f388a6148dae0fc999f1a67d0b96d76788f9b97f)
Change-Id: I21bfe3e9f75816484b0f46dbe09e0ff40b22c6d9
2024-03-04 08:52:57 +01:00
eldritch horrors 2c85fcce87 Merge pull request #9747 from awakesecurity/mz/fix-quadratic-splitString
Fix performance of builtins.substring for empty substrings

(cherry picked from commit b2deff1947c2fe57fdbf1a472eb9003eb407f8d3)
Change-Id: I4ddfc8d26a4781c9520fff9807849a073ee7bed8
2024-03-04 08:06:36 +01:00
eldritch horrors 137673de56 Merge pull request #9681 from edolstra/eval-optimisations
Optimize empty list constants

(cherry picked from commit 315aade89d00c692715e5953c36a1b7d6528b703)
Change-Id: I0f28ef8a27ccedc45acf44243eec9dc35b733300
2024-03-04 07:39:12 +01:00
eldritch horrors 6b279cd10e Merge pull request #9658 from pennae/env-diet
reduce the size of Env by one pointer

(cherry picked from commit 83f5622545a2fc31eb7e7d5105f64ed6dd3058b3)
Change-Id: I5636290526d0165cfc61aee1e7a5b94db4a26cef
2024-03-04 07:37:45 +01:00
eldritch horrors 96f1a404d0 Merge pull request #9617 from 9999years/stack-overflow-segfault
Fix segfault on infinite recursion in some cases

(cherry picked from commit bf1b294bd81ca76c5ec9fe3ecd52196bf52a8300)
Change-Id: Id137541426ec8536567835953fccf986a3aebf16
2024-03-04 07:35:20 +01:00
eldritch horrors dd180911d8 Merge pull request #9582 from pennae/misc-opts
a packet of small optimizations

(cherry picked from commit ee439734e924eb337a869ff2e48aff8b989198bc)
Change-Id: I125d870710750a32a0dece48f39a3e9132b0d023
2024-03-04 07:32:31 +01:00
eldritch horrors 67de43b1d7 Merge pull request #9645 from tweag/undeprecate-isNull
Undeprecate isNull

(cherry picked from commit 8e64cd59b0fe1a4e1d3ba8d359ea0f9af2936f94)
Change-Id: If36b85a3fc01ee700bcaf4d5d83a5884a4b5de92
2024-03-04 07:27:37 +01:00
eldritch horrors 7614aa9797 Merge pull request #4093 from matthewbauer/eval-system
Add eval-system option

(cherry picked from commit 071dbbee33af9f27338c3e53e4ea067dbfa14010)
Change-Id: Ia81358c8cfb60241da07a4d0e84b9ee62a18a53f
2024-03-04 07:21:01 +01:00
eldritch horrors 2ba9788003 Merge pull request #9555 from 9999years/positions-in-errors
Pass positions when evaluating

(cherry picked from commit c8458bd731eb1c74159bebe459ea00165e056b65)
Change-Id: I1b4a5d58973be6264ffdb23b4492da200fdb71be
2024-03-04 07:11:25 +01:00
eldritch horrors 001be52794 Merge pull request #9430 from hercules-ci/remove-vlas
Fix stack overflow in `filter`

(cherry picked from commit cb7f25869daa2491eeac5fee49ad8f31b2218c15)
Change-Id: Ib90f97a9805bbb4d0e2741551d490f054fc0a675
2024-03-04 07:11:25 +01:00
eldritch horrors a089d8f5f6 Merge pull request #9465 from obsidiansystems/build-dir
Use `buildprefix` in a few more places

(cherry picked from commit b6a3fde6b7a416929553e6be36fc991680ddf9ef)
Change-Id: I2790663fa9f8242ac2db6582b7e421d2fdf42942
2024-03-04 07:11:25 +01:00
eldritch horrors b7c61d337b Merge pull request #9462 from trofi/eval.o-dependency
libexpr: add missing dependency on 'flake/call-flake.nix.gen.hh'
(cherry picked from commit f0180487a0e4c0091b46cb1469c44144f5400240)
Change-Id: I27211872ad75034ae3f04e18b2987f9968073d00
2024-03-04 07:11:25 +01:00
eldritch horrors cad1683048 Merge pull request #9398 from Qyriad/fixes/flake-not-found
flakes: bare minimum fix the error message for untracked flake.nix
(cherry picked from commit 7f626dba332f320fafb9f9c749986ea523d20f42)
Change-Id: I470ac56a670a8c11e4164c6b059184a02344d491
2024-03-04 06:03:33 +01:00
eldritch horrors 439f88b7d7 Merge pull request #9399 from edolstra/revert-vlas
Revert use of boost::container::small_vector in the evaluator

(cherry picked from commit 6832d18ac734f4b855f97c07b158491dd01cefcd)
Change-Id: I7f10af0c8b8a8beb4b1e36424120995f4ed82738
2024-03-04 05:57:04 +01:00
eldritch horrors 3f3badffc9 Merge pull request #9395 from nbraud/buitlins
builtins.concatMap: Fix typo in error message

(cherry picked from commit 4292d997568eb30503e287f98e24821ff0bc2816)
Change-Id: Ia33d1b02e41f699ef0c8c2d6487c9f70b2cc8cf4
2024-03-04 05:56:04 +01:00
eldritch horrors 2df9c2c00f Merge pull request #9374 from hercules-ci/fix-nogc
Fix buildNoGc

(cherry picked from commit 7762335a9c4d56f7501825345728c3f94e88bc31)
Change-Id: I3ad53c9c1dd9180a2abc3528e9437c28ac95b303
2024-03-04 05:54:10 +01:00
eldritch horrors 5e182235cb Merge pull request #7348 from thufschmitt/dont-use-vlas
Remove the usage of VLAs in the code

(cherry picked from commit ac4431e9d016e62fb5dc9ae36833bd0c6cdadeec)
Change-Id: Ifbf5fbfc2e27122362a2aaea4b62c7cf3ca46b1a
2024-03-04 05:51:23 +01:00
eldritch horrors 30f3298e9d Merge pull request #9364 from edolstra/fix-coverage
Fix a broken generated header file dependency

(cherry picked from commit 57de482e00221d781f916d49c315a8fa0816e956)
Change-Id: I0580274bf51132fbeb34e8c0a91a7017071d2bb3
2024-03-04 05:48:04 +01:00
eldritch horrors 1b135e6e7b Fix boost::bad_format_string exception in builtins.addErrorContext (#9291)
* Fix boost::bad_format_string exception in builtins.addErrorContext

The message passed to addTrace was incorrectly being used as a format
string and this this would cause an exception when the string contained
a '%', which can be hit in places where arbitrary file paths are
interpolated.

* add test

(cherry picked from commit 61d6fe059e959455e156c1d57bb91155d363e983)
Change-Id: Idd671127a9c1ccc8b94e58e727632fcc064f3cbe
2024-03-04 05:39:12 +01:00
eldritch horrors ea10088703 Merge pull request #9216 from obsidiansystems/addDrvOutputDependencies-pre
Add `builtins.addDrvOutputDependencies`

(cherry picked from commit a58d7f143ec995a45745c2176bfebcc3e011db58)
Change-Id: Ia5a1790bf29dfaf29287cc35cdae6b6d650e7a83
2024-03-04 05:06:56 +01:00
eldritch horrors c0fa61340c Merge pull request #9172 from tfc/bad-moves
Fix/remove some bad std::moves

(cherry picked from commit 8c049a9f044569ebda70231709f6f15d3073894a)
Change-Id: I720273378d2506a13883acee28abd096d099b0d4
2024-03-04 04:41:52 +01:00
eldritch horrors 6feba52008 Merge pull request #8895 from hercules-ci/gc-before-stats
eval: Run a full GC before printing stats

(cherry picked from commit aeea49609be014b1928c95b7ec28dbedeb4f032a)
Change-Id: I47a23d3a7a47ea61d9a2b5727b638f879f3aaf1e
2024-03-04 04:36:47 +01:00
Eelco Dolstra 2442a40393 Merge pull request #9029 from inclyc/users/lyc/pass-value-2
libexpr: const rvalue reference -> value for nix::Expr nodes

(cherry picked from commit de99647b9c66219b2e7bc698a62377cc4f6148f6)
Change-Id: I96baf96b0a17e401548a926a7015f85e11d7c459
2024-03-04 04:35:59 +01:00
Théophane Hufschmitt f36d4aefd3 Fix the build
bef68e53b9 (backport of
31ebc6028b3682969d86a7b39ae87131c41cc604) accidentally broke the build
because of a change in the constructor of `SourcePath` between 2.18 and
master. Fix that.
2024-02-15 09:50:12 +01:00
Eelco Dolstra bef68e53b9 Fix symlink handling
This restores the symlink handling behaviour prior to
94812cca98.

Fixes #9298.

(cherry picked from commit 31ebc6028b3682969d86a7b39ae87131c41cc604)
2024-01-21 20:53:30 +00:00
Shea Levy ba48ab4b95
Merge branch '2.18-maintenance' into ifd-buildStore-2.18 2024-01-11 07:21:16 -05:00
Shea Levy e7c2b35827
Build IFD in the build store when using eval-store.
Previously, IFDs would be built within the eval store, even though one
is typically using `--eval-store` precisely to *avoid* local builds.

Because the resulting Nix expression must be copied back to the eval
store in order to be imported, this requires the eval store to trust
the build store's signatures.

(cherry picked from commit c3942ef85ffbd83391410fbf012f1de366d2463c)
2024-01-11 06:34:27 -05:00
John Ericson f7f37035c8 Move tests to separate directories, and document
Today, with the tests inside a `tests` intermingled with the
corresponding library's source code, we have a few problems:

- We have to be careful that wildcards don't end up with tests being
  built as part of Nix proper, or test headers being installed as part
  of Nix proper.

- Tests in libraries but not executables is not right:

  - It means each executable runs the previous unit tests again, because
    it needs the libraries.

  - It doesn't work right on Windows, which doesn't want you to load a
    DLL just for the side global variable . It could be made to work
    with the dlopen equivalent, but that's gross!

This reorg solves these problems.

There is a remaining problem which is that sibbling headers (like
`hash.hh` the test header vs `hash.hh` the main `libnixutil` header) end
up shadowing each other. This PR doesn't solve that. That is left as
future work for a future PR.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>

(cherry picked from commit 91b6833686a6a6d9eac7f3f66393ec89ef1d3b57)
(cherry picked from commit a61e42adb528b3d40ce43e07c79368d779a8b624)
2023-12-01 13:05:03 -05:00
John Ericson 89a66633b7 Add missing -lrapidcheck fixing build with shared lib
https://github.com/NixOS/nixpkgs/pull/269064 makes rapidcheck be build
as a shared lib, but that broke Nix because the `-lrapidcheck` was
missing. This fixes that (and doesn't break Nix what the library is a
static archive as today).

(cherry picked from commit 46131567da96ffac298b9ec54016b37114b0dfd5)
2023-11-22 04:48:11 +00:00
DavHau 23775b7364 fetchTree: clarify docs for shallow flag
(cherry picked from commit 796a7eb92d2b0caf75685126adc7460a4c39cfec)
2023-11-20 14:16:03 +00:00
Robert Hensing 860f64c345 pathExists: isDir when endswith /.
(cherry picked from commit f8a3893e8d77ce4a6e23719a0b2d88464cb84b9c)
2023-10-01 13:58:20 +01:00
Eelco Dolstra 126e2645f2 Disable rapidcheck tests in the coverage run
https://hydra.nixos.org/build/233688539
2023-09-19 16:04:00 +02:00
Matthew Kenigsberg 07545add53
Drop dead code
localPath is unused
2023-09-10 12:18:03 +02:00
Robert Hensing 5f55c33917
Merge pull request #8944 from fricklerhandwerk/fix-anchor
fix invalid anchor link
2023-09-07 17:58:16 +02:00
John Ericson 7ad66cb3ef Allow dynamic derivation deps in inputDrvs
We use the same nested map representation we used for goals, again in
order to save space. We might someday want to combine with `inputDrvs`,
by doing `V = bool` instead of `V = std::set<OutputName>`, but we are
not doing that yet for sake of a smaller diff.

The ATerm format for Derivations also needs to be extended, in addition
to the in-memory format. To accomodate this, we added a new basic
versioning scheme, so old versions of Nix will get nice errors. (And
going forward, if the ATerm format changes again the errors will be even
better.)

`parsedStrings`, an internal function used as part of parsing
derivations in A-Term format, used to consume the final `]` but expect
the initial `[` to already be consumed. This made for what looked like
unbalanced brackets at callsites, which was confusing. Now it consumes
both which is hopefully less confusing.

As part of testing, we also created a unit test for the A-Term format for
regular non-experimental derivations too.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>

Apply suggestions from code review

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-09-07 10:39:37 -04:00
Valentin Gagarin 829d4d3e03 fix invalid anchor link 2023-09-07 15:13:22 +02:00
Eelco Dolstra b88784278f
Merge pull request #8869 from hercules-ci/fix-issue-8838-pathExists-isDir
Fix #8838, pathExists: isDir when ends with `/ `
2023-09-01 13:15:54 +02:00
Robert Hensing 1e08e12d81 pathExists: isDir when endswith /
Fixes https://github.com/NixOS/nix/issues/8838
2023-08-25 17:18:37 +02:00
John Ericson 50f40ac4c0
Merge pull request #8829 from obsidiansystems/build-dynamic-derivations
Adapt scheduler to work with dynamic derivations
2023-08-25 11:13:15 -04:00
tomberek b563ef38cc
Merge pull request #8819 from VertexA115/fix/deep-follow-paths
Fix follow path checking at depths greater than 2
2023-08-25 10:33:05 -04:00
John Ericson 2f5d3da806 Introduce OutputName and OutputNameView type aliases
Hopefully they make the code easier to understand!
2023-08-25 09:55:07 -04:00
John Ericson fe71faa920 Delete EvalState::addToSearchPath
This function is now trivial enough that it doesn't need to exist.

`EvalState` can still be initialized with a custom search path, but we
don't have a need to mutate the search path after it has been
constructed, and I don't see why we would need to in the future.

Fixes #8229
2023-08-18 14:04:33 -04:00
John Ericson 665ad4f7c5
Merge pull request #8839 from obsidiansystems/string-context-7479
Refactor Raw pattern, part of #7479
2023-08-18 13:47:01 -04:00
John Ericson 9121fed4b4 Fixing #7479
Types converted:

- `NixStringContextElem`
- `OutputsSpec`
- `ExtendedOutputsSpec`
- `DerivationOutput`
- `DerivationType`

Existing ones mostly conforming the pattern cleaned up:

- `ContentAddressMethod`
- `ContentAddressWithReferences`

The `DerivationGoal::derivationType` field had a bogus initialization,
now caught, so I made it `std::optional`. I think #8829 can make it
non-optional again because it will ensure we always have the derivation
when we construct a `DerivationGoal`.

See that issue (#7479) for details on the general goal.

`git grep 'Raw::Raw'` indicates the two types I didn't yet convert
`DerivedPath` and `BuiltPath` (and their `Single` variants) . This is
because @roberth and I (can't find issue right now...) plan on reworking
them somewhat, so I didn't want to churn them more just yet.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2023-08-18 11:44:00 -04:00
Cole Helbling 1d7a57cfd9 libexpr/tests: test that parseFlakeRef doesn't percent-encode twice 2023-08-17 13:45:55 -07:00
Sergei Trofimovich b74962c92b src/libexpr/search-path.cc: avoid out-of-bounds read on string_view
Without the change build with `-D_GLIBCXX_ASSERTIONS` exposes testsuite
assertion:

    $ gdb src/libexpr/tests/libnixexpr-tests
    Reading symbols from src/libexpr/tests/libnixexpr-tests...
    (gdb) break __glibcxx_assert_fail
    (gdb) run
    (gdb) bt
    in std::__glibcxx_assert_fail(char const*, int, char const*, char const*)@plt () from /mnt/archive/big/git/nix/src/libexpr/libnixexpr.so
    in std::basic_string_view<char, std::char_traits<char> >::operator[] (this=0x7fffffff56c0, __pos=4)
        at /nix/store/r74fw2j8rx5idb0w8s1s6ynwwgs0qmh9-gcc-14.0.0/include/c++/14.0.0/string_view:258
    in nix::SearchPath::Prefix::suffixIfPotentialMatch (this=0x7fffffff5780, path=...) at src/libexpr/search-path.cc:15
    in nix::SearchPathElem_suffixIfPotentialMatch_partialPrefix_Test::TestBody (this=0x555555a17540) at src/libexpr/tests/search-path.cc:62

As string sizes are usigned types `(a - b) > 0` effectively means
`a != b`. While the intention should be `a > b`.

The change fixes test suite pass.
2023-08-14 22:07:37 +01:00
Alex Zero 1ef8008ca7 Fix follow path checking at depths greater than 2
We need to recurse into the input tree to handle follows paths that
trarverse multiple inputs that may or may not be follow paths
themselves.
2023-08-14 18:55:46 +01:00
John Ericson 44c8d83831 Create outputOf primop.
In the Nix language, given a drv path, we should be able to construct
another string referencing to one of its output. We can do this today
with `(import drvPath).output`, but this only works for derivations we
already have.

With dynamic derivations, however, that doesn't work well because the
`drvPath` isn't yet built: importing it like would need to trigger IFD,
when the whole point of this feature is to do "dynamic build graph"
without IFD!

Instead, what we want to do is create a placeholder value with the right
string context to refer to the output of the as-yet unbuilt derivation.
A new primop in the language, analogous to `builtins.placeholder` can be
used to create one. This will achieve all the right properties. The
placeholder machinery also will match out the `outPath` attribute for CA
derivations works.

In 60b7121d2c we added that type of
placeholder, and the derived path and string holder changes necessary to
support it. Then in the previous commit we cleaned up the code
(inspiration finally hit me!) to deduplicate the code and expose exactly
what we need. Now, we can wire up the primop trivally!

Part of RFC 92: dynamic derivations (tracking issue #6316)

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-08-14 09:37:37 -04:00
John Ericson e7c39ff00b Rework evaluator SingleDerivedPath infra
`EvalState::mkSingleDerivedPathString` previously contained its own
inverse (printing, rather than parsing) in order to validate what was
parsed. Now that is pulled out into its own separate function:
`EvalState::coerceToSingleDerivedPath`.

In additional that pulled out logic is deduplicated with
`EvalState::mkOutputString` via `EvalState::mkOutputStringRaw`, which is
itself deduplicated (and generalized) with
`DownstreamPlaceholder::mkOutputStringRaw`.

All these changes make the unit tests simpler.

(We would ideally write more unit tests for `mkSingleDerivedPathString`
`coerceToSingleDerivedPath` directly, but we cannot yet do that because
the IO in reading the store path won't work when the dummy store cannot
hold anything. Someday we'll have a proper in-memory store which will
work for this.)

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-08-14 08:44:50 -04:00
John Ericson a04720e68c Rename optOutputPath to optStaticOutputPath
This choice of variable name makes it more clear what is going on.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-08-14 08:44:48 -04:00
Robert Hensing 584ff408a4
Merge pull request #8735 from obsidiansystems/defexpr
Factor out `nix-defexpr` path computation
2023-08-11 20:32:03 +02:00
John Ericson 60b7121d2c Make the Derived Path family of types inductive for dynamic derivations
We want to be able to write down `foo.drv^bar.drv^baz`:
`foo.drv^bar.drv` is the dynamic derivation (since it is itself a
derivation output, `bar.drv` from `foo.drv`).

To that end, we create `Single{Derivation,BuiltPath}` types, that are
very similar except instead of having multiple outputs (in a set or
map), they have a single one. This is for everything to the left of the
rightmost `^`.

`NixStringContextElem` has an analogous change, and now can reuse
`SingleDerivedPath` at the top level. In fact, if we ever get rid of
`DrvDeep`, `NixStringContextElem` could be replaced with
`SingleDerivedPath` entirely!

Important note: some JSON formats have changed.

We already can *produce* dynamic derivations, but we can't refer to them
directly. Today, we can merely express building or example at the top
imperatively over time by building `foo.drv^bar.drv`, and then with a
second nix invocation doing `<result-from-first>^baz`, but this is not
declarative. The ethos of Nix of being able to write down the full plan
everything you want to do, and then execute than plan with a single
command, and for that we need the new inductive form of these types.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-08-10 00:08:32 -04:00
Eelco Dolstra eb1302670e
Merge pull request #8769 from edolstra/generalize-tarball-urls
Don't require .tar/.zip extension for tarball flakerefs
2023-08-07 17:02:17 +02:00
Théophane Hufschmitt 5df0f1755f
Merge pull request #8692 from obsidiansystems/add-another-xp-check
Feature gate `DownstreamPlaceholder::unknownCaOutput`
2023-08-07 13:11:44 +02:00
John Ericson 9b908fa70a Factor out nix-defexpr path computation
Avoid duplicated code, and also avoid "on the fly" path construction
(which makes it harder to keep track of which paths we use).

The factored out code doesn't create the Nix state dir anymore, but this
is fine because other in nix-env and nix-channel does:

- nix-channel: Line 158 in this commit

- nix-env: Line 1407 in this commit
2023-08-02 12:54:48 -04:00
Eelco Dolstra d00469ebf9
Merge pull request #8762 from obsidiansystems/split-out-eval-settings
Move evaluator settings (type and global) to separate file/header
2023-08-02 16:54:02 +02:00
Eelco Dolstra d9e7758f47 Don't require .tar/.zip extension for tarball flakerefs
Special-casing the file name is rather ugly, so we shouldn't do
that. So now any {file,http,https} URL is handled by
TarballInputScheme, except for non-flake inputs (i.e. inputs that have
the attribute `flake = false`).
2023-08-01 16:07:20 +02:00
John Ericson 1570e80219 Move evaluator settings (type and global) to separate file/header 2023-07-31 10:14:15 -04:00
Robert Hensing 33d58a90c2 toJSON: Add attribute path to trace 2023-07-31 13:02:54 +02:00
Alex Ameen 2d1d81114d
Add parseFlakeRef and flakeRefToString builtins (#8670)
Over the last year or so I've run into several use cases where I need to
parse and/or serialize URLs for use by `builtins.fetchTree` or
`builtins.getFlake`, largely in order to produce _lockfile-like_ files
for lang2nix frameworks or tools which use `nix` internally to drive
builds.

I've gone through the painstaking process of emulating
`nix::FlakeRef::fromAttrs` and `nix::parseFlakeRef` several times with
mixed success; but these are difficult to create and even harder to
maintain if I hope to stay aligned with changes to the real
parser/serializer.

I understand why adding new `builtins` isn't something we want to do
flagrantly. I'm recommending this addition simply because I keep
encountering use cases where I need to parse/serialize these URIs in
`nix` expressions, and I want a reliable solution.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-07-25 17:43:33 +00:00
John Ericson 4685c9b55f
Merge pull request #8664 from ncfavier/merge-dynamic-attrs
parser: merge nested dynamic attributes
2023-07-23 11:03:38 -04:00
Naïm Favier 570a1a3ad7
parser: merge nested dynamic attributes
Fixes https://github.com/NixOS/nix/issues/7115
2023-07-21 17:14:03 +02:00
Eelco Dolstra 7ac24d9525
Merge pull request #8650 from obsidiansystems/content-address-simpler
Simplify `ContentAddress`
2023-07-21 13:46:53 +02:00
Valentin Gagarin 0e4f6dfcf7 revert anchor prefix for builtin constants
the original change broke many pre-existing anchor links.

also change formatting of the constants listing slightly:
- the type should not be part of the anchor
- add highlight to the "impure only" note
2023-07-20 10:27:38 +02:00
John Ericson caabc4f648 Feature gate DownstreamPlaceholder::unknownCaOutput
This is a part of CA derivations that we forgot to put behind the
experimental feature.

This was caught by @fricklerhandwerk in
https://github.com/NixOS/nix/pull/8369#discussion_r1258133719
2023-07-13 07:56:33 -04:00
John Ericson 4a880c3cc0
Merge pull request #8579 from obsidiansystems/findPath-cleanup-2
Further search path cleanups
2023-07-10 09:59:01 -04:00
Eelco Dolstra fea7d3b1cd
Merge pull request #8681 from inclyc/libexpr/parser-move-noeffect
libexpr: remove std::move() for `basePath` in parser, it has no effect
2023-07-10 13:30:42 +02:00
Yingchi Long 3d74e7b811 libexpr: remove std::move() for basePath in parser, it has no effect 2023-07-10 12:02:29 +08:00
John Ericson 028b26a77f
Merge pull request #8370 from hercules-ci/fetchClosure-input-addressed
`fetchClosure`: input addressed and pure
2023-07-09 23:41:22 -04:00
John Ericson be518e73ae Clean up SearchPath
- Better types

- Own header / C++ file pair

- Test factored out methods

- Pass parsed thing around more than strings

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-07-09 23:22:22 -04:00
John Ericson 87dcd09047 Clean up resolveSearchPathElem
We should use `std::optional<std::string>` not `std::pair<bool,
std::string>` for an optional string.
2023-07-09 23:13:30 -04:00
John Ericson 903700c5e1 Simplify ContentAddress
Whereas `ContentAddressWithReferences` is a sum type complex because different
varieties support different notions of reference, and
`ContentAddressMethod` is a nested enum to support that,
`ContentAddress` can be a simple pair of a method and hash.

`ContentAddress` does not need to be a sum type on the outside because
the choice of method doesn't effect what type of hashes we can use.

Co-Authored-By: Cale Gibbard <cgibbard@gmail.com>
2023-07-07 07:30:01 -04:00
Robert Hensing b4b02d084f fetchClosure: Interleave the examples in the docs 2023-07-07 11:40:40 +02:00
Robert Hensing 537e8beb77
fetchClosure: Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-07-07 11:00:40 +02:00
Yingchi Long 3c90340fe6 libexpr: use thread_local to make the parser thread-safe
If we call `adjustLoc`, the global variable `prev_yylloc` is shared
between threads and racy.

Currently, nix itself does not concurrently parsing files, but this is
helpful for libexpr users. (The parser is thread-safe except this.)
2023-07-03 16:05:43 +08:00
Robert Hensing 7b39a388b3
Merge pull request #8566 from inclyc/nixd/value-print-depth
libexpr: extend `Value::print` to allow limited depth
2023-07-01 20:08:52 +02:00
Robert Hensing 40052c7613 fetchClosure: Docs and error message improvements
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-30 18:23:42 +02:00
Robert Hensing 50de11d662 doc: Improve fetchClosure documentation 2023-06-30 18:23:24 +02:00
Robert Hensing dc79636007 fetchClosure: Refactor: replace enableRewriting
A single variable is nice and self-contained.
2023-06-30 18:22:47 +02:00
Robert Hensing 5bdca46117 fetchClosure: Split into three cases 2023-06-30 18:22:47 +02:00
Robert Hensing 55888633dd makeContentAddressed: Add single path helper 2023-06-30 18:22:47 +02:00
Robert Hensing 8dca95386c fetchClosure: Disallow toPath for inputAddressed = true 2023-06-30 18:22:47 +02:00
Robert Hensing 508aa58e67 fetchClosure: Always check that inputAddressed matches the result 2023-06-30 18:22:47 +02:00
Robert Hensing ea30f152b7 fetchClosure: Allow input addressed paths in pure mode
When explicitly requested by the caller, as suggested in the meeting
(https://github.com/NixOS/nix/pull/8090#issuecomment-1531139324)

> @edolstra: { toPath } vs { fromPath } is too implicit

I've opted for the `inputAddressed = true` requirement, because it
we did not agree on renaming the path attributes.

> @roberth: more explicit
> @edolstra: except for the direction; not immediately clear in which direction the rewriting happens

This is in fact the most explicit syntax and a bit redundant, which is
good, because that redundancy lets us deliver an error message that
reminds expression authors that CA provides a better experience to
their users.
2023-06-30 18:22:43 +02:00
Robert Hensing 7e5b6d2c45 fetchClosure: Refactor: rename toCA -> enableRewriting 2023-06-30 18:19:56 +02:00
Robert Hensing 0f6d596df5 fetchClosure: Factor out attribute hint 2023-06-30 18:19:56 +02:00
Robert Hensing 1632f08ea2
Merge pull request #8600 from inclyc/libexpr/fix-leaking-in-stripIndentation
libexpr: fix leaking `es2` in stripIndentation (parser.y)
2023-06-29 11:31:53 +02:00
Yingchi Long 3468cbaf47 libexpr: fix leaking es2 in stripIndentation (parser.y) 2023-06-28 22:38:44 +08:00
John Ericson 22b278e011 Automatically document builtin constants
This is done in roughly the same way builtin functions are documented.

Also auto-link experimental features for primops, subsuming PR #8371.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-27 09:37:54 -04:00
John Ericson e8067daf09 Generialize showType 2023-06-27 09:11:42 -04:00
Eelco Dolstra 8e770dac9f
Merge pull request #8576 from obsidiansystems/findPath-cleanup
Use a struct not `std::pair` for `SearchPathElem`
2023-06-26 12:37:06 +02:00
Théophane Hufschmitt 60f06a1714
Merge pull request #5385 from Enzime/add/dirty-rev
Add `dirtyRev` and `dirtyShortRev` to `fetchGit`
2023-06-24 14:55:31 +02:00
Michael Hoang a7b49086c7 Add dirtyRev and dirtyShortRev to fetchGit
Fixes #4682
2023-06-24 14:17:25 +10:00
John Ericson 484290a9e0 Use a struct not std::pair for SearchPathElem
I got very confused trying to keep all the `first` and `second` straight
reading the code, *especially* as there is also another `(boolean,
string)` pair type also being used.

Named fields is much better.

There are other cleanups that we can do (for example, the existing
TODO), but we can do them later. Doing them now would just make this
harder to review.
2023-06-23 12:01:10 -04:00
Yingchi Long 9d8c4ac446 libexpr: remove unused token ATTRPATH in token declaration 2023-06-23 13:35:41 +08:00
Yingchi Long 1400fde144 libexpr: extend Value::print to allow limited depth 2023-06-22 18:28:30 +08:00
Yingchi Long c48277c1c1 libexpr: add tests for nix::Value::print 2023-06-22 18:28:30 +08:00
Adam Joseph 6b06e97bde src/libexpr/eval.hh: add link for allowed-uris option
This commit adds a link to the documentation for `--option
allowed-uris` where that option is mentioned while describing
`restrict-eval`.
2023-06-18 23:36:32 -04:00
Eelco Dolstra e503eadafc
Merge pull request #8477 from edolstra/tarball-flake-redirects
Tarball flake improvements
2023-06-16 18:03:50 +02:00
Adam Joseph 098fbf6273 src/libexpr/eval.hh: fix typo
The option name is `allowed-uris`, not `allowed-uri`.
2023-06-14 21:47:58 -07:00
Michal Sojka a0c4d58549 Remove RegisterPrimOp constructor without support for documentation
The remaining constructor RegisterPrimOp::RegisterPrimOp(Info && info)
allows specifying the documentation in .args and .doc members of the
Info structure.

Commit 8ec1ba0210 removed all uses of the removed constructor in the
nix binary. Here, we remove the constructor completely as well as its
use in a plugin test. According to #8515, we didn't promis to maintain
compatibility with external plugins.

Fixes #8515
2023-06-14 22:37:52 +02:00
John Ericson e377746842
Merge pull request #8491 from wentasah/builtins-doc
Document several undocumented builtin functions
2023-06-14 20:32:37 +02:00
John Ericson 61a3e1f2e2
Merge pull request #4282 from tweag/fix-ca-hash-rewriting
fix the hash rewriting for ca-derivations
2023-06-14 18:25:00 +02:00
Michal Sojka c6d7c4f9ec Document fromTOML, hasContext and getContext builtins
Until now, these functions were completely missing in the Nix manual.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-13 21:53:03 +02:00
Eelco Dolstra 1ad3328c5e Allow tarball URLs to redirect to a lockable immutable URL
Previously, for tarball flakes, we recorded the original URL of the
tarball flake, rather than the URL to which it ultimately
redirects. Thus, a flake URL like
http://example.org/patchelf-latest.tar that redirects to
http://example.org/patchelf-<revision>.tar was not really usable. We
couldn't record the redirected URL, because sites like GitHub redirect
to CDN URLs that we can't rely on to be stable.

So now we use the redirected URL only if the server returns the
`x-nix-is-immutable` or `x-amz-meta-nix-is-immutable` headers in its
response.
2023-06-13 14:17:45 +02:00
Michal Sojka 8ec1ba0210 Register all PrimOps via the Info structure
This will allow documenting them (in later commits).

Note that we keep the old constructor even if it is no longer used by
Nix code, because it is used in tests/plugins/plugintest.cc, which
suggests that it might be used by some external plugin.
2023-06-11 21:40:43 +02:00
Andrea Bedini 3c78920f73
Parse TOML timestamps (#8120)
Currently `fromTOML` throws an exception when encountering a timestamp
since the Nix language lacks a way to represent them.

This patch changes this beaviour and makes `fromTOML` parse timestamps as
attrsets of the format

  { _type = "timestamp"; value = "1979-05-27T07:32:00Z"; }

This is guarded by an experimental feature flag to leave room for iterating on the representation.
2023-06-09 11:53:18 +02:00
Eelco Dolstra 9a5b9c588f
Merge pull request #8421 from fricklerhandwerk/doc-replaceStrings
update documentation according to release notes
2023-05-31 12:10:32 +02:00
Valentin Gagarin 738c0d5064
Merge pull request #8318 from fricklerhandwerk/doc-currentTime
document `builtins.currentTime`
2023-05-31 03:15:54 +02:00
Valentin Gagarin 9afd525e18 update documentation according to release notes 2023-05-30 22:58:15 +02:00
Robert Hensing bed2fe2312
Merge pull request #8398 from polykernel/perf/lazy-eval-replacements-replacestrings
primops: lazy evaluation of replaceStrings replacements
2023-05-30 22:49:42 +02:00
Eelco Dolstra ecae62020b
Merge pull request #8406 from NixOS/fix-ca-attrs-false
Restore Nix 2.3 behaviour for {__impure,__contentAddressed} = false
2023-05-30 11:59:14 +02:00
Eelco Dolstra e4aebccf20 Restore Nix 2.3 behaviour for {__impure,__contentAddressed} = false
Fixes #8405.
2023-05-27 17:53:30 +02:00
Yorick 2c462486fe
create pathAccessible, use it to infer default dirs 2023-05-26 15:36:47 +02:00
Yorick a6c78ba367
getDefaultNixPath: ignore EPERM 2023-05-26 15:36:46 +02:00
polykernel a382919d7d
primops: lazy evaluation of replaceStrings replacements
The primop `builtins.replaceStrings` currently always strictly evaluates the
replacement strings, however time and space are wasted for their computation
if the corresponding pattern do not occur in the input string. This commit
makes the evaluation of the replacement strings lazy by deferring their
evaluation to when the corresponding pattern are matched and memoize the result
for efficient retrieval on subsequent matches.

The testcases for replaceStrings was updated to check for lazy evaluation
of the replacements. A note was also added in the release notes to
document the behavior change.
2023-05-25 18:35:23 -04:00
Théophane Hufschmitt 3ebe1341ab Make RewritingSink accept a map of rewrites
Giving it the same semantics as `rewriteStrings`.
Also add some tests for it
2023-05-24 14:11:50 +02:00
John Ericson b9e5ce4a27 Upgrade downstreamPlaceholder to a type with methods
This gets us ready for dynamic derivation dependencies (part of RFC 92).
2023-05-17 17:41:16 -04:00
Valentin Gagarin 6a5a8f51bb add cross-references to pure evaluation mode
use consistent wording everywhere.
add some details on the configuration option documentation.
2023-05-17 15:01:54 +02:00
Robert Hensing 41591b33a9
Merge pull request #8340 from NixOS/delete-commited-build-artifacts
Delete commited build artifacts
2023-05-15 17:30:20 +02:00
John Ericson 98afd6ff76 Delete commited build artifacts
They were improperly added in 8a93b5a551.

They were not `.gitignore`d because they were stale in that commit --
build artifacts no longer used that name by then and so `.gitignore` was
updated accordingly.
2023-05-15 10:50:33 -04:00
John Ericson bbd7d5de09 Fix some bounds in rapid check instances
`inRange` is exclusive not inclusive:
https://github.com/emil-e/rapidcheck/blob/master/doc/generators.md#usage

Furthermore, use `std::variant_size_v` so we use the right number
automatically.

Finally, make the `switch` assert the discriminant is in bounds as
expected.
2023-05-15 10:41:44 -04:00
John Ericson 5a23b80b0a Create EvalState::coerceToDerivedPath
This gives us some round trips to test.

`EvalState::coerceToDerivedPathUnchecked` is a factored out helper just
for unit testing.
2023-05-15 09:03:37 -04:00
John Ericson 8e1a990268 Expose mkOutputString as method of EvalState 2023-05-15 09:03:37 -04:00
John Ericson 0a9afce3b9 Split mkOutputString in two
This well help us with some unit testing
2023-05-15 09:03:37 -04:00
John Ericson 9550c3862f Remove unneeded argument for mkOutputString 2023-05-15 09:03:37 -04:00
John Ericson 914672dc4f
Merge pull request #8141 from tweag/user-files-doc
Document user files of nix
2023-05-15 07:11:47 -04:00
John Ericson 53a1354acf
Merge pull request #3959 from obsidiansystems/ca-drv-exotic
Derivations can output "text-hashed" data
2023-05-10 10:41:59 -04:00
Alex Ameen 82296f8113
prevent double quotation 2023-05-09 09:59:18 -05:00
Alex Ameen b72bc4a972
libexpr: quote reserved keys when printing
This fixes a bug in commands like `nix eval' which would emit invalid attribute
sets if they contained reserved keywords such as "assert", "let", etc.

These keywords will not be quoted when printed, making them valid expressions.
All keywords recognized by the lexer are quoted except "or", which does not
require quotation.
2023-05-09 09:45:12 -05:00
Alexander Bantyev 8a93b5a551 Document user files of nix 2023-04-26 15:38:19 +02:00
Frédéric Christ f0d2b7eef3 Doc: Improve builtins.genericClosure 2023-04-26 09:37:32 +02:00
Eelco Dolstra 87f676b3a0 Formatting 2023-04-25 16:52:02 +02:00
Eelco Dolstra 01232358ff Merge remote-tracking branch 'origin/master' into source-path 2023-04-24 13:20:36 +02:00
John Ericson 85f0cdc370 Use std::set<StringContextElem> not PathSet for string contexts
Motivation

`PathSet` is not correct because string contexts have other forms
(`Built` and `DrvDeep`) that are not rendered as plain store paths.
Instead of wrongly using `PathSet`, or "stringly typed" using
`StringSet`, use `std::std<StringContextElem>`.

-----

In support of this change, `NixStringContext` is now defined as
`std::std<StringContextElem>` not `std:vector<StringContextElem>`. The
old definition was just used by a `getContext` method which was only
used by the eval cache. It can be deleted altogether since the types are
now unified and the preexisting `copyContext` function already suffices.

Summarizing the previous paragraph:

Old:

  - `value/context.hh`: `NixStringContext = std::vector<StringContextElem>`
  - `value.hh`: `NixStringContext Value::getContext(...)`
  - `value.hh`: `copyContext(...)`

New:

  - `value/context.hh`: `NixStringContext = std::set<StringContextElem>`
  - `value.hh`: `copyContext(...)`
----

The string representation of string context elements no longer contains
the store dir. The diff of `src/libexpr/tests/value/context.cc` should
make clear what the new representation is, so we recommend reviewing
that file first. This was done for two reasons:

Less API churn:

`Value::mkString` and friends did not take a `Store` before. But if
`NixStringContextElem::{parse, to_string}` *do* take a store (as they
did before), then we cannot have the `Value` functions use them (in
order to work with the fully-structured `NixStringContext`) without
adding that argument.

That would have been a lot of churn of threading the store, and this
diff is already large enough, so the easier and less invasive thing to
do was simply make the element `parse` and `to_string` functions not
take the `Store` reference, and the easiest way to do that was to simply
drop the store dir.

Space usage:

Dropping the `/nix/store/` (or similar) from the internal representation
will safe space in the heap of the Nix programming being interpreted. If
the heap contains many strings with non-trivial contexts, the saving
could add up to something significant.

----

The eval cache version is bumped.

The eval cache serialization uses `NixStringContextElem::{parse,
to_string}`, and since those functions are changed per the above, that
means the on-disk representation is also changed.

This is simply done by changing the name of the used for the eval cache
from `eval-cache-v4` to eval-cache-v5`.

----

To avoid some duplication `EvalCache::mkPathString` is added to abstract
over the simple case of turning a store path to a string with just that
string in the context.

Context

This PR picks up where #7543 left off. That one introduced the fully
structured `NixStringContextElem` data type, but kept `PathSet context`
as an awkward middle ground between internal `char[][]` interpreter heap
string contexts and `NixStringContext` fully parsed string contexts.

The infelicity of `PathSet context` was specifically called out during
Nix team group review, but it was agreeing that fixing it could be left
as future work. This is that future work.

A possible follow-up step would be to get rid of the `char[][]`
evaluator heap representation, too, but it is not yet clear how to do
that. To use `NixStringContextElem` there we would need to get the STL
containers to GC pointers in the GC build, and I am not sure how to do
that.

----

PR #7543 effectively is writing the inverse of a `mkPathString`,
`mkOutputString`, and one more such function for the `DrvDeep` case. I
would like that PR to have property tests ensuring it is actually the
inverse as expected.

This PR sets things up nicely so that reworking that PR to be in that
more elegant and better tested way is possible.

Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2023-04-21 01:05:49 -04:00
John Ericson 7103c6da70 Remove references from fixed output derivation ab syntax
In other words, use a plain `ContentAddress` not
`ContentAddressWithReferences` for `DerivationOutput::CAFixed`.

Supporting fixed output derivations with (fixed) references would be a
cool feature, but it is out of scope at this moment.
2023-04-19 15:00:04 -04:00
John Ericson 76baaeb341 Merge remote-tracking branch 'upstream/master' into ca-drv-exotic 2023-04-19 11:32:14 -04:00
Robert Hensing 28d7ffd448
Merge pull request #8220 from accelbread/whitelist-commit-lockfile-summary
Add commit-lockfile-summary to flake nixConfig whitelist
2023-04-18 16:49:33 +02:00
John Ericson 668377f217 TextHashMethod -> TextIngestionMethod, gate with XP feature
I suppose we can use `dynamic-derivations` for the few things we neeed.
2023-04-17 19:02:45 -04:00
John Ericson e12efa3654 Merge remote-tracking branch 'upstream/master' into ca-drv-exotic 2023-04-17 10:16:57 -04:00
Robert Hensing e641de085b
Merge pull request #3746 from obsidiansystems/path-info
Introduce `StoreReferences` and `ContentAddressWithReferences`
2023-04-17 15:49:48 +02:00
Robert Hensing cb2615cf47 Merge remote-tracking branch 'upstream/master' into source-path 2023-04-17 11:41:50 +02:00
Robert Hensing 9af9c260fc
Merge pull request #8193 from hercules-ci/dry-strings
Deduplicate string literal rendering, fix 4909
2023-04-17 11:19:40 +02:00
Robert Hensing b6125772d7 libexpr: Move identifier-like printing to print.cc 2023-04-16 14:07:35 +02:00
Robert Hensing 28a5cdde02 libexpr/value/print.* -> libexpr/print.*
Generalizes the file to sensibly allow printing any part of the
language syntax.
2023-04-16 13:10:45 +02:00
Robert Hensing 1e2dd669bc printLiteral: Do not overload 2023-04-16 13:04:35 +02:00
Robert Hensing 9c74df5bb4
Format
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-04-15 20:56:51 +02:00
Archit Gupta bfc558c972 Whitelist commit-lockfile-summary in flake nixConfig 2023-04-14 11:33:38 -07:00
Eelco Dolstra 162c3f5f7c
Merge pull request #8170 from tweag/fix-aarch64-build
Explicitely define `LockFile::operator!=`
2023-04-11 12:35:23 +02:00
Robert Hensing 4e0804c920 Deduplicate string literal rendering, fix 4909 2023-04-09 22:42:20 +02:00
John Ericson fd21f9d76e Merge remote-tracking branch 'upstream/master' into path-info 2023-04-07 20:39:04 -04:00
Robert Hensing 4411c7d7e0
Merge pull request #8179 from tweag/disable-gc-on-coroutine
disable gc on coroutine
2023-04-07 18:21:11 +02:00
John Ericson 0746951be1
Finish converting existing comments for internal API docs (#8146)
* Finish converting existing comments for internal API docs

99% of this was just reformatting existing comments. Only two exceptions:

- Expanded upon `BuildResult::status` compat note

- Split up file-level `symbol-table.hh` doc comments to get
  per-definition docs

Also fixed a few whitespace goofs, turning leading tabs to spaces and
removing trailing spaces.

Picking up from #8133

* Fix two things from comments

* Use triple-backtick not indent for `dumpPath`

* Convert GNU-style `\`..'` quotes to markdown style in API docs

This will render correctly.
2023-04-07 13:55:28 +00:00
Yorick 62ddd8633c Add talkative msg for coro gc debug 2023-04-07 14:54:50 +02:00
Yorick 58d24a4cb6 Always disable GC in a coroutine unless the patch is applied 2023-04-07 14:54:38 +02:00
Yorick 00bc34430b DisableGC: replace by CoroutineContext, std::shared_ptr<void> 2023-04-07 14:53:40 +02:00
Yorick 2c53ef1bfe Disable GC inside coroutines on mac OS 2023-04-07 14:52:59 +02:00
Eelco Dolstra 6960de73af
Typo 2023-04-07 13:08:32 +02:00
Eelco Dolstra a9759407e5 Origin: Use SourcePath 2023-04-06 15:25:06 +02:00
Eelco Dolstra 94812cca98 Backport SourcePath from the lazy-trees branch
This introduces the SourcePath type from lazy-trees as an abstraction
for accessing files from inputs that may not be materialized in the
real filesystem (e.g. Git repositories). Currently, however, it's just
a wrapper around CanonPath, so it shouldn't change any behaviour. (On
lazy-trees, SourcePath is a <InputAccessor, CanonPath> tuple.)
2023-04-06 13:15:50 +02:00
Théophane Hufschmitt faefaac875 Explicitely define LockFile::operator!=
It should be syntethised in terms of `operator==`, but the GCC version
used on aarch64-linux doesn't implement that (see
https://hydra.nixos.org/build/214848896=. So explicitely define it.

Fix https://github.com/NixOS/nix/issues/8159
2023-04-05 17:20:04 +02:00
Théophane Hufschmitt 523c3f7225
Merge pull request #8157 from hercules-ci/switch-bugs
Fix current and future `switch` bugs
2023-04-03 20:43:02 +02:00
Eelco Dolstra 81491e1379
Merge pull request #8042 from lheckemann/alt-lockfiles
Allow specifying alternative paths for reading/writing flake locks
2023-04-03 19:28:09 +02:00
Robert Hensing fba7be80eb Enable -Werror=switch-enum
switch statements must now match all enum values or disable the
warning.

Explicit is good. This has helped us find two bugs, after solving
another one by debugging.

From now on, adding to an enum will raise errors where they are
not explicitly handled, which is good for productivity, and helps
us decide the correct behavior in all usages.

Notably still excluded from this though are the cases where the
warning is disabled by local pragmas.

fromTOML.cc did not build despite a top-level pragma, so I've had
to resort to a makefile solution for that.
2023-04-03 18:45:20 +02:00
Robert Hensing 9470ee877d Allow open switch-enum in 5 places 2023-04-03 18:45:20 +02:00
Robert Hensing 3dac4c7874 Add explicit case statements where -Wswitch-enum would report them 2023-04-03 18:17:32 +02:00
Robert Hensing f3a6de6ba9
Merge pull request #8148 from hercules-ci/fix-issue-8119-printValue-tBlackhole-abort
eval: Fix crash on missing printValue tBlackhole case
2023-04-03 16:05:55 +02:00
Robert Hensing 1c55544a42 eval: Fix crash on missing printValue tBlackhole case
Fixes #8119
2023-04-03 15:32:42 +02:00
John Ericson eeecfacb43 Merge branch 'path-info' into ca-drv-exotic 2023-04-01 16:40:32 -04:00
John Ericson f7f44f7c96 Merge commit 'aa99005004bccc9be506a2a2f162f78bad4bcb41' into ca-drv-exotic 2023-04-01 15:15:32 -04:00
John Ericson f4ab297b31 Ensure all headers have #pragma once and are in API docs
`///@file` makes them show up in the internal API dos. A tiny few were
missing `#pragma once`.
2023-03-31 23:19:44 -04:00
John Ericson aa99005004 Merge remote-tracking branch 'upstream/master' into path-info
Also improve content-address.hh API docs.
2023-03-30 16:28:53 -04:00
Eelco Dolstra 02050bd055
Typo 2023-03-30 10:49:21 +02:00
Joachim Breitner 1b6cfe6fa1 More precise wording 2023-03-30 10:40:04 +02:00
Joachim Breitner 2e6b95dac7
Update src/libexpr/primops.cc
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2023-03-29 13:19:51 +02:00
Joachim Breitner 3ce6d72b2b
Update src/libexpr/primops.cc
Co-authored-by: asymmetric <lorenzo@mailbox.org>
2023-03-28 14:38:39 +02:00
Joachim Breitner 605c8f7789 Docs: Explain why import nixpkgs works in flakes 2023-03-28 09:35:49 +02:00
John Ericson 1d539aa287
Merge pull request #8073 from tweag/fix-root-channels-location
Fix root channels location
2023-03-27 09:37:53 -04:00