Commit graph

2622 commits

Author SHA1 Message Date
terru - a33c95be5b avoid markdown which the repl's :doc cannot handle
code blocks, if not surrounded by empty lines, have the language
tags (in these cases, always `nix`) show up in the output of :doc.

for example:

  nix-repl> :doc builtins.parseFlakeRef
  Synopsis: builtins.parseFlakeRef flake-ref

    Parse a flake reference, and return its exploded form.

    For example: nix builtins.parseFlakeRef
    "github:NixOS/nixpkgs/23.05?dir=lib"  evaluates to: nix { dir =
    "lib"; owner = "NixOS"; ref = "23.05"; repo = "nixpkgs"; type =
    "github"; }

is now instead:
  nix-repl> :doc builtins.parseFlakeRef
  Synopsis: builtins.parseFlakeRef flake-ref

    Parse a flake reference, and return its exploded form.

    For example:

      | builtins.parseFlakeRef "github:NixOS/nixpkgs/23.05?dir=lib"

    evaluates to:

      | { dir = "lib"; owner = "NixOS"; ref = "23.05"; repo = "nixpkgs"; type = "github"; }

(closes #225)

Change-Id: I0741aeb1006a5376bb2f663d202c7a4da7e38cce
2024-04-06 13:14:14 +00:00
Qyriad a4ef195a9f always re-eval cached failures
This is terrible UX, and frankly an eval failure should be a cache
invalidation anyway.

This removes the CachedEvalError type entirely.

Fixes #223.

Change-Id: I91f8003eabd0ea45003024e96d1de3c7ae8e49d8
2024-04-06 04:35:25 +00:00
Rebecca Turner ea10fe7ab0 Add EscapeStringOptions and escapeString tests
Change-Id: I86ead2f969c9e03c9edfa51bbc92ee06393fd7d6
2024-04-03 16:25:54 +00:00
Ilya K e9960e147b libexpr/flake: don't purple the flake.lock diff
Change-Id: I6f9471fb0ddd51fadb209ad970abd215238ba5c4
2024-04-01 08:57:50 +03:00
Théophane Hufschmitt 6c29016a09 Merge pull request #9920 from 9999years/forbid-nested-debuggers
Forbid nested debuggers

(cherry picked from commit e164b39ee90fd655dbb7f479fdd4fbe38cc883bd)
Change-Id: Iff62f40fd251116516a63e2d3f9fb5b21480b16d
2024-03-31 17:28:25 +00:00
Rebecca Turner 2a98ba8b97 Add pre-commit checks
The big ones here are `trim-trailing-whitespace` and `end-of-file-fixer`
(which makes sure that every file ends with exactly one newline
character).

Change-Id: Idca73b640883188f068f9903e013cf0d82aa1123
2024-03-29 22:57:40 -07:00
jade a4f5bb951d Merge "Document ExprLambda fields" into main 2024-03-30 03:22:53 +00:00
Jade Lovelace 817322fbfb Document ExprLambda fields
We got confused what formals did and had to briefly figure it out. We
should just have docs, so these are some.

Change-Id: If3e794a401e69d022785cbfa0b0c2e2284f41f58
2024-03-29 18:29:59 -07:00
Rebecca Turner a5a25894c1 Move escapeString to its own file
Change-Id: Ie5c954ec73c46c9d3c679ef99a83a29cc7a08352
2024-03-29 16:26:29 -07:00
Winter Cute 6646b80396 meson: add missing explicit dependency on nlohmann_json
Without this, the Meson setup won't bail out if nlohmann_json is
missing, leading to subpar DX (and maybe worse, but I'm not entirely
sure).

Change-Id: I5913111060226b540dcf003257c99a08e84da0de
2024-03-29 14:16:58 -04:00
eldritch horrors 6e5db5e4a2 meson: install missing/generated headers
one headers (args/root.hh) was simply missing, and the generated headers
were not installed. not all of them *should* be installed either, only a
select few (and sadly this needs a custom target for each one, it seems)

Change-Id: I37b25517895d0e5e521abc1202fa65624de57ed1
2024-03-29 02:45:48 +00:00
eldritch horrors 69bfd21e20 meson: install pkg-config files for libraries
Change-Id: I14b9d81d09f188eacfb9c68bcfb84751c18e3779
2024-03-29 02:45:48 +00:00
Rebecca Turner 236bc046ba Merge "Remove HintFmt::operator%" into main 2024-03-29 01:13:45 +00:00
Winter Cute 80405d0626 Stop vendoring toml11
We don't apply any patches to it, and vendoring it locks users into
bugs (it hasn't been updated since its introduction in late 2021).

Closes lix-project/lix#164

Change-Id: Ied071c841fc30b0dfb575151afd1e7f66970fdb9
2024-03-27 21:04:00 -04:00
jade edba570664 HOT SALE: 15% off your build times!
This was achieved by running maintainers/buildtime_report.sh on the
build directory of a meson build, then asking "why the heck is json
eating our build times", and strategically moving the json using bits
out of widely included headers.

It turns out that putting literally any metrics whatsoever into the
build had immediate and predictable results.

Results are 1382.5s frontend time -> 1175.4s frontend time, back end
time approximately invariant.

Related: lix-project/lix#159

Change-Id: I7edea95c8536203325c8bb4dae5f32d727a21b2d
2024-03-27 03:52:57 +00:00
Rebecca Turner 8e63eca912 Remove HintFmt::operator%
Change-Id: Ibcf1a7848b4b18ec9b0807628ff229079ae7a0fe
2024-03-26 15:40:05 -07:00
Théophane Hufschmitt 86881226b0 Merge pull request #8817 from iFreilicht/flake-update-lock-overhaul
Overhaul `nix flake update` and `nix flake lock` UX

(cherry picked from commit 12a0ae73dbb37becefa5a442eb4532ff0de9ce65)
Change-Id: Iff3b4f4235ebb1948ec612036b39ab29e4ca22b2
2024-03-25 17:36:24 -06:00
Eelco Dolstra aa7653608d Minor cleanup in libexpr/flake/flake.cc
(cherry picked from commit 05316d401fa509557c71140e17bb19814412fcb8)
Change-Id: I6ba0b55709f5fe21beb4e9f3bf72ee28715d15f3
2024-03-25 15:30:36 +00:00
Eelco Dolstra b525d0f20c Input: Replace markFileChanged() by putFile()
Committing a lock file using markFileChanged() required the input to
be writable by the caller in the local filesystem (using the path
returned by getSourcePath()). putFile() abstracts over this.

(cherry picked from commit 95d657c8b3ae4282e24628ba7426edb90c8f3942)
Change-Id: Ie081c5d9eb4e923b229191c5e23ece85145557ff
2024-03-25 15:30:36 +00:00
Qyriad b4d07656ff build: optionally build and install with meson
This commit adds several meson.build, which successfully build and
install Lix executables, libraries, and headers. Meson does not yet
build docs, Perl bindings, or run tests, which will be added in
following commits. As such, this commit does not remove the existing
build system, or make it the default, and also as such, this commit has
several FIXMEs and TODOs as notes for what should be done before the
existing autoconf + make buildsystem can be removed and Meson made the
default. This commit does not modify any source files.

A Meson-enabled build is also added as a Hydra job, and to
`nix flake check`.

Change-Id: I667c8685b13b7bab91e281053f807a11616ae3d4
2024-03-22 08:36:50 -06:00
eldritch horrors 22e3f0e987 libexpr: unbreak PosTable performance
this was mostly an inconvenience for error reporting, but fully broke
the debugger (because the debugger does *a lot* of eager position
resolution). copying the line offsets into a local and filling that
local when empty without also storing the calculated offsets back does
kind of ... not cache anything.

fixes lix-project/lix#165

Change-Id: Iccb0ba193ce2f15c832978daecf7b9bebbbe8585
2024-03-20 13:45:36 +01:00
jade 4050245faa Merge changes I72c945ca,I2138bb4d,Ib96749f3 into main
* changes:
  Release notes for builtins.nixVersion change
  un-nixes ur lix, a little
  issue importer: list issues that are *not* closed when finding existing issues
2024-03-18 20:19:53 -06:00
jade 30233d87f9 un-nixes ur lix, a little
I didn't really go attack the docs because we need to pull a bunch of
PRs. I went looking for strings in the code that called lix nix.

Change-Id: I2138bb4dd239096bc530946b281db7f875195b39
2024-03-18 18:20:24 -07:00
eldritch horrors afb839a0c9 libexpr: associate let exprs with the correct StaticEnv
static env association is from expr to its enclosing scope, but let
exprs set their association to their *inner* scope. this skips one level
of envs and will cause segfaults if the parent is a with expr.

fixes #145

Change-Id: I1d22146110f071ede21b4eed7ed34b5850ef2ef3
2024-03-18 14:15:22 -07:00
eldritch horrors b3599166ad libexpr: sort binding name in debugger
not doing this exposes the binding name order to the annoying
interference of parse order on symbol order, which wouldn't be so bad if
it didn't make the tests less reliable and, importantly, dependent on
linker behavior (due to primop initialization being done in static
initializer, and the order of static initializers being defined only
within a single translation unit).

fixes #143

Change-Id: I3cf417893fbcf19e9ad3ff8986deb7cbcf3ca511
2024-03-18 20:03:31 +01:00
jade 47a237f7ec Merge "Delete hasPrefix and hasSuffix from the codebase" into main 2024-03-18 12:01:39 -06:00
eldritch horrors 86a1121d16 use byte indexed locations for PosIdx
we now keep not a table of all positions, but a table of all origins and
their sizes. position indices are now direct pointers into the virtual
concatenation of all parsed contents. this slightly reduces memory usage
and time spent in the parser, at the cost of not being able to report
positions if the total input size exceeds 4GiB. this limit is not unique
to nix though, rustc and clang also limit their input to 4GiB (although
at least clang refuses to process inputs that are larger, we will not).

this new 4GiB limit probably will not cause any problems for quite a
while, all of nixpkgs together is less than 100MiB in size and already
needs over 700MiB of memory and multiple seconds just to parse. 4GiB
worth of input will easily take multiple minutes and over 30GiB of
memory without even evaluating anything. if problems *do* arise we can
probably recover the old table-based system by adding some tracking to
Pos::Origin (or increasing the size of PosIdx outright), but for time
being this looks like more complexity than it's worth.

since we now need to read the entire input again to determine the
line/column of a position we'll make unsafeGetAttrPos slightly lazy:
mostly the set it returns is only used to determine the file of origin
of an attribute, not its exact location. the thunks do not add
measurable runtime overhead.

notably this change is necessary to allow changing the parser since
apparently nothing supports nix's very idiosyncratic line ending choice
of "anything goes", making it very hard to calculate line/column
positions in the parser (while byte offsets are very easy).

(cherry picked from commit 5d9fdab3de0ee17c71369ad05806b9ea06dfceda)
Change-Id: Ie0b2430cb120c09097afa8c0101884d94f4bbf34
2024-03-18 16:12:46 +01:00
eldritch horrors c39150e6bb diagnose "unexpected EOF" at EOF
this needs a string comparison because there seems to be no other way to
get that information out of bison. usually the location info is going to
be correct (pointing at a bad token), but since EOF isn't a token as
such it'll be wrong in that this case.

this hasn't shown up much so far because a single line ending *is* a
token, so any file formatted in the usual manner (ie, ending in a line
ending) would have its EOF position reported correctly.

(cherry picked from commit 855fd5a1bb781e4f722c1d757ba43e866d370132)
Change-Id: I120c56a962f4286b1ae3b71da7b71ce8ec3e0535
2024-03-18 16:12:46 +01:00
eldritch horrors 9cf92c012d report inherit attr errors at the duplicate name
previously we reported the error at the beginning of the binding
block (for plain inherits) or the beginning of the attr list (for
inherit-from), effectively hiding where exactly the error happened.

this also carries over to runtime positions of attributes in sets as
reported by unsafeGetAttrPos. we're not worried about this changing
observable eval behavior because it *is* marked unsafe, and the new
behavior is much more useful.

(cherry picked from commit 1edd6fada53553b89847ac3981ac28025857ca02)
Change-Id: I2f50eb9f3dc3977db4eb3e3da96f1cb37ccd5174
2024-03-18 16:12:45 +01:00
eldritch horrors d826427f02 normalize formal order on ExprLambda::show
we already normalize attr order to lexicographic, doing the same for
formals makes sense. doubly so because the order of formals would
otherwise depend on the context of the expression, which is not quite as
useful as one might expect.

(cherry picked from commit 4147ecfb1c51f3fe3b4adcbd4e753fd487dab645)
Change-Id: I3fd0dbdef3ac7447a3a03ff20bb514a0d0f23fb1
2024-03-18 07:56:34 -06:00
eldritch horrors 314f044c2b keep copies of parser inputs that are in-memory only
the parser modifies its inputs, which means that sharing them between
the error context reporting system and the parser itself can confuse the
reporting system. usually this led to early truncation of error context
reports which, while not dangerous, can be quite confusing.

(cherry picked from commit d384ecd553aa997270b79ee98d02f7cf7e1849e6)
Change-Id: I677646b5675b12b2faa787943646aa36dc6e6ee3
2024-03-18 07:56:23 -06:00
jade 61e21b2557 Delete hasPrefix and hasSuffix from the codebase
These now have equivalents in the standard lib in C++20. This change was
performed with a custom clang-tidy check which I will submit later.
Executed like so:

ninja -C build && run-clang-tidy -checks='-*,nix-*' -load=build/libnix-clang-tidy.so -p .. -fix ../tests | tee -a clang-tidy-result

Change-Id: I62679e315ff9e7ce72a40b91b79c3e9fc01b27e9
2024-03-17 20:17:19 -07:00
jade 886a418d23 builtins.nixVersion: return fixed fake version
This builtin is only going to cause us problems because we are not Nix,
so let's just falsify being in the 2.18 series, since that is the
closest target that has any meaning.

In future we might want to have a better feature detection mechanism,
for when we actually add stuff to some builtin's attr set argument. But
builtins.nixVersion is just going to be hopelessly broken and it should
be stubbed out.

Fixes lix-project/lix#144

Change-Id: Id7390b32a29c6147f2977737d81846320de5d67e
2024-03-17 00:32:19 -07:00
eldritch horrors 11f35afa6f diagnose duplicated attrs at correct path
diagnose attr duplication at the path the duplication was detected, not
at the path the current attribute wanted to place. doing the latter is
only correct if a leaf attribute was duplicated, not if an attrpath was
set to a non-attrset in one binding and a (potentially implied) attrset
in another binding.

fixes #124

Change-Id: Ic4aa9cc12a9874d4e7897c6f64408f10aa36fc82
2024-03-16 22:12:49 +01:00
Rebecca Turner 7abbce500b
Fix gc-small-vector.hh includes
Change-Id: I4abc19029fb62712582761d4fc1895156b68803d
2024-03-15 13:39:32 -07:00
eldritch horrors c26599b143 libexpr: fix elided value counting in printer
using the total-attrs-printed and total-list-items-printed counters to
calculate how many attrs were elided only works properly if no nesting
is involved. once things do nest the global counter can exceed the size
of the currently printed object, leading to unsigned wrapping and great
overestimation of elided counts. counting locally in addition to global
counts fixes this.

these are functional tests because creating these objects requires the
evaluator to not be a huge amount of code, and we also want defaults to
be tested for cli usage.

fixes #14

Change-Id: Icb9a0cb21b2f4bacbc5e9dcdd8c0b9055b4088a7
2024-03-14 01:52:19 -06:00
jade 50c401b4c1 Merge "util.hh: split out signals stuff" into main 2024-03-11 11:14:24 -06:00
jade 8be7030299 util.hh: split out signals stuff
Copies part of the changes of ac89bb064aeea85a62b82a6daf0ecca7190a28b7

Change-Id: I9ce601875cd6d4db5eb1132d7835c5bab9f126d8
2024-03-11 00:52:09 -07:00
eldritch horrors 0a4737f519 add doc comment justifying ExprInheritFrom
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
(cherry picked from commit f24e445bc024cfd3c26be5f061280af549321c22)
Change-Id: I7acda5d5c34c0914a78adc2385d32782c4c275cd
2024-03-10 03:18:32 -06:00
eldritch horrors 06764118ea remove ExprAttrs::AttrDef::inherited
it's no longer widely used and has a rather confusing meaning now that
inherit-from is handled very differently.

(cherry picked from commit 1cd87b7042d14aae1fafa47b1c28db4c5bd20de7)
Change-Id: I90bbebddf06762960d8ca4f621cf042ce8ae83f9
2024-03-10 03:18:32 -06:00
eldritch horrors b667b4cded evaluate inherit (from) exprs only once per directive
desugaring inherit-from to syntactic duplication of the source expr also
duplicates side effects of the source expr (such as trace calls) and
expensive computations (such as derivationStrict).

(cherry picked from commit cefd0302b55b3360dbca59cfcb4bf6a750d6cdcf)
Change-Id: Iff519f991adef2e51683ba2c552d37a3df7a179e
2024-03-10 03:18:32 -06:00
eldritch horrors 71e0114708 remove getDerivations deduplication
deduplication does not currently work fully, showing derivations
multiple times if they have different underlying values. this can happen
by selecting the same derivation twice for two different attributes of a
set, using inherit-from (which reduces to the previous), importing
nixpkgs twice, or any other number of things.

since users already have to deal with duplicates for this reason it
won't hurt to add *more* duplicates. the alternative would be to
deduplicate fully, which would drop derivations that are currently
returned and those pose a regression risk.

Change-Id: I64b397351237e10375d270f1bddecb71f62aa131
2024-03-10 03:18:32 -06:00
eldritch horrors 2a84123631 group inherit by source during Expr::show
for plain inherits this is really just a stylistic choice, but for
inherit-from it actually fixes an exponential size increase problem
during expr printing (as may happen during assertion failure reporting,
on during duplicate attr detection in the parser)

(cherry picked from commit ecf8b12d60ad2929f9998666cf0966475b91e291)
Change-Id: Ie55f0cb01a37e766414c31f8d40f51c2c7d106b0
2024-03-10 03:18:32 -06:00
eldritch horrors bf19eebb9b use the same bindings print for ExprAttrs and ExprLet
this also has the effect of sorting let bindings lexicographically
rather than by symbol creation order as was previously done, giving a
better canonicalization in the process.

(cherry picked from commit 6c08fba533ef31cad2bdc03ba72ecf58dc8ee5a0)
Change-Id: Ia887f629305645bb8a165fbbc0d32e620912595a
2024-03-10 03:18:32 -06:00
eldritch horrors 1cf0fa0633 add ExprAttrs::AttrDef::chooseByKind
in place of inherited() — not quite useful yet since we don't
distinguish plain and inheritFrom attr kinds so far.

(cherry picked from commit 1f542adb3e18e7078e6a589182a53a47d971748a)
Change-Id: If948c9d43e875de18f213a73a06a36f7c335b536
2024-03-10 03:18:32 -06:00
eldritch horrors 03f852b2c6 preserve information about whether/how an attribute was inherited
(cherry picked from commit c66ee57edc6cac3571bfbf77d0c0ea4d25b4e805)
Change-Id: Ie8606a8b2f5946c87dd4d16b7b46203e199a4cc1
2024-03-10 03:18:32 -06:00
eldritch horrors a9b813cc3b Merge pull request #10066 from 9999years/print-all-frames
Do not skip any stack frames when `--show-trace` is given

(cherry picked from commit 0b47783d0a879875d558f0b56e49584f25ceb2d0)
Change-Id: Ia0f18266dbcf97543110110c655c219c7a3e3270
2024-03-09 10:17:26 -07:00
eldritch horrors f2e11ddce1 Merge pull request #9914 from 9999years/debugger-on-trace
Enter debugger on `builtins.trace` with an option

(cherry picked from commit 774e7ca5847ebc392eac2a124a8f12b24da4f65a)
Change-Id: If01e2110b3a128e639b05143227e365227d149f1
2024-03-09 10:17:26 -07:00
eldritch horrors 992d99592f :quit in the debugger should quit the whole program
(cherry picked from commit 2a8fe9a93837733e9dd9ed5c078734a35b203e14)
Change-Id: I71dadfef6b24d9272b206e9e2c408040559d8a1c
2024-03-09 10:17:26 -07:00
eldritch horrors 6b11c2cd70 Extract printSpace helper
(cherry picked from commit 403c90ddf58a3f16a44dfe1f20004b6baa4e5ce2)
Change-Id: I53c9824e6b1c4c619b4dfd8346d39e5289d92265
2024-03-09 07:20:23 -07:00