Commit graph

16580 commits

Author SHA1 Message Date
jade 5530de4673 terminal code eaters: implement OSC
This is a useful piece of functionality to being able to eat URL
hyperlinks, for instance, which is a bug that Lix has while dealing with
terminal output today.

Change-Id: I77b2de107b2525cad7ea5dea28bfba2cc78b9e6d
2024-12-10 15:43:31 -08:00
jade faf00ad022 libexpr: significantly improve error messages for bad attr paths
This commit makes Lix include the summarized content of the value being
indexed when it is bad.

lix/lix2 » nix eval --expr '{x.y = 2;}' 'x.y.z'
error: the value being indexed in the selection path 'x.y.z' at 'x.y' should be a set but is an integer: 2

lix/lix2 » nix eval --expr '{x.y = { a = 3; };}' 'x.y.z'
error: attribute 'z' in selection path 'x.y.z' not found inside path 'x.y', whose contents are: { a = 3; }
       Did you mean a?

lix/lix2 » nix eval --expr '{x.y = { a = 3; };}' 'x.y.1'
error: the expression selected by the selection path 'x.y.1' should be a list but is a set: { a = 3; }

Change-Id: I3202aba0e437e00b4c6d3ee287a2d9a7c6892dbf
2024-12-10 15:43:31 -08:00
jade c0808bd855 tests/functional2: add terminal code eater
I want this for being able to write reasonable expect-test style tests
for oneliners. We will still probably want something like insta for more
complicated test cases where you actually *want* the output in a
different file, but for now this will do.

cc: #595
Change-Id: I6ddc42963cc49177762cfca206fe9a9efe1ae65d
2024-12-10 15:43:31 -08:00
jade 93d5221b9b tests/functional2: fix occasional pytest haunting
I don't know what the heck the xonsh module is doing but its obviously
crimes so it has to go. It was never intended to be running here anyway.

Fixes: #593
Change-Id: I1877698469392f85884945aaa60987c68c4e0ebc
2024-12-10 13:32:28 -08:00
jade 21ad02c1d0 attr path parser: fix bug in not rejecting empty attr paths, add unparser
The following behaviour was previously present and has been fixed:

lix/lix2 » nix eval --expr '{x."" = 2;}' 'x.""'
{ "" = 2; }
lix/lix2 » nix eval --expr '{x."".y = 2;}' 'x."".y'
error: empty attribute name in selection path 'x."".y'

Change-Id: Iad21988f1191c33a3661c72a6b7f01a8b8b3e6eb
2024-12-10 13:32:28 -08:00
jade 2a9e560570 Merge "libcmd: munge evalSettings in the right place" into main 2024-12-06 01:25:12 +00:00
eldritch horrors fb76a4d335 libcmd: munge evalSettings in the right place
these must be tampered with before the evaluator is created, *never*
after. doing it any other way leads to interesting things like #596.

fixes #596

Change-Id: Iea253ccce44b94b1243833837a3df93c795967d9
2024-12-06 00:59:53 +01:00
jade 9dd58224ea libexpr: rename misleading type alias Bindings::size_t
Calling a custom type size_t is incredibly sneaky and makes all the code
around this extremely context dependent.

Change-Id: Idae684781f45fe615020d8642f12a656ab2c15ad
2024-12-05 12:24:15 -08:00
eldritch horrors 61eed2c97c libexpr: fully separate evaluator and running evaluation
this finally gives us a witness type we can use to prove that a certain
call graph subtree can't be used in kj promises using only a single new
assumption: if EvalState& is never held as a reference member of a type
and instead only ever passes as an argument or held on the stack we can
be certain that anything that has access to en EvalState ref must never
be run inside a promise and, crucially, that anything that doesn't have
access to an EvalState& *can* be run inside a promise without problems.

Change-Id: I6c15ada479175ad7e6cd3e4a729a5586b3ba30d6
2024-12-05 13:30:35 +00:00
eldritch horrors 07a26fc776 libexpr associate caches and statistics with EvalContext
Change-Id: Ie4740070881bdd056b112ffd79634e30c29e8aaa
2024-12-04 23:50:31 +00:00
eldritch horrors 71c392ae0f libexpr: associate parsing and builtins with EvalContext
Change-Id: Iff56246bb344405c9acecb967423554d25fe47f5
2024-12-04 23:50:31 +00:00
eldritch horrors 7fa6e785ff libexpr: associate path config with EvalContext
Change-Id: Id25f990ab4ae9d9e76c91c07d7397689acf57eca
2024-12-04 23:50:31 +00:00
eldritch horrors 1f37fc85ac libexpr: associate memory with EvalContext
Change-Id: I791ae16cd42a674972c782a869b7d790880313ff
2024-12-04 23:50:31 +00:00
eldritch horrors 5db5eed43d libexpr: associate stores and repair flag with EvalContext
Change-Id: I5fd911415707f0b250888dcabdf6290691fcdbf3
2024-12-04 23:50:31 +00:00
eldritch horrors d30027a37a libexpr: associate error creation and debugging with EvalContext
Change-Id: I80beedd0c58361ebc67cebcee97d64879baed050
2024-12-04 23:50:31 +00:00
eldritch horrors 2f2bc90d92 libexpr: associate positions with EvalContext
Change-Id: I43e827a9119324c23a6d72cfc1b7ed4d5ff32dd2
2024-12-04 23:50:31 +00:00
eldritch horrors 014845ddf1 libexpr: associate symbols with EvalContext
Change-Id: I4ca99dc1f5183c7e7383db2b9c896c1946aadd29
2024-12-04 23:50:31 +00:00
eldritch horrors 8e74d1ceeb libexpr: extract persistent eval state to new struct
"persistent" here means state that outlives any particular evaluation,
i.e. everything that exist outside of a call to eval() and friends. do
note that this leave EvalState nearly empty, containing only things of
importance to evaluation *itself*, e.g. the call depth tracker. do not
worry about the self-reference ctx member. nothing to see here, go on.

Change-Id: Ie8d0af3f09d33902f81e0c36a1096dd9f334a537
2024-12-04 23:48:47 +00:00
eldritch horrors ce698198f0 libcmd: pass EvalState& to installables
for the same reason the eval caches should not keep them as references,
only worse: installables cannot fully lose their reference member since
some types of installable (flakes -sigh-) need access to caches held by
their member reference. passing that around as well at all times is not
feasible; passing in EvalState& all the time is quite bad enough as is.

Change-Id: I05d438623b5ef10fc22e7e256b89447e61a0657d
2024-12-03 20:38:41 +01:00
eldritch horrors 81559ea8ad treewide: add evaluator aliases for eval states
this is not necessary in any way, but it will make the following changes
smaller and easier to review. the aliases could also be added piecemeal,
but doing it here lets us lean heavily on our compilers for correctness.

(teacher notes: here the author foreshadows the shape of things to come.
not all names change, and only the names unchanged are those which will,
over time, become ever more unrecognizable. note especially nix/main.cc,
where `state` is not only cloned, but itself changes pointerness. it can
be seen as a nod to the trans community, but more realistically it is no
more than foreshadowing the future where `state` is only seen by proxy.)

Change-Id: I7732025e58df089b7f8e564fc63960cd91729d09
2024-12-03 20:38:41 +01:00
eldritch horrors 473c1bdcab libexpr: remove EvalState& from EvalCache constructor
it's not used for anything any more. it was only passed on to the AttrDb
constructor, which itself only stashed it away to an unused member field

Change-Id: I334fde751a3754e6580f573c21ae3d04be00345a
2024-12-03 20:38:41 +01:00
eldritch horrors 9ff702554d libexpr: eval caches should not have an EvalState&
and for the same reason DrvInfo shouldn't, only even more so. the eval
cache infrastructure wants to be a wrapper around evaluation, with its
lifetimes fully decoupled from the lifetime of the wrapped evaluation.
this can't not create problems when eval lifetime must become bounded.

Change-Id: Ib8eec649995b4decd7290c2266322f67d73b6b46
2024-12-03 20:38:41 +01:00
eldritch horrors 8468dc65c7 libcmd: DrvInfo should not have an EvalState*
in the future eval states will be asyncio root contexts. proving that
they are used correctly is impossible if the are referenced by things
with weakly scoped lifetimes, or entirely unscoped lifetimes, as we'd
have to deal with in the case of `DrvInfos` passed as out parameters.

Change-Id: I9ca5c46a714c6c914a77c7c7500cb4939ac9aff5
2024-12-03 20:38:41 +01:00
eldritch horrors f47cd0fc88 libcmd, libexpr: remove dead code
Change-Id: I555d81bd78770dd1b773c1e90f1a41779fccad7f
2024-12-03 20:38:41 +01:00
eldritch horrors d6a7fb5f6a cli: remove duplicated declarations
Change-Id: I78c6666cea8b6604a4fae9cb047662571bd6f99b
2024-12-03 20:38:41 +01:00
eldritch horrors a5d02bcdf9 libcmd: make a back-compat hack more robust
otherwise eval fails if getEvalState runs before parseInstallables runs.

Change-Id: Ia0d66dd6297212f7835c4a9b2db0fb4d9e4c9b9d
2024-12-03 20:38:41 +01:00
eldritch horrors 9a3e3a5560 libexpr: standardize on strings for attr cache traversal
it's all strings anyway. the db stores strings, the cli wants to
interact with attr paths as strings, so we will just use strings

Change-Id: Id9ea07d92343de77e8d47af8fec1e86ae225e9a1
2024-12-03 20:38:41 +01:00
eldritch horrors dd7d7450a5 libexpr: move eval statistics into new struct
there's no reason for these to be private. much more important things
that are now part of EvalState are not private, and having statistics
be private forces us to add otherwise unnecessary friend declarations

Change-Id: Ib37097d94a9f55c2b21969fb6c51049b1c914515
2024-12-03 20:38:41 +01:00
eldritch horrors 276a771210 libexpr: move thunk counter into eval state
this is relative to the current state. why was it made global?

Change-Id: I58b3ce0213e791a05ce9fd7a2cb2394b9454c653
2024-12-03 20:38:41 +01:00
eldritch horrors ea931d4d03 libexpr: mkThunk_ -> evalLazily
thunk creation doesn't have to be called that. thunks are more of an
implementation detail, calling it evalLazily seems a lot more clear.

Change-Id: I64bdac422eadb629002195cd6b6a89dce1b4a60f
2024-12-03 20:38:41 +01:00
eldritch horrors 32f7a93f71 libexpr: extract eval path handling into new type
Change-Id: Icf38113076a4dd0f8515a27b501f405033aec73b
2024-12-03 20:38:41 +01:00
eldritch horrors a65e9e5828 libexpr: extract eval error creation into new type
this will let us pass the capability to create debuggable eval errors
without having to pass an entire EvalState. we could pass symbols and
debug states around just as easily, but if we add new capabilities to
our debugger we might have to change many more places than with this.

Change-Id: I2f8893012e5d98a986ef1fc888234c2dd8d5e096
2024-12-03 20:38:41 +01:00
eldritch horrors a4fa93d469 libexpr: simplify EvalErrorBuilder interface
if we hold the error being built behind a pointer we no longer need to
allocate the error builder itself to avoid impacting eval performance.
adding && ref qualifiers to builder functions and adding the nodiscard
attribute to the class itself also makes the nodiscard attributes kept
on the builder functions unnecessary. we do keep the noinlines though,
removing them still regresses eval performance even after this change.

Change-Id: Ibc14a66955ac32142d97fb3680b0a7e14db250cd
2024-12-03 20:38:41 +01:00
eldritch horrors ef1d62ec6c libexpr: abstract DebugState from EvalState
this is necessary to share a debug state between multiple eval states.
while doings so makes little sense at present it will be necessary for
async io support since eval states must be async io roots, which means
we must create them as needed from a shared evaluation context object.

Change-Id: Id9d4b37aae40706f65c741e3b961855582e035ab
2024-12-03 20:38:41 +01:00
eldritch horrors 0bfa58ff53 libexpr: move runtime caches to new struct
Change-Id: Ic5492853d39eaffdf03d961b3e69c93c4a75ae33
2024-12-03 20:38:41 +01:00
eldritch horrors 2e5780ebc8 libexpr: simplify HAVE_BOEHMGC ifdefs
if we define a TraceableAllocator at all times and use that in places
that want maybe-traceable allocation we can simplify things a lot. we
also unconditionally allocate cache root pointers for Value and Env-1
caches, even though we don't need them without gc (they're so cheap).
defaulting to `std::allocator` without gc recovers previous behavior.

Change-Id: I236da8c3b0669b40cdfe355ec3ec4e764d096074
2024-11-29 17:55:08 +01:00
eldritch horrors 1e064e08fa libexpr: move builtin env to new class
Change-Id: Ib76891372be756917bddcf5eaa24ccf8e1288035
2024-11-29 17:55:07 +01:00
eldritch horrors 1ed0814859 libexpr: use eval() for builtin-using value printer tests
if we're relying on the behavior of builtins we can also use the parser.

Change-Id: I682722b2a22fbb6e748da2cab1bfeb76788bfb48
2024-11-29 17:45:04 +01:00
eldritch horrors 12802b492a libexpr: treat derivation like a normal primop for baseEnv construction
that way we don't need to eval while initializing the evaluator. this
will make a lot more sense once we start pushing asyncio through here

Change-Id: I3663052438ed97d48e213b71395ad1dd5e1318bb
2024-11-29 17:45:02 +01:00
eldritch horrors e9520ffd06 libexpr: add hidden pos origin for builtin code
currently only used to hide the nix code of derivation from stack traces
and debug frames, but perhaps we'll find it useful for other things too.

Change-Id: Ie5667873d8858d25dd4113bdf454e800b59082d7
2024-11-29 17:42:15 +01:00
eldritch horrors 63006438c4 libexpr: remove state reference from EvalError
nothing actually needs it. EvalErrorBuilder can have its own reference.

Change-Id: I8bf301d03a9c161519c130c95e58b1d5400e6411
2024-11-29 17:42:15 +01:00
eldritch horrors e44dbfe97a libexpr: move flake.nix attrsetness check out of eval state
this doesn't belong here, getFlake should be handling this instead. we
do lose the debug frame for loading a root flake, but that seems fine.
there's no point in starting a debugger if the root isn't even a flake

Change-Id: I24ad32b6716baee81a1a0f8bf9ce26814d97c7aa
2024-11-29 17:42:15 +01:00
eldritch horrors 94bb66c867 libexpr: remove maxPrimOpArity
this was only used to enable an optimization we no longer need such
strong guarantees for: SmallVector can be used with a dynamic size.

Change-Id: I4513fb7e665827fe363ce6308448656e6c5badb7
2024-11-29 17:42:15 +01:00
eldritch horrors 85d600ca4d libexpr: drop bison-specific \0\0 input trailer
bison needed it for internal reasons, the new parser does not.

Change-Id: I91f7aa23fb151e3152eee793a4fdce423fcf98d9
2024-11-29 17:42:15 +01:00
eldritch horrors 37aeb3059d libexpr: remove EvalState::addErrorTrace
Error::addTrace exists and is used far more often already. let's
standardize on the variant that doesn't need yet more templates.

Change-Id: If66b69ca02dbb546ce98cf385181bd13ce7ad9b5
2024-11-29 14:19:31 +00:00
eldritch horrors 105d8ceb36 libexpr: remove templated forceAttrs
the callback is always called immediately, which defeats the purpose of
the callback and the purpose of the template itself. there seems to not
be any performance impact of this. optimizing Value::determinePos would
be nice, but it's not used nearly often enough to matter at this point.

Change-Id: I2aec6a38103630652112f4b273653f11d2404c04
2024-11-29 14:19:31 +00:00
eldritch horrors 66d7128512 libexpr: remove EvalState parse caches
nothing needs these any more. the CLI calls evalFile, but only in places
where it'll only be called *once* per process lifetime. __import does so
too, but import doesn't care about the parse tree, only the eval result.

interestingly this improves eval performance by 6% on system eval unless
GC never occurs. no idea why this makes a GCing eval faster, but it does

Change-Id: I8289528550244e0a8b5ebc7284d8fb9aaac59e20
2024-11-29 13:29:31 +00:00
eldritch horrors f815b966c4 doc: remove utils.nix
only generate-manpage.nix uses it any more, so we can inline it there
instead of keeping it around as a separate generated header. doing so
will also allow us to remove caching functions needed *only for this*

Change-Id: I97ee91f1dd7140ecb69dbafd8479b82fba7981b8
2024-11-29 13:29:31 +00:00
eldritch horrors 2297d3f895 doc: remove obsolete files
these must've been forgotten during the move to generated builtins.

Change-Id: I0989847abc020e9356b996f26196d2c07953f77b
2024-11-29 13:29:31 +00:00
eldritch horrors 7c650ea241 libexpr: remove eval caches from EvalState
eval caches are not used by actual eval at all, only by the flake-shaped
wrappers around evaluation. moving caches into a subclass both clarifies
that eval caches and eval states are coupled and separates concerns that
should not have been intermixed as they were here. in the future we will
want to split up and decouple things even further. that'll have to wait.

Change-Id: I7b69510c0f8b212f05fae62e7b992d9475b4841f
2024-11-29 13:29:31 +00:00