Commit graph

2184 commits

Author SHA1 Message Date
Rebecca Turner 06996718c3 Merge "repl: tab-complete quoted attribute names" into main 2024-12-11 03:27:15 +00:00
jade cb44fe7c15 Merge changes Ic890f781,I77b2de10,I3202aba0,I6ddc4296,I18776984, ... into main
* changes:
  doc: fix broken table in operators list
  terminal code eaters: implement OSC
  libexpr: significantly improve error messages for bad attr paths
  tests/functional2: add terminal code eater
  tests/functional2: fix occasional pytest haunting
  attr path parser: fix bug in not rejecting empty attr paths, add unparser
2024-12-11 02:28:11 +00:00
Ian Chamberlain 67f07e05df
repl: tab-complete quoted attribute names
Attribute names containing special characters like @ or . need to be
quoted, so we need to do our own tokenization of the command line for
completion, and quote the attribute names when we provide the completion.

Fixes: #450

Change-Id: I55a30dd272880c89445d9ded49b3f2c90cb19326
2024-12-10 18:10:16 -08:00
beviu bc63b007e5
Mention support for ~/ paths in nix.conf in manual
Fixes: #497

Change-Id: I9606900d17f62359cef4fe2c8a01e5791390a870
2024-12-10 16:36:35 -08:00
jade f51943f171 doc: fix broken table in operators list
Fixes: #597
Change-Id: Ic890f781af10fbcab23deae5ab9c10eebfbab070
2024-12-10 15:43:31 -08:00
jade faf00ad022 libexpr: significantly improve error messages for bad attr paths
This commit makes Lix include the summarized content of the value being
indexed when it is bad.

lix/lix2 » nix eval --expr '{x.y = 2;}' 'x.y.z'
error: the value being indexed in the selection path 'x.y.z' at 'x.y' should be a set but is an integer: 2

lix/lix2 » nix eval --expr '{x.y = { a = 3; };}' 'x.y.z'
error: attribute 'z' in selection path 'x.y.z' not found inside path 'x.y', whose contents are: { a = 3; }
       Did you mean a?

lix/lix2 » nix eval --expr '{x.y = { a = 3; };}' 'x.y.1'
error: the expression selected by the selection path 'x.y.1' should be a list but is a set: { a = 3; }

Change-Id: I3202aba0e437e00b4c6d3ee287a2d9a7c6892dbf
2024-12-10 15:43:31 -08:00
piegames 7c76053d93 libexpr: Undeprecate overriding __nixPath
It being overridable was an intended feature with good use cases, and
should not have been removed. However, this feature is generally in a
bad state and needs revisiting in the future.

Fixup for 81d5f0a7d9
Fixes #599

Change-Id: I2d93e012caa65aa795bce3a71d8e56d7052ef9df
2024-12-10 19:02:41 +01:00
piegames a2ae14bfd8 libexpr: Rework error messages on ExprSelct::eval
Calls to `show` have been removed. To counter the loss of information,
the error positions have been improved and now correctly point to the
current selector instead of the entire select expression.

Change-Id: I4771fe874af1ac15828a9863550cd4369a8f0e94
2024-12-08 20:52:18 +01:00
piegames cdda2454f6 libexpr: Assert: Don't print assertion in error message
The `show` functionality needs to be removed because it is deeply
flawed, and given that we already print position information in the
error message (which probably wasn't always the case in the past) the
assertion printing is redundant anyways.

Change-Id: I1f5e05ab73aaa0ec92994c2211463260fd374898
2024-12-08 20:51:44 +01:00
eldritch horrors f815b966c4 doc: remove utils.nix
only generate-manpage.nix uses it any more, so we can inline it there
instead of keeping it around as a separate generated header. doing so
will also allow us to remove caching functions needed *only for this*

Change-Id: I97ee91f1dd7140ecb69dbafd8479b82fba7981b8
2024-11-29 13:29:31 +00:00
eldritch horrors 2297d3f895 doc: remove obsolete files
these must've been forgotten during the move to generated builtins.

Change-Id: I0989847abc020e9356b996f26196d2c07953f77b
2024-11-29 13:29:31 +00:00
piegames 81d5f0a7d9 libexpr: Deprecate overriding __sub and the like
It was never intended to be a feature to be used, and moreover it is
inconsistent: One cannot override `+`, and overriding `__lessThan` won't
affect the builtins which do comparisons.

Change-Id: Iaba54a05aa4c2eb37cdb3dc0d731fcee5a86deba
2024-11-28 18:15:52 +01:00
eldritch horrors b0d7a81613 fix tooling after include reorganization
clangd broke because it can't look through symlinks. compile_commands
manipulation does not fix it, clangd configuration does not fix it, a
vfs overlay does not fix it, and while a combination of those can fix
it with a bind mount in place that's just too cursed to even consider

clangd bug: https://github.com/llvm/llvm-project/issues/116877

Change-Id: I8e3e8489548eb3a7aa65ac9d12a5ec8abf814aec
2024-11-19 22:55:32 +00:00
jade 0bec915522 Merge "unnamed threads: Obliterate" into main 2024-11-19 05:02:48 +00:00
jade 519957bd59 unnamed threads: Obliterate
Ever read gdb output and you just kinda get a headache because you have
to infer what a thread is by reading the stack trace? It's not hard, but
we could also just never have to do that again, which is also not hard.

Sample:

(gdb) info thr
  Id   Target Id                    Frame
* 1    LWP 3719283 "nix-daemon"     0x00007e558587da0f in accept ()
   from target:/nix/store/c10zhkbp6jmyh0xc5kd123ga8yy2p4hk-glibc-2.39-52/lib/libc.so.6
  2    LWP 3719284 "signal handler" 0x00007e55857b2bea in sigtimedwait ()
   from target:/nix/store/c10zhkbp6jmyh0xc5kd123ga8yy2p4hk-glibc-2.39-52/lib/libc.so.6

The API design for this is forced by the macOS pthread_setname_np only
being able to change the current thread's name, but if we just conform
everything to that, it works everywhere.

Change-Id: I2b1d6ed41e3c94170cb0b4e73ad66f239ebd9c88
2024-11-18 18:53:40 -08:00
jade 33e832ba5a Merge "manual: Make link to install page clickable" into main 2024-11-19 01:47:03 +00:00
jade fbdb177b31 manual: Make link to install page clickable
Closes: https://github.com/lix-project/lix/pull/11

Thanks to xsova for suggesting this.

Change-Id: I5958fa4438f4b83d928e8f309aab9a33fb4cbedd
2024-11-18 16:32:20 -08:00
jade b62cc7b361 meson: prepare for include rearrangement
Context: we have include paths that are "types.hh" and similarly common
names. We currently have these compatibly available as
"lix/libutil/types.hh" externally but *not yet internally*. This is
because we don't have any way for the src directory to appear as
`"lix/"` from inside of Lix: the lix/ include directory is created by
the install process.

The goal of this whole thing is to make it clearer which component of
Lix that files are a part of, which should hopefully help at least a
little bit to new developers. One disadvantage of un-mixing these is
that it will cause some API changes if we ever move a file between
libraries, but that is not very common, and we don't care that much
about external API users.

This was planned for a while and is why we have a FixIncludes check to
begin with.

Personally I don't see a great benefit in rearranging our source code,
and in fact, it would probably be counterproductive:
- Moving the includes into a separate `include/` directory would just
  make developers have to deal with more directories, when we can
  already generate the desired layout through the build process.
- This would also decouple the .cc and .hh files which currently
  conventionally have each others' definitions and declarations
  respectively, right next to each other, making it easier for them to
  feel decoupled and diverge.

Content: Add ../include as an include directory so that lix/ in include
paths will resolve to src/ within Lix itself, just as it does externally
today. This prepares for a further series of commits applying the actual
change to each library one-by-one by accepting both include versions at
once.

This could have been done with ../ and a symlink called lix, but we
would like to not accept libexpr/foo.hh internally for it would be
broken externally, so we need an otherwise empty directory for the
include.

Change-Id: Ideac17faadae2bcea2dffbab34eb27c582ede399
2024-11-18 12:22:39 -08:00
alois31 c1746300a9
libexpr: generate builtins from data
Most builtins are now generated from data too, with two exceptions:
* Undocumented builtins, since supporting them would add complexity to the
  generator, the harms of the current implementation mostly don't apply, and
  the proper fix is to document them.
* `derivation` is somewhat magic (it is a function, but defined in the code as
  a constant), so the current treatment of having it separately documented is
  kept (for now, at least).
Since it is slightly easier to do and probably a good idea anyway, the builtin
function registrations generated this way are now processed directly in code
and don't go through global variables any more.
Unfortunately, a slight breaking change is introduced because the order of the
builtins' names in the symbol table changes. Hopefully, this will turn out to
not matter in practice.

Change-Id: I7b4379a93ae380b6524e41a916a21c5c6f70555e
2024-11-09 16:17:26 +01:00
alois31 2e0c1a5ea9
treewide: generate global settings from data
Change-Id: If048b73a55d42522827eb9c140a066ba061e957c
2024-11-09 16:17:14 +01:00
alois31 21fc0ddce5
libutil: generate experimental and deprecated features from data
Currently, a bunch of documentation is generated by embedding parts of it in
the nix executable, getting it out again by running it, and then postprocessing
the output. This is bad, since it creates a pointless dependency of the
documentation on the executable, and also makes documentation generation
impossible when cross-compiling.
Instead, both the code and the documentation should be generated from data, see
#292 . Here we start applying
this approach to the experimental and deprecated features, which are done in
one go since the technical implementation is very similar.
Of course, the actual benefits are not realised yet, since the offending
pattern is used in several more places. These will be fixed later.

Change-Id: I4c802052cc7e865c61119a34b8f1063c4decc9cb
2024-11-09 16:05:12 +01:00
Lily Ballard d0e0969810 Merge "feat: Add temp-dir setting" into main 2024-11-09 08:44:58 +00:00
Maximilian Bosch 116895acb1 Merge "libexpr/flake: (opinionated) changes to interactive flake config" into main 2024-11-08 17:35:38 +00:00
Maximilian Bosch 86eddb9e27
libexpr/flake: (opinionated) changes to interactive flake config
So I recently saw it the first time in the wild, I liked that you get
interactively asked about the nix.conf settings from the flake, but
there were a few minor things that I'd like to see changed:

* The `(y/N)` was somewhere in the middle of the line. Moved it to
  the end. At first I assumed it was a bug because another thread into
  my terminal while I was answering the question.

* I had to say no four times for a single flake with two options. So if
  you already know you don't want any of the config for _this_ flake, I
  found a `No to all` switch that ignores the rest of the nix.conf
  settings a little more ergonomic than having to stop the invocation,
  looking up the exact wording of `--no-accept-flake-config` and
  restarting it. Hence, I added it.

* Added a note where the choices which settings to trust are persisted.
  My initial assumption was that this went into `nix.conf` which is not
  writable on NixOS, so I said no there as well.

Change-Id: I0a0d9c403f0662df4707697a77f08e6cd003ec6f
2024-11-07 21:47:26 +01:00
Lily Ballard 3c8096e5cb feat: Add temp-dir setting
This adds a new temp-dir setting for controlling the temporary directory
without having to change the TMPDIR env var. This can be used to e.g.
use a path on a case-sensitive store on macOS for temporary files
without changing the TMPDIR var used by interactive shells or commands
invoked with `nix run`.

This also stops unsetting `TMPDIR` on darwin when the env var value
starts with `/var/folders/`, preferring instead to just do the check
when reading `TMPDIR`. This way the inherited `TMPDIR` env var is
preserved for child processes (such as interactive shells).

As a side effect this changes the behavior of `nix-build -o ''` to act
like `nix-build --no-out-link` instead of failing with an error caused
by trying to create a symlink at the cwd.

Fixes: #253
Fixes: #112
Change-Id: I9ee826323f2deca62854715a77ca7a373a948a29
2024-11-06 18:11:47 -08:00
Lily Ballard 72cce7be3f Merge "libstore: replace random() calls with atomic counter" into main 2024-11-07 00:15:23 +00:00
Kiara Grouwstra 72292671a9 Merge "fix(libfetchers): set GitHub API version header, closes #255" into main 2024-11-05 22:44:29 +00:00
kloenk 6b7076f81c Merge "add .mailmap" into main 2024-11-04 13:16:16 +00:00
kloenk d73211356a add .mailmap
Add .mailmap file to allow display/email changes in the git log
without rewriting the history

Change-Id: Ie507aba563cd4fa8ba24e65269aefc647c6376ed
2024-11-04 13:46:10 +01:00
raito 763a61bb7c Merge "Revert "Reject weak hash algorithms as SRIs, and warn in any other cases"" into main 2024-11-02 19:50:52 +00:00
raito 6e2349d2e1 Revert "Reject weak hash algorithms as SRIs, and warn in any other cases"
This reverts commit 02c35ea9df.

Reason for revert: this code path is also used for `Input::getRev()`, i.e. flakes VCS revision validation, which, in the case of Git, are using SHA1.
As a result, this cause too much noise due to SHA1 revisions in Flakes.

Change-Id: I8064c1ebc26e4e83b627f0803a7a9ba56cfe1f37
2024-11-01 11:59:59 +00:00
Lily Ballard 834450e237 Merge changes Ibb849b68,I501397c8 into main
* changes:
  libstore: ignore broken symlinks in ssl-cert-file default
  change-authors: add lilyball
2024-11-01 03:51:21 +00:00
Aria 8005d17365 Merge "Reject weak hash algorithms as SRIs, and warn in any other cases" into main 2024-10-31 22:13:27 +00:00
Lily Ballard 69957a971e libstore: replace random() calls with atomic counter
random() is not thread-safe, it relies on global state, and calling it
from worker threads can result in multiple threads producing the same
value. It also doesn't guarantee unique values even in single-threaded
use.

Use an atomic counter for the use-case of generating temporary paths,
and switch to a thread-local RNG for the one remaining call.

This will probably fix https://github.com/NixOS/nix/issues/7273 though
I'm not willing to risk corrupting my store to find out.

Change-Id: I4c4c4c9796613573ffefd29cc8efe3d07839facc
2024-10-30 19:54:43 -07:00
Lily Ballard 684f93e783 libstore: ignore broken symlinks in ssl-cert-file default
Also tweak `pathAccessible` to ignore other relevant errors too. It was
documented as ignoring permission errors but it was only ignoring
`EPERM`, which comes from the darwin sandbox, and not ignoring `EACCESS`
which is the real permission error. I figured it also makes sense to
ignore `ELOOP`.

Fixes: #560
Change-Id: Ibb849b68d07386eb80afb52b57f7d12b3a48a202
2024-10-30 19:50:38 -07:00
Lily Ballard 11950a0a79 change-authors: add lilyball
Change-Id: I501397c8e3a215a2ccb6074100e2508bae98d1a9
2024-10-30 19:50:38 -07:00
Linus Heckemann c95b73d8a1 Merge "libstore: report all differing outputs rather than just the first" into main 2024-10-30 19:04:57 +00:00
Kiara Grouwstra a778b0f85a fix(libfetchers): set GitHub API version header, closes #255
Sets the `X-GitHub-Api-Version` header to `2022-11-28` for calls to the
GitHub API.
This follows the later version as per
https://docs.github.com/en/rest/about-the-rest-api/api-versions?apiVersion=2022-11-28.

This affected the check on whether to use the API versus unauthenticated
calls as well, given the headers would no longer be empty if the
authentication token were missing.
The workaround used here is to use a check similar to an existing
check for the token.

In the current implementation, headers are (still) similarly sent to
non-authenticated as well as GitHub on-prem calls.
For what it's worth, manual curl calls with such a header seemed to
break nor unauthenticated calls nor ones to the github.com API.

Change-Id: I6e10839e6b99cb65eb451e923b2a64f5d3c0f578
2024-10-30 18:40:13 +01:00
V. 56ead73fda Merge "chore: remove monolithic coreutils requirement" into main 2024-10-30 16:53:43 +00:00
V. fb1b211037 chore: remove monolithic coreutils requirement
It's only used in a couple of tests, and only in such a way that
replacing it with a random command suffices.
I also removed a few pointless uses of the variable.

Fixes: #376
Change-Id: I90aedb61d64b02f7c9b007e72f9d614cc1b37a2e
2024-10-30 15:12:35 +04:00
Linus Heckemann 8b0ac51f12 libstore: report all differing outputs rather than just the first
Before:

error: derivation '/nix/store/4spy3nz1661zm15gkybsy1h5f36aliwx-python3.11-test-1.0.0.drv' may not be deterministic: output '/nix/store/ccqcp01zg18wp9iadzmzimqzdi3ll08d-python3.11-test-1.0.0-dist' differs from '/nix/store/ccqcp01zg18wp9iadzmzimqzdi3ll08d-python3.11-test-1.0.0-dist.check'

After:

error: derivation '4spy3nz1661zm15gkybsy1h5f36aliwx-python3.11-test-1.0.0.drv' may not be deterministic: outputs differ
         output differs: output '/nix/store/ccqcp01zg18wp9iadzmzimqzdi3ll08d-python3.11-test-1.0.0-dist' differs from '/nix/store/ccqcp01zg18wp9iadzmzimqzdi3ll08d-python3.11-test-1.0.0-dist.check'
         output differs: output '/nix/store/yl59v08356i841c560alb0zmk7q16klb-python3.11-test-1.0.0' differs from '/nix/store/yl59v08356i841c560alb0zmk7q16klb-python3.11-test-1.0.0.check'

Change-Id: Ib2871fa602bf1fa9c00e2565b3a2e1b26f908152
2024-10-29 18:34:54 +01:00
eldritch horrors 10488f7431 libstore: use curl content-encoding support, not our own
let's use the automatic decoding functions curl provides instead of
implementing them ourselves for the dubious ability to support both
xz and bzip2 encodings as well, neither of which anything will send

Change-Id: I3edfebeb596a0e9d5c986efca9270501c996f2dd
2024-10-28 18:52:49 +00:00
Aria 02c35ea9df Reject weak hash algorithms as SRIs, and warn in any other cases
Fixes #114

Change-Id: Ib9e68edfed5c186a029531e1eb9bda9d2e338e54
2024-10-26 22:29:54 +01:00
eldritch horrors 67f1aafd61 libstore: restrict curl protocols
previously it was possible to fetchurl a dict server, or an ldap server,
or an imap server. this is a bit of a problem, both because rare schemes
may not be available on all systems, and because some schemes (e.g. scp)
are inherently insecure in potentially surprising ways we needn't allow.

Change-Id: I18fc567c6f58c3221b5ea8ce927f4da780057828
2024-10-23 11:32:14 +00:00
jade 4180b84a67 testsuite: use xdist for parallel test running
This is capped at 12 because 3.7 seconds of startup is painful enough
and 5.5 seconds with 24 was more annoying.

Change-Id: I327db40fd98deaa5330cd9cf6de99fb07b2c1cb0
2024-10-09 14:47:39 -07:00
Rebecca Turner 0d484aa498
Add release note for CTRL-C improvements
I'm very excited for cl/2016, so others will probably be excited also!
Let's add a release note.

Change-Id: Ic84a4444241aafce4cb6d5a6d1dddb47e7a7dd7b
2024-10-05 10:40:51 -07:00
alois31 5df2cccc49
doc: install the HTML manual again
In 0e6b3435a1, installation of the HTML manual
was accidentally dropped: setting install_dir on a custom_target only sets the
directory where something is going to be installed if it is installed at all,
but does not itself trigger installation. The latter has to be explicitly
requested, which is just what we do here to get the manual back.

Change-Id: Iff8b791de7e7cb4c8d747c2a9b1154b5fcc32fe0
2024-10-05 10:49:34 +02:00
Jonas Chevalier a16ceb9411 Merge "fix(nix fmt): remove the default "." argument" into main 2024-09-30 16:10:32 +00:00
Maximilian Bosch 289e7a6b5a Merge "libfetchers/git: restore compat with builtins.fetchGit from 2.3" into main 2024-09-29 08:56:16 +00:00
Maximilian Bosch 04daff94e3
libfetchers/git: restore compat with builtins.fetchGit from 2.3
Since fb38459d6e, each `ref` is appended
with `refs/heads` unless it starts with `refs/` already. This regressed
two use-cases that worked fine before:

* Specifying a commit hash as `ref`: now, if `ref` looks like a commit
  hash it will be directly passed to `git fetch`.

* Specifying a tag without `refs/tags` as prefix: now, the fetcher prepends
  `refs/*` to a ref that doesn't start with `refs/` and doesn't look
  like a commit hash. That way, both a branch and a tag specified in
  `ref` can be fetched.

  The order of preference in git is

  * file in `refs/` (e.g. `HEAD`)
  * file in `refs/tags/`
  * file in `refs/heads` (i.e. a branch)

  After fetching `refs/*`, ref is resolved the same way as git does.

Change-Id: Idd49b97cbdc8c6fdc8faa5a48bef3dec25e4ccc3
2024-09-28 14:52:06 +02:00