Commit graph

2061 commits

Author SHA1 Message Date
jade 59b5965bbf doc: Write an index of environment variables used in testsuite
This was originally going to be just the testsuite but I kinda just
documented all of them.

I am tired of us not documenting these. This is a starting point to
producing an actually good index. I would like to enforce it in a
pre-commit hook eventually that we document all environment variables
used in Lix itself, even if it is terse dev facing docs.

This is full of a bunch of TODOs caused by auditing code. They should
probably be done at some point.

Change-Id: I7c0d3b257e19bae23d47d1efbd7361d203bccb0e
2024-06-12 15:34:23 -07:00
jade 211f79d4a2 doc/testing.md: Rewrite some outdated sections for meson and current source layout
Change-Id: Ia23f82c9a564b55bd799afbda59c28c9b0a65c13
2024-06-12 15:34:23 -07:00
jade 5f6eb6eb44 doc: rewrite the multi-user documentation to actually talk about security
It's in the security section, and it was totally outdated anyway.

I took the opportunity to write down the stuff we already believed.

Change-Id: I73e62ae85a82dad13ef846e31f377c3efce13cb0
2024-06-12 15:34:23 -07:00
jade 4392d89eea manual: rewrite the docker guide now that we have images
Change-Id: I5bdf47e67059ae4099552750a47ae070dbe094df
2024-06-09 20:33:24 -07:00
jade 611b1de441 Add meson release note
Change-Id: I4d2d08dc77a3ab4dce9fbb129c1487aa8c9f1722
2024-06-06 15:08:12 -07:00
jade 24057dcb6a Remove rl-next-dev
We realized that there's really no good place to put these dev facing
bulletins, and the user-facing release notes aren't really the worst
place to put them, I guess, and we do kind of hope that it converts
users to devs.

Change-Id: Id9387b2964fe291cb5a3f74ad6344157f19b540c
2024-06-06 15:08:12 -07:00
raito b8cb7abcf0 chore: rebrand Nix to Lix when it makes sense
Here's my guide so far:

$ rg '((?!(recursive).*) Nix
(?!(daemon|store|expression|Rocks!|Packages|language|derivation|archive|account|user|sandbox|flake).*))'
-g '!doc/' --pcre2

All items from this query have been tackled. For the documentation side:
that's for #162.

Additionally, all remaining references to github.com/NixOS/nix which
were not relevant were also replaced.

Fixes: #148.
Fixes: #162.
Change-Id: Ib3451fae5cb8ab8cd9ac9e4e4551284ee6794545
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-06-01 20:31:24 +02:00
terru - 0c6cb34de6 Merge "document context-dependent keywords" into main 2024-05-31 10:07:50 +00:00
Maximilian Bosch 3d78b4847e Merge "release-notes: add missing credits/category to consistent-nix-build entry" into main 2024-05-30 14:38:47 +00:00
Maximilian Bosch 6abac7aacc release-notes: add missing credits/category to consistent-nix-build entry
Change-Id: I737422a2ff9d66be30cc432f8c1ddba9b1e71f4f
2024-05-30 14:05:12 +00:00
Cole Helbling 031d924116 libutil/args: warn on unknown settings after parsing all flags
Upstream change: https://github.com/NixOS/nix/pull/10701

Change-Id: Icf271df57ec529dd8c64667d1ef9f6dbf02d33d3
2024-05-30 03:07:21 +00:00
Qyriad afeaa2371c docs: enable non-default TOC folding
Folding by default would prevent things like "Ctrl+F for nix-env" from
working trivially, but the user should be able to fold if they want to.

Change-Id: I5273272289f0f24e1f040c691580acfe33f66bd4
2024-05-30 00:23:01 +00:00
terru - d8e452a91b document context-dependent keywords
Documents some of the weirdness of __curPos and the or keyword.
This does not fit well into any existing section for either of
them, though the use of or as a quasi-operator is mentioned in
the section on operators.

Addresses #353

Change-Id: I7c906c8368843dca6944e8b22573b6d201cd9a76
2024-05-28 23:44:55 +02:00
alois31 f047e4357b libstore/build: always enable seccomp filtering and no-new-privileges
Seccomp filtering and the no-new-privileges functionality improve the security
of the sandbox, and have been enabled by default for a long time. In
#265 it was decided that they
should be enabled unconditionally. Accordingly, remove the allow-new-privileges
(which had weird behavior anyway) and filter-syscall settings, and force the
security features on. Syscall filtering can still be enabled at build time to
support building on architectures libseccomp doesn't support.

Change-Id: Iedbfa18d720ae557dee07a24f69b2520f30119cb
2024-05-24 21:19:29 +00:00
jade 19ea351642 Merge "clang-tidy: work with angle brackets and external projects" into main 2024-05-24 19:11:26 +00:00
Qyriad 65da3e7199 make CTRL+Z work in the REPL
Editline just wasn't being built with --enable-sigstop lol

Change-Id: I35a78f74ea100d97f26b2b41990deb373fd9cd9a
2024-05-24 03:10:12 +00:00
jade 745b5d3d4f clang-tidy: work with angle brackets and external projects
Also fix the readme

Change-Id: I422dff5536bf01d43983621aa01035bd77ac0252
2024-05-24 02:22:58 +00:00
Qyriad 00bf2b105d Merge changes I462a8cf0,I3b0bcea3,I2acd56e7,Ifc149764,I9e2ef170 into main
* changes:
  docs: linkify nix3-build mention in nix-build.md
  build: make internal-api-docs PHONY
  cleanup lookupFileArg
  add docstring to lookupFileArg
  add libcmd test for lookupFileArg
2024-05-23 23:28:42 +00:00
jade 9530b7f2b2 Merge "packaging: rename nixexpr -> lixexpr and so on" into main 2024-05-23 23:19:28 +00:00
jade c97e17144e packaging: rename nixexpr -> lixexpr and so on
This breaks downstreams linking to us on purpose to make sure that if
someone is linking to Lix they're doing it on purpose and crucially not
mixing up Nix and Lix versions in compatibility code.

We still need to fix the internal includes to follow the same schema so
we can drop the single-level include system entirely. However, this
requires a little more effort.

This adds pkg-config for libfetchers and config.h.

Migration path:
expr.hh      -> lix/libexpr/expr.hh
nix/config.h -> lix/config.h

To apply this migration automatically, remove all `<nix/>` from
includes, so: `#include <nix/expr.hh>` -> `#include <expr.hh>`. Then,
the correct paths will be resolved from the tangled mess, and the
clang-tidy automated fix will work.

Then run the following for out of tree projects:

```
lix_root=$HOME/lix
(cd $lix_root/clang-tidy && nix develop -c 'meson setup build && ninja -C build')
run-clang-tidy -checks='-*,lix-fixincludes' -load=$lix_root/clang-tidy/build/liblix-clang-tidy.so -p build/ -fix src
```

Related: lix-project/nix-eval-jobs#5
Fixes: #279
Change-Id: I7498e903afa6850a731ef8ce77a70da6b2b46966
2024-05-23 16:45:23 -06:00
Qyriad a0172dc81b docs: linkify nix3-build mention in nix-build.md
Change-Id: I462a8cf0da42b5045ce84b48dc1841ecdccbb89e
2024-05-23 14:11:10 -06:00
Qyriad a8b2fc6d41 build: make internal-api-docs PHONY
Since we're skipping Meson's dependency tracking, for the
internal-api-docs custom target, we should just consider it a phony
target and build it on every request.

Change-Id: I3b0bcea30ee9a4830023ccc5bededf995e96cccc
2024-05-23 14:11:10 -06:00
eldritch horrors 9a75150d19 doc: sort change-authors.yml
this should make it easier to spot future instances of entries being
duplicated by accident. also add a pre-commit check to remain sorted

Change-Id: I500caf862e93480b38c9d51144273bb2dcab1af0
2024-05-23 02:09:05 +00:00
jade dcc7ea5498 release notes: add a bunch of them
Also fix typos introduced by the commits I read.

I have run the addDrvOutputDependencies release note past Ericson since
I was confused by what the heck it was doing, and he was saying it was
reasonable.

Change-Id: Id015353b00938682f7faae7de43df7f991a5237e
2024-05-22 21:13:56 +02:00
alois31 0bf4c2971f Merge "doc: fix repl-interrupt release note entry" into main 2024-05-22 18:59:54 +00:00
alois31 d5fdb995d3
doc: fix repl-interrupt release note entry
The timing of the merge resulted in the newly introduced metadata not being
present.

Change-Id: I07f28cf37703ec05c3e1b96301797a42d913264b
2024-05-21 16:34:04 +02:00
Qyriad 20981461d4 print type and value in "flake attr is not a derivation" errors
This turns errors like:

error: flake output attribute 'hydraJobs' is not a derivation or path

into errors like:

error: expected flake output attribute 'hydraJobs' to be a derivation or
path but found a set: { binaryTarball = «thunk»; build = «thunk»; etc> }

This change affects all InstallableFlake commands.

Change-Id: I899757af418b6f98201006ec6ee13a448c07077c
2024-05-21 05:55:13 -06:00
puck bfb91db4f6 repl-interacter: save history after entering every line
Fixes: #328
Change-Id: Iedd79ff5f72e84766ebd234c63856170afc624f0
2024-05-19 22:47:45 +00:00
puck 40311973a8 change-authors: add puck
Change-Id: I04b8cd04a168b3adea7790f816e774d5d90fcea2
2024-05-19 22:47:45 +00:00
julia 7a3745b076
Deprecate the online flake registries and vendor the default registry
Fixes #183, #110, #116.

The default flake-registry option becomes 'vendored', and refers
to a vendored flake-registry.json file in the install path.

Vendored copy of the flake-registry is from github:NixOS/flake-registry
at commit 9c69f7bd2363e71fe5cd7f608113290c7614dcdd.

Change-Id: I752b81c85ebeaab4e582ac01c239d69d65580f37
2024-05-18 12:27:23 +10:00
julia 5b7dcb3005 Allow enabling core dumps from builds for nix & child processes
Fixes #268

Change-Id: I3f1b0ddf064f891cca8b53229c5c31c74cea3d9f
2024-05-16 17:11:21 -07:00
jade 6d84d6bdc1 builtins: fix builtins.langVersion docs to state it's deprecated
This builtin was always a problem and nixpkgs uses it in exactly one
place, to give up if the Nix version is absurdly old. It has no other
use cases, and doesn't work in a multi-implementation world anyway.

Change-Id: I03c36e118591029e2ef14b091fe14a311c66a08a
2024-05-15 21:54:12 -07:00
jade 2a7b3d7c94 build-release-notes: add change author metadata and use it
Change-Id: I6f5fb54f70b02a467bbdee4c526f59da1193f7db
2024-05-15 14:33:35 -07:00
jade 272db8af1b doc: add release note credits and categories for all the changes in Lix
This does not add missing release notes, and it doesn't do anything
about the profiles feature we would really like to have so we can have
consistent credit.

Change-Id: I72a6f7acfcff85f380be17dac76501a6f4693776
2024-05-15 14:33:35 -07:00
jade 03655c310d build-release-notes: fail if the directory does not exist
This was a combination of two problems: the python didn't throw an error
because apparently glob on a nonexistent directory doesn't crash, and
secondarily, bash ignores bad exit codes without `set -e` if they are
not in the final/only command.

Change-Id: I812bde7a4daee5c77ffe9d7c73a25fd14969f548
2024-05-15 20:18:50 +00:00
alois31 f6397cc286 doc: fix rl-next build
Use the correct directory for the rl-next build, so that the release notes
actually get built and the page doesn't end up empty. I don't know why the
exception didn't cause a build failure before.

Fixes: #297
Change-Id: Ic72b9bb4c0d2d1f633f2af90cce4a3a2796d7f9b
2024-05-15 20:18:50 +00:00
Maximilian Bosch aabc902040 Merge "nix3-build: show all FOD errors with --keep-going" into main 2024-05-15 18:46:06 +00:00
alois31 0903a99bad Merge changes I8456c47b,I48253f5f into main
* changes:
  repl: clear the interrupt before reading the next line
  libutil: remove the interrupt-blocking code
2024-05-15 15:46:11 +00:00
Maximilian Bosch e146393183
nix3-build: show all FOD errors with --keep-going
Basically I'd expect the same behavior as with `nix-build`, i.e.
with `--keep-going` the hash-mismatch error of each failing
fixed-output derivation is shown.

The approach is derived from `Store::buildPaths` (`entry-point.cc`):
instead of throwing the first build-result, check if there are any build
errors and if so, display all of them and throw after that.

Unfortunately, the BuildResult struct doesn't have an `ErrorInfo`
(there's a FIXME for that at least), so I have to construct my own here.
This is a rather cheap bugfix and I decided against touching too many
parts of libstore for that (also I don't know if that's in line with the
ongoing refactoring work).

Closes #302

Change-Id: I378ab984fa271e6808c6897c45e0f070eb4c6fac
2024-05-15 15:35:18 +02:00
jade 58ff8960cd doc: add a script to upload the nightly manual manually
This is not like, perfect, since it is a manual operation, but we can
automate it in the future. rclone is used, since it seems like awscli is
not (obviously at least?) able to sync directories such that old things
are deleted, and rclone does this thing properly.

Fixes: lix-project/meta#2
Change-Id: Ia6a46d861342a6d29b22f981ba4e35e79f79e60e
2024-05-14 14:28:01 -07:00
alois31 914b0febf7
libutil: remove the interrupt-blocking code
The interrupt-blocking code was originally introduced 20 years ago so that
trying to log an error message does not result in an interrupt exception being
thrown and then going unhandled (c8d3882cdc).
However, the logging code does not check for interrupts any more
(054be50257), so this reasoning is no longer
applicable. Delete this code so that later interrupts are unblocked again, for
example in the next line entered into the repl.

Closes: #296
Change-Id: I48253f5f4272e75001148c13046e709ef5427fbd
2024-05-13 09:03:53 +02:00
Qyriad b9be46fb31 remove the autoconf+Make buildsystem
We're not using it anymore. Any leftover bugs in the Meson buildsystem
are now just bugs.

Closes #249.

Change-Id: I0465a0c37ae819f94d40e7829f5bff046aa63d73
2024-05-07 17:04:30 -06:00
jade 748d8310fa Fix the pages in the manual for Lix
This doesn't comprehensively fix everything outdated in the manual, or
make the manual greatly better, but it does note down where at least
jade noticed it was wrong, and it does fix all the instances of
referencing Nix to conform to the style guide to the best of our
ability.

A lot of things have been commented out for being wrong, and there are
three types of FIXME introduced:

- FIXME(Lix): generically Lix needs to fix it
- FIXME(Qyriad): re #215
- FIXME(meson): docs got outdated by meson changes and need rewriting

I did fix a bunch of it that I could, but there could certainly be
mistakes and this is definitely just an incremental improvement.

Fixes: #266
Change-Id: I5993c4603d7f026a887089fce77db08394362135
2024-05-05 16:11:01 -07:00
eldritch horrors 6a5f100b8b doc: fix build littering doc/
mdbook has the unfortunate habit of creating stub files for chapters it
can't find on disk. turn off this helpful feature as it masks errors in
the summary file, and fix a recently introduced instance of this error.

Change-Id: I10d86aac0489c9c494bd5c8a50047415f4d4b18d
2024-05-05 16:13:27 +00:00
Maximilian Bosch 799d0132f0 Merge "Revert "Revert "Merge pull request #6621 from Kha/nested-follows""" into main 2024-05-04 08:52:29 +00:00
Maximilian Bosch 79d0ae6670 Merge "libstore/local-derivation-goal: prohibit creating setuid/setgid binaries" into main 2024-05-04 07:26:15 +00:00
Maximilian Bosch 0e38720502 Revert "Revert "Merge pull request #6621 from Kha/nested-follows""
This reverts commit a8b3d777fb.

This undoes the revert of PR#6621, which allows nested `follows`, i.e.

    {
      inputs = {
        foo.url = "github:bar/foo";
        foo.inputs.bar.inputs.nixpkgs = "nixpkgs";
      };
    }

does the expected thing now. This is useful to avoid the 1000 instances
of nixpkgs problem without having each flake in the dependency tree to
expose all of its transitive dependencies for modification.

This was in fact part of Nix before and the C++ changes applied w/o
conflicts. However, it got reverted then because people didn't want to
merge lazy-trees against it which was supposed to be merged soon back in
October 2022.

Fixes: #201

Change-Id: I5ddef914135b695717b2ef88862d57ced5e7aa3c
2024-05-03 22:54:38 +00:00
Maximilian Bosch 045ee37438 libstore/local-derivation-goal: prohibit creating setuid/setgid binaries
With Linux kernel >=6.6 & glibc 2.39 a `fchmodat2(2)` is available that
isn't filtered away by the libseccomp sandbox.

Being able to use this to bypass that restriction has surprising results
for some builds such as lxc[1]:

> With kernel ≥6.6 and glibc 2.39, lxc's install phase uses fchmodat2,
> which slips through 9b88e52846/src/libstore/build/local-derivation-goal.cc (L1650-L1663).
> The fixupPhase then uses fchmodat, which fails.
> With older kernel or glibc, setting the suid bit fails in the
> install phase, which is not treated as fatal, and then the
> fixup phase does not try to set it again.

Please note that there are still ways to bypass this sandbox[2] and this is
mostly a fix for the breaking builds.

This change works by creating a syscall filter for the `fchmodat2`
syscall (number 452 on most systems). The problem is that glibc 2.39
is needed to have the correct syscall number available via
`__NR_fchmodat2` / `__SNR_fchmodat2`, but this flake is still on
nixpkgs 23.11. To have this change everywhere and not dependent on the
glibc this package is built against, I added a header
"fchmodat2-compat.hh" that sets the syscall number based on the
architecture. On most platforms its 452 according to glibc with a few
exceptions:

    $ rg --pcre2 'define __NR_fchmodat2 (?!452)'
    sysdeps/unix/sysv/linux/x86_64/x32/arch-syscall.h
    58:#define __NR_fchmodat2 1073742276

    sysdeps/unix/sysv/linux/mips/mips64/n32/arch-syscall.h
    67:#define __NR_fchmodat2 6452

    sysdeps/unix/sysv/linux/mips/mips64/n64/arch-syscall.h
    62:#define __NR_fchmodat2 5452

    sysdeps/unix/sysv/linux/mips/mips32/arch-syscall.h
    70:#define __NR_fchmodat2 4452

    sysdeps/unix/sysv/linux/alpha/arch-syscall.h
    59:#define __NR_fchmodat2 562

I added a small regression-test to the setuid integration-test that
attempts to set the suid bit on a file using the fchmodat2 syscall.
I confirmed that the test fails without the change in
local-derivation-goal.

Additionally, we require libseccomp 2.5.5 or greater now: as it turns
out, libseccomp maintains an internal syscall table and
validates each rule against it. This means that when using libseccomp
2.5.4 or older, one may pass `452` as syscall number against it, but
since it doesn't exist in the internal structure, `libseccomp` will refuse
to create a filter for that. This happens with nixpkgs-23.11, i.e. on
stable NixOS and when building Lix against the project's flake.

To work around that

* a backport of libseccomp 2.5.5 on upstream nixpkgs has been
  scheduled[3].

* the package now uses libseccomp 2.5.5 on its own already. This is to
  provide a quick fix since the correct fix for 23.11 is still a staging cycle
  away.

We still need the compat header though since `SCMP_SYS(fchmodat2)`
internally transforms this into `__SNR_fchmodat2` which points to
`__NR_fchmodat2` from glibc 2.39, so it wouldn't build on glibc 2.38.
The updated syscall table from libseccomp 2.5.5 is NOT used for that
step, but used later, so we need both, our compat header and their
syscall table 🤷

Relevant PRs in CppNix:

* https://github.com/NixOS/nix/pull/10591
* https://github.com/NixOS/nix/pull/10501

[1] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2031073804
[2] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2030844251
[3] https://github.com/NixOS/nixpkgs/pull/306070

(cherry picked from commit ba6804518772e6afb403dd55478365d4b863c854)
Change-Id: I6921ab5a363188c6bff617750d00bb517276b7fe
2024-05-03 16:29:06 +02:00
Théophane Hufschmitt 8458d98b27 Rename nix show-config to nix config show
Part of #7672

My main motivation is to be able to use `nix.checkConfig`[1]. This
doesn't work with Lix currently since the module uses `nix show-config`
if the Nix version is <2.20pre and `nix config show` otherwise. I think
this is the only instance where nixpkgs checks for which Nix commands
exist that affects us now, so I figured we could just perform the rename
here as well[2] and still provide the current version number[3].

I don't have a strong opinion on whether to deprecate `nix show-config`,
the warning is added there automatically.

(cherry picked from commit f300e11b056dea414d7d77bbc6e5a7dc5d9ddd41)

[1] https://nixos.org/manual/nixos/stable/options.html#opt-nix.checkConfig
[2] I should add that I don't use the "official" ways of installing Lix
    because using the flake directly and callPackaging it seemed to fit
    better into my workflow: I already have a little mess to make
    sure Hydra from the flake uses the correct pkgs.nix and I didn't
    want to complicate it further while keeping a single package-set I
    can build in CI. Don't get me wrong, I think such a module for a
    quick-start is very important, just giving context on why I bother
    in the first place :)
[3] When we go public, I think it's worth considering to add support in
    nixpkgs itself for Lix.

Change-Id: I47b4239b05cbeda3c370d2fa56ea768b768768ac
2024-05-03 16:26:16 +02:00
Qyriad 076dfd30c6 Merge changes from topic "profile-v3" into main
* changes:
  nix3-profile: remove check "name" attr in manifests
  Add profile migration test
  nix3-profile: make element names stable
  getNameFromURL(): Support uppercase characters in attribute names
  nix3-profile: remove indices
  nix3-profile: allow using human-readable names to select packages
  implement parsing human-readable names from URLs
2024-05-02 20:15:48 +00:00