Commit graph

823 commits

Author SHA1 Message Date
eldritch horrors 4d8c66ec6f add aarch64-linux as a cross-build target
we want to be sure we can cross-build to aarch64 for releases, add a
target to our crossSystems list to make those cheacks easier to run.

Change-Id: Ieb65c1333a5232641ace0ba4d122fc7d528ebc04
2024-07-09 22:49:10 +02:00
jade d85309f7ca store: delete obsolete lsof-disabling code
Since Ifa0adda7984e, we don't use this code anymore on macOS, so we have
no reason to have a knob to disable it anymore.

Change-Id: Ie29a8a8978d9aefd4551895f4f9b3cc0827496df
2024-06-27 22:53:36 -07:00
jade 85c1241201 doc/hacking: fix up some outdated info about cross, hydra links
We would like to build these with Hydra but we do not currently have a
Hydra to build them with conveniently.

Change-Id: I0832a33881138dd1caab3805df7ad097db347e62
2024-06-25 21:46:26 -07:00
jade 33d53c4983 doc/hacking: fix internal api docs section to say to enable it
I filed a bug to build these in releng in the future:
lix-project/lix#422

Change-Id: I476a2516cc2be382d4b7c8529a02f9212a78fdb2
2024-06-25 21:26:18 -07:00
jade e537678f1e doc: Add more about the release note generator
Change-Id: I8a274227cb1b05d442d3f644603dd2844ecc9d05
2024-06-25 21:22:37 -07:00
Delan Azabani 4c3d93611f Merge "Fix build instructions in hacking.md and justfile" into main 2024-06-24 06:46:08 +00:00
Delan Azabani 5d33e4cd59 Fix build instructions in hacking.md and justfile
The stdenv phases don’t actually do anything (at least not anymore),
and our justfile doesn’t behave the same as our docs.

This patch removes the stdenv phases from the docs, documents our
usage of just, and makes `just setup` heed `$mesonFlags`.

Fixes #413.
Fixes #414.

Change-Id: Ieb0b2a8ae420526238b5f9a73d7849ec6919995d
2024-06-24 12:55:00 +08:00
kloenk da4e46dd1f libmain: add progress bar with multiple status lines
Add the log-formats `multiline` and `multiline-with-logs` which offer
multiple current active building status lines.

Change-Id: Idd8afe62f8591b5d8b70e258c5cefa09be4cab03
2024-06-22 14:20:27 +02:00
jade f95a47e8c4 release: release notes for 2.90.0
For now we just need to put the release notes in the final spot. We will
have to fix the date on both 2.90 and 2.91 branches, but such as it is.

Release created with releng/create_release.xsh

Closes: lix-project/lix#318
Change-Id: I38e79b40e7f632c8a286f2f09865a84dc93eca90
2024-06-15 18:46:18 -07:00
jade 59b5965bbf doc: Write an index of environment variables used in testsuite
This was originally going to be just the testsuite but I kinda just
documented all of them.

I am tired of us not documenting these. This is a starting point to
producing an actually good index. I would like to enforce it in a
pre-commit hook eventually that we document all environment variables
used in Lix itself, even if it is terse dev facing docs.

This is full of a bunch of TODOs caused by auditing code. They should
probably be done at some point.

Change-Id: I7c0d3b257e19bae23d47d1efbd7361d203bccb0e
2024-06-12 15:34:23 -07:00
jade 211f79d4a2 doc/testing.md: Rewrite some outdated sections for meson and current source layout
Change-Id: Ia23f82c9a564b55bd799afbda59c28c9b0a65c13
2024-06-12 15:34:23 -07:00
jade 5f6eb6eb44 doc: rewrite the multi-user documentation to actually talk about security
It's in the security section, and it was totally outdated anyway.

I took the opportunity to write down the stuff we already believed.

Change-Id: I73e62ae85a82dad13ef846e31f377c3efce13cb0
2024-06-12 15:34:23 -07:00
jade 4392d89eea manual: rewrite the docker guide now that we have images
Change-Id: I5bdf47e67059ae4099552750a47ae070dbe094df
2024-06-09 20:33:24 -07:00
raito b8cb7abcf0 chore: rebrand Nix to Lix when it makes sense
Here's my guide so far:

$ rg '((?!(recursive).*) Nix
(?!(daemon|store|expression|Rocks!|Packages|language|derivation|archive|account|user|sandbox|flake).*))'
-g '!doc/' --pcre2

All items from this query have been tackled. For the documentation side:
that's for lix-project/lix#162.

Additionally, all remaining references to github.com/NixOS/nix which
were not relevant were also replaced.

Fixes: lix-project/lix#148.
Fixes: lix-project/lix#162.
Change-Id: Ib3451fae5cb8ab8cd9ac9e4e4551284ee6794545
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-06-01 20:31:24 +02:00
terru - d8e452a91b document context-dependent keywords
Documents some of the weirdness of __curPos and the or keyword.
This does not fit well into any existing section for either of
them, though the use of or as a quasi-operator is mentioned in
the section on operators.

Addresses lix-project/lix#353

Change-Id: I7c906c8368843dca6944e8b22573b6d201cd9a76
2024-05-28 23:44:55 +02:00
alois31 f047e4357b libstore/build: always enable seccomp filtering and no-new-privileges
Seccomp filtering and the no-new-privileges functionality improve the security
of the sandbox, and have been enabled by default for a long time. In
lix-project/lix#265 it was decided that they
should be enabled unconditionally. Accordingly, remove the allow-new-privileges
(which had weird behavior anyway) and filter-syscall settings, and force the
security features on. Syscall filtering can still be enabled at build time to
support building on architectures libseccomp doesn't support.

Change-Id: Iedbfa18d720ae557dee07a24f69b2520f30119cb
2024-05-24 21:19:29 +00:00
Qyriad 00bf2b105d Merge changes I462a8cf0,I3b0bcea3,I2acd56e7,Ifc149764,I9e2ef170 into main
* changes:
  docs: linkify nix3-build mention in nix-build.md
  build: make internal-api-docs PHONY
  cleanup lookupFileArg
  add docstring to lookupFileArg
  add libcmd test for lookupFileArg
2024-05-23 23:28:42 +00:00
jade 9530b7f2b2 Merge "packaging: rename nixexpr -> lixexpr and so on" into main 2024-05-23 23:19:28 +00:00
jade c97e17144e packaging: rename nixexpr -> lixexpr and so on
This breaks downstreams linking to us on purpose to make sure that if
someone is linking to Lix they're doing it on purpose and crucially not
mixing up Nix and Lix versions in compatibility code.

We still need to fix the internal includes to follow the same schema so
we can drop the single-level include system entirely. However, this
requires a little more effort.

This adds pkg-config for libfetchers and config.h.

Migration path:
expr.hh      -> lix/libexpr/expr.hh
nix/config.h -> lix/config.h

To apply this migration automatically, remove all `<nix/>` from
includes, so: `#include <nix/expr.hh>` -> `#include <expr.hh>`. Then,
the correct paths will be resolved from the tangled mess, and the
clang-tidy automated fix will work.

Then run the following for out of tree projects:

```
lix_root=$HOME/lix
(cd $lix_root/clang-tidy && nix develop -c 'meson setup build && ninja -C build')
run-clang-tidy -checks='-*,lix-fixincludes' -load=$lix_root/clang-tidy/build/liblix-clang-tidy.so -p build/ -fix src
```

Related: lix-project/nix-eval-jobs#5
Fixes: lix-project/lix#279
Change-Id: I7498e903afa6850a731ef8ce77a70da6b2b46966
2024-05-23 16:45:23 -06:00
Qyriad a0172dc81b docs: linkify nix3-build mention in nix-build.md
Change-Id: I462a8cf0da42b5045ce84b48dc1841ecdccbb89e
2024-05-23 14:11:10 -06:00
jade dcc7ea5498 release notes: add a bunch of them
Also fix typos introduced by the commits I read.

I have run the addDrvOutputDependencies release note past Ericson since
I was confused by what the heck it was doing, and he was saying it was
reasonable.

Change-Id: Id015353b00938682f7faae7de43df7f991a5237e
2024-05-22 21:13:56 +02:00
jade 2a7b3d7c94 build-release-notes: add change author metadata and use it
Change-Id: I6f5fb54f70b02a467bbdee4c526f59da1193f7db
2024-05-15 14:33:35 -07:00
jade 03655c310d build-release-notes: fail if the directory does not exist
This was a combination of two problems: the python didn't throw an error
because apparently glob on a nonexistent directory doesn't crash, and
secondarily, bash ignores bad exit codes without `set -e` if they are
not in the final/only command.

Change-Id: I812bde7a4daee5c77ffe9d7c73a25fd14969f548
2024-05-15 20:18:50 +00:00
alois31 f6397cc286 doc: fix rl-next build
Use the correct directory for the rl-next build, so that the release notes
actually get built and the page doesn't end up empty. I don't know why the
exception didn't cause a build failure before.

Fixes: lix-project/lix#297
Change-Id: Ic72b9bb4c0d2d1f633f2af90cce4a3a2796d7f9b
2024-05-15 20:18:50 +00:00
jade 748d8310fa Fix the pages in the manual for Lix
This doesn't comprehensively fix everything outdated in the manual, or
make the manual greatly better, but it does note down where at least
jade noticed it was wrong, and it does fix all the instances of
referencing Nix to conform to the style guide to the best of our
ability.

A lot of things have been commented out for being wrong, and there are
three types of FIXME introduced:

- FIXME(Lix): generically Lix needs to fix it
- FIXME(Qyriad): re lix-project/lix#215
- FIXME(meson): docs got outdated by meson changes and need rewriting

I did fix a bunch of it that I could, but there could certainly be
mistakes and this is definitely just an incremental improvement.

Fixes: lix-project/lix#266
Change-Id: I5993c4603d7f026a887089fce77db08394362135
2024-05-05 16:11:01 -07:00
eldritch horrors 6a5f100b8b doc: fix build littering doc/
mdbook has the unfortunate habit of creating stub files for chapters it
can't find on disk. turn off this helpful feature as it masks errors in
the summary file, and fix a recently introduced instance of this error.

Change-Id: I10d86aac0489c9c494bd5c8a50047415f4d4b18d
2024-05-05 16:13:27 +00:00
Théophane Hufschmitt 8458d98b27 Rename nix show-config to nix config show
Part of #7672

My main motivation is to be able to use `nix.checkConfig`[1]. This
doesn't work with Lix currently since the module uses `nix show-config`
if the Nix version is <2.20pre and `nix config show` otherwise. I think
this is the only instance where nixpkgs checks for which Nix commands
exist that affects us now, so I figured we could just perform the rename
here as well[2] and still provide the current version number[3].

I don't have a strong opinion on whether to deprecate `nix show-config`,
the warning is added there automatically.

(cherry picked from commit f300e11b056dea414d7d77bbc6e5a7dc5d9ddd41)

[1] https://nixos.org/manual/nixos/stable/options.html#opt-nix.checkConfig
[2] I should add that I don't use the "official" ways of installing Lix
    because using the flake directly and callPackaging it seemed to fit
    better into my workflow: I already have a little mess to make
    sure Hydra from the flake uses the correct pkgs.nix and I didn't
    want to complicate it further while keeping a single package-set I
    can build in CI. Don't get me wrong, I think such a module for a
    quick-start is very important, just giving context on why I bother
    in the first place :)
[3] When we go public, I think it's worth considering to add support in
    nixpkgs itself for Lix.

Change-Id: I47b4239b05cbeda3c370d2fa56ea768b768768ac
2024-05-03 16:26:16 +02:00
Qyriad 78ce710722 docs(nix-env): summarize of each subcommand in --help
This should have been there from the beginning. As much as nix-env is a
pile of problems we don't need trivial docs papercuts like this adding
to it.

Change-Id: I0c53e4b146af2fefdd0e4743d850672729cb2194
2024-04-26 21:56:08 -06:00
Qyriad b913a939b0 meson: flip the switch!!
This commit makes Meson the default buildsystem for Lix.
The Make buildsystem is now deprecated and will be removed soon, but has
not yet, which will be done in a later commit when all seems good. The
mesonBuild jobs have been removed, and have not been replaced with
equivalent jobs to ensure the Make buildsystem still works.

The full, new commands in a development shell are:

$ meson setup ./build "--prefix=$out" $mesonFlags

(A simple `meson setup ./build` will also build, but will do a different
thing, not having the settings from package.nix applied.)

$ meson compile -C build
$ meson test -C build --suite=check
$ meson install -C build
$ meson test -C build --suite=installcheck

(Check and installcheck may both be done after install, allowing you to
omit the --suite argument entirely, but this is the order package.nix
runs them in.)

If tests fail and Meson helpfully has no output for why, use the
`--print-error-logs` option to `meson test`. Why this is not the default
I cannot explain.

If you change a setting in the buildsystem, most cases will
automatically regenerate the Meson configuration, but some cases, like
trying to build a specific target whose name is new to the buildsystem
(e.g. `meson compile -C build src/libmelt/libmelt.dylib`, when
`libmelt.dylib` did not exist as a target the last time the buildsystem
was generated), then you can reconfigure using new settings but
existing options, and only recompiling stuff affected by the changes:

$ meson setup --reconfigure build

Note that changes to the default values in `meson.options` or in the
`default_options :` argument to project() are NOT propagated with
`--reconfigure`.

If you want a totally clean build, you can use:

$ meson setup --wipe build

That will work regardless of if `./build` exists or not.

Specific, named targets may be addressed in
`meson build -C build <target>` with the "target ID" if there is one,
which is the first string argument passed to target functions that
have one, and unrelated to the variable name, e.g.:

libexpr_dylib = library('nixexpr', …)

can be addressed with:

$ meson compile -C build nixexpr

All targets may be addressed as their output, relative to the build
directory, e.g.:

$ meson compile -C build src/libexpr/libnixexpr.so

But Meson does not consider intermediate files like object files
targets. To build a specific object file, use Ninja directly and
specify the output file relative to the build directory:

$ ninja -C build src/libexpr/libnixexpr.so.p/nixexpr.cc.o

To inspect the canonical source of truth on what the state of the
buildsystem configuration is, use:

$ meson introspect

Have fun!

Change-Id: Ia3e7b1e6fae26daf3162e655b4ded611a5cd57ad
2024-04-22 21:41:58 -06:00
eldritch horrors adab839c98 manual: build docs with dummy envs
this was previously used because the macOS docs build would otherwise
pull files out of the host nix store. or something. not sure about it

Change-Id: I76b51eac1ebc5de5f00e2e4be086dd8db3eeb8e6
2024-04-11 13:36:04 +00:00
eldritch horrors 725f5cd358 docs: redo content generation for mdbook and manual
manpages can be rendered using the markdown output of mdbook, the rest
of the manual can generated out of the main doc/manual source tree. we
still use lowdown to actually render manpages instead of eg mdbook-man
because lowdown does generate reasonably good manpages (though that is
also somewhat debatable, but they're a lot better than mdbook-man).

doing this not only lets us drastically simplify the lowdown pipeline,
but also remove all custom {{#include}} handling since now mdbook does
all of it, even for the manpage builds. even the lowdown wrapper isn't
entirely necessary because lowdown can take all wrapper arguments with
command line flags rather than bits of input file content.

This also implements running mdbook in Meson, in order to generate the
manpages. The mdbook outputs are also installed in the usual location.

Co-authored-by: Qyriad <qyriad@qyriad.me>

Change-Id: I60193f9fd0f15d48872f071af35855cda2a0f40b
2024-04-11 13:32:06 +00:00
eldritch horrors c1ae35d34a docs: don't compute SUMMARY.md during build
this would make meson build compatibility unnecessarily hard and
the cli does not change often enough to justify this complexity.

Change-Id: I17b1870cdf8538feeaa01a9945db97af2175a642
2024-04-09 02:09:36 +00:00
eldritch horrors d9bc197ff4 docs: don't compute rl-next.md during build
not sure why this was done the way it was considering that includes are
a feature the doc toolchain had previously. let's just always have some
kind of entry for the upcoming release in the dev manual builds even if
that means having a completely empty release notes chapter.

the release notes generation script isn't entirely functional right now
due to pre-commit hooks, but it's good enough for time being. we need a
better release process for notes anyway.

Change-Id: Ifda6912cf5233db013f72a30247a62d6f22b1565

Change-Id: I9eb347ec4aabc5be2b816ff0fd3e4be45f93b934
2024-04-09 02:09:36 +00:00
eldritch horrors 14088e1ec5 docs: remove many link-to-self
for some reason these three were anchors, not links, but had they been
links they wouldn't've worked because they're not defined anywhere but
here. in the print version of the manual they're duplicated many times
over (creating id collisions), so we should better remove them anyway.

Change-Id: I8988a7c32c812dee0f0b6d4953faa7cd1255228d
2024-04-09 02:09:36 +00:00
Qyriad 9166babbaf Revert "meson: move nix3 manpage generation into command-ref/"
This reverts commit 7095423374.

This seems to have broken running ninja on warm build directories, which
is not what we want. Reverted until we figure out something better

Change-Id: I9623ae078917e7c59a930bf8044a216501d4bb20
2024-04-04 11:59:08 -06:00
Qyriad c96c5db030 meson: move nix3 manpage generation into command-ref/
This puts the generated files where they are for the make system.
This is in preparation for further meson-mdbook stuff.

Change-Id: I934df6854a80af5ccf381cf1da0bda0187a8bcfc
2024-04-03 13:52:56 -06:00
Rebecca Turner 2a98ba8b97 Add pre-commit checks
The big ones here are `trim-trailing-whitespace` and `end-of-file-fixer`
(which makes sure that every file ends with exactly one newline
character).

Change-Id: Idca73b640883188f068f9903e013cf0d82aa1123
2024-03-29 22:57:40 -07:00
eldritch horrors 279e30e7ef build: replace changelog-d with local script
hacking changelog-d to support not just github but also forgejo and
gerrit is a lot more complicated than it's worth, even moreso since
the entire thing can just as well be done with ~60 lines of python.
this new script is also much cheaper to instantiate (being python),
so having it enabled in all shells is far less of a hassle.

we've also adjusted existing release notes that referenced a gerrit
cl to auto-link to the cl in question, making the diff a bit bigger

closes lix-project/lix#176

Change-Id: I8ba7dd0070aad9ba4474401731215fcf5d9d2130
2024-03-27 03:09:14 +00:00
eldritch horrors 7e1f8b09a4 Merge pull request #9755 from 9999years/printer-followup
Printer followup

(cherry picked from commit 51f524c629b778b75cb62a9e0c85bae655984abc)
Change-Id: I29214cc86c5e846cbcfec382022293a70011b316
2024-03-09 04:04:46 +01:00
eldritch horrors 512c1f05c3 Unify and refactor value printing
Previously, there were two mostly-identical value printers -- one in
`libexpr/eval.cc` (which didn't force values) and one in
`libcmd/repl.cc` (which did force values and also printed ANSI color
codes).

This PR unifies both of these printers into `print.cc` and provides a
`PrintOptions` struct for controlling the output, which allows for
toggling whether values are forced, whether repeated values are tracked,
and whether ANSI color codes are displayed.

Additionally, `PrintOptions` allows tuning the maximum number of
attributes, list items, and bytes in a string that will be displayed;
this makes it ideal for contexts where printing too much output (e.g.
all of Nixpkgs) is distracting. (As requested by @roberth in
https://github.com/NixOS/nix/pull/9554#issuecomment-1845095735)

Please read the tests for example output.

Future work:
- It would be nice to provide this function as a builtin, perhaps
  `builtins.toStringDebug` -- a printing function that never fails would
  be useful when debugging Nix code.
- It would be nice to support customizing `PrintOptions` members on the
  command line, e.g. `--option to-string-max-attrs 1000`.

(cherry picked from commit 0fa08b451682fb3311fe58112ff05c4fe5bee3a4, )

===

Restore ambiguous value printer for `nix-instantiate`

The Nix team has requested that this output format remain unchanged.
I've added a warning to the man page explaining that `nix-instantiate
--eval` output will not parse correctly in many situations.

(cherry picked from commit df84dd4d8dd3fd6381ac2ca3064432ab31a16b79)

Change-Id: I7cca6b4b53cd0642f2d49af657d5676a8554c9f8
2024-03-09 03:50:06 +01:00
eldritch horrors 9eb58f5209 Merge pull request #9032 from Ma27/structured-attrs-env-vars
structured attrs: improve support / usage of NIX_ATTRS_{SH,JSON}_FILE

(cherry picked from commit 3c042f3b0b0a7ef9c47bf049f5410dbd4aac9e90)
Change-Id: I7e41838338ee1edf31fff6f9e354c3db2bba6c0e
2024-03-07 10:46:47 +01:00
eldritch horrors 4c56629a52 Merge pull request #9692 from 9999years/update-clang-stdenv
Remove `clang11Stdenv`

(cherry picked from commit cbf99c71c6f58579174c1c7649a5421c1d2ba9b0)
Change-Id: I716ce1a54f2006c4a9dc9716e8529fe7858ecebb
2024-03-04 07:48:42 +01:00
eldritch horrors 116e48fad3 Merge pull request #9573 from hercules-ci/rl-next-md-frontmatter
rl-next: Fix and support markdown frontmatter syntax
(cherry picked from commit 69b7876a0810269ad71807594cfd99b26cd8a5ff)
Change-Id: I8bfb8967af0943080fdd70d257c34abaf0a9fedf
2024-03-04 07:12:09 +01:00
eldritch horrors a089d8f5f6 Merge pull request #9465 from obsidiansystems/build-dir
Use `buildprefix` in a few more places

(cherry picked from commit b6a3fde6b7a416929553e6be36fc991680ddf9ef)
Change-Id: I2790663fa9f8242ac2db6582b7e421d2fdf42942
2024-03-04 07:11:25 +01:00
eldritch horrors e8a488236c Merge pull request #9393 from hercules-ci/changelog-d
Automatically compile hand-written release notes with `changelog-d`

(cherry picked from commit 928f0c13414d20c1af88b30bd6700fd730ee0bab)
Change-Id: Ia0685835c52edf185b64dd696b19305746c077e5
2024-03-04 07:11:19 +01:00
eldritch horrors 340a6cf977 Merge pull request #9257 from Artturin/nixenvjsondrvpath
`nix-env --query`: fix `--json` ignoring `--drv-path`

(cherry picked from commit 516e7ddc41f39ff939b5d5b5dc71e590f24890d4)
Change-Id: I84b5bccea9d0383e2e74544743b703942e7be547
2024-03-04 05:59:15 +01:00
eldritch horrors 032eff7f69 Merge pull request #8470 from ncfavier/shebang-single-quotes
nix-shell: support single quotes in shebangs, fix whitespace parsing
(cherry picked from commit 3b99c6291377cbd22607896af9dfafa857d2f2dc)
Change-Id: I2a431b21c3467eefa1ef95d5a36d672f45b6937a
2024-03-04 05:06:32 +01:00
eldritch horrors aeb803de9a Merge pull request #8047 from lovesegfault/always-allow-substitutes
feat: add always-allow-substitutes

(cherry picked from commit da2b59a08878b3c6c7074595e3b6d26b6928b4c1)
Change-Id: I50481cd8fe643c673c610fec28bad84519a4d650
2024-03-04 04:37:03 +01:00
eldritch horrors da0aa66d98 Merge pull request #9131 from obsidiansystems/delete-bootstrap-script
Get rid of `bootstrap.sh`

(cherry picked from commit aaef47a08eaf54a8856dd25c784fd85d8d7b0e22)
Change-Id: I1a74bed0c23d6fda06d5dfd8ecad443b9122da12
2024-03-04 04:36:52 +01:00
eldritch horrors f17e7b1855 Merge pull request #8923 from obsidiansystems/test-proto
Unit test some worker protocol serializers

(cherry picked from commit c6faef61a6f31c71146aee5d88168e861df9a22a)
Change-Id: I99e36f5f17eb7642211a4e42a16b143424f164b4
2024-03-04 04:36:14 +01:00