Commit graph

314 commits

Author SHA1 Message Date
Pierre Bourdon f7b6552699 [resubmit] flake: update nixpkgs pin 23.11->24.05 (+ boehmgc compat changes)
-- message from cl/1418 --

The boehmgc changes are bundled into this commit because doing otherwise
would require an annoying dance of "adding compatibility for < 8.2.6 and
>= 8.2.6" then updating the pin then removing the (now unneeded)
compatibility. It doesn't seem worth the trouble to me given the low
complexity of said changes.

Rebased coroutine-sp-fallback.diff patch taken from https://github.com/NixOS/nixpkgs/pull/317227

-- jade resubmit changes --

This is a resubmission of https://gerrit.lix.systems/c/lix/+/1418, which
was reverted in https://gerrit.lix.systems/c/lix/+/1432 for breaking CI
evaluation without being detected.

I have run `nix flake check -Lv` on this one before submission and it
passes on my machine and crucially without eval errors, so the CI result
should be accurate.

It seems like someone renamed forbiddenDependenciesRegex to
forbiddenDependenciesRegexes in nixpkgs and also changed the type
incompatibly. That's pretty silly, but at least it's just an eval error.

Also, `xonsh` regressed the availability of `xonsh-unwrapped`, but it
was fixed by us in https://github.com/NixOS/nixpkgs/pull/317636, which
is now in our channel, so we update nixpkgs compared to the original
iteration of this to simply get that.

We originally had a regression related to some reorganization of the
nixpkgs lib test suite in which there was broken parameter passing.
This, too, we got quickfixed in nixpkgs, so we don't need any changes
for it: https://github.com/NixOS/nixpkgs/pull/317772

Related: https://gerrit.lix.systems/c/lix/+/1428
Fixes: lix-project/lix#385

Change-Id: I26d41ea826fec900ebcad0f82a727feb6bcd28f3
2024-06-12 15:34:22 -07:00
jade 9bb7fb8f69 Rewrite docker to be sensible and smaller
I have checked the image can build things and inspected `diff -ru`
compared to the old image. As far as I can tell it is more or less
the same besides the later git change.

Layers are now 65MB or less, and we aren't against the maxLayers limit
for the broken automatic layering to do anything but shove one store
path in a layer (which is good behaviour, actually).

This uses nix2container which streams images, so the build time is much
shorter.

I have also taken the opportunity to, in addition to fixing the 400MB
single layer (terrible, and what motivated this in the first place),
delete about 200MB of closure size inflicted by git vs gitMinimal
causing both perl and python to get into closure.

People mostly use this thing for CI, so I don't really think you need
advanced git operations, and large git can be added at the user side if
really motivated.

With love for whichever container developer somewhat ironically assumed
that one would not run skopeo in a minimal container that doesn't have a
/var/tmp.

Fixes: lix-project/lix#378

Change-Id: Icc3aa20e64446276716fbbb87535fd5b50628010
2024-06-09 20:33:24 -07:00
jade bdf1b264ad Expose officialRelease from the flake
Change-Id: If87beb3f31dfb5d59862294ac2e1c821ea864277
2024-06-06 20:53:08 -07:00
jade c32a01f9eb Put into place initial release engineering
This can release x86_64-linux binaries to staging, with ephemeral keys.
I think it's good enough to review at least at this point, so we don't
keep adding more stuff to it to make it harder to review.

Change-Id: Ie95e8f35d1252f5d014e819566f170b30eda152e
2024-06-06 20:53:08 -07:00
jade 9c77c62e73 Move version to a JSON file so we can have release names
Change-Id: I5ff3396a302565ee5ee6c2db97e048e403779076
2024-06-06 15:08:12 -07:00
jade 24057dcb6a Remove rl-next-dev
We realized that there's really no good place to put these dev facing
bulletins, and the user-facing release notes aren't really the worst
place to put them, I guess, and we do kind of hope that it converts
users to devs.

Change-Id: Id9387b2964fe291cb5a3f74ad6344157f19b540c
2024-06-06 15:08:12 -07:00
raito b8cb7abcf0 chore: rebrand Nix to Lix when it makes sense
Here's my guide so far:

$ rg '((?!(recursive).*) Nix
(?!(daemon|store|expression|Rocks!|Packages|language|derivation|archive|account|user|sandbox|flake).*))'
-g '!doc/' --pcre2

All items from this query have been tackled. For the documentation side:
that's for lix-project/lix#162.

Additionally, all remaining references to github.com/NixOS/nix which
were not relevant were also replaced.

Fixes: lix-project/lix#148.
Fixes: lix-project/lix#162.
Change-Id: Ib3451fae5cb8ab8cd9ac9e4e4551284ee6794545
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-06-01 20:31:24 +02:00
jade 7cfaf057e3 release-notes: build unreleased release notes by default
Surely if you have unreleased changes you want them on a page right?

`officialRelease` means "this is a *release version*", which is a
reasonable case to not want it, but we are not that here.

I understand wanting to be able to turn it off for deps reasons or
something, but other than that, uhh, seems better to just turn it on
always; it is basically free compute-wise to the point we run it on
pre-commit.

Part two of fixing lix#297.

Fixes: lix-project/lix#297
Change-Id: I0f8dd1ae42458df371aef529c456e47a7ac04ae0
2024-05-15 15:01:38 -07:00
jade 2a7b3d7c94 build-release-notes: add change author metadata and use it
Change-Id: I6f5fb54f70b02a467bbdee4c526f59da1193f7db
2024-05-15 14:33:35 -07:00
jade d5804d0c6d Merge "fix: eval error of .#devShells.x86_64-linux.x86_64-freebsd13" into main 2024-05-09 15:22:03 +00:00
jade d1dacad708 fix: eval error of .#devShells.x86_64-linux.x86_64-freebsd13
This is broken and our resident nixbsd maintainers say it should
probably just be temporarily removed till we switch to 24.05 instead of
diagnosing it.

Originally introduced in: https://github.com/nixos/nix/pull/8887

Fixes: lix-project/lix#277
Change-Id: I1e7db8859620024a7b37dbd0cc1c5ec139b9e5cb
2024-05-09 13:07:30 +00:00
Maximilian Bosch a4c943403f
flake: update nixpkgs input to latest nixos-23.11
This includes the update to libseccomp 2.5.5[1], so we don't need to
override it on our own.

[1] https://nixpk.gs/pr-tracker.html?pr=306070

Change-Id: I1fa9c7fcc23e501d75f774745107c6bb086ced70
2024-05-08 23:03:28 +02:00
Qyriad f782c8a60a flake: refactor devShell creation
Now instead of a derivation overridden from Lix, we use a mkShell
derivation parameterized on an already called package.nix. This also
lets callPackage take care of the buildPackages distinction for the
devShell.

Change-Id: I5ddfec40d83fa6136032da7606fe6d3d5014ef42
2024-05-07 18:09:51 -06:00
Qyriad 8822fd7dd5 package: default the build-release-notes arg like we do with lix-doc
Change-Id: I0e2df55efc1cd6ea0a3252b9f26676e84612fdb6
2024-05-07 17:07:53 -06:00
Qyriad b9be46fb31 remove the autoconf+Make buildsystem
We're not using it anymore. Any leftover bugs in the Meson buildsystem
are now just bugs.

Closes #249.

Change-Id: I0465a0c37ae819f94d40e7829f5bff046aa63d73
2024-05-07 17:04:30 -06:00
Patrick Jackson d184981af0 Merge "feat: setup gerrit commit-msg hook with nix develop" into main 2024-05-07 22:51:53 +00:00
Qyriad 7e940cc170 flake: fix devShell on i686-linux by disabling ClangBuildAnalyzer on it
ClangBuildAnalyzer doesn't build on i686-linux due to
`long long int`/`size_t` conversion errors, so let's just exclude it
from the devshell on that platform

Change-Id: If1077a7b3860db4381999c8e304f6d4b2bc96a05
2024-05-07 15:31:25 -06:00
Patrick Jackson 9af8694367 feat: setup gerrit commit-msg hook with nix develop
Closes #273

Change-Id: Id883d2cda06adbcae53b8c360ad015330f0af81b
2024-05-07 14:20:09 -07:00
Qyriad aac32327d5 flake: fix eval of checks & devshell on i686-linux
Change-Id: I62da3161327051005e3f48f83974140efef4417e
2024-05-07 12:38:01 -06:00
Qyriad 4f98d21b71 flake: move the pre-commit definition to its own file
It's a good hundred LOC, and wasn't coupled to the actual flake logic at
all.

Change-Id: Iebb4667b3197dbd8cb2b019014e99fa651848832
2024-05-07 12:38:01 -06:00
Qyriad 10c1081b88 add a contributor notice message to the dev shell hook
It can be turned off by creating a file `.nocontribmsg` in the root
of the repo.

Change-Id: Iecc5c647c824a0416e527550226447780b94c08e
2024-05-05 11:22:38 -06:00
Maximilian Bosch 045ee37438 libstore/local-derivation-goal: prohibit creating setuid/setgid binaries
With Linux kernel >=6.6 & glibc 2.39 a `fchmodat2(2)` is available that
isn't filtered away by the libseccomp sandbox.

Being able to use this to bypass that restriction has surprising results
for some builds such as lxc[1]:

> With kernel ≥6.6 and glibc 2.39, lxc's install phase uses fchmodat2,
> which slips through 9b88e52846/src/libstore/build/local-derivation-goal.cc (L1650-L1663).
> The fixupPhase then uses fchmodat, which fails.
> With older kernel or glibc, setting the suid bit fails in the
> install phase, which is not treated as fatal, and then the
> fixup phase does not try to set it again.

Please note that there are still ways to bypass this sandbox[2] and this is
mostly a fix for the breaking builds.

This change works by creating a syscall filter for the `fchmodat2`
syscall (number 452 on most systems). The problem is that glibc 2.39
is needed to have the correct syscall number available via
`__NR_fchmodat2` / `__SNR_fchmodat2`, but this flake is still on
nixpkgs 23.11. To have this change everywhere and not dependent on the
glibc this package is built against, I added a header
"fchmodat2-compat.hh" that sets the syscall number based on the
architecture. On most platforms its 452 according to glibc with a few
exceptions:

    $ rg --pcre2 'define __NR_fchmodat2 (?!452)'
    sysdeps/unix/sysv/linux/x86_64/x32/arch-syscall.h
    58:#define __NR_fchmodat2 1073742276

    sysdeps/unix/sysv/linux/mips/mips64/n32/arch-syscall.h
    67:#define __NR_fchmodat2 6452

    sysdeps/unix/sysv/linux/mips/mips64/n64/arch-syscall.h
    62:#define __NR_fchmodat2 5452

    sysdeps/unix/sysv/linux/mips/mips32/arch-syscall.h
    70:#define __NR_fchmodat2 4452

    sysdeps/unix/sysv/linux/alpha/arch-syscall.h
    59:#define __NR_fchmodat2 562

I added a small regression-test to the setuid integration-test that
attempts to set the suid bit on a file using the fchmodat2 syscall.
I confirmed that the test fails without the change in
local-derivation-goal.

Additionally, we require libseccomp 2.5.5 or greater now: as it turns
out, libseccomp maintains an internal syscall table and
validates each rule against it. This means that when using libseccomp
2.5.4 or older, one may pass `452` as syscall number against it, but
since it doesn't exist in the internal structure, `libseccomp` will refuse
to create a filter for that. This happens with nixpkgs-23.11, i.e. on
stable NixOS and when building Lix against the project's flake.

To work around that

* a backport of libseccomp 2.5.5 on upstream nixpkgs has been
  scheduled[3].

* the package now uses libseccomp 2.5.5 on its own already. This is to
  provide a quick fix since the correct fix for 23.11 is still a staging cycle
  away.

We still need the compat header though since `SCMP_SYS(fchmodat2)`
internally transforms this into `__SNR_fchmodat2` which points to
`__NR_fchmodat2` from glibc 2.39, so it wouldn't build on glibc 2.38.
The updated syscall table from libseccomp 2.5.5 is NOT used for that
step, but used later, so we need both, our compat header and their
syscall table 🤷

Relevant PRs in CppNix:

* https://github.com/NixOS/nix/pull/10591
* https://github.com/NixOS/nix/pull/10501

[1] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2031073804
[2] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2030844251
[3] https://github.com/NixOS/nixpkgs/pull/306070

(cherry picked from commit ba6804518772e6afb403dd55478365d4b863c854)
Change-Id: I6921ab5a363188c6bff617750d00bb517276b7fe
2024-05-03 16:29:06 +02:00
Qyriad b913a939b0 meson: flip the switch!!
This commit makes Meson the default buildsystem for Lix.
The Make buildsystem is now deprecated and will be removed soon, but has
not yet, which will be done in a later commit when all seems good. The
mesonBuild jobs have been removed, and have not been replaced with
equivalent jobs to ensure the Make buildsystem still works.

The full, new commands in a development shell are:

$ meson setup ./build "--prefix=$out" $mesonFlags

(A simple `meson setup ./build` will also build, but will do a different
thing, not having the settings from package.nix applied.)

$ meson compile -C build
$ meson test -C build --suite=check
$ meson install -C build
$ meson test -C build --suite=installcheck

(Check and installcheck may both be done after install, allowing you to
omit the --suite argument entirely, but this is the order package.nix
runs them in.)

If tests fail and Meson helpfully has no output for why, use the
`--print-error-logs` option to `meson test`. Why this is not the default
I cannot explain.

If you change a setting in the buildsystem, most cases will
automatically regenerate the Meson configuration, but some cases, like
trying to build a specific target whose name is new to the buildsystem
(e.g. `meson compile -C build src/libmelt/libmelt.dylib`, when
`libmelt.dylib` did not exist as a target the last time the buildsystem
was generated), then you can reconfigure using new settings but
existing options, and only recompiling stuff affected by the changes:

$ meson setup --reconfigure build

Note that changes to the default values in `meson.options` or in the
`default_options :` argument to project() are NOT propagated with
`--reconfigure`.

If you want a totally clean build, you can use:

$ meson setup --wipe build

That will work regardless of if `./build` exists or not.

Specific, named targets may be addressed in
`meson build -C build <target>` with the "target ID" if there is one,
which is the first string argument passed to target functions that
have one, and unrelated to the variable name, e.g.:

libexpr_dylib = library('nixexpr', …)

can be addressed with:

$ meson compile -C build nixexpr

All targets may be addressed as their output, relative to the build
directory, e.g.:

$ meson compile -C build src/libexpr/libnixexpr.so

But Meson does not consider intermediate files like object files
targets. To build a specific object file, use Ninja directly and
specify the output file relative to the build directory:

$ ninja -C build src/libexpr/libnixexpr.so.p/nixexpr.cc.o

To inspect the canonical source of truth on what the state of the
buildsystem configuration is, use:

$ meson introspect

Have fun!

Change-Id: Ia3e7b1e6fae26daf3162e655b4ded611a5cd57ad
2024-04-22 21:41:58 -06:00
Qyriad cf0744ceed Merge "build internal API docs with Meson" into main 2024-04-17 21:48:25 +00:00
Qyriad b81eec6ed5 build internal API docs with Meson
This commit adds the capability for building the Doxygen internal API
docs in the Meson buildsystem, and also makes doing so the default for
the internal-api-docs hydra job. Aside from the /nix-support directory,
which differed only by the hash part of a store path, the outputs of
hydraJobs.internal-api-docs before and after this commit were
bit-for-bit identical on my machine.

Change-Id: I98f0017891c25b06866c15f7652fe74f706ec8e1
2024-04-15 19:05:07 -06:00
Qyriad 629351163d flake: factor out binary tarball into its own file
Bit-for-bit identical, and this one is callPackage-able

Change-Id: Ic635687b0054e107271a9c24ae69101f5e0fba9e
2024-04-12 06:35:54 -06:00
jade 1e74bffd5c pre-commit check for pragma once and ///@file
This is in our style guide, we can cheaply enforce it, let's do it.

```
$ pre-commit
check-case-conflicts.....................................................Passed
check-executables-have-shebangs..........................................Passed
check-headers............................................................Failed
- hook id: check-headers
- exit code: 1

Missing pattern @file in file src/libexpr/value.hh

We found some header files that don't conform to the style guide.

The Lix style guide requests that header files:
- Begin with `#pragma once` so they only get parsed once
- Contain a doxygen comment (`/**` or `///`) containing `@file`, for
  example, `///@file`, which will make doxygen generate docs for them.

  When adding that, consider also adding a `@brief` with a sentence
  explaining what the header is for.

For more details: https://wiki.lix.systems/link/3#bkmrk-header-files

check-merge-conflicts....................................................Passed
check-shebang-scripts-are-executable.....................................Passed
check-symlinks.......................................(no files to check)Skipped
end-of-file-fixer........................................................Passed
mixed-line-endings.......................................................Passed
no-commit-to-branch......................................................Passed
release-notes........................................(no files to check)Skipped
treefmt..................................................................Passed
trim-trailing-whitespace.................................................Passed
```

Fixes: lix-project/lix#233
Change-Id: I77150b9298c844ffedd0f85cc5250ae9208502e3
2024-04-08 16:10:57 -07:00
jade 06f17a5c78 release-notes: check with pre-commit
This required making the build-release-notes script understand how to
check multiple directories.

Change-Id: I057f5f636155ab6c6fb5755da5217b7e72249ece
2024-04-08 15:40:12 -07:00
jade 6fcab7ee95 pre-commit: stop using the flake
The flake for pre-commit-checks is rather questionable. We ignored
it so it uses our own nixpkgs and doesn't reimport nixpkgs. This should
save a couple of seconds of eval time!

Change-Id: I4584982beb32e0122f791fa29f6a544bdbb9e201
2024-04-08 15:29:23 -07:00
Rebecca Turner cfbcf12276 Format Nix code with nixfmt
Change-Id: I61efeb666ff7481c05fcb247168290e86a250151
2024-04-08 13:00:00 -07:00
Rebecca Turner b323340538 Add nixfmt
Change-Id: I7f21695e3971cfd02b2cce0dd016ff6eb3389905
2024-04-08 09:42:34 -07:00
jade 0145d45806 nix-doc -> lix-doc, make self-contained in package.nix
package.nix previously needed this callPackage'd externally, which
didn't make a lot of sense to us since this is an internal dependency.
Thus we changed it to make it more self contained.

Change-Id: I4935bc0bc80e1a132bc9b1519e917791da95037c
2024-04-08 04:05:13 +00:00
jade de20392c37 flake: remove dead code
Some of this code existed for installer tests, and indeed its removal is
an indication that our daemon cross-compatibility tests were removed.
Although these are not like, super critical tests, we would like to
restore them.

See: lix-project/lix#33
Change-Id: I75c733b25c00eca3a9676d498703bbfc1d6ec21b
2024-04-07 17:30:22 -07:00
Qyriad 84599abd4c Merge "flake: fix rl-next and rl-next-dev checks after 32eaa8a29 (fix nix flake check)" into main 2024-04-07 23:26:19 +00:00
Qyriad 0c5e2cfb33 package: just use fileset from lib
The following command is now sufficient to build Lix from outside of the
flake:

nix-build -E 'let pkgs = import <nixpkgs> { }; in pkgs.callPackage
./package.nix { build-release-notes = false; nix-doc = pkgs.callPackage
./nix-doc/package.nix { }; }'

Change-Id: Ie6b14b446480ac07c7266d4fba20042b04cc35b9
2024-04-07 23:10:05 +00:00
Qyriad 55e0c73695 flake: fix rl-next and rl-next-dev checks after 32eaa8a29 (fix nix flake check)
follow-up to 32eaa8a29[1] "flake: move release note checks to hydraJobs",
this commit fixes a load-bearing typo for`checks.rl-next` and
`checks.rl-next-dev`.

[1]: 32eaa8a2910793538deab31f85534faf7e722ef7

Change-Id: I9383ed21f7eccc337c0c2f65525418b735a94a1d
2024-04-07 17:05:51 -06:00
Qyriad 3ac2dd0613 package: put boehmgc patch logic in package.nix
In our view it really doesn't make sense to not have this in in
package.nix in some way. These patches aren't just for performance or
something -- Lix flat out doesn't build without these patches.

(Arguably that makes them a buildsystem responsibility as well, but that
can wait for when we're ready to start adding subproject fallback
dependency resolution to Meson.)

This is a step towards making `package.nix` more self-sufficient and
`callPackage`able without excessive external logic.

With this change the following command is enough to build Lix from out
of the flake:

nix-build -E 'let pkgs = import <nixpkgs> { }; in pkgs.callPackage
./package.nix { build-release-notes = false; inherit (pkgs.lib) fileset;
nix-doc = pkgs.callPackage ./nix-doc/package.nix { }; }'

Change-Id: Ia37fe8171f87d3293033de8be07d9bab12716f1d
2024-04-07 22:44:04 +00:00
eldritch horrors 1b5b9de04e flake: move release note checks to hydraJobs
having them in checks only does not run them in CI, which can cause
broken release notes entries to pass.

fixes #228

Change-Id: If0ba7b1be0b6525fc884a27e941cbc84b5a160f9
2024-04-06 15:51:52 +00:00
Lunaphied 7a1054fa5f repl: improve :doc builtin repl command to support lambdas.
For a long time `nix repl` has supported displaying documentation set on
builtins, however, it has long been convention to use Markdown comments
on Nix functions themselves for documentation. This exposes that
information to `nix repl` users in a nice and formatted way.

NixOS/rfcs#145 doc-comments are primarily what this feature is intended
to consume, however, support for lambda documentation in the repl is
experimental. We do our best effort to support the RFC here.

These changes are based on [the nix-doc library](https://github.com/lf-/nix-doc) and
are licensed under the terms described in the relevant source files.

Change-Id: Ic6fe947d39a22540705d890737e336c4720b0a22
2024-04-03 13:47:22 -06:00
Rebecca Turner 56c7dfd652 Merge "Add treefmt pre-commit hook" into main 2024-04-03 00:09:09 +00:00
Rebecca Turner 71ef7fd428 Merge "Use upstream cachix/git-hooks.nix for pre-commit" into main 2024-04-02 17:20:33 +00:00
Rebecca Turner ca9acf8184 Use upstream cachix/git-hooks.nix for pre-commit
Change-Id: I6bbc7d6da9accd7d2daffa9d780384df7226670e
2024-04-01 10:38:50 -07:00
Artemis Tosini 9adefa9de4 flake: Add cmake to devShell
Meson fails to setup when cmake is not found.
Add cmake to the default devShell so meson build works.

Change-Id: I4d933efac9540c564f3171e43c23e7a645722ef7
2024-03-31 21:02:50 +00:00
Rebecca Turner 655dae03de Add treefmt pre-commit hook
This lets us use different formatters for different filetypes.

Change-Id: Ib52383dd5097c8919a65e299aca2b5a55412223c
2024-03-29 22:57:43 -07:00
Rebecca Turner 2a98ba8b97 Add pre-commit checks
The big ones here are `trim-trailing-whitespace` and `end-of-file-fixer`
(which makes sure that every file ends with exactly one newline
character).

Change-Id: Idca73b640883188f068f9903e013cf0d82aa1123
2024-03-29 22:57:40 -07:00
Rebecca Turner 0b4e738944 Add a justfile to make meson friendly :)
Change-Id: Id6e4528392266c6f2444e030b67293abe297ed17
2024-03-30 03:03:46 +00:00
jade 412a9c9f67 Enable clang build timing analysis
I didn't enable this by default for clang due to making the build time
10% worse or so. Unfortunate, but tbh devs for whom 10% of build time is
not *that* bad should probably simply enable this.

Change-Id: I8d1e5b6f3f76c649a4e2f115f534f7f97cee46e6
2024-03-27 03:52:57 +00:00
jade 50c6feeb77 Add release notes system for dev facing release notes
We keep changing dev stuff and we probably should keep the news up to
date?

Change-Id: I819da6a29f1c56c8ab8d758c159a9c96164cb04e
2024-03-27 03:52:57 +00:00
eldritch horrors 0436f4cfa6 flake: always build release notes in devshell
Change-Id: I0e02567fe8f102a8a8f1558aa094eefacdac9393
2024-03-27 03:09:14 +00:00
eldritch horrors 279e30e7ef build: replace changelog-d with local script
hacking changelog-d to support not just github but also forgejo and
gerrit is a lot more complicated than it's worth, even moreso since
the entire thing can just as well be done with ~60 lines of python.
this new script is also much cheaper to instantiate (being python),
so having it enabled in all shells is far less of a hassle.

we've also adjusted existing release notes that referenced a gerrit
cl to auto-link to the cl in question, making the diff a bit bigger

closes lix-project/lix#176

Change-Id: I8ba7dd0070aad9ba4474401731215fcf5d9d2130
2024-03-27 03:09:14 +00:00