Compare commits

...

112 commits

Author SHA1 Message Date
Qyriad 9e1b0b04ad meson: flip the switch!!
This commit makes Meson the default buildsystem for Lix.
The Make buildsystem is now deprecated and will be removed soon, but has
not yet, which will be done in a later commit when all seems good. The
mesonBuild jobs have been removed, and have not been replaced with
equivalent jobs to ensure the Make buildsystem still works.

The full, new commands in a development shell are:

$ meson setup ./build "--prefix=$out" $mesonFlags

(A simple `meson setup ./build` will also build, but will do a different
thing, not having the settings from package.nix applied.)

$ meson compile -C build
$ meson test -C build --suite=check
$ meson install -C build
$ meson test -C build --suite=installcheck

(Check and installcheck may both be done after install, allowing you to
omit the --suite argument entirely, but this is the order package.nix
runs them in.)

If tests fail and Meson helpfully has no output for why, use the
`--print-error-logs` option to `meson test`. Why this is not the default
I cannot explain.

If you change a setting in the buildsystem, most cases will automatically
regenerate the Meson configuration, but some cases, like trying to build
a specific target whose name is new to the buildsystem (e.g.
`meson compile -C build src/libmelt/libmelt.dylib`, when
`libmelt.dylib` did not exist as a target the last time the buildsystem
was generated), then you can reconfigure using new settings but existing
options, and only recompiling stuff affected by the changes:

$ meson setup --reconfigure build

Note that changes to the default values in `meson.options` or in the
`default_options :` argument to project() are NOT propagated with
`--reconfigure`.

If you want a totally clean build, you can use:

$ meson setup --wipe build

That will work regardless of if `./build` exists or not.

Specific, named targets may be addressed in `meson build -C build <target>`
with "target ID" if there is one, which is the first string argument
passed to target functions that have one, and unrelated to the variable
name, e.g.:

libexpr_dylib = library('nixexpr', …)

can be addressed with:

$ meson compile -C build nixexpr

All targets may be addressed as their output, relative to the build
directory, e.g.:

$ meson compile -C build src/libexpr/libnixexpr.so

But Meson does not consider intermediate files like object files
targets. To build a specific object file, use Ninja directly and
specify the output file relative to the build directory:

$ ninja -C build src/libexpr/libnixexpr.so.p/nixexpr.cc.o

To inspect the canonical source of truth on what the state of the
buildsystem configuration is, use:

$ meson introspect

Have fun!

Change-Id: Ia3e7b1e6fae26daf3162e655b4ded611a5cd57ad
2024-04-18 16:15:58 -06:00
Qyriad 111db8b38f meson: correctly embed sandbox shell when asked
Change-Id: I2f6c0d42245204a516d2e424eea26a6391e975ad
2024-04-18 16:15:58 -06:00
eldritch horrors a326344253 tests: unhaunt the flakes nixos tests
these should really wait for networks to come up, otherwise they can fail.

fixes #235

Change-Id: I08989e8bdb0de280df74660ac43983de5c34fa9d
2024-04-18 20:09:19 +00:00
Qyriad f9d08cc44c meson: embed source paths as relative to the source root and avoid ../src
Change-Id: Ifab83cb7a3bfde717a4d6032ede8be75dc61f2b1
2024-04-18 10:45:27 -06:00
Qyriad 077f45ee38 meson: correctly set -DSANDBOX_SHELL if we have it
The statically embedded busybox is not required for Lix to work, but
package.nix explicitly sets this, which was accidentally being ignored.

Change-Id: Ieeff830ac7d1f5fabe84d1a6cfd82f13d79035bf
2024-04-18 10:45:27 -06:00
Qyriad cf0744ceed Merge "build internal API docs with Meson" into main 2024-04-17 21:48:25 +00:00
Ilya K 6d79aa3d70 Merge "libstore/build: set NO_NEW_PRIVS for the sandbox" into main 2024-04-16 05:33:41 +00:00
Qyriad b81eec6ed5 build internal API docs with Meson
This commit adds the capability for building the Doxygen internal API
docs in the Meson buildsystem, and also makes doing so the default for
the internal-api-docs hydra job. Aside from the /nix-support directory,
which differed only by the hash part of a store path, the outputs of
hydraJobs.internal-api-docs before and after this commit were
bit-for-bit identical on my machine.

Change-Id: I98f0017891c25b06866c15f7652fe74f706ec8e1
2024-04-15 19:05:07 -06:00
Qyriad a41abb4594 fix probable format bug in DerivationGoal::buildDone
Either the contents of `line` could cause format errors, or this usage
is Technically safe. However, I trust nothing, especially with
boost::format.

Change-Id: I07933b20bde3b305a6e5d61c2a7bab6ecb042ad9
2024-04-15 23:09:40 +00:00
Qyriad 4e68deef80 abort with a descriptive message on bad HintFmt usage
Change-Id: Ic2f05572042343a8160fd971394372f5f2706fc4
2024-04-15 23:09:16 +00:00
Ilya K effc28f6f5 libstore/build: set NO_NEW_PRIVS for the sandbox
Change-Id: I711f64e2b68495ed9c85c1a4bd5025405805e43a
2024-04-15 10:25:29 +03:00
Qyriad 80bbfe2034 don't throw an exception for the trivial case of isStorePath()...
Previously if isStorePath() was called on anything other than a
top-level /nix/store/some-path, it would throw a BadStorePath exception.
This commit duplicates the absolutely trivial check, into
maybeParseStorePath(), and leaves exception throwing to
parseStorePath(), the function that assumes you're already giving a
valid path instead of the one whose purpose is to check if its valid or
not...

Change-Id: I8dda548f0f88d14ca8c3ee927d64e0ec0681fc7b
2024-04-14 21:08:07 +00:00
Qyriad ddb4d3fa4c Merge "don't boost::to_few_args when an eval cached string type errors" into main 2024-04-14 21:07:47 +00:00
Ilya K 8d15e6af4b Merge "libstore/build: just copy the magic /etc files into the sandbox" into main 2024-04-13 12:15:20 +00:00
Ilya K b469c6509b libstore/build: just copy the magic /etc files into the sandbox
Saves us a bunch of thinking about how to handle symlinks, and prevents
the DNS config from changing on the fly under the build, which may or may
not be a good thing?

Change-Id: I071e6ae7e220884690b788d94f480866f428db71
2024-04-13 12:43:19 +03:00
Qyriad ded64e2822 Merge changes I60d8e6f7,Ic635687b into main
* changes:
  binary tarball: include cacert in root paths
  flake: factor out binary tarball into its own file
2024-04-12 13:24:47 +00:00
Qyriad a3be742bda binary tarball: include cacert in root paths
93cc06334 removed nss-cacert from the binary tarball, but they're
necessary for global compatibility (and for our installer). This is what
results in cacerts being in the default profile, so e.g. the daemon has
TLS certs without having to use the system ones.

There's a fallback behavior in the daemon script in case these wind up
missing from the profile, but we don't want to have to rely on that,
since the fallback fails if it doesn't recognize one of a handful of
distros.

Change-Id: I60d8e6f734469548e80d5f38113ef168f67cbf7d
2024-04-12 07:04:37 -06:00
Qyriad 629351163d flake: factor out binary tarball into its own file
Bit-for-bit identical, and this one is callPackage-able

Change-Id: Ic635687b0054e107271a9c24ae69101f5e0fba9e
2024-04-12 06:35:54 -06:00
Ilya K d363bc2f12 Merge "Merge pull request #10456 from NixOS/fixpermdeniedbind" into main 2024-04-11 19:08:33 +00:00
eldritch horrors e4a8c01bdf Merge changes Iedf46484,I76b51eac,I6a084827,I60193f9f into main
* changes:
  meson: fix log-dir
  manual: build docs with dummy envs
  libcmd: install generated headers as well
  docs: redo content generation for mdbook and manual
2024-04-11 14:33:16 +00:00
Ilya K d106bb553b Merge "Merge pull request #10362 from obsidiansystems/maybeLstat" into main 2024-04-11 13:45:46 +00:00
eldritch horrors cd79b8d65a meson: fix log-dir
the make build system sets this with an extra /nix segment.

Change-Id: Iedf464843196faeae5b59698837faca3a4f23586
2024-04-11 13:36:04 +00:00
eldritch horrors adab839c98 manual: build docs with dummy envs
this was previously used because the macOS docs build would otherwise
pull files out of the host nix store. or something. not sure about it

Change-Id: I76b51eac1ebc5de5f00e2e4be086dd8db3eeb8e6
2024-04-11 13:36:04 +00:00
eldritch horrors f42678802c libcmd: install generated headers as well
these seem to have been forgotten.

Change-Id: I6a084827d087f8098c19b62f2060a874d87202a1
2024-04-11 13:36:04 +00:00
eldritch horrors 725f5cd358 docs: redo content generation for mdbook and manual
manpages can be rendered using the markdown output of mdbook, the rest
of the manual can generated out of the main doc/manual source tree. we
still use lowdown to actually render manpages instead of eg mdbook-man
because lowdown does generate reasonably good manpages (though that is
also somewhat debatable, but they're a lot better than mdbook-man).

doing this not only lets us drastically simplify the lowdown pipeline,
but also remove all custom {{#include}} handling since now mdbook does
all of it, even for the manpage builds. even the lowdown wrapper isn't
entirely necessary because lowdown can take all wrapper arguments with
command line flags rather than bits of input file content.

This also implements running mdbook in Meson, in order to generate the
manpages. The mdbook outputs are also installed in the usual location.

Co-authored-by: Qyriad <qyriad@qyriad.me>

Change-Id: I60193f9fd0f15d48872f071af35855cda2a0f40b
2024-04-11 13:32:06 +00:00
Théophane Hufschmitt 07b627cc6d Merge pull request #10456 from NixOS/fixpermdeniedbind
Fix adding symlink to the sandbox paths

(cherry-picked from commit da1e977bf48cff2a635034c85e7c13878e38efc2)

Change-Id: I221c85a38180800ec6552d2e86a88df48398fad8
2024-04-11 15:43:58 +03:00
John Ericson aeee22e5a1 Merge pull request #10362 from obsidiansystems/maybeLstat
Factor out `nix::maybeLstat`

(cherry-picked from commit 9b88e5284608116b7db0dbd3d5dd7a33b90d52d7)

Change-Id: Id890525e847c890fad6593c594772826ac4d1d50
2024-04-11 15:43:41 +03:00
eldritch horrors a0875f6adf libstore: fix glossary link in documentation
this should be a link, not an anchor. it should also point to the
`gloss-store` element, not the `#gloss-store` element.

Change-Id: I1f2803093179549637e10f917ad73399a419131b
2024-04-11 02:34:45 +02:00
Qyriad 70af056de8 don't boost::to_few_args when an eval cached string type errors
Change-Id: Id3cb762622e156ceaf9d5bb95c2c704ffe474d0e
2024-04-10 18:30:12 -06:00
Rebecca Turner 99845e0e01 Merge "Print top-level errors normally in nix repl" into main 2024-04-10 15:40:03 +00:00
Qyriad 784a46654c Merge "docs: generalize manpage generation script as json-to-tree.py" into main 2024-04-10 13:40:47 +00:00
Qyriad 723ddadf92 docs: generalize manpage generation script as json-to-tree.py
This should be capable of replacing every invocation of
nix eval --write-to.

Change-Id: I60387bc9b0fc54a91244eddb639beaa64d705878
2024-04-10 15:04:34 +02:00
Rebecca Turner 9e7e927837 Print top-level errors normally in nix repl
Previously, errors while printing values in `nix repl` would be printed
in `«error: ...»` brackets rather than displayed normally:

```
nix-repl> legacyPackages.aarch64-darwin.pythonPackages.APScheduler
«error: Package ‘python-2.7.18.7’ in /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/cpython/2.7/default.nix:335 is marked as insecure, refusing to evaluate.»
```

Now, errors will be displayed normally if they're emitted at the
top-level of an expression:

```
nix-repl> legacyPackages.aarch64-darwin.pythonPackages.APScheduler
error:
       … in the condition of the assert statement
         at /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/lib/customisation.nix:268:17:
          267|     in commonAttrs // {
          268|       drvPath = assert condition; drv.drvPath;
             |                 ^
          269|       outPath = assert condition; drv.outPath;

       … in the left operand of the OR (||) operator
         at /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/passthrufun.nix:28:45:
           27|         if lib.isDerivation value then
           28|           lib.extendDerivation (valid value || throw "${name} should use `buildPythonPackage` or `toPythonModule` if it is to be part of the Python packages set.") {} value
             |                                             ^
           29|         else

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: Package ‘python-2.7.18.7’ in /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/cpython/2.7/default.nix:335 is marked as insecure, refusing to evaluate.
```

Errors emitted in nested structures (like e.g. when printing `nixpkgs`)
will still be printed in brackets.

Change-Id: I25aeddf08c017582718cb9772a677bf51b9fc2ad
2024-04-09 08:34:40 -07:00
Qyriad 4b99f43aeb meson: correctly differentiate $profiledir and $sysconfdir/profile.d
The configured sysconfdir is used to look for nix.conf, so it needs
to be /etc, and not $out/etc, so we separate out the place where shell
profile files are installed, which is the only other place sysconfdir is
at all used.

See lix-project/lix#231 (comment)
for more info.

Change-Id: Idbed8ba82e711b8a9d6b6127904befa27d58e279
2024-04-09 02:25:58 -06:00
Qyriad 2a98fe28cb meson: correctly set NIX_CONF_DIR to $sysconfdir/nix
Instead of $sysconfdir.

Fixes #231, but there's more to do in following commits to make
Meson-built Lix actually look in /etc/nix.

Change-Id: Ia8d627070f405843add46e05cff5134b76b8eb48
2024-04-09 02:25:58 -06:00
Qyriad 7e139c52dd format repl-overlays.nix to fix pre-commit checks
Change-Id: I9487f5832160c1093d8752f883c1be85f8310e0f
2024-04-09 02:21:26 -06:00
Rebecca Turner a95c62673b Merge "Add repl-overlays" into main 2024-04-09 05:29:48 +00:00
jade b0293f2de1 Merge "Add benchmarking scripts" into main 2024-04-09 03:01:00 +00:00
jade f4fc2b5d10 Add benchmarking scripts
These scripts were originally written by horrors, and have since been
hacked up a lot by jade. We are putting them up as a CL since it is
better to have checked in benchmarking scripts than to not have
benchmarking scripts.

cc: lix-project/lix#23

Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I95c2f9d24753ac468944c5781deec9508fd5cb8c
2024-04-08 19:50:24 -07:00
eldritch horrors e6aab36d60 docs: put the manual build into a known tmpdir
this isn't strictly necessary, but it'll make it a lot easier to put the
generated files used by the autoconf build system in this directory too.
doing this now already will make the meson transition a lot easier later

Change-Id: I5fb39eade2ff88b6093c9ee436c9e8db793e9448
2024-04-09 02:09:36 +00:00
eldritch horrors c1ae35d34a docs: don't compute SUMMARY.md during build
this would make meson build compatibility unnecessarily hard and
the cli does not change often enough to justify this complexity.

Change-Id: I17b1870cdf8538feeaa01a9945db97af2175a642
2024-04-09 02:09:36 +00:00
eldritch horrors d9bc197ff4 docs: don't compute rl-next.md during build
not sure why this was done the way it was considering that includes are
a feature the doc toolchain had previously. let's just always have some
kind of entry for the upcoming release in the dev manual builds even if
that means having a completely empty release notes chapter.

the release notes generation script isn't entirely functional right now
due to pre-commit hooks, but it's good enough for time being. we need a
better release process for notes anyway.

Change-Id: Ifda6912cf5233db013f72a30247a62d6f22b1565

Change-Id: I9eb347ec4aabc5be2b816ff0fd3e4be45f93b934
2024-04-09 02:09:36 +00:00
eldritch horrors 07ef9f9f5d docs: skip do custom include processing for mdbook
mdbook already does include processing of its own, and the custom
processing code has always admitted as much. we don't need it for
the mdbook build at this point if we run our preprocessors in the
right order, and maybe we can even have mdbook to return complete
pages to us that we only have to pass to lowdown without any more
preprocessing of our own.

Change-Id: Icd978acbc3b1e215fee8f062c53ab2cb2a222ab1
2024-04-09 02:09:36 +00:00
eldritch horrors 14088e1ec5 docs: remove many link-to-self
for some reason these three were anchors, not links, but had they been
links they wouldn't've worked because they're not defined anywhere but
here. in the print version of the manual they're duplicated many times
over (creating id collisions), so we should better remove them anyway.

Change-Id: I8988a7c32c812dee0f0b6d4953faa7cd1255228d
2024-04-09 02:09:36 +00:00
Rebecca Turner 727b43478c Add repl-overlays
Adds a `repl-overlays` option, which specifies files that can overlay
and modify the top-level bindings in `nix repl`. For example, with the
following contents in `~/.config/nix/repl.nix`:

    info: final: prev: let
      optionalAttrs = predicate: attrs:
        if predicate
        then attrs
        else {};
    in
      optionalAttrs (prev ? legacyPackages && prev.legacyPackages ? ${info.currentSystem})
      {
        pkgs = prev.legacyPackages.${info.currentSystem};
      }

We can run `nix repl` and use `pkgs` to refer to `legacyPackages.${currentSystem}`:

    $ nix repl --repl-overlays ~/.config/nix/repl.nix nixpkgs
    Lix 2.90.0
    Type :? for help.
    Loading installable 'flake:nixpkgs#'...
    Added 5 variables.
    Loading 'repl-overlays'...
    Added 6 variables.
    nix-repl> pkgs.bash
    «derivation /nix/store/g08b5vkwwh0j8ic9rkmd8mpj878rk62z-bash-5.2p26.drv»

Change-Id: Ic12e0f2f210b2f46e920c33088dfe1083f42391a
2024-04-08 17:11:47 -07:00
jade 1e74bffd5c pre-commit check for pragma once and ///@file
This is in our style guide, we can cheaply enforce it, let's do it.

```
$ pre-commit
check-case-conflicts.....................................................Passed
check-executables-have-shebangs..........................................Passed
check-headers............................................................Failed
- hook id: check-headers
- exit code: 1

Missing pattern @file in file src/libexpr/value.hh

We found some header files that don't conform to the style guide.

The Lix style guide requests that header files:
- Begin with `#pragma once` so they only get parsed once
- Contain a doxygen comment (`/**` or `///`) containing `@file`, for
  example, `///@file`, which will make doxygen generate docs for them.

  When adding that, consider also adding a `@brief` with a sentence
  explaining what the header is for.

For more details: https://wiki.lix.systems/link/3#bkmrk-header-files

check-merge-conflicts....................................................Passed
check-shebang-scripts-are-executable.....................................Passed
check-symlinks.......................................(no files to check)Skipped
end-of-file-fixer........................................................Passed
mixed-line-endings.......................................................Passed
no-commit-to-branch......................................................Passed
release-notes........................................(no files to check)Skipped
treefmt..................................................................Passed
trim-trailing-whitespace.................................................Passed
```

Fixes: lix-project/lix#233
Change-Id: I77150b9298c844ffedd0f85cc5250ae9208502e3
2024-04-08 16:10:57 -07:00
jade c58e3f826e pragma once and ///@file everything missing it
Change-Id: Ia1a72348336794b5fb9f2694dd750266089b904e
2024-04-08 15:40:12 -07:00
jade 54bc1f1b98 "but doctor, I AM the untrusted store": nix doctor had wrong trustedness
This probably snuck in in a refactor using truthiness or so. The
trustedness flag was having the optional fullness checked, rather than
the actual contained trust level.

Also adds some tests.

```
m1@6876551b-255d-4cb0-af02-8a4f17b27e2e ~ % nix store ping
warning: 'nix store ping' is a deprecated alias for 'nix store info'
Store URL: daemon
Version: 2.20.4
Trusted: 0
m1@6876551b-255d-4cb0-af02-8a4f17b27e2e ~ % nix doctor
warning: 'doctor' is a deprecated alias for 'config check'
[PASS] PATH contains only one nix version.
[PASS] All profiles are gcroots.
[PASS] Client protocol matches store protocol.
[INFO] You are trusted by store uri: daemon
```

Fixes: lix-project/lix#232
Change-Id: I21576e2a0a755036edf8814133345987617ba3d0
2024-04-08 15:40:12 -07:00
jade 0d37b99a15 build: enable libstdc++ assertions
Benchmarks say that it does not regress performance by more than 1%
(which is where it gets really hard to measure accurately anyhow).

Meson appears to be planning to do this for us without asking us in a
release we will get in the future, and it seems good enough to ship
today:
https://mesonbuild.com/Release-notes-for-1-4-0.html#ndebug-setting-now-controls-c-stdlib-assertions

Benchmarks:

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `result-asserts/bin/nix --extra-experimental-features 'nix-command flakes' eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix` | 418.4 ± 25.0 | 396.9 | 451.2 | 1.01 ± 0.08 |
| `result/bin/nix --extra-experimental-features 'nix-command flakes' eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix` | 416.1 ± 23.9 | 397.1 | 445.4 | 1.00 |

| Command | Mean [s] | Min [s] | Max [s] | Relative |
|:---|---:|---:|---:|---:|
| `GC_INITIAL_HEAP_SIZE=10g result-asserts/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 4.147 ± 0.021 | 4.123 | 4.195 | 1.00 |
| `GC_INITIAL_HEAP_SIZE=10g result/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 4.149 ± 0.027 | 4.126 | 4.215 | 1.00 ± 0.01 |

| Command | Mean [s] | Min [s] | Max [s] | Relative |
|:---|---:|---:|---:|---:|
| `result-asserts/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 5.838 ± 0.023 | 5.799 | 5.867 | 1.01 ± 0.01 |
| `result/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 5.788 ± 0.044 | 5.715 | 5.876 | 1.00 |

| Command | Mean [s] | Min [s] | Max [s] | Relative |
|:---|---:|---:|---:|---:|
| `result-asserts/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello` | 15.993 ± 0.081 | 15.829 | 16.096 | 1.01 ± 0.01 |
| `result/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello` | 15.897 ± 0.075 | 15.807 | 16.047 | 1.00 |

Fixes: lix-project/lix#4

Change-Id: Id3a6f38274ba94d5d10b09edd19dfd96bc3e7d5f
2024-04-08 15:40:12 -07:00
jade 06f17a5c78 release-notes: check with pre-commit
This required making the build-release-notes script understand how to
check multiple directories.

Change-Id: I057f5f636155ab6c6fb5755da5217b7e72249ece
2024-04-08 15:40:12 -07:00
jade 6fcab7ee95 pre-commit: stop using the flake
The flake for pre-commit-checks is rather questionable. We ignored
it so it uses our own nixpkgs and doesn't reimport nixpkgs. This should
save a couple of seconds of eval time!

Change-Id: I4584982beb32e0122f791fa29f6a544bdbb9e201
2024-04-08 15:29:23 -07:00
jade 860b34cf07 build-release-notes: support multiple PRs/CLs
Change-Id: I923e4af099d02d2324777cc9f7e6d38b130aac10
2024-04-08 15:29:19 -07:00
Rebecca Turner 5ec256625e Merge "Don't run libstore unit tests in project root" into main 2024-04-08 22:08:34 +00:00
Rebecca Turner 3910d4aa68 Merge "Fix REPL test parser nits" into main 2024-04-08 21:52:06 +00:00
Rebecca Turner cfbcf12276 Format Nix code with nixfmt
Change-Id: I61efeb666ff7481c05fcb247168290e86a250151
2024-04-08 13:00:00 -07:00
Rebecca Turner 12f2565f91 Don't run libstore unit tests in project root
This keeps the libstore unit tests from writing
`libstore-unit-tests.xml` to the project root.

Change-Id: I0d9909aabf9f3574cc1e72a5ae81daefba9a394b
2024-04-08 12:43:35 -07:00
Rebecca Turner 4a49a497eb Fix REPL test parser nits
Follow-up to https://gerrit.lix.systems/c/lix/+/546

Change-Id: Ie603f01e5520329bf879e061cea9e3fba45213fd
2024-04-08 11:14:34 -07:00
Rebecca Turner b323340538 Add nixfmt
Change-Id: I7f21695e3971cfd02b2cce0dd016ff6eb3389905
2024-04-08 09:42:34 -07:00
jade e55fc5af71 Merge "nix-doc -> lix-doc, make self-contained in package.nix" into main 2024-04-08 07:45:00 +00:00
Rebecca Turner 4600eabdae Merge "Rename some parse failure tests to parse-fail-*" into main 2024-04-08 06:33:51 +00:00
Rebecca Turner 8ed8b9aef0 Merge "Add PathsSetting" into main 2024-04-08 06:31:11 +00:00
jade 0145d45806 nix-doc -> lix-doc, make self-contained in package.nix
package.nix previously needed this callPackage'd externally, which
didn't make a lot of sense to us since this is an internal dependency.
Thus we changed it to make it more self contained.

Change-Id: I4935bc0bc80e1a132bc9b1519e917791da95037c
2024-04-08 04:05:13 +00:00
Robert Hensing b995c17f0e Merge pull request #10208 from 9999years/print-strings-directly
`:print` strings directly in `nix repl`

(cherry picked from commit 3539172fd2f7cee639ce46423c58beca4231f2db)
Change-Id: I1972f3bf3b56312851f38288509d371d37f21677
Upstream-PR: https://github.com/NixOS/nix/pull/10208
2024-04-07 19:10:43 -07:00
Rebecca Turner 6f863e8ccd Add PathsSetting
Change-Id: I1165f6ef033a5f757ca3716d3f8008ba36b01fd0
2024-04-07 18:22:15 -07:00
jade de20392c37 flake: remove dead code
Some of this code existed for installer tests, and indeed its removal is
an indication that our daemon cross-compatibility tests were removed.
Although these are not like, super critical tests, we would like to
restore them.

See: lix-project/lix#33
Change-Id: I75c733b25c00eca3a9676d498703bbfc1d6ec21b
2024-04-07 17:30:22 -07:00
Qyriad 84599abd4c Merge "flake: fix rl-next and rl-next-dev checks after 32eaa8a29 (fix nix flake check)" into main 2024-04-07 23:26:19 +00:00
Qyriad 0c5e2cfb33 package: just use fileset from lib
The following command is now sufficient to build Lix from outside of the
flake:

nix-build -E 'let pkgs = import <nixpkgs> { }; in pkgs.callPackage
./package.nix { build-release-notes = false; nix-doc = pkgs.callPackage
./nix-doc/package.nix { }; }'

Change-Id: Ie6b14b446480ac07c7266d4fba20042b04cc35b9
2024-04-07 23:10:05 +00:00
Qyriad 55e0c73695 flake: fix rl-next and rl-next-dev checks after 32eaa8a29 (fix nix flake check)
follow-up to 32eaa8a29[1] "flake: move release note checks to hydraJobs",
this commit fixes a load-bearing typo for`checks.rl-next` and
`checks.rl-next-dev`.

[1]: 32eaa8a291

Change-Id: I9383ed21f7eccc337c0c2f65525418b735a94a1d
2024-04-07 17:05:51 -06:00
Qyriad 3ac2dd0613 package: put boehmgc patch logic in package.nix
In our view it really doesn't make sense to not have this in in
package.nix in some way. These patches aren't just for performance or
something -- Lix flat out doesn't build without these patches.

(Arguably that makes them a buildsystem responsibility as well, but that
can wait for when we're ready to start adding subproject fallback
dependency resolution to Meson.)

This is a step towards making `package.nix` more self-sufficient and
`callPackage`able without excessive external logic.

With this change the following command is enough to build Lix from out
of the flake:

nix-build -E 'let pkgs = import <nixpkgs> { }; in pkgs.callPackage
./package.nix { build-release-notes = false; inherit (pkgs.lib) fileset;
nix-doc = pkgs.callPackage ./nix-doc/package.nix { }; }'

Change-Id: Ia37fe8171f87d3293033de8be07d9bab12716f1d
2024-04-07 22:44:04 +00:00
Rebecca Turner 06e11778b5 Merge "Rewrite REPL test parser" into main 2024-04-06 21:36:26 +00:00
Qyriad 5bac308c7c meson: fix warm nix3 CLI manual generation
`nix eval --write-to` refuses to write to a directory that exists at
all, so now we generate in a temporary directory, and copy the generated
tree to the build directory. This is equivalent to what the Make
buildsystem did, actually, but hopefully more robust.

Future work: documenting the doc generation architecture in the
top-level meson.build outline comment.

Change-Id: Ic3eb6d26e3cc249a1c042fd3ced22d637ac66a69
2024-04-06 14:43:14 -06:00
eldritch horrors 1b5b9de04e flake: move release note checks to hydraJobs
having them in checks only does not run them in CI, which can cause
broken release notes entries to pass.

fixes #228

Change-Id: If0ba7b1be0b6525fc884a27e941cbc84b5a160f9
2024-04-06 15:51:52 +00:00
terru - a33c95be5b avoid markdown which the repl's :doc cannot handle
code blocks, if not surrounded by empty lines, have the language
tags (in these cases, always `nix`) show up in the output of :doc.

for example:

  nix-repl> :doc builtins.parseFlakeRef
  Synopsis: builtins.parseFlakeRef flake-ref

    Parse a flake reference, and return its exploded form.

    For example: nix builtins.parseFlakeRef
    "github:NixOS/nixpkgs/23.05?dir=lib"  evaluates to: nix { dir =
    "lib"; owner = "NixOS"; ref = "23.05"; repo = "nixpkgs"; type =
    "github"; }

is now instead:
  nix-repl> :doc builtins.parseFlakeRef
  Synopsis: builtins.parseFlakeRef flake-ref

    Parse a flake reference, and return its exploded form.

    For example:

      | builtins.parseFlakeRef "github:NixOS/nixpkgs/23.05?dir=lib"

    evaluates to:

      | { dir = "lib"; owner = "NixOS"; ref = "23.05"; repo = "nixpkgs"; type = "github"; }

(closes #225)

Change-Id: I0741aeb1006a5376bb2f663d202c7a4da7e38cce
2024-04-06 13:14:14 +00:00
Qyriad fae48884c5 Merge "package: add python to nativeBuildInputs" into main 2024-04-06 05:29:34 +00:00
Qyriad 9bd52cb952 Merge "always re-eval cached failures" into main 2024-04-06 05:20:14 +00:00
jade 9e166f3456 Merge "Create clang-tidy check to rename all our includes" into main 2024-04-06 05:01:21 +00:00
jade 43cf487c25 Create clang-tidy check to rename all our includes
It is a little bit scuffed, but it seems to produce correct results. We
can run it at a later date when we want to explode every in-flight
commit in existence and then need to filter-branch them.

Fixes: lix-project/lix#188

Change-Id: Id97e4651f78804a941d941df02c7c1b21ce453b6
2024-04-06 04:40:19 +00:00
Qyriad a4ef195a9f always re-eval cached failures
This is terrible UX, and frankly an eval failure should be a cache
invalidation anyway.

This removes the CachedEvalError type entirely.

Fixes #223.

Change-Id: I91f8003eabd0ea45003024e96d1de3c7ae8e49d8
2024-04-06 04:35:25 +00:00
Qyriad 11ad383327 package: add python to nativeBuildInputs
Commit c21d11ac0 "docs: replace sed invocation with an mdbook
preprocessor for @docroot@" added a direct build dependency on Python.
This has been accidentally working so far because Python is already a
*transitive* dependency of Lix's derivation.

Change-Id: I32d6b4f2665dbbfad7014613457dd58aa4ec73da
2024-04-05 22:28:27 -06:00
Qyriad b47c159aec Merge "docs: replace sed invocation with an mdbook preprocessor for @docroot@" into main 2024-04-06 01:05:32 +00:00
Qyriad 4e11b0d04d Merge "meson: add missing tests: ca, dyn-drv, plugins, libstoreconsumer" into main 2024-04-06 01:05:09 +00:00
eldritch horrors e9e1b6963c Merge changes I1fa30114,I3ca208b6,Ide4c6e00,I74c46b9f,I05fa6a9d, ... into main
* changes:
  Revert "libutil: drop Pool resources on exceptional free"
  Revert "libutil: remove Pool::Handle::bad"
  Revert "libstore: remove one Resource::good flag"
  Revert "libstore: using throwing finally in withFramedSink"
  Revert "libutil: allow graceful dropping of Pool::Handle"
  Revert "libutil: drop Fs{Source,Sink}::good"
  libutil: guard Finally against invalid exception throws
2024-04-05 23:17:18 +00:00
Qyriad 405e41e288 Merge "meson: run repl characterization test" into main 2024-04-05 20:57:42 +00:00
Rebecca Turner ee423f391d Rewrite REPL test parser
- Use a recursive descent parser so that it's easy to extend.
- Add `@args` to enable customizing command-line arguments
- Add `@should-start` to enable `nix repl` tests that error before
  entering the REPL
- Make sure to read all stdout output before comparing. This catches
  some extra output we were tossing out before!

Change-Id: I5522555df4c313024ab15cd10f9f04e7293bda3a
2024-04-05 13:14:21 -07:00
eldritch horrors 38dc6f5b69 Revert "libutil: drop Pool resources on exceptional free"
This reverts commit de2884b82b.

Change-Id: I1fa301149d7c2ed3d266a40c15b2d010e12e44e6
2024-04-05 20:13:02 +00:00
eldritch horrors 52f741c23a Revert "libutil: remove Pool::Handle::bad"
This reverts commit 792844fb86.

Change-Id: I3ca208b62edfd5cd1199478f75cd2edf19a364f6
2024-04-05 20:13:02 +00:00
eldritch horrors 0b8a17cab6 Revert "libstore: remove one Resource::good flag"
This reverts commit 87249eb579.

Change-Id: Ide4c6e00c4155216a17e46671ff47151d7bb85b4
2024-04-05 20:13:02 +00:00
eldritch horrors ad30972867 Revert "libstore: using throwing finally in withFramedSink"
This reverts commit 491caad6f6.

this is not actually legal for nix! throwing exceptions in destructors
is fine, but the way nix is set up we'll end up throwing the exception
we received from the remote *twice* in some cases, and such cases will
cause an immediate terminate without active exception.

Change-Id: I74c46b9f26fd791086e4193ec60eb1deb9a5bb2a
2024-04-05 20:13:02 +00:00
eldritch horrors c77b6e1fdd Revert "libutil: allow graceful dropping of Pool::Handle"
This reverts commit 8075541d82.

Change-Id: I05fa6a9de1308a4827a6557cf2807eb47ca64da6
2024-04-05 20:13:02 +00:00
eldritch horrors 821ad98beb Revert "libutil: drop Fs{Source,Sink}::good"
This reverts commit 1340807e30.

Change-Id: I34d2a80eb3c3e9d79cb02b92cd1189da32d18cb6
2024-04-05 20:13:02 +00:00
eldritch horrors 6c777476c9 libutil: guard Finally against invalid exception throws
throwing exceptions is fine, but throwing exceptions during exception
handling is hard enough to do correctly that we should just forbid it
entirely out of an overabundance of caution. in cases where terminate
is the correct answer the users of Finally must call it manually now.

Change-Id: Ia51a2cb4a0638500550bfabc89cf01a6d8098983
2024-04-05 20:13:02 +00:00
Rebecca Turner 5081109592 Fix Boost with make build
This prevents the autotools build from discovering Boost in Homebrew
installation directories on macOS.

Change-Id: I624309165c9371c391fd657424ba4c4f3182b385
2024-04-05 11:09:59 -07:00
Qyriad 07e1ddc23d meson: add missing tests: ca, dyn-drv, plugins, libstoreconsumer
Change-Id: I6a74ebaf93697cb99aadd6b51538c2766b0a808a
2024-04-04 20:43:38 -06:00
Rebecca Turner 83d720b730 Merge "Make repl_characterization.cc constants string_views" into main 2024-04-04 23:08:09 +00:00
Rebecca Turner 0fec7d2db6 Rename some parse failure tests to parse-fail-*
These were mistakenly labeled `eval-fail-*`.

Note that the `lang.sh` runner passes `parse-fail-*` tests on stdin, so
filenames are removed from error messages.

Change-Id: I7f3a0d78b6cfa87af29aaa1b7af19d5a57fd4ade
2024-04-04 15:49:30 -07:00
Qyriad c355354772 docs: replace sed invocation with an mdbook preprocessor for @docroot@
We're not entirely clear on why the links preprocessor has to be done
*before* rather than after, but we assume it is probably that as a
builtin preprocessor it does some processing on the raw book source,
and not just the JSON data.

Also a real use for Python pattern matching? I know I was surprised too.

Change-Id: Ibe8b59e7b5bd5f357a655d8b4c5f0b0f58a67d6b
2024-04-04 21:43:19 +00:00
Qyriad 9166babbaf Revert "meson: move nix3 manpage generation into command-ref/"
This reverts commit 7095423374.

This seems to have broken running ninja on warm build directories, which
is not what we want. Reverted until we figure out something better

Change-Id: I9623ae078917e7c59a930bf8044a216501d4bb20
2024-04-04 11:59:08 -06:00
Rebecca Turner b44514819b Merge "Add EscapeStringOptions and escapeString tests" into main 2024-04-04 17:41:13 +00:00
Qyriad 3f4c7a68f4 meson: run repl characterization test
This was mistakenly left out of 69c3363f2.

Fixes #205.

Change-Id: Ib9a2088c9eee3a192623bee107ba32221d048640
2024-04-03 17:05:30 -06:00
Qyriad c263554ec6 Merge "meson: move nix3 manpage generation into command-ref/" into main 2024-04-03 21:34:27 +00:00
Qyriad c96c5db030 meson: move nix3 manpage generation into command-ref/
This puts the generated files where they are for the make system.
This is in preparation for further meson-mdbook stuff.

Change-Id: I934df6854a80af5ccf381cf1da0bda0187a8bcfc
2024-04-03 13:52:56 -06:00
Lunaphied 7a1054fa5f repl: improve :doc builtin repl command to support lambdas.
For a long time `nix repl` has supported displaying documentation set on
builtins, however, it has long been convention to use Markdown comments
on Nix functions themselves for documentation. This exposes that
information to `nix repl` users in a nice and formatted way.

NixOS/rfcs#145 doc-comments are primarily what this feature is intended
to consume, however, support for lambda documentation in the repl is
experimental. We do our best effort to support the RFC here.

These changes are based on [the nix-doc library](https://github.com/lf-/nix-doc) and
are licensed under the terms described in the relevant source files.

Change-Id: Ic6fe947d39a22540705d890737e336c4720b0a22
2024-04-03 13:47:22 -06:00
Rebecca Turner 84018e1264 Make repl_characterization.cc constants string_views
Change-Id: I96455ee365799ae87d9ab433cf102c691616a45e
2024-04-03 16:25:54 +00:00
Rebecca Turner ea10fe7ab0 Add EscapeStringOptions and escapeString tests
Change-Id: I86ead2f969c9e03c9edfa51bbc92ee06393fd7d6
2024-04-03 16:25:54 +00:00
Rebecca Turner 56c7dfd652 Merge "Add treefmt pre-commit hook" into main 2024-04-03 00:09:09 +00:00
Qyriad fa4a0fc6a4 Merge "meson: implement perl bindings" into main 2024-04-02 19:08:24 +00:00
Qyriad 4f5e532b8b meson: implement perl bindings
Change-Id: Ie1bfb0aa784e6136a82d518a652d0ae60c4b047a
2024-04-02 11:22:42 -06:00
Rebecca Turner 71ef7fd428 Merge "Use upstream cachix/git-hooks.nix for pre-commit" into main 2024-04-02 17:20:33 +00:00
eldritch horrors d2aca15f31 doc: remove the -- unescaping hack
lowdown has an option to disable this behavior. use it.

Change-Id: I06d4201b7f528021ddb8e94ac6baff385402e080
2024-04-01 20:15:28 +00:00
eldritch horrors ed2bb919e7 meson: add switch to disable docs
the make build system can do this too.

Change-Id: I8c07d159cab54a8749c50dc33615f60bd251a86d
2024-04-01 20:15:28 +00:00
Rebecca Turner ca9acf8184 Use upstream cachix/git-hooks.nix for pre-commit
Change-Id: I6bbc7d6da9accd7d2daffa9d780384df7226670e
2024-04-01 10:38:50 -07:00
Rebecca Turner 655dae03de Add treefmt pre-commit hook
This lets us use different formatters for different filetypes.

Change-Id: Ib52383dd5097c8919a65e299aca2b5a55412223c
2024-03-29 22:57:43 -07:00
288 changed files with 6052 additions and 1905 deletions

8
.gitignore vendored
View file

@ -21,15 +21,9 @@ perl/Makefile.config
/doc/manual/conf-file.json
/doc/manual/language.json
/doc/manual/xp-features.json
/doc/manual/src/SUMMARY.md
/doc/manual/src/SUMMARY-rl-next.md
/doc/manual/src/command-ref/new-cli
/doc/manual/src/command-ref/conf-file.md
/doc/manual/src/command-ref/experimental-features-shortlist.md
/doc/manual/src/contributing/experimental-feature-descriptions.md
/doc/manual/src/language/builtins.md
/doc/manual/src/language/builtin-constants.md
/doc/manual/src/release-notes/rl-next.md
/doc/manual/src/release-notes/rl-next-generated.md
# /scripts/
/scripts/nix-profile.sh

View file

@ -19,6 +19,7 @@ LIBBROTLI_LIBS = @LIBBROTLI_LIBS@
LIBCURL_LIBS = @LIBCURL_LIBS@
LIBSECCOMP_LIBS = @LIBSECCOMP_LIBS@
LOWDOWN_LIBS = @LOWDOWN_LIBS@
NIXDOC_LIBS = -llix_doc
OPENSSL_LIBS = @OPENSSL_LIBS@
PACKAGE_NAME = @PACKAGE_NAME@
PACKAGE_VERSION = @PACKAGE_VERSION@

3
bench/.gitignore vendored Normal file
View file

@ -0,0 +1,3 @@
bench-*.json
bench-*.md
nixpkgs

91
bench/README.md Normal file
View file

@ -0,0 +1,91 @@
# Benchmarking scripts for Lix
These are very much WIP, and have a few clumsy assumptions that we would
somewhat rather be fixed, but we have committed them to let others be able to
do benchmarking in the mean time.
## Benchmarking procedure
Build some Lixes you want to compare, by whichever means you wish.
Get a computer that is not busy and *strongly preferably* is bare-metal or at
least not a cloud VM (e.g. go make coffee when running benchmarks).
From the root of a Lix checkout, run `./bench/bench.sh resultlink-one
resultlink-two`, where `resultlink-one` and `resultlink-two` are the result
links from the builds you want to test (they can be any directory with bin/nix
in it, however).
To get the summary again, run `./bench/summarize.jq bench/bench-*.json`.
## Example results
(vim tip: `:r !bench/summarize.jq bench/bench-*.json` to dump it directly into
your editor)
```
result-asserts/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6
c6fe19ccb59cac54b5b3f25e160870 hello
mean: 15.993s ± 0.081s
user: 13.321s | system: 1.865s
median: 15.994s
range: 15.829s ... 16.096s
relative: 1
result/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19cc
b59cac54b5b3f25e160870 hello
mean: 15.897s ± 0.075s
user: 13.248s | system: 1.843s
median: 15.88s
range: 15.807s ... 16.047s
relative: 0.994
---
result/bin/nix --extra-experimental-features 'nix-command flakes' eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
mean: 0.4s ± 0.024s
user: 0.335s | system: 0.046s
median: 0.386s
range: 0.379s ... 0.43s
relative: 1
result-asserts/bin/nix --extra-experimental-features 'nix-command flakes' eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
mean: 0.404s ± 0.024s
user: 0.338s | system: 0.046s
median: 0.386s
range: 0.384s ... 0.436s
relative: 1.008
---
result-asserts/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
mean: 5.838s ± 0.023s
user: 5.083s | system: 0.464s
median: 5.845s
range: 5.799s ... 5.867s
relative: 1
result/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
mean: 5.788s ± 0.044s
user: 5.056s | system: 0.439s
median: 5.79s
range: 5.715s ... 5.876s
relative: 0.991
---
GC_INITIAL_HEAP_SIZE=10g result-asserts/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
mean: 4.147s ± 0.021s
user: 3.457s | system: 0.487s
median: 4.147s
range: 4.123s ... 4.195s
relative: 1
GC_INITIAL_HEAP_SIZE=10g result/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
mean: 4.149s ± 0.027s
user: 3.483s | system: 0.456s
median: 4.142s
range: 4.126s ... 4.215s
relative: 1
---
```

62
bench/bench.sh Executable file
View file

@ -0,0 +1,62 @@
#!/usr/bin/env bash
set -euo pipefail
shopt -s inherit_errexit
scriptdir=$(cd "$(dirname -- "$0")" ; pwd -P)
cd "$scriptdir/.."
if [[ $# -lt 2 ]]; then
# FIXME(jade): it is a reasonable use case to want to run a benchmark run
# on just one build. However, since we are using hyperfine in comparison
# mode, we would have to combine the JSON ourselves to support that, which
# would probably be better done by writing a benchmarking script in
# not-bash.
echo "Fewer than two result dirs given, nothing to compare!" >&2
echo "Pass some directories (with names indicating which alternative they are) with bin/nix in them" >&2
echo "Usage: ./bench/bench.sh result-1 result-2 [result-3...]" >&2
exit 1
fi
_exit=""
trap "$_exit" EXIT
# XXX: yes this is very silly. flakes~!!
nix build --impure --expr '(builtins.getFlake "git+file:.").inputs.nixpkgs.outPath' -o bench/nixpkgs
export NIX_REMOTE="$(mktemp -d)"
_exit='rm -rfv "$NIX_REMOTE"; $_exit'
export NIX_PATH="nixpkgs=bench/nixpkgs:nixos-config=bench/configuration.nix"
builds=("$@")
flake_args="--extra-experimental-features 'nix-command flakes'"
hyperfineArgs=(
--parameter-list BUILD "$(IFS=,; echo "${builds[*]}")"
--warmup 2 --runs 10
)
declare -A cases
cases=(
[search]="{BUILD}/bin/nix $flake_args search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello"
[rebuild]="{BUILD}/bin/nix $flake_args eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'"
[rebuild-lh]="GC_INITIAL_HEAP_SIZE=10g {BUILD}/bin/nix eval $flake_args --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'"
[parse]="{BUILD}/bin/nix $flake_args eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix"
)
benches=(
rebuild
rebuild-lh
search
parse
)
for k in "${benches[@]}"; do
taskset -c 2,3 \
chrt -f 50 \
hyperfine "${hyperfineArgs[@]}" --export-json="bench/bench-${k}.json" --export-markdown="bench/bench-${k}.md" "${cases[$k]}"
done
echo "Benchmarks summary (from ./bench/summarize.jq bench/bench-*.json)"
bench/summarize.jq bench/*.json

325
bench/configuration.nix Normal file
View file

@ -0,0 +1,325 @@
{
config,
pkgs,
lib,
...
}:
{
boot = {
initrd = {
availableKernelModules = [
"xhci_pci"
"ahci"
];
kernelModules = [ "dm-snapshot" ];
luks.devices = {
croot = {
device = "/dev/sdb";
allowDiscards = true;
};
};
};
kernelModules = [ "kvm-intel" ];
kernelPackages = pkgs.linuxPackages_latest;
loader = {
systemd-boot.enable = true;
efi.canTouchEfiVariables = true;
};
};
hardware = {
enableRedistributableFirmware = true;
cpu.intel.updateMicrocode = true;
opengl.driSupport32Bit = true;
opengl.extraPackages = with pkgs; [
vaapiIntel
intel-media-driver
intel-compute-runtime
];
};
fileSystems = {
"/" = {
device = "/dev/sda2";
fsType = "xfs";
options = [ "noatime" ];
};
"/boot" = {
device = "/dev/sda1";
fsType = "vfat";
};
"/nas" = {
device = "nas:/";
fsType = "nfs4";
options = [
"ro"
"x-systemd.automount"
];
};
};
swapDevices = [ { device = "/dev/swap"; } ];
networking = {
useDHCP = false;
hostName = "host";
wireless = {
enable = true;
interfaces = [ "eth1" ];
};
interfaces = {
eth0.useDHCP = true;
eth1.useDHCP = true;
};
wg-quick.interfaces = {
wg0 = {
address = [ "2001:db8::1" ];
privateKeyFile = "/etc/secrets/wg0.key";
peers = [
{
publicKey = "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";
endpoint = "[2001:db8::2]:61021";
allowedIPs = [ "2001::db8:1::/64" ];
}
];
};
};
firewall.allowedUDPPorts = [ 4567 ];
};
i18n = {
defaultLocale = "en_US.UTF-8";
inputMethod.enabled = "ibus";
};
services = {
xserver = {
enable = true;
layout = "us";
xkbVariant = "altgr-intl";
xkbOptions = "ctrl:nocaps";
libinput.enable = true;
wacom.enable = true;
videoDrivers = [ "modesetting" ];
modules = [ pkgs.xf86_input_wacom ];
displayManager.sx.enable = true;
windowManager.i3.enable = true;
};
udev.extraHwdb = ''
# not like this mattered at all
# we're not running udev from here
'';
udev.extraRules = ''
# ACTION=="add", SUBSYSTEM=="input", ...
'';
};
sound.enable = true;
hardware.pulseaudio = {
enable = true;
package = pkgs.pulseaudioFull;
daemon.config = {
lock-memory = "yes";
realtime-scheduling = "yes";
rlimit-rtprio = "-1";
};
};
programs = {
light.enable = true;
wireshark = {
enable = true;
package = pkgs.wireshark-qt;
};
gnupg.agent = {
enable = true;
};
};
fonts.packages = with pkgs; [
font-awesome
noto-fonts
noto-fonts-cjk
noto-fonts-emoji
noto-fonts-extra
dejavu_fonts
powerline-fonts
source-code-pro
cantarell-fonts
];
users = {
mutableUsers = false;
users = {
user = {
isNormalUser = true;
group = "user";
extraGroups = [
"wheel"
"video"
"audio"
"dialout"
"users"
"kvm"
"wireshark"
];
password = "unimportant";
};
};
groups = {
user = { };
};
};
security = {
pam.loginLimits = [
{
domain = "@audio";
item = "memlock";
type = "-";
value = "unlimited";
}
{
domain = "@audio";
item = "rtprio";
type = "-";
value = "99";
}
{
domain = "@audio";
item = "nofile";
type = "soft";
value = "99999";
}
{
domain = "@audio";
item = "nofile";
type = "hard";
value = "99999";
}
];
sudo.extraRules = [
{
users = [ "user" ];
commands = [
{
command = "${pkgs.linuxPackages.cpupower}/bin/cpupower";
options = [ "NOPASSWD" ];
}
];
}
];
};
environment.systemPackages = with pkgs; [
a2jmidid
age
ardour
bemenu
blender
breeze-icons
breeze-qt5
bubblewrap
calf
claws-mail
darktable
duperemove
emacs
feh
file
firefox
fluidsynth
gnome3.adwaita-icon-theme
gnuplot
graphviz
helm
i3status-rust
inkscape
jack2
jq
krita
ldns
libqalculate
libreoffice
man-pages
nheko
nix-diff
nix-index
nix-output-monitor
open-music-kontrollers.patchmatrix
pamixer
pavucontrol
pciutils
picom
pwgen
redshift
ripgrep
rlwrap
silver-searcher
soundfont-fluid
whois
wol
xclip
xdot
xdotool
xorg.xkbcomp
yt-dlp
zathura
borgbackup
linuxPackages.cpupower
mtr
kitty
xf86_input_wacom
];
environment.pathsToLink = [ "/share/soundfonts" ];
systemd.user.services.run-python = {
after = [ "network-online.target" ];
script = ''
exec ${pkgs.python3}/bin/python
'';
serviceConfig = {
CapabilityBoundingSet = [ "" ];
KeyringMode = "private";
LockPersonality = true;
MemoryDenyWriteExecute = true;
NoNewPrivileges = true;
PrivateDevices = true;
PrivateTmp = true;
PrivateUsers = true;
ProcSubset = "pid";
ProtectClock = true;
ProtectControlGroups = true;
ProtectHome = true;
ProtectHostname = true;
ProtectKernelLogs = true;
ProtectKernelModules = true;
ProtectKernelTunables = true;
ProtectProc = "invisible";
ProtectSystem = "strict";
RestrictAddressFamilies = "AF_INET AF_INET6";
RestrictNamespaces = true;
RestrictRealtime = true;
RestrictSUIDSGID = true;
SystemCallArchitectures = "native";
SystemCallFilter = [
"@system-service"
"~ @resources @privileged"
];
UMask = "077";
};
};
system.stateVersion = "23.11";
}

22
bench/summarize.jq Executable file
View file

@ -0,0 +1,22 @@
#!/usr/bin/env -S jq -Mrf
def round3:
. * 1000 | round | . / 1000
;
def stats($first):
[
" mean: \(.mean | round3)s ± \(.stddev | round3)s",
" user: \(.user | round3)s | system: \(.system | round3)s",
" median: \(.median | round3)s",
" range: \(.min | round3)s ... \(.max | round3)s",
" relative: \(.mean / $first.mean | round3)"
]
| join("\n")
;
def fmt($first):
"\(.command)\n" + (. | stats($first))
;
[.results | .[0] as $first | .[] | fmt($first)] | join("\n\n") | (. + "\n\n---\n")

4
clang-tidy/.editorconfig Normal file
View file

@ -0,0 +1,4 @@
# LLVM style code is 2-space indented
[*.{cc,hh}]
indent_style = space
indent_size = 2

81
clang-tidy/FixIncludes.cc Normal file
View file

@ -0,0 +1,81 @@
#include "FixIncludes.hh"
#include <clang-tidy/ClangTidyCheck.h>
#include <clang/Basic/Diagnostic.h>
#include <clang/Basic/SourceManager.h>
#include <clang/Lex/PPCallbacks.h>
#include <clang/Lex/Preprocessor.h>
#include <llvm/ADT/StringRef.h>
#include <llvm/Support/Debug.h>
#include <memory>
#include <set>
#include <string>
namespace nix::clang_tidy {
using namespace clang;
using namespace clang::tidy;
class FixIncludesCallbacks : public PPCallbacks {
public:
ClangTidyCheck &Check;
Preprocessor &PP;
FixIncludesCallbacks(ClangTidyCheck &Check, Preprocessor &PP)
: Check(Check), PP(PP) {}
private:
bool Ignore = false;
virtual void LexedFileChanged(FileID FID, LexedFileChangeReason Reason,
SrcMgr::CharacteristicKind FileType,
FileID PrevFID, SourceLocation Loc) override;
virtual void InclusionDirective(SourceLocation HashLoc,
const Token &IncludeTok, StringRef FileName,
bool IsAngled, CharSourceRange FilenameRange,
OptionalFileEntryRef File,
StringRef SearchPath, StringRef RelativePath,
const Module *Imported,
SrcMgr::CharacteristicKind FileType) override;
};
void FixIncludesCallbacks::LexedFileChanged(FileID, LexedFileChangeReason,
SrcMgr::CharacteristicKind FileType,
FileID, SourceLocation) {
Ignore = FileType != SrcMgr::C_User;
}
void FixIncludesCallbacks::InclusionDirective(
SourceLocation, const Token &, StringRef, bool,
CharSourceRange FilenameRange, OptionalFileEntryRef File, StringRef,
StringRef, const Module *, SrcMgr::CharacteristicKind) {
if (Ignore)
return;
// FIXME: this is kinda evil, but this is a one-time fixup
const std::string SourceDir = "src/";
if (File && File->getNameAsRequested().contains(SourceDir)) {
StringRef Name = File->getNameAsRequested();
auto Idx = Name.find(SourceDir);
assert(Idx != std::string::npos);
StringRef Suffix = Name.drop_front(Idx + SourceDir.length());
if (!Suffix.starts_with("lib")) {
llvm::dbgs() << "ignored: " << Suffix << "\n";
return;
}
auto Diag = Check.diag(FilenameRange.getBegin(),
"include needs to specify the source subdir");
Diag << FilenameRange
<< FixItHint::CreateReplacement(FilenameRange,
("\"" + Suffix + "\"").str());
}
}
void FixIncludesCheck::registerPPCallbacks(const SourceManager &,
Preprocessor *PP, Preprocessor *) {
PP->addPPCallbacks(std::make_unique<FixIncludesCallbacks>(*this, *PP));
}
}; // namespace nix::clang_tidy

21
clang-tidy/FixIncludes.hh Normal file
View file

@ -0,0 +1,21 @@
#pragma once
///@file
#include <clang-tidy/ClangTidyCheck.h>
#include <clang/ASTMatchers/ASTMatchFinder.h>
#include <llvm/ADT/StringRef.h>
namespace nix::clang_tidy {
using namespace clang;
using namespace clang::tidy;
class FixIncludesCheck : public ClangTidyCheck {
public:
FixIncludesCheck(StringRef Name, ClangTidyContext *Context)
: ClangTidyCheck(Name, Context) {}
void registerPPCallbacks(const SourceManager &SM, Preprocessor *PP, Preprocessor *ModuleExpanderPP) override;
};
};

View file

@ -1,5 +1,6 @@
#include <clang-tidy/ClangTidyModule.h>
#include <clang-tidy/ClangTidyModuleRegistry.h>
#include "FixIncludes.hh"
#include "HasPrefixSuffix.hh"
namespace nix::clang_tidy {
@ -9,9 +10,10 @@ using namespace clang::tidy;
class NixClangTidyChecks : public ClangTidyModule {
public:
void addCheckFactories(ClangTidyCheckFactories &CheckFactories) override {
CheckFactories.registerCheck<HasPrefixSuffixCheck>("nix-hasprefixsuffix");
CheckFactories.registerCheck<HasPrefixSuffixCheck>("lix-hasprefixsuffix");
CheckFactories.registerCheck<FixIncludesCheck>("lix-fixincludes");
}
};
static ClangTidyModuleRegistry::Add<NixClangTidyChecks> X("nix-module", "Adds nix specific checks");
static ClangTidyModuleRegistry::Add<NixClangTidyChecks> X("lix-module", "Adds lix specific checks");
};

View file

@ -1,8 +1,13 @@
project('nix-clang-tidy', ['cpp', 'c'],
project('lix-clang-tidy', ['cpp', 'c'],
version : '0.1',
default_options : ['warning_level=3', 'cpp_std=c++20'])
llvm = dependency('Clang', version: '>= 14', modules: ['libclang'])
sources = ['HasPrefixSuffix.cc', 'NixClangTidyChecks.cc']
shared_module('nix-clang-tidy', sources,
sources = files(
'HasPrefixSuffix.cc',
'LixClangTidyChecks.cc',
'FixIncludes.cc',
)
shared_module('lix-clang-tidy', sources,
dependencies: llvm)

View file

@ -1,10 +1,9 @@
(import
(
let lock = builtins.fromJSON (builtins.readFile ./flake.lock); in
fetchTarball {
url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
sha256 = lock.nodes.flake-compat.locked.narHash;
}
)
{ src = ./.; }
).defaultNix
(import (
let
lock = builtins.fromJSON (builtins.readFile ./flake.lock);
in
fetchTarball {
url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
sha256 = lock.nodes.flake-compat.locked.narHash;
}
) { src = ./.; }).defaultNix

View file

@ -14,6 +14,8 @@ PROJECT_NAME = "Nix"
PROJECT_NUMBER = @PACKAGE_VERSION@
OUTPUT_DIRECTORY = @docdir@
# Using the PROJECT_BRIEF tag one can provide an optional one line description
# for a project that appears at the top of each page and should give viewer a
# quick idea about the purpose of the project. Keep the description short.

View file

@ -0,0 +1,33 @@
doxygen_cfg = configure_file(
input : 'doxygen.cfg.in',
output : 'doxygen.cfg',
configuration : {
'PACKAGE_VERSION': meson.project_version(),
'RAPIDCHECK_HEADERS': rapidcheck_meson.get_variable('includedir'),
'docdir' : meson.current_build_dir(),
},
)
internal_api_docs = custom_target(
'internal-api-docs',
command : [
bash,
# Meson can you please just give us a `workdir` argument to custom targets...
'-c',
# We have to prefix the doxygen_cfg path with the project build root
# because of the cd in front.
'cd @0@ && @1@ @2@/@INPUT0@'.format(
meson.project_source_root(),
doxygen.full_path(),
meson.project_build_root(),
),
],
input : [
doxygen_cfg,
],
output : 'html',
install : true,
install_dir : datadir / 'doc/nix/internal-api',
)
alias_target('internal-api-html', internal_api_docs)

View file

@ -7,10 +7,22 @@ additional-js = ["redirects.js"]
edit-url-template = "https://github.com/NixOS/nix/tree/master/doc/manual/{path}"
git-repository-url = "https://github.com/NixOS/nix"
# Handles replacing @docroot@ with a path to ./src relative to that markdown file,
# {{#include handlebars}}, and the @generated@ syntax used within these. it mostly
# but not entirely replaces the links preprocessor (which we cannot simply use due
# to @generated@ files living in a different directory to make meson happy). we do
# not want to disable the links preprocessor entirely though because that requires
# disabling *all* built-in preprocessors and selectively reenabling those we want.
[preprocessor.substitute]
command = "python3 doc/manual/substitute.py"
before = ["anchors", "links"]
[preprocessor.anchors]
renderers = ["html"]
command = "jq --from-file doc/manual/anchors.jq"
[output.markdown]
[output.linkcheck]
# no Internet during the build (in the sandbox)
follow-web-links = false

View file

@ -5,7 +5,13 @@ in
builtinsInfo:
let
showBuiltin = name: { doc, type, impure-only }:
showBuiltin =
name:
{
doc,
type,
impure-only,
}:
let
type' = optionalString (type != null) " (${type})";

View file

@ -5,7 +5,14 @@ in
builtinsInfo:
let
showBuiltin = name: { doc, args, arity, experimental-feature }:
showBuiltin =
name:
{
doc,
args,
arity,
experimental-feature,
}:
let
experimentalNotice = optionalString (experimental-feature != null) ''
This function is only available if the [${experimental-feature}](@docroot@/contributing/experimental-features.md#xp-feature-${experimental-feature}) experimental feature is enabled.

22
doc/manual/generate-deps.py Executable file
View file

@ -0,0 +1,22 @@
#!/usr/bin/env python3
import glob
import sys
# meson expects makefile-style dependency declarations, i.e.
#
# target: dependency...
#
# meson seems to pass depfiles straight on to ninja even though
# it also parses the file itself (or at least has code to do so
# in its tree), so we must live by ninja's rules: only slashes,
# spaces and octothorpes can be escaped, anything else is taken
# literally. since the rules for these aren't even the same for
# all three we will just fail when we encounter any of them (if
# asserts are off for some reason the depfile will likely point
# to nonexistant paths, making everything phony and thus fine.)
for path in glob.glob(sys.argv[1] + '/**', recursive=True):
assert '\\' not in path
assert ' ' not in path
assert '#' not in path
print("ignored:", path)

View file

@ -1,8 +1,26 @@
let
inherit (builtins)
attrNames attrValues fromJSON listToAttrs mapAttrs
concatStringsSep concatMap length lessThan replaceStrings sort;
inherit (import ./utils.nix) concatStrings optionalString filterAttrs trim squash unique showSettings;
attrNames
attrValues
fromJSON
listToAttrs
mapAttrs
concatStringsSep
concatMap
length
lessThan
replaceStrings
sort
;
inherit (import ./utils.nix)
concatStrings
optionalString
filterAttrs
trim
squash
unique
showSettings
;
in
inlineHTML: commandDump:
@ -11,7 +29,13 @@ let
commandInfo = fromJSON commandDump;
showCommand = { command, details, filename, toplevel }:
showCommand =
{
command,
details,
filename,
toplevel,
}:
let
result = ''
@ -35,26 +59,27 @@ let
${maybeOptions}
'';
showSynopsis = command: args:
showSynopsis =
command: args:
let
showArgument = arg: "*${arg.label}*" + optionalString (! arg ? arity) "...";
showArgument = arg: "*${arg.label}*" + optionalString (!arg ? arity) "...";
arguments = concatStringsSep " " (map showArgument args);
in ''
`${command}` [*option*...] ${arguments}
in
''
`${command}` [*option*...] ${arguments}
'';
maybeSubcommands = optionalString (details ? commands && details.commands != {})
''
where *subcommand* is one of the following:
maybeSubcommands = optionalString (details ? commands && details.commands != { }) ''
where *subcommand* is one of the following:
${subcommands}
'';
${subcommands}
'';
subcommands = if length categories > 1
then listCategories
else listSubcommands details.commands;
subcommands = if length categories > 1 then listCategories else listSubcommands details.commands;
categories = sort (x: y: x.id < y.id) (unique (map (cmd: cmd.category) (attrValues details.commands)));
categories = sort (x: y: x.id < y.id) (
unique (map (cmd: cmd.category) (attrValues details.commands))
);
listCategories = concatStrings (map showCategory categories);
@ -71,11 +96,11 @@ let
'';
# TODO: move this confusing special case out of here when implementing #8496
maybeStoreDocs = optionalString
(details ? doc)
(replaceStrings ["@stores@"] [storeDocs] details.doc);
maybeStoreDocs = optionalString (details ? doc) (
replaceStrings [ "@stores@" ] [ storeDocs ] details.doc
);
maybeOptions = optionalString (details.flags != {}) ''
maybeOptions = optionalString (details.flags != { }) ''
# Options
${showOptions details.flags toplevel.flags}
@ -85,51 +110,70 @@ let
> See [`man nix.conf`](@docroot@/command-ref/conf-file.md#command-line-flags) for overriding configuration settings with command line flags.
'';
showOptions = options: commonOptions:
showOptions =
options: commonOptions:
let
allOptions = options // commonOptions;
showCategory = cat: ''
${optionalString (cat != "") "**${cat}:**"}
${listOptions (filterAttrs (n: v: v.category == cat) allOptions)}
'';
'';
listOptions = opts: concatStringsSep "\n" (attrValues (mapAttrs showOption opts));
showOption = name: option:
showOption =
name: option:
let
result = trim ''
- ${item}
${option.description}
'';
item = if inlineHTML
then ''<span id="opt-${name}">[`--${name}`](#opt-${name})</span> ${shortName} ${labels}''
else "`--${name}` ${shortName} ${labels}";
shortName = optionalString
(option ? shortName)
("/ `-${option.shortName}`");
labels = optionalString
(option ? labels)
(concatStringsSep " " (map (s: "*${s}*") option.labels));
in result;
item =
if inlineHTML then
''<span id="opt-${name}">[`--${name}`](#opt-${name})</span> ${shortName} ${labels}''
else
"`--${name}` ${shortName} ${labels}";
shortName = optionalString (option ? shortName) ("/ `-${option.shortName}`");
labels = optionalString (option ? labels) (concatStringsSep " " (map (s: "*${s}*") option.labels));
in
result;
categories = sort lessThan (unique (map (cmd: cmd.category) (attrValues allOptions)));
in concatStrings (map showCategory categories);
in squash result;
in
concatStrings (map showCategory categories);
in
squash result;
appendName = filename: name: (if filename == "nix" then "nix3" else filename) + "-" + name;
processCommand = { command, details, filename, toplevel }:
processCommand =
{
command,
details,
filename,
toplevel,
}:
let
cmd = {
inherit command;
name = filename + ".md";
value = showCommand { inherit command details filename toplevel; };
value = showCommand {
inherit
command
details
filename
toplevel
;
};
};
subcommand = subCmd: processCommand {
command = command + " " + subCmd;
details = details.commands.${subCmd};
filename = appendName filename subCmd;
inherit toplevel;
};
in [ cmd ] ++ concatMap subcommand (attrNames details.commands or {});
subcommand =
subCmd:
processCommand {
command = command + " " + subCmd;
details = details.commands.${subCmd};
filename = appendName filename subCmd;
inherit toplevel;
};
in
[ cmd ] ++ concatMap subcommand (attrNames details.commands or { });
manpages = processCommand {
command = "nix";
@ -138,14 +182,15 @@ let
toplevel = commandInfo.args;
};
tableOfContents = let
showEntry = page:
" - [${page.command}](command-ref/new-cli/${page.name})";
in concatStringsSep "\n" (map showEntry manpages) + "\n";
storeDocs =
let
showStore = name: { settings, doc, experimentalFeature }:
showStore =
name:
{
settings,
doc,
experimentalFeature,
}:
let
experimentalFeatureNote = optionalString (experimentalFeature != null) ''
> **Warning**
@ -161,7 +206,8 @@ let
extra-experimental-features = ${experimentalFeature}
```
'';
in ''
in
''
## ${name}
${doc}
@ -172,6 +218,7 @@ let
${showSettings { inherit inlineHTML; } settings}
'';
in concatStrings (attrValues (mapAttrs showStore commandInfo.stores));
in (listToAttrs manpages) // { "SUMMARY.md" = tableOfContents; }
in
concatStrings (attrValues (mapAttrs showStore commandInfo.stores));
in
listToAttrs manpages

View file

@ -2,8 +2,8 @@ with builtins;
with import ./utils.nix;
let
showExperimentalFeature = name: doc:
''
- [`${name}`](@docroot@/contributing/experimental-features.md#xp-feature-${name})
'';
in xps: indent " " (concatStrings (attrValues (mapAttrs showExperimentalFeature xps)))
showExperimentalFeature = name: doc: ''
- [`${name}`](@docroot@/contributing/experimental-features.md#xp-feature-${name})
'';
in
xps: indent " " (concatStrings (attrValues (mapAttrs showExperimentalFeature xps)))

View file

@ -2,10 +2,12 @@ with builtins;
with import ./utils.nix;
let
showExperimentalFeature = name: doc:
showExperimentalFeature =
name: doc:
squash ''
## [`${name}`]{#xp-feature-${name}}
${doc}
'';
in xps: (concatStringsSep "\n" (attrValues (mapAttrs showExperimentalFeature xps)))
in
xps: (concatStringsSep "\n" (attrValues (mapAttrs showExperimentalFeature xps)))

61
doc/manual/json-to-tree.py Executable file
View file

@ -0,0 +1,61 @@
#!/usr/bin/env python3
"""
This script is a helper for this project's Meson buildsystem, to replace its
usage of `nix eval --write-to`. Writing a JSON object as a nested directory
tree is more generic, easier to maintain, and far, far less cursed. Nix
has 'good' support for JSON output. Let's just use it.
"""
import argparse
from pathlib import Path
import json
import sys
name = 'json-to-tree.py'
def log(*args, **kwargs):
kwargs['file'] = sys.stderr
return print(f'{name}:', *args, **kwargs)
def write_dict_to_directory(current_directory: Path, data: dict, files_written=0):
current_directory.mkdir(parents=True, exist_ok=True)
for key, value in data.items():
nested_path = current_directory / key
match value:
case dict(nested_data):
files_written += write_dict_to_directory(nested_path, nested_data)
case str(content):
nested_path.write_text(content)
files_written += 1
case rest:
assert False, \
f'should have been called on a dict or string, not {type(rest)=}\n\t{rest=}'
return files_written
def main():
parser = argparse.ArgumentParser(name)
parser.add_argument('-i', '--input', type=argparse.FileType('r'), default='-',
help='The JSON input to operate on and output as a directory tree',
)
parser.add_argument('-o', '--output', type=Path, required=True,
help='The place to put the directory tree',
)
args = parser.parse_args()
json_string = args.input.read()
try:
data = json.loads(json_string)
except json.JSONDecodeError:
log(f'could not decode JSON from input: {json_string}')
raise
files_written = write_dict_to_directory(args.output, data)
log(f'wrote {files_written} files')
sys.exit(main())

View file

@ -17,14 +17,14 @@ man-pages := $(foreach n, \
nix-hash.1 nix-copy-closure.1 \
nix.conf.5 nix-daemon.8 \
nix-profiles.5 \
, $(d)/$(n))
, doc/manual/generated/in/$(n))
# man pages for subcommands
# convert from `$(d)/src/command-ref/nix-{1}/{2}.md` to `$(d)/nix-{1}-{2}.1`
# FIXME: unify with how nix3-cli man pages are generated
man-pages += $(foreach subcommand, \
$(filter-out %opt-common.md %env-common.md, $(wildcard $(d)/src/command-ref/nix-*/*.md)), \
$(d)/$(subst /,-,$(subst $(d)/src/command-ref/,,$(subst .md,.1,$(subcommand)))))
doc/manual/generated/in/$(subst /,-,$(subst $(d)/src/command-ref/,,$(subst .md,.1,$(subcommand)))))
clean-files += $(d)/*.1 $(d)/*.5 $(d)/*.8
@ -39,81 +39,91 @@ dummy-env = env -i \
nix-eval = $(dummy-env) $(doc_nix) eval --experimental-features nix-command -I nix/corepkgs=corepkgs --store dummy:// --impure --raw
$(d)/nix-env-%.1: $(d)/src/command-ref/nix-env/%.md
doc/manual/generated/in/nix-env-%.1: doc/manual/generated/out
$(trace-gen) doc/manual/render-manpage.sh \
--unescape-dashes "$(subst nix-env-,nix-env --,$$(basename "$@" .1))" 1 $^ $^.tmp $@
--out-no-smarty "$(subst nix-env-,nix-env --,$$(basename "$@" .1))" 1 \
doc/manual/generated/out/markdown/command-ref/nix-env/$*.md \
$@
$(d)/nix-store-%.1: $(d)/src/command-ref/nix-store/%.md
doc/manual/generated/in/nix-store-%.1: doc/manual/generated/out
$(trace-gen) doc/manual/render-manpage.sh \
--unescape-dashes "$(subst nix-store-,nix-store --,$$(basename "$@" .1))" 1 $^ $^.tmp $@
--out-no-smarty "$(subst nix-store-,nix-store --,$$(basename "$@" .1))" 1 \
doc/manual/generated/out/markdown/command-ref/nix-store/$*.md \
$@
$(d)/%.1: $(d)/src/command-ref/%.md
$(trace-gen) doc/manual/render-manpage.sh "$$(basename $@ .1)" 1 $^ $^.tmp $@
doc/manual/generated/in/%.1: doc/manual/generated/out
$(trace-gen) doc/manual/render-manpage.sh "$$(basename $@ .1)" 1 \
doc/manual/generated/out/markdown/command-ref/$*.md \
$@
$(d)/%.8: $(d)/src/command-ref/%.md
$(trace-gen) doc/manual/render-manpage.sh "$$(basename $@ .8)" 8 $^ $^.tmp $@
doc/manual/generated/in/%.8: doc/manual/generated/out
$(trace-gen) doc/manual/render-manpage.sh "$$(basename $@ .8)" 8 \
doc/manual/generated/out/markdown/command-ref/$*.md \
$@
$(d)/nix.conf.5: $(d)/src/command-ref/conf-file.md
$(trace-gen) doc/manual/render-manpage.sh "$$(basename $@ .5)" 5 $^ $^.tmp $@
doc/manual/generated/in/nix.conf.5: doc/manual/generated/out
$(trace-gen) doc/manual/render-manpage.sh "$$(basename $@ .5)" 5 \
doc/manual/generated/out/markdown/command-ref/conf-file.md \
$@
$(d)/nix-profiles.5: $(d)/src/command-ref/files/profiles.md
$(trace-gen) doc/manual/render-manpage.sh "$$(basename $@ .5)" 5 $^ $^.tmp $@
doc/manual/generated/in/nix-profiles.5: doc/manual/generated/out
$(trace-gen) doc/manual/render-manpage.sh "$$(basename $@ .5)" 5 \
doc/manual/generated/out/markdown/command-ref/files/profiles.md \
$@
$(d)/src/SUMMARY.md: $(d)/src/SUMMARY.md.in $(d)/src/SUMMARY-rl-next.md $(d)/src/command-ref/new-cli $(d)/src/contributing/experimental-feature-descriptions.md
@cp $< $@
@doc/manual/process-includes.sh $@ $@
$(d)/src/command-ref/new-cli: $(d)/nix.json $(d)/utils.nix $(d)/generate-manpage.nix $(doc_nix)
doc/manual/generated/in/command-ref/new-cli: doc/manual/generated/in/nix.json $(d)/utils.nix $(d)/generate-manpage.nix $(doc_nix)
@mkdir -p doc/manual/generated/in/command-ref
@rm -rf $@ $@.tmp
$(trace-gen) $(nix-eval) --write-to $@.tmp --expr 'import doc/manual/generate-manpage.nix true (builtins.readFile $<)'
@mv $@.tmp $@
$(d)/src/command-ref/conf-file.md: $(d)/conf-file.json $(d)/utils.nix $(d)/src/command-ref/conf-file-prefix.md $(d)/src/command-ref/experimental-features-shortlist.md $(doc_nix)
@cat doc/manual/src/command-ref/conf-file-prefix.md > $@.tmp
$(trace-gen) $(nix-eval) --expr '(import doc/manual/utils.nix).showSettings { inlineHTML = true; } (builtins.fromJSON (builtins.readFile $<))' >> $@.tmp;
@mv $@.tmp $@
doc/manual/generated/in/command-ref/conf-file.md: doc/manual/generated/in/conf-file.json $(d)/utils.nix doc/manual/generated/in/command-ref/experimental-features-shortlist.md $(doc_nix)
@mkdir -p doc/manual/generated/in/command-ref
$(trace-gen) $(nix-eval) --expr '(import doc/manual/utils.nix).showSettings { inlineHTML = true; } (builtins.fromJSON (builtins.readFile $<))' >> $@
$(d)/nix.json: $(doc_nix)
doc/manual/generated/in/nix.json: $(doc_nix)
@mkdir -p doc/manual/generated/in
$(trace-gen) $(dummy-env) $(doc_nix) __dump-cli > $@.tmp
@mv $@.tmp $@
$(d)/conf-file.json: $(doc_nix)
doc/manual/generated/in/conf-file.json: $(doc_nix)
@mkdir -p doc/manual/generated/in
$(trace-gen) $(dummy-env) $(doc_nix) show-config --json --experimental-features nix-command > $@.tmp
@mv $@.tmp $@
$(d)/src/contributing/experimental-feature-descriptions.md: $(d)/xp-features.json $(d)/utils.nix $(d)/generate-xp-features.nix $(doc_nix)
doc/manual/generated/in/contributing/experimental-feature-descriptions.md: doc/manual/generated/in/xp-features.json $(d)/utils.nix $(d)/generate-xp-features.nix $(doc_nix)
@mkdir -p doc/manual/generated/in/contributing
@rm -rf $@ $@.tmp
$(trace-gen) $(nix-eval) --write-to $@.tmp --expr 'import doc/manual/generate-xp-features.nix (builtins.fromJSON (builtins.readFile $<))'
@mv $@.tmp $@
$(d)/src/command-ref/experimental-features-shortlist.md: $(d)/xp-features.json $(d)/utils.nix $(d)/generate-xp-features-shortlist.nix $(doc_nix)
doc/manual/generated/in/command-ref/experimental-features-shortlist.md: doc/manual/generated/in/xp-features.json $(d)/utils.nix $(d)/generate-xp-features-shortlist.nix $(doc_nix)
@mkdir -p doc/manual/generated/in/command-ref
@rm -rf $@ $@.tmp
$(trace-gen) $(nix-eval) --write-to $@.tmp --expr 'import doc/manual/generate-xp-features-shortlist.nix (builtins.fromJSON (builtins.readFile $<))'
@mv $@.tmp $@
$(d)/xp-features.json: $(doc_nix)
doc/manual/generated/in/xp-features.json: $(doc_nix)
$(trace-gen) $(dummy-env) NIX_PATH=nix/corepkgs=corepkgs $(doc_nix) __dump-xp-features > $@.tmp
@mv $@.tmp $@
$(d)/src/language/builtins.md: $(d)/language.json $(d)/generate-builtins.nix $(d)/src/language/builtins-prefix.md $(doc_nix)
@cat doc/manual/src/language/builtins-prefix.md > $@.tmp
$(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-builtins.nix (builtins.fromJSON (builtins.readFile $<)).builtins' >> $@.tmp;
@cat doc/manual/src/language/builtins-suffix.md >> $@.tmp
@mv $@.tmp $@
doc/manual/generated/in/language/builtins.md: doc/manual/generated/in/language.json $(d)/generate-builtins.nix $(doc_nix)
@mkdir -p doc/manual/generated/in/language
$(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-builtins.nix (builtins.fromJSON (builtins.readFile $<)).builtins' >> $@
$(d)/src/language/builtin-constants.md: $(d)/language.json $(d)/generate-builtin-constants.nix $(d)/src/language/builtin-constants-prefix.md $(doc_nix)
@cat doc/manual/src/language/builtin-constants-prefix.md > $@.tmp
$(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-builtin-constants.nix (builtins.fromJSON (builtins.readFile $<)).constants' >> $@.tmp;
@cat doc/manual/src/language/builtin-constants-suffix.md >> $@.tmp
@mv $@.tmp $@
doc/manual/generated/in/language/builtin-constants.md: doc/manual/generated/in/language.json $(d)/generate-builtin-constants.nix $(doc_nix)
@mkdir -p doc/manual/generated/in/language
$(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-builtin-constants.nix (builtins.fromJSON (builtins.readFile $<)).constants' >> $@
$(d)/language.json: $(doc_nix)
doc/manual/generated/in/language.json: $(doc_nix)
@mkdir -p doc/manual/generated/in
$(trace-gen) $(dummy-env) NIX_PATH=nix/corepkgs=corepkgs $(doc_nix) __dump-language > $@.tmp
@mv $@.tmp $@
# Generate "Upcoming release" notes (or clear it and remove from menu)
$(d)/src/release-notes/rl-next.md: $(d)/rl-next $(d)/rl-next/*
doc/manual/generated/in/release-notes/rl-next-generated.md: $(d)/rl-next $(d)/rl-next/*
@mkdir -p doc/manual/generated/in/release-notes
@if type -p build-release-notes > /dev/null; then \
echo " GEN " $@; \
build-release-notes doc/manual/rl-next > $@; \
@ -122,14 +132,6 @@ $(d)/src/release-notes/rl-next.md: $(d)/rl-next $(d)/rl-next/*
true > $@; \
fi
$(d)/src/SUMMARY-rl-next.md: $(d)/src/release-notes/rl-next.md
$(trace-gen) true
@if [ -s $< ]; then \
echo ' - [Upcoming release](release-notes/rl-next.md)' > $@; \
else \
true > $@; \
fi
# Generate the HTML manual.
.PHONY: manual-html
manual-html: $(docdir)/manual/index.html
@ -146,9 +148,9 @@ $(mandir)/man1/nix3-manpages: doc/manual/generated/man1/nix3-manpages
@mkdir -p $(DESTDIR)$$(dirname $@)
$(trace-install) install -m 0644 $$(dirname $<)/* $(DESTDIR)$$(dirname $@)
doc/manual/generated/man1/nix3-manpages: $(d)/src/command-ref/new-cli
doc/manual/generated/man1/nix3-manpages: doc/manual/generated/out
@mkdir -p $(DESTDIR)$$(dirname $@)
$(trace-gen) for i in doc/manual/src/command-ref/new-cli/*.md; do \
$(trace-gen) for i in doc/manual/generated/out/markdown/command-ref/new-cli/*.md; do \
name=$$(basename $$i .md); \
tmpFile=$$(mktemp); \
if [[ $$name = SUMMARY ]]; then continue; fi; \
@ -159,23 +161,18 @@ doc/manual/generated/man1/nix3-manpages: $(d)/src/command-ref/new-cli
done
@touch $@
$(docdir)/manual/index.html: $(MANUAL_SRCS) $(d)/book.toml $(d)/anchors.jq $(d)/custom.css $(d)/src/SUMMARY.md $(d)/src/command-ref/new-cli $(d)/src/contributing/experimental-feature-descriptions.md $(d)/src/command-ref/conf-file.md $(d)/src/language/builtins.md $(d)/src/language/builtin-constants.md $(d)/src/release-notes/rl-next.md
doc/manual/generated/out: $(MANUAL_SRCS) $(d)/book.toml $(d)/anchors.jq $(d)/custom.css $(d)/src/SUMMARY.md doc/manual/generated/in/command-ref/new-cli doc/manual/generated/in/command-ref/experimental-features-shortlist.md doc/manual/generated/in/contributing/experimental-feature-descriptions.md doc/manual/generated/in/command-ref/conf-file.md doc/manual/generated/in/language/builtins.md doc/manual/generated/in/language/builtin-constants.md doc/manual/generated/in/release-notes/rl-next-generated.md $(d)/substitute.py
@rm -rf $@
$(trace-gen) \
tmp="$$(mktemp -d)"; \
cp -r doc/manual "$$tmp"; \
find "$$tmp" -name '*.md' | while read -r file; do \
doc/manual/process-includes.sh $$file $$file; \
done; \
find "$$tmp" -name '*.md' | while read -r file; do \
docroot="$$(realpath --relative-to="$$(dirname "$$file")" $$tmp/manual/src)"; \
sed -i "s,@docroot@,$$docroot,g" "$$file"; \
done; \
set -euo pipefail; \
RUST_LOG=warn mdbook build "$$tmp/manual" -d $(DESTDIR)$(docdir)/manual.tmp 2>&1 \
| { grep -Fv "because fragment resolution isn't implemented" || :; }; \
rm -rf "$$tmp/manual"
MDBOOK_SUBSTITUTE_SEARCH=doc/manual/generated/in \
RUST_LOG=warn \
mdbook build doc/manual -d generated/out 2>&1 \
| { grep -Fv "because fragment resolution isn't implemented" || :; }
@find $@ -iname meson.build -delete
$(docdir)/manual/index.html: doc/manual/generated/out
@mkdir -p $(DESTDIR)$(docdir)
@rm -rf $(DESTDIR)$(docdir)/manual
@mv $(DESTDIR)$(docdir)/manual.tmp/html $(DESTDIR)$(docdir)/manual
@rm -rf $(DESTDIR)$(docdir)/manual.tmp
@cp -r $</html $(DESTDIR)$(docdir)/manual
endif

View file

@ -7,15 +7,15 @@ nix_env_for_docs = {
}
nix_for_docs = [ nix, '--experimental-features', 'nix-command' ]
nix_eval_for_docs = nix_for_docs + [
nix_eval_for_docs_common = nix_for_docs + [
'eval',
'-I', 'nix/corepkgs=corepkgs',
'--store', 'dummy://',
'--impure',
'--raw',
]
nix_eval_for_docs = nix_eval_for_docs_common + '--raw'
nix_conf_file_json = custom_target(
conf_file_json = custom_target(
command : nix_for_docs + [ 'show-config', '--json' ],
capture : true,
output : 'conf-file.json',
@ -30,7 +30,7 @@ nix_conf_file_md_body = custom_target(
capture : true,
input : [
'utils.nix',
nix_conf_file_json,
conf_file_json,
],
output : 'conf-file.md.body',
env : nix_env_for_docs,
@ -40,7 +40,7 @@ nix_conf_file_md = custom_target(
command : [ 'cat', '@INPUT@' ],
capture : true,
input : [
'src/command-ref/conf-file-prefix.md',
'src/command-ref/conf-file.md',
nix_conf_file_md_body,
],
output : 'conf-file.md',
@ -51,20 +51,84 @@ nix_exp_features_json = custom_target(
capture : true,
output : 'xp-features.json',
)
nix_exp_feature_shortlist = custom_target(
command : nix_eval_for_docs + [
'--expr',
'import @INPUT0@ (builtins.fromJSON (builtins.readFile @INPUT1@))',
],
input : [
'generate-xp-features-shortlist.nix',
nix_exp_features_json,
],
output : 'experimental-features-shortlist.md',
language_json = custom_target(
command: [nix, '__dump-language'],
output : 'language.json',
capture : true,
env : nix_env_for_docs,
)
nix3_cli_json = custom_target(
command : [ nix, '__dump-cli' ],
capture : true,
output : 'nix.json',
env : nix_env_for_docs,
)
generate_manual_deps = files(
'generate-deps.py',
)
# Generates builtins.md and builtin-constants.md.
subdir('src/language')
# Generates new-cli pages, experimental-features-shortlist.md, and conf-file.md.
subdir('src/command-ref')
# Generates experimental-feature-descriptions.md.
subdir('src/contributing')
# Generates rl-next-generated.md.
subdir('src/release-notes')
manual = custom_target(
'manual',
command : [
bash,
'-euo', 'pipefail',
'-c',
'''
@0@ @INPUT0@ @CURRENT_SOURCE_DIR@ > @DEPFILE@
cd @SOURCE_ROOT@
@1@ build doc/manual -d @2@ | { grep -Fv "because fragment resolution isn't implemented" || :; }
rm -rf @2@/manual
mv @2@/html @2@/manual
find @2@/manual -iname meson.build -delete
'''.format(
python.full_path(),
mdbook.full_path(),
meson.current_build_dir(),
),
],
input : [
generate_manual_deps,
'book.toml',
'anchors.jq',
'custom.css',
nix3_cli_files,
experimental_features_shortlist_md,
experimental_feature_descriptions_md,
conf_file_md,
builtins_md,
builtin_constants_md,
rl_next_generated,
],
output : [
'manual',
'markdown',
],
depfile : 'manual.d',
env : {
'RUST_LOG': 'info',
'MDBOOK_SUBSTITUTE_SEARCH': meson.current_build_dir() / 'src',
},
)
manual_html = manual[0]
manual_md = manual[1]
install_subdir(
manual_html.full_path(),
install_dir : datadir / 'doc/nix',
)
nix_nested_manpages = [
[ 'nix-env',
[
@ -109,17 +173,20 @@ nix_nested_manpages = [
foreach command : nix_nested_manpages
foreach page : command[1]
title = command[0] + ' --' + page
section = '1'
custom_target(
command : [
'./render-manpage.sh',
'--unescape-dashes',
command[0] + ' --' + page,
'1',
'@INPUT@',
'@OUTPUT@.tmp',
'@OUTPUT@',
'--out-no-smarty',
title,
section,
'@INPUT0@/command-ref' / command[0] / (page + '.md'),
'@OUTPUT0@',
],
input : [
manual_md,
],
input : 'src/command-ref' / command[0] / (page + '.md'),
output : command[0] + '-' + page + '.1',
install : true,
install_dir : mandir / 'man1',
@ -127,25 +194,6 @@ foreach command : nix_nested_manpages
endforeach
endforeach
nix3_cli_json = custom_target(
command : [ nix, '__dump-cli' ],
capture : true,
output : 'nix.json',
)
nix3_cli_files = custom_target(
command : nix_eval_for_docs + [
'--write-to', '@OUTPUT@',
'--expr',
'import @INPUT1@ true (builtins.readFile @INPUT0@)',
],
input : [
nix3_cli_json,
'generate-manpage.nix',
'utils.nix',
],
output : 'new-cli',
)
nix3_manpages = [
'nix3-build',
'nix3-bundle',
@ -236,16 +284,20 @@ nix3_manpages = [
]
foreach page : nix3_manpages
section = '1'
custom_target(
command : [
'./render-manpage.sh',
bash,
'@INPUT0@',
page,
'1',
'@INPUT0@/' + page + '.md',
'@OUTPUT@.tmp',
section,
'@INPUT1@/command-ref/new-cli/@0@.md'.format(page),
'@OUTPUT@',
],
input : nix3_cli_files,
input : [
'render-manpage.sh',
manual_md,
],
output : page + '.1',
install : true,
install_dir : mandir / 'man1',
@ -263,24 +315,30 @@ nix_manpages = [
[ 'nix-channel', 1 ],
[ 'nix-hash', 1 ],
[ 'nix-copy-closure', 1 ],
[ 'nix.conf', 5, nix_conf_file_md, nix_exp_feature_shortlist ],
[ 'nix.conf', 5, 'conf-file.md' ],
[ 'nix-daemon', 8 ],
[ 'nix-profiles', 5, 'src/command-ref/files/profiles.md', nix_exp_feature_shortlist ],
[ 'nix-profiles', 5, 'files/profiles.md' ],
]
foreach entry : nix_manpages
title = entry[0]
# nix.conf.5 and nix-profiles.5 are based off of conf-file.md and files/profiles.md,
# rather than a stem identical to its mdbook source.
# Therefore we use an optional third element of this array to override the name pattern
md_file = entry.get(2, title + '.md')
section = entry[1].to_string()
custom_target(
command : [
'./render-manpage.sh',
entry[0],
entry[1].to_string(),
bash,
'@INPUT0@',
'@OUTPUT@.tmp',
title,
section,
'@INPUT1@/command-ref/@0@'.format(md_file),
'@OUTPUT@',
meson.current_build_dir(),
],
input : [
entry.get(2, 'src/command-ref' / (entry[0] + '.md')),
'render-manpage.sh',
manual_md,
entry.get(3, []),
],
output : '@0@.@1@'.format(entry[0], entry[1]),

View file

@ -2,26 +2,24 @@
set -euo pipefail
unescape_dashes=
lowdown_args=
if [ "$1" = --unescape-dashes ]; then
unescape_dashes=yes
if [ "$1" = --out-no-smarty ]; then
lowdown_args=--out-no-smarty
shift
fi
[ "$#" = 4 ] || {
echo "wrong number of args passed" >&2
exit 1
}
title="$1"
section="$2"
infile="$3"
tmpfile="$4"
outfile="$5"
outfile="$4"
printf "Title: %s\n\n" "$title" > "$tmpfile"
cat "$infile" >> "$tmpfile"
"$(dirname "$0")"/process-includes.sh "$infile" "$tmpfile"
lowdown -sT man --nroff-nolinks -M section="$section" "$tmpfile" -o "$outfile"
if [ -n "$unescape_dashes" ]; then
# fix up `lowdown`'s automatic escaping of `--`
# https://github.com/kristapsdz/lowdown/blob/edca6ce6d5336efb147321a43c47a698de41bb7c/entity.c#L202
sed -i 's/\e\[u2013\]/--/' "$outfile"
fi
rm "$tmpfile"
(
printf "Title: %s\n\n" "$title"
cat "$infile"
) | lowdown -sT man --nroff-nolinks $lowdown_args -M section="$section" -o "$outfile"

View file

@ -1,6 +1,7 @@
---
synopsis: Concise error printing in `nix repl`
prs: 9928
cls: 811
---
Previously, if an element of a list or attribute set threw an error while

View file

@ -0,0 +1,23 @@
---
synopsis: Lix turns more internal bugs into crashes
cls: [797, 626]
---
Lix now enables build options such as trapping on signed overflow and enabling
libstdc++ assertions by default. These may find new bugs in Lix, which will
present themselves as Lix processes aborting, potentially without an error
message.
If Lix processes abort on your machine, this is a bug. Please file a bug,
ideally with the core dump (or information from it).
On Linux, run `coredumpctl list`, find the crashed process's PID at
the bottom of the list, then run `coredumpctl info THE-PID`. You can then paste
the output into a bug report.
On macOS, open the Console app from Applications/Utilities, select Crash
Reports, select the crash report in question. Right click on it, select Open In
Finder, then include that file in your bug report. [See the Apple
documentation][apple-crashreport] for more details.
[apple-crashreport]: https://developer.apple.com/documentation/xcode/acquiring-crash-reports-and-diagnostic-logs#Locate-crash-reports-and-memory-logs-on-the-device

View file

@ -0,0 +1,8 @@
---
synopsis: re-evaluate cached evaluation errors
cls: 771
---
"cached failure of [expr]" errors have been removed: expressions already in the
eval cache as a failure will now simply be re-evaluated, removing the need to
set `--no-eval-cache` or similar to see the error.

View file

@ -0,0 +1,13 @@
---
synopsis: Experimental REPL support for documentation comments using `:doc`
cls: 564
---
Using `:doc` in the REPL now supports showing documentation comments when defined on a function.
Previously this was only able to document builtins, however it now will show comments defined on a lambda as well.
This support is experimental and relies on an embedded version of [nix-doc](https://github.com/lf-/nix-doc).
The logic also supports limited Markdown formatting of doccomments and should easily support any [RFC 145](https://github.com/NixOS/rfcs/blob/master/rfcs/0145-doc-strings.md)
compatible documentation comments in addition to simple commented documentation.

View file

@ -0,0 +1,36 @@
---
synopsis: Add `repl-overlays` option
prs: 10203
cls: 504
---
A `repl-overlays` option has been added, which specifies files that can overlay
and modify the top-level bindings in `nix repl`. For example, with the
following contents in `~/.config/nix/repl.nix`:
```nix
info: final: prev: let
optionalAttrs = predicate: attrs:
if predicate
then attrs
else {};
in
optionalAttrs (prev ? legacyPackages && prev.legacyPackages ? ${info.currentSystem})
{
pkgs = prev.legacyPackages.${info.currentSystem};
}
```
We can run `nix repl` and use `pkgs` to refer to `legacyPackages.${currentSystem}`:
```ShellSession
$ nix repl --repl-overlays ~/.config/nix/repl.nix nixpkgs
Lix 2.90.0
Type :? for help.
Loading installable 'flake:nixpkgs#'...
Added 5 variables.
Loading 'repl-overlays'...
Added 6 variables.
nix-repl> pkgs.bash
«derivation /nix/store/g08b5vkwwh0j8ic9rkmd8mpj878rk62z-bash-5.2p26.drv»
```

View file

@ -88,7 +88,92 @@
- [nix-instantiate](command-ref/nix-instantiate.md)
- [nix-prefetch-url](command-ref/nix-prefetch-url.md)
- [Experimental Commands](command-ref/experimental-commands.md)
{{#include ./command-ref/new-cli/SUMMARY.md}}
- [nix](command-ref/new-cli/nix.md)
- [nix build](command-ref/new-cli/nix3-build.md)
- [nix bundle](command-ref/new-cli/nix3-bundle.md)
- [nix copy](command-ref/new-cli/nix3-copy.md)
- [nix daemon](command-ref/new-cli/nix3-daemon.md)
- [nix derivation](command-ref/new-cli/nix3-derivation.md)
- [nix derivation add](command-ref/new-cli/nix3-derivation-add.md)
- [nix derivation show](command-ref/new-cli/nix3-derivation-show.md)
- [nix develop](command-ref/new-cli/nix3-develop.md)
- [nix doctor](command-ref/new-cli/nix3-doctor.md)
- [nix edit](command-ref/new-cli/nix3-edit.md)
- [nix eval](command-ref/new-cli/nix3-eval.md)
- [nix flake](command-ref/new-cli/nix3-flake.md)
- [nix flake archive](command-ref/new-cli/nix3-flake-archive.md)
- [nix flake check](command-ref/new-cli/nix3-flake-check.md)
- [nix flake clone](command-ref/new-cli/nix3-flake-clone.md)
- [nix flake info](command-ref/new-cli/nix3-flake-info.md)
- [nix flake init](command-ref/new-cli/nix3-flake-init.md)
- [nix flake lock](command-ref/new-cli/nix3-flake-lock.md)
- [nix flake metadata](command-ref/new-cli/nix3-flake-metadata.md)
- [nix flake new](command-ref/new-cli/nix3-flake-new.md)
- [nix flake prefetch](command-ref/new-cli/nix3-flake-prefetch.md)
- [nix flake show](command-ref/new-cli/nix3-flake-show.md)
- [nix flake update](command-ref/new-cli/nix3-flake-update.md)
- [nix fmt](command-ref/new-cli/nix3-fmt.md)
- [nix hash](command-ref/new-cli/nix3-hash.md)
- [nix hash file](command-ref/new-cli/nix3-hash-file.md)
- [nix hash path](command-ref/new-cli/nix3-hash-path.md)
- [nix hash to-base16](command-ref/new-cli/nix3-hash-to-base16.md)
- [nix hash to-base32](command-ref/new-cli/nix3-hash-to-base32.md)
- [nix hash to-base64](command-ref/new-cli/nix3-hash-to-base64.md)
- [nix hash to-sri](command-ref/new-cli/nix3-hash-to-sri.md)
- [nix help](command-ref/new-cli/nix3-help.md)
- [nix help-stores](command-ref/new-cli/nix3-help-stores.md)
- [nix key](command-ref/new-cli/nix3-key.md)
- [nix key convert-secret-to-public](command-ref/new-cli/nix3-key-convert-secret-to-public.md)
- [nix key generate-secret](command-ref/new-cli/nix3-key-generate-secret.md)
- [nix log](command-ref/new-cli/nix3-log.md)
- [nix nar](command-ref/new-cli/nix3-nar.md)
- [nix nar cat](command-ref/new-cli/nix3-nar-cat.md)
- [nix nar dump-path](command-ref/new-cli/nix3-nar-dump-path.md)
- [nix nar ls](command-ref/new-cli/nix3-nar-ls.md)
- [nix path-info](command-ref/new-cli/nix3-path-info.md)
- [nix print-dev-env](command-ref/new-cli/nix3-print-dev-env.md)
- [nix profile](command-ref/new-cli/nix3-profile.md)
- [nix profile diff-closures](command-ref/new-cli/nix3-profile-diff-closures.md)
- [nix profile history](command-ref/new-cli/nix3-profile-history.md)
- [nix profile install](command-ref/new-cli/nix3-profile-install.md)
- [nix profile list](command-ref/new-cli/nix3-profile-list.md)
- [nix profile remove](command-ref/new-cli/nix3-profile-remove.md)
- [nix profile rollback](command-ref/new-cli/nix3-profile-rollback.md)
- [nix profile upgrade](command-ref/new-cli/nix3-profile-upgrade.md)
- [nix profile wipe-history](command-ref/new-cli/nix3-profile-wipe-history.md)
- [nix realisation](command-ref/new-cli/nix3-realisation.md)
- [nix realisation info](command-ref/new-cli/nix3-realisation-info.md)
- [nix registry](command-ref/new-cli/nix3-registry.md)
- [nix registry add](command-ref/new-cli/nix3-registry-add.md)
- [nix registry list](command-ref/new-cli/nix3-registry-list.md)
- [nix registry pin](command-ref/new-cli/nix3-registry-pin.md)
- [nix registry remove](command-ref/new-cli/nix3-registry-remove.md)
- [nix repl](command-ref/new-cli/nix3-repl.md)
- [nix run](command-ref/new-cli/nix3-run.md)
- [nix search](command-ref/new-cli/nix3-search.md)
- [nix shell](command-ref/new-cli/nix3-shell.md)
- [nix show-config](command-ref/new-cli/nix3-show-config.md)
- [nix store](command-ref/new-cli/nix3-store.md)
- [nix store add-file](command-ref/new-cli/nix3-store-add-file.md)
- [nix store add-path](command-ref/new-cli/nix3-store-add-path.md)
- [nix store cat](command-ref/new-cli/nix3-store-cat.md)
- [nix store copy-log](command-ref/new-cli/nix3-store-copy-log.md)
- [nix store copy-sigs](command-ref/new-cli/nix3-store-copy-sigs.md)
- [nix store delete](command-ref/new-cli/nix3-store-delete.md)
- [nix store diff-closures](command-ref/new-cli/nix3-store-diff-closures.md)
- [nix store dump-path](command-ref/new-cli/nix3-store-dump-path.md)
- [nix store gc](command-ref/new-cli/nix3-store-gc.md)
- [nix store ls](command-ref/new-cli/nix3-store-ls.md)
- [nix store make-content-addressed](command-ref/new-cli/nix3-store-make-content-addressed.md)
- [nix store optimise](command-ref/new-cli/nix3-store-optimise.md)
- [nix store path-from-hash-part](command-ref/new-cli/nix3-store-path-from-hash-part.md)
- [nix store ping](command-ref/new-cli/nix3-store-ping.md)
- [nix store prefetch-file](command-ref/new-cli/nix3-store-prefetch-file.md)
- [nix store repair](command-ref/new-cli/nix3-store-repair.md)
- [nix store sign](command-ref/new-cli/nix3-store-sign.md)
- [nix store verify](command-ref/new-cli/nix3-store-verify.md)
- [nix upgrade-nix](command-ref/new-cli/nix3-upgrade-nix.md)
- [nix why-depends](command-ref/new-cli/nix3-why-depends.md)
- [Files](command-ref/files.md)
- [nix.conf](command-ref/conf-file.md)
- [Profiles](command-ref/files/profiles.md)
@ -109,7 +194,7 @@
- [CLI guideline](contributing/cli-guideline.md)
- [C++ style guide](contributing/cxx.md)
- [Release Notes](release-notes/release-notes.md)
{{#include ./SUMMARY-rl-next.md}}
- [Upcoming release](release-notes/rl-next.md)
- [Release 2.18 (2023-09-20)](release-notes/rl-2.18.md)
- [Release 2.17 (2023-07-24)](release-notes/rl-2.17.md)
- [Release 2.16 (2023-05-31)](release-notes/rl-2.16.md)

View file

@ -17,7 +17,7 @@ By default Nix reads settings from the following places, in that order:
1. If [`NIX_USER_CONF_FILES`](./env-common.md#env-NIX_USER_CONF_FILES) is set, then each path separated by `:` will be loaded in reverse order.
Otherwise it will look for `nix/nix.conf` files in `XDG_CONFIG_DIRS` and [`XDG_CONFIG_HOME`](./env-common.md#env-XDG_CONFIG_HOME).
Otherwise it will look for `nix/nix.conf` files in `XDG_CONFIG_DIRS` and `XDG_CONFIG_HOME`.
If unset, `XDG_CONFIG_DIRS` defaults to `/etc/xdg`, and `XDG_CONFIG_HOME` defaults to `$HOME/.config` as per [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html).
1. If [`NIX_CONFIG`](./env-common.md#env-NIX_CONFIG) is set, its contents are treated as the contents of a configuration file.
@ -67,3 +67,5 @@ Configuration options can be set on the command line, overriding the values set
The `extra-` prefix is supported for settings that take a list of items (e.g. `--extra-trusted users alice` or `--option extra-trusted-users alice`).
# Available settings
{{#include @generated@/command-ref/conf-file.md}}

View file

@ -115,9 +115,9 @@ For backwards compatibility, Nix commands will follow the standard only when [`u
The following environment variables are used to determine locations of various state and configuration files:
- [`XDG_CONFIG_HOME`]{#env-XDG_CONFIG_HOME} (default `~/.config`)
- [`XDG_STATE_HOME`]{#env-XDG_STATE_HOME} (default `~/.local/state`)
- [`XDG_CACHE_HOME`]{#env-XDG_CACHE_HOME} (default `~/.cache`)
- `XDG_CONFIG_HOME` (default `~/.config`)
- `XDG_STATE_HOME` (default `~/.local/state`)
- `XDG_CACHE_HOME` (default `~/.cache`)
[XDG Base Directory Specification]: https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html

View file

@ -0,0 +1,66 @@
xp_features_json = custom_target(
command : [nix, '__dump-xp-features'],
capture : true,
output : 'xp-features.json',
)
experimental_features_shortlist_md = custom_target(
command : nix_eval_for_docs + [
'--expr',
'import @INPUT0@ (builtins.fromJSON (builtins.readFile @INPUT1@))',
],
input : [
'../../generate-xp-features-shortlist.nix',
xp_features_json,
],
capture : true,
output : 'experimental-features-shortlist.md',
env : nix_env_for_docs,
)
# Intermediate step for manpage generation.
# This splorks the output of generate-manpage.nix as JSON,
# which gets written as a directory tree below.
nix3_cli_files_json = custom_target(
command : nix_eval_for_docs_common + [
'--json',
'--expr',
'import @INPUT0@ true (builtins.readFile @INPUT1@)',
],
input : [
'../../generate-manpage.nix',
nix3_cli_json,
],
capture : true,
output : 'new-cli.json',
env : nix_env_for_docs,
)
nix3_cli_files = custom_target(
command : [
python,
'@INPUT0@',
'-i', '@INPUT1@',
'-o', '@OUTPUT@',
],
input : [
'../../json-to-tree.py',
nix3_cli_files_json,
],
output : 'new-cli',
)
conf_file_md = custom_target(
command : [
nix_eval_for_docs,
'--expr',
'(import @INPUT0@).showSettings { inlineHTML = true; } (builtins.fromJSON (builtins.readFile @INPUT1@))',
],
capture : true,
input : [
'../../utils.nix',
conf_file_json,
experimental_features_shortlist_md,
],
output : 'conf-file.md',
env : nix_env_for_docs,
)

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-build.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-bundle.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-copy.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-daemon.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-derivation-add.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-derivation-show.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-derivation.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-develop.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-doctor.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-edit.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-eval.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-archive.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-check.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-clone.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-info.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-init.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-lock.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-metadata.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-new.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-prefetch.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-show.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-update.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-fmt.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-hash-file.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-hash-path.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-hash-to-base16.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-hash-to-base32.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-hash-to-base64.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-hash-to-sri.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-hash.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-help-stores.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-help.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-key-convert-secret-to-public.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-key-generate-secret.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-key.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-log.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-nar-cat.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-nar-dump-path.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-nar-ls.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-nar.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-path-info.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-print-dev-env.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-profile-diff-closures.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-profile-history.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-profile-install.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-profile-list.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-profile-remove.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-profile-rollback.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-profile-upgrade.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-profile-wipe-history.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-profile.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-realisation-info.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-realisation.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-registry-add.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-registry-list.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-registry-pin.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-registry-remove.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-registry.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-repl.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-run.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-search.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-shell.md}}

View file

@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-show-config.md}}

Some files were not shown because too many files have changed in this diff Show more