This builtin was always a problem and nixpkgs uses it in exactly one
place, to give up if the Nix version is absurdly old. It has no other
use cases, and doesn't work in a multi-implementation world anyway.
Change-Id: I03c36e118591029e2ef14b091fe14a311c66a08a
This does not add missing release notes, and it doesn't do anything
about the profiles feature we would really like to have so we can have
consistent credit.
Change-Id: I72a6f7acfcff85f380be17dac76501a6f4693776
This was a combination of two problems: the python didn't throw an error
because apparently glob on a nonexistent directory doesn't crash, and
secondarily, bash ignores bad exit codes without `set -e` if they are
not in the final/only command.
Change-Id: I812bde7a4daee5c77ffe9d7c73a25fd14969f548
Use the correct directory for the rl-next build, so that the release notes
actually get built and the page doesn't end up empty. I don't know why the
exception didn't cause a build failure before.
Fixes: #297
Change-Id: Ic72b9bb4c0d2d1f633f2af90cce4a3a2796d7f9b
Basically I'd expect the same behavior as with `nix-build`, i.e.
with `--keep-going` the hash-mismatch error of each failing
fixed-output derivation is shown.
The approach is derived from `Store::buildPaths` (`entry-point.cc`):
instead of throwing the first build-result, check if there are any build
errors and if so, display all of them and throw after that.
Unfortunately, the BuildResult struct doesn't have an `ErrorInfo`
(there's a FIXME for that at least), so I have to construct my own here.
This is a rather cheap bugfix and I decided against touching too many
parts of libstore for that (also I don't know if that's in line with the
ongoing refactoring work).
Closes #302
Change-Id: I378ab984fa271e6808c6897c45e0f070eb4c6fac
This is not like, perfect, since it is a manual operation, but we can
automate it in the future. rclone is used, since it seems like awscli is
not (obviously at least?) able to sync directories such that old things
are deleted, and rclone does this thing properly.
Fixes: lix-project/meta#2
Change-Id: Ia6a46d861342a6d29b22f981ba4e35e79f79e60e
The interrupt-blocking code was originally introduced 20 years ago so that
trying to log an error message does not result in an interrupt exception being
thrown and then going unhandled (c8d3882cdc).
However, the logging code does not check for interrupts any more
(054be50257), so this reasoning is no longer
applicable. Delete this code so that later interrupts are unblocked again, for
example in the next line entered into the repl.
Closes: #296
Change-Id: I48253f5f4272e75001148c13046e709ef5427fbd
This doesn't comprehensively fix everything outdated in the manual, or
make the manual greatly better, but it does note down where at least
jade noticed it was wrong, and it does fix all the instances of
referencing Nix to conform to the style guide to the best of our
ability.
A lot of things have been commented out for being wrong, and there are
three types of FIXME introduced:
- FIXME(Lix): generically Lix needs to fix it
- FIXME(Qyriad): re #215
- FIXME(meson): docs got outdated by meson changes and need rewriting
I did fix a bunch of it that I could, but there could certainly be
mistakes and this is definitely just an incremental improvement.
Fixes: #266
Change-Id: I5993c4603d7f026a887089fce77db08394362135
mdbook has the unfortunate habit of creating stub files for chapters it
can't find on disk. turn off this helpful feature as it masks errors in
the summary file, and fix a recently introduced instance of this error.
Change-Id: I10d86aac0489c9c494bd5c8a50047415f4d4b18d
This reverts commit a8b3d777fb.
This undoes the revert of PR#6621, which allows nested `follows`, i.e.
{
inputs = {
foo.url = "github:bar/foo";
foo.inputs.bar.inputs.nixpkgs = "nixpkgs";
};
}
does the expected thing now. This is useful to avoid the 1000 instances
of nixpkgs problem without having each flake in the dependency tree to
expose all of its transitive dependencies for modification.
This was in fact part of Nix before and the C++ changes applied w/o
conflicts. However, it got reverted then because people didn't want to
merge lazy-trees against it which was supposed to be merged soon back in
October 2022.
Fixes: #201
Change-Id: I5ddef914135b695717b2ef88862d57ced5e7aa3c
With Linux kernel >=6.6 & glibc 2.39 a `fchmodat2(2)` is available that
isn't filtered away by the libseccomp sandbox.
Being able to use this to bypass that restriction has surprising results
for some builds such as lxc[1]:
> With kernel ≥6.6 and glibc 2.39, lxc's install phase uses fchmodat2,
> which slips through 9b88e52846/src/libstore/build/local-derivation-goal.cc (L1650-L1663).
> The fixupPhase then uses fchmodat, which fails.
> With older kernel or glibc, setting the suid bit fails in the
> install phase, which is not treated as fatal, and then the
> fixup phase does not try to set it again.
Please note that there are still ways to bypass this sandbox[2] and this is
mostly a fix for the breaking builds.
This change works by creating a syscall filter for the `fchmodat2`
syscall (number 452 on most systems). The problem is that glibc 2.39
is needed to have the correct syscall number available via
`__NR_fchmodat2` / `__SNR_fchmodat2`, but this flake is still on
nixpkgs 23.11. To have this change everywhere and not dependent on the
glibc this package is built against, I added a header
"fchmodat2-compat.hh" that sets the syscall number based on the
architecture. On most platforms its 452 according to glibc with a few
exceptions:
$ rg --pcre2 'define __NR_fchmodat2 (?!452)'
sysdeps/unix/sysv/linux/x86_64/x32/arch-syscall.h
58:#define __NR_fchmodat2 1073742276
sysdeps/unix/sysv/linux/mips/mips64/n32/arch-syscall.h
67:#define __NR_fchmodat2 6452
sysdeps/unix/sysv/linux/mips/mips64/n64/arch-syscall.h
62:#define __NR_fchmodat2 5452
sysdeps/unix/sysv/linux/mips/mips32/arch-syscall.h
70:#define __NR_fchmodat2 4452
sysdeps/unix/sysv/linux/alpha/arch-syscall.h
59:#define __NR_fchmodat2 562
I added a small regression-test to the setuid integration-test that
attempts to set the suid bit on a file using the fchmodat2 syscall.
I confirmed that the test fails without the change in
local-derivation-goal.
Additionally, we require libseccomp 2.5.5 or greater now: as it turns
out, libseccomp maintains an internal syscall table and
validates each rule against it. This means that when using libseccomp
2.5.4 or older, one may pass `452` as syscall number against it, but
since it doesn't exist in the internal structure, `libseccomp` will refuse
to create a filter for that. This happens with nixpkgs-23.11, i.e. on
stable NixOS and when building Lix against the project's flake.
To work around that
* a backport of libseccomp 2.5.5 on upstream nixpkgs has been
scheduled[3].
* the package now uses libseccomp 2.5.5 on its own already. This is to
provide a quick fix since the correct fix for 23.11 is still a staging cycle
away.
We still need the compat header though since `SCMP_SYS(fchmodat2)`
internally transforms this into `__SNR_fchmodat2` which points to
`__NR_fchmodat2` from glibc 2.39, so it wouldn't build on glibc 2.38.
The updated syscall table from libseccomp 2.5.5 is NOT used for that
step, but used later, so we need both, our compat header and their
syscall table 🤷
Relevant PRs in CppNix:
* https://github.com/NixOS/nix/pull/10591
* https://github.com/NixOS/nix/pull/10501
[1] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2031073804
[2] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2030844251
[3] https://github.com/NixOS/nixpkgs/pull/306070
(cherry picked from commit ba6804518772e6afb403dd55478365d4b863c854)
Change-Id: I6921ab5a363188c6bff617750d00bb517276b7fe
Part of #7672
My main motivation is to be able to use `nix.checkConfig`[1]. This
doesn't work with Lix currently since the module uses `nix show-config`
if the Nix version is <2.20pre and `nix config show` otherwise. I think
this is the only instance where nixpkgs checks for which Nix commands
exist that affects us now, so I figured we could just perform the rename
here as well[2] and still provide the current version number[3].
I don't have a strong opinion on whether to deprecate `nix show-config`,
the warning is added there automatically.
(cherry picked from commit f300e11b056dea414d7d77bbc6e5a7dc5d9ddd41)
[1] https://nixos.org/manual/nixos/stable/options.html#opt-nix.checkConfig
[2] I should add that I don't use the "official" ways of installing Lix
because using the flake directly and callPackaging it seemed to fit
better into my workflow: I already have a little mess to make
sure Hydra from the flake uses the correct pkgs.nix and I didn't
want to complicate it further while keeping a single package-set I
can build in CI. Don't get me wrong, I think such a module for a
quick-start is very important, just giving context on why I bother
in the first place :)
[3] When we go public, I think it's worth considering to add support in
nixpkgs itself for Lix.
Change-Id: I47b4239b05cbeda3c370d2fa56ea768b768768ac
* changes:
nix3-profile: remove check "name" attr in manifests
Add profile migration test
nix3-profile: make element names stable
getNameFromURL(): Support uppercase characters in attribute names
nix3-profile: remove indices
nix3-profile: allow using human-readable names to select packages
implement parsing human-readable names from URLs
As discussed in the maintainer meeting on 2024-01-29.
Mainly this is to avoid a situation where the name is parsed and
treated as a file name, mostly to protect users.
.-* and ..-* are also considered invalid because they might strip
on that separator to remove versions. Doesn't really work, but that's
what we decided, and I won't argue with it, because .-* probably
doesn't seem to have a real world application anyway.
We do still permit a 1-character name that's just "-", which still
poses a similar risk in such a situation. We can't start disallowing
trailing -, because a non-zero number of users will need it and we've
seen how annoying and painful such a change is.
What matters most is preventing a situation where . or .. can be
injected, and to just get this done.
(cherry picked from commit f1b4663805a9dbcb1ace64ec110092d17c9155e0)
Change-Id: I900a8509933cee662f888c3c76fa8986b0058839
nix3-profile automatically migrates any profile its used on to its style
of profile -- the ones with manifest.json instead of manifest.nix. On
non-NixOS systems, Nix is conventionally installed to the profile at
/nix/var/nix/profiles/default, so if a user passed that to `--profile`
of `nix profile`, then it would break upgrade-nix from ever working
again, without recreating the profile.
This commit fixes that, and allows upgrade-nix to work on either kind of
profile.
Fixes#16.
Change-Id: I4c49b1beba93bb50e8f8a107edc451affe08c3f7
This should have been there from the beginning. As much as nix-env is a
pile of problems we don't need trivial docs papercuts like this adding
to it.
Change-Id: I0c53e4b146af2fefdd0e4743d850672729cb2194
This commit makes Meson the default buildsystem for Lix.
The Make buildsystem is now deprecated and will be removed soon, but has
not yet, which will be done in a later commit when all seems good. The
mesonBuild jobs have been removed, and have not been replaced with
equivalent jobs to ensure the Make buildsystem still works.
The full, new commands in a development shell are:
$ meson setup ./build "--prefix=$out" $mesonFlags
(A simple `meson setup ./build` will also build, but will do a different
thing, not having the settings from package.nix applied.)
$ meson compile -C build
$ meson test -C build --suite=check
$ meson install -C build
$ meson test -C build --suite=installcheck
(Check and installcheck may both be done after install, allowing you to
omit the --suite argument entirely, but this is the order package.nix
runs them in.)
If tests fail and Meson helpfully has no output for why, use the
`--print-error-logs` option to `meson test`. Why this is not the default
I cannot explain.
If you change a setting in the buildsystem, most cases will
automatically regenerate the Meson configuration, but some cases, like
trying to build a specific target whose name is new to the buildsystem
(e.g. `meson compile -C build src/libmelt/libmelt.dylib`, when
`libmelt.dylib` did not exist as a target the last time the buildsystem
was generated), then you can reconfigure using new settings but
existing options, and only recompiling stuff affected by the changes:
$ meson setup --reconfigure build
Note that changes to the default values in `meson.options` or in the
`default_options :` argument to project() are NOT propagated with
`--reconfigure`.
If you want a totally clean build, you can use:
$ meson setup --wipe build
That will work regardless of if `./build` exists or not.
Specific, named targets may be addressed in
`meson build -C build <target>` with the "target ID" if there is one,
which is the first string argument passed to target functions that
have one, and unrelated to the variable name, e.g.:
libexpr_dylib = library('nixexpr', …)
can be addressed with:
$ meson compile -C build nixexpr
All targets may be addressed as their output, relative to the build
directory, e.g.:
$ meson compile -C build src/libexpr/libnixexpr.so
But Meson does not consider intermediate files like object files
targets. To build a specific object file, use Ninja directly and
specify the output file relative to the build directory:
$ ninja -C build src/libexpr/libnixexpr.so.p/nixexpr.cc.o
To inspect the canonical source of truth on what the state of the
buildsystem configuration is, use:
$ meson introspect
Have fun!
Change-Id: Ia3e7b1e6fae26daf3162e655b4ded611a5cd57ad
This commit adds the capability for building the Doxygen internal API
docs in the Meson buildsystem, and also makes doing so the default for
the internal-api-docs hydra job. Aside from the /nix-support directory,
which differed only by the hash part of a store path, the outputs of
hydraJobs.internal-api-docs before and after this commit were
bit-for-bit identical on my machine.
Change-Id: I98f0017891c25b06866c15f7652fe74f706ec8e1
this was previously used because the macOS docs build would otherwise
pull files out of the host nix store. or something. not sure about it
Change-Id: I76b51eac1ebc5de5f00e2e4be086dd8db3eeb8e6
manpages can be rendered using the markdown output of mdbook, the rest
of the manual can generated out of the main doc/manual source tree. we
still use lowdown to actually render manpages instead of eg mdbook-man
because lowdown does generate reasonably good manpages (though that is
also somewhat debatable, but they're a lot better than mdbook-man).
doing this not only lets us drastically simplify the lowdown pipeline,
but also remove all custom {{#include}} handling since now mdbook does
all of it, even for the manpage builds. even the lowdown wrapper isn't
entirely necessary because lowdown can take all wrapper arguments with
command line flags rather than bits of input file content.
This also implements running mdbook in Meson, in order to generate the
manpages. The mdbook outputs are also installed in the usual location.
Co-authored-by: Qyriad <qyriad@qyriad.me>
Change-Id: I60193f9fd0f15d48872f071af35855cda2a0f40b
Previously, errors while printing values in `nix repl` would be printed
in `«error: ...»` brackets rather than displayed normally:
```
nix-repl> legacyPackages.aarch64-darwin.pythonPackages.APScheduler
«error: Package ‘python-2.7.18.7’ in /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/cpython/2.7/default.nix:335 is marked as insecure, refusing to evaluate.»
```
Now, errors will be displayed normally if they're emitted at the
top-level of an expression:
```
nix-repl> legacyPackages.aarch64-darwin.pythonPackages.APScheduler
error:
… in the condition of the assert statement
at /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/lib/customisation.nix:268:17:
267| in commonAttrs // {
268| drvPath = assert condition; drv.drvPath;
| ^
269| outPath = assert condition; drv.outPath;
… in the left operand of the OR (||) operator
at /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/passthrufun.nix:28:45:
27| if lib.isDerivation value then
28| lib.extendDerivation (valid value || throw "${name} should use `buildPythonPackage` or `toPythonModule` if it is to be part of the Python packages set.") {} value
| ^
29| else
(stack trace truncated; use '--show-trace' to show the full trace)
error: Package ‘python-2.7.18.7’ in /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/cpython/2.7/default.nix:335 is marked as insecure, refusing to evaluate.
```
Errors emitted in nested structures (like e.g. when printing `nixpkgs`)
will still be printed in brackets.
Change-Id: I25aeddf08c017582718cb9772a677bf51b9fc2ad
this isn't strictly necessary, but it'll make it a lot easier to put the
generated files used by the autoconf build system in this directory too.
doing this now already will make the meson transition a lot easier later
Change-Id: I5fb39eade2ff88b6093c9ee436c9e8db793e9448
this would make meson build compatibility unnecessarily hard and
the cli does not change often enough to justify this complexity.
Change-Id: I17b1870cdf8538feeaa01a9945db97af2175a642
not sure why this was done the way it was considering that includes are
a feature the doc toolchain had previously. let's just always have some
kind of entry for the upcoming release in the dev manual builds even if
that means having a completely empty release notes chapter.
the release notes generation script isn't entirely functional right now
due to pre-commit hooks, but it's good enough for time being. we need a
better release process for notes anyway.
Change-Id: Ifda6912cf5233db013f72a30247a62d6f22b1565
Change-Id: I9eb347ec4aabc5be2b816ff0fd3e4be45f93b934
mdbook already does include processing of its own, and the custom
processing code has always admitted as much. we don't need it for
the mdbook build at this point if we run our preprocessors in the
right order, and maybe we can even have mdbook to return complete
pages to us that we only have to pass to lowdown without any more
preprocessing of our own.
Change-Id: Icd978acbc3b1e215fee8f062c53ab2cb2a222ab1
for some reason these three were anchors, not links, but had they been
links they wouldn't've worked because they're not defined anywhere but
here. in the print version of the manual they're duplicated many times
over (creating id collisions), so we should better remove them anyway.
Change-Id: I8988a7c32c812dee0f0b6d4953faa7cd1255228d
Adds a `repl-overlays` option, which specifies files that can overlay
and modify the top-level bindings in `nix repl`. For example, with the
following contents in `~/.config/nix/repl.nix`:
info: final: prev: let
optionalAttrs = predicate: attrs:
if predicate
then attrs
else {};
in
optionalAttrs (prev ? legacyPackages && prev.legacyPackages ? ${info.currentSystem})
{
pkgs = prev.legacyPackages.${info.currentSystem};
}
We can run `nix repl` and use `pkgs` to refer to `legacyPackages.${currentSystem}`:
$ nix repl --repl-overlays ~/.config/nix/repl.nix nixpkgs
Lix 2.90.0
Type :? for help.
Loading installable 'flake:nixpkgs#'...
Added 5 variables.
Loading 'repl-overlays'...
Added 6 variables.
nix-repl> pkgs.bash
«derivation /nix/store/g08b5vkwwh0j8ic9rkmd8mpj878rk62z-bash-5.2p26.drv»
Change-Id: Ic12e0f2f210b2f46e920c33088dfe1083f42391a
`nix eval --write-to` refuses to write to a directory that exists at
all, so now we generate in a temporary directory, and copy the generated
tree to the build directory. This is equivalent to what the Make
buildsystem did, actually, but hopefully more robust.
Future work: documenting the doc generation architecture in the
top-level meson.build outline comment.
Change-Id: Ic3eb6d26e3cc249a1c042fd3ced22d637ac66a69
This is terrible UX, and frankly an eval failure should be a cache
invalidation anyway.
This removes the CachedEvalError type entirely.
Fixes#223.
Change-Id: I91f8003eabd0ea45003024e96d1de3c7ae8e49d8
We're not entirely clear on why the links preprocessor has to be done
*before* rather than after, but we assume it is probably that as a
builtin preprocessor it does some processing on the raw book source,
and not just the JSON data.
Also a real use for Python pattern matching? I know I was surprised too.
Change-Id: Ibe8b59e7b5bd5f357a655d8b4c5f0b0f58a67d6b
This reverts commit 70954233743a233744787103d3211237a28ddbca.
This seems to have broken running ninja on warm build directories, which
is not what we want. Reverted until we figure out something better
Change-Id: I9623ae078917e7c59a930bf8044a216501d4bb20
This puts the generated files where they are for the make system.
This is in preparation for further meson-mdbook stuff.
Change-Id: I934df6854a80af5ccf381cf1da0bda0187a8bcfc
For a long time `nix repl` has supported displaying documentation set on
builtins, however, it has long been convention to use Markdown comments
on Nix functions themselves for documentation. This exposes that
information to `nix repl` users in a nice and formatted way.
NixOS/rfcs#145 doc-comments are primarily what this feature is intended
to consume, however, support for lambda documentation in the repl is
experimental. We do our best effort to support the RFC here.
These changes are based on [the nix-doc library](https://github.com/lf-/nix-doc) and
are licensed under the terms described in the relevant source files.
Change-Id: Ic6fe947d39a22540705d890737e336c4720b0a22
this notably does *not* install the `nix3-manpages` manpage the old
system generated, mostly because that page was empty and just a bug
with a coat of documentation paint.
Change-Id: I7a4248a72e7bb5e0cc925a6311a33b6b72589569
we'll want to use these for the meson builds, and probably eventually
rewrite them in something that isn't plain shell. diffoscope confirms
that out/share and doc/share are equal before and after these changes
Change-Id: I49aa418fc8615cad86d67328e08c28a7405ec952
The big ones here are `trim-trailing-whitespace` and `end-of-file-fixer`
(which makes sure that every file ends with exactly one newline
character).
Change-Id: Idca73b640883188f068f9903e013cf0d82aa1123
We don't apply any patches to it, and vendoring it locks users into
bugs (it hasn't been updated since its introduction in late 2021).
Closes #164
Change-Id: Ied071c841fc30b0dfb575151afd1e7f66970fdb9
I didn't enable this by default for clang due to making the build time
10% worse or so. Unfortunate, but tbh devs for whom 10% of build time is
not *that* bad should probably simply enable this.
Change-Id: I8d1e5b6f3f76c649a4e2f115f534f7f97cee46e6
hacking changelog-d to support not just github but also forgejo and
gerrit is a lot more complicated than it's worth, even moreso since
the entire thing can just as well be done with ~60 lines of python.
this new script is also much cheaper to instantiate (being python),
so having it enabled in all shells is far less of a hassle.
we've also adjusted existing release notes that referenced a gerrit
cl to auto-link to the cl in question, making the diff a bit bigger
closes #176
Change-Id: I8ba7dd0070aad9ba4474401731215fcf5d9d2130
previously we reported the error at the beginning of the binding
block (for plain inherits) or the beginning of the attr list (for
inherit-from), effectively hiding where exactly the error happened.
this also carries over to runtime positions of attributes in sets as
reported by unsafeGetAttrPos. we're not worried about this changing
observable eval behavior because it *is* marked unsafe, and the new
behavior is much more useful.
(cherry picked from commit 1edd6fada53553b89847ac3981ac28025857ca02)
Change-Id: I2f50eb9f3dc3977db4eb3e3da96f1cb37ccd5174
we already normalize attr order to lexicographic, doing the same for
formals makes sense. doubly so because the order of formals would
otherwise depend on the context of the expression, which is not quite as
useful as one might expect.
(cherry picked from commit 4147ecfb1c51f3fe3b4adcbd4e753fd487dab645)
Change-Id: I3fd0dbdef3ac7447a3a03ff20bb514a0d0f23fb1
diagnose attr duplication at the path the duplication was detected, not
at the path the current attribute wanted to place. doing the latter is
only correct if a leaf attribute was duplicated, not if an attrpath was
set to a non-attrset in one binding and a (potentially implied) attrset
in another binding.
fixes#124
Change-Id: Ic4aa9cc12a9874d4e7897c6f64408f10aa36fc82
desugaring inherit-from to syntactic duplication of the source expr also
duplicates side effects of the source expr (such as trace calls) and
expensive computations (such as derivationStrict).
(cherry picked from commit cefd0302b55b3360dbca59cfcb4bf6a750d6cdcf)
Change-Id: Iff519f991adef2e51683ba2c552d37a3df7a179e
Enter debugger on `builtins.trace` with an option
(cherry picked from commit 774e7ca5847ebc392eac2a124a8f12b24da4f65a)
Change-Id: If01e2110b3a128e639b05143227e365227d149f1
Enter debugger more reliably in `let` expressions and function calls
(cherry picked from commit c4ed92fa6f836d3d8eb354a48c37a2f9eeecc3aa)
Change-Id: I16d0cad7e898feecd2399723b92ba8df67222fb4
Print the value in `error: cannot coerce` messages
(cherry picked from commit 5b7bfd2d6b89d7dd5f54c1ca6c8072358d31a84e)
===
test taken from 6e8d5983143ae576e3f4b1d2954a5267f2943a49; it was added
previously (and not backported because its pr was a mostly-revert), but
it's useful to have around.
Change-Id: Icbd14b55e3610ce7b774667bf14b82e6dc717982
Print the value in `value is X while a Y is expected` error
(cherry picked from commit 5f72a97092da6af28a7d2b2a50d74e9d34fae7e1)
Change-Id: Idb4bc903ae59a0f5b6fb3b1da4d47970fe0a6efe
Previously, there were two mostly-identical value printers -- one in
`libexpr/eval.cc` (which didn't force values) and one in
`libcmd/repl.cc` (which did force values and also printed ANSI color
codes).
This PR unifies both of these printers into `print.cc` and provides a
`PrintOptions` struct for controlling the output, which allows for
toggling whether values are forced, whether repeated values are tracked,
and whether ANSI color codes are displayed.
Additionally, `PrintOptions` allows tuning the maximum number of
attributes, list items, and bytes in a string that will be displayed;
this makes it ideal for contexts where printing too much output (e.g.
all of Nixpkgs) is distracting. (As requested by @roberth in
https://github.com/NixOS/nix/pull/9554#issuecomment-1845095735)
Please read the tests for example output.
Future work:
- It would be nice to provide this function as a builtin, perhaps
`builtins.toStringDebug` -- a printing function that never fails would
be useful when debugging Nix code.
- It would be nice to support customizing `PrintOptions` members on the
command line, e.g. `--option to-string-max-attrs 1000`.
(cherry picked from commit 0fa08b451682fb3311fe58112ff05c4fe5bee3a4, )
===
Restore ambiguous value printer for `nix-instantiate`
The Nix team has requested that this output format remain unchanged.
I've added a warning to the man page explaining that `nix-instantiate
--eval` output will not parse correctly in many situations.
(cherry picked from commit df84dd4d8dd3fd6381ac2ca3064432ab31a16b79)
Change-Id: I7cca6b4b53cd0642f2d49af657d5676a8554c9f8
(cherry picked from commit 561a56cd13b4f12e3dfb6c5e3f42e5d8add04ecc)
===
Modified the release notes' synopsis to make it match its contents,
probably a copy-paste.
Co-authored-by: Raito Bezarius <raito@lix.systems>
Change-Id: I03bbff940b93e7df4b6c2fe9159c49a59ed47b55
structured attrs: improve support / usage of NIX_ATTRS_{SH,JSON}_FILE
(cherry picked from commit 3c042f3b0b0a7ef9c47bf049f5410dbd4aac9e90)
Change-Id: I7e41838338ee1edf31fff6f9e354c3db2bba6c0e
Docs build: depend on locally built nix executable and not installed one
(cherry picked from commit ca72e3e7e8f69526f028475a7a9b40812da1acdd)
===
includes changes from (because not doing so removes manpages):
Merge pull request #9976 from alois31/restore-manual-pages
Restore manual pages
(cherry picked from commit d3c1997127e0fc08576e842b2bfe046d8a28d2f4)
Change-Id: I685ff16163ac552a1754570c03c992c63a461d50
Add position information to `while evaluating the attribute` errors in the debugger
(cherry picked from commit ffe67c86a8ef3695e5c8b9c9800c192ac633dded)
Change-Id: I177ea5ec60898abe09fb9d80d9602b2a32ff8f44
Expose locals from `let` expressions to the debugger
(cherry picked from commit acef4f17a2daab4ccdf656bdf229792db2f779e4)
Change-Id: Ib3623254f67ac762f4e7230d625e9f87dff38a84
Fix segfault on infinite recursion in some cases
(cherry picked from commit bf1b294bd81ca76c5ec9fe3ecd52196bf52a8300)
Change-Id: Id137541426ec8536567835953fccf986a3aebf16
rl-next: Fix and support markdown frontmatter syntax
(cherry picked from commit 69b7876a0810269ad71807594cfd99b26cd8a5ff)
Change-Id: I8bfb8967af0943080fdd70d257c34abaf0a9fedf
nix-shell: support single quotes in shebangs, fix whitespace parsing
(cherry picked from commit 3b99c6291377cbd22607896af9dfafa857d2f2dc)
Change-Id: I2a431b21c3467eefa1ef95d5a36d672f45b6937a
Factor out bits of the worker protocol to use elsewhere
(cherry picked from commit 4b1a97338f517f45e6169d3d8845c5caa5724e97)
Change-Id: If93afa0f8b1cf9b0e705b34fa71e6fd708752758
Unit test some worker protocol serializers
(cherry picked from commit c6faef61a6f31c71146aee5d88168e861df9a22a)
Change-Id: I99e36f5f17eb7642211a4e42a16b143424f164b4
do not show configuration override flags for each command
(cherry picked from commit f89b84919c1a5c796512c50311821e7779b3678b)
Change-Id: Ib98b739bd6c9a1e94f94a78a47d84d72e435e7c0
this part must have been moved quite a while ago, but apparently so far
no one noticed
(cherry picked from commit 6db805b3d1e4eccd0103d9856b8ab3d01efba51f)
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Today, with the tests inside a `tests` intermingled with the
corresponding library's source code, we have a few problems:
- We have to be careful that wildcards don't end up with tests being
built as part of Nix proper, or test headers being installed as part
of Nix proper.
- Tests in libraries but not executables is not right:
- It means each executable runs the previous unit tests again, because
it needs the libraries.
- It doesn't work right on Windows, which doesn't want you to load a
DLL just for the side global variable . It could be made to work
with the dlopen equivalent, but that's gross!
This reorg solves these problems.
There is a remaining problem which is that sibbling headers (like
`hash.hh` the test header vs `hash.hh` the main `libnixutil` header) end
up shadowing each other. This PR doesn't solve that. That is left as
future work for a future PR.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
(cherry picked from commit 91b6833686a6a6d9eac7f3f66393ec89ef1d3b57)
(cherry picked from commit a61e42adb528b3d40ce43e07c79368d779a8b624)
I think it is bad for these reasons when `tests/` contains a mix of
functional and integration tests
- Concepts is harder to understand, the documentation makes a good
unit vs functional vs integration distinction, but when the
integration tests are just two subdirs within `tests/` this is not
clear.
- Source filtering in the `flake.nix` is more complex. We need to
filter out some of the dirs from `tests/`, rather than simply pick
the dirs we want and take all of them. This is a good sign the
structure of what we are trying to do is not matching the structure
of the files.
With this change we have a clean:
```shell-session
$ git show 'HEAD:tests'
tree HEAD:tests
functional/
installer/
nixos/
```
(cherry picked from commit 68c81c737571794f7246db53fb4774e94fcf4b7e)
up to now, those were managed outside of this repo, which as
unsurprisingly a real hassle to deal with if one wanted to prevent URLs
from breaking when moving pages around. this change removes a large part
of the friction involved in moving content in the Nix manual.
possible next steps for further automation:
- check for content that moved and warn if it's not reachable from
links that were valid prior to a change
- create redirect rules automatically based on this information
(cherry picked from commit 2b7016cc56d12e67de9f1f25b18311866a26a5fe)
We use the same nested map representation we used for goals, again in
order to save space. We might someday want to combine with `inputDrvs`,
by doing `V = bool` instead of `V = std::set<OutputName>`, but we are
not doing that yet for sake of a smaller diff.
The ATerm format for Derivations also needs to be extended, in addition
to the in-memory format. To accomodate this, we added a new basic
versioning scheme, so old versions of Nix will get nice errors. (And
going forward, if the ATerm format changes again the errors will be even
better.)
`parsedStrings`, an internal function used as part of parsing
derivations in A-Term format, used to consume the final `]` but expect
the initial `[` to already be consumed. This made for what looked like
unbalanced brackets at callsites, which was confusing. Now it consumes
both which is hopefully less confusing.
As part of testing, we also created a unit test for the A-Term format for
regular non-experimental derivations too.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
this is a pure reformatting, contents were not changed
one sentence per line makes reviewing diffs and making suggestions much
more convenient. the indentation was an artifat of the DocBook
migration.
In the Nix language, given a drv path, we should be able to construct
another string referencing to one of its output. We can do this today
with `(import drvPath).output`, but this only works for derivations we
already have.
With dynamic derivations, however, that doesn't work well because the
`drvPath` isn't yet built: importing it like would need to trigger IFD,
when the whole point of this feature is to do "dynamic build graph"
without IFD!
Instead, what we want to do is create a placeholder value with the right
string context to refer to the output of the as-yet unbuilt derivation.
A new primop in the language, analogous to `builtins.placeholder` can be
used to create one. This will achieve all the right properties. The
placeholder machinery also will match out the `outPath` attribute for CA
derivations works.
In 60b7121d2c we added that type of
placeholder, and the derived path and string holder changes necessary to
support it. Then in the previous commit we cleaned up the code
(inspiration finally hit me!) to deduplicate the code and expose exactly
what we need. Now, we can wire up the primop trivally!
Part of RFC 92: dynamic derivations (tracking issue #6316)
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
We want to be able to write down `foo.drv^bar.drv^baz`:
`foo.drv^bar.drv` is the dynamic derivation (since it is itself a
derivation output, `bar.drv` from `foo.drv`).
To that end, we create `Single{Derivation,BuiltPath}` types, that are
very similar except instead of having multiple outputs (in a set or
map), they have a single one. This is for everything to the left of the
rightmost `^`.
`NixStringContextElem` has an analogous change, and now can reuse
`SingleDerivedPath` at the top level. In fact, if we ever get rid of
`DrvDeep`, `NixStringContextElem` could be replaced with
`SingleDerivedPath` entirely!
Important note: some JSON formats have changed.
We already can *produce* dynamic derivations, but we can't refer to them
directly. Today, we can merely express building or example at the top
imperatively over time by building `foo.drv^bar.drv`, and then with a
second nix invocation doing `<result-from-first>^baz`, but this is not
declarative. The ethos of Nix of being able to write down the full plan
everything you want to do, and then execute than plan with a single
command, and for that we need the new inductive form of these types.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
When loading a derivation from a JSON, malformed input would trigger
cryptic "assertion failed" errors. Simply replacing calls to `operator []`
with calls to `.at()` was not enough, as this would cause json.execptions
to be printed verbatim.
Display nice error messages instead and give some indication where the
error happened.
*Before:*
```
$ echo 4 | nix derivation add
error: [json.exception.type_error.305] cannot use operator[] with a string argument with number
$ nix derivation show nixpkgs#hello | nix derivation add
Assertion failed: (it != m_value.object->end()), function operator[], file /nix/store/8h9pxgq1776ns6qi5arx08ifgnhmgl22-nlohmann_json-3.11.2/include/nlohmann/json.hpp, line 2135.
$ nix derivation show nixpkgs#hello | jq '.[] | .name = 5' | nix derivation add
error: [json.exception.type_error.302] type must be string, but is object
$ nix derivation show nixpkgs#hello | jq '.[] | .outputs = { out: "/nix/store/8j3f8j-hello" }' | nix derivation add
error: [json.exception.type_error.302] type must be object, but is string
```
*After:*
```
$ echo 4 | nix derivation add
error: Expected JSON of derivation to be of type 'object', but it is of type 'number'
$ nix derivation show nixpkgs#hello | nix derivation add
error: Expected JSON object to contain key 'name' but it doesn't
$ nix derivation show nixpkgs#hello | jq '.[] | .name = 5' | nix derivation add
error: Expected JSON value to be of type 'string' but it is of type 'number'
$ nix derivation show nixpkgs#hello | jq '.[] | .outputs = { out: "/nix/store/8j3f8j-hello" }' | nix derivation add
error:
… while reading key 'outputs'
error: Expected JSON value to be of type 'object' but it is of type 'string'
```
Over the last year or so I've run into several use cases where I need to
parse and/or serialize URLs for use by `builtins.fetchTree` or
`builtins.getFlake`, largely in order to produce _lockfile-like_ files
for lang2nix frameworks or tools which use `nix` internally to drive
builds.
I've gone through the painstaking process of emulating
`nix::FlakeRef::fromAttrs` and `nix::parseFlakeRef` several times with
mixed success; but these are difficult to create and even harder to
maintain if I hope to stay aligned with changes to the real
parser/serializer.
I understand why adding new `builtins` isn't something we want to do
flagrantly. I'm recommending this addition simply because I keep
encountering use cases where I need to parse/serialize these URIs in
`nix` expressions, and I want a reliable solution.
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
the original change broke many pre-existing anchor links.
also change formatting of the constants listing slightly:
- the type should not be part of the anchor
- add highlight to the "impure only" note
* clarify wording on args@ default handling
Most importantly use shorter sentences and emphasize the key point that defaults aren't taken into account
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
this is a how-to guide which should not be in the reference manual.
it also refers to `nix-env`, which should not be the first thing readers
of the reference manual encounter, as it behaves very differently in
spirit from the rest of Nix.
slightly reword the documentation to be more concise and informative.
Grouping our tests should make it easier to understand the intent than
one long poorly-arranged list. It also is convenient for running just
the tests for a specific component when working on that component.
We need at least one test group so this isn't dead code; I decided to
collect the tests for the `ca-derivations` and `dynamic-derivations`
experimental features in groups. Do
```bash
make ca.test-group -jN
```
and
```bash
make dyn-drv.test-group -jN
```
to try running just them.
I originally did this as part of #8397 for being able to just the local
overlay store alone. I am PRing it separately now so we can separate
general infra from new features.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
* Lang now verifies errors and parse output
* Some new miscellaneous tests
* Easy way to update the tests
* Document workflow in manual
* Use `!` not `~` as separater char for sed
It is confusing to use `~` when we are talking about paths and home
directories!
* Test test suite itself (`test/lang-test/infra.sh`)
Additionally, run shellcheck on `tests/lang.sh` to help ensure it is
correct, now that is is more complex.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
This is done in roughly the same way builtin functions are documented.
Also auto-link experimental features for primops, subsuming PR #8371.
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
- Improved API docs from comment
- Exit codes are for `nix-build`, not just `nix-store --release`
- Make note in tests so the magic numbers are not surprising
Picking up where #8387 left off.
Deleting store info corrected (there is a foot-gun in Nix with
`--delete-generations old`!)
Also a few things are cleaned up based on feedback.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Uninstall instructions were moved to their own page in #8267. The
overall section link was redirected in #8286, but platform-specific
links (which I give out frequently when I triage installer trouble)
weren't included.
This is generally a fine practice: Putting implementations in headers
makes them harder to read and slows compilation. Unfortunately it is
necessary for templates, but we can ameliorate that by putting them in a
separate header. Only files which need to instantiate those templates
will need to include the header with the implementation; the rest can
just include the declaration.
This is now documenting in the contributing guide.
Also, it just happens that these polymorphic serializers are the
protocol agnostic ones. (Worker and serve protocol have the same logic
for these container types.) This means by doing this general template
cleanup, we are also getting a head start on better indicating which
code is protocol-specific and which code is shared between protocols.
While this is not actually a notion in the implementation, it is
explicitly described in the thesis and quite important for understanding
how the store works.
Co-authored-by: John Ericson <git@JohnEricson.me>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
jade
alois31
Maximilian Bosch
Qyriad
eldritch horrors
Robert Hensing
Rebecca Turner
Lunaphied
Winter Cute