-O3 does not measurably improve performance of the resulting binaries,
neither with lto enabled nor with lto disabled. what it does to however
is cause gcc warning spew in libstdc++ that we can't do anything
about (and that upon inspection of libstdc++ source looks like a gcc
bug).
with lto, -O3:
Benchmark 1: GC_INITIAL_HEAP_SIZE=10g nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
Time (mean ± σ): 4.608 s ± 0.027 s [User: 3.866 s, System: 0.522 s]
Range (min … max): 4.579 s … 4.640 s 10 runs
Benchmark 2: nix eval -f <nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix>
Time (mean ± σ): 408.1 ms ± 25.5 ms [User: 360.0 ms, System: 28.1 ms]
Range (min … max): 387.6 ms … 439.0 ms 10 runs
with lto, -O2:
Benchmark 1: GC_INITIAL_HEAP_SIZE=10g nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
Time (mean ± σ): 4.632 s ± 0.044 s [User: 3.874 s, System: 0.544 s]
Range (min … max): 4.563 s … 4.673 s 10 runs
Benchmark 2: nix eval -f <nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix>
Time (mean ± σ): 394.0 ms ± 23.9 ms [User: 351.2 ms, System: 27.6 ms]
Range (min … max): 377.8 ms … 429.3 ms 10 runs
without lto, -O3:
Benchmark 1: GC_INITIAL_HEAP_SIZE=10g nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
Time (mean ± σ): 4.700 s ± 0.024 s [User: 3.906 s, System: 0.559 s]
Range (min … max): 4.663 s … 4.717 s 10 runs
Benchmark 2: nix eval -f <nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix>
Time (mean ± σ): 400.4 ms ± 25.6 ms [User: 353.7 ms, System: 26.8 ms]
Range (min … max): 379.8 ms … 430.6 ms 10 runs
without lto, -O2:
Benchmark 1: GC_INITIAL_HEAP_SIZE=10g nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
Time (mean ± σ): 4.724 s ± 0.030 s [User: 3.924 s, System: 0.570 s]
Range (min … max): 4.687 s … 4.749 s 10 runs
Benchmark 2: nix eval -f <nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix>
Time (mean ± σ): 392.4 ms ± 24.3 ms [User: 350.9 ms, System: 26.4 ms]
Range (min … max): 376.9 ms … 428.0 ms 10 runs
fixes#46
Change-Id: Ib8afad8a07c278f57f2e3317d00cce4f9ec0f338
Docs build: depend on locally built nix executable and not installed one
(cherry picked from commit ca72e3e7e8f69526f028475a7a9b40812da1acdd)
===
includes changes from (because not doing so removes manpages):
Merge pull request #9976 from alois31/restore-manual-pages
Restore manual pages
(cherry picked from commit d3c1997127e0fc08576e842b2bfe046d8a28d2f4)
Change-Id: I685ff16163ac552a1754570c03c992c63a461d50
Including `config.h` also needs `$(buildprefix)`
(cherry picked from commit 96fdea3394ff61e24c53358644a5064218218d13)
Change-Id: I8b5c0b1826aa007aa681c8b199f9b1489cac6784
Use positive source filtering for the standalone functional tests job and Perl bindings
(cherry picked from commit 6b6bd9003062c86a49d4384381941cf57f269c45)
Change-Id: I896be67654f893d543ed6beb5d0d0d6c6d36e027
Unit test some worker protocol serializers
(cherry picked from commit c6faef61a6f31c71146aee5d88168e861df9a22a)
Change-Id: I99e36f5f17eb7642211a4e42a16b143424f164b4
Today, with the tests inside a `tests` intermingled with the
corresponding library's source code, we have a few problems:
- We have to be careful that wildcards don't end up with tests being
built as part of Nix proper, or test headers being installed as part
of Nix proper.
- Tests in libraries but not executables is not right:
- It means each executable runs the previous unit tests again, because
it needs the libraries.
- It doesn't work right on Windows, which doesn't want you to load a
DLL just for the side global variable . It could be made to work
with the dlopen equivalent, but that's gross!
This reorg solves these problems.
There is a remaining problem which is that sibbling headers (like
`hash.hh` the test header vs `hash.hh` the main `libnixutil` header) end
up shadowing each other. This PR doesn't solve that. That is left as
future work for a future PR.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
(cherry picked from commit 91b6833686a6a6d9eac7f3f66393ec89ef1d3b57)
(cherry picked from commit a61e42adb528b3d40ce43e07c79368d779a8b624)
I think it is bad for these reasons when `tests/` contains a mix of
functional and integration tests
- Concepts is harder to understand, the documentation makes a good
unit vs functional vs integration distinction, but when the
integration tests are just two subdirs within `tests/` this is not
clear.
- Source filtering in the `flake.nix` is more complex. We need to
filter out some of the dirs from `tests/`, rather than simply pick
the dirs we want and take all of them. This is a good sign the
structure of what we are trying to do is not matching the structure
of the files.
With this change we have a clean:
```shell-session
$ git show 'HEAD:tests'
tree HEAD:tests
functional/
installer/
nixos/
```
(cherry picked from commit 68c81c737571794f7246db53fb4774e94fcf4b7e)
Grouping our tests should make it easier to understand the intent than
one long poorly-arranged list. It also is convenient for running just
the tests for a specific component when working on that component.
We need at least one test group so this isn't dead code; I decided to
collect the tests for the `ca-derivations` and `dynamic-derivations`
experimental features in groups. Do
```bash
make ca.test-group -jN
```
and
```bash
make dyn-drv.test-group -jN
```
to try running just them.
I originally did this as part of #8397 for being able to just the local
overlay store alone. I am PRing it separately now so we can separate
general infra from new features.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
A library shouldn't require changes to the caller's argument handling,
especially if it doesn't have to, and indeed we don't have to.
This changes the lookup order to prioritize the hardcoded path to nix
if it exists. The static executable still finds itself through /proc
and the like.
The motivation is as stated in issue #7814: even though the the C++ API
is internal and unstable, people still want it to be well documented for
sake of learning, code review, and other purposes that aren't predicated
on it being stable.
Fixes#7814
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Building without tests is useful for bootstrapping with a smaller footprint
or running the tests in a separate derivation. Otherwise, we do compile and
run them.
This isn't fine grained as to allow picking `check` but not `installcheck`
or vice versa, but it's good enough for now.
I've tried to use Nixpkgs' `checkInputs`, but those inputs weren't discovered
properly by the configure script. We can emulate its behavior very well though.
Nixpkgs on aarch64-linux is currently stuck on GCC 9
(https://github.com/NixOS/nixpkgs/issues/208412) and using gcc11Stdenv
doesn't work either.
So use c++2a instead of c++20 for now. Unfortunately this means we
can't use some C++20 features for now (like std::span).
* libexpr: fix builtins.split example
The example was previously indicating that multiple whitespaces would be
collapsed into a single captured whitespace. That isn't true and was
likely a mistake when being documented initially.
* Fix segfault on unitilized list when printing value
Since lists are just chunks of memory the individual elements in the
list might be unitilized when a programming error happens within Nix.
In this case the values are null-initialized (at least with Boehm GC)
and we can avoid a nullptr deref when printing them.
I ran into this issue while ensuring that new expression tests would
show the actual value on an assertion failure.
This is unlikely to cause any runtime performance regressions as
printing values is not really in the hot path (unless the repl is the
primary use case).
* Add operator<< for ValueTypes
* Add libexpr tests
This introduces tests for libexpr that evalulate various trivial Nix
language expressions and primop invocations that should be good smoke
tests wheter or not the implementation is behaving as expected.
gives 2-5% performance improvement across a board of tests.
LTO is broken when using clang; some libs link fine while others crash
the linker with a segfault in the llvm linker plugin. 🙁
These headers are included by the libexpr, libfetchers, libstore
and libutil headers.
Considering that these are vendored sources, Nix should expose them,
as it is not a good idea for reverse dependencies to rely on a
potentially different source that can go out of sync.
This is only rudimentary support as allowed by `NIX_GET_COMPLETIONS`.
In the future, we could use complete’s `--wraps` argument to autocomplete arguments for programs after `nix shell -c`.
Otherwise the build is cluttered with
```
/nix/store/fwpn2f7a4iqszyydw7ag61zlnp6xk5d3-glibc-2.30-dev/include/features.h:382:4: warning: #warning _FORTIFY_SOURCE requires compiling with optimization (-O) [-Wcpp]
382 | # warning _FORTIFY_SOURCE requires compiling with optimization (-O)
| ^~~~~~~
```
when building with `OPTIMIZE=0`
On nix-env -qa -f '<nixpkgs>', this reduces maximum RSS by 20970 KiB
and runtime by 0.8%. This is mostly because we're not parsing the hash
part as a hash anymore (just validating that it consists of base-32
characters).
Also, replace storePathToHash() by StorePath::hashPart().
This is a proof on concept to evaluate writing unit tests for Nix using
google test (https://github.com/google/googletest).
In order to execute tests:
$ make unit-tests
$ ./unit-tests
The Makefile rules for `unit-tests` is a complete hack.
This provides a pluggable mechanism for defining new fetchers. It adds
a builtin function 'fetchTree' that generalizes existing fetchers like
'fetchGit', 'fetchMercurial' and 'fetchTarball'. 'fetchTree' takes a
set of attributes, e.g.
fetchTree {
type = "git";
url = "https://example.org/repo.git";
ref = "some-branch";
rev = "abcdef...";
}
The existing fetchers are just wrappers around this. Note that the
input attributes to fetchTree are the same as flake input
specifications and flake lock file entries.
All fetchers share a common cache stored in
~/.cache/nix/fetcher-cache-v1.sqlite. This replaces the ad hoc caching
mechanisms in fetchGit and download.cc (e.g. ~/.cache/nix/{tarballs,git-revs*}).
This also adds support for Git worktrees (c169ea5904).
jade
eldritch horrors
Robert Hensing