Commit graph

16339 commits

Author SHA1 Message Date
eldritch horrors 314f044c2b keep copies of parser inputs that are in-memory only
the parser modifies its inputs, which means that sharing them between
the error context reporting system and the parser itself can confuse the
reporting system. usually this led to early truncation of error context
reports which, while not dangerous, can be quite confusing.

(cherry picked from commit d384ecd553aa997270b79ee98d02f7cf7e1849e6)
Change-Id: I677646b5675b12b2faa787943646aa36dc6e6ee3
2024-03-18 07:56:23 -06:00
eldritch horrors 1f8b85786e libutil: remove vfork
vfork confers a large performance advantage over fork, measured locally
at 16µs per vfork agains 90µs per fork. however nix *almost always*
follows a vfork up with an execve-family call, melting the performance
advantage from 6x to only 15%. in most of those cases it's doing things
that are undefined behavior (like manipulating the heap, or even
throwing exceptions and trashing the parent process stack).

most notably the one place that could benefit from the vfork performance
improvement is linux derivation sandbox setup—which doesn't use vfork.

Change-Id: I2037b7384d5a4ca24da219a569e1b1f39531410e
2024-03-18 06:10:41 -06:00
jade 2890840b96 Merge "Forgejo issue importer" into main 2024-03-17 22:08:59 -06:00
jade 61e21b2557 Delete hasPrefix and hasSuffix from the codebase
These now have equivalents in the standard lib in C++20. This change was
performed with a custom clang-tidy check which I will submit later.
Executed like so:

ninja -C build && run-clang-tidy -checks='-*,nix-*' -load=build/libnix-clang-tidy.so -p .. -fix ../tests | tee -a clang-tidy-result

Change-Id: I62679e315ff9e7ce72a40b91b79c3e9fc01b27e9
2024-03-17 20:17:19 -07:00
jade 706cee5c49 Merge "builtins.nixVersion: return fixed fake version" into main 2024-03-17 12:13:21 -06:00
Qyriad 32d6e58069 flake: fix musl static stdenv devShell (fix nix flake check)
pkgs.pkgsStatic.glibcLocales is null, so the string coercion was failing
for devShells against static stdenvs

Change-Id: Iee8e1042a852133ce0432627d72a85e97c17055e
2024-03-17 10:05:18 -06:00
jade 886a418d23 builtins.nixVersion: return fixed fake version
This builtin is only going to cause us problems because we are not Nix,
so let's just falsify being in the 2.18 series, since that is the
closest target that has any meaning.

In future we might want to have a better feature detection mechanism,
for when we actually add stuff to some builtin's attr set argument. But
builtins.nixVersion is just going to be hopelessly broken and it should
be stubbed out.

Fixes #144

Change-Id: Id7390b32a29c6147f2977737d81846320de5d67e
2024-03-17 00:32:19 -07:00
eldritch horrors 11f35afa6f diagnose duplicated attrs at correct path
diagnose attr duplication at the path the duplication was detected, not
at the path the current attribute wanted to place. doing the latter is
only correct if a leaf attribute was duplicated, not if an attrpath was
set to a non-attrset in one binding and a (potentially implied) attrset
in another binding.

fixes #124

Change-Id: Ic4aa9cc12a9874d4e7897c6f64408f10aa36fc82
2024-03-16 22:12:49 +01:00
jade 3392020710 Forgejo issue importer
We needed a script to go yoink all the real NixOS/Nix issues from our
mirror into the Lix repo.

Change-Id: If8c8ebfb58634c675eae450454c0189288c6b18a
2024-03-16 00:22:33 -07:00
Rebecca Turner e257ff10fd Merge "Fix gc-small-vector.hh includes" into main 2024-03-15 16:20:40 -06:00
eldritch horrors 3c52344300 out with the -O3, in with the -O2
-O3 does not measurably improve performance of the resulting binaries,
neither with lto enabled nor with lto disabled. what it does to however
is cause gcc warning spew in libstdc++ that we can't do anything
about (and that upon inspection of libstdc++ source looks like a gcc
bug).

with lto, -O3:

Benchmark 1: GC_INITIAL_HEAP_SIZE=10g nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      4.608 s ±  0.027 s    [User: 3.866 s, System: 0.522 s]
    Range (min … max):    4.579 s …  4.640 s    10 runs

Benchmark 2:  nix eval -f <nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix>
    Time (mean ± σ):     408.1 ms ±  25.5 ms    [User: 360.0 ms, System: 28.1 ms]
    Range (min … max):   387.6 ms … 439.0 ms    10 runs

with lto, -O2:

Benchmark 1: GC_INITIAL_HEAP_SIZE=10g nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      4.632 s ±  0.044 s    [User: 3.874 s, System: 0.544 s]
    Range (min … max):    4.563 s …  4.673 s    10 runs

Benchmark 2:  nix eval -f <nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix>
    Time (mean ± σ):     394.0 ms ±  23.9 ms    [User: 351.2 ms, System: 27.6 ms]
    Range (min … max):   377.8 ms … 429.3 ms    10 runs

without lto, -O3:

Benchmark 1: GC_INITIAL_HEAP_SIZE=10g nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      4.700 s ±  0.024 s    [User: 3.906 s, System: 0.559 s]
    Range (min … max):    4.663 s …  4.717 s    10 runs

Benchmark 2:  nix eval -f <nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix>
    Time (mean ± σ):     400.4 ms ±  25.6 ms    [User: 353.7 ms, System: 26.8 ms]
    Range (min … max):   379.8 ms … 430.6 ms    10 runs

without lto, -O2:

Benchmark 1: GC_INITIAL_HEAP_SIZE=10g nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      4.724 s ±  0.030 s    [User: 3.924 s, System: 0.570 s]
    Range (min … max):    4.687 s …  4.749 s    10 runs

Benchmark 2:  nix eval -f <nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix>
    Time (mean ± σ):     392.4 ms ±  24.3 ms    [User: 350.9 ms, System: 26.4 ms]
    Range (min … max):   376.9 ms … 428.0 ms    10 runs

fixes #46

Change-Id: Ib8afad8a07c278f57f2e3317d00cce4f9ec0f338
2024-03-15 15:31:49 -06:00
Rebecca Turner 7abbce500b
Fix gc-small-vector.hh includes
Change-Id: I4abc19029fb62712582761d4fc1895156b68803d
2024-03-15 13:39:32 -07:00
jade 0d85875c3a Allow dlopen of plugins to fail
It happens with some frequency that plugins that might be unimportant to
the evaluation at hand mismatch with the nix version, leading to
spurious load failures. Let's make these non fatal.

Change-Id: Iba10e951d171725ccf1a121bcd9be1e1d6ad69eb
2024-03-15 12:31:16 -07:00
jade 7d361f1a82 Test that :st does ... something
Change-Id: I97c00b5eb1288f68d8c2b484436cc185d040b8b2
2024-03-15 12:31:16 -07:00
jade af066af7f3 repl_characterization: Also verify the stack trace exists
Change-Id: I8b2d8211a24011fae1586a1182d7d0772a039cd7
2024-03-15 12:31:16 -07:00
jade 78513b1fc8 repl_characterization: eat newlines after commands and source-dir paths
This is because they are unrepresentable in the source files with
commentary but not in the output, so we should just eat them in
normalization. It's ok.

Change-Id: I2cb7e8b3fc7b00874885bb287cbaa200b41cb16b
2024-03-15 12:31:16 -07:00
jade 8a8715af89 Add regression tests for #9917, #9918
Change-Id: Ib0591e1499c5dba5e5a83ee75a899c9d16986827
2024-03-15 12:31:16 -07:00
jade 18ed6c3bdf Implement a repl characterization test system
This allows for automating using the repl without needing a PTY, with
very easy to write test files.

Change-Id: Ia8d7854edd91f93477638942cb6fc261354e6035
2024-03-15 12:31:16 -07:00
jade 38571c50e6 Implement a parser for a literate testing system for the repl
This parser can be reused for other purposes. It's inspired by
https://bitheap.org/cram/

Although eelco's impostor exists https://github.com/mobusoperandi/eelco,
it is not very nice to depend on out of tree testing frameworks with no
way to customize them.

Change-Id: Ifca50177e09730182baf0ebf829c3505bbb0274a
2024-03-14 14:30:38 -07:00
Cole Helbling 84727bebb4
Backport PR#9633 by cole-h: package: don't set sysconfdir in devShells
(cherry-picked from commit ba0087316acc2aba999cabe5e1a159da636b2569)

Change-Id: I5a50afb3b7b65516df798ee51b74f06727a91928
2024-03-14 14:06:31 -07:00
Rebecca Turner 9f242fae76
Only set LOCALE_ARCHIVE on Linux
`macOS` does not have `glibcLocales`:

    error:
       … while calling the 'derivationStrict' builtin

         at /derivation-internal.nix:9:12:

            8|
            9|   strict = derivationStrict drvAttrs;
             |            ^
           10|

       … while evaluating derivation 'nix-2.90.0'
         whose name attribute is located at /nix/store/y0c95bwyvs80pm69hdd4b11pyq2ghiwh-source
/pkgs/stdenv/generic/make-derivation.nix:348:7

       … while evaluating attribute 'LOCALE_ARCHIVE' of derivation 'nix-2.90.0'

         at /nix/store/ng5qzbyv4902b4pw7g35caqw5cnmryf9-source/flake.nix:331:15:

          330|               # Required to make non-NixOS Linux not complain about missing loc

Change-Id: I4464484a0eca12b5e073d49d900b6f25886245c1
2024-03-14 14:06:23 -07:00
puck caded0d55e Merge "Delete the existing installer" into main 2024-03-14 13:14:21 -06:00
jade c79b5dca2a Merge "flake: dev shells should include the LOCALE_ARCHIVE so configure does not complain" into main 2024-03-14 12:22:35 -06:00
jade cc55da201b Merge "un-thumbs-up ur github templates" into main 2024-03-14 12:22:09 -06:00
puck 93cc063344 Delete the existing installer
We're not going to use it.

Fixes: #31
Change-Id: Ib17a2eb6cae1ecbbf9ad1062e576ba6107a3c13b
2024-03-14 18:15:46 +00:00
eldritch horrors c26599b143 libexpr: fix elided value counting in printer
using the total-attrs-printed and total-list-items-printed counters to
calculate how many attrs were elided only works properly if no nesting
is involved. once things do nest the global counter can exceed the size
of the currently printed object, leading to unsigned wrapping and great
overestimation of elided counts. counting locally in addition to global
counts fixes this.

these are functional tests because creating these objects requires the
evaluator to not be a huge amount of code, and we also want defaults to
be tested for cli usage.

fixes #14

Change-Id: Icb9a0cb21b2f4bacbc5e9dcdd8c0b9055b4088a7
2024-03-14 01:52:19 -06:00
jade 0278c03de5 un-thumbs-up ur github templates
These are not the way that we want to do things.

Change-Id: I5f3706cf50d007a6659edb96a6230d52e18a769a
2024-03-13 23:10:25 -07:00
jade 2a8f579c53 Merge "Backport PR#10204 by 9999years: Replace foo with __NIX_STR in cxx-big-literal" into main 2024-03-14 00:07:21 -06:00
jade ab5ff86917 flake: dev shells should include the LOCALE_ARCHIVE so configure does not complain
Change-Id: Id661bd8c90696cb663a25503878a697f596d1e77
2024-03-13 12:25:23 -07:00
eldritch horrors 06952cf7c4 support <program>_ENV variables
this lets us set per-test-program environment variables rather than only
a single, global default. this was supported in nix originally but
might've gone partially missing in the upstream backports process?

Change-Id: Iad0919841b1b6d11e0b7ebd3920449a62f544e77
2024-03-13 19:48:26 +01:00
Rebecca Turner da2f165128 Backport PR#10204 by 9999years: Replace foo with __NIX_STR in cxx-big-literal
Looks a little nicer when you check the generated sources.

(cherry-picked from commit e65e9114d2797cc4380da218972979dda7395df6)

Change-Id: I91bd185bf12deef72d20fba36178ff42a686c518
Upstream-PR: https://github.com/NixOS/nix/pull/10204
2024-03-12 11:58:36 -07:00
jade 1b8662b85c import the revisions to the characterization test framework from cppnix
This has some Flaws for sure (like, it is going to be a bit stretched to
use for repl characterization), but it is a start.

Change-Id: I258c8beb3aee236f45818a03be83bcda858120c9
2024-03-11 14:14:43 -07:00
jade be2b87ed4d add automated usage mode to the repl
This is definitely not a stable thing, but it does feel slightly crimes
to put it as an experimental feature. Shrug, up for bikeshedding.

Change-Id: I6ef176e3dee6fb1cac9c0a7a60d553a2c63ea728
2024-03-11 14:14:43 -07:00
jade b06a392114 Merge "refactor: repl prompts are now the job of the interacter" into main 2024-03-11 15:12:09 -06:00
jade dd05106d1c Merge "refactor: move readline stuff into its own file" into main 2024-03-11 15:11:56 -06:00
jade df2723b972 Merge "finally.hh: delete copy constructor which is a bad idea" into main 2024-03-11 15:11:20 -06:00
jade d9367da027 Merge "Add box_ptr: nonnull unique_ptr with value semantics" into main 2024-03-11 15:11:15 -06:00
Qyriad f9701da8f6 Merge changes from topic "maint/flake-package.nix" into main
* changes:
  package: cleanup of all intermediaries
  package: migrate devShells
  package: migrate internal-api-docs
  package: migrate testNixVersions
  package: use pname, version, and dontBuild (first change with diff hash)
  package: refactor Nix out of flake.nix and into package.nix
2024-03-11 14:32:37 -06:00
jade 50c401b4c1 Merge "util.hh: split out signals stuff" into main 2024-03-11 11:14:24 -06:00
Qyriad 36a8d151e1 package: cleanup of all intermediaries
Change-Id: I0da5182de6b01c192cfcba407959d659d70c6dc9
2024-03-11 04:26:35 -06:00
Qyriad 529a01ade2 package: migrate devShells
Change-Id: Ic63721667edd4bef79aa699a0de8411639e5159b
2024-03-11 04:26:35 -06:00
Qyriad b072c069b7 package: migrate internal-api-docs
Change-Id: I344d73a412c2c6e4bb2eb14bd4859056324f1ba7
2024-03-11 04:26:35 -06:00
Qyriad 4ad3446311 package: migrate testNixVersions
Change-Id: I71845f8a6d7b77c3617d055e726ed4a28cd05fa3
2024-03-11 04:26:35 -06:00
Qyriad 875b76d0c7 package: use pname, version, and dontBuild (first change with diff hash)
The src fileset, preConfigure, and separateDebugInfo also respond to doBuild if its overridden

This commit is logically just a continuation of the previous commit's
refactor, but exists separately to delineate when the core Nix
derivation hash changed (this commit).

Change-Id: I67a61bc9608d91b6a833ebc5c3894b2d2e694050
2024-03-11 04:26:35 -06:00
Qyriad 15380b4c6e package: refactor Nix out of flake.nix and into package.nix
This series takes a somewhat different approach from the flake rework
done in NixOS/nix. The package.nix here does not provide callPackage
options for all the various settings in the build, and instead the other
places Nix derivations are used (like internal-api-docs) will .overrideAttrs
the normal Nix package derivation. This more closely matches how these
things were structured originally, and results in less churn and more
atomicity in these changes.

In the future, package.nix likely will migrate to have more build
options in the callPackage arguments, but we are also planning to
rewrite the build system anyway.

Change-Id: I170c4e5a4184bab62e1fd75e56db876d4ff116cf
2024-03-11 04:26:35 -06:00
jade 1758a6ef25 refactor: repl prompts are now the job of the interacter
Change-Id: I17c2873dfbbff303cdbdc7a8903deb8409ce3026
2024-03-11 01:04:52 -07:00
jade 95a87f2c2a refactor: move readline stuff into its own file
This is in direct preparation for an automation mode of nix repl.

Change-Id: I26e6ca88ef1c48aab11a2d1e939ff769f1770caa
2024-03-11 01:04:52 -07:00
jade 45f6e3521a finally.hh: delete copy constructor which is a bad idea
Change-Id: I6d0b5736893c44bddc6f5789b452b434f8671b9b
2024-03-11 01:04:52 -07:00
jade af515baf6e Add box_ptr: nonnull unique_ptr with value semantics
This solves the problem of collections of boxed subclasses with virtual
dispatch, which should still be treated as values, since the
indirection is only there due to the virtual dispatch.

Change-Id: I368daedd3f31298e99c6e56a15606337a55494c6
2024-03-11 01:04:52 -07:00
jade 8be7030299 util.hh: split out signals stuff
Copies part of the changes of ac89bb064aeea85a62b82a6daf0ecca7190a28b7

Change-Id: I9ce601875cd6d4db5eb1132d7835c5bab9f126d8
2024-03-11 00:52:09 -07:00