lix-project/lix
18
38
Fork 14
Code Issues 287 Code Review (Gerrit) Projects 1 Releases Wiki Activity
lix/tests/functional
History
eldritch horrors e6cd67591b libexpr: rewrite the parser with pegtl instead of flex/bison 
this gives about 20% performance improvements on pure parsing. obviously
it will be less on full eval, but depending on how much parsing is to be
done (e.g. including hackage-packages.nix or not) it's more like 4%-10%.

this has been tested (with thousands of core hours of fuzzing) to ensure
that the ASTs produced by the new parser are exactly the same as the old
one would have produced. error messages will change (sometimes by a lot)
and are not yet perfect, but we would rather leave this as is for later.

test results for running only the parser (excluding the variable binding
code) in a tight loop with inputs and parameters as given are promising:

  - 40% faster on lix's package.nix at 10000 iterations
  - 1.3% faster on nixpkgs all-packages.nix at 1000 iterations
  - equivalent on all of nixpkgs concatenated at 100 iterations
    (excluding invalid files, each file surrounded with parens)

more realistic benchmarks are somewhere in between the extremes, parsing
once again getting the largest uplift. other realistic workloads improve
by a few percentage points as well, notably system builds are 4% faster.

Benchmarks summary (from ./bench/summarize.jq bench/bench-*.json)
old/bin/nix --extra-experimental-features 'nix-command flakes' eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  mean:     0.408s ± 0.025s
            user: 0.355s | system: 0.033s
  median:   0.389s
  range:    0.388s ... 0.442s
  relative: 1

new/bin/nix --extra-experimental-features 'nix-command flakes' eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  mean:     0.332s ± 0.024s
            user: 0.279s | system: 0.033s
  median:   0.314s
  range:    0.313s ... 0.361s
  relative: 0.814

---

old/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  mean:     6.133s ± 0.022s
            user: 5.395s | system: 0.437s
  median:   6.128s
  range:    6.099s ... 6.183s
  relative: 1

new/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  mean:     5.925s ± 0.025s
            user: 5.176s | system: 0.456s
  median:   5.934s
  range:    5.861s ... 5.943s
  relative: 0.966

---

GC_INITIAL_HEAP_SIZE=10g old/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  mean:     4.503s ± 0.027s
            user: 3.731s | system: 0.547s
  median:   4.499s
  range:    4.478s ... 4.541s
  relative: 1

GC_INITIAL_HEAP_SIZE=10g new/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  mean:     4.285s ± 0.031s
            user: 3.504s | system: 0.571s
  median:   4.281s
  range:    4.221s ... 4.328s
  relative: 0.951

---

old/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello
  mean:     16.475s ± 0.07s
            user: 14.088s | system: 1.572s
  median:   16.495s
  range:    16.351s ... 16.536s
  relative: 1

new/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello
  mean:     15.973s ± 0.013s
            user: 13.558s | system: 1.615s
  median:   15.973s
  range:    15.946s ... 15.99s
  relative: 0.97

---

Change-Id: Ie66ec2d045dec964632c6541e25f8f0797319ee2
2024-06-25 12:24:58 +00:00
..
ca chore: rebrand Nix to Lix when it makes sense 2024-06-01 20:31:24 +02:00
common Add build-dir setting, clean up default TMPDIR handling 2024-06-24 11:30:32 +03:00
config Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
dyn-drv remove the autoconf+Make buildsystem 2024-05-07 17:04:30 -06:00
flakes flakes: add --commit-lock-file message test 2024-06-23 17:29:40 +00:00
lang libexpr: rewrite the parser with pegtl instead of flex/bison 2024-06-25 12:24:58 +00:00
nested-sandboxing chore: rebrand Nix to Lix when it makes sense 2024-06-01 20:31:24 +02:00
plugins remove the autoconf+Make buildsystem 2024-05-07 17:04:30 -06:00
repl_characterization libutil: return Pid from startProcess, not pid_t 2024-06-23 11:52:49 +00:00
test-libstoreconsumer remove the autoconf+Make buildsystem 2024-05-07 17:04:30 -06:00
add.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
bad.tar.xz Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
bash-profile.sh Fix failing darwin tests 2024-05-06 18:56:40 +02:00
big-derivation-attr.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
binary-cache-build-remote.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
binary-cache.sh chore: rebrand Nix to Lix when it makes sense 2024-06-01 20:31:24 +02:00
brotli.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
build-delete.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
build-dry.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
build-hook-ca-fixed.nix Merge pull request #9032 from Ma27/structured-attrs-env-vars 2024-03-07 10:46:47 +01:00
build-hook-ca-floating.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
build-hook.nix Merge pull request #9032 from Ma27/structured-attrs-env-vars 2024-03-07 10:46:47 +01:00
build-remote-content-addressed-fixed.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
build-remote-content-addressed-floating.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
build-remote-input-addressed.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
build-remote-trustless-after.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
build-remote-trustless-should-fail-0.sh chore: rebrand Nix to Lix when it makes sense 2024-06-01 20:31:24 +02:00
build-remote-trustless-should-pass-0.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
build-remote-trustless-should-pass-1.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
build-remote-trustless-should-pass-2.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
build-remote-trustless-should-pass-3.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
build-remote-trustless.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
build-remote.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
build.sh nix3-build: show all FOD errors with --keep-going 2024-05-15 15:35:18 +02:00
ca-shell.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
case-hack.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
case.nar Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
check-refs.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
check-refs.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
check-reqs.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
check-reqs.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
check.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
check.sh Add build-dir setting, clean up default TMPDIR handling 2024-06-24 11:30:32 +03:00
common.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
completions.sh Merge pull request #8817 from iFreilicht/flake-update-lock-overhaul 2024-03-25 17:36:24 -06:00
compression-levels.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
compute-levels.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
config.nix.in Merge pull request #9032 from Ma27/structured-attrs-env-vars 2024-03-07 10:46:47 +01:00
config.sh Rename nix show-config to nix config show 2024-05-03 16:26:16 +02:00
db-migration.sh chore: rebrand Nix to Lix when it makes sense 2024-06-01 20:31:24 +02:00
debugger.sh Merge pull request #9933 from pennae/debugger-fix 2024-03-04 08:57:44 +01:00
dependencies.builder0.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
dependencies.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
dependencies.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
derivation-json.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
dummy Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
dump-db.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
eval-store.sh Merge pull request #9589 from obsidiansystems/floating-content-addressing-derivations-eval-store 2024-03-04 07:15:08 +01:00
eval.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
eval.sh libutil/args: warn on unknown settings after parsing all flags 2024-05-30 03:07:21 +00:00
experimental-features.sh Rename nix show-config to nix config show 2024-05-03 16:26:16 +02:00
export-graph.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
export-graph.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
export.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
extra-sandbox-profile.nix libstore: Fix sandbox=relaxed 2024-05-08 19:31:43 +00:00
extra-sandbox-profile.sh libstore: Fix sandbox=relaxed 2024-05-08 19:31:43 +00:00
failing.nix Merge pull request #9032 from Ma27/structured-attrs-env-vars 2024-03-07 10:46:47 +01:00
fetchClosure.sh Add pre-commit checks 2024-03-29 22:57:40 -07:00
fetchGit.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
fetchGitRefs.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
fetchGitSubmodules.sh libfetchers: represent unfetched submodules consistently 2024-06-18 00:54:51 +00:00
fetchMercurial.sh tests: actually run mercurial tests 2024-05-04 16:48:07 +02:00
fetchPath.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
fetchTree-file.sh chore: rebrand Nix to Lix when it makes sense 2024-06-01 20:31:24 +02:00
fetchurl.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
filter-source.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
filter-source.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
fixed.builder1.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
fixed.builder2.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
fixed.nix Merge pull request #9662 from shlevy/flat-fixed-references-assert 2024-03-04 07:50:02 +01:00
fixed.sh Merge pull request #9662 from shlevy/flat-fixed-references-assert 2024-03-04 07:50:02 +01:00
fmt.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
fmt.simple.sh Add pre-commit checks 2024-03-29 22:57:40 -07:00
fod-failing.nix nix3-build: show all FOD errors with --keep-going 2024-05-15 15:35:18 +02:00
function-trace.sh Add pre-commit checks 2024-03-29 22:57:40 -07:00
gc-auto.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
gc-concurrent.builder.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
gc-concurrent.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
gc-concurrent.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
gc-concurrent2.builder.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
gc-non-blocking.sh Merge pull request #8544 from edolstra/handle-missing-gc-socket 2024-03-07 00:43:51 -07:00
gc-runtime.nix gc: Find roots using libproc on Darwin 2024-04-25 23:24:21 -04:00
gc-runtime.sh gc: Find roots using libproc on Darwin 2024-04-25 23:24:21 -04:00
gc.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
hash-check.nix Add pre-commit checks 2024-03-29 22:57:40 -07:00
hash.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
hermetic.nix Merge pull request #9032 from Ma27/structured-attrs-env-vars 2024-03-07 10:46:47 +01:00
ifd.nix Merge branch '2.18-maintenance' into ifd-buildStore-2.18 2024-01-11 07:21:16 -05:00
import-derivation.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
import-derivation.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
impure-derivations.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
impure-derivations.sh chore: rebrand Nix to Lix when it makes sense 2024-06-01 20:31:24 +02:00
impure-eval.sh Merge pull request #4093 from matthewbauer/eval-system 2024-03-04 07:21:01 +01:00
init.sh Fix failing darwin tests 2024-05-06 18:56:40 +02:00
install-darwin.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
lang-test-infra.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
lang.sh Add pre-commit checks 2024-03-29 22:57:40 -07:00
legacy-ssh-store.sh "but doctor, I AM the untrusted store": nix doctor had wrong trustedness 2024-04-08 15:40:12 -07:00
linux-sandbox-cert-test.nix Add pre-commit checks 2024-03-29 22:57:40 -07:00
linux-sandbox.sh libstore/build: copy ca-certificates too 2024-05-31 07:54:18 +00:00
local-store.sh "but doctor, I AM the untrusted store": nix doctor had wrong trustedness 2024-04-08 15:40:12 -07:00
logging.sh Merge pull request #9637 from 9999years/fix-logging-test-on-macos 2024-03-04 07:26:34 +01:00
meson.build Merge "Revert "libfetchers: make attribute / URL query handling consistent"" into main 2024-06-25 10:19:52 +00:00
misc.sh libutil/args: warn on unknown settings after parsing all flags 2024-05-30 03:07:21 +00:00
multiple-outputs.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
multiple-outputs.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
nar-access.nix Add pre-commit checks 2024-03-29 22:57:40 -07:00
nar-access.sh Merge pull request #9288 from edolstra/fix-nar-access-test 2024-03-04 05:27:18 +01:00
nested-sandboxing.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
nix-build-examples.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
nix-build.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
nix-channel.sh Add pre-commit checks 2024-03-29 22:57:40 -07:00
nix-collect-garbage-d.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
nix-copy-ssh-common.sh Merge pull request #9604 from obsidiansystems/test-substitute-on-destination 2024-03-07 03:34:00 +01:00
nix-copy-ssh-ng.sh Merge pull request #9604 from obsidiansystems/test-substitute-on-destination 2024-03-07 03:34:00 +01:00
nix-copy-ssh.sh Merge pull request #9604 from obsidiansystems/test-substitute-on-destination 2024-03-07 03:34:00 +01:00
nix-daemon-untrusting.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
nix-profile.sh libfetchers: log fetches by URL just before they happen 2024-05-24 15:15:42 -06:00
nix-shell.sh Merge pull request #9930 from rvl/print-dev-env-unbound-variables 2024-03-04 08:51:49 +01:00
nix_path.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
optimise-store.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
output-normalization.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
parallel.builder.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
parallel.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
parallel.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
pass-as-file.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
path-from-hash-part.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
path.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
placeholders.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
plugins.sh tests: don't build test plugin shared libs on static builds 2024-05-12 23:04:21 +02:00
post-hook.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
pure-eval.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
pure-eval.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
push-to-store-old.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
push-to-store.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
read-only-store.sh Add pre-commit checks 2024-03-29 22:57:40 -07:00
readfile-context.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
readfile-context.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
recursive.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
recursive.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
referrers.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
remote-store.sh "but doctor, I AM the untrusted store": nix doctor had wrong trustedness 2024-04-08 15:40:12 -07:00
repair.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
repl.sh Pretty-print values in the REPL 2024-03-09 07:20:23 -07:00
restricted.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
restricted.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
search.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
search.sh Merge pull request #9481 from iFreilicht/disallow-nix-search-without-search-terms 2024-03-04 08:40:12 +01:00
secure-drv-outputs.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
secure-drv-outputs.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
selfref-gc.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
shell-hello.nix Merge pull request #9648 from cole-h/nix-shell-ordering 2024-03-04 07:50:55 +01:00
shell.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
shell.sh Merge pull request #9648 from cole-h/nix-shell-ordering 2024-03-04 07:50:55 +01:00
shell.shebang.nix Add pre-commit checks 2024-03-29 22:57:40 -07:00
shell.shebang.rb Add pre-commit checks 2024-03-29 22:57:40 -07:00
shell.shebang.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
signing.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
simple-failing.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
simple.builder.sh Add pre-commit checks 2024-03-29 22:57:40 -07:00
simple.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
simple.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
ssh-relay.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
store-ping.sh Merge pull request #9632 from cole-h/nix-daemon-testing 2024-03-04 08:17:32 +01:00
structured-attrs-shell.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
structured-attrs.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
structured-attrs.sh Merge pull request #9032 from Ma27/structured-attrs-env-vars 2024-03-07 10:46:47 +01:00
substitute-truncated-nar.sh libutil: remove sinkToSource eof callback 2024-06-23 11:52:49 +00:00
substitute-with-invalid-ca.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
suggestions.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
supplementary-groups.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
symlink-derivation.nix Merge pull request #10456 from NixOS/fixpermdeniedbind 2024-04-11 15:43:58 +03:00
tarball.sh Merge pull request #9225 from drupol/nixpkgs-pr-107251/make-gnutar-reproducible 2024-03-04 05:34:19 +01:00
test-infra.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
test-libstoreconsumer.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
test-repl-characterization.sh Implement a parser for a literate testing system for the repl 2024-03-14 14:30:38 -07:00
timeout.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
timeout.sh tests: fix functional-timeout 2024-05-28 04:20:00 +00:00
toString-path.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
undefined-variable.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
user-envs-migration.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
user-envs.builder.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
user-envs.nix Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
user-envs.sh Merge pull request #9257 from Artturin/nixenvjsondrvpath 2024-03-04 05:59:15 +01:00
why-depends.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00
zstd.sh Put functional tests in tests/functional 2023-12-01 12:06:43 -05:00