Commit graph

11337 commits

Author SHA1 Message Date
Eelco Dolstra 04432f2510 Add examples 2022-01-18 17:37:04 +01:00
Eelco Dolstra 5fe1ec8a05 Add a test for 'nix store copy-log' and 'nix log' 2022-01-18 17:30:50 +01:00
Eelco Dolstra 3876238546 Add Installable::toDrvPaths()
This is needed to get the path of a derivation that might not exist
(e.g. for 'nix store copy-log').

InstallableStorePath::toDerivedPaths() cannot be used for this because
it calls readDerivation(), so it fails if the store doesn't have the
derivation.
2022-01-18 17:28:18 +01:00
Eelco Dolstra 5b243a2b4b BinaryCacheStore: Implement addBuildLog() 2022-01-18 16:14:01 +01:00
Eelco Dolstra 4dda1f92aa Add command 'nix store copy-log'
Fixes #5222.
2022-01-18 14:08:49 +01:00
Eelco Dolstra 6448ea84ab Factor out --from / --to logic 2022-01-18 14:08:49 +01:00
Eelco Dolstra f6f0bcf11f
Doh 2022-01-18 14:06:51 +01:00
Eelco Dolstra 50be51d9a8 Doh 2022-01-18 13:50:25 +01:00
Eelco Dolstra 3157028fc1
Merge pull request #5932 from edolstra/remove-shared-strings
Remove shared strings
2022-01-18 11:14:18 +01:00
Eelco Dolstra d62a9390fc Get rid of std::shared_ptr<std::string> and ref<std::string>
These were needed back in the pre-C++11 era because we didn't have
move semantics. But now we do.
2022-01-18 11:12:30 +01:00
Théophane Hufschmitt dc4730ee94
Merge pull request #5720 from tomberek/flake_search
flakes: search up to git or filesystem boundary
2022-01-18 10:59:12 +01:00
regnat 5753f6efbb Fix the rendering of the example directory tree 2022-01-18 10:55:00 +01:00
Eelco Dolstra 52ee7ec002 StringSource: Use std::string_view 2022-01-17 22:20:23 +01:00
Eelco Dolstra 776eb97a43 serialise.hh: Use std::string_view 2022-01-17 22:20:23 +01:00
Eelco Dolstra fc2443a67c
Merge pull request #5812 from pennae/small-perf-improvements
improve parser performance a bit
2022-01-17 19:49:52 +01:00
Alexander Bantyev 34b66aab00
Update documentation for paths on command line
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-01-17 20:00:04 +03:00
Eelco Dolstra 008ddef4b0
Merge pull request #5915 from NixOS/disallow-boost-reference
Forbid runtime references to boost
2022-01-17 10:14:45 +01:00
Eelco Dolstra bb24d1edd7
Merge pull request #5914 from NixOS/5911-fix-nix-why-depends-pager
Start the pager early-enough in `nix why-depends`
2022-01-17 10:10:31 +01:00
Rok Garbas e1720b8e9d
Merge pull request #5919 from johnae/var-tmp-in-docker-image
docker: also create var/tmp as some tools rely on it
2022-01-16 21:01:47 +01:00
John Axel Eriksson 84507daaaa docker: var/tmp make add -p option to mkdir to also create parent dirs 2022-01-15 14:11:37 +01:00
regnat dd3aa1e515 Remove the references to boost on darwin 2022-01-15 11:34:15 +01:00
John Axel Eriksson 3fff0196cd docker: also create var/tmp as some tools rely on it 2022-01-15 10:20:18 +01:00
Rok Garbas 34f2eebd5b
Merge pull request #5916 from NixOS/5896-git-in-docker-image
Add git to the docker image
2022-01-14 21:28:39 +01:00
Alexander Bantyev b9f5dccdbe
Check that we don't search past a git repo 2022-01-14 18:03:47 +03:00
Alexander Bantyev f055cc5a0b
Document searching upwards and fix documentation for installables 2022-01-14 17:58:03 +03:00
regnat 1dace02866 Add git to the docker image
Fix #5896

See https://github.com/NixOS/docker/issues/33
2022-01-14 15:55:36 +01:00
regnat de4489a672 Forbid runtime references to boost
We explicitly hack around to remove them, so might as well check that
the hack is useful.

(Introduced because I feared that the changes of
https://github.com/NixOS/nix/pull/5906#discussion_r784810238 would bring
back some runtime references)
2022-01-14 15:41:14 +01:00
Alexander Bantyev e3690ab393
Add more tests for flake upward searching 2022-01-14 17:21:18 +03:00
tomberek 2dead20924
Apply suggestions from code review
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-01-14 09:16:34 -05:00
regnat 81cd0a113b Start the pager early-enough in nix why-depends
`nix why-depends` is piping its output into a pager by default.
However the pager was only started after the first path is printed,
causing it to be excluded from the pager output.

(Actually the pager was started *inside* the recursive function that was
printing the dependency chain, so a new instance was started at each
level. It’s a little miracle that it worked at all).

Fix #5911
2022-01-14 13:41:45 +01:00
pennae 72f42093e7 optimize unescapeStr
mainly to avoid an allocation and a copy of a string that can be
modified in place (ever since EvalState holds on to the buffer, not the
generated parser itself).

 # before

Benchmark 1: nix search --offline nixpkgs hello
  Time (mean ± σ):     571.7 ms ±   2.4 ms    [User: 563.3 ms, System: 8.0 ms]
  Range (min … max):   566.7 ms … 579.7 ms    50 runs

Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  Time (mean ± σ):     376.6 ms ±   1.0 ms    [User: 345.8 ms, System: 30.5 ms]
  Range (min … max):   374.5 ms … 379.1 ms    50 runs

Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      2.922 s ±  0.006 s    [User: 2.707 s, System: 0.215 s]
  Range (min … max):    2.906 s …  2.934 s    50 runs

 # after

Benchmark 1: nix search --offline nixpkgs hello
  Time (mean ± σ):     570.4 ms ±   2.8 ms    [User: 561.3 ms, System: 8.6 ms]
  Range (min … max):   564.6 ms … 578.1 ms    50 runs

Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  Time (mean ± σ):     375.4 ms ±   1.3 ms    [User: 343.2 ms, System: 31.7 ms]
  Range (min … max):   373.4 ms … 378.2 ms    50 runs

Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      2.925 s ±  0.006 s    [User: 2.704 s, System: 0.219 s]
  Range (min … max):    2.910 s …  2.942 s    50 runs
2022-01-13 18:06:15 +01:00
pennae 34e3bd10e3 avoid copies of parser input data
when given a string yacc will copy the entire input to a newly allocated
location so that it can add a second terminating NUL byte. since the
parser is a very internal thing to EvalState we can ensure that having
two terminating NUL bytes is always possible without copying, and have
the parser itself merely check that the expected NULs are present.

 # before

Benchmark 1: nix search --offline nixpkgs hello
  Time (mean ± σ):     572.4 ms ±   2.3 ms    [User: 563.4 ms, System: 8.6 ms]
  Range (min … max):   566.9 ms … 579.1 ms    50 runs

Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  Time (mean ± σ):     381.7 ms ±   1.0 ms    [User: 348.3 ms, System: 33.1 ms]
  Range (min … max):   380.2 ms … 387.7 ms    50 runs

Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      2.936 s ±  0.005 s    [User: 2.715 s, System: 0.221 s]
  Range (min … max):    2.923 s …  2.946 s    50 runs

 # after

Benchmark 1: nix search --offline nixpkgs hello
  Time (mean ± σ):     571.7 ms ±   2.4 ms    [User: 563.3 ms, System: 8.0 ms]
  Range (min … max):   566.7 ms … 579.7 ms    50 runs

Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  Time (mean ± σ):     376.6 ms ±   1.0 ms    [User: 345.8 ms, System: 30.5 ms]
  Range (min … max):   374.5 ms … 379.1 ms    50 runs

Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      2.922 s ±  0.006 s    [User: 2.707 s, System: 0.215 s]
  Range (min … max):    2.906 s …  2.934 s    50 runs
2022-01-13 18:06:15 +01:00
pennae eee0bcee22 avoid allocations in SymbolTable::create
speeds up parsing by ~3%, system builds by a bit more than 1%

 # before

Benchmark 1: nix search --offline nixpkgs hello
  Time (mean ± σ):     574.7 ms ±   2.8 ms    [User: 566.3 ms, System: 8.0 ms]
  Range (min … max):   569.2 ms … 580.7 ms    50 runs

Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  Time (mean ± σ):     394.4 ms ±   0.8 ms    [User: 361.8 ms, System: 32.3 ms]
  Range (min … max):   392.7 ms … 395.7 ms    50 runs

Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      2.976 s ±  0.005 s    [User: 2.757 s, System: 0.218 s]
  Range (min … max):    2.966 s …  2.990 s    50 runs

 # after

Benchmark 1: nix search --offline nixpkgs hello
  Time (mean ± σ):     572.4 ms ±   2.3 ms    [User: 563.4 ms, System: 8.6 ms]
  Range (min … max):   566.9 ms … 579.1 ms    50 runs

Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  Time (mean ± σ):     381.7 ms ±   1.0 ms    [User: 348.3 ms, System: 33.1 ms]
  Range (min … max):   380.2 ms … 387.7 ms    50 runs

Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      2.936 s ±  0.005 s    [User: 2.715 s, System: 0.221 s]
  Range (min … max):    2.923 s …  2.946 s    50 runs
2022-01-13 18:06:15 +01:00
pennae 61a9d16d5c don't strdup tokens in the lexer
every stringy token the lexer returns is turned into a Symbol and not
used further, so we don't have to strdup. using a string_view is
sufficient, but due to limitations of the current parser we have to use
a POD type that holds the same information.

gives ~2% on system build, 6% on search, 8% on parsing alone

 # before

Benchmark 1: nix search --offline nixpkgs hello
  Time (mean ± σ):     610.6 ms ±   2.4 ms    [User: 602.5 ms, System: 7.8 ms]
  Range (min … max):   606.6 ms … 617.3 ms    50 runs

Benchmark 2: nix eval -f hackage-packages.nix
  Time (mean ± σ):     430.1 ms ±   1.4 ms    [User: 393.1 ms, System: 36.7 ms]
  Range (min … max):   428.2 ms … 434.2 ms    50 runs

Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      3.032 s ±  0.005 s    [User: 2.808 s, System: 0.223 s]
  Range (min … max):    3.023 s …  3.041 s    50 runs

 # after

Benchmark 1: nix search --offline nixpkgs hello
  Time (mean ± σ):     574.7 ms ±   2.8 ms    [User: 566.3 ms, System: 8.0 ms]
  Range (min … max):   569.2 ms … 580.7 ms    50 runs

Benchmark 2: nix eval -f hackage-packages.nix
  Time (mean ± σ):     394.4 ms ±   0.8 ms    [User: 361.8 ms, System: 32.3 ms]
  Range (min … max):   392.7 ms … 395.7 ms    50 runs

Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      2.976 s ±  0.005 s    [User: 2.757 s, System: 0.218 s]
  Range (min … max):    2.966 s …  2.990 s    50 runs
2022-01-13 18:06:14 +01:00
Eelco Dolstra 5e9653c370 Tweak 2022-01-13 14:33:41 +01:00
Eelco Dolstra 66424cd29f Merge branch 'patch-1' of https://github.com/TheodorRene/nix 2022-01-13 14:31:49 +01:00
Eelco Dolstra a2811f8499
Merge pull request #5903 from Profpatsch/release-notes-2.4-add-indirect-noop
Release Notes 2.4: add `--indirect` no-op change
2022-01-13 11:18:41 +01:00
Profpatsch a51c457204 Release Notes 2.4: add --indirect no-op change
Since
00d25e8457
which was first included in nix 2.4.

It is a backwards-compatible change since the flag will just be
ignored.
2022-01-12 23:19:15 +01:00
Théophane Hufschmitt e61c4bc25a
Merge pull request #5887 from pennae/avoid-streams
avoid std::?stream overhead when it's not helpful
2022-01-12 10:52:40 +01:00
pennae 73fcc40fa4 use boost::lexical_cast for string2*
this avoids one copy from `s` into `str`, and possibly another copy needed to
construct `s` at the call site. lexical_cast is also more efficient in general.
2022-01-12 10:07:21 +01:00
pennae 5838354d34 optimize ExprConcatStrings::eval
constructing an ostringstream for non-string concats (like integer addition) is
a small constant cost that we can avoid. for string concats we can keep all the
string temporaries we get from coerceToString and concatenate them in one go,
which saves a lot of intermediate temporaries and copies in ostringstream. we
can also avoid copying the concatenated string again by directly allocating it
in GC memory and moving ownership of the concatenated string into the target
value.

saves about 2% on system eval.

before:

  Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.837 s ±  0.031 s    [User: 2.562 s, System: 0.191 s]
    Range (min … max):    2.796 s …  2.892 s    20 runs

after:

  Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.790 s ±  0.035 s    [User: 2.532 s, System: 0.187 s]
    Range (min … max):    2.722 s …  2.836 s    20 runs
2022-01-12 10:07:21 +01:00
Theodor René Carlsen 89b4df8d92
Add link to explanation when introducing a new operator
The logical implication operator is included in this section but never explained. It might stump new readers with a pretty uncommon operator, and it's never referenced explicitly.
2022-01-11 17:01:43 +01:00
Théophane Hufschmitt d023903b6f
Merge pull request #5883 from fzakaria/faridzakaria/check-meta
Check for `--meta` before emitting it for `--json`
2022-01-11 16:57:28 +01:00
Eelco Dolstra 8a446aff75
Merge pull request #5898 from layus/repair-path-links
Make --repair-path also repair corrupt optimised links
2022-01-11 14:14:44 +01:00
Guillaume Maudoux 9f9f39a24b Prefer RepairFlag over bool when applicable 2022-01-11 13:38:28 +01:00
Guillaume Maudoux e9a4abdb5d Make --repair-path also repair corrupt optimised links
There already existed a smoke test for the link content length,
but it appears that there exists some corruptions pernicious enough
to replace the file content with zeros, and keeping the same length.

--repair-path now goes as far as checking the content of the link,
making it true to its name and actually repairing the path for such
coruption cases.
2022-01-11 11:57:45 +01:00
Eelco Dolstra f39d94a55b Merge branch 'readfile' of https://github.com/abbradar/nix 2022-01-10 16:55:28 +01:00
Eelco Dolstra 19fd6e585d 'target' points to 'source' 2022-01-10 16:52:25 +01:00
Eelco Dolstra df11e75d0e Merge branch 'recursive-nix-fix' of https://github.com/L-as/nix 2022-01-10 16:51:59 +01:00
Nikolay Amiantov 120ca245d1 Add test for readFile keeping context 2022-01-09 13:42:36 +03:00