Commit graph

13843 commits

Author SHA1 Message Date
Eelco Dolstra c536e00c9d Move pseudoterminal slave open to child
Hopefully this fixes "unexpected EOF" failures on macOS
(#3137, #3605, #7242, #7702).

The problem appears to be that under some circumstances, macOS
discards the output written to the slave side of the
pseudoterminal. Hence the parent never sees the "sandbox initialized"
message from the child, even though it succeeded. The conditions are:

* The child finishes very quickly. That's why this bug is likely to
  trigger in nix-env tests, since that uses a builtin builder. Adding
  a short sleep before the child exits makes the problem go away.

* The parent has closed its duplicate of the slave file
  descriptor. This shouldn't matter, since the child has a duplicate
  as well, but it does. E.g. moving the close to the bottom of
  startBuilder() makes the problem go away. However, that's not a
  solution because it would make Nix hang if the child dies before
  sending the "sandbox initialized" message.

* The system is under high load. E.g. "make installcheck -j16" makes
  the issue pretty reproducible, while it's very rare under "make
  installcheck -j1".

As a fix/workaround, we now open the pseudoterminal slave in the
child, rather than the parent. This removes the second condition
(i.e. the parent no longer needs to close the slave fd) and I haven't
been able to reproduce the "unexpected EOF" with this.
2023-03-15 10:58:37 +01:00
Eelco Dolstra 19326ac297 Simplify commonChildInit() 2023-03-14 18:58:12 +01:00
Robert Hensing a387f46967
Merge pull request #8033 from lbodor/stop-adding-dot-to-nix-dev-env-path
`print-dev-env`: stop inadvertently adding `.` to `PATH`
2023-03-13 19:52:41 +01:00
Valentin Gagarin 4a96125c3c
Merge pull request #6338 from Artturin/labeler
add labeler workflow
2023-03-13 17:57:39 +01:00
Artturin 9667516566 add labeler workflow 2023-03-13 18:01:21 +02:00
Lazar Bodor a3a6909bc8 Use $TEST_ROOT 2023-03-14 01:48:12 +11:00
Valentin Gagarin d37b8a29ce
Merge pull request #8036 from Freed-Wu/patch-1 2023-03-13 14:35:09 +01:00
Eelco Dolstra 879472e414
Merge pull request #8040 from NixOS/macos-warning
Fix macOS warning
2023-03-13 14:15:54 +01:00
Eelco Dolstra 6f22e8b7d8
Merge pull request #7486 from fricklerhandwerk/doc-reference
reword definition of "reference"
2023-03-13 14:04:14 +01:00
Valentin Gagarin d9f60ffb81
Merge pull request #7534 from fricklerhandwerk/maintainers-goals
add objectives and tasks for Nix team
2023-03-13 14:01:15 +01:00
Eelco Dolstra f32ca06a83
Merge pull request #8037 from edolstra/fix-make-clean
make clean: Delete vars-and-functions.sh
2023-03-13 13:41:24 +01:00
Eelco Dolstra fcd0b0fbd5 Fix macOS warning 2023-03-13 05:31:03 -07:00
Eelco Dolstra f056468959 make clean: Delete vars-and-functions.sh 2023-03-13 13:11:03 +01:00
wzy 5bf4ade0fa
Fix a typo of values.md 2023-03-13 15:34:01 +08:00
Lazar Bodor e210de4799 print-dev-env: test the case when PATH is empty 2023-03-13 17:50:36 +11:00
Lazar Bodor 526bdbda3c print-dev-env: stop inadvertently adding . to PATH 2023-03-12 22:40:47 +11:00
John Ericson 208c855124
Merge pull request #7896 from obsidiansystems/doxygen
Generate API docs with Doxygen
2023-03-10 13:17:43 -05:00
John Ericson 6910f5dcb6 Generate API docs with Doxygen
The motivation is as stated in issue #7814: even though the the C++ API
is internal and unstable, people still want it to be well documented for
sake of learning, code review, and other purposes that aren't predicated
on it being stable.

Fixes #7814

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-03-10 12:51:06 -05:00
Valentin Gagarin 66f49864f3
Merge pull request #7928 from serokell/doc-includes
Documentation: process `#include` directives
2023-03-10 17:06:42 +01:00
Théophane Hufschmitt 99af56cd0d
Merge pull request #8015 from tweag/progress-during-nix-copy
Display progress when running copyPaths (nix copy)
2023-03-10 16:23:22 +01:00
Eelco Dolstra 523913d091
Merge pull request #8020 from edolstra/no-such-process
Make findRuntimeRoots() more resilient to disappearing processes
2023-03-09 19:57:47 +01:00
Eelco Dolstra 4f3a4b732c Make findRuntimeRoots() more resilient to disappearing processes
I saw this random failure in https://hydra.nixos.org/build/211811692:

  error: opening /proc/15307/fd: No such process

while running nix-collect-garbage in a readfile-context.sh. This is
because we're not handling ESRCH errors reading /proc/<pid>/fd. So
just move the read inside the try/catch where we do handle it.
2023-03-09 16:44:51 +01:00
Eelco Dolstra c44750982d
Merge pull request #8019 from edolstra/fix-read-from-stdin
Fix another uninitialized variable
2023-03-09 16:04:32 +01:00
Eelco Dolstra 8f96720b9f Fix another uninitialized variable
https://hydra.nixos.org/build/211811494
2023-03-09 15:30:53 +01:00
Alexander Bantyev 4bef2016a1
Display progress when running copyPaths (nix copy)
`nix copy` operations did not show progress. This is quite confusing.

Add a `progressSink` which displays the progress during `copyPaths`,
pretty much copied from `copyStorePath`.

Fixes https://github.com/NixOS/nix/issues/8000
2023-03-09 16:03:48 +04:00
Eelco Dolstra 1ba40e959b
Merge pull request #8012 from NixOS/revert-7725-check-coro-gc
Revert "Disable GC during coroutine execution + test"
2023-03-09 12:37:38 +01:00
Théophane Hufschmitt d25322e7c0
Merge pull request #5754 from obsidiansystems/harden-tests-set-u
Harden tests' bash
2023-03-08 21:25:30 +01:00
Théophane Hufschmitt 39700c5cbe
Revert "Disable GC during coroutine execution + test" 2023-03-08 20:47:52 +01:00
Eelco Dolstra e8415dc439
Merge pull request #8005 from edolstra/run-installcheck
Run 'make installcheck' again
2023-03-08 17:40:12 +01:00
John Ericson c11836126b Harden tests' bash
Use `set -u` and `set -o pipefail` to catch accidental mistakes and
failures more strongly.

 - `set -u` catches the use of undefined variables
 - `set -o pipefail` catches failures (like `set -e`) earlier in the
   pipeline.

This makes the tests a bit more robust. It is nice to read code not
worrying about these spurious success paths (via uncaught) errors
undermining the tests. Indeed, I caught some bugs doing this.

There are a few tests where we run a command that should fail, and then
search its output to make sure the failure message is one that we
expect. Before, since the `grep` was the last command in the pipeline
the exit code of those failing programs was silently ignored. Now with
`set -o pipefail` it won't be, and we have to do something so the
expected failure doesn't accidentally fail the test.

To do that we use `expect` and a new `expectStderr` to check for the
exact failing exit code. See the comments on each for why.

`grep -q` is replaced with `grepQuiet`, see the comments on that
function for why.

`grep -v` when we just want the exit code is replaced with `grepInverse,
see the comments on that function for why.

`grep -q -v` together is, surprise surprise, replaced with
`grepQuietInverse`, which is both combined.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-03-08 10:26:30 -05:00
Eelco Dolstra 693b1be81f Run 'make installcheck' again
This was failing because the check for the existence of the
'installcheck' target failed silently, so the whole phase got
skipped. It works by running 'make -n installcheck 2> /dev/null',
which however barfs with

  /nix/store/039g378vc3pc3dvi9dzdlrd0i4q93qwf-binutils-2.39/bin/ld.gold: error: cannot open tests/plugins/plugintest.o: No such file or directory

Fixes #8004.
2023-03-08 14:48:29 +01:00
Eelco Dolstra 0159dfad3f
Merge pull request #8007 from edolstra/fix-coverage
Fix coverage job
2023-03-08 14:45:13 +01:00
Eelco Dolstra 1bb81286d1
Merge pull request #8006 from edolstra/fix-read-from-stdin
Fix uninitialized readFromStdIn variable
2023-03-08 14:29:27 +01:00
Eelco Dolstra bda8d7f165 Fix coverage job
https://hydra.nixos.org/build/211747539
2023-03-08 14:20:10 +01:00
Eelco Dolstra 1607ad5076 Fix uninitialized readFromStdIn variable
This was causing random failures in tests/ca/substitute.ca: 'nix copy
--file ./content-addressed.nix' wouldn't get the default installable
'.' applied in InstallablesCommand::load(), so it would do nothing.
2023-03-08 13:53:20 +01:00
Eelco Dolstra df48040786
Merge pull request #7994 from edolstra/fix-ca-crash
Fix crash/hang with CA derivations
2023-03-08 12:15:55 +01:00
Eelco Dolstra 6b243ef2d5
Merge pull request #7993 from tweag/fix-profile-gc
Fix `nix-collect-garbage -d` with the new profile location
2023-03-08 11:10:45 +01:00
Eelco Dolstra 7bfed34367 Fix crash/hang with CA derivations
The curl download can outlive DrvOutputSubstitutionGoal (if some other
error occurs), so at shutdown setting the promise to an exception will
fail because 'this' is no longer valid in the callback. This can
manifest itself as a segfault, "corrupted double-linked list" or hang.
2023-03-08 11:09:15 +01:00
Théophane Hufschmitt 4a6244dcf7
Merge pull request #7725 from yorickvP/check-coro-gc
Disable GC during coroutine execution + test
2023-03-08 09:51:46 +01:00
Théophane Hufschmitt 2272bc6049 Fix nix-collect-garbage -d with the new profile location
Low-hanging fix for https://github.com/NixOS/nix/pull/5226#issuecomment-1454669399
2023-03-07 13:08:04 +01:00
Théophane Hufschmitt ba0486f045
Merge pull request #7889 from sidkshatriya/sorted-fetch-paths
Print the store paths to be fetched sorted by StorePath name()
2023-03-07 11:58:10 +01:00
Eelco Dolstra 02bf521968
Merge pull request #7580 from fricklerhandwerk/nix-team-add-john
add @Ericson2314 to the Nix team
2023-03-06 15:02:07 +01:00
Eelco Dolstra fc76852b66
Merge pull request #7957 from NixOS/update-cachix-install-nix
Switch to cachix/install-nix-action@v20
2023-03-06 11:09:39 +01:00
Eelco Dolstra 3f4dc61f7f
Merge pull request #7958 from amesgen/issue-7955
InstallableFlake: Apply nix config in `getCursors`
2023-03-06 11:08:51 +01:00
Eelco Dolstra 92771ca1be
Merge pull request #7975 from fricklerhandwerk/fix-fetchgit
remove indentation in `fetchGit` attribute listing
2023-03-06 11:00:40 +01:00
Théophane Hufschmitt 2fbb2562c1
Merge pull request #7376 from fricklerhandwerk/installable
clarify definition of "installable"
2023-03-06 10:44:14 +01:00
Valentin Gagarin dfeb83cac1 output paths of store derivations are now addressed explicitly 2023-03-06 03:32:14 +01:00
Valentin Gagarin f69d453c3e remove indentation in fetchGit attribute listing
also reword a confusing sentence and add links to Git terminology
2023-03-05 04:44:11 +01:00
Valentin Gagarin 2af9fd20c6 clarify definition of "installable"
the term was hard to discover, as its definition and explanation were in
a very long document lacking an overview section.
search did not help because it occurs so often.

- clarify wording in the definition
- add an overview of installable types
- add "installable" to glossary
- link to definition from occurrences of the term
- be more precise about where store derivation outputs are processed
- installable Nix expressions must evaluate to a derivation

Co-authored-by: Adam Joseph <54836058+amjoseph-nixpkgs@users.noreply.github.com>
2023-03-05 01:46:17 +01:00
Valentin Gagarin 1e87d5f1ea make descriptions of each installable type an own subsection
this is easier to edit, provides anchors for free, and renders correctly
on the terminal without additional effort.
2023-03-05 01:46:17 +01:00