Commit graph

14957 commits

Author SHA1 Message Date
Eelco Dolstra 16db8dc96f Open slave pseudoterminal before CLONE_NEWUSER
Otherwise, when running as root and user namespaces are enabled,
opening the slave fails with EPERM.

Fixes "opening pseudoterminal slave: Permission denied" followed by a
hang (https://hydra.nixos.org/build/213104244), and "error: getting
sandbox mount namespace: No such file or directory" (#8072), which
happens when the child fails very quickly and consequently reading
/proc/<child>/ns fails.
2023-03-20 17:58:36 +01:00
John Ericson 4607ac7aed Fix handling of experimental features mid-parse
If we conditionally "declare" the argument, as we did before, based upon
weather the feature is enabled, commands like

    nix --experimental-features=foo ... --thing-gated-on-foo

won't work, because the experimental feature isn't enabled until *after*
we start parsing.

Instead, allow arguments to also be associated with experimental
features (just as we did for builtins and settings), and then the
command line parser will filter out the experimental ones.

Since the effects of arguments (handler functions) are performed right
away, we get the required behavior: earlier arguments can enable later
arguments enabled!

There is just one catch: we want to keep non-positional
flags...non-positional. So if

    nix --experimental-features=foo ... --thing-gated-on-foo

works, then

    nix --thing-gated-on-foo --experimental-features=foo ...

should also work.

This is not my favorite long-term solution, but for now this is
implemented by delaying the requirement of needed experimental features
until *after* all the arguments have been parsed.
2023-03-20 11:35:34 -04:00
John Ericson aa663b7e89 Mark experimental features on settings
We hide them in various ways if the experimental feature isn't enabled.

To do this, we had to move the experimental features list out of
libnixstore, because the setting machinary itself depends on it. To do
that, we made a new `ExperimentalFeatureSettings`.
2023-03-20 11:06:40 -04:00
John Ericson 296831f641 Move enabled experimental feature to libutil struct
This is needed in subsequent commits to allow the settings and CLI args
infrastructure itself to read this setting.
2023-03-20 11:05:22 -04:00
John Ericson 1b6c96bbcb Write test, will fail until rest of PR 2023-03-20 09:17:33 -04:00
Eelco Dolstra 83b977ff07
Merge pull request #8075 from edolstra/api-docs
Fix internal-api rendering in Hydra
2023-03-20 13:51:07 +01:00
Eelco Dolstra 658847179a Fix internal-api rendering in Hydra
Currently it gives a 500 error with "Do not know how to serve path
'/nix/store/bym5sm8z2wpavnvzancb9gjdlgyzs1l8-nix-internal-api-docs-2.15.0pre20230320_e37f436/share/doc/nix/internal-api'."
2023-03-20 13:13:57 +01:00
Théophane Hufschmitt 1fcbdecf15
Merge pull request #7968 from bobvanderlinden/pr-contributing
Add CONTRIBUTING.md
2023-03-20 13:13:12 +01:00
Eelco Dolstra e37f436eb4
Merge pull request #8070 from figsoda/quote
docs: quote URL literals
2023-03-20 11:18:09 +01:00
Eelco Dolstra 0d314bbfe8
Merge pull request #8068 from DieracDelta/jr/compile_darwin
Fix build aarch64-darwin for static library
2023-03-20 11:17:46 +01:00
figsoda 54b8da0b10 docs: quote URL literals 2023-03-19 11:28:10 -04:00
Linus Heckemann 3c3bd0767f Create test lockfiles in TEST_ROOT 2023-03-19 14:14:30 +01:00
Linus Heckemann f1c9d83697 Only allow reference lock files when allow-dirty is set 2023-03-19 14:12:49 +01:00
Linus Heckemann ea207a2eed Add tests for alternate lockfile path functionality 2023-03-19 14:11:19 +01:00
Justin Restivo 220aa8e0ac fix: build aarch64-darwin 2023-03-18 17:53:45 -04:00
Robert Hensing b5b6055bbf
Merge pull request #8064 from knedlsepp/flake-in-submodule
Add a test with flake.nix in a git submodule
2023-03-18 15:07:38 +01:00
Robert Hensing bf0c8c34e7
tests/flake-in-submodule: Remove unnecessary rm -rf 2023-03-18 14:39:38 +01:00
Josef Kemetmüller f9c24d67b9 Add a test with flake.nix in a submodule
I noticed a regression in the lazy-trees branch, which I'm trying to
capture with this test. While the tests succeeds in master, the
lazy-trees branch gives the following error message:

    error: access to path
    '/build/nix-test/tests/flakes/flake-in-submodule/rootRepo/submodule/flake.nix'
    is forbidden because it is not under Git control; maybe you should
    'git add' it to the repository
    '/build/nix-test/tests/flakes/flake-in-submodule/rootRepo'?
2023-03-17 22:10:28 +01:00
Eelco Dolstra e53e5c38d4 Add a setting for configuring the SSL certificates file
This provides a platform-independent way to configure the SSL
certificates file in the Nix daemon. Previously we provided
instructions for overriding the environment variable in launchd, but
that obviously doesn't work with systemd. Now we can just tell users
to add

  ssl-cert-file = /etc/ssl/my-certificate-bundle.crt

to their nix.conf.
2023-03-17 18:32:18 +01:00
John Ericson 790dd2555b
Merge pull request #7891 from mupdt/shared-fs-out-links
no-op refactor: extract outLink generation into a function
2023-03-17 10:57:07 -04:00
John Ericson 95dabbadd8 Prioritize testing in the maintainers process docs
PRs that don't increase our ongoing obligations (i.e. by adding new
features) but do increase test coverage of existing features are good
things to merge for the health of the project, and thus good to
prioritize.
2023-03-16 22:27:46 -04:00
John Ericson c998e0172f Move value-only methods to InstallableValue
These methods would previously fail on the other `Installable`s, so
moving them to this class is more correct as to where they actually
work.

Additionally, a `InstallableValueCommand` is created to make it easier
(or rather no worse than before) to write commands that just work on
`InstallableValue`s.

Besides being a cleanup to avoid failing default methods, this gets us
closer to https://github.com/NixOS/rfcs/pull/134.
2023-03-16 20:06:46 -04:00
John Ericson acd707acca
Merge pull request #8057 from obsidiansystems/slight-test-cleanup
Cleanup	test skipping
2023-03-16 19:13:03 -04:00
John Ericson bfb9eb87fe Cleanup test skipping
- Try not to put cryptic "99" in many places

  Factor out `exit 99` into `skipTest` function

- Alows make sure skipping a test is done with a reason

  `skipTest` takes a mandatory argument

- Separate pure conditionals vs side-effectful test skipping.

  "require daemon" already had this, but "sandbox support" did not.
2023-03-16 18:43:03 -04:00
Eelco Dolstra 7f46ebcf90
Merge pull request #8049 from edolstra/unexpected-eof
Fix "unexpected EOF" errors on macOS
2023-03-16 16:13:42 +01:00
Eelco Dolstra 581e11cd55
Merge pull request #8053 from DeterminateSystems/fix-structuredattrs-sandbox
LocalDerivationGoal: set NIX_ATTRS_*_FILE correctly for sandboxed builds
2023-03-16 16:04:30 +01:00
Linus Heckemann af4cbdafe7 LocalDerivationGoal: set NIX_ATTRS_*_FILE correctly for sandboxed builds 2023-03-16 15:26:52 +01:00
John Ericson eb56cb7cc7
Merge pull request #7750 from obsidiansystems/no-args-prepare
Make command infra less stateful and more regular
2023-03-15 17:23:40 -04:00
John Ericson bc23a44c54 Make command infra less stateful and more regular
Already, we had classes like `BuiltPathsCommand` and `StorePathsCommand`
which provided alternative `run` virtual functions providing the
implementation with more arguments. This was a very nice and easy way to
make writing command; just fill in the virtual functions and it is
fairly clear what to do.

However, exception to this pattern were `Installable{,s}Command`. These
two classes instead just had a field where the installables would be
stored, and various side-effecting `prepare` and `load` machinery too
fill them in. Command would wish out those fields.

This isn't so clear to use.

What this commit does is make those command classes like the others,
with richer `run` functions.

Not only does this restore the pattern making commands easier to write,
it has a number of other benefits:

- `prepare` and `load` are gone entirely! One command just hands just
  hands off to the next.

- `useDefaultInstallables` because `defaultInstallables`. This takes
  over `prepare` for the one case that needs it, and provides enough
  flexiblity to handle `nix repl`'s idiosyncratic migration.

- We can use `ref` instead of `std::shared_ptr`. The former must be
  initialized (so it is like Rust's `Box` rather than `Option<Box>`,
  This expresses the invariant that the installable are in fact
  initialized much better.

  This is possible because since we just have local variables not
  fields, we can stop worrying about the not-yet-initialized case.

- Fewer lines of code! (Finally I have a large refactor that makes the
  number go down not up...)

- `nix repl` is now implemented in a clearer way.

The last item deserves further mention. `nix repl` is not like the other
installable commands because instead working from once-loaded
installables, it needs to be able to load them again and again.

To properly support this, we make a new superclass
`RawInstallablesCommand`. This class has the argument parsing and
completion logic, but does *not* hand off parsed installables but
instead just the raw string arguments.

This is exactly what `nix repl` needs, and allows us to instead of
having the logic awkwardly split between `prepare`,
`useDefaultInstallables,` and `load`, have everything right next to each
other. I think this will enable future simplifications of that argument
defaulting logic, but I am saving those for a future PR --- best to keep
code motion and more complicated boolean expression rewriting separate
steps.

The "diagnostic ignored `-Woverloaded-virtual`" pragma helps because C++
doesn't like our many `run` methods. In our case, we don't mind the
shadowing it all --- it is *intentional* that the derived class only
provides a `run` method, and doesn't call any of the overridden `run`
methods.

Helps with https://github.com/NixOS/rfcs/pull/134
2023-03-15 16:29:07 -04:00
John Ericson 0a140a92bb
Merge pull request #7690 from ShamrockLee/nix-hash-sri
nix-hash: support base-64 and SRI format
2023-03-15 16:02:46 -04:00
Yueh-Shun Li c27d358abb nix-hash: support base-64 and SRI format
Add the --base64 and --sri flags for the Base64 and SRI format output.

Add the --base16 flag to explicitly specify the hexadecimal format.

Add the --to-base64 and --to-sri flag to convert a hash to the above
mentioned format.
2023-03-16 03:08:42 +08:00
Eelco Dolstra 1b8c13cbba
Merge pull request #8050 from figsoda/allrefs
respect allRefs=1 when using `nix flake prefetch`
2023-03-15 17:58:01 +01:00
Yueh-Shun Li 08510c5ed0 test/hash.sh: add to-base32 test for nix hash 2023-03-16 00:50:26 +08:00
Yueh-Shun Li 2f32303f93 tests/hash.sh: try: Use FORMAT_FLAG instead of EXTRA
Do not rely on the "multiple format flag specified" behavior.

Explicitly test without the format flag / with the --base16 flag.
2023-03-16 00:49:57 +08:00
figsoda 31d588db8b respect allRefs=1 when using nix flake prefetch 2023-03-15 11:29:01 -04:00
Eelco Dolstra 5eb8bfd0f1 Remove a variable in LocalDerivationGoal 2023-03-15 10:59:10 +01:00
Eelco Dolstra 6029c763c2 Change builderOut from Pipe to AutoCloseFD 2023-03-15 10:59:10 +01:00
Eelco Dolstra cb28e4fe2a Remove "unexpected EOF" retry hack 2023-03-15 10:59:10 +01:00
Eelco Dolstra c536e00c9d Move pseudoterminal slave open to child
Hopefully this fixes "unexpected EOF" failures on macOS
(#3137, #3605, #7242, #7702).

The problem appears to be that under some circumstances, macOS
discards the output written to the slave side of the
pseudoterminal. Hence the parent never sees the "sandbox initialized"
message from the child, even though it succeeded. The conditions are:

* The child finishes very quickly. That's why this bug is likely to
  trigger in nix-env tests, since that uses a builtin builder. Adding
  a short sleep before the child exits makes the problem go away.

* The parent has closed its duplicate of the slave file
  descriptor. This shouldn't matter, since the child has a duplicate
  as well, but it does. E.g. moving the close to the bottom of
  startBuilder() makes the problem go away. However, that's not a
  solution because it would make Nix hang if the child dies before
  sending the "sandbox initialized" message.

* The system is under high load. E.g. "make installcheck -j16" makes
  the issue pretty reproducible, while it's very rare under "make
  installcheck -j1".

As a fix/workaround, we now open the pseudoterminal slave in the
child, rather than the parent. This removes the second condition
(i.e. the parent no longer needs to close the slave fd) and I haven't
been able to reproduce the "unexpected EOF" with this.
2023-03-15 10:58:37 +01:00
Eelco Dolstra 19326ac297 Simplify commonChildInit() 2023-03-14 18:58:12 +01:00
Théophane Hufschmitt 9ec1a3ae60
Merge pull request #7989 from sysedwinistrator/flake-compat-sha256-mr
add flake-compat to flake.nix and use sha256 in default.nix
2023-03-14 17:12:50 +01:00
Linus Heckemann 3a1de4c3fe Apply review suggestions
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2023-03-14 12:02:03 +01:00
Linus Heckemann e91596eb69 Allow specifying alternative paths for reading/writing flake locks
This allows having multiple separate lockfiles for a single
project, which can be useful for testing against different versions of
nixpkgs; it also allows tracking custom input overrides for remote
flakes without requiring local clones of these flakes.

For example, if I want to build Nix against my locally pinned nixpkgs,
and have a lock file tracking this override independently of future
updates to said nixpkgs:

nix flake lock --output-lock-file /tmp/nix-flake.lock --override-input nixpkgs flake:nixpkgs
nix build --reference-lock-file /tmp/nix-flake.lock

Co-Authored-By: Will Fancher <elvishjerricco@gmail.com>
2023-03-13 21:08:52 +01:00
Robert Hensing a387f46967
Merge pull request #8033 from lbodor/stop-adding-dot-to-nix-dev-env-path
`print-dev-env`: stop inadvertently adding `.` to `PATH`
2023-03-13 19:52:41 +01:00
Valentin Gagarin 4a96125c3c
Merge pull request #6338 from Artturin/labeler
add labeler workflow
2023-03-13 17:57:39 +01:00
Artturin 9667516566 add labeler workflow 2023-03-13 18:01:21 +02:00
Lazar Bodor a3a6909bc8 Use $TEST_ROOT 2023-03-14 01:48:12 +11:00
Valentin Gagarin d37b8a29ce
Merge pull request #8036 from Freed-Wu/patch-1 2023-03-13 14:35:09 +01:00
Eelco Dolstra 879472e414
Merge pull request #8040 from NixOS/macos-warning
Fix macOS warning
2023-03-13 14:15:54 +01:00
Eelco Dolstra 6f22e8b7d8
Merge pull request #7486 from fricklerhandwerk/doc-reference
reword definition of "reference"
2023-03-13 14:04:14 +01:00