Commit graph

15267 commits

Author SHA1 Message Date
John Ericson 211b6e1855
Merge pull request #9517 from NixOS/2.18-flatten-tests
[Backport 2.18-maintanence] Backport test source layout reorgs
2023-12-01 13:53:13 -05:00
John Ericson f7f37035c8 Move tests to separate directories, and document
Today, with the tests inside a `tests` intermingled with the
corresponding library's source code, we have a few problems:

- We have to be careful that wildcards don't end up with tests being
  built as part of Nix proper, or test headers being installed as part
  of Nix proper.

- Tests in libraries but not executables is not right:

  - It means each executable runs the previous unit tests again, because
    it needs the libraries.

  - It doesn't work right on Windows, which doesn't want you to load a
    DLL just for the side global variable . It could be made to work
    with the dlopen equivalent, but that's gross!

This reorg solves these problems.

There is a remaining problem which is that sibbling headers (like
`hash.hh` the test header vs `hash.hh` the main `libnixutil` header) end
up shadowing each other. This PR doesn't solve that. That is left as
future work for a future PR.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>

(cherry picked from commit 91b6833686a6a6d9eac7f3f66393ec89ef1d3b57)
(cherry picked from commit a61e42adb528b3d40ce43e07c79368d779a8b624)
2023-12-01 13:05:03 -05:00
John Ericson 30dcc19d1f Put functional tests in tests/functional
I think it is bad for these reasons when `tests/` contains a mix of
functional and integration tests

 - Concepts is harder to understand, the documentation makes a good
   unit vs functional vs integration distinction, but when the
   integration tests are just two subdirs within `tests/` this is not
   clear.

 - Source filtering in the `flake.nix` is more complex. We need to
   filter out some of the dirs from `tests/`, rather than simply pick
   the dirs we want and take all of them. This is a good sign the
   structure of what we are trying to do is not matching the structure
   of the files.

With this change we have a clean:
```shell-session
$ git show 'HEAD:tests'
tree HEAD:tests

functional/
installer/
nixos/
```

(cherry picked from commit 68c81c737571794f7246db53fb4774e94fcf4b7e)
2023-12-01 12:06:43 -05:00
John Ericson 7242521265
Merge pull request #9514 from NixOS/2.18-nixpkgs-bump
[Backport 2.18-maintainence] nixpkgs bump
2023-12-01 11:30:05 -05:00
Robert Hensing 016f936df6 flake.lock: Update
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/31ed632c692e6a36cfc18083b88ece892f863ed4' (2023-09-21)
  → 'github:NixOS/nixpkgs/9ba29e2346bc542e9909d1021e8fd7d4b3f64db0' (2023-11-23)

(cherry picked from commit fe4f573d49a5c47cf9ffd0bd3fe8868104550818)
(cherry picked from commit f01baf5f0657ce58c9c14d7ab5de910f58e423c
2023-12-01 11:08:15 -05:00
Robert Hensing 083b198bb7
Merge pull request #9471 from NixOS/backport-9459-to-2.18-maintenance
[Backport 2.18-maintenance] add path based redirects
2023-11-27 15:15:52 +01:00
Valentin Gagarin 8fb477a6e7 add path based redirects
up to now, those were managed outside of this repo, which as
unsurprisingly a real hassle to deal with if one wanted to prevent URLs
from breaking when moving pages around. this change removes a large part
of the friction involved in moving content in the Nix manual.

possible next steps for further automation:
- check for content that moved and warn if it's not reachable from
  links that were valid prior to a change
- create redirect rules automatically based on this information

(cherry picked from commit 2b7016cc56d12e67de9f1f25b18311866a26a5fe)
2023-11-27 13:29:00 +00:00
John Ericson 05b7e2dc57
Merge pull request #9436 from NixOS/backport-9431-to-2.18-maintenance
[Backport 2.18-maintenance] Add missing `-lrapidcheck` fixing build with shared lib
2023-11-22 09:35:24 -05:00
John Ericson 89a66633b7 Add missing -lrapidcheck fixing build with shared lib
https://github.com/NixOS/nixpkgs/pull/269064 makes rapidcheck be build
as a shared lib, but that broke Nix because the `-lrapidcheck` was
missing. This fixes that (and doesn't break Nix what the library is a
static archive as today).

(cherry picked from commit 46131567da96ffac298b9ec54016b37114b0dfd5)
2023-11-22 04:48:11 +00:00
Théophane Hufschmitt fadad86276
Merge pull request #9406 from NixOS/backport-9390-to-2.18-maintenance
[Backport 2.18-maintenance] fetchTree: clarify docs for shallow flag
2023-11-20 15:38:25 +01:00
DavHau 23775b7364 fetchTree: clarify docs for shallow flag
(cherry picked from commit 796a7eb92d2b0caf75685126adc7460a4c39cfec)
2023-11-20 14:16:03 +00:00
Robert Hensing 8e72a529da
Merge pull request #9366 from NixOS/backport-9187-to-2.18-maintenance
[2.18-maintenance] Backport #9187 apple-virt
2023-11-16 19:44:32 +01:00
Robert Hensing af21431140 libstore: Add apple-virt to system features when available
I'm sure that we'll adjust the implementation over time, but this
at least discerns between an apple silicon bare metal machine and
a tart VM.

(cherry picked from commit 9277eb276bf0a942e88fcf499f6a6b9c262be853)
2023-11-16 18:59:06 +01:00
Eelco Dolstra 184a20ec04
Merge pull request #9334 from fricklerhandwerk/backport-2.18-fix-anchors
backport fix for the `--help` output to 2.18
2023-11-16 15:02:18 +01:00
Valentin Gagarin a68bf15fe6 backport fix for the --help output 2023-11-10 19:22:42 +01:00
John Ericson 78fd621397
Merge pull request #9122 from NixOS/backport-9095-to-2.18-maintenance
[Backport 2.18-maintenance] StorePath: reject names starting with '.'
2023-10-09 10:27:51 -04:00
edef 82040664e4 StorePath: reject names starting with '.'
This has been the behaviour before Nix 2.4. It was dropped in a rewrite
in 759947bf72, allowing the creation of
store paths that aren't considered valid by older Nix versions or other
Nix tooling.

Nix 2.4 didn't ship in NixOS until 22.05, and stdenv.mkDerivation in
nixpkgs drops leading periods since April 2022, so it's unlikely anyone
is relying on the current lax behaviour.

Closes #9091.

Change-Id: I4a57bd9899e1b0dba56870ae5a1b680918a18ce9
(cherry picked from commit 24bda0c7b381e1a017023c6f7cb9661fae8560bd)
2023-10-08 12:21:18 +00:00
Eelco Dolstra f5f4de6a55
Merge pull request #9086 from NixOS/backport-9081-to-2.18-maintenance
[Backport 2.18-maintenance] Revert "Adapt scheduler to work with dynamic derivations"
2023-10-02 17:05:54 +02:00
John Ericson 72b65981f9 Revert "Adapt scheduler to work with dynamic derivations"
This reverts commit 5e3986f59c. This
un-implements RFC 92 but fixes the critical bug #9052 which many people
are hitting. This is a decent stop-gap until a minimal reproduction of
that bug is found and a proper fix can be made.

Mostly fixed #9052, but I would like to leave that issue open until we
have a regression test, so I can then properly fix the bug (unbreaking
RFC 92) later.

(cherry picked from commit 8440afbed756254784d9fea3eaab06649dffd390)
2023-10-02 15:05:23 +00:00
Eelco Dolstra 7e2399b123
Merge pull request #9079 from hercules-ci/backport-9022-to-2.18-maintenance
[Backport 2.18-maintenance] pathExists: isDir when endswith /.
2023-10-02 14:08:25 +02:00
Robert Hensing 860f64c345 pathExists: isDir when endswith /.
(cherry picked from commit f8a3893e8d77ce4a6e23719a0b2d88464cb84b9c)
2023-10-01 13:58:20 +01:00
Eelco Dolstra 9e212344f9
Merge pull request #9036 from NixOS/backport-9026-to-2.18-maintenance
[Backport 2.18-maintenance] flake: complete update to 23.05
2023-09-25 15:05:25 +02:00
Gerg-L 277ba90779 flake: complete update to 23.05
(cherry picked from commit f264d9ff0862277523a207c7afaed4894a40dc11)
2023-09-25 08:43:54 +00:00
Robert Hensing a4445859ab
Merge pull request #9009 from NixOS/backport-9008-to-2.18-maintenance
[Backport 2.18-maintenance] Re-enable systemd-nspawn test
2023-09-21 13:13:16 +02:00
Cole Helbling f3005632c4 Re-enable systemd-nspawn test
It was disabled in c6953d1ff6 because
a recent Nixpkgs bump brought in a new systemd which changed how
systemd-nspawn worked.

As far as I can tell, the issue was caused by this upstream systemd
commit:
b71a0192c0

Bind-mounting the host's `/sys` and `/proc` into the container's
`/run/host/{sys,proc}` fixes the issue and allows the test to succeed.

(cherry picked from commit 883092e3f78d4efb1066a2e24e343b307035a04c)
2023-09-20 17:03:47 +00:00
Eelco Dolstra 40a014416b Bump version 2023-09-20 15:20:52 +02:00
Eelco Dolstra 44fb119218 Mark official release 2023-09-20 12:49:01 +02:00
Eelco Dolstra d40e91440d
Merge pull request #9002 from NixOS/release-notes
Release notes
2023-09-20 12:41:30 +02:00
Eelco Dolstra 10ad052f7d Release notes 2023-09-20 11:42:49 +02:00
Eelco Dolstra a4a7eca679
Merge pull request #8991 from NixOS/dependabot/github_actions/docker/login-action-3
Bump docker/login-action from 2 to 3
2023-09-20 11:37:36 +02:00
Eelco Dolstra 6ce358b13c
Merge pull request #8569 from vcunat/p/flake-update
flake: update nixpkgs: 22.11 -> 23.05
2023-09-20 11:30:55 +02:00
Eelco Dolstra b6b2a0aea9 Use "touch -h"
https://hydra.nixos.org/build/235888160

This is needed because Nixpkgs now contains dangling symlinks
(pkgs/test/nixpkgs-check-by-name/tests/symlink-invalid/pkgs/by-name/fo/foo/foo.nix).
2023-09-19 17:21:07 +02:00
Eelco Dolstra c6953d1ff6 Disable systemd-nspawn test
This is broken because of a change in systemd in NixOS 23.05. It fails
with

  Failed to mount proc (type proc) on /proc (MS_NOSUID|MS_NODEV|MS_NOEXEC ""): Operation not permitted
2023-09-19 17:03:21 +02:00
Eelco Dolstra 126e2645f2 Disable rapidcheck tests in the coverage run
https://hydra.nixos.org/build/233688539
2023-09-19 16:04:00 +02:00
Eelco Dolstra c8afa01bc2 Try aws-sdk-cpp fix 2023-09-19 14:51:50 +02:00
Eelco Dolstra c451b48993 Merge remote-tracking branch 'origin/master' into p/flake-update 2023-09-19 13:33:56 +02:00
dependabot[bot] dd3bf4dbda
Bump docker/login-action from 2 to 3
Bumps [docker/login-action](https://github.com/docker/login-action) from 2 to 3.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-18 22:16:55 +00:00
Théophane Hufschmitt 2a52ec4e92
Merge pull request #7661 from henrik-ch/repl-doc
improved help command listing.
2023-09-14 16:43:02 +02:00
Théophane Hufschmitt b99fdcf8db
Merge pull request #8959 from maralorn/print-parent
Print parent activity field in json log
2023-09-13 11:41:47 +02:00
Emil Nikolov 21783cff16
docs: make the nix develop --command example unambiguous (#8952) 2023-09-12 17:15:36 +02:00
John Ericson d98337d18f
Merge pull request #8962 from mkenigs/dead-code
Drop dead code
2023-09-11 10:03:32 -07:00
Matthew Kenigsberg 07545add53
Drop dead code
localPath is unused
2023-09-10 12:18:03 +02:00
maralorn 682dbcab9a
Print parent activity field in json log 2023-09-09 18:01:10 +02:00
Emil Nikolov 2cdc9c32e7
docs: fixed the default priority of nix-env --install (#8945) 2023-09-09 06:54:39 +00:00
thenbe 5473e10249
fix: nix shell multiple commands example (#8950)
The `-c` flag belongs to `sh` not `nix shell`. As it stands, the command errors with:

```
$ nix shell nixpkgs#gnumake --command sh --command "cd src && make"
sh: --command: invalid option
```

https://github.com/NixOS/nix/pull/8276 was good for readability, but it missed this since that PR used a find/replace script.
2023-09-07 22:25:32 +00:00
Robert Hensing 5f55c33917
Merge pull request #8944 from fricklerhandwerk/fix-anchor
fix invalid anchor link
2023-09-07 17:58:16 +02:00
Robert Hensing e34493a70e
Merge pull request #4628 from obsidiansystems/dynamic-drvs
Dynamic derivations RFC 92
2023-09-07 17:33:02 +02:00
John Ericson 37d6fff113
Merge pull request #8938 from obsidiansystems/better-drv-parsing
Improve derivation parsing
2023-09-07 11:07:02 -04:00
John Ericson 80d7994f52 Special-case error message to add extra information
The Derivation parser and old ATerm unfortunately leaves few ways to get
nice errors when an old version of Nix encounters a new version of the
format. The most likely scenario for this to occur is with a new client
making a derivation that the old daemon it is communicating with cannot
understand.

The extensions we just created for dynamic derivation deps will add a
version field, solving the problem going forward, but there is still the
issue of what to do about old versions of Nix up to now.

The solution here is to carefully catch the bad error from the daemon
that is likely to indicate this problem, and add some extra context to
it.

There is another "Ugly backwards compatibility hack" in
`remote-store.cc` that also works by transforming an error.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-09-07 10:39:37 -04:00
John Ericson 7ad66cb3ef Allow dynamic derivation deps in inputDrvs
We use the same nested map representation we used for goals, again in
order to save space. We might someday want to combine with `inputDrvs`,
by doing `V = bool` instead of `V = std::set<OutputName>`, but we are
not doing that yet for sake of a smaller diff.

The ATerm format for Derivations also needs to be extended, in addition
to the in-memory format. To accomodate this, we added a new basic
versioning scheme, so old versions of Nix will get nice errors. (And
going forward, if the ATerm format changes again the errors will be even
better.)

`parsedStrings`, an internal function used as part of parsing
derivations in A-Term format, used to consume the final `]` but expect
the initial `[` to already be consumed. This made for what looked like
unbalanced brackets at callsites, which was confusing. Now it consumes
both which is hopefully less confusing.

As part of testing, we also created a unit test for the A-Term format for
regular non-experimental derivations too.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>

Apply suggestions from code review

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-09-07 10:39:37 -04:00