Commit graph

301 commits

Author SHA1 Message Date
John Ericson 4ab27e5595 Merge remote-tracking branch 'upstream/master' into libgit2 2023-11-20 08:30:26 -05:00
Robert Hensing fe4f573d49 flake.nix: Update nixpkgs: release-23.05 -> nixos-23.05-small
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/9eb24edd6a0027fed010ccfe300a9734d029983c' (2023-11-01)
  → 'github:NixOS/nixpkgs/decdf666c833a325cb4417041a90681499e06a41' (2023-11-18)
2023-11-19 11:38:58 +01:00
John Ericson 6c8f4ef350 Allow installing unit tests
Closes #9343

See that issue for motivation.

Installing these is disabled by default, but we enable it (and the
additional output we want isntall these too so as not to clutter the
existing ones) to use in cross builds and dev shells.
2023-11-16 09:55:42 -05:00
Eelco Dolstra 21bb180547 Use libgit2 with ssh-exec support
See https://github.com/libgit2/libgit2/pull/6617. This ensures that we
get support for ~/.ssh/config, known_hosts etc.
2023-11-14 13:30:51 +01:00
Eelco Dolstra 98a120b8b8 Merge remote-tracking branch 'origin/master' into libgit2 2023-11-09 16:48:41 +01:00
BootRhetoric 271932782d
fetchGit and flake: add commit signature verification tests
This adds simple tests of the commit signature verification mechanism of
fetchGit and its flake input wrapper.
OpenSSH is added to the build dependencies since it's needed to create
a key when testing the functionality. It is neither a built- nor a
runtime dependency.
2023-11-03 20:23:31 +01:00
Eelco Dolstra 39ea46abb1 Merge remote-tracking branch 'origin/master' into libgit2 2023-11-02 16:16:37 +01:00
John Ericson d7710a40be flake: Temporarily get Nixpkgs ahead of Hydra
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/31ed632c692e6a36cfc18083b88ece892f863ed4' (2023-09-21)
  → 'github:NixOS/nixpkgs/9eb24edd6a0027fed010ccfe300a9734d029983c' (2023-11-01)
2023-11-01 20:09:55 -04:00
Eelco Dolstra ee36a44bf2 GitInputScheme: Use libgit2
This replaces most calls to the "git" binary with libgit2.
2023-10-31 14:52:21 +01:00
Eelco Dolstra a9b8595fd9
Merge pull request #9162 from eclairevoyant/add-mainprogram
explicitly set meta.mainProgram
2023-10-17 14:44:29 +02:00
éclairevoyant e5ce53f3db
explicitly set meta.mainProgram 2023-10-15 15:08:07 -04:00
John Ericson a23cc147cb Factor out Perl bindings Nix package
Progress breaking up `flake.nix` by introducing separate `default.nix`
files which make sense on their own. (This one is a regular
`callPackage`-able package.)
2023-10-13 11:04:51 -04:00
John Ericson 6654b4e3b4 Use positive source filtering for the Perl bindings 2023-10-09 08:29:45 -04:00
John Ericson 47b3508665 Use positive source filtering for the standalone functional tests job
Additionally this skipping of the building is reimplemented to be a bit
more robust and use the same idioms as the functionality for skipping
the tests. In particular, it will now work even if the source files
exist, so we can do this during development too.
2023-10-09 08:29:27 -04:00
John Ericson 644ebaab5f Define NixOS tests in tests/nixos/default.nix rather than flake.nix
I think the our `flake.nix` is currently too large and too scary looking.
I think this matters --- if Nix cannot dog-food itself in a way that is
elegant, why should other people have confidence that their own code can
be elegant and easy to maintain?

We could do this at many points in time, but I think around now, when we
are thinking about stabilizing parts of Flakes, is an especially good
time.

This is a first step to make the `flake.nix` smaller, and make
individual components responsible for their own packaging. I hope we can
do this many more follow-ups like it, until the top-level `flake.nix` is
very small and just coordinates between other things.
2023-10-06 10:58:17 -04:00
John Ericson 68c81c7375 Put functional tests in tests/functional
I think it is bad for these reasons when `tests/` contains a mix of
functional and integration tests

 - Concepts is harder to understand, the documentation makes a good
   unit vs functional vs integration distinction, but when the
   integration tests are just two subdirs within `tests/` this is not
   clear.

 - Source filtering in the `flake.nix` is more complex. We need to
   filter out some of the dirs from `tests/`, rather than simply pick
   the dirs we want and take all of them. This is a good sign the
   structure of what we are trying to do is not matching the structure
   of the files.

With this change we have a clean:
```shell-session
$ git show 'HEAD:tests'
tree HEAD:tests

functional/
installer/
nixos/
```
2023-10-06 09:05:56 -04:00
Théophane Hufschmitt 92e8e1b1bb Poison the build on the test derivation
Make sure that we're not accidentally rebuilding Nix here as it's just
wasteful and awful for CI times.
2023-10-05 16:43:26 +02:00
Théophane Hufschmitt eb68454be6 Don't run the tests that require building if we're not building
A couple of tests require building some libraries that depend on Nix,
and assume it to be built locally.
Don't run these if we only want to run the install tests.

This prevents the CI from rebuilding several times Nix (like in
https://github.com/NixOS/nix/actions/runs/6404422275/job/17384964033#step:6:6412), thus removing a fair amount of build time.
2023-10-05 16:43:26 +02:00
Théophane Hufschmitt c6faef61a6
Merge pull request #8923 from obsidiansystems/test-proto
Unit test some worker protocol serializers
2023-09-26 17:12:24 +02:00
John Ericson 1f3fc08c59
Merge pull request #8887 from obsidiansystems/bsd-cross-ci
Support cross compiling to BSD and CI it
2023-09-25 13:46:55 -04:00
Gerg-L f264d9ff08
flake: complete update to 23.05 2023-09-21 21:00:53 -04:00
John Ericson 0db251e4ad Do not build docs in cross devShell
Coppied from the main build; we really should deduplicate this more.
2023-09-20 09:04:42 -04:00
John Ericson 564392b57b Make libsodium an unconditional dependency
The configure script will not tolerate it being missing.
2023-09-20 09:04:42 -04:00
John Ericson 28850ee900 Make dev shells work for cross
Need to get tools from right package set. Could build clang tools but I
don't want to wait :D.
2023-09-20 09:04:42 -04:00
John Ericson e44d2a6bbe Add FreeBSD and NetBSD cross to Nix's flake 2023-09-20 09:04:42 -04:00
Eelco Dolstra 126e2645f2 Disable rapidcheck tests in the coverage run
https://hydra.nixos.org/build/233688539
2023-09-19 16:04:00 +02:00
Eelco Dolstra c8afa01bc2 Try aws-sdk-cpp fix 2023-09-19 14:51:50 +02:00
John Ericson 7ff43435f9 Unit test some worker protocol serializers
Continue with the characterization testing idioms begun in
c70484454f, but this time for unit tests.

Co-authored-by: Andreas Rammhold <andreas@rammhold.de>
2023-09-05 10:48:02 -04:00
Vladimír Čunát 539cc5e5f0 flake: update nixpkgs: 22.11 -> 23.05
The lowdown input can't be updated; `nix build` would fail to find it.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-09-01 14:07:25 +02:00
Robert Hensing 3384f70a3d nixpkgsLibTests: Only test our Nix
Interface has changed upstream.
It *should* be fine to test 23.05's other Nix versions as those
*should* succeed, but that's not the case and it's obfuscating
our terrible CI setup's log.
2023-08-30 19:34:35 -04:00
p01arst0rm 7d82341633 update system definitions 2023-08-23 19:28:24 +01:00
Robert Hensing 21a188a2b4 Add gc root for nixpkgs/lib content 2023-08-16 16:01:46 +02:00
Robert Hensing 63e0b5d081 GC root for fetched nixpkgs/lib content 2023-08-16 15:46:37 +02:00
Robert Hensing b13fc7101f Add positive source filter
Source filtering is a really cool Nix feature that lets us avoid a
lot of rebuilds, which speeds up the iteration cycle a lot in cases
where the relevant source files aren't actually modified.

We used to have a source filter that marked a few files as irrelevant,
but this is the wrong approach, as we have many more files that are
irrelevant. We may call this negative filtering.

This commit switches the source filtering to positive filtering, which
is a lot more robust. Instead of marking which files we don't need
we marked the files that we do need.

It's a superior approach because it is fail safe. Instead of allowing
build performance problems to creep in over time, we require that all
source inputs are declared.

I shouldn't have to explain that declaring inputs is a good practice,
so I'll stop over-explaining here.

I do have to acknowledge that this will cause a build failure when the
filter is incomplete. This is *good*, because it's the only realistic
way we could be reminded of these problems. These events will be
infrequent, so the small cost of extending the filter is worth it,
compared to the hidden cost of longer dev cycles for things like tests,
docker image, etc, etc.

(Also rebuilding Nix for stupid unnecessary reasons makes my blood boil)
2023-08-16 14:21:59 +02:00
Eelco Dolstra 1ad3328c5e Allow tarball URLs to redirect to a lockable immutable URL
Previously, for tarball flakes, we recorded the original URL of the
tarball flake, rather than the URL to which it ultimately
redirects. Thus, a flake URL like
http://example.org/patchelf-latest.tar that redirects to
http://example.org/patchelf-<revision>.tar was not really usable. We
couldn't record the redirected URL, because sites like GitHub redirect
to CDN URLs that we can't rely on to be stable.

So now we use the redirected URL only if the server returns the
`x-nix-is-immutable` or `x-amz-meta-nix-is-immutable` headers in its
response.
2023-06-13 14:17:45 +02:00
Philipp Otterbein ca6b759f4e fix failing configure in nix-tests 2023-04-09 02:33:53 +02:00
Eelco Dolstra 2425121a59 Remove nixpkgsFor flake output
Fixes "warning: unknown flake output 'nixpkgsFor'".
2023-03-31 16:08:16 +02:00
Théophane Hufschmitt e32ca3cf16
Merge pull request #8018 from tweag/ssh-password-prompt
SSH: don't erase password prompt if it is displayed
2023-03-31 12:06:10 +02:00
Alexander Bantyev 80f9231b69
Filter tests/nixos from source 2023-03-24 14:29:28 +04:00
Alexander Bantyev 85a2d1d94f
Add a test for nix copy over ssh
Check that nix copy can copy stuff, refuses to copy unsigned paths by
default, and doesn't hide the ssh password prompt.
2023-03-22 09:45:08 +04:00
Eelco Dolstra 658847179a Fix internal-api rendering in Hydra
Currently it gives a 500 error with "Do not know how to serve path
'/nix/store/bym5sm8z2wpavnvzancb9gjdlgyzs1l8-nix-internal-api-docs-2.15.0pre20230320_e37f436/share/doc/nix/internal-api'."
2023-03-20 13:13:57 +01:00
Théophane Hufschmitt 9ec1a3ae60
Merge pull request #7989 from sysedwinistrator/flake-compat-sha256-mr
add flake-compat to flake.nix and use sha256 in default.nix
2023-03-14 17:12:50 +01:00
John Ericson 6910f5dcb6 Generate API docs with Doxygen
The motivation is as stated in issue #7814: even though the the C++ API
is internal and unstable, people still want it to be well documented for
sake of learning, code review, and other purposes that aren't predicated
on it being stable.

Fixes #7814

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-03-10 12:51:06 -05:00
Eelco Dolstra 693b1be81f Run 'make installcheck' again
This was failing because the check for the existence of the
'installcheck' target failed silently, so the whole phase got
skipped. It works by running 'make -n installcheck 2> /dev/null',
which however barfs with

  /nix/store/039g378vc3pc3dvi9dzdlrd0i4q93qwf-binutils-2.39/bin/ld.gold: error: cannot open tests/plugins/plugintest.o: No such file or directory

Fixes #8004.
2023-03-08 14:48:29 +01:00
Eelco Dolstra bda8d7f165 Fix coverage job
https://hydra.nixos.org/build/211747539
2023-03-08 14:20:10 +01:00
Edwin Mackenzie-Owen 934431d06c add flake-compat to flake.nix and use sha256 in default.nix 2023-03-06 21:11:24 +01:00
Eelco Dolstra 19c1a4699b
Merge pull request #7946 from cole-h/restore-static-bin-dist
flake: restore binary-dist artifact to Hydra static builds
2023-03-03 10:23:17 +01:00
Cole Helbling a8d0ff1a11 flake: restore binary-dist artifact to Hydra static builds 2023-03-02 10:02:55 -08:00
Eelco Dolstra 9c79ce353d Fix 'make check' inside 'nix develop' 2023-03-02 16:11:49 +01:00
Robert Hensing 892d46adbb
flake.nix: Force the ./configure tests setting
This always forces the setting, rather than relying on its default, and cleans up the code a bit.

Co-authored-by: John Ericson <git@JohnEricson.me>
2023-02-28 15:56:46 +01:00
Robert Hensing 8648ebc2cc Add ./configure --disable-tests option
Building without tests is useful for bootstrapping with a smaller footprint
or running the tests in a separate derivation. Otherwise, we do compile and
run them.

This isn't fine grained as to allow picking `check` but not `installcheck`
or vice versa, but it's good enough for now.

I've tried to use Nixpkgs' `checkInputs`, but those inputs weren't discovered
properly by the configure script. We can emulate its behavior very well though.
2023-02-24 09:50:21 +01:00
Eelco Dolstra c30907829c Fix the static build
It doesn't produce a "debug" output, so the build failed without an
error message in Hydra (https://hydra.nixos.org/build/210121811).
2023-02-22 14:10:07 +01:00
John Ericson 16111aa32e Fix isStatic arguments to commonDeps
Some dependencies supposed to be skipped in the cross build, along with
not using the gold linker. But in https://github.com/NixOS/nix/pull/6538
this was accidentally not preserved.

Also since https://github.com/NixOS/nix/pull/6538 we saw some new
aarch64-linux static build failures. This is a first attempt to try to
fix those failures. If this is not sufficient, there are other things we
can try next.
2023-02-21 10:21:51 -05:00
John Ericson d7a4f08d42
Nix's own flake: Dedup and memoize more
- `nixpkgsFor` does all of native, static, cross, and the different stdenvs.

- The main Nix derivation is no longer duplicated for static.

- DRY nixpkgs.lib and lib.genAttrs calls.
2023-02-20 11:35:51 +01:00
Théophane Hufschmitt 9a3f66d9d9
Merge pull request #7433 from yorickvP/improv-onboarding
Improve hacking.md and add clangd+bear to devshell
2023-02-20 10:50:08 +01:00
Yorick f2e427942d
Improve hacking.md
- Refer to current version in readme
- Split into flakes and non-flakes section
- Change order to move nix-build to the end, since people often start
  with it in the beginning.
- Use proper "Note" syntax
- Add notes about editor integration
- Move information about target platforms and stdenvs into separate
  sections

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Alexander Bantyev <alexander.bantyev@tweag.io>
Co-authored-by: Théophane Hufschmitt <theophane.hufschmitt@tweag.io>
2023-02-13 12:00:00 +04:00
Yorick 012ddaa322
flake.nix: add clangd and bear 2023-02-10 14:29:59 +04:00
Robert Hensing 72b18f05a2 Add a basic daemon authorization test 2023-02-07 16:43:09 +01:00
Théophane Hufschmitt ccaadc9575
Merge pull request #7648 from hercules-ci/move-nixos-tests
Move nixos tests
2023-01-27 15:11:48 +01:00
John Ericson 75892710f8 Fix the coverage job
See https://hydra.nixos.org/build/206790960
2023-01-24 19:19:19 -05:00
Robert Hensing f233fd496d
Merge pull request #7679 from hercules-ci/re-add-boehmgc-patch
Revert "fixup: remove boehmgc patch"
2023-01-24 16:26:47 +01:00
Robert Hensing 8270dccf60 Actually complete the revert 2023-01-24 14:57:18 +01:00
John Ericson a91709a604 Try to fix #7669
The issue *seems* to be the cross jobs, which are missing the `CXXFLAGS`
needed to get rapidcheck.

PR #6538 would be really nice to resurrect which will prevent the
`configureFlags` from going out of sync between the regular build and
the cross build again.
2023-01-23 15:47:26 -05:00
John Ericson 7fe308c2f8 Add rapidcheck dependency for testing
Property tests are great!

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2023-01-23 07:05:50 -05:00
Robert Hensing 261c25601d Use the official, documented NixOS runTest interface 2023-01-20 16:23:52 +01:00
Robert Hensing 74026bb101 tests: Move NixOS tests to tests/nixos
This will allow contributors to find them more easily.
2023-01-20 15:33:13 +01:00
Robert Hensing 620e4fb89b flake.nix: Add nixpkgs/lib/tests as regression test 2023-01-18 01:55:21 +01:00
Robert Hensing be10c09d23 manual: Check links
mdbook-linkcheck is not consistent about its warning setting.
It disables some warnings, but not the warnings about lack of
fragment checking support; hence the extra filtering.
2023-01-10 22:30:41 +01:00
Eelco Dolstra 3dbf9b5af5
Merge pull request #7367 from lheckemann/nixpkgs-22.11
Bump nixpkgs to 22.11
2022-12-23 23:20:48 +01:00
Théophane Hufschmitt 2ec6685eb0 Build Nix with the GC disabled in hydra
Make sure that it still compiles as it's easy to accidentally break one
of the `#if` guarded clauses
2022-12-13 10:44:07 +01:00
Robert Hensing af8136afd4 flake.nix: Use nixpkgs.lib.fileContents 2022-12-07 13:55:31 +01:00
Robert Hensing cc27bd0633 .version: Add newline 2022-12-07 13:39:10 +01:00
Eelco Dolstra e4a2a08b04 Merge remote-tracking branch 'origin/master' into nixpkgs-22.11 2022-11-30 13:09:45 +01:00
Linus Heckemann 9c90452f9d bump nixpkgs to 22.11 beta 2022-11-29 12:06:57 +01:00
Eelco Dolstra f1b5c6876b Add tests for auto-uid-allocation, uid-range and cgroups 2022-11-27 16:38:34 +01:00
Eelco Dolstra dfc3a9afc3
Merge pull request #7043 from edolstra/installer-test
Add an installer test
2022-09-26 17:34:23 +02:00
Théophane Hufschmitt 6a64820cb4
Merge pull request #7082 from rapenne-s/ccache
add ccacheStdenv
2022-09-24 12:45:27 +02:00
Solène Rapenne e04b38f789 add ccacheStdenv
when using ccache, rebuild time has been measured
89% faster while not slowing the speed of cold builds
2022-09-23 11:21:19 +02:00
Eelco Dolstra fae3b4fe8a
Add an installer test
This runs the installer in a QEMU VM. Unlike the old installer test
that ran inside a declaratively built RedHat/Debian image, this uses
an image from Vagrant.
2022-09-14 15:41:11 +02:00
Théophane Hufschmitt 3fca5f6c69 Installer: Reset the timestamps in the tarball
Otherwise it isn't reproducible.

Fix https://github.com/NixOS/nix/issues/7001
2022-09-05 14:44:01 +02:00
Domen Kožar 99208bb8cc curl: patch for netrc regression in Nix 2022-07-14 17:45:02 -05:00
Eelco Dolstra c364e0b098 Fix buildStatic.aarch64-linux
Commit 925b975224 accidentally enabled
use of the gold linker on aarch64-linux, which apparently doesn't work.

https://hydra.nixos.org/build/183500621
2022-07-11 20:57:29 +02:00
Eelco Dolstra 749d914d10
Add reminder to remove nix-static testing hack
https://github.com/NixOS/nix/pull/6708#issuecomment-1165912951
2022-06-24 23:14:56 +02:00
Eelco Dolstra 925b975224 Embed the sandbox shell into the statically linked 'nix' binary
With this, Nix will write a copy of the sandbox shell to /bin/sh in
the sandbox rather than bind-mounting it from the host filesystem.
This makes /bin/sh work out of the box with nix-static, i.e. you no
longer get

  /nix/store/qa36xhc5gpf42l3z1a8m1lysi40l9p7s-bootstrap-stage4-stdenv-linux/setup: ./configure: /bin/sh: bad interpreter: No such file or directory
2022-06-23 04:08:28 +02:00
Eelco Dolstra 8bbbb6e737 Enable/fix tests in nix-static
pkgsStatic is apparently considered a cross environment, so checkPhase
and installCheckPhase are disabled even when we ask for them.
2022-06-22 23:33:15 +02:00
Eelco Dolstra 12e86c0735
Merge pull request #6630 from edolstra/boost-dylib
Remove ${boost}/lib from the RPATH
2022-06-08 17:55:21 +02:00
Eelco Dolstra 4a3f217bde
Remove ${boost}/lib from the RPATH 2022-06-08 15:51:35 +02:00
Théophane Hufschmitt 694b12052a
Merge pull request #6624 from NixOS/new-flake-style
flake.nix: Convert to new naming convention
2022-06-08 13:34:09 +02:00
Eelco Dolstra faf80fa920
Convert to new flake style
https://github.com/NixOS/nix/issues/5532
2022-06-07 14:36:23 +02:00
Eelco Dolstra 0f8754cd30
Fix 22.05 eval warnings 2022-06-07 14:27:10 +02:00
Eelco Dolstra 5a9d83aa59
Disable cross builds on platforms other than x86_64-linux
Needed because evaluation was broken on x86_64-darwin.
2022-06-07 14:27:06 +02:00
Cole Helbling bd0192d0bb flake: update to 22.05
The static build works now :)
2022-05-31 11:51:17 -07:00
Eelco Dolstra 403201e011
Merge pull request #6537 from cole-h/update-flake
flake: use github: reference to nixpkgs
2022-05-31 08:30:42 +02:00
Eelco Dolstra 452dba510d
Mark nix-perl as a Perl module
The call to perl.withPackages in nix-serve expects this.
2022-05-30 14:10:26 +02:00
Eelco Dolstra 948515efb7
Set meta.platforms
'nix-serve' in nixpkgs expects the nix package to set this.
2022-05-30 13:35:28 +02:00
Eelco Dolstra d67a2ae52d
Merge branch 'doc-redirects' of https://github.com/jtojnar/nix 2022-05-27 11:12:41 +02:00
Jan Tojnar 3272afa17b doc: Port anchors preprocessor to jq script
Python is only pulled into the build closure by Mercurial, which might end up being removed.
Let’s port the script to jq, which is more likely to stay.
2022-05-26 18:17:21 +02:00
Eelco Dolstra 5b27040203
Remove unused Perl dependency 2022-05-26 14:49:17 +02:00
Cole Helbling 685107c6c8 flake: use github: reference to nixpkgs
This allows flakes that don't override the Nixpkgs input and also have a
different flake registry.
2022-05-16 11:46:44 -07:00
Eelco Dolstra cb5361628d Add libcxxStdenv devshell 2022-04-28 14:36:48 +02:00