Commit graph

3452 commits

Author SHA1 Message Date
Graham Christensen d5282de131
Merge pull request #1042 from DeterminateSystems/declarative-jobsets-no-jq
GitHubPulls: output sorted json without using jq
2021-10-20 10:33:46 -04:00
Graham Christensen 3c38629fad GitHubPulls: output sorted json without using jq 2021-10-20 10:05:31 -04:00
Eelco Dolstra 2ee584e7ae
Merge pull request #1040 from NixOS/update-api
Update hydra-api.yaml
2021-10-12 16:46:16 +02:00
Eelco Dolstra 2e43a9644f Update hydra-api.yaml 2021-10-12 14:31:46 +02:00
Eelco Dolstra 8755321ec7
Merge pull request #1039 from NixOS/return-flake
/eval: Return more columns
2021-10-12 14:09:52 +02:00
Eelco Dolstra 888ba467fd /eval: Return more columns
Fixes #1038.
2021-10-12 12:41:37 +02:00
Graham Christensen 0be3bed757
Merge pull request #1033 from Mic92/plugins
add basic documentation for all plugins
2021-10-08 09:17:41 -04:00
Jörg Thalheim 243291f9bd add basic documentation for all plugins
Co-authored-by: Pablo Ovelleiro Corral <mail@pablo.tools>
Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-10-03 13:56:44 +02:00
Graham Christensen dcee1a35ad
Merge pull request #1035 from DeterminateSystems/fixup-perlcritic-after-constituents
constituents test: add use warnings
2021-09-30 09:41:27 -04:00
Graham Christensen 3990bd12e9 constituents test: add use warnings
This is causing CI to fail after #1026 merged. #1026 had a green
bill of health, but #1003 increased perlcritic to level 4. #1003
was not part of #1026 so it was not checked at perlcritic level 4.
2021-09-30 09:13:01 -04:00
Graham Christensen 4c8279044b
Merge pull request #1026 from DeterminateSystems/constituents
add api endpoint: /build/<id>/constituents
2021-09-28 09:21:21 -04:00
Eelco Dolstra 19e668b7cd
Merge pull request #981 from jonringer/pure_eval
Allow for evaluator_pure_eval to be configured
2021-09-27 20:25:01 +02:00
Eelco Dolstra 2745226ada
Merge pull request #1003 from DeterminateSystems/perlcritic-level-4
perlcritic: level 4
2021-09-27 20:23:55 +02:00
Eelco Dolstra 37af3e69eb
Merge pull request #1030 from NixOS/ublock
Fix 'Error loading tab: 0 error' with uBlock Origin
2021-09-27 20:21:39 +02:00
Eelco Dolstra b622337186
Merge pull request #1029 from NixOS/fix-make
Fix building $(srcdir)/static
2021-09-27 17:45:01 +02:00
Eelco Dolstra 66375fd2d9 Fix 'Error loading tab: 0 error' with uBlock Origin
/metrics-tab matches a rule in one of uBlock Origin's default
blocklists, so let's name it something else.
2021-09-27 17:42:56 +02:00
Eelco Dolstra 9bb3e30a46 Fix building $(srcdir)/static
Fixes

  mv: cannot move './static/bootstrap-4.3.1-dist' to './static/bootstrap/bootstrap-4.3.1-dist': Directory not empty

when 'make' is called more than once.
2021-09-27 17:01:25 +02:00
Graham Christensen b550145967
Merge pull request #1017 from Ma27/compare-full
jobset-eval: pass `full` parameter to "Compare to..."
2021-09-24 22:00:05 -04:00
Graham Christensen 8d7116522f
Merge pull request #1024 from Ma27/hydra-home-creation
hydra-module: don't use `createHome = true;` to create Hydra's base-dir
2021-09-24 21:59:26 -04:00
Graham Christensen c60c8d10ea add api endpoint: /build/<id>/constituents
Returns a list of constituent builds
2021-09-24 16:30:44 -04:00
Graham Christensen 99161c7c53
Merge pull request #1012 from helsinki-systems/fix/ldap-test
fix ldap nixos test
2021-09-24 14:27:26 -04:00
Graham Christensen 047fbbdd6b
Merge pull request #1021 from NixOS/dependabot/github_actions/cachix/install-nix-action-14
build(deps): bump cachix/install-nix-action from 13 to 14
2021-09-24 14:26:52 -04:00
Jörg Thalheim e985a277b3 migrate to new openldap module syntax 2021-09-23 20:32:09 +02:00
Maximilian Bosch dd2ce84fe5
hydra-module: don't use createHome = true; to create Hydra's base-dir
In NixOS, the user generation script was changed to set the permissions `0700`
to a home-directory that's specified in the `users.users`-submodule with
`createHome` being set to `true`[1].

However, the home-directory of `hydra` is also the base directory of other services using
other users (e.g. `hydra-queue-runner`). With permissions being `0700`, processes with
such a user cannot traverse into `/var/lib/hydra` and thus not into subdirectories.

I guess that this issue was kind of hidden because `hydra-init.service` ensures
proper permissions[2]. However, if `hydra-init.service` is not restarted on a
system-activation, the permissions of `/var/lib/hydra` will be set back to `0700`
by the activation script that runs on each activation.

This has lead to errors like this in `hydra-queue-runner` on my Hydra:

```
Sep 20 09:11:30 hydra hydra-queue-runner[306]: error (ignored): error: cannot unlink '/var/lib/hydra/build-logs/7h/dssz03gazrkqzfmlr5cprd0dvkg4db-squashfs.img.drv': Permission denied
Sep 20 09:11:30 hydra hydra-queue-runner[306]: error (ignored): error: cannot unlink '/var/lib/hydra/build-logs/b9/350vd8jpv1f86i312c9pkdcd2z56aw-squashfs.img.drv': Permission denied
Sep 20 09:11:30 hydra hydra-queue-runner[306]: error (ignored): error: cannot unlink '/var/lib/hydra/build-logs/kz/vlq4v9a1rylcp4fsqqav3lcjgskky4-squashfs.img.drv': Permission denied
Sep 20 09:11:30 hydra hydra-queue-runner[306]: error (ignored): error: cannot unlink '/var/lib/hydra/build-logs/xd/hkjnbbr9jp7364pkn8zpk9v8xapj2c-nix-2.4pre20210917_37cc50f.drv': Permission denied
Sep 20 09:11:30 hydra hydra-queue-runner[306]: error (ignored): error: cannot unlink '/var/lib/hydra/build-logs/zn/9df7225fl8p7iavqqfvlyay4rf0msw-nix-2.4pre20210917_37cc50f.drv': Permission denied
Sep 20 09:11:30 hydra hydra-queue-runner[306]: possibly transient failure building ‘/nix/store/7hdssz03gazrkqzfmlr5cprd0dvkg4db-squashfs.img.drv’ on ‘roflmayr’: error: creating directory '/var/lib/hydra/build-logs': Permission denied
Sep 20 09:11:30 hydra hydra-queue-runner[306]: will retry ‘/nix/store/7hdssz03gazrkqzfmlr5cprd0dvkg4db-squashfs.img.drv’ after 543s
```

Because of that, I decided to remove the `createHome = true;` setting and instead used
`systemd-tmpfiles`[3] which can not only ensure that certain directories
exist, but also proper permissions.

With this change, we can also get rid of the manual setup in
`hydra-init.service` since `systemd-tmpfiles` will be executed by
`switch-to-configuration` before *any* systemd service gets started. On
startup, `systemd-tmpfiles-setup.service` is invoked within
`sysinit.target` being reached, so when `hydra-init.service` gets called
in `multi-user.target`, the structure already exists.

[1] fa0d499dbf
[2] 3cec908738/hydra-module.nix (L260-L262)
[3] https://www.freedesktop.org/software/systemd/man/systemd-tmpfiles.html
2021-09-22 00:08:15 +02:00
dependabot[bot] ac0529cacb
build(deps): bump cachix/install-nix-action from 13 to 14
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 13 to 14.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Commits](https://github.com/cachix/install-nix-action/compare/v13...v14)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-13 15:00:47 +00:00
Maximilian Bosch 255d4a91bc
jobset-eval: pass full parameter to "Compare to..."
When I take a look at *all* failing builds (by clicking at `[...] more
jobs omitted`) and I try to compare the failures to another jobset, I'd
like to still view *all* failing builds in the compare-view.

This wasn't the case before since the `full=`-param was ignored by the
compare-buttons.
2021-09-08 23:23:29 +02:00
Graham Christensen 741590c923 perlcritic: explicitly assign the result of readdir/file reads 2021-09-07 21:55:55 -04:00
Graham Christensen 85bb1c7ef9 (perlcritic) JSON::Any -> JSON::MaybeXS 2021-09-07 21:55:55 -04:00
Graham Christensen c603ae35f0 perlcritic: Don't conditionally declare variables 2021-09-07 21:35:01 -04:00
Graham Christensen 3cec908738
Merge pull request #1016 from DeterminateSystems/fixup/1011
ResultSet::TaskRetries: add missing method, get_retryable_task
2021-09-07 11:20:29 -04:00
Graham Christensen b1879132af ResultSet::TaskRetries: add missing method, get_retryable_task
Yet again, manual testing is proving to be insufficient. I'm pretty
sure I wrote this code but lost it in a rebase, or perhaps the switch
to result classes.

At any rate, this implements the actual "fetch a retry row and run it"
for the hydra-notify daemon.

Tested by hand.
2021-09-07 11:12:10 -04:00
Graham Christensen d6aa3f8d15
Merge pull request #1011 from DeterminateSystems/retryable-notifications
Retryable notifications
2021-09-07 10:04:32 -04:00
Graham Christensen 21e1ff0da1 perlcritic: don't open files as bare words 2021-09-06 22:20:17 -04:00
Graham Christensen efd1d78b97 Hydra::Helper::Nix: getHydraNotifyPrometheusConfig fixup strict warning around double-declaring cfg 2021-09-06 22:13:33 -04:00
Graham Christensen c880888f1e File::Slurp -> File::Slurper 2021-09-06 22:13:33 -04:00
Your Name c54629b732 Helper/Nix.pm: perlcritic statsd in scalar context 2021-09-06 22:13:33 -04:00
Your Name 55b7802b61 t/lib/Setup.pm: nix_config: default to empty string 2021-09-06 22:13:33 -04:00
Your Name 4677a7c894 perlcritic: use strict, use warnings 2021-09-06 22:13:33 -04:00
Your Name d3feb8b9a5 perlcritic: ratchet to severity 4 2021-09-06 21:35:35 -04:00
Graham Christensen dd06ab7a99 Merge pull request #1004 from DeterminateSystems/avoid-a-b
Avoid $a, $b
2021-09-06 21:35:17 -04:00
Graham Christensen ca4b8d449f
BitBucketStatus: tbuild -> build (typo) 2021-09-06 21:12:13 -04:00
Graham Christensen d9df26ac5a
POD: improve wording, punctuation, caps
Co-authored-by: Cole Helbling <cole.helbling@determinate.systems>
2021-09-03 09:13:34 -04:00
Graham Christensen ea3ae0693e Hook up the retryable tasks with hydra-notify 2021-09-02 10:06:26 -04:00
Graham Christensen c0e86faa78 hydra-notify: call TaskRetries->save_task if a task fails 2021-09-02 10:06:26 -04:00
Graham Christensen b0055a23df TaskDispatcher: Support re-queueing tasks, and dropping tasks after 100 failures. 2021-09-02 10:06:26 -04:00
Graham Christensen d0b0fc21b3 ResultSet::TaskRetries: teach about saving tasks 2021-09-02 10:06:26 -04:00
Graham Christensen 147fa4d029 Result::TaskRetries: Teach about requeue 2021-09-02 10:06:26 -04:00
Graham Christensen 42c2d2f387 Hydra::Math: add an exponential_backoff function 2021-09-02 10:06:26 -04:00
Graham Christensen 29738364fb ResultSet::TaskRetries: add get_seconds_to_next_retry
Get the number of seconds before the next retriable task is ready.

This number is specifically intended to be used as a timeout, where
`undef` means never time out.
2021-09-02 10:06:26 -04:00
Graham Christensen c4134c8e84 TaskRetries: init table 2021-09-02 10:06:26 -04:00