the-distro/hydra
1
0
Fork 0
forked from lix-project/hydra
Code Pull requests Activity
3597 commits 1 branch 0 tags 20 MiB
Commit graph

2688 commits

Author SHA1 Message Date
Graham Christensen
d19890a4e1 JobsetEval/cancel: test & fix 2021-10-27 11:43:36 -04:00
Graham Christensen
ef9a9fa481
Merge pull request #1025 from lukegb/hydra-better-errors 
Produce better errors for failing jobsets
 2021-10-26 12:35:45 -04:00
Luke Granger-Brown
67ebce8493 Output evaluation errors without crashing if aggregate job is broken. 
At the moment, aggregate jobs can easily break and cause the entire
evaluation to fail, which is not ideal. For Nixpkgs, we do have some
important aggregate jobs (like `tested`), but for debugging and building
purposes it's still useful to get a partial result even if the channel
won't actually advance.

This commit changes the behaviour of hydra-eval-jobs such that it
aggregates any errors found during the construction of an aggregate, and
will instead annotate the job with the evaluation failure such that it
shows up in a "cleaner" way.

There are really two types of failure that we care about: one is where
the attribute just ends up missing altogether in the final output, and
also where the attribute is in the output but fails to evaluate. Both
are handled here.

Note that this does mean that the same error message may be output
multiple times, but this aids debuggability because it'll be much
clearer what's blocking the job from being created.
 2021-10-26 10:14:34 +01:00
Graham Christensen
ffedbe5996 restart/cancelBuilds: always pass resultsets explicitly 2021-10-25 22:21:48 -04:00
Graham Christensen
592fef9d82
Merge pull request #1049 from DeterminateSystems/drop-systemtypes 
SystemTypes: drop database table.
 2021-10-24 21:46:57 -04:00
Graham Christensen
802a5dfe68
Merge pull request #1048 from DeterminateSystems/fixup-1003 
Hydra::Helper::Nix::getMachines: add a test
 2021-10-24 21:38:38 -04:00
Graham Christensen
ff888032eb SystemTypes: drop database table. It was originally removed in #65, but put back in fcd511c4de, and now totally unused. 2021-10-24 21:38:04 -04:00
Graham Christensen
5fbf1470bd (perl) machines file: support machine lines with multiple spaces between fields 2021-10-24 21:30:53 -04:00
Graham Christensen
b817124337 Hydra::Helper::Nix::getMachines: add a test 
Fix parsing breakage from #1003: assigning the lines to $lines broke chomp and the filters.

This test validates the parsing works as expected, and also fixes
a minor bug where '-' in features isn't pruned, like in the C++
repo.
 2021-10-23 22:53:55 -04:00
Graham Christensen
8cf158b7a9
Revert "Fix unhelpful error messages in aggregate jobs." 2021-10-23 22:23:29 -04:00
Graham Christensen
3516950d3c
Merge pull request #825 from samueldr/fix/unhelpful-errors-in-aggregates 
Fix unhelpful error messages in aggregate jobs.
 2021-10-23 20:34:50 -04:00
Graham Christensen
d98d74fa4f perlcritic: doesn't like prototypes 2021-10-20 13:09:39 -04:00
Graham Christensen
7095d00608 perlcritic: make all open() calls three-argument 
Two-argument calls end up parsing the second argument to guess what
should happen, three-arg parses ... less?
 2021-10-20 13:09:39 -04:00
Graham Christensen
f0d0358ee4
Merge pull request #1043 from DeterminateSystems/perl-warnings 
Fixup Perl warnings around undefined variables
 2021-10-20 10:48:05 -04:00
Graham Christensen
a887b3d346 fixup! EmailNotification: address Use of uninitialized value in numeric eq (==) 2021-10-20 10:40:08 -04:00
Graham Christensen
a36d23c1dd fixup! BitBucketStatus: address Use of uninitialized value in numeric eq (==) 2021-10-20 10:40:08 -04:00
Graham Christensen
3c38629fad GitHubPulls: output sorted json without using jq 2021-10-20 10:05:31 -04:00
Graham Christensen
30d36da057 CoverityScan: fixup tarballshandle introduced in 21e1ff0da1 2021-10-19 22:04:57 -04:00
Graham Christensen
093e235bf0 GithubRefs: fixup "my" variable $type masks earlier declaration in same scope 2021-10-19 22:02:41 -04:00
Graham Christensen
0f8d02894a hydra-eval-jobset: Scalar value @declInputs[0] better written as $declInputs[0] at hydra-eval-jobset line 570. 2021-10-19 21:52:39 -04:00
Graham Christensen
31cf249aed S3Backup: check for bzip2 and xz Nix Config support (Use of uninitialized value ::Config::xz in concatenation) 2021-10-19 21:52:39 -04:00
Graham Christensen
bf692c9e8c BitBucketStatus: address Use of uninitialized value in numeric eq (==) 2021-10-19 21:52:39 -04:00
Graham Christensen
4fd90ec784 EmailNotification: address Use of uninitialized value in numeric eq (==) 2021-10-19 21:52:39 -04:00
Graham Christensen
01112e9bd3 GitInput: deal with undefined deepClone 2021-10-19 21:52:39 -04:00
Eelco Dolstra
888ba467fd /eval: Return more columns 
Fixes #1038.
 2021-10-12 12:41:37 +02:00
Graham Christensen
4c8279044b
Merge pull request #1026 from DeterminateSystems/constituents 
add api endpoint: /build/<id>/constituents
 2021-09-28 09:21:21 -04:00
Eelco Dolstra
19e668b7cd
Merge pull request #981 from jonringer/pure_eval 
Allow for evaluator_pure_eval to be configured
 2021-09-27 20:25:01 +02:00
Eelco Dolstra
2745226ada
Merge pull request #1003 from DeterminateSystems/perlcritic-level-4 
perlcritic: level 4
 2021-09-27 20:23:55 +02:00
Eelco Dolstra
66375fd2d9 Fix 'Error loading tab: 0 error' with uBlock Origin 
/metrics-tab matches a rule in one of uBlock Origin's default
blocklists, so let's name it something else.
 2021-09-27 17:42:56 +02:00
Eelco Dolstra
9bb3e30a46 Fix building $(srcdir)/static 
Fixes

  mv: cannot move './static/bootstrap-4.3.1-dist' to './static/bootstrap/bootstrap-4.3.1-dist': Directory not empty

when 'make' is called more than once.
 2021-09-27 17:01:25 +02:00
Graham Christensen
c60c8d10ea add api endpoint: /build/<id>/constituents 
Returns a list of constituent builds
 2021-09-24 16:30:44 -04:00
Maximilian Bosch
255d4a91bc
jobset-eval: pass full parameter to "Compare to..." 
When I take a look at *all* failing builds (by clicking at `[...] more
jobs omitted`) and I try to compare the failures to another jobset, I'd
like to still view *all* failing builds in the compare-view.

This wasn't the case before since the `full=`-param was ignored by the
compare-buttons.
 2021-09-08 23:23:29 +02:00
Graham Christensen
741590c923 perlcritic: explicitly assign the result of readdir/file reads 2021-09-07 21:55:55 -04:00
Graham Christensen
85bb1c7ef9 (perlcritic) JSON::Any -> JSON::MaybeXS 2021-09-07 21:55:55 -04:00
Graham Christensen
c603ae35f0 perlcritic: Don't conditionally declare variables 2021-09-07 21:35:01 -04:00
Graham Christensen
b1879132af ResultSet::TaskRetries: add missing method, get_retryable_task 
Yet again, manual testing is proving to be insufficient. I'm pretty
sure I wrote this code but lost it in a rebase, or perhaps the switch
to result classes.

At any rate, this implements the actual "fetch a retry row and run it"
for the hydra-notify daemon.

Tested by hand.
 2021-09-07 11:12:10 -04:00
Graham Christensen
d6aa3f8d15
Merge pull request #1011 from DeterminateSystems/retryable-notifications 
Retryable notifications
 2021-09-07 10:04:32 -04:00
Graham Christensen
21e1ff0da1 perlcritic: don't open files as bare words 2021-09-06 22:20:17 -04:00
Graham Christensen
efd1d78b97 Hydra::Helper::Nix: getHydraNotifyPrometheusConfig fixup strict warning around double-declaring cfg 2021-09-06 22:13:33 -04:00
Graham Christensen
c880888f1e File::Slurp -> File::Slurper 2021-09-06 22:13:33 -04:00
Your Name
c54629b732 Helper/Nix.pm: perlcritic statsd in scalar context 2021-09-06 22:13:33 -04:00
Your Name
4677a7c894 perlcritic: use strict, use warnings 2021-09-06 22:13:33 -04:00
Graham Christensen
dd06ab7a99 Merge pull request #1004 from DeterminateSystems/avoid-a-b 
Avoid $a, $b
 2021-09-06 21:35:17 -04:00
Graham Christensen
ca4b8d449f
BitBucketStatus: tbuild -> build (typo) 2021-09-06 21:12:13 -04:00
Graham Christensen
d9df26ac5a
POD: improve wording, punctuation, caps 
Co-authored-by: Cole Helbling <cole.helbling@determinate.systems>
 2021-09-03 09:13:34 -04:00
Graham Christensen
ea3ae0693e Hook up the retryable tasks with hydra-notify 2021-09-02 10:06:26 -04:00
Graham Christensen
c0e86faa78 hydra-notify: call TaskRetries->save_task if a task fails 2021-09-02 10:06:26 -04:00
Graham Christensen
b0055a23df TaskDispatcher: Support re-queueing tasks, and dropping tasks after 100 failures. 2021-09-02 10:06:26 -04:00
Graham Christensen
d0b0fc21b3 ResultSet::TaskRetries: teach about saving tasks 2021-09-02 10:06:26 -04:00
Graham Christensen
147fa4d029 Result::TaskRetries: Teach about requeue 2021-09-02 10:06:26 -04:00
Graham Christensen
42c2d2f387 Hydra::Math: add an exponential_backoff function 2021-09-02 10:06:26 -04:00
Graham Christensen
29738364fb ResultSet::TaskRetries: add get_seconds_to_next_retry 
Get the number of seconds before the next retriable task is ready.

This number is specifically intended to be used as a timeout, where
`undef` means never time out.
 2021-09-02 10:06:26 -04:00
Graham Christensen
c4134c8e84 TaskRetries: init table 2021-09-02 10:06:26 -04:00
Graham Christensen
1f2adf61aa hydra-notify: extract runPluginsForEvent to a TaskDispatcher 2021-09-02 10:06:26 -04:00
Graham Christensen
fa57fb8f25 hydra.sql: explain update-dbix.pl map 2021-08-26 22:10:19 -04:00
Graham Christensen
397d13a300 DBIx::Class: migrate to use_namespaces 
This gives us a place to put helper functions that act on entire
tables, not just individual records.

This should be a backwards compatible change, except in places we're
manually using result class names.
 2021-08-26 12:37:19 -04:00
Your Name
24467a7bde Nix::getHydraNotifyPrometheusConfig: print errors if the configuration provided is invalid. 2021-08-24 11:35:38 -04:00
Your Name
4ebdcc290e fixup! hydra-notify: pre-declare metrics 2021-08-24 10:57:23 -04:00
Your Name
45e8fa5319 hydra-notify: support sending diagnostic dumps to STDERR on request 2021-08-24 10:56:13 -04:00
Your Name
de2282bcf4 hydra-notify: print out log lines indicating it is or is not launching the exporter 2021-08-24 10:56:13 -04:00
Your Name
5c1228e141 hydra-notify: pre-declare metrics 2021-08-24 10:56:13 -04:00
Your Name
6d7ee27d25 hydra-notify: make the prometheus endpoint configurable, default-off 2021-08-24 10:56:13 -04:00
Your Name
5d0ad5f649 hydra-notify: initial scratch take of prometheus events 2021-08-24 10:56:12 -04:00
Graham Christensen
e9ba0b56f6 BuildFinished : $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen
5e10648a98 CoverityScan: $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen
c929f5cac0 SlackNotification: $build -> $topbuild, $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen
b4f9807290 GithubStatus: $build -> $topbuild, $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen
5c9d8794b1 GiteaStatus: $build -> $topbuild, $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen
3920bf1df3 BitBucketStatus: $build -> $topbuild, $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen
bcf37ab705 GitlabStatus: $build -> $topbuild, $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen
b4a19fd3c8 HIpChatNotification: $build -> $topbuild, $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen
0d4ce15c62 EmailNotification: $build -> $topbuild, $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen
0f88014631 CircleCiNotification: $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen
b6714cdce3 JobsetEval: $a, $b to $left, $right (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen
d0fb3cbf8f Job: $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen
1cda08351f CatalystUtils: rename $b to $builds (perlcritic) 2021-08-20 21:52:32 -04:00
Your Name
4632132d02 fixup perlcritic: indirect new 
> Subroutine new called using indirect syntax at line 14, column 14.  See page 349 of PBP.  (Severity: 5)
 2021-08-20 11:39:19 -04:00
Your Name
6e65c3b320 hydra-notify: fixup printing of build IDs 
Used to print:

    sending notifications for build Hydra::Model::DB::Builds=HASH(0x124cf960)->id...

Now it prints:

    sending notifications for build 123...
 2021-08-16 16:09:05 -04:00
Your Name
2c50227082 hydra-notify: properly call new_event 2021-08-16 15:52:25 -04:00
Your Name
e572a5e576 hydra-notify: use Hydra::Event 2021-08-16 15:52:14 -04:00
Graham Christensen
fda3ed1d4e
Merge pull request #999 from DeterminateSystems/fixup/pr-998 
lib/makefile: include Event/*.pm
 2021-08-16 14:13:18 -04:00
Your Name
cfd3034af8 lib/makefile: include Event/*.pm 2021-08-16 14:04:48 -04:00
Graham Christensen
fa6d7abc13 hydra-notify: move BuildFinished processing to an Event 2021-08-13 16:51:29 -04:00
Graham Christensen
4a1389e36e hydra-notify: move StepFinished processing to an Event 2021-08-13 16:51:29 -04:00
Graham Christensen
4fdb20d3bd hydra-notify: move BuildStarted processing to an Event 2021-08-13 16:51:29 -04:00
Graham Christensen
10e85e3422 hydra-notify: Create a helper for running each plugin on an event 2021-08-13 16:51:29 -04:00
Graham Christensen
d02c6794f4 Event: teach how to execute a plugin 2021-08-13 16:51:29 -04:00
Graham Christensen
4e86e55008 Event.pm: add a new_event helper to parse and construct an Event 
An Event will be part of many Tasks
 2021-08-13 16:51:29 -04:00
Graham Christensen
64a3e75c10 Event: init structure and parse existing messages 2021-08-13 16:51:29 -04:00
Graham Christensen
a14c8ad5f8
Merge pull request #995 from DeterminateSystems/declarative-jobsets-plugin 
Declarative jobsets: move event handling to a plugin
 2021-08-12 15:56:13 -04:00
Graham Christensen
0f958f3425
Merge pull request #997 from DeterminateSystems/abstract-listener 
Abstract over postgres' LISTEN/NOTIFY
 2021-08-12 14:00:34 -04:00
Graham Christensen
5027003285 Abstract over postgres' LISTEN/NOTIFY 
This lets us test the event loop if we wanted, and lets us
test the listening behavior in isolation.
 2021-08-12 13:54:05 -04:00
Graham Christensen
593af41808 Declarative jobsets: move event handling to a plugin 
Declarative jobsets were sort of tucked in to the event hanlder
itself. It turned out that it could have been implemented as a
plugin without much trouble.
 2021-08-12 12:48:18 -04:00
Graham Christensen
9c5f317453 hydra-notify: move buildFinished query in to the function impl 
This is more consistent with the other event handlers, of dealing
with IDs and not objects.
 2021-08-12 12:30:35 -04:00
Eelco Dolstra
5edb58b314 Fix build 2021-08-10 13:47:16 +02:00
Graham Christensen
c7c322545d
Merge pull request #992 from DeterminateSystems/sql/fixup-comment 
hydra.sql: Update comment on regeneration
 2021-08-06 14:54:12 -04:00
Graham Christensen
4169f22231 update-dbix.pl: correct indentation 2021-08-06 14:40:57 -04:00
Graham Christensen
5bd8dc171b hydra.sql: Update comment on regeneration 
We no longer need to generate the hydra-postgres.sql document,
that is a relic from when sqlite was also supported.
 2021-08-06 14:40:34 -04:00
Shay Bergmann
503b0e0b6f
Remove unnecessary comment 2021-07-26 17:16:40 +00:00
Shay Bergmann
173ecb5d92
Apply requested changes to Config.pm 2021-07-26 17:16:40 +00:00