hydra/t
Luke Granger-Brown 67ebce8493 Output evaluation errors without crashing if aggregate job is broken.
At the moment, aggregate jobs can easily break and cause the entire
evaluation to fail, which is not ideal. For Nixpkgs, we do have some
important aggregate jobs (like `tested`), but for debugging and building
purposes it's still useful to get a partial result even if the channel
won't actually advance.

This commit changes the behaviour of hydra-eval-jobs such that it
aggregates any errors found during the construction of an aggregate, and
will instead annotate the job with the evaluation failure such that it
shows up in a "cleaner" way.

There are really two types of failure that we care about: one is where
the attribute just ends up missing altogether in the final output, and
also where the attribute is in the output but fails to evaluate. Both
are handled here.

Note that this does mean that the same error message may be output
multiple times, but this aids debuggability because it'll be much
clearer what's blocking the job from being created.
2021-10-26 10:14:34 +01:00
..
Config perlcritic: use strict, use warnings 2021-09-06 22:13:33 -04:00
Controller constituents test: add use warnings 2021-09-30 09:13:01 -04:00
Event perlcritic: use strict, use warnings 2021-09-06 22:13:33 -04:00
Helper (perl) machines file: support machine lines with multiple spaces between fields 2021-10-24 21:30:53 -04:00
input-types perlcritic: use strict, use warnings 2021-09-06 22:13:33 -04:00
jobs Output evaluation errors without crashing if aggregate job is broken. 2021-10-26 10:14:34 +01:00
lib Test Setup: fix evalSucceeds to actually output evaluation errors 2021-10-26 10:13:58 +01:00
plugins gittea test: prune trailing space 2021-10-20 13:09:39 -04:00
queue-runner Output evaluation errors without crashing if aggregate job is broken. 2021-10-26 10:14:34 +01:00
Schema Merge pull request #1003 from DeterminateSystems/perlcritic-level-4 2021-09-27 20:23:55 +02:00
scripts perlcritic: use strict, use warnings 2021-09-06 22:13:33 -04:00
api-test.t perlcritic: use strict, use warnings 2021-09-06 22:13:33 -04:00
build-products.t perlcritic: use strict, use warnings 2021-09-06 22:13:33 -04:00
evaluate-basic.t perlcritic: use strict, use warnings 2021-09-06 22:13:33 -04:00
evaluate-dependent-jobsets.t perlcritic: use strict, use warnings 2021-09-06 22:13:33 -04:00
Event.t perlcritic: use strict, use warnings 2021-09-06 22:13:33 -04:00
Makefile.am Start checking PRs with perlcritic 2021-08-20 11:06:43 -04:00
Math.t Hydra::Math: add an exponential_backoff function 2021-09-02 10:06:26 -04:00
perlcritic.pl Start checking PRs with perlcritic 2021-08-20 11:06:43 -04:00
PostgresListener.t perlcritic: use strict, use warnings 2021-09-06 22:13:33 -04:00
s3-backup-test.config tests: move to t, allow yath test from root 2021-03-05 09:49:06 -08:00
s3-backup-test.pl perlcritic: explicitly assign the result of readdir/file reads 2021-09-07 21:55:55 -04:00
setup-notifications-jobset.pl perlcritic: use strict, use warnings 2021-09-06 22:13:33 -04:00
TaskDispatcher.t t/TaskDispatcher.t: array / arrayref consistency 2021-10-20 11:38:27 -04:00
test.pl tests: move to t, allow yath test from root 2021-03-05 09:49:06 -08:00