ma27/hydra
0
0
Fork 0
forked from lix-project/hydra
Code Pull requests Activity
3530 commits 4 branches 0 tags 26 MiB
Commit graph

432 commits

Author SHA1 Message Date
Graham Christensen 2cbeca5c44
Merge pull request #1071 from DeterminateSystems/log-fetches-evals 
hydra-eval-jobset: log fetches and evaluations
 2021-12-08 16:00:29 -05:00
Graham Christensen 264092169c hydra-eval-jobset: log fetches and evaluations 2021-12-08 09:25:27 -05:00
Cole Helbling 47cabac4bf hydra-eval-jobset: fix use of uninitialized value 
This happens with flake jobsets for obvious reasons (namely, that nixexprinput
and nixexprpath may be undefined for a flake jobset).

12:38:59 hydra-evaluator.1    | Use of uninitialized value $args[0] in join or string at /home/vin/workspace/vcs/hydra/src/script/hydra-eval-jobset line 648.
12:38:59 hydra-evaluator.1    | Use of uninitialized value $args[1] in join or string at /home/vin/workspace/vcs/hydra/src/script/hydra-eval-jobset line 648.
 2021-12-03 12:12:53 -08:00
Graham Christensen 0f8d02894a hydra-eval-jobset: Scalar value @declInputs[0] better written as $declInputs[0] at hydra-eval-jobset line 570. 2021-10-19 21:52:39 -04:00
Eelco Dolstra 2745226ada
Merge pull request #1003 from DeterminateSystems/perlcritic-level-4 
perlcritic: level 4
 2021-09-27 20:23:55 +02:00
Graham Christensen c603ae35f0 perlcritic: Don't conditionally declare variables 2021-09-07 21:35:01 -04:00
Graham Christensen b1879132af ResultSet::TaskRetries: add missing method, get_retryable_task 
Yet again, manual testing is proving to be insufficient. I'm pretty
sure I wrote this code but lost it in a rebase, or perhaps the switch
to result classes.

At any rate, this implements the actual "fetch a retry row and run it"
for the hydra-notify daemon.

Tested by hand.
 2021-09-07 11:12:10 -04:00
Graham Christensen 21e1ff0da1 perlcritic: don't open files as bare words 2021-09-06 22:20:17 -04:00
Graham Christensen c880888f1e File::Slurp -> File::Slurper 2021-09-06 22:13:33 -04:00
Your Name 4677a7c894 perlcritic: use strict, use warnings 2021-09-06 22:13:33 -04:00
Graham Christensen ea3ae0693e Hook up the retryable tasks with hydra-notify 2021-09-02 10:06:26 -04:00
Graham Christensen c0e86faa78 hydra-notify: call TaskRetries->save_task if a task fails 2021-09-02 10:06:26 -04:00
Graham Christensen 1f2adf61aa hydra-notify: extract runPluginsForEvent to a TaskDispatcher 2021-09-02 10:06:26 -04:00
Your Name 4ebdcc290e fixup! hydra-notify: pre-declare metrics 2021-08-24 10:57:23 -04:00
Your Name 45e8fa5319 hydra-notify: support sending diagnostic dumps to STDERR on request 2021-08-24 10:56:13 -04:00
Your Name de2282bcf4 hydra-notify: print out log lines indicating it is or is not launching the exporter 2021-08-24 10:56:13 -04:00
Your Name 5c1228e141 hydra-notify: pre-declare metrics 2021-08-24 10:56:13 -04:00
Your Name 6d7ee27d25 hydra-notify: make the prometheus endpoint configurable, default-off 2021-08-24 10:56:13 -04:00
Your Name 5d0ad5f649 hydra-notify: initial scratch take of prometheus events 2021-08-24 10:56:12 -04:00
Your Name 6e65c3b320 hydra-notify: fixup printing of build IDs 
Used to print:

    sending notifications for build Hydra::Model::DB::Builds=HASH(0x124cf960)->id...

Now it prints:

    sending notifications for build 123...
 2021-08-16 16:09:05 -04:00
Your Name 2c50227082 hydra-notify: properly call new_event 2021-08-16 15:52:25 -04:00
Your Name e572a5e576 hydra-notify: use Hydra::Event 2021-08-16 15:52:14 -04:00
Graham Christensen fa6d7abc13 hydra-notify: move BuildFinished processing to an Event 2021-08-13 16:51:29 -04:00
Graham Christensen 4a1389e36e hydra-notify: move StepFinished processing to an Event 2021-08-13 16:51:29 -04:00
Graham Christensen 4fdb20d3bd hydra-notify: move BuildStarted processing to an Event 2021-08-13 16:51:29 -04:00
Graham Christensen 10e85e3422 hydra-notify: Create a helper for running each plugin on an event 2021-08-13 16:51:29 -04:00
Graham Christensen a14c8ad5f8
Merge pull request #995 from DeterminateSystems/declarative-jobsets-plugin 
Declarative jobsets: move event handling to a plugin
 2021-08-12 15:56:13 -04:00
Graham Christensen 0f958f3425
Merge pull request #997 from DeterminateSystems/abstract-listener 
Abstract over postgres' LISTEN/NOTIFY
 2021-08-12 14:00:34 -04:00
Graham Christensen 5027003285 Abstract over postgres' LISTEN/NOTIFY 
This lets us test the event loop if we wanted, and lets us
test the listening behavior in isolation.
 2021-08-12 13:54:05 -04:00
Graham Christensen 593af41808 Declarative jobsets: move event handling to a plugin 
Declarative jobsets were sort of tucked in to the event hanlder
itself. It turned out that it could have been implemented as a
plugin without much trouble.
 2021-08-12 12:48:18 -04:00
Graham Christensen 9c5f317453 hydra-notify: move buildFinished query in to the function impl 
This is more consistent with the other event handlers, of dealing
with IDs and not objects.
 2021-08-12 12:30:35 -04:00
Graham Christensen 508d99d611 Join to builds via jobset_id when easy 2021-06-01 11:16:47 -04:00
Graham Christensen 05636de7d2 hydra-init: upgrade passwords to Argon2 on startup 2021-04-16 12:32:13 -04:00
Graham Christensen 79b0ddc27d hydra-create-user: re-hash sha1 as Argon2 2021-04-16 12:32:13 -04:00
Graham Christensen 1d956be61e hydra-create-user: support Argon2 
Co-authored-by: Graham Christensen <graham@grahamc.com>
 2021-04-15 11:35:16 -04:00
Graham Christensen 425c7ff17f
hydra-send-stats: add a --once option for testing 2021-03-20 09:16:08 -04:00
Jörg Thalheim 6bb180a0f2
hydra-send-stats: fix imports 2021-03-20 09:16:04 -04:00
Janne Heß 3c86083d21
Fixup #717 "Add the project name to declarative inputs" 
```
Mar 10 16:22:35 hydra-b hydra-evaluator[41419]: DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR:  null value in column "type" violates not-null constraint
Mar 10 16:22:35 hydra-b hydra-evaluator[41419]: DETAIL:  Failing row contains (62358, projectName, 0, null, null, null, hackworthltd, null, , null). [for Statement "INSERT INTO jobsetevalinputs ( altnr, dependency, eval, name, path, revision, sha256hash, type, uri, value) VALUES ( ?, ?, ?, ?, ?, ?, ?, ?, ?, ? )" with ParamValues: 1='0', 2=undef, 3='62358', 4='projectName', 5='', 6=undef, 7=undef, 8=undef, 9=undef, 10='hackworthltd'] at /nix/store/cmqblv437mp57yz5lwvkzcqca4ldf3r5-hydra-0.1.20210308.ebf1cd2/bin/.hydra-eval-jobset-wrapped line 793
Mar 10 16:22:35 hydra-b hydra-evaluator[25828]: evaluation of jobset ‘hackworthltd:.jobsets (jobset#1)’ failed with exit code 1
```

Use the abstraction for creating inputs for simulating the project
name input.

Co-authored-by: Graham Christensen <graham@grahamc.com>
 2021-03-16 09:52:36 -04:00
Janne Heß 9e018d5443
Add the project name to declarative inputs 
This allows for more generic declarative configurations which can be
shared between projects.
 2021-03-08 17:36:52 +01:00
Matej Cotman a551fba346
statsd: add a chance to set hostname and port in hydra.conf 
Co-authored-by: Graham Christensen <graham@grahamc.com>
 2021-03-08 10:03:16 -05:00
Graham Christensen 3fda37f65a
RunCommand: Test 2021-02-24 13:43:25 -05:00
Graham Christensen f1e75c8bff
Move evaluation errors from evaluations to EvaluationErrors, a new table 
DBIx likes to eagerly select all columns without a way to really tell
it so. Therefore, this splits this one large column in to its own
table.

I'd also like to make "jobsets" use this table too, but that is on hold
to stop the bleeding caused by the extreme amount of traffic this is
causing.
 2021-02-01 21:33:14 -05:00
Graham Christensen 091d58c128
hydra-dev-server: autoflush stderr/stdout 2021-01-26 13:51:39 -05:00
Graham Christensen 54b8cb188e
perl: jobsetevals -> jobset via by jobset_id 
Frankly, this was suspiciously little work.
 2021-01-26 13:51:39 -05:00
Graham Christensen 9516b256f1
Normalize nixexpr{input,path} from builds to jobsetevals. 
Duplicating this data on every record of the builds table cost
approximately 4G of duplication.

Note that the database migration included took about 4h45m on an
untuned server which uses very slow rotational disks in a RAID5 setup,
with not a lot of RAM. I imagine in production it might take an hour
or two, but not 4. If this should become a chunked migration, I can do
that.

Note: Because of the question about chunked migrations, I have NOT
YET tested this migration thoroughly enough for merge.
 2021-01-22 09:10:18 -05:00
Graham Christensen 086eed5147
hydra-eval-jobs: write evaluation errorMsg to the jobseteval table 2021-01-21 13:10:41 -05:00
Jelle Besseling bbd4891133
Implement GitHub logins 
Requires the following configuration options
enable_github_login = 1
github_client_id
github_client_secret
Or github_client_secret_file which points to a file with the secret
 2020-12-28 14:37:03 +01:00
Eelco Dolstra be709d450b Fix sysbuild 
596f4cf4b9
 2020-10-22 13:27:52 +02:00
Janne Heß 24acb9d6bb
Fix non-static declarative jobsets 
With the current implementation, if ANY hash was found inside the decl
spec, the spec would be treated as static. This is problematic since
`inputs` is a hash and hence any configuration would be handled as a
static one.
This fixes the code to match the documentation and only switch to static
processing when ALL values are hashes.
 2020-09-13 18:21:38 +02:00
Eelco Dolstra e707990e2d
Merge pull request #804 from grahamc/fully-static-jobsets 
declarative projects: support fully static, declarative configuration
 2020-09-02 21:09:01 +02:00