ma27/hydra
0
0
Fork 0
forked from lix-project/hydra
Code Pull requests Activity
4209 commits 4 branches 0 tags 26 MiB
Commit graph

282 commits

Author SHA1 Message Date
emily ab6d81fad4
api: fix github webhook 2024-08-26 20:26:21 +02:00
Rick van Schijndel 8858abb1a6
t/test.pl: increase event-timeout, set qvf 
Only log issues/failures when something's actually up.
It has irked me for a long time that so much output came
out of running the tests, this seems to silence it.
It does hide some warnings, but I think it makes the output
so much more readable that it's worth the tradeoff.

Helps for highly parallel running of jobs, sometimes they'd not give output for a while.
Setting this timeout higher appears to help.
Not completely sure if this is the right place to do it, but it works fine for me.
 2024-08-11 16:08:35 +02:00
Rick van Schijndel ef619eca99
t: increase timeouts for slow commands with high load 
We've seen many fails on ofborg, at lot of them ultimately appear to come down to
a timeout being hit, resulting in something like this:

Failure executing slapadd -F /<path>/slap.d -b dc=example -l /<path>/load.ldif.

Hopefully this resolves it for most cases.
I've done some endurance testing and this helps a lot.
some other commands also regularly time-out with high load:

- hydra-init
- hydra-create-user
- nix-store --delete

This should address most issues with tests randomly failing.

Used the following script for endurance testing:

```

import os
import subprocess

run_counter = 0
fail_counter = 0

while True:
    try:
        run_counter += 1
        print(f"Starting run {run_counter}")
        env = os.environ
        env["YATH_JOB_COUNT"] = "20"
        result = subprocess.run(["perl", "t/test.pl"], env=env)
        if (result.returncode != 0):
            fail_counter += 1
        print(f"Finish run {run_counter}, total fail count: {fail_counter}")
    except KeyboardInterrupt:
        print(f"Finished {run_counter} runs with {fail_counter} fails")
        break
```

In case someone else wants to do it on their system :).
Note that YATH_JOB_COUNT may need to be changed loosely based on your
cores.
I only have 4 cores (8 threads), so for others higher numbers might
yield better results in hashing out unstable tests.
 2024-08-11 16:08:09 +02:00
Pierre Bourdon 4b886d9c45
autotools -> meson 
There are some known regressions regarding local testing setups - since
everything was kinda half written with the expectation that build dir =
source dir (which should not be true anymore). But everything builds and
the test suite runs fine, after several hours spent debugging random
crashes in libpqxx with MALLOC_PERTURB_...
 2024-07-22 22:30:41 +02:00
Pierre Bourdon 9a4a5dd624
jobset-eval: fix actions not showing up sometimes for new jobs 
New jobs have their "new" status take precedence over them being
"failed" or "queued", which means actions that can act on "failed" or
"queued" jobs weren't shown to the user when they could only act on
"new" jobs.
 2024-07-20 13:09:39 +02:00
Pierre Bourdon 370a4bf138
treewide: start removing tests related to constituents 
The feature cannot easily be ported to nix-eval-jobs since it requires
deep integration into the evaluator, and h.n.o doesn't use it. Later
more of this will be ripped out.
 2024-07-17 08:31:19 +02:00
Pierre Bourdon ed7c58708c
hydra-eval-jobs: remove, replaced by nix-eval-jobs 2024-07-17 08:31:19 +02:00
Pierre Bourdon 6d4ccff43c
hydra-eval-jobset: use nix-eval-jobs instead of hydra-eval-jobs 2024-07-17 08:31:19 +02:00
Pierre Bourdon 6189ba9c5e
web: replace 'errormsg' with 'errormsg IS NULL' in most cases 
This is implement in an extremely hacky way due to poor DBIx feature
support. Ideally, what we'd need is a way to tell DBIx to ignore the
errormsg column unless explicitly requested, and to automatically add a
computed 'errormsg IS NULL' column in others. Since it does not support
that, this commit instead hacks some support via method overrides while
taking care to not break anything obvious.
 2024-04-12 20:14:09 +02:00
ajs124 6ed21490ee lazy-load evaluation errors 
Closes #1362
 2024-03-16 22:10:40 +01:00
John Ericson c62eaf248f Remove now-unneeded workaround 2024-01-26 01:20:07 -05:00
John Ericson 13b5f007ef Merge branch 'master' into ca-no-new-col 2024-01-26 01:19:45 -05:00
John Ericson 5ee0e443e4 Remove now-unneeded workaround 2024-01-26 01:08:11 -05:00
John Ericson 323b556dc8 Minimal CA support 
This verison has a worse UI, but also chnages the schema less: One
non-null constraint is removed, but no new columns are added.

Co-Authored-By: Andrea Ciceri <andrea.ciceri@autistici.org>
Co-Authored-By: regnat <rg@regnat.ovh>
 2024-01-26 00:34:58 -05:00
John Ericson fcde5908d8 More CA derivations prep 
Again, with care not to change the schema in any way.
 2024-01-25 21:32:22 -05:00
John Ericson 411e4d0c24
Let tests themselves intentionally leak temp dir (#1320) 
* Let tests themselves intentionally leak temp dir

By default Yath will clean up temporary files, so the result is the
same. But `--keep-dirs` can be passed to `yath test` telling Yath to
*not* clean them up instead. This is very useful for debugging.

* Update t/lib/HydraTestContext.pm

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
 2023-12-08 16:30:31 +00:00
Eelco Dolstra ce001bb142 Relax time interval checks 
I saw one of these failing randomly.
 2023-06-23 15:09:09 +02:00
Maximilian Bosch fd765bc97a
Fix "My Jobs" tab in user dashboard 
Nowadays `Builds` doesn't reference `Project` directly anymore. This
means that simply resolving both `jobset` and `project` with a single
JOIN from `Builds` doesn't work anymore. Instead we need to resolve the
relation to `jobset` first and then the relation to `project`.

For similar fixes see e.g. c7c4759600.
 2022-11-22 20:54:51 +01:00
Maximilian Bosch d3fe4ffbf6
Job: expose closuresize and size (output size in the UI) as prometheus metrics 2022-09-22 10:47:22 +02:00
Eelco Dolstra c72bed5cb4 Fix tests 
Use $NIX_REMOTE instead of the legacy environment variables.
 2022-07-12 14:45:30 +02:00
ajs124 bb1f04ed86 AddBuilds: fix declarative jobsets with dynamic runcommand enabled 
$project->{enable_dynamic_run_command} is undefined
 2022-06-30 01:49:30 +02:00
Kayla Firestack 065039beba feat(t/evaluator/evaluate-oom): comment intentions 2022-05-02 15:26:26 -04:00
Kayla Firestack 87f610e7c1 fix(t/evaluator/evaluate-oom): use test_context to get path to ./t/jobs instead of relative paths 2022-05-02 15:14:46 -04:00
Kayla Firestack 013a1dcabc fix(t/evaluator/evaluate-oom): check that the exit value of the systemd-run check is zero. Rework skip messages 2022-05-02 15:13:59 -04:00
Kayla Firestack e917d9e546 fix(t/evaluator/evaluate-oom): convert systemd-run presence check to eval, fix indentaion, show relationships between flags and commands with indentation 2022-05-02 14:40:13 -04:00
Kayla Firestack 01ec004108 feat(t/evaluator/evaluate-oom-job): skip test if systemd-run is not present 2022-05-02 14:08:50 -04:00
Kayla Firestack 2c909c038f feat(t/evaluator/hydra-eval-jobs): add basic evaluation test for hydra-eval-jobs 2022-05-02 13:50:57 -04:00
Kayla Firestack 90769ab5ad feat(t/jobs): add test job to cause an OOM 2022-05-02 13:49:32 -04:00
Graham Christensen 5c90edd19f
Merge pull request #1103 from DeterminateSystems/runcommand/dynamic 
Dynamic RunCommand
 2022-04-19 10:09:47 -04:00
Graham Christensen e1965250b5
Merge pull request #1173 from DeterminateSystems/queue-runner-exporter 
hydra-queue-runner metrics
 2022-04-07 12:27:33 -04:00
Cole Helbling edf3c348f2 hydra-queue-runner: make entire address configurable 2022-04-06 10:59:45 -07:00
Cole Helbling 9c1f36c47c t/lib/HydraTestContext: set queue runner port to 0 
This makes the exposer choose a random, available port.
 2022-03-29 11:41:23 -07:00
Graham Christensen e5393c2cf8 fixup: make id non-ambiguous 2022-03-19 23:56:47 -04:00
Graham Christensen a582e4c485 HydraTestContext: add \n's to various dies 2022-03-19 14:46:53 -04:00
Graham Christensen 0c51de6334 hydra-evaluate-jobset: assert it logs errored constituents properly 2022-03-19 14:35:30 -04:00
Graham Christensen 25f6bae847 HydraTestContext: make it easy to create a jobset without evaluating 2022-03-19 14:34:43 -04:00
Graham Christensen e0921eba0a Create a basic test which verifies we can't delete the derivation of aggregate jobs 2022-02-20 12:28:40 -05:00
Graham Christensen be46f02164 tests: relocate evaluator tests 2022-02-20 12:28:40 -05:00
Graham Christensen 5d169e3a2e Add a test validating direct and indirect constituents 2022-02-20 12:28:40 -05:00
Graham Christensen dfb3eccfaa
Merge pull request #1140 from Ma27/nix-update 
Update Nix to 2.6
 2022-02-19 08:38:34 -05:00
Cole Helbling a22a8fa62d AddBuilds: reject declarative jobsets with dynamic runcommand enabled if disabled elsewhere 2022-02-11 14:35:52 -05:00
Cole Helbling 928ba9e854 Controller/{Jobset,Project}: error when enabling dynamic runcommand but it's disabled elsewhere 2022-02-11 14:35:52 -05:00
Graham Christensen 78e9872251 ldap.t: write the password to an external .conf file 2022-02-11 11:27:10 -05:00
Graham Christensen 848fb3b265 ldap-legacy.t: specify the root password manually 2022-02-11 11:26:56 -05:00
Graham Christensen 0bd4a75918 HydraTestContext: support running a sub before running hydra-init. 2022-02-11 11:26:27 -05:00
Graham Christensen e13d80f5cf LDAPContext: take a root_password argument or generate one 2022-02-11 11:02:04 -05:00
Graham Christensen 86bb16d07b LDAPContext: sort $self hash keys 2022-02-11 11:01:25 -05:00
Graham Christensen 6637c03985 fixup normalization error regex 2022-02-11 10:59:24 -05:00
Graham Christensen 71c06f2ce7 LDAP normalization errors: note that the error came while normalizing the roles. 2022-02-11 10:55:27 -05:00
Graham Christensen d6dea39912 ldap_role_map.t: fixup indentation 2022-02-11 10:53:08 -05:00