ma27/hydra
0
0
Fork 0
forked from lix-project/hydra
Code Pull requests Activity
4203 commits 4 branches 0 tags 26 MiB
Commit graph

281 commits

Author SHA1 Message Date
Cole Helbling
0eb952d72d t/BuildDiff: init 2022-01-10 12:11:05 -08:00
Graham Christensen
98c88a4dbf
Merge pull request #1097 from DeterminateSystems/api-test-warnings 
Api test warnings: clean up
 2022-01-10 14:03:38 -05:00
Graham Christensen
d8025fcf04 api-test.t: don't print request content 
It adds noise to the logs
 2022-01-10 11:02:01 -05:00
Graham Christensen
6b2f0a9fe1 api-test.t: set a default method 2022-01-10 11:02:01 -05:00
Cole Helbling
35f55279c1 HydraTestContext: prefix names with t 
This is necessary because jobset and project names are not allowed to
begin with a digit, and yet the generated jobset and project names would
do just that.

Not the most elegant solution, but it works.
 2022-01-10 09:39:42 -05:00
Graham Christensen
06360bfbb6 RunCommandLogs.t: use test_context to get a specific build 
Fixes a build error: Can't use global @_ in 'my' at Schema/Result/RunCommandLogs.t line 20, near ', @_'
 2022-01-07 15:05:33 -05:00
Graham Christensen
a7aeb766aa RunCommandLogs: add a helper for if it failed with an exec error 2022-01-07 15:05:33 -05:00
Graham Christensen
d849856dcd RunCommandLogs: add helpers for if it is running/failed/signaled 2022-01-07 15:05:33 -05:00
Graham Christensen
c81acbf93e RunCommandLogs: test recording a signaled process without a core dump 2022-01-07 15:05:33 -05:00
Graham Christensen
d003fec8a5 RunCommandLogs: add a did_succeed helper 2022-01-07 15:05:33 -05:00
Graham Christensen
5bb3e2be78 Record the errno if exec fails 2022-01-07 15:05:33 -05:00
Graham Christensen
4ce8239cea RunCommand: create run logs for each execution 2022-01-07 15:05:33 -05:00
Graham Christensen
a14501c616 Tasks: only execute the event if the plugin is interested in it 2021-12-21 20:57:53 -05:00
Graham Christensen
d195e545f5 hydra-notify: listen for build_queued events 2021-12-21 20:57:53 -05:00
Graham Christensen
f2f8d571f3 queue-runner-status: fixup from perlcritic-level-2 
>
> from_json was legacy / deprecated. This was the only use.
 2021-12-15 17:24:23 -05:00
Graham Christensen
fbce3b6ed1 default-machine-file: use makeAndEvaluateJobset 2021-12-14 20:52:40 -05:00
Graham Christensen
06f824ca48 notifications.t: use system() with lists 2021-12-14 20:48:19 -05:00
Graham Christensen
5836bc9a11 HydraTestContext::makeAndEvaluateJobset: make a random user 2021-12-14 20:48:07 -05:00
Graham Christensen
c2384a04d8 notifications.t: move to makeAndEvaluateJobset 2021-12-14 20:41:21 -05:00
Graham Christensen
008321d972 build-products: switch to makeAndEvaluateJobset 2021-12-14 20:32:13 -05:00
Graham Christensen
7333d444c6 evaluate-basic.t: move to makeAndEvaluateJobset 2021-12-14 20:25:54 -05:00
Graham Christensen
adfe74b76a HydraTestContext: give a helper for creating a project, jobset, evaluating jobs, and optionally building them. 
In return, get a hash of all the build records.
 2021-12-14 20:25:42 -05:00
Graham Christensen
3238496b3d t/Setup: move subs calling hydra-* programs to a CliPrograms module 
Makes it so HydraTestContext can call them without an import loop
 2021-12-14 20:25:02 -05:00
Graham Christensen
1fa141229f HydraTestContext: explicitly stop the db 
This might, hopefully, I don't know, possibly force the
database to live a little while longer and *reduce* but not
eliminate errors around stopping the database before we lose all
our DB::PG handles to it.
 2021-12-14 19:51:12 -05:00
Graham Christensen
eca09bc980 Attempt to fix errors during test teardown 2021-12-14 16:31:41 -05:00
Graham Christensen
fc49a7129d JSON -> JSON::MaybeXS 2021-12-14 10:16:25 -05:00
Graham Christensen
30a7f068b7 AttributeSet helper: skip wantarray 2021-12-14 10:16:25 -05:00
Graham Christensen
327d79bc1d pin 2021-12-14 10:16:25 -05:00
Graham Christensen
7dcf6a01c6 JSON -> JSON::MaybeXS 2021-12-13 15:37:56 -05:00
Graham Christensen
26b197ea62 RunCommand: calculate all the commands to run against before starting 2021-12-08 20:29:32 -05:00
Graham Christensen
b7962c3882 RunCommand: Move the json validation testing to its own test 2021-12-08 20:29:27 -05:00
Graham Christensen
bb91f96381 RunCommand: test isEnabled, configSectionMatches, and eventMatches 2021-12-08 20:03:43 -05:00
Graham Christensen
9dfa34682d RunCommand tests: move in to a subdirectory 2021-12-08 20:03:43 -05:00
Graham Christensen
7227395a72
Merge pull request #1068 from DeterminateSystems/fix-some-warnings 
Fix some warnings
 2021-12-03 15:38:46 -05:00
Cole Helbling
ecb1454cf1 t/Controller/Root: test narinfo path 2021-12-03 12:12:53 -08:00
Graham Christensen
085b02e092 t/Helper/CatalystUtils.t: test trim functionality 2021-12-03 11:58:19 -08:00
Graham Christensen
4d5dfec8c7 TT.t: fixup $_, since _ is a special variable, not a throwaway 2021-12-01 13:46:58 -05:00
Graham Christensen
ce6805f02b fixup: TT.t: move the :'s to the next line 2021-11-22 20:20:06 -05:00
Graham Christensen
03323f6ef1 TT: add helpers for linking to jobs, jobsets, and projects, and for generating colon separated names. 2021-11-22 13:38:53 -05:00
Graham Christensen
5917de4948 t/plugins/runcommand.t: output the filename if it fails to open 2021-11-17 15:05:28 -05:00
Graham Christensen
e5a4147f0a runcommand.t: stop dumping the json 2021-11-17 15:05:28 -05:00
Eelco Dolstra
9ae676072c
Merge pull request #1051 from DeterminateSystems/restart-cancel-resultset-p2 
Validate and fix canceling builds
 2021-10-27 18:22:20 +02:00
Graham Christensen
c1630e6d80 Admin/clear-queue-non-current: test behavior 2021-10-27 11:43:58 -04:00
Graham Christensen
d19890a4e1 JobsetEval/cancel: test & fix 2021-10-27 11:43:36 -04:00
Graham Christensen
ef9a9fa481
Merge pull request #1025 from lukegb/hydra-better-errors 
Produce better errors for failing jobsets
 2021-10-26 12:35:45 -04:00
Graham Christensen
a62c2a4d71 JobsetEval/restart-*: Write tests for the behavior 2021-10-26 10:45:33 -04:00
Graham Christensen
383b395b00 Test canceling builds. 2021-10-26 09:44:58 -04:00
Luke Granger-Brown
67ebce8493 Output evaluation errors without crashing if aggregate job is broken. 
At the moment, aggregate jobs can easily break and cause the entire
evaluation to fail, which is not ideal. For Nixpkgs, we do have some
important aggregate jobs (like `tested`), but for debugging and building
purposes it's still useful to get a partial result even if the channel
won't actually advance.

This commit changes the behaviour of hydra-eval-jobs such that it
aggregates any errors found during the construction of an aggregate, and
will instead annotate the job with the evaluation failure such that it
shows up in a "cleaner" way.

There are really two types of failure that we care about: one is where
the attribute just ends up missing altogether in the final output, and
also where the attribute is in the output but fails to evaluate. Both
are handled here.

Note that this does mean that the same error message may be output
multiple times, but this aids debuggability because it'll be much
clearer what's blocking the job from being created.
 2021-10-26 10:14:34 +01:00
Luke Granger-Brown
f2b51a017b Test Setup: fix evalSucceeds to actually output evaluation errors 
At the moment, the jobset object is unlikely to actually retrieve the
evaluation error output, because it isn't refreshed after
hydra-eval-jobsets is run.

Explicitly calling DBIx::Class::Row->discard_changes causes any updated
data to be refreshed, at the cost of losing any not-yet committed
changes to the row.
 2021-10-26 10:13:58 +01:00
Graham Christensen
d52e397503 Builds controller: add a test for restarting builds, fix restarting builds 2021-10-25 22:22:08 -04:00
Graham Christensen
5fbf1470bd (perl) machines file: support machine lines with multiple spaces between fields 2021-10-24 21:30:53 -04:00
Graham Christensen
b817124337 Hydra::Helper::Nix::getMachines: add a test 
Fix parsing breakage from #1003: assigning the lines to $lines broke chomp and the filters.

This test validates the parsing works as expected, and also fixes
a minor bug where '-' in features isn't pruned, like in the C++
repo.
 2021-10-23 22:53:55 -04:00
Graham Christensen
5285d0b6cb gittea test: prune trailing space 2021-10-20 13:09:39 -04:00
Graham Christensen
7095d00608 perlcritic: make all open() calls three-argument 
Two-argument calls end up parsing the second argument to guess what
should happen, three-arg parses ... less?
 2021-10-20 13:09:39 -04:00
Graham Christensen
43c056bb7b t/TaskDispatcher.t: array / arrayref consistency 
Also fixes a test which was comparing the length of two lists
instead of the values inside the lists. Dang, Perl.
 2021-10-20 11:38:27 -04:00
Graham Christensen
3990bd12e9 constituents test: add use warnings 
This is causing CI to fail after #1026 merged. #1026 had a green
bill of health, but #1003 increased perlcritic to level 4. #1003
was not part of #1026 so it was not checked at perlcritic level 4.
 2021-09-30 09:13:01 -04:00
Graham Christensen
4c8279044b
Merge pull request #1026 from DeterminateSystems/constituents 
add api endpoint: /build/<id>/constituents
 2021-09-28 09:21:21 -04:00
Eelco Dolstra
2745226ada
Merge pull request #1003 from DeterminateSystems/perlcritic-level-4 
perlcritic: level 4
 2021-09-27 20:23:55 +02:00
Graham Christensen
c60c8d10ea add api endpoint: /build/<id>/constituents 
Returns a list of constituent builds
 2021-09-24 16:30:44 -04:00
Graham Christensen
741590c923 perlcritic: explicitly assign the result of readdir/file reads 2021-09-07 21:55:55 -04:00
Graham Christensen
b1879132af ResultSet::TaskRetries: add missing method, get_retryable_task 
Yet again, manual testing is proving to be insufficient. I'm pretty
sure I wrote this code but lost it in a rebase, or perhaps the switch
to result classes.

At any rate, this implements the actual "fetch a retry row and run it"
for the hydra-notify daemon.

Tested by hand.
 2021-09-07 11:12:10 -04:00
Graham Christensen
21e1ff0da1 perlcritic: don't open files as bare words 2021-09-06 22:20:17 -04:00
Your Name
55b7802b61 t/lib/Setup.pm: nix_config: default to empty string 2021-09-06 22:13:33 -04:00
Your Name
4677a7c894 perlcritic: use strict, use warnings 2021-09-06 22:13:33 -04:00
Graham Christensen
b0055a23df TaskDispatcher: Support re-queueing tasks, and dropping tasks after 100 failures. 2021-09-02 10:06:26 -04:00
Graham Christensen
d0b0fc21b3 ResultSet::TaskRetries: teach about saving tasks 2021-09-02 10:06:26 -04:00
Graham Christensen
147fa4d029 Result::TaskRetries: Teach about requeue 2021-09-02 10:06:26 -04:00
Graham Christensen
42c2d2f387 Hydra::Math: add an exponential_backoff function 2021-09-02 10:06:26 -04:00
Graham Christensen
29738364fb ResultSet::TaskRetries: add get_seconds_to_next_retry 
Get the number of seconds before the next retriable task is ready.

This number is specifically intended to be used as a timeout, where
`undef` means never time out.
 2021-09-02 10:06:26 -04:00
Graham Christensen
1f2adf61aa hydra-notify: extract runPluginsForEvent to a TaskDispatcher 2021-09-02 10:06:26 -04:00
Your Name
6d7ee27d25 hydra-notify: make the prometheus endpoint configurable, default-off 2021-08-24 10:56:13 -04:00
Your Name
241985fb2d Start checking PRs with perlcritic 2021-08-20 11:06:43 -04:00
Graham Christensen
fa6d7abc13 hydra-notify: move BuildFinished processing to an Event 2021-08-13 16:51:29 -04:00
Graham Christensen
4a1389e36e hydra-notify: move StepFinished processing to an Event 2021-08-13 16:51:29 -04:00
Graham Christensen
4fdb20d3bd hydra-notify: move BuildStarted processing to an Event 2021-08-13 16:51:29 -04:00
Graham Christensen
4e86e55008 Event.pm: add a new_event helper to parse and construct an Event 
An Event will be part of many Tasks
 2021-08-13 16:51:29 -04:00
Graham Christensen
64a3e75c10 Event: init structure and parse existing messages 2021-08-13 16:51:29 -04:00
Graham Christensen
5027003285 Abstract over postgres' LISTEN/NOTIFY 
This lets us test the event loop if we wanted, and lets us
test the listening behavior in isolation.
 2021-08-12 13:54:05 -04:00
Shay Bergmann
ba557972c7
Add test to verify includes work 2021-07-26 17:16:40 +00:00
Cole Helbling
e6a8412a12 Jobset: test type-related constraints 
Prior to https://github.com/NixOS/hydra/pull/856, the `type`-related constraints
were a bit too loose, allowing some variations that should have been seen as
invalid. This commit adds a test (inspired by
https://github.com/NixOS/hydra/pull/856#issuecomment-772820929) to ensure these
constraints stay unviolated.
 2021-07-20 10:38:22 -07:00
Graham Christensen
cb8929b7ed Tighten up 'should exit with return code' 2021-06-16 11:48:49 -04:00
Graham Christensen
2ac47e8013 Test that each page listing evals works 
PR #860 caused a regression that broke some loads.
 2021-06-16 11:22:40 -04:00
Graham Christensen
71453dd341 Expose Prometheus metrics at /metrics 
Exposes metrics:

* http_request_duration_seconds_bucket
* http_request_size_bytes_bucket
* http_response_size_bytes_bucket
* http_requests_total

with labels of action and controller to help identify popular
endpoints and their performance characteristics.
 2021-06-10 20:02:42 -04:00
Cole Helbling
45ced33ab6
Projects: don't respond with unnecessary declarative field 
If the project isn't declarative, who cares about it in the response? After
setting the `declfile` to an empty string, everything related to declarative-
ness is wiped out, anyways.
 2021-05-24 15:26:34 -07:00
Cole Helbling
588a3a774f
Project: add test for declarative->normal project transition 
Also split into subtests.
 2021-05-06 08:47:29 -07:00
Cole Helbling
ad13d13436
Project: add declfile, decltype, declvalue to API 
This makes it possible to create declarative projects via a PUT request, and
also exposes the currently-configured values to GET requests.
 2021-05-02 17:45:14 -07:00
Cole Helbling
d0682c804d
api-test: test deleting a project 
As of this commit, this test will fail, because the underlying cause hasn't been
fixed.
 2021-04-30 10:42:38 -07:00
Cole Helbling
03d5a858a7
Jobset: list known inputs in invalid input type response 2021-04-29 18:19:44 -07:00
Cole Helbling
f1dd5d202e Jobsets: update schema to align with the API 
To further align with the API, we return custom JSON in order to display a
`visible` field rather than `hidden` -- a `PUT` request expects `visible`, while
a `GET` request returns `hidden`.

This also allows us to rename the `jobsetinputs` field to `inputs` for the same
reason: `PUT` expects `inputs`, while `GET` returns `jobsetinputs`.
 2021-04-29 07:50:23 -04:00
Graham Christensen
29028258e3
Merge pull request #940 from regnat/test-with-remote-dest-store 
Run the tests with a remote dest store
 2021-04-29 07:27:42 -04:00
Graham Christensen
e1a761194f
Merge pull request #943 from regnat/include-localhost-features 
Cleanup the localhost-features test
 2021-04-29 07:26:50 -04:00
regnat
bf3e1cf77b Disable the remote dest store for the channel test 
Looks like feature is only works when the destination store is the local
store
 2021-04-29 07:24:20 +02:00
regnat
9c787e5e94 Document the use_external_destination_store test_init param 2021-04-29 07:24:20 +02:00
Théophane Hufschmitt
7af63658f1 tests: Make the external destination store optional 
Co-authored-by: Graham Christensen <graham@grahamc.com>
 2021-04-29 07:24:20 +02:00
regnat
305c27d3fb Move the default-machine-file test under queue-runner 
Just a bit of housekeeping to keep the testsuite clean
 2021-04-29 06:55:23 +02:00
regnat
f61a13d2db Only add the Nix system feature to the test that requires it 2021-04-29 06:54:25 +02:00
Graham Christensen
a44e5b73a8 api-test: clean up, use subtests 2021-04-28 19:41:49 -04:00
Graham Christensen
5c5b53961e fixup: api-test.t expected to pass inputs as inputs 2021-04-28 18:30:53 -04:00
Graham Christensen
a34f23f7e6 api-test: use is for nicer errors 2021-04-28 18:28:35 -04:00
Graham Christensen
20df86c275 api-test.t: make it work under yath 
1. Configure the in-memory Hydra instance with a null path input cache
time to avoid caching slowing the test down.
2. Use the Catalyst::Test helpers so we talk to the application and skip
needing to actually run a webserver.
3. Change path references to use a tempdir, since this is running while
other tests are also running.
4. Change the login flow to save a cookie and pass it manually. A bit
weird, but it avoids a dependency on heavier browser-mimicking
libraries.
 2021-04-28 18:28:35 -04:00
Graham Christensen
38c7f88b7c rename api-test.pl to .t so it runs under yath 2021-04-28 18:28:35 -04:00
Graham Christensen
dd6cd33e98 move the api-test.nix in to the jobsdir 2021-04-28 18:28:35 -04:00
Graham Christensen
823da22e4f
Merge pull request #937 from DeterminateSystems/flesh-out-api 
hydra-api: flesh out Jobset, JobsetInput schemas; implement DELETE /jobset/{project-id}/{jobset-id}
 2021-04-28 13:43:04 -04:00
Cole Helbling
948b3cf073
Jobset: add HTTP API test 2021-04-28 10:37:06 -07:00
regnat
0d3977866e Run the tests with a remote dest store 
Set `dest_store` in the test hydra config, so that the testsuite ensures
that the distinction between the local store and the destination store
is properly taken into account.

Fix #938
 2021-04-28 12:22:54 +02:00
regnat
abff212d06 Use system-features from the Nix conf in the default machine file 
Fix #936
 2021-04-28 11:43:04 +02:00
regnat
d58b3274f9 test: Don't generate a Nix conf in the makefile 
The config is overriden by the `test_init` perl function, so it’s at
best misleading to create a first one there
 2021-04-28 11:41:37 +02:00
Cole Helbling
50fab154a4
ToJSON: serialize string_columns to JSON 
If the column is undefined, then it should be an empty string according to your
API spec.
 2021-04-26 16:39:13 -07:00
Graham Christensen
f0ede5f8b8
t: Test Projects JSON serialization and deserialization 2021-04-26 16:39:13 -07:00
Graham Christensen
05636de7d2 hydra-init: upgrade passwords to Argon2 on startup 2021-04-16 12:32:13 -04:00
Graham Christensen
79b0ddc27d hydra-create-user: re-hash sha1 as Argon2 2021-04-16 12:32:13 -04:00
Graham Christensen
d10d8964f2 Users: add a validation step which lets the user's password be a Argon2 hashed sha1 hash. 
OWASP suggests expiring all passwords and requiring users to update their password.
However, we don't have a way to do this. They suggest this mechanism
as a good alternative:
https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html#upgrading-legacy-hashes
 2021-04-16 12:32:13 -04:00
Graham Christensen
1da70030b7 Users: transparently upgrade passwords to Argon2 
Passwords that are sha1 will be transparently upgraded to argon2,
and future comparisons will use Argon2

Co-authored-by: Graham Christensen <graham@grahamc.com>
 2021-04-15 11:35:11 -04:00
Graham Christensen
29620df85e Passwords: check in constant time 
The default password comparison logic does not use
constant time validation. Switching to constant time
offers a meager improvement by removing a timing
oracle.

A prepatory step in moving to Argon2id password storage, since we'll need this change anyway after
for validating existing passwords.

Co-authored-by: Graham Christensen <graham@grahamc.com>
 2021-04-15 11:34:56 -04:00
Graham Christensen
cf4434bc9f queue runner: test notifications 
Especially, test the difference in behavior of substituted and unsubstituted builds.
 2021-04-14 14:19:10 -04:00
Graham Christensen
e45f852277 tests: allow specifying some nix config 2021-04-14 14:19:10 -04:00
Graham Christensen
c7ac123dc5 Setup.pm: specify NIX_ env vars for running yath 
Otherwise yath will try to use global configuration.
 2021-04-14 14:19:10 -04:00
Graham Christensen
74d34c0f80 t/Setup.pm: sort NIX_ env vars 2021-04-14 14:19:09 -04:00
Graham Christensen
8aa59dcc1b Drop references to NIX_MANFIESTS_DIR and NIX_BUILD_HOOK 
Neither of these have been supported in ~years.
 2021-04-14 14:19:09 -04:00
Maximilian Bosch
2beb1f5405
Replace TestHTTPMockServer with python script 
This seems to work fine in a `nix build`-sandbox as it doesn't depend on
`getprotobyname`.
 2021-04-03 01:07:07 +02:00
Maximilian Bosch
d16bf5b8cd
Implement yath-test for the new Gitea plugin 2021-04-02 19:11:25 +02:00
Graham Christensen
9bb04ed97a
Merge pull request #893 from grahamc/fake-channel-nested 
Test the fake derivations channel, asserting nested packages are properly represented.
 2021-03-24 17:56:55 -04:00
Graham Christensen
425c7ff17f
hydra-send-stats: add a --once option for testing 2021-03-20 09:16:08 -04:00
Graham Christensen
6f662a606a
hydra-send-stats: add a failing test asserting it can run 2021-03-20 09:14:14 -04:00
Graham Christensen
019aef3d41
Test the fake derivations channel, asserting nested packages are properly represented. 
This is a breaking change. Previously, packages named `packageset.foo`
would be exposed in the fake derivation channel as `packageset-foo`.

Presumably this was done to avoid needing to track attribute sets, and
to avoid the complexity. I think this now correctly handles the
complexity and properly mirrors the input expressions layout.
 2021-03-18 11:33:37 -04:00
Graham Christensen
88e0198a8e
Create a helper for dealing with nested attribute sets 2021-03-18 11:33:36 -04:00
Graham Christensen
d62a2c1657
NixExprs: extract the escape function and test it 2021-03-18 11:24:17 -04:00
Graham Christensen
b9fb66401b
Merge pull request #880 from grahamc/runcommand-finished-bool 
RunCommand: emit the `finished` field as a boolean
 2021-03-09 09:58:43 -05:00
Graham Christensen
2179b4b4b0
RunCommand: emit the finished field as a boolean 2021-03-08 12:11:20 -05:00
Matej Cotman
a551fba346
statsd: add a chance to set hostname and port in hydra.conf 
Co-authored-by: Graham Christensen <graham@grahamc.com>
 2021-03-08 10:03:16 -05:00
Cole Helbling
025be052b7
tests: move to t, allow yath test from root 
By moving the tests subdirectory to t, we gain the ability to run `yath
test` with no arguments from inside `nix develop` in the root of the
the repo.

(`nix develop` is necessary in order to set the proper env vars for
`yath` to find our test libraries.)
 2021-03-05 09:49:06 -08:00