Commit graph

3357 commits

Author SHA1 Message Date
Damien Diederen df7dab1291 GitInput: Include deepClone option in the cache key
Without this commit, two jobsets using the same repository as input,
but different `deepClone` options, end up incorrectly sharing the same
"checkout" for a given (`uri`, `branch`, `revision`) tuple.  The
presence or absence of `.git` is determined by the jobset execution
order.

This patch adds the missing `isDeepClone` boolean to the cache key.

The database upgrade script empties the `CachedGitInputs` table, as we
don't know if existing checkouts are deep clones.  Unfortunately, this
generally forces rebuilds even for correct `deepClone` checkouts, as
the binary contents of `.git` are not deterministic.

Fixes #510
2021-06-19 17:37:40 +02:00
Graham Christensen b6921c2006
Merge pull request #974 from AmandaCameron/patch-1
flake: stdenv.lib -> lib
2021-06-17 17:11:51 -04:00
Amanda Cameron b0f8a207fe
flake: stdenv.lib -> lib
Nixpkgs on unstable has removed `stdenv.lib` as they've been warning for a while now. This removes the extra references to it in the flake.nix

I'm not entirely sure if this is right, but I figured it was trivial enough to give a quick try using the GH Editor while I was waiting for a job to finish
2021-06-17 16:50:38 -04:00
Graham Christensen ac1ef9361e
Merge pull request #913 from grahamc/notif-docs
Notifications: write up some docs
2021-06-17 13:27:21 -04:00
Graham Christensen 6eb701fcf2 notifications: Document an example scenario where builds_finished is triggered twice. 2021-06-17 13:20:13 -04:00
Graham Christensen e8afde5079 Write up some documentation on notifications 2021-06-17 13:15:34 -04:00
Graham Christensen 4375268276
Merge pull request #973 from DeterminateSystems/build-evals
/build/ID/evals: fix after #860
2021-06-16 13:36:20 -04:00
Graham Christensen 09ad52ab60 Document getEvals a bit
Also drop the $self parameter.

Co-authored-by: gustavderdrache
2021-06-16 13:15:01 -04:00
Graham Christensen 5d95abf540 getBuilds: clarify the names of evals vs. the query builder 2021-06-16 11:53:04 -04:00
Graham Christensen cb8929b7ed Tighten up 'should exit with return code' 2021-06-16 11:48:49 -04:00
Graham Christensen bf5c76feb6 getEvals: order by the eval table's ID
I broke this when I added `me.` in f1e75c8bff

I added me. to disambiguate `id`, but:

* eval.id works on the per-build page
* me.id works on the other pages
* Just id works everywhere if I drop:

    , prefetch => { evaluationerror => [ ] },

  but this causes a query per row to collect the evaluationerror
  records later, this becomes significantly slow on non-trivial
  datasets.

Using evals->current_source_alias will use the correct alias
whether it is me or eval or something else.
2021-06-16 11:24:30 -04:00
Graham Christensen 2ac47e8013 Test that each page listing evals works
PR #860 caused a regression that broke some loads.
2021-06-16 11:22:40 -04:00
Graham Christensen 75855c202f
Merge pull request #972 from DeterminateSystems/prom
Prometheus Metrics for the Webserver
2021-06-11 09:16:15 -04:00
Graham Christensen 5700c351e8 Init a docs section for monitoring, document queue-runner-status and the prometheus metrics 2021-06-10 20:13:08 -04:00
Graham Christensen 71453dd341 Expose Prometheus metrics at /metrics
Exposes metrics:

* http_request_duration_seconds_bucket
* http_request_size_bytes_bucket
* http_response_size_bytes_bucket
* http_requests_total

with labels of action and controller to help identify popular
endpoints and their performance characteristics.
2021-06-10 20:02:42 -04:00
Graham Christensen db6bad108b Use PrometheusTinyShared 2021-06-10 17:47:11 -04:00
Graham Christensen 1652631b21 Prometheus: init Tiny, TinyShared 2021-06-10 17:47:11 -04:00
Graham Christensen c6670b805f init: HashSharedMem for Prometheus 2021-06-10 17:47:11 -04:00
Graham Christensen b1fcc0c25b init DataRandom perl package for Prometheus stats 2021-06-10 17:47:11 -04:00
Graham Christensen e04dc14d94
Merge pull request #969 from DeterminateSystems/builds-index-jobset-id
Builds: index on jobset_id, largely join by jobset_id
2021-06-01 17:59:39 -04:00
Graham Christensen 382ba590ad Drop unused query from GET Build. 2021-06-01 11:47:05 -04:00
Graham Christensen a9e4ede006 SQL: create better indexes for builds based on the jobset id
These are primarily used by the jobsetOverview renders.
2021-06-01 11:23:22 -04:00
Graham Christensen 719d0a6134 jobset overview: join jobsets to builds by jobset id 2021-06-01 11:19:33 -04:00
Graham Christensen 508d99d611 Join to builds via jobset_id when easy 2021-06-01 11:16:47 -04:00
Graham Christensen 5b6b8261fc
Merge pull request #968 from DeterminateSystems/unnecessary-declarative-field
Projects: don't respond with unnecessary declarative field
2021-05-24 19:21:00 -04:00
Cole Helbling 45ced33ab6
Projects: don't respond with unnecessary declarative field
If the project isn't declarative, who cares about it in the response? After
setting the `declfile` to an empty string, everything related to declarative-
ness is wiped out, anyways.
2021-05-24 15:26:34 -07:00
Eelco Dolstra cdba2b86c0
Merge pull request #967 from Ma27/hydra-system-users
Set `isSystemUser = true;` for each statically declared user
2021-05-24 16:09:15 +02:00
Maximilian Bosch 302f5e0de9
Set isSystemUser = true; for each statically declared user
This is needed to evaluate the module on 21.05.
2021-05-24 14:47:30 +02:00
Graham Christensen 3dc745de68
Merge pull request #964 from DeterminateSystems/fix-cascade
Jobsets: remove defunct Jobs relationship
2021-05-09 21:17:12 -04:00
Cole Helbling 7ba58ca3ea
Jobsets: remove defunct Jobs relationship
It appears the Jobs table was removed in
8adb433e3b, but the Jobsets schema was never
updated to reflect this. This relationship was added in
efa1f1d4fb, roughly 3 months prior.

Previously, one would see a message similar to the following logged when
deleting a jobset:

    17:38:23 hydra-server.1       | DBIx::Class::Relationship::CascadeActions::delete(): Skipping cascade delete on relationship 'jobs' - related resultsource 'Hydra::Schema::Jobs' is not registered with this schema at /home/vin/workspace/vcs/hydra/src/script/../lib/Hydra/Controller/Jobset.pm line 106
2021-05-09 17:52:48 -07:00
Graham Christensen 67b6f0d7ed
Merge pull request #962 from DeterminateSystems/del-.jobsets
Project: delete the `.jobsets` jobset if project is no longer declarative
2021-05-06 11:57:34 -04:00
Cole Helbling 588a3a774f
Project: add test for declarative->normal project transition
Also split into subtests.
2021-05-06 08:47:29 -07:00
Cole Helbling 6107040bf5
Project: clear decltype and declvalue when project is no longer declarative 2021-05-06 08:47:18 -07:00
Cole Helbling bd9c52dbd0
Project: delete the .jobsets jobset if project is no longer declarative
"No longer declarative" as defined by the "Edit project" page is an empty spec
file.
2021-05-05 14:03:51 -07:00
Graham Christensen bc8619af3f
Merge pull request #961 from DeterminateSystems/fix-959
lazy tabs: trigger the load event
2021-05-05 14:51:33 -04:00
Graham Christensen 6c1151b14a lazy tabs: trigger the load event
Something in the upgrade of Bootstrap and JQuery broke lazy tab loading.
I don't understand what is providing the tab behavior, how it should
work, or what the correct fix is.

I can tell you that this patch fixes the issue: when loading a tab
with a URL fragment deep-linking to a lazily loaded tab... it now
loads.

Close #959
2021-05-05 14:42:12 -04:00
Eelco Dolstra 886e6f85e4
Merge pull request #924 from hackworthltd/fix-flake-github-status
Fix GitHub status update for private flakes.
2021-05-03 16:24:51 +02:00
Graham Christensen 6e537671df
Merge pull request #954 from DeterminateSystems/projects-json-schema
Projects: use JSON in Schema
2021-05-02 21:35:04 -04:00
Cole Helbling f3947acfc4
Projects: use JSON in Schema 2021-05-02 18:25:37 -07:00
Graham Christensen 5520f4b7b6
Merge pull request #952 from DeterminateSystems/decl-api
Project: add declfile, decltype, declvalue to API
2021-05-02 20:51:27 -04:00
Cole Helbling ad13d13436
Project: add declfile, decltype, declvalue to API
This makes it possible to create declarative projects via a PUT request, and
also exposes the currently-configured values to GET requests.
2021-05-02 17:45:14 -07:00
Graham Christensen e9a06113c9
Merge pull request #951 from DeterminateSystems/delete-project
Fix deleting projects
2021-05-01 13:01:39 -04:00
Cole Helbling 7c9ed6b919
Project: fix DELETE route
This appears to have been broken in ac3e8a4a59,
which removed the `jobsetevals` column from the Projects schema, but didn't
update the Controller accordingly.

Fixes the test added in the previous commit.
2021-04-30 10:43:21 -07:00
Cole Helbling d0682c804d
api-test: test deleting a project
As of this commit, this test will fail, because the underlying cause hasn't been
fixed.
2021-04-30 10:42:38 -07:00
Graham Christensen 6047b1dd04
Merge pull request #948 from DeterminateSystems/known-inputs
Jobset: list known inputs in invalid input type response
2021-04-29 21:45:16 -04:00
Cole Helbling 14aff5678c
hydra-api: don't restrict input types
Plugins can add their own input types; we don't want to restrict this.
2021-04-29 18:19:44 -07:00
Cole Helbling 03d5a858a7
Jobset: list known inputs in invalid input type response 2021-04-29 18:19:44 -07:00
Graham Christensen 18d271655c
Merge pull request #945 from DeterminateSystems/fixup-ToJSON
Jobsets,JobsetInputs: use JSON
2021-04-29 08:38:45 -04:00
Graham Christensen 8492db033e Jobsets,JobsetInputs: use JSON 2021-04-29 08:32:47 -04:00
Graham Christensen ec288e6d88
Merge pull request #941 from DeterminateSystems/align-api
ToJSON: allow custom as_json function; JobsetInputs, Jobsets: update schema to align with the API
2021-04-29 07:57:39 -04:00