Commit graph

2596 commits

Author SHA1 Message Date
Graham Christensen a14c8ad5f8
Merge pull request #995 from DeterminateSystems/declarative-jobsets-plugin
Declarative jobsets: move event handling to a plugin
2021-08-12 15:56:13 -04:00
Graham Christensen 0f958f3425
Merge pull request #997 from DeterminateSystems/abstract-listener
Abstract over postgres' LISTEN/NOTIFY
2021-08-12 14:00:34 -04:00
Graham Christensen 5027003285 Abstract over postgres' LISTEN/NOTIFY
This lets us test the event loop if we wanted, and lets us
test the listening behavior in isolation.
2021-08-12 13:54:05 -04:00
Graham Christensen 593af41808 Declarative jobsets: move event handling to a plugin
Declarative jobsets were sort of tucked in to the event hanlder
itself. It turned out that it could have been implemented as a
plugin without much trouble.
2021-08-12 12:48:18 -04:00
Graham Christensen 9c5f317453 hydra-notify: move buildFinished query in to the function impl
This is more consistent with the other event handlers, of dealing
with IDs and not objects.
2021-08-12 12:30:35 -04:00
Eelco Dolstra 5edb58b314 Fix build 2021-08-10 13:47:16 +02:00
Graham Christensen c7c322545d
Merge pull request #992 from DeterminateSystems/sql/fixup-comment
hydra.sql: Update comment on regeneration
2021-08-06 14:54:12 -04:00
Graham Christensen 4169f22231 update-dbix.pl: correct indentation 2021-08-06 14:40:57 -04:00
Graham Christensen 5bd8dc171b hydra.sql: Update comment on regeneration
We no longer need to generate the hydra-postgres.sql document,
that is a relic from when sqlite was also supported.
2021-08-06 14:40:34 -04:00
Shay Bergmann 503b0e0b6f
Remove unnecessary comment 2021-07-26 17:16:40 +00:00
Shay Bergmann 173ecb5d92
Apply requested changes to Config.pm 2021-07-26 17:16:40 +00:00
Shay Bergmann 4f95774ca3
Factor out common options 2021-07-26 03:43:09 +00:00
Shay Bergmann e98bd0ec8d
Comment around dupe to avoid drift 2021-07-22 23:27:21 -04:00
Shay Bergmann 54c8028ec0
Make hydra-server honor config file includes 2021-07-22 17:25:30 -04:00
Nathan van Doorn 5c47343b40 Fix deprecation warnings with libpqxx usage 2021-07-05 19:45:25 +01:00
Graham Christensen e02f34722f
Merge pull request #887 from helsinki-systems/feat/eval-overview-duration
Show evaluation time in the evaluations overview
2021-06-29 10:59:02 -04:00
Janne Heß 94b88b34ae
Show evaluation time in the evaluations overview
Parts of this code have been authored by @grahamc
2021-06-29 16:53:10 +02:00
Damien Diederen df7dab1291 GitInput: Include deepClone option in the cache key
Without this commit, two jobsets using the same repository as input,
but different `deepClone` options, end up incorrectly sharing the same
"checkout" for a given (`uri`, `branch`, `revision`) tuple.  The
presence or absence of `.git` is determined by the jobset execution
order.

This patch adds the missing `isDeepClone` boolean to the cache key.

The database upgrade script empties the `CachedGitInputs` table, as we
don't know if existing checkouts are deep clones.  Unfortunately, this
generally forces rebuilds even for correct `deepClone` checkouts, as
the binary contents of `.git` are not deterministic.

Fixes #510
2021-06-19 17:37:40 +02:00
Graham Christensen 09ad52ab60 Document getEvals a bit
Also drop the $self parameter.

Co-authored-by: gustavderdrache
2021-06-16 13:15:01 -04:00
Graham Christensen 5d95abf540 getBuilds: clarify the names of evals vs. the query builder 2021-06-16 11:53:04 -04:00
Graham Christensen bf5c76feb6 getEvals: order by the eval table's ID
I broke this when I added `me.` in f1e75c8bff

I added me. to disambiguate `id`, but:

* eval.id works on the per-build page
* me.id works on the other pages
* Just id works everywhere if I drop:

    , prefetch => { evaluationerror => [ ] },

  but this causes a query per row to collect the evaluationerror
  records later, this becomes significantly slow on non-trivial
  datasets.

Using evals->current_source_alias will use the correct alias
whether it is me or eval or something else.
2021-06-16 11:24:30 -04:00
Graham Christensen 71453dd341 Expose Prometheus metrics at /metrics
Exposes metrics:

* http_request_duration_seconds_bucket
* http_request_size_bytes_bucket
* http_response_size_bytes_bucket
* http_requests_total

with labels of action and controller to help identify popular
endpoints and their performance characteristics.
2021-06-10 20:02:42 -04:00
Graham Christensen 382ba590ad Drop unused query from GET Build. 2021-06-01 11:47:05 -04:00
Graham Christensen a9e4ede006 SQL: create better indexes for builds based on the jobset id
These are primarily used by the jobsetOverview renders.
2021-06-01 11:23:22 -04:00
Graham Christensen 719d0a6134 jobset overview: join jobsets to builds by jobset id 2021-06-01 11:19:33 -04:00
Graham Christensen 508d99d611 Join to builds via jobset_id when easy 2021-06-01 11:16:47 -04:00
Cole Helbling 45ced33ab6
Projects: don't respond with unnecessary declarative field
If the project isn't declarative, who cares about it in the response? After
setting the `declfile` to an empty string, everything related to declarative-
ness is wiped out, anyways.
2021-05-24 15:26:34 -07:00
Graham Christensen 3dc745de68
Merge pull request #964 from DeterminateSystems/fix-cascade
Jobsets: remove defunct Jobs relationship
2021-05-09 21:17:12 -04:00
Cole Helbling 7ba58ca3ea
Jobsets: remove defunct Jobs relationship
It appears the Jobs table was removed in
8adb433e3b, but the Jobsets schema was never
updated to reflect this. This relationship was added in
efa1f1d4fb, roughly 3 months prior.

Previously, one would see a message similar to the following logged when
deleting a jobset:

    17:38:23 hydra-server.1       | DBIx::Class::Relationship::CascadeActions::delete(): Skipping cascade delete on relationship 'jobs' - related resultsource 'Hydra::Schema::Jobs' is not registered with this schema at /home/vin/workspace/vcs/hydra/src/script/../lib/Hydra/Controller/Jobset.pm line 106
2021-05-09 17:52:48 -07:00
Graham Christensen 67b6f0d7ed
Merge pull request #962 from DeterminateSystems/del-.jobsets
Project: delete the `.jobsets` jobset if project is no longer declarative
2021-05-06 11:57:34 -04:00
Cole Helbling 6107040bf5
Project: clear decltype and declvalue when project is no longer declarative 2021-05-06 08:47:18 -07:00
Cole Helbling bd9c52dbd0
Project: delete the .jobsets jobset if project is no longer declarative
"No longer declarative" as defined by the "Edit project" page is an empty spec
file.
2021-05-05 14:03:51 -07:00
Graham Christensen 6c1151b14a lazy tabs: trigger the load event
Something in the upgrade of Bootstrap and JQuery broke lazy tab loading.
I don't understand what is providing the tab behavior, how it should
work, or what the correct fix is.

I can tell you that this patch fixes the issue: when loading a tab
with a URL fragment deep-linking to a lazily loaded tab... it now
loads.

Close #959
2021-05-05 14:42:12 -04:00
Eelco Dolstra 886e6f85e4
Merge pull request #924 from hackworthltd/fix-flake-github-status
Fix GitHub status update for private flakes.
2021-05-03 16:24:51 +02:00
Cole Helbling f3947acfc4
Projects: use JSON in Schema 2021-05-02 18:25:37 -07:00
Cole Helbling ad13d13436
Project: add declfile, decltype, declvalue to API
This makes it possible to create declarative projects via a PUT request, and
also exposes the currently-configured values to GET requests.
2021-05-02 17:45:14 -07:00
Cole Helbling 7c9ed6b919
Project: fix DELETE route
This appears to have been broken in ac3e8a4a59,
which removed the `jobsetevals` column from the Projects schema, but didn't
update the Controller accordingly.

Fixes the test added in the previous commit.
2021-04-30 10:43:21 -07:00
Cole Helbling 03d5a858a7
Jobset: list known inputs in invalid input type response 2021-04-29 18:19:44 -07:00
Graham Christensen 8492db033e Jobsets,JobsetInputs: use JSON 2021-04-29 08:32:47 -04:00
Cole Helbling f1dd5d202e Jobsets: update schema to align with the API
To further align with the API, we return custom JSON in order to display a
`visible` field rather than `hidden` -- a `PUT` request expects `visible`, while
a `GET` request returns `hidden`.

This also allows us to rename the `jobsetinputs` field to `inputs` for the same
reason: `PUT` expects `inputs`, while `GET` returns `jobsetinputs`.
2021-04-29 07:50:23 -04:00
Cole Helbling d23f431889 JobsetInputs: update schema to align with the API
`PUT /jobsets/{project-id}/{jobset-id}` expects a JSON object `inputs` which
maps a name to a name, a type, a value, and a boolean that enables emailing
responsible parties. However, `GET /jobsets/{project-id}/{jobset-id}` responds
with an object that doesn't contain a value, but does contain a jobsetinputalts
(which is old and should be unused).

This commit aligns the two by removing the old and unused `jobsetinputalts` from
the response and replaces it with `value`.
2021-04-29 07:50:23 -04:00
Cole Helbling fff0db10e3 ToJSON: allow custom as_json function
This allows us to modify what the API responds with, which in turn lets us unify
the OpenAPI specification and the actual API's responses.
2021-04-29 07:50:23 -04:00
Graham Christensen fdb6e7dd97 fixup: return the user entity on login 2021-04-28 18:30:35 -04:00
Graham Christensen 823da22e4f
Merge pull request #937 from DeterminateSystems/flesh-out-api
hydra-api: flesh out Jobset, JobsetInput schemas; implement DELETE /jobset/{project-id}/{jobset-id}
2021-04-28 13:43:04 -04:00
Graham Christensen 725c9c2f81
login: redirect to the current-user page 2021-04-28 08:32:10 -07:00
Graham Christensen d589db2ed9
login: missing parameters are 400s 2021-04-28 08:31:59 -07:00
regnat abff212d06 Use system-features from the Nix conf in the default machine file
Fix #936
2021-04-28 11:43:04 +02:00
Cole Helbling 72fec31dbb
hydra-api: flesh out JobsetInput schema 2021-04-27 16:16:42 -07:00
Cole Helbling 2600810551
hydra-api: flesh out Jobset schema
* made all columns available via the API (except for forceeval)
* renamed flakeref to flake to unify the API with the database schema
* renamed inputs to jobsetinputs to unify the API with the database schema
2021-04-27 16:16:42 -07:00
Cole Helbling 50fab154a4
ToJSON: serialize string_columns to JSON
If the column is undefined, then it should be an empty string according to your
API spec.
2021-04-26 16:39:13 -07:00
Graham Christensen f2b9649bf2
Projects: serialize enabled and hidden as boolean 2021-04-26 16:03:32 -07:00
Graham Christensen 4aea02e1e1
ToJSON: serialize boolean_columns to JSON boolean 2021-04-26 16:03:32 -07:00
Cole Helbling c757867b9e
Add homepage to Projects schema 2021-04-26 15:46:30 -07:00
Graham Christensen 453b8479be
Merge pull request #927 from cole-h/nonexistent-user-400
Return HTTP 400 when creating Project with nonexistent user
2021-04-26 14:40:15 -04:00
Cole Helbling 47e19ba22c
Return HTTP 400 when creating Project with nonexistent user 2021-04-26 11:32:39 -07:00
Drew Hess 523d6df5b8
Fix GitHub status update for private flakes.
Also, if the parse fails, don't try to update the GitHub status, as
this will eventually cause rate-limiting.
2021-04-26 01:38:24 +01:00
Maximilian Bosch 21ed005c84
Make it possible to enable email notifications when creating a jobset
The checkbox is only enabled if `email_notification = 1` is set in
`hydra.conf`. However, when creating jobset (in contrast to the edit
form), the checkbox is always disabled because the `emailNotification`
parameter in Catalyst's stash was missing.
2021-04-24 19:48:43 +02:00
Graham Christensen 05636de7d2 hydra-init: upgrade passwords to Argon2 on startup 2021-04-16 12:32:13 -04:00
Graham Christensen 79b0ddc27d hydra-create-user: re-hash sha1 as Argon2 2021-04-16 12:32:13 -04:00
Graham Christensen d10d8964f2 Users: add a validation step which lets the user's password be a Argon2 hashed sha1 hash.
OWASP suggests expiring all passwords and requiring users to update their password.
However, we don't have a way to do this. They suggest this mechanism
as a good alternative:
https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html#upgrading-legacy-hashes
2021-04-16 12:32:13 -04:00
Eelco Dolstra fa924ea697
Merge pull request #915 from grahamc/hydra-auth
Hydra auth: support Argon2, transparently upgrade hashes
2021-04-15 17:40:32 +02:00
Graham Christensen 9225be0897 Drop remaining sha1_hex references
Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:35:18 -04:00
Graham Christensen 1d956be61e hydra-create-user: support Argon2
Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:35:16 -04:00
Graham Christensen beb5be4302 Users: password changes via the web UI now use Argon2
Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:35:13 -04:00
Graham Christensen 1da70030b7 Users: transparently upgrade passwords to Argon2
Passwords that are sha1 will be transparently upgraded to argon2,
and future comparisons will use Argon2

Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:35:11 -04:00
Graham Christensen 29620df85e Passwords: check in constant time
The default password comparison logic does not use
constant time validation. Switching to constant time
offers a meager improvement by removing a timing
oracle.

A prepatory step in moving to Argon2id password storage, since we'll need this change anyway after
for validating existing passwords.

Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:34:56 -04:00
Graham Christensen d4d8f1ba1b Plugin::Authentication config: modernize
Some time in the last decade the plugin switched to preferring
a flatter namespace for realm config.

Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:34:47 -04:00
Eelco Dolstra 0bee194ce9
Merge pull request #914 from Ma27/fix-remote-builds
Fix `std::bad_alloc` errors for remote builds
2021-04-15 17:05:54 +02:00
Maximilian Bosch 2808227eb7
Fix std::bad_alloc errors for remote builds
In Nix the protocol was slightly altered[1] to also contain more
information about realisations. This however wasn't read from the pipe
that was used to read from the store.

After the `cmdBuildDerivation` command which caused this issue, Hydra
will issue a `cmdQueryPathInfos` that tries to read from the remote
store as well. However, there's still left over to read from the
previous command and thus Nix fails to properly allocate the expected
string.

[1] See rev a2b69660a9b326b95d48bd222993c5225bbd5b5f

Fixes #898
2021-04-15 15:16:52 +02:00
Graham Christensen b9bcedbfdb
Merge pull request #596 from kquick/local_inp_url
Update prompt for Local path input to indicate a URL is also valid.
2021-04-14 20:01:58 +00:00
Graham Christensen ffc4be6743
Merge pull request #910 from grahamc/ui-fixups
UI fixups
2021-04-12 17:42:01 +00:00
Graham Christensen afd064d19d
Merge pull request #867 from ck3d/fix-proxy-login
Fix login if Hydra runs behind HTTP proxy with sub-path location
2021-04-12 17:36:55 +00:00
Graham Christensen 82953389e4 Make new jobsets enabled by default in the UI
If the jobset is brand new, none of the options matched. This
makes all new jobsets default to Enabled.
2021-04-12 12:01:24 -04:00
Graham Christensen 0fe4cba5b7 edit-jobset: make the "add a new input" button span the whole way
Previously it would only cross 4 of the 5 columns.
2021-04-12 12:01:24 -04:00
Graham Christensen e674fb6139 edit user: make the role changer checkboxes
Also, it makes the role changer a bit of a dangerzone visually for
admins.
2021-04-09 14:07:10 +00:00
Graham Christensen 258b39f1e5
Merge pull request #677 from twhitehead/javascripts-update
Javascript libraries update
2021-04-09 14:05:47 +00:00
Eelco Dolstra 20c1efeb5b
Merge pull request #904 from Ma27/gitea-integration
Add `GiteaStatus`-Plugin
2021-04-08 17:57:38 +02:00
Tyson Whitehead e51a6a4bca
Cards are now required to decorate pre blocks 2021-04-08 11:32:30 -04:00
Tyson Whitehead 7d3bd08c9b
Fix tab link spacing issue by dropping old sticky navbar css hack 2021-04-08 11:32:29 -04:00
Samuel Dionne-Riel def05105bb Add top spacing to tab contents 2021-04-08 11:32:26 -04:00
Samuel Dionne-Riel d85b846357 Fix spacing with search inputs
And more importantly, crucially, fix the "appended" button so the round
corners are on the right (heh) side.
2021-04-08 11:32:08 -04:00
Samuel Dionne-Riel 20e3c51ba9 Style unstyled alerts 2021-04-08 11:32:02 -04:00
Graham Christensen db2c9fe96a layout.tt: make the header constrained to the center 2021-04-08 11:31:56 -04:00
Graham Christensen 9c903e9991 renderEvals: fixup colspan on Jobs and More 2021-04-08 11:31:47 -04:00
Graham Christensen c8dd5b57c4 src/root/Makefile: fixup extraction of the fontawesome and bootstrap archives 2021-04-08 11:31:39 -04:00
Tyson Whitehead abe082c1d6
Couple of upstream/bootstrap items missed in rebase 2021-04-08 11:31:34 -04:00
Tyson Whitehead b5e18ce62c
Fix the empty popovers 2021-04-08 11:31:34 -04:00
Tyson Whitehead 2902684c61
Update jobset comparison button 2021-04-08 11:31:34 -04:00
Tyson Whitehead 3f77bd240f
Bring deep-embedding (tab anchors) javascript up-to-date 2021-04-08 11:31:34 -04:00
Tyson Whitehead b6f5b6eb38
Labels are now badges 2021-04-08 11:31:34 -04:00
Tyson Whitehead 70df5e6312
Update button styles as btn-mini is no more 2021-04-08 11:31:34 -04:00
Graham Christensen 1e34cd3d3d fixups to the rebase 2021-04-08 11:30:49 -04:00
regnat 26ffd4a93e Fix build with latest master 2021-04-08 17:11:15 +02:00
Graham Christensen cc9c91fe12 jobsets: put hidden and enabled jobsets at the end
Allows for generally correct zebra striping
2021-03-31 14:33:20 +00:00
Graham Christensen a46f655c56 root project listing: show hidden projects at the end
Makes the zebra striping correct.
2021-03-31 14:33:20 +00:00
Maximilian Bosch f9f5ab2fb1
Make gitea public URL configurable
Otherwise, it will be obtained from the jobset input that contains the
URL to the git repo to build.
2021-03-30 23:01:36 +02:00
Maximilian Bosch eecea56131
Implement VM-test for gitea plugin 2021-03-30 22:35:39 +02:00
Maximilian Bosch 56997d8e8b
Fix error codes for GiteaStatus plugin
* `failure` if a build error occurred, on e.g. an aborted build send
  `error`.
2021-03-30 14:13:46 +02:00
Maximilian Bosch fef142f13a
Implement simple status notifications for Git repos hosted on gitea 2021-03-30 14:10:21 +02:00
Graham Christensen 9bb04ed97a
Merge pull request #893 from grahamc/fake-channel-nested
Test the fake derivations channel, asserting nested packages are properly represented.
2021-03-24 17:56:55 -04:00