Commit graph

2569 commits

Author SHA1 Message Date
Graham Christensen 3dc745de68
Merge pull request #964 from DeterminateSystems/fix-cascade
Jobsets: remove defunct Jobs relationship
2021-05-09 21:17:12 -04:00
Cole Helbling 7ba58ca3ea
Jobsets: remove defunct Jobs relationship
It appears the Jobs table was removed in
8adb433e3b, but the Jobsets schema was never
updated to reflect this. This relationship was added in
efa1f1d4fb, roughly 3 months prior.

Previously, one would see a message similar to the following logged when
deleting a jobset:

    17:38:23 hydra-server.1       | DBIx::Class::Relationship::CascadeActions::delete(): Skipping cascade delete on relationship 'jobs' - related resultsource 'Hydra::Schema::Jobs' is not registered with this schema at /home/vin/workspace/vcs/hydra/src/script/../lib/Hydra/Controller/Jobset.pm line 106
2021-05-09 17:52:48 -07:00
Graham Christensen 67b6f0d7ed
Merge pull request #962 from DeterminateSystems/del-.jobsets
Project: delete the `.jobsets` jobset if project is no longer declarative
2021-05-06 11:57:34 -04:00
Cole Helbling 6107040bf5
Project: clear decltype and declvalue when project is no longer declarative 2021-05-06 08:47:18 -07:00
Cole Helbling bd9c52dbd0
Project: delete the .jobsets jobset if project is no longer declarative
"No longer declarative" as defined by the "Edit project" page is an empty spec
file.
2021-05-05 14:03:51 -07:00
Graham Christensen 6c1151b14a lazy tabs: trigger the load event
Something in the upgrade of Bootstrap and JQuery broke lazy tab loading.
I don't understand what is providing the tab behavior, how it should
work, or what the correct fix is.

I can tell you that this patch fixes the issue: when loading a tab
with a URL fragment deep-linking to a lazily loaded tab... it now
loads.

Close #959
2021-05-05 14:42:12 -04:00
Eelco Dolstra 886e6f85e4
Merge pull request #924 from hackworthltd/fix-flake-github-status
Fix GitHub status update for private flakes.
2021-05-03 16:24:51 +02:00
Cole Helbling f3947acfc4
Projects: use JSON in Schema 2021-05-02 18:25:37 -07:00
Cole Helbling ad13d13436
Project: add declfile, decltype, declvalue to API
This makes it possible to create declarative projects via a PUT request, and
also exposes the currently-configured values to GET requests.
2021-05-02 17:45:14 -07:00
Cole Helbling 7c9ed6b919
Project: fix DELETE route
This appears to have been broken in ac3e8a4a59,
which removed the `jobsetevals` column from the Projects schema, but didn't
update the Controller accordingly.

Fixes the test added in the previous commit.
2021-04-30 10:43:21 -07:00
Cole Helbling 03d5a858a7
Jobset: list known inputs in invalid input type response 2021-04-29 18:19:44 -07:00
Graham Christensen 8492db033e Jobsets,JobsetInputs: use JSON 2021-04-29 08:32:47 -04:00
Cole Helbling f1dd5d202e Jobsets: update schema to align with the API
To further align with the API, we return custom JSON in order to display a
`visible` field rather than `hidden` -- a `PUT` request expects `visible`, while
a `GET` request returns `hidden`.

This also allows us to rename the `jobsetinputs` field to `inputs` for the same
reason: `PUT` expects `inputs`, while `GET` returns `jobsetinputs`.
2021-04-29 07:50:23 -04:00
Cole Helbling d23f431889 JobsetInputs: update schema to align with the API
`PUT /jobsets/{project-id}/{jobset-id}` expects a JSON object `inputs` which
maps a name to a name, a type, a value, and a boolean that enables emailing
responsible parties. However, `GET /jobsets/{project-id}/{jobset-id}` responds
with an object that doesn't contain a value, but does contain a jobsetinputalts
(which is old and should be unused).

This commit aligns the two by removing the old and unused `jobsetinputalts` from
the response and replaces it with `value`.
2021-04-29 07:50:23 -04:00
Cole Helbling fff0db10e3 ToJSON: allow custom as_json function
This allows us to modify what the API responds with, which in turn lets us unify
the OpenAPI specification and the actual API's responses.
2021-04-29 07:50:23 -04:00
Graham Christensen fdb6e7dd97 fixup: return the user entity on login 2021-04-28 18:30:35 -04:00
Graham Christensen 823da22e4f
Merge pull request #937 from DeterminateSystems/flesh-out-api
hydra-api: flesh out Jobset, JobsetInput schemas; implement DELETE /jobset/{project-id}/{jobset-id}
2021-04-28 13:43:04 -04:00
Graham Christensen 725c9c2f81
login: redirect to the current-user page 2021-04-28 08:32:10 -07:00
Graham Christensen d589db2ed9
login: missing parameters are 400s 2021-04-28 08:31:59 -07:00
regnat abff212d06 Use system-features from the Nix conf in the default machine file
Fix #936
2021-04-28 11:43:04 +02:00
Cole Helbling 72fec31dbb
hydra-api: flesh out JobsetInput schema 2021-04-27 16:16:42 -07:00
Cole Helbling 2600810551
hydra-api: flesh out Jobset schema
* made all columns available via the API (except for forceeval)
* renamed flakeref to flake to unify the API with the database schema
* renamed inputs to jobsetinputs to unify the API with the database schema
2021-04-27 16:16:42 -07:00
Cole Helbling 50fab154a4
ToJSON: serialize string_columns to JSON
If the column is undefined, then it should be an empty string according to your
API spec.
2021-04-26 16:39:13 -07:00
Graham Christensen f2b9649bf2
Projects: serialize enabled and hidden as boolean 2021-04-26 16:03:32 -07:00
Graham Christensen 4aea02e1e1
ToJSON: serialize boolean_columns to JSON boolean 2021-04-26 16:03:32 -07:00
Cole Helbling c757867b9e
Add homepage to Projects schema 2021-04-26 15:46:30 -07:00
Graham Christensen 453b8479be
Merge pull request #927 from cole-h/nonexistent-user-400
Return HTTP 400 when creating Project with nonexistent user
2021-04-26 14:40:15 -04:00
Cole Helbling 47e19ba22c
Return HTTP 400 when creating Project with nonexistent user 2021-04-26 11:32:39 -07:00
Drew Hess 523d6df5b8
Fix GitHub status update for private flakes.
Also, if the parse fails, don't try to update the GitHub status, as
this will eventually cause rate-limiting.
2021-04-26 01:38:24 +01:00
Maximilian Bosch 21ed005c84
Make it possible to enable email notifications when creating a jobset
The checkbox is only enabled if `email_notification = 1` is set in
`hydra.conf`. However, when creating jobset (in contrast to the edit
form), the checkbox is always disabled because the `emailNotification`
parameter in Catalyst's stash was missing.
2021-04-24 19:48:43 +02:00
Graham Christensen 05636de7d2 hydra-init: upgrade passwords to Argon2 on startup 2021-04-16 12:32:13 -04:00
Graham Christensen 79b0ddc27d hydra-create-user: re-hash sha1 as Argon2 2021-04-16 12:32:13 -04:00
Graham Christensen d10d8964f2 Users: add a validation step which lets the user's password be a Argon2 hashed sha1 hash.
OWASP suggests expiring all passwords and requiring users to update their password.
However, we don't have a way to do this. They suggest this mechanism
as a good alternative:
https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html#upgrading-legacy-hashes
2021-04-16 12:32:13 -04:00
Eelco Dolstra fa924ea697
Merge pull request #915 from grahamc/hydra-auth
Hydra auth: support Argon2, transparently upgrade hashes
2021-04-15 17:40:32 +02:00
Graham Christensen 9225be0897 Drop remaining sha1_hex references
Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:35:18 -04:00
Graham Christensen 1d956be61e hydra-create-user: support Argon2
Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:35:16 -04:00
Graham Christensen beb5be4302 Users: password changes via the web UI now use Argon2
Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:35:13 -04:00
Graham Christensen 1da70030b7 Users: transparently upgrade passwords to Argon2
Passwords that are sha1 will be transparently upgraded to argon2,
and future comparisons will use Argon2

Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:35:11 -04:00
Graham Christensen 29620df85e Passwords: check in constant time
The default password comparison logic does not use
constant time validation. Switching to constant time
offers a meager improvement by removing a timing
oracle.

A prepatory step in moving to Argon2id password storage, since we'll need this change anyway after
for validating existing passwords.

Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:34:56 -04:00
Graham Christensen d4d8f1ba1b Plugin::Authentication config: modernize
Some time in the last decade the plugin switched to preferring
a flatter namespace for realm config.

Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:34:47 -04:00
Eelco Dolstra 0bee194ce9
Merge pull request #914 from Ma27/fix-remote-builds
Fix `std::bad_alloc` errors for remote builds
2021-04-15 17:05:54 +02:00
Maximilian Bosch 2808227eb7
Fix std::bad_alloc errors for remote builds
In Nix the protocol was slightly altered[1] to also contain more
information about realisations. This however wasn't read from the pipe
that was used to read from the store.

After the `cmdBuildDerivation` command which caused this issue, Hydra
will issue a `cmdQueryPathInfos` that tries to read from the remote
store as well. However, there's still left over to read from the
previous command and thus Nix fails to properly allocate the expected
string.

[1] See rev a2b69660a9b326b95d48bd222993c5225bbd5b5f

Fixes #898
2021-04-15 15:16:52 +02:00
Graham Christensen b9bcedbfdb
Merge pull request #596 from kquick/local_inp_url
Update prompt for Local path input to indicate a URL is also valid.
2021-04-14 20:01:58 +00:00
Graham Christensen ffc4be6743
Merge pull request #910 from grahamc/ui-fixups
UI fixups
2021-04-12 17:42:01 +00:00
Graham Christensen afd064d19d
Merge pull request #867 from ck3d/fix-proxy-login
Fix login if Hydra runs behind HTTP proxy with sub-path location
2021-04-12 17:36:55 +00:00
Graham Christensen 82953389e4 Make new jobsets enabled by default in the UI
If the jobset is brand new, none of the options matched. This
makes all new jobsets default to Enabled.
2021-04-12 12:01:24 -04:00
Graham Christensen 0fe4cba5b7 edit-jobset: make the "add a new input" button span the whole way
Previously it would only cross 4 of the 5 columns.
2021-04-12 12:01:24 -04:00
Graham Christensen e674fb6139 edit user: make the role changer checkboxes
Also, it makes the role changer a bit of a dangerzone visually for
admins.
2021-04-09 14:07:10 +00:00
Graham Christensen 258b39f1e5
Merge pull request #677 from twhitehead/javascripts-update
Javascript libraries update
2021-04-09 14:05:47 +00:00
Eelco Dolstra 20c1efeb5b
Merge pull request #904 from Ma27/gitea-integration
Add `GiteaStatus`-Plugin
2021-04-08 17:57:38 +02:00
Tyson Whitehead e51a6a4bca
Cards are now required to decorate pre blocks 2021-04-08 11:32:30 -04:00
Tyson Whitehead 7d3bd08c9b
Fix tab link spacing issue by dropping old sticky navbar css hack 2021-04-08 11:32:29 -04:00
Samuel Dionne-Riel def05105bb Add top spacing to tab contents 2021-04-08 11:32:26 -04:00
Samuel Dionne-Riel d85b846357 Fix spacing with search inputs
And more importantly, crucially, fix the "appended" button so the round
corners are on the right (heh) side.
2021-04-08 11:32:08 -04:00
Samuel Dionne-Riel 20e3c51ba9 Style unstyled alerts 2021-04-08 11:32:02 -04:00
Graham Christensen db2c9fe96a layout.tt: make the header constrained to the center 2021-04-08 11:31:56 -04:00
Graham Christensen 9c903e9991 renderEvals: fixup colspan on Jobs and More 2021-04-08 11:31:47 -04:00
Graham Christensen c8dd5b57c4 src/root/Makefile: fixup extraction of the fontawesome and bootstrap archives 2021-04-08 11:31:39 -04:00
Tyson Whitehead abe082c1d6
Couple of upstream/bootstrap items missed in rebase 2021-04-08 11:31:34 -04:00
Tyson Whitehead b5e18ce62c
Fix the empty popovers 2021-04-08 11:31:34 -04:00
Tyson Whitehead 2902684c61
Update jobset comparison button 2021-04-08 11:31:34 -04:00
Tyson Whitehead 3f77bd240f
Bring deep-embedding (tab anchors) javascript up-to-date 2021-04-08 11:31:34 -04:00
Tyson Whitehead b6f5b6eb38
Labels are now badges 2021-04-08 11:31:34 -04:00
Tyson Whitehead 70df5e6312
Update button styles as btn-mini is no more 2021-04-08 11:31:34 -04:00
Graham Christensen 1e34cd3d3d fixups to the rebase 2021-04-08 11:30:49 -04:00
regnat 26ffd4a93e Fix build with latest master 2021-04-08 17:11:15 +02:00
Graham Christensen cc9c91fe12 jobsets: put hidden and enabled jobsets at the end
Allows for generally correct zebra striping
2021-03-31 14:33:20 +00:00
Graham Christensen a46f655c56 root project listing: show hidden projects at the end
Makes the zebra striping correct.
2021-03-31 14:33:20 +00:00
Maximilian Bosch f9f5ab2fb1
Make gitea public URL configurable
Otherwise, it will be obtained from the jobset input that contains the
URL to the git repo to build.
2021-03-30 23:01:36 +02:00
Maximilian Bosch eecea56131
Implement VM-test for gitea plugin 2021-03-30 22:35:39 +02:00
Maximilian Bosch 56997d8e8b
Fix error codes for GiteaStatus plugin
* `failure` if a build error occurred, on e.g. an aborted build send
  `error`.
2021-03-30 14:13:46 +02:00
Maximilian Bosch fef142f13a
Implement simple status notifications for Git repos hosted on gitea 2021-03-30 14:10:21 +02:00
Graham Christensen 9bb04ed97a
Merge pull request #893 from grahamc/fake-channel-nested
Test the fake derivations channel, asserting nested packages are properly represented.
2021-03-24 17:56:55 -04:00
Tyson Whitehead eb8a0f279a
Add fontawesome as required by newer bootstrap 2021-03-24 17:10:28 -04:00
Tyson Whitehead cd0ff484f7
Add popper.js as required by newer bootstrap 2021-03-24 17:10:28 -04:00
Tyson Whitehead 4d881b59ad
Update bootbox to latest 5.2.0 2021-03-24 17:10:27 -04:00
Tyson Whitehead 230a0387d2
Update boostrap to latest 4.3.1
Co-authored-by: Graham Christensen <graham@grahamc.com>
... but just fixing up merge conflicts from the introduction of flakes
and the removal of the Jobs table.
2021-03-24 17:10:27 -04:00
Tyson Whitehead 627af61abe
Update jquery to latest 3.4.1 (considered by some as more secure) 2021-03-24 17:10:27 -04:00
Graham Christensen 425c7ff17f
hydra-send-stats: add a --once option for testing 2021-03-20 09:16:08 -04:00
Jörg Thalheim 6bb180a0f2
hydra-send-stats: fix imports 2021-03-20 09:16:04 -04:00
Graham Christensen 6b7ca554f9
Update src/lib/Hydra/Helper/Escape.pm: fewer ()s
Co-authored-by: Stig <stig@stig.io>
2021-03-18 16:27:21 -04:00
Graham Christensen 019aef3d41
Test the fake derivations channel, asserting nested packages are properly represented.
This is a breaking change. Previously, packages named `packageset.foo`
would be exposed in the fake derivation channel as `packageset-foo`.

Presumably this was done to avoid needing to track attribute sets, and
to avoid the complexity. I think this now correctly handles the
complexity and properly mirrors the input expressions layout.
2021-03-18 11:33:37 -04:00
Graham Christensen 88e0198a8e
Create a helper for dealing with nested attribute sets 2021-03-18 11:33:36 -04:00
Graham Christensen d62a2c1657
NixExprs: extract the escape function and test it 2021-03-18 11:24:17 -04:00
Eelco Dolstra aeb3d2f44c
Merge pull request #892 from grahamc/hydra-queue-runner-build-one
hydra-queue-runner: --build-one: correctly handle a cached build
2021-03-16 21:28:32 +01:00
Graham Christensen 87d46ad5d6
hydra-queue-runner: --build-one: correctly handle a cached build
Previously, the build ID would never flow through channels which
exited.

This patch tracks the buildOne state as part of State and exits avoids
waiting forever for new work.

The code around buildOnly is a bit rough, making this a bit weird to
implement but since it is only used for testing the value of improving
it on its own is a bit questionable.
2021-03-16 16:13:38 -04:00
Janne Heß 3c86083d21
Fixup #717 "Add the project name to declarative inputs"
```
Mar 10 16:22:35 hydra-b hydra-evaluator[41419]: DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR:  null value in column "type" violates not-null constraint
Mar 10 16:22:35 hydra-b hydra-evaluator[41419]: DETAIL:  Failing row contains (62358, projectName, 0, null, null, null, hackworthltd, null, , null). [for Statement "INSERT INTO jobsetevalinputs ( altnr, dependency, eval, name, path, revision, sha256hash, type, uri, value) VALUES ( ?, ?, ?, ?, ?, ?, ?, ?, ?, ? )" with ParamValues: 1='0', 2=undef, 3='62358', 4='projectName', 5='', 6=undef, 7=undef, 8=undef, 9=undef, 10='hackworthltd'] at /nix/store/cmqblv437mp57yz5lwvkzcqca4ldf3r5-hydra-0.1.20210308.ebf1cd2/bin/.hydra-eval-jobset-wrapped line 793
Mar 10 16:22:35 hydra-b hydra-evaluator[25828]: evaluation of jobset ‘hackworthltd:.jobsets (jobset#1)’ failed with exit code 1
```

Use the abstraction for creating inputs for simulating the project
name input.

Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-03-16 09:52:36 -04:00
Shea Levy 930f05c38e
Bump Nix version 2021-03-10 12:53:03 -05:00
Graham Christensen b9fb66401b
Merge pull request #880 from grahamc/runcommand-finished-bool
RunCommand: emit the `finished` field as a boolean
2021-03-09 09:58:43 -05:00
Graham Christensen 2179b4b4b0
RunCommand: emit the finished field as a boolean 2021-03-08 12:11:20 -05:00
Janne Heß 9e018d5443
Add the project name to declarative inputs
This allows for more generic declarative configurations which can be
shared between projects.
2021-03-08 17:36:52 +01:00
Matej Cotman a551fba346
statsd: add a chance to set hostname and port in hydra.conf
Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-03-08 10:03:16 -05:00
Josh McSavaney e0d3a1c1a5
Make nix-build args copy-pastable via set -x
A reproduce script includes a logline that may resemble:

> using these flags: --arg nixpkgs { outPath = /tmp/build-137689173/nixpkgs/source; rev = "fdc872fa200a32456f12cc849d33b1fdbd6a933c"; shortRev = "fdc872f"; revCount = 273100; } -I nixpkgs=/tmp/build-137689173/nixpkgs/source --arg officialRelease false --option extra-binary-caches https://hydra.nixos.org/ --option system x86_64-linux /tmp/build-137689173/nixpkgs/source/pkgs/top-level/release.nix -A 

These are passed along to nix-build and that's fine and dandy, but you can't just copy-paste this as is, as the `{}` introduces a syntax error and the value accompanying `-A` is `''`.

A very naive approach is to just `printf "%q"` the individual args, which makes them safe to copy-paste. Unfortunately, this looks awful due to the liberal usage of slashes:

```
$ printf "%q" '{ outPath = /tmp/build-137689173/nixpkgs/source; rev = "fdc872fa200a32456f12cc849d33b1fdbd6a933c"; shortRev = "fdc872f"; revCount = 273100; }'
\{\ outPath\ =\ /tmp/build-137689173/nixpkgs/source\;\ rev\ =\ \"fdc872fa200a32456f12cc849d33b1fdbd6a933c\"\;\ shortRev\ =\ \"fdc872f\"\;\ revCount\ =\ 273100\;\ \}
```

Alternatively, if we just use `set -x` before we execute nix-build, we'll get the whole invocation in a friendly, copy-pastable format that nicely displays `{}`-enclosed content and preserves the empty arg following `-A`:

```
running nix-build...
using this invocation: 
+ nix-build --arg nixpkgs '{ outPath = /tmp/build-138165173/nixpkgs/source; rev = "e0e4484f2c028d2269f5ebad0660a51bbe46caa4"; shortRev = "e0e4484"; revCount = 274008; }' -I nixpkgs=/tmp/build-138165173/nixpkgs/source --arg officialRelease false --option extra-binary-caches https://hydra.nixos.org/ --option system x86_64-linux /tmp/build-138165173/nixpkgs/source/pkgs/top-level/release.nix -A ''
```
2021-03-06 23:25:26 -05:00
Graham Christensen 68ac64dbd9
Merge pull request #832 from wizeman/fix-hash-mismatch
Fix persistent hash mismatch errors when importing
2021-03-02 16:04:23 -05:00
Graham Christensen a756614fa1
RunCommand: pass homepage, description, license, system, and nixname 2021-02-24 16:13:09 -05:00
Graham Christensen 3fda37f65a
RunCommand: Test 2021-02-24 13:43:25 -05:00
Graham Christensen e4cda87b5a
db.hh: use hasPrefix for prefix comparisons
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-02-24 07:00:26 -05:00
Graham Christensen fe1f2f0806
Create an ephemeral PostgreSQL database per test 2021-02-23 21:12:06 -05:00
regnat f602ed0d86 Remove the sendDerivation logic from the builder
The queue runner used to special-case `localhost` as a remote builder:
Rather than using the normal remote-build (using the
`cmdBuildDerivation` command), it was using the (generally less
efficient, except when running against localhost) `cmdBuildPaths`
command because the latter didn't require a privileged Nix user (so made
testing easier − allowing to run hydra in a container in particular).

However:
1. this means that the build loop can follow two discint code paths depending
   on the setup, the irony being that the most commonly used one in production
   (the “non-localhost” case) isn't the one used in the testsuite (because all
   the tests run against a local store);
2. It turns out that the “localhost” version is buggy in relatively obvious
   ways − in particular a failure in a fixed-output derivation or a hash
   mismatch isn't reported properly;
3. If the “run in a container” use-case is indeed that important, it can be
   (partially) restored using a chroot store (which wouldn't behave excactly
   the same way of course, but would be more than good-enough for testing)
2021-02-23 09:50:15 +01:00
Eelco Dolstra 107d60027f hydra-eval-jobs: Fix unexpected EOF when a top-level attr fails 2021-02-22 16:29:07 +01:00