Commit graph

2791 commits

Author SHA1 Message Date
Graham Christensen b0055a23df TaskDispatcher: Support re-queueing tasks, and dropping tasks after 100 failures. 2021-09-02 10:06:26 -04:00
Graham Christensen d0b0fc21b3 ResultSet::TaskRetries: teach about saving tasks 2021-09-02 10:06:26 -04:00
Graham Christensen 147fa4d029 Result::TaskRetries: Teach about requeue 2021-09-02 10:06:26 -04:00
Graham Christensen 42c2d2f387 Hydra::Math: add an exponential_backoff function 2021-09-02 10:06:26 -04:00
Graham Christensen 29738364fb ResultSet::TaskRetries: add get_seconds_to_next_retry
Get the number of seconds before the next retriable task is ready.

This number is specifically intended to be used as a timeout, where
`undef` means never time out.
2021-09-02 10:06:26 -04:00
Graham Christensen c4134c8e84 TaskRetries: init table 2021-09-02 10:06:26 -04:00
Graham Christensen 1f2adf61aa hydra-notify: extract runPluginsForEvent to a TaskDispatcher 2021-09-02 10:06:26 -04:00
Graham Christensen fa57fb8f25 hydra.sql: explain update-dbix.pl map 2021-08-26 22:10:19 -04:00
Graham Christensen 397d13a300 DBIx::Class: migrate to use_namespaces
This gives us a place to put helper functions that act on entire
tables, not just individual records.

This should be a backwards compatible change, except in places we're
manually using result class names.
2021-08-26 12:37:19 -04:00
Your Name 24467a7bde Nix::getHydraNotifyPrometheusConfig: print errors if the configuration provided is invalid. 2021-08-24 11:35:38 -04:00
Your Name 4ebdcc290e fixup! hydra-notify: pre-declare metrics 2021-08-24 10:57:23 -04:00
Your Name 45e8fa5319 hydra-notify: support sending diagnostic dumps to STDERR on request 2021-08-24 10:56:13 -04:00
Your Name de2282bcf4 hydra-notify: print out log lines indicating it is or is not launching the exporter 2021-08-24 10:56:13 -04:00
Your Name 5c1228e141 hydra-notify: pre-declare metrics 2021-08-24 10:56:13 -04:00
Your Name 6d7ee27d25 hydra-notify: make the prometheus endpoint configurable, default-off 2021-08-24 10:56:13 -04:00
Your Name 5d0ad5f649 hydra-notify: initial scratch take of prometheus events 2021-08-24 10:56:12 -04:00
Graham Christensen e9ba0b56f6 BuildFinished : $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen 5e10648a98 CoverityScan: $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen c929f5cac0 SlackNotification: $build -> $topbuild, $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen b4f9807290 GithubStatus: $build -> $topbuild, $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen 5c9d8794b1 GiteaStatus: $build -> $topbuild, $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen 3920bf1df3 BitBucketStatus: $build -> $topbuild, $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen bcf37ab705 GitlabStatus: $build -> $topbuild, $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen b4a19fd3c8 HIpChatNotification: $build -> $topbuild, $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen 0d4ce15c62 EmailNotification: $build -> $topbuild, $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen 0f88014631 CircleCiNotification: $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen b6714cdce3 JobsetEval: $a, $b to $left, $right (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen d0fb3cbf8f Job: $b -> $build (perlcritic) 2021-08-20 21:52:32 -04:00
Graham Christensen 1cda08351f CatalystUtils: rename $b to $builds (perlcritic) 2021-08-20 21:52:32 -04:00
Your Name 4632132d02 fixup perlcritic: indirect new
> Subroutine new called using indirect syntax at line 14, column 14.  See page 349 of PBP.  (Severity: 5)
2021-08-20 11:39:19 -04:00
Your Name 6e65c3b320 hydra-notify: fixup printing of build IDs
Used to print:

    sending notifications for build Hydra::Model::DB::Builds=HASH(0x124cf960)->id...

Now it prints:

    sending notifications for build 123...
2021-08-16 16:09:05 -04:00
Your Name 2c50227082 hydra-notify: properly call new_event 2021-08-16 15:52:25 -04:00
Your Name e572a5e576 hydra-notify: use Hydra::Event 2021-08-16 15:52:14 -04:00
Graham Christensen fda3ed1d4e
Merge pull request #999 from DeterminateSystems/fixup/pr-998
lib/makefile: include Event/*.pm
2021-08-16 14:13:18 -04:00
Your Name cfd3034af8 lib/makefile: include Event/*.pm 2021-08-16 14:04:48 -04:00
Graham Christensen fa6d7abc13 hydra-notify: move BuildFinished processing to an Event 2021-08-13 16:51:29 -04:00
Graham Christensen 4a1389e36e hydra-notify: move StepFinished processing to an Event 2021-08-13 16:51:29 -04:00
Graham Christensen 4fdb20d3bd hydra-notify: move BuildStarted processing to an Event 2021-08-13 16:51:29 -04:00
Graham Christensen 10e85e3422 hydra-notify: Create a helper for running each plugin on an event 2021-08-13 16:51:29 -04:00
Graham Christensen d02c6794f4 Event: teach how to execute a plugin 2021-08-13 16:51:29 -04:00
Graham Christensen 4e86e55008 Event.pm: add a new_event helper to parse and construct an Event
An Event will be part of many Tasks
2021-08-13 16:51:29 -04:00
Graham Christensen 64a3e75c10 Event: init structure and parse existing messages 2021-08-13 16:51:29 -04:00
Graham Christensen a14c8ad5f8
Merge pull request #995 from DeterminateSystems/declarative-jobsets-plugin
Declarative jobsets: move event handling to a plugin
2021-08-12 15:56:13 -04:00
Graham Christensen 0f958f3425
Merge pull request #997 from DeterminateSystems/abstract-listener
Abstract over postgres' LISTEN/NOTIFY
2021-08-12 14:00:34 -04:00
Graham Christensen 5027003285 Abstract over postgres' LISTEN/NOTIFY
This lets us test the event loop if we wanted, and lets us
test the listening behavior in isolation.
2021-08-12 13:54:05 -04:00
Graham Christensen 593af41808 Declarative jobsets: move event handling to a plugin
Declarative jobsets were sort of tucked in to the event hanlder
itself. It turned out that it could have been implemented as a
plugin without much trouble.
2021-08-12 12:48:18 -04:00
Graham Christensen 9c5f317453 hydra-notify: move buildFinished query in to the function impl
This is more consistent with the other event handlers, of dealing
with IDs and not objects.
2021-08-12 12:30:35 -04:00
Eelco Dolstra 5edb58b314 Fix build 2021-08-10 13:47:16 +02:00
Graham Christensen c7c322545d
Merge pull request #992 from DeterminateSystems/sql/fixup-comment
hydra.sql: Update comment on regeneration
2021-08-06 14:54:12 -04:00
Graham Christensen 4169f22231 update-dbix.pl: correct indentation 2021-08-06 14:40:57 -04:00
Graham Christensen 5bd8dc171b hydra.sql: Update comment on regeneration
We no longer need to generate the hydra-postgres.sql document,
that is a relic from when sqlite was also supported.
2021-08-06 14:40:34 -04:00
Shay Bergmann 503b0e0b6f
Remove unnecessary comment 2021-07-26 17:16:40 +00:00
Shay Bergmann 173ecb5d92
Apply requested changes to Config.pm 2021-07-26 17:16:40 +00:00
Shay Bergmann 4f95774ca3
Factor out common options 2021-07-26 03:43:09 +00:00
Shay Bergmann e98bd0ec8d
Comment around dupe to avoid drift 2021-07-22 23:27:21 -04:00
Shay Bergmann 54c8028ec0
Make hydra-server honor config file includes 2021-07-22 17:25:30 -04:00
Nathan van Doorn 5c47343b40 Fix deprecation warnings with libpqxx usage 2021-07-05 19:45:25 +01:00
Graham Christensen e02f34722f
Merge pull request #887 from helsinki-systems/feat/eval-overview-duration
Show evaluation time in the evaluations overview
2021-06-29 10:59:02 -04:00
Janne Heß 94b88b34ae
Show evaluation time in the evaluations overview
Parts of this code have been authored by @grahamc
2021-06-29 16:53:10 +02:00
Jonathan Ringer 24959a3ca6 Allow for evaluator_pure_eval to be configured 2021-06-28 16:24:40 -07:00
Damien Diederen df7dab1291 GitInput: Include deepClone option in the cache key
Without this commit, two jobsets using the same repository as input,
but different `deepClone` options, end up incorrectly sharing the same
"checkout" for a given (`uri`, `branch`, `revision`) tuple.  The
presence or absence of `.git` is determined by the jobset execution
order.

This patch adds the missing `isDeepClone` boolean to the cache key.

The database upgrade script empties the `CachedGitInputs` table, as we
don't know if existing checkouts are deep clones.  Unfortunately, this
generally forces rebuilds even for correct `deepClone` checkouts, as
the binary contents of `.git` are not deterministic.

Fixes #510
2021-06-19 17:37:40 +02:00
Graham Christensen 09ad52ab60 Document getEvals a bit
Also drop the $self parameter.

Co-authored-by: gustavderdrache
2021-06-16 13:15:01 -04:00
Graham Christensen 5d95abf540 getBuilds: clarify the names of evals vs. the query builder 2021-06-16 11:53:04 -04:00
Graham Christensen bf5c76feb6 getEvals: order by the eval table's ID
I broke this when I added `me.` in f1e75c8bff

I added me. to disambiguate `id`, but:

* eval.id works on the per-build page
* me.id works on the other pages
* Just id works everywhere if I drop:

    , prefetch => { evaluationerror => [ ] },

  but this causes a query per row to collect the evaluationerror
  records later, this becomes significantly slow on non-trivial
  datasets.

Using evals->current_source_alias will use the correct alias
whether it is me or eval or something else.
2021-06-16 11:24:30 -04:00
Graham Christensen 71453dd341 Expose Prometheus metrics at /metrics
Exposes metrics:

* http_request_duration_seconds_bucket
* http_request_size_bytes_bucket
* http_response_size_bytes_bucket
* http_requests_total

with labels of action and controller to help identify popular
endpoints and their performance characteristics.
2021-06-10 20:02:42 -04:00
Graham Christensen 382ba590ad Drop unused query from GET Build. 2021-06-01 11:47:05 -04:00
Graham Christensen a9e4ede006 SQL: create better indexes for builds based on the jobset id
These are primarily used by the jobsetOverview renders.
2021-06-01 11:23:22 -04:00
Graham Christensen 719d0a6134 jobset overview: join jobsets to builds by jobset id 2021-06-01 11:19:33 -04:00
Graham Christensen 508d99d611 Join to builds via jobset_id when easy 2021-06-01 11:16:47 -04:00
Cole Helbling 45ced33ab6
Projects: don't respond with unnecessary declarative field
If the project isn't declarative, who cares about it in the response? After
setting the `declfile` to an empty string, everything related to declarative-
ness is wiped out, anyways.
2021-05-24 15:26:34 -07:00
Graham Christensen 3dc745de68
Merge pull request #964 from DeterminateSystems/fix-cascade
Jobsets: remove defunct Jobs relationship
2021-05-09 21:17:12 -04:00
Cole Helbling 7ba58ca3ea
Jobsets: remove defunct Jobs relationship
It appears the Jobs table was removed in
8adb433e3b, but the Jobsets schema was never
updated to reflect this. This relationship was added in
efa1f1d4fb, roughly 3 months prior.

Previously, one would see a message similar to the following logged when
deleting a jobset:

    17:38:23 hydra-server.1       | DBIx::Class::Relationship::CascadeActions::delete(): Skipping cascade delete on relationship 'jobs' - related resultsource 'Hydra::Schema::Jobs' is not registered with this schema at /home/vin/workspace/vcs/hydra/src/script/../lib/Hydra/Controller/Jobset.pm line 106
2021-05-09 17:52:48 -07:00
Graham Christensen 67b6f0d7ed
Merge pull request #962 from DeterminateSystems/del-.jobsets
Project: delete the `.jobsets` jobset if project is no longer declarative
2021-05-06 11:57:34 -04:00
Cole Helbling 6107040bf5
Project: clear decltype and declvalue when project is no longer declarative 2021-05-06 08:47:18 -07:00
Cole Helbling bd9c52dbd0
Project: delete the .jobsets jobset if project is no longer declarative
"No longer declarative" as defined by the "Edit project" page is an empty spec
file.
2021-05-05 14:03:51 -07:00
Graham Christensen 6c1151b14a lazy tabs: trigger the load event
Something in the upgrade of Bootstrap and JQuery broke lazy tab loading.
I don't understand what is providing the tab behavior, how it should
work, or what the correct fix is.

I can tell you that this patch fixes the issue: when loading a tab
with a URL fragment deep-linking to a lazily loaded tab... it now
loads.

Close #959
2021-05-05 14:42:12 -04:00
Eelco Dolstra 886e6f85e4
Merge pull request #924 from hackworthltd/fix-flake-github-status
Fix GitHub status update for private flakes.
2021-05-03 16:24:51 +02:00
Cole Helbling f3947acfc4
Projects: use JSON in Schema 2021-05-02 18:25:37 -07:00
Cole Helbling ad13d13436
Project: add declfile, decltype, declvalue to API
This makes it possible to create declarative projects via a PUT request, and
also exposes the currently-configured values to GET requests.
2021-05-02 17:45:14 -07:00
Cole Helbling 7c9ed6b919
Project: fix DELETE route
This appears to have been broken in ac3e8a4a59,
which removed the `jobsetevals` column from the Projects schema, but didn't
update the Controller accordingly.

Fixes the test added in the previous commit.
2021-04-30 10:43:21 -07:00
Cole Helbling 03d5a858a7
Jobset: list known inputs in invalid input type response 2021-04-29 18:19:44 -07:00
Graham Christensen 8492db033e Jobsets,JobsetInputs: use JSON 2021-04-29 08:32:47 -04:00
Cole Helbling f1dd5d202e Jobsets: update schema to align with the API
To further align with the API, we return custom JSON in order to display a
`visible` field rather than `hidden` -- a `PUT` request expects `visible`, while
a `GET` request returns `hidden`.

This also allows us to rename the `jobsetinputs` field to `inputs` for the same
reason: `PUT` expects `inputs`, while `GET` returns `jobsetinputs`.
2021-04-29 07:50:23 -04:00
Cole Helbling d23f431889 JobsetInputs: update schema to align with the API
`PUT /jobsets/{project-id}/{jobset-id}` expects a JSON object `inputs` which
maps a name to a name, a type, a value, and a boolean that enables emailing
responsible parties. However, `GET /jobsets/{project-id}/{jobset-id}` responds
with an object that doesn't contain a value, but does contain a jobsetinputalts
(which is old and should be unused).

This commit aligns the two by removing the old and unused `jobsetinputalts` from
the response and replaces it with `value`.
2021-04-29 07:50:23 -04:00
Cole Helbling fff0db10e3 ToJSON: allow custom as_json function
This allows us to modify what the API responds with, which in turn lets us unify
the OpenAPI specification and the actual API's responses.
2021-04-29 07:50:23 -04:00
Graham Christensen fdb6e7dd97 fixup: return the user entity on login 2021-04-28 18:30:35 -04:00
Graham Christensen 823da22e4f
Merge pull request #937 from DeterminateSystems/flesh-out-api
hydra-api: flesh out Jobset, JobsetInput schemas; implement DELETE /jobset/{project-id}/{jobset-id}
2021-04-28 13:43:04 -04:00
Graham Christensen 725c9c2f81
login: redirect to the current-user page 2021-04-28 08:32:10 -07:00
Graham Christensen d589db2ed9
login: missing parameters are 400s 2021-04-28 08:31:59 -07:00
regnat abff212d06 Use system-features from the Nix conf in the default machine file
Fix #936
2021-04-28 11:43:04 +02:00
Cole Helbling 72fec31dbb
hydra-api: flesh out JobsetInput schema 2021-04-27 16:16:42 -07:00
Cole Helbling 2600810551
hydra-api: flesh out Jobset schema
* made all columns available via the API (except for forceeval)
* renamed flakeref to flake to unify the API with the database schema
* renamed inputs to jobsetinputs to unify the API with the database schema
2021-04-27 16:16:42 -07:00
Cole Helbling 50fab154a4
ToJSON: serialize string_columns to JSON
If the column is undefined, then it should be an empty string according to your
API spec.
2021-04-26 16:39:13 -07:00
Graham Christensen f2b9649bf2
Projects: serialize enabled and hidden as boolean 2021-04-26 16:03:32 -07:00
Graham Christensen 4aea02e1e1
ToJSON: serialize boolean_columns to JSON boolean 2021-04-26 16:03:32 -07:00
Cole Helbling c757867b9e
Add homepage to Projects schema 2021-04-26 15:46:30 -07:00
Graham Christensen 453b8479be
Merge pull request #927 from cole-h/nonexistent-user-400
Return HTTP 400 when creating Project with nonexistent user
2021-04-26 14:40:15 -04:00
Cole Helbling 47e19ba22c
Return HTTP 400 when creating Project with nonexistent user 2021-04-26 11:32:39 -07:00
Drew Hess 523d6df5b8
Fix GitHub status update for private flakes.
Also, if the parse fails, don't try to update the GitHub status, as
this will eventually cause rate-limiting.
2021-04-26 01:38:24 +01:00
Maximilian Bosch 21ed005c84
Make it possible to enable email notifications when creating a jobset
The checkbox is only enabled if `email_notification = 1` is set in
`hydra.conf`. However, when creating jobset (in contrast to the edit
form), the checkbox is always disabled because the `emailNotification`
parameter in Catalyst's stash was missing.
2021-04-24 19:48:43 +02:00
Graham Christensen 05636de7d2 hydra-init: upgrade passwords to Argon2 on startup 2021-04-16 12:32:13 -04:00
Graham Christensen 79b0ddc27d hydra-create-user: re-hash sha1 as Argon2 2021-04-16 12:32:13 -04:00
Graham Christensen d10d8964f2 Users: add a validation step which lets the user's password be a Argon2 hashed sha1 hash.
OWASP suggests expiring all passwords and requiring users to update their password.
However, we don't have a way to do this. They suggest this mechanism
as a good alternative:
https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html#upgrading-legacy-hashes
2021-04-16 12:32:13 -04:00
Eelco Dolstra fa924ea697
Merge pull request #915 from grahamc/hydra-auth
Hydra auth: support Argon2, transparently upgrade hashes
2021-04-15 17:40:32 +02:00
Graham Christensen 9225be0897 Drop remaining sha1_hex references
Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:35:18 -04:00
Graham Christensen 1d956be61e hydra-create-user: support Argon2
Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:35:16 -04:00
Graham Christensen beb5be4302 Users: password changes via the web UI now use Argon2
Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:35:13 -04:00
Graham Christensen 1da70030b7 Users: transparently upgrade passwords to Argon2
Passwords that are sha1 will be transparently upgraded to argon2,
and future comparisons will use Argon2

Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:35:11 -04:00
Graham Christensen 29620df85e Passwords: check in constant time
The default password comparison logic does not use
constant time validation. Switching to constant time
offers a meager improvement by removing a timing
oracle.

A prepatory step in moving to Argon2id password storage, since we'll need this change anyway after
for validating existing passwords.

Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:34:56 -04:00
Graham Christensen d4d8f1ba1b Plugin::Authentication config: modernize
Some time in the last decade the plugin switched to preferring
a flatter namespace for realm config.

Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:34:47 -04:00
Eelco Dolstra 0bee194ce9
Merge pull request #914 from Ma27/fix-remote-builds
Fix `std::bad_alloc` errors for remote builds
2021-04-15 17:05:54 +02:00
Maximilian Bosch 2808227eb7
Fix std::bad_alloc errors for remote builds
In Nix the protocol was slightly altered[1] to also contain more
information about realisations. This however wasn't read from the pipe
that was used to read from the store.

After the `cmdBuildDerivation` command which caused this issue, Hydra
will issue a `cmdQueryPathInfos` that tries to read from the remote
store as well. However, there's still left over to read from the
previous command and thus Nix fails to properly allocate the expected
string.

[1] See rev a2b69660a9b326b95d48bd222993c5225bbd5b5f

Fixes #898
2021-04-15 15:16:52 +02:00
Graham Christensen b9bcedbfdb
Merge pull request #596 from kquick/local_inp_url
Update prompt for Local path input to indicate a URL is also valid.
2021-04-14 20:01:58 +00:00
Graham Christensen ffc4be6743
Merge pull request #910 from grahamc/ui-fixups
UI fixups
2021-04-12 17:42:01 +00:00
Graham Christensen afd064d19d
Merge pull request #867 from ck3d/fix-proxy-login
Fix login if Hydra runs behind HTTP proxy with sub-path location
2021-04-12 17:36:55 +00:00
Graham Christensen 82953389e4 Make new jobsets enabled by default in the UI
If the jobset is brand new, none of the options matched. This
makes all new jobsets default to Enabled.
2021-04-12 12:01:24 -04:00
Graham Christensen 0fe4cba5b7 edit-jobset: make the "add a new input" button span the whole way
Previously it would only cross 4 of the 5 columns.
2021-04-12 12:01:24 -04:00
Graham Christensen e674fb6139 edit user: make the role changer checkboxes
Also, it makes the role changer a bit of a dangerzone visually for
admins.
2021-04-09 14:07:10 +00:00
Graham Christensen 258b39f1e5
Merge pull request #677 from twhitehead/javascripts-update
Javascript libraries update
2021-04-09 14:05:47 +00:00
Eelco Dolstra 20c1efeb5b
Merge pull request #904 from Ma27/gitea-integration
Add `GiteaStatus`-Plugin
2021-04-08 17:57:38 +02:00
Tyson Whitehead e51a6a4bca
Cards are now required to decorate pre blocks 2021-04-08 11:32:30 -04:00
Tyson Whitehead 7d3bd08c9b
Fix tab link spacing issue by dropping old sticky navbar css hack 2021-04-08 11:32:29 -04:00
Samuel Dionne-Riel def05105bb Add top spacing to tab contents 2021-04-08 11:32:26 -04:00
Samuel Dionne-Riel d85b846357 Fix spacing with search inputs
And more importantly, crucially, fix the "appended" button so the round
corners are on the right (heh) side.
2021-04-08 11:32:08 -04:00
Samuel Dionne-Riel 20e3c51ba9 Style unstyled alerts 2021-04-08 11:32:02 -04:00
Graham Christensen db2c9fe96a layout.tt: make the header constrained to the center 2021-04-08 11:31:56 -04:00
Graham Christensen 9c903e9991 renderEvals: fixup colspan on Jobs and More 2021-04-08 11:31:47 -04:00
Graham Christensen c8dd5b57c4 src/root/Makefile: fixup extraction of the fontawesome and bootstrap archives 2021-04-08 11:31:39 -04:00
Tyson Whitehead abe082c1d6
Couple of upstream/bootstrap items missed in rebase 2021-04-08 11:31:34 -04:00
Tyson Whitehead b5e18ce62c
Fix the empty popovers 2021-04-08 11:31:34 -04:00
Tyson Whitehead 2902684c61
Update jobset comparison button 2021-04-08 11:31:34 -04:00
Tyson Whitehead 3f77bd240f
Bring deep-embedding (tab anchors) javascript up-to-date 2021-04-08 11:31:34 -04:00
Tyson Whitehead b6f5b6eb38
Labels are now badges 2021-04-08 11:31:34 -04:00
Tyson Whitehead 70df5e6312
Update button styles as btn-mini is no more 2021-04-08 11:31:34 -04:00
Graham Christensen 1e34cd3d3d fixups to the rebase 2021-04-08 11:30:49 -04:00
regnat 26ffd4a93e Fix build with latest master 2021-04-08 17:11:15 +02:00
Graham Christensen cc9c91fe12 jobsets: put hidden and enabled jobsets at the end
Allows for generally correct zebra striping
2021-03-31 14:33:20 +00:00
Graham Christensen a46f655c56 root project listing: show hidden projects at the end
Makes the zebra striping correct.
2021-03-31 14:33:20 +00:00
Maximilian Bosch f9f5ab2fb1
Make gitea public URL configurable
Otherwise, it will be obtained from the jobset input that contains the
URL to the git repo to build.
2021-03-30 23:01:36 +02:00
Maximilian Bosch eecea56131
Implement VM-test for gitea plugin 2021-03-30 22:35:39 +02:00
Maximilian Bosch 56997d8e8b
Fix error codes for GiteaStatus plugin
* `failure` if a build error occurred, on e.g. an aborted build send
  `error`.
2021-03-30 14:13:46 +02:00
Maximilian Bosch fef142f13a
Implement simple status notifications for Git repos hosted on gitea 2021-03-30 14:10:21 +02:00
Graham Christensen 9bb04ed97a
Merge pull request #893 from grahamc/fake-channel-nested
Test the fake derivations channel, asserting nested packages are properly represented.
2021-03-24 17:56:55 -04:00
Tyson Whitehead eb8a0f279a
Add fontawesome as required by newer bootstrap 2021-03-24 17:10:28 -04:00
Tyson Whitehead cd0ff484f7
Add popper.js as required by newer bootstrap 2021-03-24 17:10:28 -04:00
Tyson Whitehead 4d881b59ad
Update bootbox to latest 5.2.0 2021-03-24 17:10:27 -04:00
Tyson Whitehead 230a0387d2
Update boostrap to latest 4.3.1
Co-authored-by: Graham Christensen <graham@grahamc.com>
... but just fixing up merge conflicts from the introduction of flakes
and the removal of the Jobs table.
2021-03-24 17:10:27 -04:00
Tyson Whitehead 627af61abe
Update jquery to latest 3.4.1 (considered by some as more secure) 2021-03-24 17:10:27 -04:00
Graham Christensen 425c7ff17f
hydra-send-stats: add a --once option for testing 2021-03-20 09:16:08 -04:00
Jörg Thalheim 6bb180a0f2
hydra-send-stats: fix imports 2021-03-20 09:16:04 -04:00
Graham Christensen 6b7ca554f9
Update src/lib/Hydra/Helper/Escape.pm: fewer ()s
Co-authored-by: Stig <stig@stig.io>
2021-03-18 16:27:21 -04:00
Graham Christensen 019aef3d41
Test the fake derivations channel, asserting nested packages are properly represented.
This is a breaking change. Previously, packages named `packageset.foo`
would be exposed in the fake derivation channel as `packageset-foo`.

Presumably this was done to avoid needing to track attribute sets, and
to avoid the complexity. I think this now correctly handles the
complexity and properly mirrors the input expressions layout.
2021-03-18 11:33:37 -04:00
Graham Christensen 88e0198a8e
Create a helper for dealing with nested attribute sets 2021-03-18 11:33:36 -04:00
Graham Christensen d62a2c1657
NixExprs: extract the escape function and test it 2021-03-18 11:24:17 -04:00
Eelco Dolstra aeb3d2f44c
Merge pull request #892 from grahamc/hydra-queue-runner-build-one
hydra-queue-runner: --build-one: correctly handle a cached build
2021-03-16 21:28:32 +01:00
Graham Christensen 87d46ad5d6
hydra-queue-runner: --build-one: correctly handle a cached build
Previously, the build ID would never flow through channels which
exited.

This patch tracks the buildOne state as part of State and exits avoids
waiting forever for new work.

The code around buildOnly is a bit rough, making this a bit weird to
implement but since it is only used for testing the value of improving
it on its own is a bit questionable.
2021-03-16 16:13:38 -04:00
Janne Heß 3c86083d21
Fixup #717 "Add the project name to declarative inputs"
```
Mar 10 16:22:35 hydra-b hydra-evaluator[41419]: DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR:  null value in column "type" violates not-null constraint
Mar 10 16:22:35 hydra-b hydra-evaluator[41419]: DETAIL:  Failing row contains (62358, projectName, 0, null, null, null, hackworthltd, null, , null). [for Statement "INSERT INTO jobsetevalinputs ( altnr, dependency, eval, name, path, revision, sha256hash, type, uri, value) VALUES ( ?, ?, ?, ?, ?, ?, ?, ?, ?, ? )" with ParamValues: 1='0', 2=undef, 3='62358', 4='projectName', 5='', 6=undef, 7=undef, 8=undef, 9=undef, 10='hackworthltd'] at /nix/store/cmqblv437mp57yz5lwvkzcqca4ldf3r5-hydra-0.1.20210308.ebf1cd2/bin/.hydra-eval-jobset-wrapped line 793
Mar 10 16:22:35 hydra-b hydra-evaluator[25828]: evaluation of jobset ‘hackworthltd:.jobsets (jobset#1)’ failed with exit code 1
```

Use the abstraction for creating inputs for simulating the project
name input.

Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-03-16 09:52:36 -04:00
Shea Levy 930f05c38e
Bump Nix version 2021-03-10 12:53:03 -05:00
Graham Christensen b9fb66401b
Merge pull request #880 from grahamc/runcommand-finished-bool
RunCommand: emit the `finished` field as a boolean
2021-03-09 09:58:43 -05:00
Graham Christensen 2179b4b4b0
RunCommand: emit the finished field as a boolean 2021-03-08 12:11:20 -05:00
Janne Heß 9e018d5443
Add the project name to declarative inputs
This allows for more generic declarative configurations which can be
shared between projects.
2021-03-08 17:36:52 +01:00
Matej Cotman a551fba346
statsd: add a chance to set hostname and port in hydra.conf
Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-03-08 10:03:16 -05:00
Josh McSavaney e0d3a1c1a5
Make nix-build args copy-pastable via set -x
A reproduce script includes a logline that may resemble:

> using these flags: --arg nixpkgs { outPath = /tmp/build-137689173/nixpkgs/source; rev = "fdc872fa200a32456f12cc849d33b1fdbd6a933c"; shortRev = "fdc872f"; revCount = 273100; } -I nixpkgs=/tmp/build-137689173/nixpkgs/source --arg officialRelease false --option extra-binary-caches https://hydra.nixos.org/ --option system x86_64-linux /tmp/build-137689173/nixpkgs/source/pkgs/top-level/release.nix -A 

These are passed along to nix-build and that's fine and dandy, but you can't just copy-paste this as is, as the `{}` introduces a syntax error and the value accompanying `-A` is `''`.

A very naive approach is to just `printf "%q"` the individual args, which makes them safe to copy-paste. Unfortunately, this looks awful due to the liberal usage of slashes:

```
$ printf "%q" '{ outPath = /tmp/build-137689173/nixpkgs/source; rev = "fdc872fa200a32456f12cc849d33b1fdbd6a933c"; shortRev = "fdc872f"; revCount = 273100; }'
\{\ outPath\ =\ /tmp/build-137689173/nixpkgs/source\;\ rev\ =\ \"fdc872fa200a32456f12cc849d33b1fdbd6a933c\"\;\ shortRev\ =\ \"fdc872f\"\;\ revCount\ =\ 273100\;\ \}
```

Alternatively, if we just use `set -x` before we execute nix-build, we'll get the whole invocation in a friendly, copy-pastable format that nicely displays `{}`-enclosed content and preserves the empty arg following `-A`:

```
running nix-build...
using this invocation: 
+ nix-build --arg nixpkgs '{ outPath = /tmp/build-138165173/nixpkgs/source; rev = "e0e4484f2c028d2269f5ebad0660a51bbe46caa4"; shortRev = "e0e4484"; revCount = 274008; }' -I nixpkgs=/tmp/build-138165173/nixpkgs/source --arg officialRelease false --option extra-binary-caches https://hydra.nixos.org/ --option system x86_64-linux /tmp/build-138165173/nixpkgs/source/pkgs/top-level/release.nix -A ''
```
2021-03-06 23:25:26 -05:00
Graham Christensen 68ac64dbd9
Merge pull request #832 from wizeman/fix-hash-mismatch
Fix persistent hash mismatch errors when importing
2021-03-02 16:04:23 -05:00
Graham Christensen a756614fa1
RunCommand: pass homepage, description, license, system, and nixname 2021-02-24 16:13:09 -05:00
Graham Christensen 3fda37f65a
RunCommand: Test 2021-02-24 13:43:25 -05:00
Graham Christensen e4cda87b5a
db.hh: use hasPrefix for prefix comparisons
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-02-24 07:00:26 -05:00
Graham Christensen fe1f2f0806
Create an ephemeral PostgreSQL database per test 2021-02-23 21:12:06 -05:00
regnat f602ed0d86 Remove the sendDerivation logic from the builder
The queue runner used to special-case `localhost` as a remote builder:
Rather than using the normal remote-build (using the
`cmdBuildDerivation` command), it was using the (generally less
efficient, except when running against localhost) `cmdBuildPaths`
command because the latter didn't require a privileged Nix user (so made
testing easier − allowing to run hydra in a container in particular).

However:
1. this means that the build loop can follow two discint code paths depending
   on the setup, the irony being that the most commonly used one in production
   (the “non-localhost” case) isn't the one used in the testsuite (because all
   the tests run against a local store);
2. It turns out that the “localhost” version is buggy in relatively obvious
   ways − in particular a failure in a fixed-output derivation or a hash
   mismatch isn't reported properly;
3. If the “run in a container” use-case is indeed that important, it can be
   (partially) restored using a chroot store (which wouldn't behave excactly
   the same way of course, but would be more than good-enough for testing)
2021-02-23 09:50:15 +01:00
Eelco Dolstra 107d60027f hydra-eval-jobs: Fix unexpected EOF when a top-level attr fails 2021-02-22 16:29:07 +01:00
Eelco Dolstra a7d8ee98da Fix build 2021-02-22 15:10:24 +01:00
Eelco Dolstra a39b479280
Merge pull request #866 from Infinisil/github-status-flakes
Fix Github status plugin for flakes
2021-02-16 17:00:46 +01:00
Christian Kögler 150213cbb3 Fix login if Hydra runs behind HTTP proxy with sub-path location 2021-02-07 19:18:29 +01:00
Silvan Mosberger 58dd7f9ed3
Fix Github status plugin for flakes
If the root flake is a github: one, github status notifications are sent
to it. The githubstatus->inputs configuration isn't used for flakes.
2021-02-06 00:02:30 +01:00
Ismaël Bouya 339a09f2e4
Fix check in jobsets
The current check happening in jobsets is incorrect.
The wanted constraint is stated as follow :
- If type is 0 (legacy), then the flake field should be null, and
  both nixExprInput and nixExprPath should be non-null
- If type is 1 (flake), then the flake field should be non-null, and
  both nixExprInput and nixExprPath should be null

The current version will not catch (i.e. it will accept) situations
where you have for instance :
type = 1, nixExprPath null, nixExprInput non-null, flake non-null

This commit fixes that.

I split(ted) that into two constraints, to make it more readable and
easier to extend if a new type appears in the future.

The complete query could be instead :
( type = 0
  AND nixExprInput IS NOT NULL AND nixExprPath IS NOT NULL AND flake IS NULL )
OR ( type = 1
  AND nixExprInput IS NULL AND nixExprPath IS NULL AND flake IS NOT NULL )

(but an "OR" cannot be split, hence the other formulation)
2021-02-03 22:14:53 +01:00
Graham Christensen bc12fe19f9
Merge pull request #855 from grahamc/jobsetevals-fixups
JobsetEvals: fixup permission references
2021-02-02 11:04:18 -05:00
Graham Christensen 6de9c6540c
Merge pull request #858 from Infinisil/fix-declarative-flakes
Fix transition from declarative non-flake to flake jobset
2021-02-02 11:04:05 -05:00
Graham Christensen f1e75c8bff
Move evaluation errors from evaluations to EvaluationErrors, a new table
DBIx likes to eagerly select all columns without a way to really tell
it so. Therefore, this splits this one large column in to its own
table.

I'd also like to make "jobsets" use this table too, but that is on hold
to stop the bleeding caused by the extreme amount of traffic this is
causing.
2021-02-01 21:33:14 -05:00
Silvan Mosberger 1d45b63516
Fix transition from declarative non-flake to flake jobset
The database has these constraints:

    check ((type = 0) = (nixExprInput is not null and nixExprPath is not null)),
    check ((type = 1) = (flake is not null)),

which prevented switching to flakes in a declarative jobspec, since the
nixexpr{path,input} fields were not nulled in such an update

Co-Authored-By: Graham Christensen <graham@grahamc.com>
2021-02-01 18:57:40 +01:00
Graham Christensen 8d7bfe1706
JobsetEvals: fixup permission references
Going from an eval to a project now requires hopping through the jobset
2021-02-01 10:31:05 -05:00
Graham Christensen 91e63fb7da
search: limit queries to 20s
Even 20s is really long, but it cuts off queries which are today
running for 500+s.
2021-01-30 11:51:20 -05:00
Graham Christensen 4f308b1f2f
search: limit results to 50, default to 10
This search query is pretty heavy. Defaulting to 500 has caused
Hydra's web UI to appear to be down. Since 500 can take it down, users
probably shouldn't be allowed t ask for that many.
2021-01-30 08:37:57 -05:00
Graham Christensen 5fcc2018db
hydra-evaluator: clean up names, clean up & / * spacing 2021-01-28 09:15:19 -05:00
Graham Christensen 091d58c128
hydra-dev-server: autoflush stderr/stdout 2021-01-26 13:51:39 -05:00
Graham Christensen 54b8cb188e
perl: jobsetevals -> jobset via by jobset_id
Frankly, this was suspiciously little work.
2021-01-26 13:51:39 -05:00
Graham Christensen 54341cd9f6
hydra-evaluator: deal in jobset IDs 2021-01-26 13:51:31 -05:00
Graham Christensen cb01859718
hydra-evaluator: JobsetName -> JobsetIdentity 2021-01-26 11:50:38 -05:00
Graham Christensen 705a45df2b
hydra-evaluator: reformat readJobsets query 2021-01-26 11:50:37 -05:00
Graham Christensen ac3e8a4a59
jobsetevals: refer to jobset by ID 2021-01-26 11:50:37 -05:00
Graham Christensen 99e3c83358
JobsetEvals: noop: re-run the generator to update the order of fields 2021-01-26 11:50:36 -05:00
Graham Christensen bf674a9653
hydra.sql: embed some in-line docs about schema changes 2021-01-26 11:50:36 -05:00
Graham Christensen dc5a0d59c5
sql: Stop loading SQL if an error occurs
Otherwise we may go ahead and create DBIx classes for a half-loaded schema.
2021-01-26 11:50:32 -05:00
Graham Christensen 9516b256f1
Normalize nixexpr{input,path} from builds to jobsetevals.
Duplicating this data on every record of the builds table cost
approximately 4G of duplication.

Note that the database migration included took about 4h45m on an
untuned server which uses very slow rotational disks in a RAID5 setup,
with not a lot of RAM. I imagine in production it might take an hour
or two, but not 4. If this should become a chunked migration, I can do
that.

Note: Because of the question about chunked migrations, I have NOT
YET tested this migration thoroughly enough for merge.
2021-01-22 09:10:18 -05:00
Graham Christensen c64c4aac4f
jobset page: render error labels per eval 2021-01-21 17:08:02 -05:00
Graham Christensen 805dd6e7ee
Evaluation page: render evaluation errors 2021-01-21 13:11:05 -05:00
Graham Christensen 086eed5147
hydra-eval-jobs: write evaluation errorMsg to the jobseteval table 2021-01-21 13:10:41 -05:00
Graham Christensen d9989b7fa1
Schema: add errorMsg, errorTime to JobsetEvals 2021-01-21 13:10:41 -05:00
Graham Christensen bc4b96d053
BuildOutputs: index path with HASH
Looking at AWS' Performance Insights for a Hydra instance, I found
the hydra-queue-runner's query:

    select id, buildStatus, releaseName, closureSize, size
    from Builds b
    join BuildOutputs o on b.id = o.build
    where
      finished = ?
      and (buildStatus = ? or buildStatus = ?)
      and path = $1

was the slowest query by at least 10x. Running an explain on this
showed why:

hydra=> explain select id, buildStatus, releaseName, closureSize, size
    from Builds b join BuildOutputs o on b.id = o.build where
    finished = 1 and (buildStatus = 0 or buildStatus = 6) and
    path = '/nix/store/s93khs2dncf2cy273mbyr4fb4ns3db20-MIDIVisualizer-5.1';

                                                     QUERY PLAN
    ------------------------------------------------------------------------
     Gather  (cost=1000.43..33718.98 rows=2 width=56)
       Workers Planned: 2
       ->  Nested Loop  (cost=0.43..32718.78 rows=1 width=56)
             ->  Parallel Seq Scan on buildoutputs o  (cost=0.00..32710.32
                                                       rows=1
                                                        width=4)
                   Filter: (path = '/nix/store/s93kh...snip...'::text)
             ->  Index Scan using indexbuildsonjobsetidfinishedid on builds b
                                            (cost=0.43..8.45 rows=1 width=56)
                   Index Cond: ((id = o.build) AND (finished = 1))
                   Filter: ((buildstatus = 0) OR (buildstatus = 6))
    (8 rows)

A paralell sequential scan is definitely better than a sequential scan, but the
cost ranging from 0 to 32710 is not great. Looking at the table, I saw the `path`
column is completely unindex:

    hydra=> \d buildoutputs
                Table "public.buildoutputs"
    Column |  Type   | Collation | Nullable | Default
    --------+---------+-----------+----------+---------
    build  | integer |           | not null |
    name   | text    |           | not null |
    path   | text    |           | not null |
    Indexes:
        "buildoutputs_pkey" PRIMARY KEY, btree (build, name)
    Foreign-key constraints:
        "buildoutputs_build_fkey" FOREIGN KEY (build) REFERENCES builds(id)
            ON DELETE CASCADE

Since we always do exact matches on the path and don't care about ordering,
and since the path column is very high cardinality a `hash` index is a
good candidate. Note that I did test a btree index and it performed
similarly well, but slightly worse.

After creating the index (this took about 10 seconds) on a test database:

    create index IndexBuildOutputsPath on BuildOutputs using hash(path);

We get a *significantly* reduced cost:

    hydra=> explain select id, buildStatus, releaseName, closureSize, size
    hydra->     from Builds b join BuildOutputs o on b.id = o.build where
    hydra->     finished = 1 and (buildStatus = 0 or buildStatus = 6) and
    hydra->     path = '/nix/store/s93khs2dncf2cy273mbyr4fb4ns3db20-MIDIVisualizer-5.1';
                                                QUERY PLAN
    -------------------------------------------------------------------------------------------------------
    Nested Loop  (cost=0.43..41.41 rows=2 width=56)
    ->  Index Scan using buildoutputs_path_hash on buildoutputs o  (cost=0.00..16.05 rows=3 width=4)
            Index Cond: (path = '/nix/store/s93khs2dncf2cy273mbyr4fb4ns3db20-MIDIVisualizer-5.1'::text)
    ->  Index Scan using indexbuildsonjobsetidfinishedid on builds b  (cost=0.43..8.45 rows=1 width=56)
            Index Cond: ((id = o.build) AND (finished = 1))
            Filter: ((buildstatus = 0) OR (buildstatus = 6))
    (6 rows)

For direct comparison, the overall query plan was changed:

    From: Gather      (cost=1000.43..33718.98 rows=2 width=56)
    To:   Nested Loop (cost=   0.43.....41.41 rows=2 width=56)

and the query plan for buildoutputs changed from a maximum cost of
32,710 down to 16.

In practical terms, the query's planning and execution time was reduced:

Before (ms) | Try 1   | Try 2   | Try 3
------------+---------+---------+--------
Planning    |   0.898 |   0.416 |   0.383
Execution   | 138.644 | 172.331 | 375.585

After (ms)  | Try 1   | Try 2   | Try 3
------------+---------+---------+--------
Planning    |   0.298 |   0.290 |   0.296
Execution   | 219.625 |   0.035 |   0.034
2021-01-18 11:28:05 -05:00
Eelco Dolstra be0aa7eb85
Merge pull request #841 from pingiun/github-login
Implement GitHub logins
2021-01-05 14:51:51 +01:00
Jelle Besseling 43d662f63a
Don't use enable_github_login option after all
Instead the github_client_id option is used to detect if github logins
should be enabled.
2021-01-04 18:09:49 +01:00
Jelle Besseling c49ca66689
Die when no email is found 2021-01-04 18:09:05 +01:00
Jelle Besseling 20d8134936
Update src/lib/Hydra/Controller/User.pm
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-01-04 17:48:43 +01:00
Jelle Besseling 19f9d8249f
Update src/lib/Hydra/Controller/User.pm
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-01-04 17:48:37 +01:00
Eelco Dolstra 20d09518f8
Merge pull request #839 from pingiun/shield-io
Add endpoint to generate a shields.io badge
2021-01-04 14:09:09 +01:00
Eelco Dolstra c4062c2772
Merge pull request #842 from pingiun/also-trigger-flakes
Also trigger flake based jobsets with push-github endpoint
2021-01-03 18:28:29 +01:00
Eelco Dolstra b59a5850a8 Merge branch 'receiveContents' of https://github.com/orivej/hydra 2021-01-03 18:26:04 +01:00
Jelle Besseling 5f4eddbe57
Use email scope 2020-12-31 13:40:33 +01:00
Jelle Besseling e88355b3d4
Use email api call 2020-12-31 13:40:32 +01:00
Matthew Bauer d38ef4492e Update for Nix Value::type API changes
This updates hydra to be compatible with Nix https://github.com/NixOS/nix/pull/4355.

Along with https://github.com/NixOS/hydra/pull/840 needed for https://github.com/NixOS/nixpkgs/pull/107909

/cc @edolstra
2020-12-29 17:25:51 -06:00
Jelle Besseling 1b3000e132
Allow push-github endpoint to also trigger flakes 2020-12-28 15:27:09 +01:00
Jelle Besseling bbd4891133
Implement GitHub logins
Requires the following configuration options
enable_github_login = 1
github_client_id
github_client_secret
Or github_client_secret_file which points to a file with the secret
2020-12-28 14:37:03 +01:00
Orivej Desh 34a856c7ab Update for receiveContents taking string_view
nix change: https://github.com/NixOS/nix/commit/faa31f40
2020-12-26 11:23:26 +00:00
Jelle Besseling f64230b45e
Add endpoint to generate a shields.io badge 2020-12-25 15:05:34 +01:00
Eelco Dolstra bde8d81876
Merge pull request #811 from helsinki-systems/fix/override-constraint
Stop violating not null constraint
2020-11-22 00:02:26 +01:00
Janne Heß bd0ab9a5fb
Stop violating not null constraint
Fixes this error:

ERROR: failed to process declarative jobset test:inputs,
DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st
execute failed: ERROR:  null value in column "emailoverride" violates
not-null constraint
2020-11-21 22:04:40 +01:00
Ricardo M. Correia f47749a62d Fix persistent hash mismatch errors when importing
This would start happening if the network connection between the Hydra
server and the remote build server breaks after sucessfully importing
at least one output of a derivation, but before having finished
importing all outputs.

Fixes #816.
2020-11-10 04:50:35 +01:00
Eelco Dolstra 73dfef364b Copy deriver field to the binary cache
Fixes https://github.com/NixOS/nixos-org-configurations/issues/129.
2020-11-02 17:08:02 +01:00
Nathan van Doorn 2742fde8c2
Remove Debug prints from GitLabStatus.pm
These make the hydra-queue-runner logs very noisy even when not using the GitlabStatus plugin.
Also, they shouldn't be necessary except when developing the plugin itself and should have been removed before release.
2020-11-02 10:14:54 +00:00
Eelco Dolstra 87317812a8 Fix some broken indices
These indices basically did nothing since they put "id" first.

In particular this makes /job/.../all much faster.
2020-10-28 14:30:44 +01:00
Eelco Dolstra 8bb23905c3 Build: Remove unused prevBuild
This speeds up loading the page a lot in the case where there is no
previous evaluation (for some reason).
2020-10-28 13:29:31 +01:00
Eelco Dolstra d9dc7ca18b getPreviousBuild: Get previous build in the job, not jobset
Broken since 8adb433e3.
2020-10-28 13:29:02 +01:00
Eelco Dolstra a1b5418114
Remove dead link 2020-10-27 12:35:36 +01:00
Samuel Dionne-Riel b5140c1da1 hydra-eval-jobs: Transmit original Nix error when handling aggregate jobs
It might happen that a job from the aggregate returned an error!

This is what the vague "[json.exception.type_error.302] type must be string, but is null"
was all about in this instance; there was no `drvPath` to stringify!

So we now actively watch for errors and copy them to the aggregate job.
2020-10-25 20:13:07 -04:00
Samuel Dionne-Riel 68e689cace hydra-eval-jobs: Identify unexpected errors in handling aggregate jobs
The vague "[json.exception.type_error.302] type must be string, but is null"
is **absolutely** unhelpful in the way Hydra currently handles it on
evaluation.

This is handling *unexpected* errors only; the following commit will
handle the specific instance of the previously mentioned error.
2020-10-25 19:27:15 -04:00
Eelco Dolstra be709d450b Fix sysbuild
596f4cf4b9
2020-10-22 13:27:52 +02:00
Eelco Dolstra 4e05acc471 Fix localhost builds 2020-10-20 12:11:46 +02:00
Eelco Dolstra 6cd2bb6954 Fix build 2020-10-18 21:01:06 +02:00
Maximilian Bosch 9cc76f6d69
Fix build with latest Nix
Recently a few internal APIs have changed[1]. The `outputPaths` function
has been removed and a lot of data structures are modeled with
`std::optional` which broke compilation.

This patch updates the code in `hydra-queue-runner` accordingly to make
sure that Hydra compiles again.

[1] https://github.com/NixOS/nix/pull/3883
2020-09-26 23:37:39 +02:00
Eelco Dolstra 46a8d6d531 Merge branch 'fix/addPermRoot' of https://github.com/helsinki-systems/hydra 2020-09-15 13:22:53 +02:00
Eelco Dolstra b8a4ed0c77
Merge pull request #815 from helsinki-systems/fix/non-static-decl
Fix non-static declarative jobsets
2020-09-14 15:52:05 +02:00
Eelco Dolstra 2c76e9848a
Merge pull request #812 from helsinki-systems/no-accesslog
Disable access log
2020-09-14 15:38:13 +02:00
Eelco Dolstra a6c4e4b067
Merge pull request #807 from plutotulp/bugfix-revcount-as-integer
reproduce bash script must supply revCount as int
2020-09-14 15:34:03 +02:00
ajs124 2c61779816 fix build after nix commit 82b77a77262c414044fffc7ad8b955ad91827995 2020-09-14 15:30:59 +02:00
Janne Heß 24acb9d6bb
Fix non-static declarative jobsets
With the current implementation, if ANY hash was found inside the decl
spec, the spec would be treated as static. This is problematic since
`inputs` is a hash and hence any configuration would be handled as a
static one.
This fixes the code to match the documentation and only switch to static
processing when ALL values are hashes.
2020-09-13 18:21:38 +02:00
Janne Heß 971dcc46a2
Disable access log
This is annoying and mostly redundant to nginx.
2020-09-13 17:51:21 +02:00
Andreas Rammhold 6a07712e1d LDAP: only try LDAP authentication when the realm is configured 2020-09-12 19:57:24 +02:00
Erlend Pedersen 1d11b29e73 reproduce bash script must supply revCount as int
The reproduce script would fail for me, because it passed revCount as
a string to nix-build, which would then fail to subtract from it.
2020-09-12 14:04:43 +02:00
edef c00b42dced don't try to load HYDRA_LDAP_CONFIG if none is provided 2020-09-09 13:02:51 +02:00
ajs124 28646e1c5f Initial attempt at adding LDAP login support 2020-09-09 13:00:34 +02:00
Eelco Dolstra e707990e2d
Merge pull request #804 from grahamc/fully-static-jobsets
declarative projects: support fully static, declarative configuration
2020-09-02 21:09:01 +02:00
Graham Christensen 7f16c0d243
declarative projects: support fully static, declarative configuration 2020-09-02 12:35:41 -04:00
Graham Christensen 648eb980dd
hydra-notify: autoflush stdout too 2020-09-02 12:35:18 -04:00
Casey Ransom 03be8ae7a1 Remove image dependency on hydra.nixos.org
Also standardize the slack image size on 256 pixels.
2020-09-01 15:13:12 -04:00
Eelco Dolstra 405c52b589
Fix build 2020-08-27 17:46:36 +02:00
Eelco Dolstra 1113c2895a Fix build 2020-08-07 21:42:09 +02:00
Eelco Dolstra f8e15bc311 Revive putBytes 2020-08-04 18:25:21 +02:00
Eelco Dolstra 4b5813051b
unsigned long long -> uint64_t 2020-08-04 11:38:22 +02:00
Eelco Dolstra 7d3ba616a9
Fix build 2020-08-04 11:33:29 +02:00
Eelco Dolstra 77c33c1d71
Restore NoCheckSigs
https://github.com/NixOS/nixpkgs/pull/93945#issuecomment-668244478
2020-08-04 10:53:06 +02:00
Eelco Dolstra 8722927c08
Copy paths in the right order 2020-07-28 13:46:57 +02:00