Commit graph

3849 commits

Author SHA1 Message Date
Graham Christensen 59ac96a99c Track the number of steps created 2022-04-06 20:23:02 -04:00
Graham Christensen 1c12c5882f hydra queue runner: instrument the process of loading new builds with prom 2022-04-06 20:18:29 -04:00
Graham Christensen 5de08d412e queue metrics: refactor the metrics into a struct 2022-04-06 20:00:30 -04:00
Graham Christensen 46f52b4c4e bring back the working version Cole made 2022-04-06 15:49:38 -04:00
Cole Helbling 5bff730f2c WIP: I love it when they delete the assignment operator :) 2022-04-06 11:41:40 -07:00
Cole Helbling 15e8fa8aff doc/manual: document queue-runner prometheus exporter configuration 2022-04-06 11:41:40 -07:00
Cole Helbling edf3c348f2 hydra-queue-runner: make entire address configurable 2022-04-06 10:59:45 -07:00
Cole Helbling 33bc60b83c hydra-queue-runner: move exporter back to State::run
It's (arguably) better than risking pinning the thread at 100% due to
the busy `while` loop.
2022-04-06 10:49:14 -07:00
Cole Helbling 8c5636fe18
hydra-queue-runner: use port 9198 by default
Co-authored-by: Graham Christensen <graham@grahamc.com>
2022-04-02 17:32:14 -07:00
Cole Helbling 9c1f36c47c t/lib/HydraTestContext: set queue runner port to 0
This makes the exposer choose a random, available port.
2022-03-29 11:41:23 -07:00
Cole Helbling 4789eba92c hydra-queue-runer: split metrics functionality into its own function 2022-03-29 10:55:28 -07:00
Cole Helbling 928b3b8268 hydra-queue-runner: fix priority of flag over config file 2022-03-29 10:42:07 -07:00
Cole Helbling 5ddb9a98ca fixup! hydra-queue-runner: log message before and after exporter is started 2022-03-29 08:47:41 -07:00
Cole Helbling 905a7a7beb hydra-queue-runner: read metrics port from queue_runner_metrics_port config 2022-03-29 08:46:43 -07:00
Cole Helbling 9cdc5aceed hydra-queue-runner: log message before and after exporter is started
This way, if something goes wrong between the two, it's easier to narrow
down where the issue lies.
2022-03-29 08:41:19 -07:00
Cole Helbling 8503a7917b fixup! hydra-queue-runner: make registry member of State, configurable metrics port 2022-03-22 13:38:13 -07:00
Cole Helbling b0c17112c9 flake: update to nixos-unstable-small
https://github.com/NixOS/nixpkgs/pull/163695 was merged, so no longer
need to use my commit!
2022-03-18 11:10:57 -07:00
Cole Helbling c0f826b92d hydra-queue-runner: get the listening port from the exposer itself
Otherwise, when the port is randomly chosen (e.g. by specifying no port,
or a port of 0), it will just show that the port is 0 and not the port
that is actually serving the metrics.
2022-03-14 08:41:45 -07:00
Cole Helbling 52a29d43e6 hydra-queue-runner: make registry member of State, configurable metrics port
Thanks to the updated prometheus-cpp library, specifying a port of 0
will cause it to pick a random (available) port -- ideal for tests.
2022-03-11 11:58:10 -08:00
Cole Helbling 6e6475d860 flake: replace aliases with their proper names
Newer Nixpkgs have added a throw for these aliases.
2022-03-11 11:58:10 -08:00
Cole Helbling a0cb73579d flake: update newNixpkgs for newer prometheus-cpp 2022-03-11 11:58:10 -08:00
Cole Helbling 3bf31bd6a6 hydra-queue-runner: add simple "up" exporter
There are probably better ways to achieve this (and will likely need to
be refactored a bit to support further metrics).
2022-03-10 12:36:58 -08:00
Cole Helbling 5bbaa18a8f flake: fix foreman execution
[vin@scadrial:~/workspace/vcs/hydra]$ foreman -h
    Warning: the running version of Bundler (2.1.4) is older than the version that created the lockfile (2.2.20). We suggest you to upgrade to the version that created the lockfile by running `gem install bundler:2.2.20`.
    Traceback (most recent call last):
    	2: from /nix/store/ycshcdssxcj9sjf6yzb1ydw4fcglf66y-foreman-0.87.2/bin/foreman:20:in `<main>'
    	1: from /nix/store/ggqacj06n6qfm1iww0bih9ph0j89wcna-bundler-2.1.4/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/lib/bundler/rubygems_integration.rb:413:in `block in replace_bin_path'
    /nix/store/ggqacj06n6qfm1iww0bih9ph0j89wcna-bundler-2.1.4/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/lib/bundler/rubygems_integration.rb:374:in `block in replace_bin_path': can't find executable foreman for gem foreman. foreman is not currently included in the bundle, perhaps you meant to add it to your Gemfile? (Gem::Exception)
2022-03-10 12:26:13 -08:00
Graham Christensen 28b682b85b
Merge pull request #1170 from NixOS/dependabot/github_actions/actions/checkout-3
build(deps): bump actions/checkout from 2 to 3
2022-03-09 21:13:47 -05:00
dependabot[bot] b9ec3a41d6
build(deps): bump actions/checkout from 2 to 3
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-07 15:01:23 +00:00
Graham Christensen a2546121f0
Merge pull request #1163 from DeterminateSystems/constituents
Constituents: Add a GC root for indirect aggregates
2022-02-21 20:42:42 -05:00
Graham Christensen 9316544abf
src/hydra-eval-jobs/hydra-eval-jobs.cc: .get<std::string> for drvPath
Co-authored-by: Kayla Fire <firestack@users.noreply.github.com>
2022-02-21 12:41:21 -05:00
Graham Christensen 290e0653ad hydra-eval-jobs: GC root aggregate jobs 2022-02-20 12:28:40 -05:00
Graham Christensen e0921eba0a Create a basic test which verifies we can't delete the derivation of aggregate jobs 2022-02-20 12:28:40 -05:00
Graham Christensen be46f02164 tests: relocate evaluator tests 2022-02-20 12:28:40 -05:00
Graham Christensen 5d169e3a2e Add a test validating direct and indirect constituents 2022-02-20 12:28:40 -05:00
Graham Christensen dfb3eccfaa
Merge pull request #1140 from Ma27/nix-update
Update Nix to 2.6
2022-02-19 08:38:34 -05:00
Graham Christensen 4c41ca08e1
Merge pull request #1155 from helsinki-systems/fix/graph-readability
build-graphs: Fix readability in dark mode
2022-02-14 11:27:37 -05:00
Graham Christensen 1124230d9f
Merge pull request #1151 from DeterminateSystems/ldap-tests-inc-mapping
ldap: support configurable roles with backwards compatibility
2022-02-14 09:30:54 -05:00
Janne Heß 6d146deaf0
build-graphs: Fix readability in dark mode 2022-02-13 14:00:17 +01:00
Graham Christensen 78e9872251 ldap.t: write the password to an external .conf file 2022-02-11 11:27:10 -05:00
Graham Christensen 848fb3b265 ldap-legacy.t: specify the root password manually 2022-02-11 11:26:56 -05:00
Graham Christensen 0bd4a75918 HydraTestContext: support running a sub before running hydra-init. 2022-02-11 11:26:27 -05:00
Graham Christensen 05ca71069f ldap config: document putting the password in a separate file 2022-02-11 11:24:45 -05:00
Graham Christensen e13d80f5cf LDAPContext: take a root_password argument or generate one 2022-02-11 11:02:04 -05:00
Graham Christensen 86bb16d07b LDAPContext: sort $self hash keys 2022-02-11 11:01:25 -05:00
Graham Christensen 6637c03985 fixup normalization error regex 2022-02-11 10:59:24 -05:00
Graham Christensen 71c06f2ce7 LDAP normalization errors: note that the error came while normalizing the roles. 2022-02-11 10:55:27 -05:00
Graham Christensen d6dea39912 ldap_role_map.t: fixup indentation 2022-02-11 10:53:08 -05:00
Graham Christensen 185100adb8 docs: fixup 2022-02-11 10:50:58 -05:00
Graham Christensen f07fb7d279 LDAP support: include BC support for the YAML based loading
Includes a refactoring of the configuration loader.
2022-02-11 10:49:38 -05:00
Janne Heß 61d74a7194 Redo LDAP config in the main configuration and add role mappings 2022-02-11 10:49:38 -05:00
Graham Christensen 76b4b43ac5 Move ldap.t to a legacy-ldap.t, make ldap.t use the new format config. 2022-02-11 10:49:38 -05:00
Graham Christensen d0bc0d0eda
Merge pull request #1152 from DeterminateSystems/parallel-tests
Parallel tests, fix a hydra-queue-runner race condition
2022-02-10 12:11:20 -05:00
Graham Christensen 74a29e8ea9
Merge pull request #1150 from DeterminateSystems/ldap-tests
LDAP: Test without a VM
2022-02-10 12:05:19 -05:00