ma27/hydra
0
0
Fork 0
forked from lix-project/hydra
Code Pull requests Activity
3893 commits 4 branches 0 tags 25 MiB
Commit graph

3893 commits

This branch
This branch
All branches
Author SHA1 Message Date
Graham Christensen 137be3452e Reduce the jobset cols on the remaining two queries 2022-03-19 23:56:47 -04:00
Graham Christensen f353a7ac41 update-gc-roots: try subselecting the jobset table 2022-03-19 23:56:47 -04:00
Graham Christensen 145667cb53 hydra-update-gc-roots: allow cached refs to the build's jobset 
Re-executing this search_related on every access turned out to
create very problematic performance. If a jobset had a lot of
error output stored in the jobset, and there were many hundreds
or thousands of active jobs, this could easily cause >1Gbps of
network traffic.
 2022-03-19 23:56:47 -04:00
Graham Christensen 22026da4f8
Merge pull request #1176 from DeterminateSystems/broken-constituent 
Broken constituents: emit useful log messages on evaluation errors on constituents
 2022-03-19 14:55:17 -04:00
Graham Christensen a582e4c485 HydraTestContext: add \n's to various dies 2022-03-19 14:46:53 -04:00
Graham Christensen 074a2f96bf hydra-eval-jobset: emit a useful error if constituents errored 2022-03-19 14:37:12 -04:00
Graham Christensen 0c51de6334 hydra-evaluate-jobset: assert it logs errored constituents properly 2022-03-19 14:35:30 -04:00
Graham Christensen 25f6bae847 HydraTestContext: make it easy to create a jobset without evaluating 2022-03-19 14:34:43 -04:00
Cole Helbling b0c17112c9 flake: update to nixos-unstable-small 
https://github.com/NixOS/nixpkgs/pull/163695 was merged, so no longer
need to use my commit!
 2022-03-18 11:10:57 -07:00
Cole Helbling c0f826b92d hydra-queue-runner: get the listening port from the exposer itself 
Otherwise, when the port is randomly chosen (e.g. by specifying no port,
or a port of 0), it will just show that the port is 0 and not the port
that is actually serving the metrics.
 2022-03-14 08:41:45 -07:00
Cole Helbling 52a29d43e6 hydra-queue-runner: make registry member of State, configurable metrics port 
Thanks to the updated prometheus-cpp library, specifying a port of 0
will cause it to pick a random (available) port -- ideal for tests.
 2022-03-11 11:58:10 -08:00
Cole Helbling 6e6475d860 flake: replace aliases with their proper names 
Newer Nixpkgs have added a throw for these aliases.
 2022-03-11 11:58:10 -08:00
Cole Helbling a0cb73579d flake: update newNixpkgs for newer prometheus-cpp 2022-03-11 11:58:10 -08:00
Cole Helbling 3bf31bd6a6 hydra-queue-runner: add simple "up" exporter 
There are probably better ways to achieve this (and will likely need to
be refactored a bit to support further metrics).
 2022-03-10 12:36:58 -08:00
Cole Helbling 5bbaa18a8f flake: fix foreman execution 
[vin@scadrial:~/workspace/vcs/hydra]$ foreman -h
    Warning: the running version of Bundler (2.1.4) is older than the version that created the lockfile (2.2.20). We suggest you to upgrade to the version that created the lockfile by running `gem install bundler:2.2.20`.
    Traceback (most recent call last):
    	2: from /nix/store/ycshcdssxcj9sjf6yzb1ydw4fcglf66y-foreman-0.87.2/bin/foreman:20:in `<main>'
    	1: from /nix/store/ggqacj06n6qfm1iww0bih9ph0j89wcna-bundler-2.1.4/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/lib/bundler/rubygems_integration.rb:413:in `block in replace_bin_path'
    /nix/store/ggqacj06n6qfm1iww0bih9ph0j89wcna-bundler-2.1.4/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/lib/bundler/rubygems_integration.rb:374:in `block in replace_bin_path': can't find executable foreman for gem foreman. foreman is not currently included in the bundle, perhaps you meant to add it to your Gemfile? (Gem::Exception)
 2022-03-10 12:26:13 -08:00
Graham Christensen 28b682b85b
Merge pull request #1170 from NixOS/dependabot/github_actions/actions/checkout-3 
build(deps): bump actions/checkout from 2 to 3
 2022-03-09 21:13:47 -05:00
dependabot[bot] b9ec3a41d6
build(deps): bump actions/checkout from 2 to 3 
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-07 15:01:23 +00:00
Graham Christensen a2546121f0
Merge pull request #1163 from DeterminateSystems/constituents 
Constituents: Add a GC root for indirect aggregates
 2022-02-21 20:42:42 -05:00
Graham Christensen 9316544abf
src/hydra-eval-jobs/hydra-eval-jobs.cc: .get<std::string> for drvPath 
Co-authored-by: Kayla Fire <firestack@users.noreply.github.com>
 2022-02-21 12:41:21 -05:00
Graham Christensen 290e0653ad hydra-eval-jobs: GC root aggregate jobs 2022-02-20 12:28:40 -05:00
Graham Christensen e0921eba0a Create a basic test which verifies we can't delete the derivation of aggregate jobs 2022-02-20 12:28:40 -05:00
Graham Christensen be46f02164 tests: relocate evaluator tests 2022-02-20 12:28:40 -05:00
Graham Christensen 5d169e3a2e Add a test validating direct and indirect constituents 2022-02-20 12:28:40 -05:00
John Ericson 445bba337b Make copyClosureTo take a regular C++ ref to the store 
This is syntactically lighter wait, and demonstates there are no weird
dynamic lifetimes involved, just regular passing reference to callee
which it only borrows for the duration of the call.
 2022-02-20 17:22:43 +00:00
John Ericson f14c583ce5 Use copyClosure instead of computeFSClosure + copyPaths 
It is more terse, and in the future it is possible `copyClosure` will
become more sophisticated.
 2022-02-19 11:59:17 -05:00
Graham Christensen dfb3eccfaa
Merge pull request #1140 from Ma27/nix-update 
Update Nix to 2.6
 2022-02-19 08:38:34 -05:00
Graham Christensen 4c41ca08e1
Merge pull request #1155 from helsinki-systems/fix/graph-readability 
build-graphs: Fix readability in dark mode
 2022-02-14 11:27:37 -05:00
Graham Christensen 1124230d9f
Merge pull request #1151 from DeterminateSystems/ldap-tests-inc-mapping 
ldap: support configurable roles with backwards compatibility
 2022-02-14 09:30:54 -05:00
Janne Heß 6d146deaf0
build-graphs: Fix readability in dark mode 2022-02-13 14:00:17 +01:00
Graham Christensen 78e9872251 ldap.t: write the password to an external .conf file 2022-02-11 11:27:10 -05:00
Graham Christensen 848fb3b265 ldap-legacy.t: specify the root password manually 2022-02-11 11:26:56 -05:00
Graham Christensen 0bd4a75918 HydraTestContext: support running a sub before running hydra-init. 2022-02-11 11:26:27 -05:00
Graham Christensen 05ca71069f ldap config: document putting the password in a separate file 2022-02-11 11:24:45 -05:00
Graham Christensen e13d80f5cf LDAPContext: take a root_password argument or generate one 2022-02-11 11:02:04 -05:00
Graham Christensen 86bb16d07b LDAPContext: sort $self hash keys 2022-02-11 11:01:25 -05:00
Graham Christensen 6637c03985 fixup normalization error regex 2022-02-11 10:59:24 -05:00
Graham Christensen 71c06f2ce7 LDAP normalization errors: note that the error came while normalizing the roles. 2022-02-11 10:55:27 -05:00
Graham Christensen d6dea39912 ldap_role_map.t: fixup indentation 2022-02-11 10:53:08 -05:00
Graham Christensen 185100adb8 docs: fixup 2022-02-11 10:50:58 -05:00
Graham Christensen f07fb7d279 LDAP support: include BC support for the YAML based loading 
Includes a refactoring of the configuration loader.
 2022-02-11 10:49:38 -05:00
Janne Heß 61d74a7194 Redo LDAP config in the main configuration and add role mappings 2022-02-11 10:49:38 -05:00
Graham Christensen 76b4b43ac5 Move ldap.t to a legacy-ldap.t, make ldap.t use the new format config. 2022-02-11 10:49:38 -05:00
Graham Christensen d0bc0d0eda
Merge pull request #1152 from DeterminateSystems/parallel-tests 
Parallel tests, fix a hydra-queue-runner race condition
 2022-02-10 12:11:20 -05:00
Graham Christensen 74a29e8ea9
Merge pull request #1150 from DeterminateSystems/ldap-tests 
LDAP: Test without a VM
 2022-02-10 12:05:19 -05:00
Graham Christensen 4f9aea9434
t/Hydra/Plugin/gitea.t: explain why we loop a few times 
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
 2022-02-10 12:02:29 -05:00
Graham Christensen a1579007f4
Update t/test.pl: fixup the list of env vars yath looks at for job count 2022-02-10 11:53:10 -05:00
Graham Christensen b1b1b4b74b
t/lib/LDAPContext.pm: error message nit 
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
 2022-02-10 11:31:49 -05:00
Graham Christensen 33f4c4c13d build-locally-with-substitutable-path.t: give nix-store --delete a bit more time to run 
Under high load, like 64-128 tests at once, this can take more than a second.
 2022-02-10 11:13:31 -05:00
Graham Christensen e709a17508 gitea.t: try opening the file a few times 2022-02-10 11:01:09 -05:00
Graham Christensen 4acaf9c8b0 hydra-queue-runner: don't dispatch until the machines parser has completed one run 
Periodically, I have seen tests fail because of out of order queue runner behavior:

    checking the queue for builds > 0...
    loading build 1 (tests:basic:empty_dir)
    aborting unsupported build step '...-empty-dir.drv' (type 'x86_64-linux')
    marking build 1 as failed
    adding new machine ‘localhost’

This patch should prevent the dispatcher from running before any machines are
made available.
 2022-02-10 10:54:30 -05:00