Commit graph

12 commits

Author SHA1 Message Date
Graham Christensen 5fae9d96a2
hydra-evaluator: add a 'ONE_AT_A_TIME' evaluator style
In the past, jobsets which are automatically evaluated are evaluated
regularly, on a schedule. This schedule means a new evaluation is
created every checkInterval seconds (assuming something changed.)

This model works well for architectures where our build farm can
easily keep up with demand.

This commit adds a new type of evaluation, called ONE_AT_A_TIME, which
only schedules a new evaluation if the previous evaluation of the
jobset has no unfinished builds.

This model of evaluation lets us have 'low-tier' architectures.

For example, we could now have a jobset for ARMv7l builds, where
the buildfarm only has a single, underpowered ARMv7l builder.
Configuring that jobset as ONE_AT_A_TIME will create an evaluation
and then won't schedule another evaluation until every job of
the existing evaluation is complete.

This way, the cache will have a complete collection of pre-built
software for some commits, but the underpowered architecture will
never become backlogged in ancient revisions.
2020-03-03 19:28:44 -05:00
Graham Christensen eaa65f51f4
hydra-evaluator: make the logic of the scheduler easier to read 2020-03-03 18:17:21 -05:00
Eelco Dolstra b790a00729
Disable deprecation warnings
(cherry picked from commit 950e8bef6c)
2020-02-20 10:22:21 +01:00
Hamish Mackenzie c40c887e50
Fixes for macOS
Building on macOS with the latest nixpkgs master and NixOS/nixpkgs#77147
fails.  It seems some `std::experimental` (optional) for instance are
not available as `experimental`, but are in `std`.  Also `toJSON` is
missing for `atomic< unsigned long long >`.
2020-01-07 12:38:06 +13:00
Eelco Dolstra 09a4c9fad2
hydra-evaluator: Allow setting the maximum number of concurrent evaluations 2018-05-16 14:22:00 +02:00
Eelco Dolstra f0d24af940
Fix "Evaluation pending since..." message even though the jobset has been evaluated 2017-06-12 14:15:00 +02:00
Eelco Dolstra 285754aff6
hydra-evaluator improvements
* The "Jobset" page now shows when evaluations are in progress (rather
  than just pending).

* Restored the ability to do a single evaluation from the command line
  by doing "hydra-evaluator <project> <jobset>".

* Fix some consistency issues between jobset status in PostgreSQL and
  in hydra-evaluator. In particular, "lastCheckedTime" was never
  updated internally.
2017-03-15 16:59:57 +01:00
Eelco Dolstra 242ea8009f
hydra-evaluator: Don't use the "timeout" command
"timeout" does not propagate PR_SET_PDEATHSIG, so it prevents
hydra-eval-jobset from being killed when hydra-evaluator quits.
2017-03-15 16:59:56 +01:00
Eelco Dolstra 52753e49c2
hydra-evaluator: Handle SIGINT
This makes "systemctl stop hydra-evaluator" work promptly again.
2017-02-21 17:54:31 +01:00
Eelco Dolstra e2a012de8c hydra-evaluator: Set jobset error if hydra-eval-jobset dies 2016-11-08 17:08:54 +01:00
Eelco Dolstra 4576b3d35b hydra-evaluator: Show friendlier exit status 2016-11-08 17:00:17 +01:00
Eelco Dolstra e0b2921ff2 Concurrent hydra-evaluator
This rewrites the top-level loop of hydra-evaluator in C++. The Perl
stuff is moved into hydra-eval-jobset. (Rewriting the entire evaluator
would be nice but is a bit too much work.) The new version has some
advantages:

* It can run multiple jobset evaluations in parallel.

* It uses PostgreSQL notifications so it doesn't have to poll the
  database. So if a jobset is triggered via the web interface or from
  a GitHub / Bitbucket webhook, evaluation of the jobset will start
  almost instantaneously (assuming the evaluator is not at its
  concurrency limit).

* It imposes a timeout on evaluations. So if e.g. hydra-eval-jobset
  hangs connecting to a Mercurial server, it will eventually be
  killed.
2016-10-14 14:22:12 +02:00