Commit graph

2404 commits

Author SHA1 Message Date
Eelco Dolstra
4d26546d3c Add support for tracking custom metrics
Builds can now emit metrics that Hydra will store in its database and
render as time series via flot charts. Typical applications are to
keep track of performance indicators, coverage percentages, artifact
sizes, and so on.

For example, a coverage build can emit the coverage percentage as
follows:

  echo "lineCoverage $pct %" > $out/nix-support/hydra-metrics

Graphs of all metrics for a job can be seen at

  http://.../job/<project>/<jobset>/<job>#tabs-charts

Specific metrics are also visible at

  http://.../job/<project>/<jobset>/<job>/metric/<metric>

The latter URL also allows getting the data in JSON format (e.g. via
"curl -H 'Accept: application/json'").
2015-07-31 00:57:30 +02:00
Eelco Dolstra
8092149a9f Use nixUnstable 2015-07-30 16:55:11 +02:00
Eelco Dolstra
9afaf322b3 Use latest DBIx::Class::Schema::Loader 2015-07-30 16:55:11 +02:00
Eelco Dolstra
fed1cc88c0 Merge pull request #225 from aszlig/fix-scmdiff-url
common.tt: Properly generate URL for /api/scmdiff.
2015-07-22 13:00:21 +02:00
aszlig
36cbdd1bd8
common.tt: Properly generate URL for /api/scmdiff.
If Hydra isn't hosted on https://example.com/ but something like
https://example.com/hydra/, the URL for /api/scmdiff would have ended up
on /api/scmdiff rather than /hydra/api/scmdiff.

This is because we didn't use the URI resolver from the controller,
hence we're using it now to build up the whole URL including the query
string.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-07-22 12:00:15 +02:00
Eelco Dolstra
c18fb0ad74 Temporarily disable machines after a connection failure 2015-07-21 15:58:47 +02:00
Eelco Dolstra
7e026d35f7 Split hydra-queue-runner.cc more 2015-07-21 15:14:17 +02:00
Eelco Dolstra
6ddcd37df1 hydra-queue-runner.service: Set IN_SYSTEMD 2015-07-21 01:54:24 +02:00
Eelco Dolstra
5370be9f52 hydra-queue-runner: Use cmdBuildDerivation
See 1511aa9f48 and eda2f36c2a.
2015-07-21 01:54:24 +02:00
Eelco Dolstra
62b1b095ad Handle IP addresses as machine names 2015-07-21 01:54:24 +02:00
Rob Vermaas
fbd48f5684 Make cloning of jobsets work again.
(cherry picked from commit 706551453800fb3a4bb3d8b681dad157a8cea98b)
2015-07-14 09:07:10 +00:00
Eelco Dolstra
3ded87329d Keep track of how many threads are waiting 2015-07-10 19:10:14 +02:00
Eelco Dolstra
06d75699a7 Fix restarting a build 2015-07-10 16:56:53 +02:00
Eelco Dolstra
7f865a30d5 hydra-evaluator: Fix input change check
Because inputs were processed in random order by inputsToArgs, the
inputs hash could be different every time, leading to unnecessary
re-evaluations.
2015-07-10 16:44:06 +02:00
Eelco Dolstra
3e7bbec40b hydra-evaluator: Send statistics to statsd 2015-07-10 16:40:50 +02:00
Eelco Dolstra
5919e911db Don't show how long a machine has been idle
Without an index on (machine, stoptime desc), this requires a
sequential scan. And adding a whole index for this seems
overkill. (Possibly the queue runner could maintain this info more
efficiently.)
2015-07-10 15:41:57 +02:00
Eelco Dolstra
3bb9e17e5c IndexJobsetEvalsOnJobsetId: Only index jobsets with new builds 2015-07-10 15:41:57 +02:00
Eelco Dolstra
b09f7e0989 Add page showing latest build steps 2015-07-10 15:41:57 +02:00
Shea Levy
fc16c53886 Merge branch 'no-local-psql' 2015-07-10 09:09:34 -04:00
Eelco Dolstra
0da08df4eb Stream logs if possible and remove size limit 2015-07-08 19:05:17 +02:00
Eelco Dolstra
f5548dc225 Fix Persona logins 2015-07-08 16:57:33 +02:00
Eelco Dolstra
f933762358 Disable building the PDF manual 2015-07-08 16:51:05 +02:00
Eelco Dolstra
296213e90a Drop obsolete graphviz dependency 2015-07-08 16:30:58 +02:00
Eelco Dolstra
d8acaf2181 Index BuildSteps on propagatedFrom
This significantly speeds up deleting Builds, since it removes the
need for a sequential scan on BuildSteps.
2015-07-08 12:12:44 +02:00
Eelco Dolstra
bbee81efae Use triggers for all notifications on Builds table changes 2015-07-08 12:05:32 +02:00
Eelco Dolstra
89fb723ace Notify the queue runner when a build is deleted 2015-07-08 11:43:35 +02:00
Eelco Dolstra
95c4294560 Allow cancelling builds marked as busy
Note that if there are active build *steps*, this won't cancel them.
2015-07-07 14:08:46 +02:00
Eelco Dolstra
35b7c4f82b Allow only 1 thread to send a closure to a given machine at the same time
This prevents a race where multiple threads see that machine X is
missing path P, and start sending it concurrently. Nix handles this
correctly, but it's still wasteful (especially for the case where P ==
GHC).

A more refined scheme would be to have per machine, per path locks.
2015-07-07 14:06:48 +02:00
Eelco Dolstra
16696a4aee Namespace cleanup 2015-07-07 10:29:43 +02:00
Eelco Dolstra
63745b8e25 Move buildRemote() into State 2015-07-07 10:25:33 +02:00
Eelco Dolstra
df29527531 Refactor 2015-07-07 10:17:21 +02:00
Eelco Dolstra
dd4f6e695e Merge branch 'master' into build-ng 2015-07-06 17:17:51 +02:00
Eelco Dolstra
ccf6e6062c Store full Mercurial revision hashes 2015-07-06 17:17:17 +02:00
Eelco Dolstra
309ef5baa9 Merge branch 'master' into build-ng 2015-07-06 15:57:09 +02:00
Eelco Dolstra
b85e9ef1cd Support using Git revisions as branch names 2015-07-06 15:56:24 +02:00
Eelco Dolstra
b03de925cb Allow a jobset to be created from an evaluation
Fixes #150.
2015-07-06 15:56:20 +02:00
Eelco Dolstra
535e936732 Hackery 2015-07-02 02:55:15 +02:00
Eelco Dolstra
377d9d37f3 Fix paths 2015-07-02 02:54:59 +02:00
Eelco Dolstra
8ea05cc7f2 Don't require postgresql.service 2015-07-02 02:54:30 +02:00
Eelco Dolstra
dffb629b8a Unify Hydra's NixOS module with the one used for hydra.nixos.org
In particular, the queue runner and web server now run under different
UIDs.
2015-07-02 01:01:44 +02:00
Eelco Dolstra
3e0f5f664a GitInput plugin: Don't clone during getCommits
This doesn't work if hydra-queue-runner has no write access to the scm
directory, and in any case races with the evaluator.
2015-07-02 00:44:40 +02:00
Eelco Dolstra
ae52fc7f61 Remove display of queue runner log file (it no longer exists) 2015-07-02 00:18:33 +02:00
Eelco Dolstra
e35b704d80 Drop the 5 minute minimum interval between triggered evals 2015-07-01 14:45:39 +02:00
Eelco Dolstra
85a1ce99c9 Only include Persona JS when Persona is enabled 2015-07-01 14:24:18 +02:00
Eelco Dolstra
1e87b15f91 Remove Nix options not needed by the new queue runner 2015-07-01 11:40:00 +02:00
Eelco Dolstra
3c665dac82 Remove superfluous HYDRA_LOGO environment variable 2015-07-01 11:34:19 +02:00
Eelco Dolstra
60e8b9881b The new queue runner requires "hydra" to be a trusted Nix user
(Because it does unsigned imports into the store.)
2015-07-01 10:47:43 +02:00
Eelco Dolstra
7e6135a8c6 Don't repeat links to build step logs
Hydra only stores the last log for a particular derivation, so only
show log links for the last one.
2015-06-30 00:27:31 +02:00
Eelco Dolstra
2ece42b2b9 Support preferLocalBuild
Derivations with "preferLocalBuild = true" can now be executed on
specific machines (typically localhost) by setting the mandary system
features field to include "local". For example:

  localhost x86_64-linux,i686-linux - 10 100 - local

says that "localhost" can *only* do builds with "preferLocalBuild =
true". The speed factor of 100 will make the machine almost always win
over other machines.
2015-06-30 00:20:19 +02:00
Eelco Dolstra
171303864e Fix type 2015-06-29 09:29:04 +00:00