ma27/hydra
0
0
Fork 0
forked from lix-project/hydra
Code Pull requests Activity
2052 commits 4 branches 0 tags 26 MiB
Commit graph

300 commits

Author SHA1 Message Date
Eelco Dolstra 2b9bf7526d hydra-send-stats: Keep track of the number of machines 2015-08-27 12:05:06 +02:00
Eelco Dolstra a6e3cb53b9 Use /usr/bin/env to find perl 
This is nicer in nix-shell.
 2015-08-17 14:18:20 +02:00
Eelco Dolstra 3ded87329d Keep track of how many threads are waiting 2015-07-10 19:10:14 +02:00
Eelco Dolstra 3e7bbec40b hydra-evaluator: Send statistics to statsd 2015-07-10 16:40:50 +02:00
Eelco Dolstra bbee81efae Use triggers for all notifications on Builds table changes 2015-07-08 12:05:32 +02:00
Eelco Dolstra e35b704d80 Drop the 5 minute minimum interval between triggered evals 2015-07-01 14:45:39 +02:00
Eelco Dolstra f5e5a1b96e Don't wake up the queue runner for cached evals 2015-06-26 20:59:14 +02:00
Eelco Dolstra 401f5bdce2 Add a unit for hydra-send-stats 2015-06-26 15:24:12 +02:00
Eelco Dolstra c6fcce3b3b Moar stats 2015-06-25 16:47:39 +02:00
Eelco Dolstra 1a0e1eb5a0 More stats 2015-06-24 13:19:27 +02:00
Eelco Dolstra 62219adaf3 Send queue runner stats to statsd 
This is currently done by a separate program that periodically
calls "hydra-queue-runner --status". Eventually, I'll do this
in the queue runner directly.

Fixes #220.
 2015-06-23 14:56:43 +02:00
Eelco Dolstra a317d24b29 hydra-queue-runner: Send build notifications 
Since our notification plugins are written in Perl, sending
notification from C++ requires a small Perl helper named
‘hydra-notify’.
 2015-06-23 00:14:49 +02:00
Eelco Dolstra e32ee3d5b9 Remove hydra-build and the old hydra-queue-runner 2015-06-22 15:43:15 +02:00
Eelco Dolstra a984c0badc Merge branch 'master' into build-ng 2015-06-15 18:21:07 +02:00
Eelco Dolstra c08883966c Use PostgreSQL notifications for queue events 
Hydra-queue-runner now no longer polls the queue periodically, but
instead sleeps until it receives a notification from PostgreSQL about
a change to the queue (build added, build cancelled or build
restarted).

Also, for the "build added" case, we now only check for builds with an
ID greater than the previous greatest ID. This is much more efficient
if the queue is large.
 2015-06-11 17:41:59 +02:00
Eelco Dolstra 672bbb1c67 hydra-update-gc-roots: Get Builds and BuildOutputs in the same query 
This greatly reduces the number of roundtrips to the database.
 2015-06-11 14:10:21 +02:00
Eelco Dolstra 9989a90e99 Keep the most recent successful build of current jobs 
Fixes #140.
 2015-06-11 14:10:21 +02:00
Eelco Dolstra e003665146 Split timeSpent query into 2 separate queries, as postgresql isn't able to figure out a decent query plan. With 120k jobs in queue, this makes some queries go from 100s to 1-2s. 2015-05-01 12:28:29 +02:00
Eelco Dolstra 01cd6397cb Better error message 2015-04-14 15:16:24 +02:00
Eelco Dolstra 63306aaf5a hydra-evaluator: Add some debug code 2015-04-09 17:35:04 +02:00
Eelco Dolstra 9e664cf8b0 Fix not-null constraint violation inserting build step 
DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR:  null value in column "machine" violates not-null constraint
 2015-04-07 13:34:00 +02:00
Eelco Dolstra ad2b7646ac Don't show missing paths in logs 2015-03-04 15:44:04 +01:00
Eelco Dolstra adc72d2409 Record which build a failed build step was propagated from 2015-02-25 16:42:32 +01:00
Eelco Dolstra 48af914e28 When propagating failure, propagate the duration and machine 
Previously the duration would just show as "0" and you would have to
search for the original build to get the duration.
 2015-02-25 16:21:54 +01:00
Eelco Dolstra a12135fc51 Don't use Perl's -w flag 2014-12-12 17:39:52 +01:00
Eelco Dolstra 6a0b9a3476 hydra-build: Handle new trace messages 
Now build step duration no longer includes network overhead (i.e.
time to copy closures to the build machine).
 2014-12-12 17:20:50 +01:00
Eelco Dolstra c0ca5489e1 Don't use given/when 
These give warnings in Perl >= 5.18:

  given is experimental at /home/hydra/src/hydra/src/lib/Hydra/Helper/CatalystUtils.pm line 241.
  when is experimental at /home/hydra/src/hydra/src/lib/Hydra/Helper/CatalystUtils.pm line 242.
  ...
 2014-12-12 11:27:17 +01:00
Eelco Dolstra 8523130ebb Use Email::MIME instead of Email::Simple 
Email::Simple cannot handle non-ASCII characters.

Fixes #191.
 2014-11-19 14:45:46 +01:00
Eelco Dolstra 5a7efc0469 Fix aggregate handling 2014-10-01 15:34:05 +02:00
Eelco Dolstra 9b38b5f134 Remove the longDescription field 
It's not useful and takes up a lot of space.
 2014-09-30 15:44:09 +02:00
Eelco Dolstra 5b4de2dee6 hydra-evaluator: Reduce verbosity 2014-09-30 15:44:08 +02:00
Eelco Dolstra 09a96c642a hydra-eval-jobs: Use JSON instead of XML 
XML::Simple is pretty slow - reading the output for the Nixpkgs jobset
takes half a minute or so. JSON is pretty much instantaneous.
 2014-09-30 15:44:08 +02:00
Eelco Dolstra 1c20cfdf24 Drop the errorMsg column in the Jobs table 
We're not using it anywhere.
 2014-09-29 19:46:11 +02:00
aszlig 9c7f303255 Use mktemp for tempdir creation in prefetchers. 
This incorporates the following two commits from <nixpkgs>:

NixOS/nixpkgs@f83af95f8a
NixOS/nixpkgs@5e7a1cf955

Hydra was the original reason why I was fixing tempdir creation in the
first place. Seeing that Hydra ships its own versions of these scripts,
we need to patch them here as well.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
 2014-08-29 12:40:11 +02:00
Eelco Dolstra a80bfceaca Remove timeout detection hack 2014-08-17 19:26:03 +02:00
Eelco Dolstra 69e3aa0438 Write Hydra roots as regular files instead of symlinks 
Note that this requires at least NixOS/Nix@1c208f2b7e.
 2014-08-01 17:24:55 +02:00
Eelco Dolstra 365de86ead Fix hydra-update-gc-roots 2014-07-16 23:20:58 +02:00
Eelco Dolstra fb5f01097b Fix race between hydra-eval-jobs and hydra-update-gc-roots 
If hydra-eval-jobs creates a new root, and hydra-update-gc-roots runs
before hydra-evaluator has had a chance to add the corresponding build
to the database, then hydra-update-gc-roots will remove the root. If
subsequently the Nix garbage collector kicks in, it may remove the
build's .drv file before the build is performed. Since evaluation of
the Nixpkgs and NixOS jobsets nowadays takes a lot of time (e.g. an
hour), the probability of this happening is fairly high.

The quick fix is not to delete roots that are less than a day old. So
long as evaluation doesn't take longer than a day, this should be fine
;-)

Fixes #166.
 2014-07-14 13:18:07 +02:00
Ludovic Courtès 4471cae07e hydra-eval-guile-jobs: Add "This file is part of Hydra". 2014-04-08 18:22:21 +02:00
Ludovic Courtès cc46456598 hydra-eval-guile-jobs: Register derivations as GC roots. 
* src/script/hydra-eval-guile-jobs.in (register-gc-root): New
  procedure.
  (job-evaluations->sxml): Add #:gc-roots-dir parameter.  Call
  'register-gc-root'.
  (job-evaluations->xml): Add #:gc-roots-dir parameter; pass it to
  'job-evaluations->sxml'.
  (eval-guile-jobs): Warn when --gc-roots-dir isn't passed.  Pass
  GC-ROOTS-DIR to 'job-evaluations->xml'.
 2014-04-08 18:22:21 +02:00
Ludovic Courtès e3e8c1bc74 hydra-eval-guile-jobs: Honor $NIX_STORE_DIR. 
* src/script/hydra-eval-guile-jobs.in (strip-store-path): Honor
  $NIX_STORE_DIR.
 2014-04-08 18:22:21 +02:00
Eelco Dolstra 4cf0d7c4b3 hydra-update-gc-roots: Keep derivations of failed builds 
By keeping the derivations of failed builds in the most recent
evaluations, we ensure that failed builds can be restarted.
 2014-04-08 17:54:11 +02:00
Ludovic Courtès 61448ca2bd guile: Raise default absolute timeout to from 2h to 20h. 2014-02-27 22:19:52 +01:00
Shea Levy a92a57f3b0 hydra-evaluator: When evaluating a single jobset, exit with a failure code if evaluation fails 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-11-27 14:29:05 -05:00
Eelco Dolstra 55f9d23933 Add a command `hydra-create-user' for managing user accounts 2013-11-06 13:36:29 +01:00
Eelco Dolstra 3315d1ea51 Remove obsolete hydra-control script 2013-11-06 11:42:49 +00:00
Eelco Dolstra 7a18e5f0c1 Remove unused file 2013-11-06 11:42:04 +00:00
Eelco Dolstra ecadcef642 Prevent a division by zero in hydra-queue-runner 
Fixes #131.
 2013-11-06 12:15:11 +01:00
Eelco Dolstra 647d6bc3f1 Force creation of a new uncached eval if jobs have been removed 
Previously we only checked if jobs had been added.  We should probably
rename the "hasNewBuilds" field in the database.
 2013-11-01 19:24:52 +01:00
Shea Levy 0db950931a Merge remote-tracking branch 'upstream/who-broke-builds' into upstream-master 
Include information about who changed the build status in notification
emails, and enable optional per-input notification of said committers.

Conflicts due to two branches modifying the database schema.

Signed-off-by: Shea Levy <shea@shealevy.com>

Conflicts:
	src/lib/Hydra/Schema/Jobsets.pm
	src/sql/upgrade-23.sql
 2013-10-15 09:49:20 -04:00