ma27/hydra
0
0
Fork 0
forked from lix-project/hydra
Code Pull requests Activity
2440 commits 4 branches 0 tags 26 MiB
Commit graph

2440 commits

This branch
This branch
All branches
Author SHA1 Message Date
Eelco Dolstra bd76f9120a Cache .narinfo lookups 2016-02-19 16:19:40 +01:00
Eelco Dolstra a0f74047da Keep some statistics for the binary cache stores 2016-02-19 14:24:23 +01:00
Eelco Dolstra dc4a00347d Use a single BinaryCacheStore for all threads 
This will make it easier to do caching / keep stats. Also, we won't
have S3Client's connection pooling if we create multiple S3Client
instances.
 2016-02-18 17:31:19 +01:00
Eelco Dolstra 00a7be13a2 Make queue runner internal status available under /queue-runner-status 2016-02-18 17:11:46 +01:00
Eelco Dolstra 8c9fc677c1 Typo 2016-02-18 16:43:24 +01:00
Eelco Dolstra db3fcc0f5e Enable substitution on the build machines 
If properly configured, this allows them to get store paths directly
from S3, rather than having to receive them from the queue runner.
 2016-02-18 16:42:05 +01:00
Eelco Dolstra 2d40888e2e Add an S3-backed binary cache store 2016-02-18 16:18:50 +01:00
Eelco Dolstra 0e254ca66d Refactor local binary cache code into a subclass 2016-02-18 14:06:17 +01:00
Eelco Dolstra a992f688d1 Rename class 2016-02-18 13:02:20 +01:00
Eelco Dolstra de77cc2910 Rename file 2016-02-18 13:02:20 +01:00
Rob Vermaas 633381a501 Merge pull request #271 from gulrotkake/master 
Construct uri using uri_for, which respects X-Request-Base.
 2016-02-18 10:27:20 +01:00
Michael Mortensen d9a4f7a211 Construct uri using uri_for, which respects X-Request-Base. 2016-02-17 21:27:44 +01:00
Eelco Dolstra ce5790285a Merge remote-tracking branch 'origin/master' into binary-cache 2016-02-17 11:54:59 +01:00
Eelco Dolstra d7a123fcd4 Keep track of the time we spend copying to/from build machines 2016-02-17 10:30:23 +01:00
Eelco Dolstra 25022bf5fd hydra-queue-runner: Support generating a signed binary cache 2016-02-16 16:41:42 +01:00
Eelco Dolstra 744cee134e hydra-queue-runner: Compress binary cache NARs using xz 2016-02-15 21:56:53 +01:00
Eelco Dolstra 2d0dd7fb49 hydra-queue-runner: Write directly to a binary cache 2016-02-15 21:10:29 +01:00
Eelco Dolstra e46acbf05b Remove the errorMsg column from the Builds table 
The queue runner no longer uses this field, and it doesn't provide
very interesting historical data (mostly SSH failures), but it takes
up a lot of space. Also, it contained some bad UTF-8 which was
preventing an upgrade to Postgres 9.5, so a good occasion to get rid
of it.
 2016-02-12 17:21:55 +01:00
Eelco Dolstra 5c5bb3762c Fix display of builds that don't have an eval 
Like http://hydra.nixos.org/build/22872.
 2016-02-12 16:27:25 +01:00
Eelco Dolstra 92d8b59361 Process Nix API changes 2016-02-11 15:59:47 +01:00
Eelco Dolstra 993647d1e3 Use Google's verifier 2016-01-14 12:54:47 +01:00
Eelco Dolstra 5a580b1bb2 Add support for logging in via a Google account 
The required configuration in hydra.conf:

  enable_google_login = 1
  google_client_id = 238429sdjkds....apps.googleusercontent.com

and optionally persona_allowed_domains to restrict to one or more
domains.
 2016-01-13 17:32:52 +01:00
Eelco Dolstra f11ce7e219 Bump evaluation timeout to 6 hours 
This is necessary given the current size of the Nixpkgs/NixOS
jobsets. Once we have a Nix store + Postgres on SSD, we can reduce
this again.

Should really make this configurable...
 2016-01-07 16:19:54 +01:00
Eelco Dolstra 0ca6bb79af Shut up warning 2016-01-07 16:19:54 +01:00
Eelco Dolstra 97f8c61928 Fix hydra-queue-runner --build-one 2015-12-29 17:53:33 +01:00
Eelco Dolstra 29db16bc69 Fix 'Can't locate object method "buildstepoutputs"' 
Template::Toolkit braindamage strikes again. See
8f85fe67d4.
 2015-12-15 11:55:57 +01:00
Eelco Dolstra b081133dcc Move GC roots to /nix/var/nix/gcroots/hydra 
The uid split a while back caused the web interface to create GC roots
in /nix/var/nix/gcroots/per-user/hydra-www, where they wouldn't be
purged by hydra-update-gc-roots. Thus restarted builds would
accumulate forever. The fix is to keep the roots in a shared directory
with gid=hydra.
 2015-12-14 13:31:24 +01:00
Eelco Dolstra e04fb5c8e7 Sort build steps 2015-12-14 13:01:35 +01:00
Eelco Dolstra 8f7614030e Better fix for dots in jobset names 2015-11-17 11:31:11 +01:00
Eelco Dolstra ea80a75204 Move getBaseUrl 2015-11-17 11:21:20 +01:00
Rob Vermaas dddb9a281d Allow dots in job specifier of input type 'Previous build' 2015-11-17 08:36:46 +00:00
Eelco Dolstra b6d2c6f03a Further fix for the channel regression 2015-11-02 14:58:23 +01:00
aszlig 20b412224e ListBuilds: Fix legacy channel URLs. 
Regression introduced by 1fdc258de0.

The commit introduced a channel/custom PathPart which uses the new
custom channel expressions, but I forgot to remove CaptureArgs, so the
URL really is channel/latest/ignored-value.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Reported-by: Peter Simons <simons@cryp.to>
 2015-11-02 14:51:42 +01:00
Eelco Dolstra c087472c71 Remove superfluous "has" function 2015-11-02 14:29:12 +01:00
Eelco Dolstra a5879cbeb8 Consistent terminology: "Clear" -> "Cancel" 2015-11-02 14:22:25 +01:00
Eelco Dolstra e1f9a209c5 Fix cancelling non-current builds 2015-11-02 14:20:27 +01:00
Eelco Dolstra 1b1fdb3f8d Revert "Don't wrap hydra-queue-runner / hydra-eval-jobs" 
This reverts commit f8ee98f455.
 2015-10-31 15:32:19 +01:00
Eelco Dolstra 2d128d2a6f Don't show redundant "removing machine..." messages 2015-10-30 18:22:43 +01:00
Eelco Dolstra f8ee98f455 Don't wrap hydra-queue-runner / hydra-eval-jobs 2015-10-30 18:13:23 +01:00
Eelco Dolstra 53d270c007 Enable debug info 2015-10-30 18:12:30 +01:00
Eelco Dolstra d8d188301d Fix division-by-zero crash 
Not clear why step_->jobsets was empty...
 2015-10-30 18:01:48 +01:00
Eelco Dolstra 1ff48da3d3 int2String -> std::to_string 2015-10-30 18:01:38 +01:00
Eelco Dolstra 19b52e907f Add action to restart all failed builds in an eval 2015-10-28 15:04:12 +01:00
Eelco Dolstra b9ebd38edf Fix restarting builds 
Failed paths were not cleared because the $builds query becomes empty
as soon as the "finished" flag has been cleared. So this needs to be
done last.
 2015-10-28 14:54:54 +01:00
Eelco Dolstra bbe1fb2f9f Ignore non-existence of IndexBuildsOnFinishedBusy 
Apparently for some reason not all Hydra installations had this
index. Another -1 for non-declarative schema definition languages.
 2015-10-28 14:09:45 +01:00
Eelco Dolstra d22e7ac03d hydra-compress-logs: Use bzip2 -f flag 2015-10-27 16:10:02 +01:00
Eelco Dolstra 4d1816b152 Remove obsolete Builds columns and provide accurate "Running builds" 
This removes the "busy", "locker" and "logfile" columns, which are no
longer used by the queue runner. The "Running builds" page now only
shows builds that have an active build step.
 2015-10-27 15:37:17 +01:00
Eelco Dolstra 9fb91460e3 Restart jobset evals efficiently 
Fixes

  DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR: stack depth limit exceeded
  HINT: Increase the configuration parameter "max_stack_depth" (currently 2048kB), after ensuring the platform's stack depth limit is adequate. [for Statement "UPDATE Builds SET busy = ?, finished = ?, iscachedbuild = ?, locker = ? WHERE ( ( id = ? OR id = ? OR ...
 2015-10-27 13:43:19 +01:00
Eelco Dolstra 53c80d9526 getQueuedBuilds(): Periodically stop to handle priority bumps 
Previously, priority bumps could take a long time to get noticed if
getQueuedBuilds() was busy processing zillions of queue
additions. (This was made worse by the reintroduction of substitute
checking.)
 2015-10-22 17:00:46 +02:00
Eelco Dolstra 71bf7e02d5 Use nix::willBuildLocally() 2015-10-21 15:44:29 +02:00