vriska/hydra
0
0
Fork 0
forked from lix-project/hydra
Code Pull requests Activity
2228 commits 1 branch 0 tags 17 MiB
Commit graph

2228 commits

This branch
This branch
All branches
Author SHA1 Message Date
Eelco Dolstra 95c4294560 Allow cancelling builds marked as busy 
Note that if there are active build *steps*, this won't cancel them.
 2015-07-07 14:08:46 +02:00
Eelco Dolstra 35b7c4f82b Allow only 1 thread to send a closure to a given machine at the same time 
This prevents a race where multiple threads see that machine X is
missing path P, and start sending it concurrently. Nix handles this
correctly, but it's still wasteful (especially for the case where P ==
GHC).

A more refined scheme would be to have per machine, per path locks.
 2015-07-07 14:06:48 +02:00
Eelco Dolstra 16696a4aee Namespace cleanup 2015-07-07 10:29:43 +02:00
Eelco Dolstra 63745b8e25 Move buildRemote() into State 2015-07-07 10:25:33 +02:00
Eelco Dolstra df29527531 Refactor 2015-07-07 10:17:21 +02:00
Eelco Dolstra dd4f6e695e Merge branch 'master' into build-ng 2015-07-06 17:17:51 +02:00
Eelco Dolstra ccf6e6062c Store full Mercurial revision hashes 2015-07-06 17:17:17 +02:00
Eelco Dolstra 309ef5baa9 Merge branch 'master' into build-ng 2015-07-06 15:57:09 +02:00
Eelco Dolstra b85e9ef1cd Support using Git revisions as branch names 2015-07-06 15:56:24 +02:00
Eelco Dolstra b03de925cb Allow a jobset to be created from an evaluation 
Fixes #150.
 2015-07-06 15:56:20 +02:00
Eelco Dolstra 535e936732 Hackery 2015-07-02 02:55:15 +02:00
Eelco Dolstra 377d9d37f3 Fix paths 2015-07-02 02:54:59 +02:00
Eelco Dolstra 8ea05cc7f2 Don't require postgresql.service 2015-07-02 02:54:30 +02:00
Eelco Dolstra dffb629b8a Unify Hydra's NixOS module with the one used for hydra.nixos.org 
In particular, the queue runner and web server now run under different
UIDs.
 2015-07-02 01:01:44 +02:00
Eelco Dolstra 3e0f5f664a GitInput plugin: Don't clone during getCommits 
This doesn't work if hydra-queue-runner has no write access to the scm
directory, and in any case races with the evaluator.
 2015-07-02 00:44:40 +02:00
Eelco Dolstra ae52fc7f61 Remove display of queue runner log file (it no longer exists) 2015-07-02 00:18:33 +02:00
Eelco Dolstra e35b704d80 Drop the 5 minute minimum interval between triggered evals 2015-07-01 14:45:39 +02:00
Eelco Dolstra 85a1ce99c9 Only include Persona JS when Persona is enabled 2015-07-01 14:24:18 +02:00
Eelco Dolstra 1e87b15f91 Remove Nix options not needed by the new queue runner 2015-07-01 11:40:00 +02:00
Eelco Dolstra 3c665dac82 Remove superfluous HYDRA_LOGO environment variable 2015-07-01 11:34:19 +02:00
Eelco Dolstra 60e8b9881b The new queue runner requires "hydra" to be a trusted Nix user 
(Because it does unsigned imports into the store.)
 2015-07-01 10:47:43 +02:00
Eelco Dolstra 7e6135a8c6 Don't repeat links to build step logs 
Hydra only stores the last log for a particular derivation, so only
show log links for the last one.
 2015-06-30 00:27:31 +02:00
Eelco Dolstra 2ece42b2b9 Support preferLocalBuild 
Derivations with "preferLocalBuild = true" can now be executed on
specific machines (typically localhost) by setting the mandary system
features field to include "local". For example:

  localhost x86_64-linux,i686-linux - 10 100 - local

says that "localhost" can *only* do builds with "preferLocalBuild =
true". The speed factor of 100 will make the machine almost always win
over other machines.
 2015-06-30 00:20:19 +02:00
Eelco Dolstra 171303864e Fix type 2015-06-29 09:29:04 +00:00
Eelco Dolstra 008d610467 getQueuedBuilds(): Don't catch errors while loading a build from the queue 
Otherwise we never recover from reset daemon connections, e.g.

  hydra-queue-runner[16106]: while loading build 599369: cannot start daemon worker: reading from file: Connection reset by peer
  hydra-queue-runner[16106]: while loading build 599236: writing to file: Broken pipe
  ...

The error is now handled queueMonitor(), causing the next call to
queueMonitorLoop() to create a new connection.
 2015-06-26 21:06:35 +02:00
Eelco Dolstra f5e5a1b96e Don't wake up the queue runner for cached evals 2015-06-26 20:59:14 +02:00
Eelco Dolstra 401f5bdce2 Add a unit for hydra-send-stats 2015-06-26 15:24:12 +02:00
Eelco Dolstra 17924ce012 Note in the log when the queue runner / evaluator gets killed 2015-06-26 11:56:11 +02:00
Eelco Dolstra aabe514d07 Update the hydra-queue-runner unit 2015-06-26 11:29:30 +02:00
Eelco Dolstra 73a78c2073 Give services.hydra.package a reasonable default 2015-06-26 11:29:19 +02:00
Eelco Dolstra 9a041f9a36 Restart builds failed due to unsupported system type 2015-06-26 11:28:38 +02:00
Eelco Dolstra 2f4676bd97 JSONObject doesn't handle 64-bit integers 2015-06-25 16:59:48 +02:00
Eelco Dolstra c54a04688e Fix email sender address when notification_sender is not set 2015-06-25 16:49:01 +02:00
Eelco Dolstra c6fcce3b3b Moar stats 2015-06-25 16:47:39 +02:00
Eelco Dolstra b5815e2aa6 Run PostgreSQL during "make check" 2015-06-25 16:47:39 +02:00
Eelco Dolstra 18a3c3ff1c Update "make check" for the new queue runner 
Also, if the machines file contains an entry for localhost, then run
"nix-store --serve" directly, without going through SSH.
 2015-06-25 16:47:39 +02:00
Eelco Dolstra 32210905d8 Automatically reload $NIX_REMOTE_SYSTEMS when it changes 
Otherwise, you'd have to restart the queue runner to add or remove
machines.
 2015-06-25 16:47:25 +02:00
Eelco Dolstra 1a0e1eb5a0 More stats 2015-06-24 13:19:27 +02:00
Eelco Dolstra 3f8891b6ff Fix incorrect debug message 2015-06-23 17:53:15 +02:00
Eelco Dolstra 62219adaf3 Send queue runner stats to statsd 
This is currently done by a separate program that periodically
calls "hydra-queue-runner --status". Eventually, I'll do this
in the queue runner directly.

Fixes #220.
 2015-06-23 14:56:43 +02:00
Eelco Dolstra af5cbe97aa createStep(): Cache finished derivations 
This gets rid of a lot of redundant calls to readDerivation().
 2015-06-23 03:25:31 +02:00
Eelco Dolstra 681f63a382 Typo 2015-06-23 02:15:11 +02:00
Eelco Dolstra 524ee295e0 Fix sending notifications in the successful case 2015-06-23 02:13:06 +02:00
Eelco Dolstra 4db7c51b5c Rate-limit the number of threads copying closures at the same time 
Having a hundred threads doing I/O at the same time is bad on magnetic
disks because of the excessive disk seeks. So allow only 4 threads to
copy closures in parallel.
 2015-06-23 01:49:14 +02:00
Eelco Dolstra a317d24b29 hydra-queue-runner: Send build notifications 
Since our notification plugins are written in Perl, sending
notification from C++ requires a small Perl helper named
‘hydra-notify’.
 2015-06-23 00:14:49 +02:00
Eelco Dolstra 5312e1209b Keep per-machine stats 2015-06-22 17:11:17 +02:00
Eelco Dolstra d06366e7cf Remove obsolete comment 2015-06-22 16:59:50 +02:00
Eelco Dolstra e069ee960e Doh 2015-06-22 16:58:40 +02:00
Eelco Dolstra a757b783f4 Update $PATH 2015-06-22 16:15:16 +02:00
Eelco Dolstra e32ee3d5b9 Remove hydra-build and the old hydra-queue-runner 2015-06-22 15:43:15 +02:00