Eelco Dolstra
b9afaadfb3
Keep better bytesReceived/bytesSent stats
2016-02-26 16:17:05 +01:00
Eelco Dolstra
6d741d2ffa
Prevent download of NARs we just uploaded
2016-02-26 15:21:44 +01:00
Eelco Dolstra
db3fcc0f5e
Enable substitution on the build machines
...
If properly configured, this allows them to get store paths directly
from S3, rather than having to receive them from the queue runner.
2016-02-18 16:42:05 +01:00
Eelco Dolstra
ce5790285a
Merge remote-tracking branch 'origin/master' into binary-cache
2016-02-17 11:54:59 +01:00
Eelco Dolstra
d7a123fcd4
Keep track of the time we spend copying to/from build machines
2016-02-17 10:30:23 +01:00
Eelco Dolstra
2d0dd7fb49
hydra-queue-runner: Write directly to a binary cache
2016-02-15 21:10:29 +01:00
Eelco Dolstra
92d8b59361
Process Nix API changes
2016-02-11 15:59:47 +01:00
Eelco Dolstra
8e8e31ce86
Re-implement log size limits
...
The old queue runner already had this. However, we now store "log
limit exceeded" as a separate status code in the database.
2015-10-06 17:35:08 +02:00
Eelco Dolstra
6075ac6fed
Remove localhost hack
2015-09-09 16:50:59 +02:00
Eelco Dolstra
2a7fbd57cc
Allow the machines file to specify host public keys
...
It's easier for the Hydra provisioner to put host public keys in the
machines file than to separately manage the known_hosts file
(especially when the provisioner runs on a different machine).
2015-08-26 13:43:02 +02:00
Eelco Dolstra
ff3f5eb4d8
Fix remote building on Nix 1.10
2015-07-31 03:41:55 +02:00
Eelco Dolstra
5b9a288123
Workaround for RemoteStore not supporting cmdBuildDerivation yet
2015-07-31 03:39:20 +02:00
Eelco Dolstra
c18fb0ad74
Temporarily disable machines after a connection failure
2015-07-21 15:58:47 +02:00
Eelco Dolstra
5370be9f52
hydra-queue-runner: Use cmdBuildDerivation
...
See 1511aa9f48eda2f36c2a
2015-07-21 01:54:24 +02:00
Eelco Dolstra
3ded87329d
Keep track of how many threads are waiting
2015-07-10 19:10:14 +02:00
Eelco Dolstra
35b7c4f82b
Allow only 1 thread to send a closure to a given machine at the same time
...
This prevents a race where multiple threads see that machine X is
missing path P, and start sending it concurrently. Nix handles this
correctly, but it's still wasteful (especially for the case where P ==
GHC).
A more refined scheme would be to have per machine, per path locks.
2015-07-07 14:06:48 +02:00
Eelco Dolstra
63745b8e25
Move buildRemote() into State
2015-07-07 10:25:33 +02:00
Eelco Dolstra
c6fcce3b3b
Moar stats
2015-06-25 16:47:39 +02:00
Eelco Dolstra
18a3c3ff1c
Update "make check" for the new queue runner
...
Also, if the machines file contains an entry for localhost, then run
"nix-store --serve" directly, without going through SSH.
2015-06-25 16:47:39 +02:00
Eelco Dolstra
1a0e1eb5a0
More stats
2015-06-24 13:19:27 +02:00
Eelco Dolstra
681f63a382
Typo
2015-06-23 02:15:11 +02:00
Eelco Dolstra
4db7c51b5c
Rate-limit the number of threads copying closures at the same time
...
Having a hundred threads doing I/O at the same time is bad on magnetic
disks because of the excessive disk seeks. So allow only 4 threads to
copy closures in parallel.
2015-06-23 01:49:14 +02:00
Eelco Dolstra
44a2b74f5a
Keep track of the number of build steps that are being built
...
(As opposed to being in the closure copying stage.)
2015-06-22 11:23:00 +02:00
Eelco Dolstra
133d298e26
Asynchronously compress build logs
2015-06-19 15:06:12 +02:00
Eelco Dolstra
3855131185
hydra-queue-runner: Improve SSH flags
2015-06-18 00:50:48 +02:00
Eelco Dolstra
f57d0b0c54
hydra-queue-runner: Maintain count of active build steps
2015-06-18 00:24:56 +02:00
Eelco Dolstra
ca48818b30
Fix remote building
2015-06-17 17:28:59 +02:00
Eelco Dolstra
745efce828
hydra-queue-runner: Implement timeouts
...
Also, keep track of timeouts in the database as a distinct build
status.
2015-06-17 13:32:33 +02:00
Eelco Dolstra
b91a616520
Automatically retry aborted builds
...
Aborted builds are now put back on the runnable queue and retried
after a certain time interval (currently 60 seconds for the first
retry, then tripled on each subsequent retry).
2015-06-17 11:45:20 +02:00
Eelco Dolstra
c68036f8b0
Pass ssh key
2015-06-10 14:57:07 +02:00
Eelco Dolstra
7dd1f0097e
Finish copyClosure
2015-06-09 16:03:41 +02:00
Eelco Dolstra
8b12ac1f6d
Basic remote building
...
This removes the need for Nix's build-remote.pl.
Build logs are now written to $HYDRA_DATA/build-logs because
hydra-queue-runner doesn't have write permission to /nix/var/log.
2015-06-09 14:21:21 +02:00
