Eelco Dolstra
610a8d67ae
Better AWS error messages
2016-02-26 22:40:27 +01:00
Eelco Dolstra
1693354506
Remove unnecessary call to hydra-queue-runner --unlock
2016-02-26 21:45:59 +01:00
Eelco Dolstra
1a055e7e9e
Reduce severity level of some message
2016-02-26 21:31:08 +01:00
Eelco Dolstra
6bb860fd6e
Add FIXME
2016-02-26 21:15:05 +01:00
Eelco Dolstra
e8cdfe5171
hydra-server: Don't barf if the binary cache public key can't be read
2016-02-26 21:14:40 +01:00
Eelco Dolstra
53ca41ef9f
Use US standard S3 region
2016-02-26 20:57:47 +01:00
Eelco Dolstra
c635f5d0ea
Fix Makefile.am
2016-02-26 19:54:55 +01:00
Eelco Dolstra
b1ce76c2b4
Fix test
...
nix-support/failed is supposed to be a file, not a directory.
2016-02-26 19:54:32 +01:00
Eelco Dolstra
07e5fc5618
Hackery to make downloads work when using a binary cache
2016-02-26 17:28:26 +01:00
Eelco Dolstra
b00bdefa98
Fix hydra-server signing
2016-02-26 17:28:16 +01:00
Eelco Dolstra
9de336de7c
Proxy local binary caches via hydra-server
2016-02-26 17:27:30 +01:00
Eelco Dolstra
b9afaadfb3
Keep better bytesReceived/bytesSent stats
2016-02-26 16:17:05 +01:00
Eelco Dolstra
6d741d2ffa
Prevent download of NARs we just uploaded
2016-02-26 15:21:44 +01:00
Eelco Dolstra
02190b0fef
Support hydra-build-products on binary cache stores
2016-02-26 14:45:03 +01:00
Eelco Dolstra
8e24ad6f0d
Sync with Nix
2016-02-25 10:58:31 +01:00
Eelco Dolstra
8321a3eb27
Sync with Nix
2016-02-24 14:04:31 +01:00
Eelco Dolstra
7b509237cd
Bleh Automake
2016-02-22 18:05:15 +01:00
Eelco Dolstra
6c3ae36648
hydra-queue-runner: Get store mode configuration from hydra.conf
...
To use the local Nix store (default):
store_mode = direct
To use a local binary cache:
store_mode = local-binary-cache
binary_cache_dir = /var/lib/hydra/binary-cache
To use an S3 bucket:
store_mode = s3-binary-cache
binary_cache_s3_bucket = my-nix-bucket
Also, respect binary_cache_{secret,public}_key_file for signing the
binary cache.
2016-02-22 17:23:06 +01:00
Eelco Dolstra
94817d77d9
BinaryCacheStore: Respect build-use-substitutes
2016-02-22 17:21:39 +01:00
Eelco Dolstra
5668aa5f71
After uploading a .narinfo, add it to the LRU cache
2016-02-20 10:35:16 +01:00
Eelco Dolstra
88a05763cc
Pool local store connections
2016-02-20 00:04:08 +01:00
Eelco Dolstra
1cefd6cac8
Fix log message
2016-02-20 00:02:37 +01:00
Eelco Dolstra
a593ebc58e
Add missing file
2016-02-19 20:58:40 +01:00
Eelco Dolstra
2b76094a23
S3BinaryCacheStore::isValidPath(): Do a GET instead of HEAD
2016-02-19 17:41:11 +01:00
Eelco Dolstra
bd76f9120a
Cache .narinfo lookups
2016-02-19 16:19:40 +01:00
Eelco Dolstra
a0f74047da
Keep some statistics for the binary cache stores
2016-02-19 14:24:23 +01:00
Eelco Dolstra
dc4a00347d
Use a single BinaryCacheStore for all threads
...
This will make it easier to do caching / keep stats. Also, we won't
have S3Client's connection pooling if we create multiple S3Client
instances.
2016-02-18 17:31:19 +01:00
Eelco Dolstra
00a7be13a2
Make queue runner internal status available under /queue-runner-status
2016-02-18 17:11:46 +01:00
Eelco Dolstra
8c9fc677c1
Typo
2016-02-18 16:43:24 +01:00
Eelco Dolstra
db3fcc0f5e
Enable substitution on the build machines
...
If properly configured, this allows them to get store paths directly
from S3, rather than having to receive them from the queue runner.
2016-02-18 16:42:05 +01:00
Eelco Dolstra
2d40888e2e
Add an S3-backed binary cache store
2016-02-18 16:18:50 +01:00
Eelco Dolstra
0e254ca66d
Refactor local binary cache code into a subclass
2016-02-18 14:06:17 +01:00
Eelco Dolstra
a992f688d1
Rename class
2016-02-18 13:02:20 +01:00
Eelco Dolstra
de77cc2910
Rename file
2016-02-18 13:02:20 +01:00
Rob Vermaas
633381a501
Merge pull request #271 from gulrotkake/master
...
Construct uri using uri_for, which respects X-Request-Base.
2016-02-18 10:27:20 +01:00
Michael Mortensen
d9a4f7a211
Construct uri using uri_for, which respects X-Request-Base.
2016-02-17 21:27:44 +01:00
Eelco Dolstra
ce5790285a
Merge remote-tracking branch 'origin/master' into binary-cache
2016-02-17 11:54:59 +01:00
Eelco Dolstra
d7a123fcd4
Keep track of the time we spend copying to/from build machines
2016-02-17 10:30:23 +01:00
Eelco Dolstra
25022bf5fd
hydra-queue-runner: Support generating a signed binary cache
2016-02-16 16:41:42 +01:00
Eelco Dolstra
744cee134e
hydra-queue-runner: Compress binary cache NARs using xz
2016-02-15 21:56:53 +01:00
Eelco Dolstra
2d0dd7fb49
hydra-queue-runner: Write directly to a binary cache
2016-02-15 21:10:29 +01:00
Eelco Dolstra
e46acbf05b
Remove the errorMsg column from the Builds table
...
The queue runner no longer uses this field, and it doesn't provide
very interesting historical data (mostly SSH failures), but it takes
up a lot of space. Also, it contained some bad UTF-8 which was
preventing an upgrade to Postgres 9.5, so a good occasion to get rid
of it.
2016-02-12 17:21:55 +01:00
Eelco Dolstra
5c5bb3762c
Fix display of builds that don't have an eval
...
Like http://hydra.nixos.org/build/22872 .
2016-02-12 16:27:25 +01:00
Eelco Dolstra
92d8b59361
Process Nix API changes
2016-02-11 15:59:47 +01:00
Eelco Dolstra
993647d1e3
Use Google's verifier
2016-01-14 12:54:47 +01:00
Eelco Dolstra
5a580b1bb2
Add support for logging in via a Google account
...
The required configuration in hydra.conf:
enable_google_login = 1
google_client_id = 238429sdjkds....apps.googleusercontent.com
and optionally persona_allowed_domains to restrict to one or more
domains.
2016-01-13 17:32:52 +01:00
Eelco Dolstra
f11ce7e219
Bump evaluation timeout to 6 hours
...
This is necessary given the current size of the Nixpkgs/NixOS
jobsets. Once we have a Nix store + Postgres on SSD, we can reduce
this again.
Should really make this configurable...
2016-01-07 16:19:54 +01:00
Eelco Dolstra
0ca6bb79af
Shut up warning
2016-01-07 16:19:54 +01:00
Eelco Dolstra
97f8c61928
Fix hydra-queue-runner --build-one
2015-12-29 17:53:33 +01:00
Eelco Dolstra
29db16bc69
Fix 'Can't locate object method "buildstepoutputs"'
...
Template::Toolkit braindamage strikes again. See
8f85fe67d4
2015-12-15 11:55:57 +01:00
