lix-project/hydra
3
0
Fork 6
Code Issues 2 Pull requests Packages Projects Releases Wiki Activity
2185 commits 3 branches 0 tags 20 MiB
Commit graph

166 commits

Author SHA1 Message Date
Eelco Dolstra 2ab8e9a1e0 hydra-queue-runner: Fix handling of missing derivations 
This barfed with 'queue monitor: ERROR: column "errormsg" of relation
"builds" does not exist' due to the removal of the errorMsg column.
 2016-03-07 19:05:24 +01:00
Eelco Dolstra e7ce225558 Fix build 2016-03-04 17:51:32 +01:00
Eelco Dolstra 86a2d6471c Fix a boost format string abort 2016-03-02 20:06:48 +01:00
Eelco Dolstra 232ca8fea2 Fix build 2016-03-02 17:05:07 +01:00
Eelco Dolstra b98a061c24 Add some instrumentation to keep track of dispatcher cost 2016-03-02 14:18:39 +01:00
Eelco Dolstra 6beee0ab49 Fix segfault sorting runnable steps 
Same problem as d744362e4a.

    at /nix/store/ksvsbr7pg4z69bv6fbbc8h7x7rm2104m-gcc-4.9.3/include/c++/4.9.3/bits/predefined_ops.h:166
    __last@entry=..., __comp=...) at /nix/store/ksvsbr7pg4z69bv6fbbc8h7x7rm2104m-gcc-4.9.3/include/c++/4.9.3/bits/stl_algo.h:1827
    __comp=...) at /nix/store/ksvsbr7pg4z69bv6fbbc8h7x7rm2104m-gcc-4.9.3/include/c++/4.9.3/bits/stl_algo.h:4717
 2016-03-02 13:59:24 +01:00
Eelco Dolstra 7cd08c7c46 Warn if PostgreSQL appears stalled 2016-02-29 15:10:30 +01:00
Eelco Dolstra 922dc541c2 Add log message 2016-02-29 11:58:06 +01:00
Eelco Dolstra 610a8d67ae Better AWS error messages 2016-02-26 22:40:27 +01:00
Eelco Dolstra 1a055e7e9e Reduce severity level of some message 2016-02-26 21:31:08 +01:00
Eelco Dolstra 6bb860fd6e Add FIXME 2016-02-26 21:15:05 +01:00
Eelco Dolstra 53ca41ef9f Use US standard S3 region 2016-02-26 20:57:47 +01:00
Eelco Dolstra c635f5d0ea Fix Makefile.am 2016-02-26 19:54:55 +01:00
Eelco Dolstra b9afaadfb3 Keep better bytesReceived/bytesSent stats 2016-02-26 16:17:05 +01:00
Eelco Dolstra 6d741d2ffa Prevent download of NARs we just uploaded 2016-02-26 15:21:44 +01:00
Eelco Dolstra 02190b0fef Support hydra-build-products on binary cache stores 2016-02-26 14:45:03 +01:00
Eelco Dolstra 8e24ad6f0d Sync with Nix 2016-02-25 10:58:31 +01:00
Eelco Dolstra 8321a3eb27 Sync with Nix 2016-02-24 14:04:31 +01:00
Eelco Dolstra 7b509237cd Bleh Automake 2016-02-22 18:05:15 +01:00
Eelco Dolstra 6c3ae36648 hydra-queue-runner: Get store mode configuration from hydra.conf 
To use the local Nix store (default):

  store_mode = direct

To use a local binary cache:

  store_mode = local-binary-cache
  binary_cache_dir = /var/lib/hydra/binary-cache

To use an S3 bucket:

  store_mode = s3-binary-cache
  binary_cache_s3_bucket = my-nix-bucket

Also, respect binary_cache_{secret,public}_key_file for signing the
binary cache.
 2016-02-22 17:23:06 +01:00
Eelco Dolstra 94817d77d9 BinaryCacheStore: Respect build-use-substitutes 2016-02-22 17:21:39 +01:00
Eelco Dolstra 5668aa5f71 After uploading a .narinfo, add it to the LRU cache 2016-02-20 10:35:16 +01:00
Eelco Dolstra 88a05763cc Pool local store connections 2016-02-20 00:04:08 +01:00
Eelco Dolstra 1cefd6cac8 Fix log message 2016-02-20 00:02:37 +01:00
Eelco Dolstra 2b76094a23 S3BinaryCacheStore::isValidPath(): Do a GET instead of HEAD 2016-02-19 17:41:11 +01:00
Eelco Dolstra bd76f9120a Cache .narinfo lookups 2016-02-19 16:19:40 +01:00
Eelco Dolstra a0f74047da Keep some statistics for the binary cache stores 2016-02-19 14:24:23 +01:00
Eelco Dolstra dc4a00347d Use a single BinaryCacheStore for all threads 
This will make it easier to do caching / keep stats. Also, we won't
have S3Client's connection pooling if we create multiple S3Client
instances.
 2016-02-18 17:31:19 +01:00
Eelco Dolstra 00a7be13a2 Make queue runner internal status available under /queue-runner-status 2016-02-18 17:11:46 +01:00
Eelco Dolstra 8c9fc677c1 Typo 2016-02-18 16:43:24 +01:00
Eelco Dolstra db3fcc0f5e Enable substitution on the build machines 
If properly configured, this allows them to get store paths directly
from S3, rather than having to receive them from the queue runner.
 2016-02-18 16:42:05 +01:00
Eelco Dolstra 2d40888e2e Add an S3-backed binary cache store 2016-02-18 16:18:50 +01:00
Eelco Dolstra 0e254ca66d Refactor local binary cache code into a subclass 2016-02-18 14:06:17 +01:00
Eelco Dolstra a992f688d1 Rename class 2016-02-18 13:02:20 +01:00
Eelco Dolstra de77cc2910 Rename file 2016-02-18 13:02:20 +01:00
Eelco Dolstra ce5790285a Merge remote-tracking branch 'origin/master' into binary-cache 2016-02-17 11:54:59 +01:00
Eelco Dolstra d7a123fcd4 Keep track of the time we spend copying to/from build machines 2016-02-17 10:30:23 +01:00
Eelco Dolstra 25022bf5fd hydra-queue-runner: Support generating a signed binary cache 2016-02-16 16:41:42 +01:00
Eelco Dolstra 744cee134e hydra-queue-runner: Compress binary cache NARs using xz 2016-02-15 21:56:53 +01:00
Eelco Dolstra 2d0dd7fb49 hydra-queue-runner: Write directly to a binary cache 2016-02-15 21:10:29 +01:00
Eelco Dolstra 92d8b59361 Process Nix API changes 2016-02-11 15:59:47 +01:00
Eelco Dolstra 97f8c61928 Fix hydra-queue-runner --build-one 2015-12-29 17:53:33 +01:00
Eelco Dolstra c087472c71 Remove superfluous "has" function 2015-11-02 14:29:12 +01:00
Eelco Dolstra 2d128d2a6f Don't show redundant "removing machine..." messages 2015-10-30 18:22:43 +01:00
Eelco Dolstra d8d188301d Fix division-by-zero crash 
Not clear why step_->jobsets was empty...
 2015-10-30 18:01:48 +01:00
Eelco Dolstra 1ff48da3d3 int2String -> std::to_string 2015-10-30 18:01:38 +01:00
Eelco Dolstra 4d1816b152 Remove obsolete Builds columns and provide accurate "Running builds" 
This removes the "busy", "locker" and "logfile" columns, which are no
longer used by the queue runner. The "Running builds" page now only
shows builds that have an active build step.
 2015-10-27 15:37:17 +01:00
Eelco Dolstra 53c80d9526 getQueuedBuilds(): Periodically stop to handle priority bumps 
Previously, priority bumps could take a long time to get noticed if
getQueuedBuilds() was busy processing zillions of queue
additions. (This was made worse by the reintroduction of substitute
checking.)
 2015-10-22 17:00:46 +02:00
Eelco Dolstra 71bf7e02d5 Use nix::willBuildLocally() 2015-10-21 15:44:29 +02:00
Eelco Dolstra 8e8e31ce86 Re-implement log size limits 
The old queue runner already had this. However, we now store "log
limit exceeded" as a separate status code in the database.
 2015-10-06 17:35:08 +02:00