0
0
Fork 0
forked from lix-project/hydra
Commit graph

18 commits

Author SHA1 Message Date
Eelco Dolstra 18a3c3ff1c Update "make check" for the new queue runner
Also, if the machines file contains an entry for localhost, then run
"nix-store --serve" directly, without going through SSH.
2015-06-25 16:47:39 +02:00
Shea Levy 74388353b5 Add a plugin for backing up builds in s3
In your hydra config, you can add an arbitrary number of <s3config>
sections, with the following options:

* name (required): Bucket name
* jobs (required): A regex to match job names (in project:jobset:job
  format) that should be backed up to this bucket
* compression_type: bzip2 (default), xz, or none
* prefix: String to prepend to all hydra-created s3 keys (if this is
  meant to represent a directory, you should include the trailing slash,
  e.g. "cache/"). Default "".

After each build with an output (i.e. successful or failed-with-output
builds), the output path and its closure are uploaded to the bucket as
.nar files, with corresponding .narinfos to enable use as a binary
cache.

This plugin requires that s3 credentials be available. It uses
Net::Amazon::S3, which as of this commit the nixpkgs version can
retrieve s3 credentials from the AWS_ACCESS_KEY_ID and
AWS_SECRET_ACCESS_KEY environment variables, or from ec2 instance
metadata when using an IAM role.

This commit also adds a hydra-s3-backup-collect-garbage program, which
uses hydra's gc roots directory to determine which paths are live, and
then deletes all files except nix-cache-info and any .nar or .narinfo
files corresponding to live paths. hydra-s3-backup-collect-garbage
respects the prefix configuration option, so it won't delete anything
outside of the hierarchy you give it, and it has the same credential
requirements as the plugin. Probably a timer unit running the garbage
collection periodically should be added to hydra-module.nix

Note that two of the added tests fail, due to a bug in the interaction
between Net::Amazon::S3 and fake-s3. Those behaviors work against real
s3 though, so I'm committing this even with the broken tests.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-09-18 18:32:58 +02:00
Eelco Dolstra a77161e40a Allow users to edit their own settings
Also, don't use the flash anymore for going back to the referer.
2013-03-04 15:25:23 +01:00
Eelco Dolstra 58dd49e645 Fix handling of IPC::Run::run exit status
Turns out that the exit status is returned in $?, not as the return
value of run().  So our error checking was completely bogus.
2013-01-23 15:56:28 +01:00
Eelco Dolstra 2d36b75d3e Fix indentation 2013-01-23 12:49:26 +00:00
Eelco Dolstra 67aefde62c Remove trailing whitespace 2013-01-22 14:41:02 +01:00
Eelco Dolstra 854513be8e I should test first 2012-04-30 17:57:13 +02:00
Nicolas Pierron 69f9175986 Add tests for "git describe --long".
* Move test logic to scm-update.sh scripts.
    * Check for "git describe --long".
2012-04-29 20:59:43 -07:00
Nicolas Pierron 19d9955e89 Optimize fetch-git. 2012-04-14 18:17:35 -07:00
Eelco Dolstra fa62c8b7f2 Fix tests 2012-03-13 13:18:40 +01:00
Eelco Dolstra 93efae0c23 Fix "make check" 2012-03-12 21:13:50 +01:00
Eelco Dolstra 12a7b5737d Test environment cleanups
In particular, clear $NIX_BUILD_HOOK to prevent tests
from failing mysteriously.  Also print stdout/stderr
output of hydra-evaluator.
2011-12-04 22:50:03 +01:00
Rob Vermaas ae26e6db2a fix assignment in evalSucceeds (always returned 3, therefore evalSucceeds always succeeded 2011-12-01 08:31:07 -05:00
Eelco Dolstra dd6e5d49ba Fix the test 2011-11-30 18:14:48 +01:00
Rob Vermaas 55b392a20d wrong altnr for jobset inputs in tests 2011-03-21 12:30:47 +00:00
Rob Vermaas 9f33d4c98d 2011-03-17 13:25:27 +00:00
Rob Vermaas 597e7c04e1 cleanup 2011-03-17 11:55:39 +00:00
Rob Vermaas 890a7867b4 add first evaluations tests 2011-03-16 13:18:12 +00:00