Commit graph

144 commits

Author SHA1 Message Date
Eelco Dolstra 1b1fdb3f8d Revert "Don't wrap hydra-queue-runner / hydra-eval-jobs"
This reverts commit f8ee98f455.
2015-10-31 15:32:19 +01:00
Eelco Dolstra f8ee98f455 Don't wrap hydra-queue-runner / hydra-eval-jobs 2015-10-30 18:13:23 +01:00
Eelco Dolstra 53d270c007 Enable debug info 2015-10-30 18:12:30 +01:00
Eelco Dolstra 7b9f488949 Check Nix version 2015-10-14 13:17:24 +02:00
Eelco Dolstra 69e9f73cf6 Use nixUnstable for the tarball
Not needed for building, only for dev-shell.
2015-08-17 11:17:54 +02:00
Eelco Dolstra 48d6f0de2a Fix the API test 2015-08-12 16:07:19 +02:00
Eelco Dolstra 8092149a9f Use nixUnstable 2015-07-30 16:55:11 +02:00
Eelco Dolstra f933762358 Disable building the PDF manual 2015-07-08 16:51:05 +02:00
Eelco Dolstra 296213e90a Drop obsolete graphviz dependency 2015-07-08 16:30:58 +02:00
Eelco Dolstra b5815e2aa6 Run PostgreSQL during "make check" 2015-06-25 16:47:39 +02:00
Eelco Dolstra 62219adaf3 Send queue runner stats to statsd
This is currently done by a separate program that periodically
calls "hydra-queue-runner --status". Eventually, I'll do this
in the queue runner directly.

Fixes #220.
2015-06-23 14:56:43 +02:00
Eelco Dolstra a757b783f4 Update $PATH 2015-06-22 16:15:16 +02:00
Eelco Dolstra a0c4120232 Don't copy src for nix-shell 2015-06-19 17:45:11 +02:00
Eelco Dolstra a984c0badc Merge branch 'master' into build-ng 2015-06-15 18:21:07 +02:00
Eelco Dolstra bf87d3a6ed Use stable Nix 2015-06-15 14:51:38 +02:00
Eelco Dolstra d6354cbe1f Fix Perl error in tests.api
Doing "su hydra" causes Perl to be invoked with a wrong PERL5LIB
(pointing to root's profile), leading to "Can't locate strict.pm".
2015-06-15 11:56:51 +02:00
Eelco Dolstra abca7a87da Cleanup 2015-06-12 18:15:58 +02:00
Eelco Dolstra f06ec78859 Handle building from a dirty Git tree 2015-06-12 18:15:58 +02:00
Eelco Dolstra 18e0a62e24 Disable 32-bit builds again
They're failing consistently:

http://hydra.nixos.org/job/hydra/master/build.i686-linux/all
2015-06-12 15:35:14 +02:00
Eelco Dolstra dc446c3980 Start of single-process hydra-queue-runner 2015-05-28 17:39:29 +02:00
Eelco Dolstra c93ff1e817 Doh 2015-05-26 15:25:21 +02:00
Chris Warburton d64ba15357 Allow running on i686-linux 2015-05-17 18:41:27 +01:00
Eelco Dolstra 8523130ebb Use Email::MIME instead of Email::Simple
Email::Simple cannot handle non-ASCII characters.

Fixes #191.
2014-11-19 14:45:46 +01:00
Eelco Dolstra d28ee3fd3f Remove Term::Size::Any dependency
This is now propagated by Catalyst::Runtime.
2014-11-12 13:49:04 +01:00
Eelco Dolstra eb2965071e Depend on Term::Size::Any
This shuts up a warning from Catalyst.
2014-11-06 15:36:46 +01:00
Eelco Dolstra 594fb7a009 Disable the S3 backup test
This test has never succeeded
(http://hydra.nixos.org/job/hydra/master/tests.s3backup.x86_64-linux)
so until somebody fixes it, there is not much point in building it.
2014-09-24 15:21:01 +02:00
Eelco Dolstra 4727165832 Use pkgconfig to find Nix 2014-09-18 12:24:05 +02:00
Eelco Dolstra 1306291c6c Handle utf-8 properly
Fixes errors like:

  Caught exception in engine "Wide character in syswrite at /nix/store/498lwsrn5kkdh1q8kn3vcpd3457w6m7a-hydra-perl-deps/lib/perl5/site_perl/5.16.3/Starman/Server.pm line 547."

Note that these errors didn't happen if the database encoding was set
to SQL_ASCII (which was the case for hydra.nixos.org, explaining why
it didn't get these errors). However, now the encoding must be
UTF8. To change it, do:

  update pg_database set encoding = pg_char_to_encoding('UTF8') where datname = 'hydra';
2014-05-01 16:33:25 +02:00
Eelco Dolstra c92410c147 Use hydra-module.nix in the tests 2013-11-06 18:10:52 +01:00
Eelco Dolstra bb2976693c Use hydra-init / hydra-create-user in the tests 2013-11-06 16:12:07 +01:00
Eelco Dolstra 0fd6bb4e3f Drop building on i686-linux
Nobody cares.
2013-11-05 00:09:33 +01:00
Eelco Dolstra 0babdf3532 Adjust to the NixOS/Nixpkgs merge 2013-10-11 10:58:34 +02:00
Shea Levy 74388353b5 Add a plugin for backing up builds in s3
In your hydra config, you can add an arbitrary number of <s3config>
sections, with the following options:

* name (required): Bucket name
* jobs (required): A regex to match job names (in project:jobset:job
  format) that should be backed up to this bucket
* compression_type: bzip2 (default), xz, or none
* prefix: String to prepend to all hydra-created s3 keys (if this is
  meant to represent a directory, you should include the trailing slash,
  e.g. "cache/"). Default "".

After each build with an output (i.e. successful or failed-with-output
builds), the output path and its closure are uploaded to the bucket as
.nar files, with corresponding .narinfos to enable use as a binary
cache.

This plugin requires that s3 credentials be available. It uses
Net::Amazon::S3, which as of this commit the nixpkgs version can
retrieve s3 credentials from the AWS_ACCESS_KEY_ID and
AWS_SECRET_ACCESS_KEY environment variables, or from ec2 instance
metadata when using an IAM role.

This commit also adds a hydra-s3-backup-collect-garbage program, which
uses hydra's gc roots directory to determine which paths are live, and
then deletes all files except nix-cache-info and any .nar or .narinfo
files corresponding to live paths. hydra-s3-backup-collect-garbage
respects the prefix configuration option, so it won't delete anything
outside of the hierarchy you give it, and it has the same credential
requirements as the plugin. Probably a timer unit running the garbage
collection periodically should be added to hydra-module.nix

Note that two of the added tests fail, due to a bug in the interaction
between Net::Amazon::S3 and fake-s3. Those behaviors work against real
s3 though, so I'm committing this even with the broken tests.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-09-18 18:32:58 +02:00
Rob Vermaas 597fd827b1 Merge pull request #81 from mornfall/master
DARCS support
2013-09-14 06:18:48 -07:00
Eelco Dolstra 3f68076577 hydra-build: Don't send a giant query to the database
We had Postgres barfing with this error:

  DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR: stack depth limit exceeded

because the ‘drvpath => [ @dependentDrvs ]’ in failDependents can
cause a query of unbounded size.  (In this specific case there was a
failure of Bison, which has > 10000 dependent derivations.)  So now we
just get all scheduled builds from the DB.
2013-09-10 11:01:29 +00:00
Petr Rockai 66f3e60e2a Add a test for darcs inputs. 2013-09-07 13:28:53 +02:00
Eelco Dolstra 5078730cb5 Use evalFile() instead of parseExprFromFile() 2013-09-06 15:17:27 +00:00
Eelco Dolstra 9a9b798939 Work around 9P corruption on 32-bit
On 32-bit, Linux 3.4, and if the memory size is bigger than a certain
value, starting the stage 2 init script fails with "Exec format error"
because the 9P filesystem is returning garbage.  No such problem with
Linux 3.10.

http://hydra.nixos.org/build/5737226
2013-08-21 15:10:40 +02:00
Eelco Dolstra 056e2ce503 Don't mess with $LOGNAME in nix-shell 2013-08-16 16:38:09 +02:00
Eelco Dolstra 1481badf21 For nix-shell, set some more variables in preHook 2013-08-12 17:23:33 +02:00
Eelco Dolstra 659c829e88 Tweaks for nix-shell 2013-07-19 14:36:52 +02:00
Shea Levy 117ae78a45 32-bit qemu can only do 2047 MB
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-03 16:42:10 -04:00
Shea Levy 002ac9ef63 Merge in the first bits of the API work
The catalyst-action-rest branch from shlevy/hydra was an exploration of
using Catalyst::Action::REST to create a JSON API for hydra. This commit
merges in the best bits from that experiment, with the goal that further
API endpoints can be added incrementally.

In addition to migrating more endpoints, there is potential for
improvement in what's already been done:
* The web interface can be updated to use the same non-GET endpoints as
  the JSON interface (using x-tunneled-method) instead of having a
  separate endpoint
* The web rendering should use the $c->stash->{resource} data structure
  where applicable rather than putting the same data in two places in
  the stash
* Which columns to render for each endpoint is a completely debatable
  question
* Hydra::Component::ToJSON should turn has_many relations that have
  strings as their primary keys into objects instead of arrays

Fixes NixOS/hydra#98

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-02 14:00:46 -04:00
Eelco Dolstra 663b6a747b release.nix: Get rid of the "system" argument 2013-07-01 19:37:14 +02:00
Eelco Dolstra 9939ddf22c Add a plugin for HipChat notification
This plugin sends notification of build failure or success to a
HipChat room, if the status differs from the last build.

The plugin can be configured by adding one or more of these stanzas to
hydra.conf:

  <hipchat>
    jobs = (patchelf|nixops):.*:.*
    room = 1234
    token = 39ab2198fe...
  </hipchat>

Here "jobs" is a regular expression against which the fully qualified
job name of the build is matched (so for instance
"nixops:master:tarball" will match the stanza above).
2013-06-27 18:56:31 +02:00
Eelco Dolstra 0046c81857 Remove Switch dependency, we're not using it 2013-06-13 15:14:33 +02:00
Eelco Dolstra 7a926935a5 Add Catalyst::DispatchType::Regex for forward compatibility 2013-06-13 15:13:52 +02:00
Eelco Dolstra 66c695e1c3 Fix ‘Unable to handle files with the extension 'conf'’
Config::Any uses Module::Pluggable to seach for plugins, so it needs
the patched Module::Pluggable in Nixpkgs (rather than the one in Perl
itself) to properly find plugins in symlink trees created by buildEnv.
2013-06-02 23:29:24 +02:00
Eelco Dolstra 5e0542d3af Use buildEnv to combine Hydra's Perl dependencies
This makes the Perl search path ($PERL5LIB) much shorter, cutting down
the number of stat() calls when starting hydra-server from ~432000 to
~9000.
2013-05-24 15:26:47 -04:00
Eelco Dolstra 7807ff27b7 Fix the test 2013-04-12 01:24:16 +02:00