Commit graph

117 commits

Author SHA1 Message Date
Eelco Dolstra 1306291c6c Handle utf-8 properly
Fixes errors like:

  Caught exception in engine "Wide character in syswrite at /nix/store/498lwsrn5kkdh1q8kn3vcpd3457w6m7a-hydra-perl-deps/lib/perl5/site_perl/5.16.3/Starman/Server.pm line 547."

Note that these errors didn't happen if the database encoding was set
to SQL_ASCII (which was the case for hydra.nixos.org, explaining why
it didn't get these errors). However, now the encoding must be
UTF8. To change it, do:

  update pg_database set encoding = pg_char_to_encoding('UTF8') where datname = 'hydra';
2014-05-01 16:33:25 +02:00
Eelco Dolstra c92410c147 Use hydra-module.nix in the tests 2013-11-06 18:10:52 +01:00
Eelco Dolstra bb2976693c Use hydra-init / hydra-create-user in the tests 2013-11-06 16:12:07 +01:00
Eelco Dolstra 0fd6bb4e3f Drop building on i686-linux
Nobody cares.
2013-11-05 00:09:33 +01:00
Eelco Dolstra 0babdf3532 Adjust to the NixOS/Nixpkgs merge 2013-10-11 10:58:34 +02:00
Shea Levy 74388353b5 Add a plugin for backing up builds in s3
In your hydra config, you can add an arbitrary number of <s3config>
sections, with the following options:

* name (required): Bucket name
* jobs (required): A regex to match job names (in project:jobset:job
  format) that should be backed up to this bucket
* compression_type: bzip2 (default), xz, or none
* prefix: String to prepend to all hydra-created s3 keys (if this is
  meant to represent a directory, you should include the trailing slash,
  e.g. "cache/"). Default "".

After each build with an output (i.e. successful or failed-with-output
builds), the output path and its closure are uploaded to the bucket as
.nar files, with corresponding .narinfos to enable use as a binary
cache.

This plugin requires that s3 credentials be available. It uses
Net::Amazon::S3, which as of this commit the nixpkgs version can
retrieve s3 credentials from the AWS_ACCESS_KEY_ID and
AWS_SECRET_ACCESS_KEY environment variables, or from ec2 instance
metadata when using an IAM role.

This commit also adds a hydra-s3-backup-collect-garbage program, which
uses hydra's gc roots directory to determine which paths are live, and
then deletes all files except nix-cache-info and any .nar or .narinfo
files corresponding to live paths. hydra-s3-backup-collect-garbage
respects the prefix configuration option, so it won't delete anything
outside of the hierarchy you give it, and it has the same credential
requirements as the plugin. Probably a timer unit running the garbage
collection periodically should be added to hydra-module.nix

Note that two of the added tests fail, due to a bug in the interaction
between Net::Amazon::S3 and fake-s3. Those behaviors work against real
s3 though, so I'm committing this even with the broken tests.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-09-18 18:32:58 +02:00
Rob Vermaas 597fd827b1 Merge pull request #81 from mornfall/master
DARCS support
2013-09-14 06:18:48 -07:00
Eelco Dolstra 3f68076577 hydra-build: Don't send a giant query to the database
We had Postgres barfing with this error:

  DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR: stack depth limit exceeded

because the ‘drvpath => [ @dependentDrvs ]’ in failDependents can
cause a query of unbounded size.  (In this specific case there was a
failure of Bison, which has > 10000 dependent derivations.)  So now we
just get all scheduled builds from the DB.
2013-09-10 11:01:29 +00:00
Petr Rockai 66f3e60e2a Add a test for darcs inputs. 2013-09-07 13:28:53 +02:00
Eelco Dolstra 5078730cb5 Use evalFile() instead of parseExprFromFile() 2013-09-06 15:17:27 +00:00
Eelco Dolstra 9a9b798939 Work around 9P corruption on 32-bit
On 32-bit, Linux 3.4, and if the memory size is bigger than a certain
value, starting the stage 2 init script fails with "Exec format error"
because the 9P filesystem is returning garbage.  No such problem with
Linux 3.10.

http://hydra.nixos.org/build/5737226
2013-08-21 15:10:40 +02:00
Eelco Dolstra 056e2ce503 Don't mess with $LOGNAME in nix-shell 2013-08-16 16:38:09 +02:00
Eelco Dolstra 1481badf21 For nix-shell, set some more variables in preHook 2013-08-12 17:23:33 +02:00
Eelco Dolstra 659c829e88 Tweaks for nix-shell 2013-07-19 14:36:52 +02:00
Shea Levy 117ae78a45 32-bit qemu can only do 2047 MB
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-03 16:42:10 -04:00
Shea Levy 002ac9ef63 Merge in the first bits of the API work
The catalyst-action-rest branch from shlevy/hydra was an exploration of
using Catalyst::Action::REST to create a JSON API for hydra. This commit
merges in the best bits from that experiment, with the goal that further
API endpoints can be added incrementally.

In addition to migrating more endpoints, there is potential for
improvement in what's already been done:
* The web interface can be updated to use the same non-GET endpoints as
  the JSON interface (using x-tunneled-method) instead of having a
  separate endpoint
* The web rendering should use the $c->stash->{resource} data structure
  where applicable rather than putting the same data in two places in
  the stash
* Which columns to render for each endpoint is a completely debatable
  question
* Hydra::Component::ToJSON should turn has_many relations that have
  strings as their primary keys into objects instead of arrays

Fixes NixOS/hydra#98

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-02 14:00:46 -04:00
Eelco Dolstra 663b6a747b release.nix: Get rid of the "system" argument 2013-07-01 19:37:14 +02:00
Eelco Dolstra 9939ddf22c Add a plugin for HipChat notification
This plugin sends notification of build failure or success to a
HipChat room, if the status differs from the last build.

The plugin can be configured by adding one or more of these stanzas to
hydra.conf:

  <hipchat>
    jobs = (patchelf|nixops):.*:.*
    room = 1234
    token = 39ab2198fe...
  </hipchat>

Here "jobs" is a regular expression against which the fully qualified
job name of the build is matched (so for instance
"nixops:master:tarball" will match the stanza above).
2013-06-27 18:56:31 +02:00
Eelco Dolstra 0046c81857 Remove Switch dependency, we're not using it 2013-06-13 15:14:33 +02:00
Eelco Dolstra 7a926935a5 Add Catalyst::DispatchType::Regex for forward compatibility 2013-06-13 15:13:52 +02:00
Eelco Dolstra 66c695e1c3 Fix ‘Unable to handle files with the extension 'conf'’
Config::Any uses Module::Pluggable to seach for plugins, so it needs
the patched Module::Pluggable in Nixpkgs (rather than the one in Perl
itself) to properly find plugins in symlink trees created by buildEnv.
2013-06-02 23:29:24 +02:00
Eelco Dolstra 5e0542d3af Use buildEnv to combine Hydra's Perl dependencies
This makes the Perl search path ($PERL5LIB) much shorter, cutting down
the number of stat() calls when starting hydra-server from ~432000 to
~9000.
2013-05-24 15:26:47 -04:00
Eelco Dolstra 7807ff27b7 Fix the test 2013-04-12 01:24:16 +02:00
Eelco Dolstra 60e36d3d1a Cleanup 2013-01-23 15:47:42 +01:00
Eelco Dolstra b1f7aa0583 Fail on failure 2013-01-23 12:46:01 +00:00
Eelco Dolstra 418e57ec3f VM test: start the web server 2013-01-23 13:00:07 +01:00
Eelco Dolstra 15ee6147d0 Fix VM test 2013-01-23 12:55:29 +01:00
Eelco Dolstra 67aefde62c Remove trailing whitespace 2013-01-22 14:41:02 +01:00
Rob Vermaas 12223d6202 Add bzip2 to buildInputs. 2012-06-25 12:04:25 +02:00
Eelco Dolstra 854513be8e I should test first 2012-04-30 17:57:13 +02:00
Eelco Dolstra 7734de5817 Add OpenSSL as a dependency because Nix needs it
Actually the real problem is that a) Nix doesn't propagate the OpenSSL
dependency; b) Nix shouldn't link against OpenSSL by default anyway.
2012-04-04 11:43:14 +02:00
Rob Vermaas 1904f82f06 add openssl to deps 2012-04-01 18:15:00 +02:00
Eelco Dolstra 914c5e248a Use <...> syntax 2012-03-13 11:03:32 +01:00
Eelco Dolstra 1d96b0ba67 Use a nicer version string 2011-12-05 15:53:23 +01:00
Ludovic Courtès 6c4e83d454 More command renaming. 2011-12-02 15:58:26 +01:00
Eelco Dolstra dd6e5d49ba Fix the test 2011-11-30 18:14:48 +01:00
Ludovic Courtès 5d1ae69696 Second attempt to get Hydra to see style.css & co. 2011-10-12 15:04:13 +00:00
Ludovic Courtès 1007c0ca20 Specify all of $(htmldir) as the doc. 2011-10-12 14:54:23 +00:00
Rob Vermaas 926c622252 remove nix from buildinputs 2011-10-04 20:56:31 +00:00
Rob Vermaas 8e87f2cf61 add nix to buildinputs 2011-10-04 20:23:49 +00:00
Eelco Dolstra 7764095835 * Pass nixUnstable to the tarball build. 2011-08-19 15:43:43 +00:00
Eelco Dolstra 5a8e0d1e13 * Fix symbol to look for. 2011-08-19 15:28:32 +00:00
Ludovic Courtès 1ddd8b29fb release.nix: Install the manual under the right prefix. 2011-07-04 20:11:51 +00:00
Ludovic Courtès 1e737f95f8 release.nix: Add --with-docbook-xsl' to the configure flags for tarball'. 2011-07-04 15:36:38 +00:00
Ludovic Courtès 2e723601ca release.nix: Make the HTML manual a Hydra build product. 2011-07-04 14:48:44 +00:00
Ludovic Courtès 63e3e869c9 release.nix: Remove `services' arguments to ${nixos}/lib/testing.nix. 2011-07-04 14:06:05 +00:00
Rob Vermaas ffa4ba3b9e nix-prefetch-* scripts are in hydra svn now 2011-03-25 08:18:26 +00:00
Rob Vermaas 385b70c898 hydra: fixed and re-enabled evaluation tests 2011-03-24 13:45:01 +00:00
Rob Vermaas d0ba7678d5 oops, should have added topgit to hydraPath 2011-03-21 13:01:54 +00:00
Rob Vermaas d939b06ced add topgit to hydra's path 2011-03-21 12:52:11 +00:00