raito/hydra
0
0
Fork 0
forked from lix-project/hydra
Code Pull requests Activity
%!d() commits 2 branches 0 tags 17 MiB
Commit graph

163 commits

Author SHA1 Message Date
Eelco Dolstra e0b2921ff2 Concurrent hydra-evaluator 
This rewrites the top-level loop of hydra-evaluator in C++. The Perl
stuff is moved into hydra-eval-jobset. (Rewriting the entire evaluator
would be nice but is a bit too much work.) The new version has some
advantages:

* It can run multiple jobset evaluations in parallel.

* It uses PostgreSQL notifications so it doesn't have to poll the
  database. So if a jobset is triggered via the web interface or from
  a GitHub / Bitbucket webhook, evaluation of the jobset will start
  almost instantaneously (assuming the evaluator is not at its
  concurrency limit).

* It imposes a timeout on evaluations. So if e.g. hydra-eval-jobset
  hangs connecting to a Mercurial server, it will eventually be
  killed.
 2016-10-14 14:22:12 +02:00
Eelco Dolstra 303718dbd6 Doh 2016-10-10 15:26:50 +02:00
Eelco Dolstra 0b5a7472be Add a job for the manual 2016-10-10 15:26:18 +02:00
Eelco Dolstra 633724cc8a services.hydra -> services.hydra-dev 
Since Nixpkgs 16.09 contains a Hydra module, we were getting a
conflict. I guess we should have a way to override modules in
Nixpkgs...
 2016-10-10 11:29:15 +02:00
Eelco Dolstra 2f0d2f8d91 Bump Nix 2016-10-10 11:18:17 +02:00
Eelco Dolstra fb0d2d2dda Build against the bleeding edge of Nix 2016-10-06 17:26:12 +02:00
Eelco Dolstra 934829a1a9 Remove NetStatsd since it's also in Nixpkgs 2016-10-06 14:50:59 +02:00
Domen Kožar 23bf99b8e2 Use pixz instead of bzip2 for compressing nars 
According to following two benchmarks:

- https://www.rootusers.com/gzip-vs-bzip2-vs-xz-performance-comparison/
- http://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO

xz has better compression ratio than bzip2 at lowest compression rate.

https://github.com/vasi/pixz has been chosen as it can scale compressing
over multiple cores linearly.

We're using this in snabblab for a month now and it has improved CPU
wise the main Hydra server.
 2016-08-12 11:39:07 +02:00
Alexander Ried 2c78a92ac5 release.nix: add missing dependency on nix 2016-05-09 18:35:24 +02:00
Alexander Ried 3f5ccfc7f3 release.nix use makeBinPath to construct PATH 
makeBinPath takes care to use the correct output.

nix-repl> lib.makeSearchPath "bin" [pkgs.nix]
"/nix/store/zpp83pr21ihxwsr15l6mkzwkr49zj71d-nix-1.11.2-dev/bin"

nix-repl> lib.makeBinPath [pkgs.nix]
"/nix/store/9n8c3g541qn43yjjs94f1a0m69wp8scg-nix-1.11.2/bin"
 2016-04-29 04:23:54 +02:00
Eelco Dolstra aa4c1fb1ab Fix version 2016-03-22 17:26:50 +01:00
Eelco Dolstra aba2356932 Restore path in nix-shell 2016-03-22 16:59:05 +01:00
Eelco Dolstra 1332463b02 Don't wrap C++ programs 2016-03-22 13:35:09 +01:00
Eelco Dolstra e624652dd8 Use patched aws-sdk-cpp 2016-03-22 13:11:30 +01:00
Eelco Dolstra a727643286 inNixShell considered harmful 2016-03-22 13:10:37 +01:00
Eelco Dolstra 74426e6820 Simplify running nix-shell 
This also removes building a separate source tarball or building a PDF
manual since it's unlikely anybody cares.
 2016-03-22 12:53:28 +01:00
Eelco Dolstra 2d40888e2e Add an S3-backed binary cache store 2016-02-18 16:18:50 +01:00
Eelco Dolstra 993647d1e3 Use Google's verifier 2016-01-14 12:54:47 +01:00
Eelco Dolstra 5a580b1bb2 Add support for logging in via a Google account 
The required configuration in hydra.conf:

  enable_google_login = 1
  google_client_id = 238429sdjkds....apps.googleusercontent.com

and optionally persona_allowed_domains to restrict to one or more
domains.
 2016-01-13 17:32:52 +01:00
Eelco Dolstra 1b1fdb3f8d Revert "Don't wrap hydra-queue-runner / hydra-eval-jobs" 
This reverts commit f8ee98f455.
 2015-10-31 15:32:19 +01:00
Eelco Dolstra f8ee98f455 Don't wrap hydra-queue-runner / hydra-eval-jobs 2015-10-30 18:13:23 +01:00
Eelco Dolstra 53d270c007 Enable debug info 2015-10-30 18:12:30 +01:00
Eelco Dolstra 7b9f488949 Check Nix version 2015-10-14 13:17:24 +02:00
Eelco Dolstra 69e9f73cf6 Use nixUnstable for the tarball 
Not needed for building, only for dev-shell.
 2015-08-17 11:17:54 +02:00
Eelco Dolstra 48d6f0de2a Fix the API test 2015-08-12 16:07:19 +02:00
Eelco Dolstra 8092149a9f Use nixUnstable 2015-07-30 16:55:11 +02:00
Eelco Dolstra f933762358 Disable building the PDF manual 2015-07-08 16:51:05 +02:00
Eelco Dolstra 296213e90a Drop obsolete graphviz dependency 2015-07-08 16:30:58 +02:00
Eelco Dolstra b5815e2aa6 Run PostgreSQL during "make check" 2015-06-25 16:47:39 +02:00
Eelco Dolstra 62219adaf3 Send queue runner stats to statsd 
This is currently done by a separate program that periodically
calls "hydra-queue-runner --status". Eventually, I'll do this
in the queue runner directly.

Fixes #220.
 2015-06-23 14:56:43 +02:00
Eelco Dolstra a757b783f4 Update $PATH 2015-06-22 16:15:16 +02:00
Eelco Dolstra a0c4120232 Don't copy src for nix-shell 2015-06-19 17:45:11 +02:00
Eelco Dolstra a984c0badc Merge branch 'master' into build-ng 2015-06-15 18:21:07 +02:00
Eelco Dolstra bf87d3a6ed Use stable Nix 2015-06-15 14:51:38 +02:00
Eelco Dolstra d6354cbe1f Fix Perl error in tests.api 
Doing "su hydra" causes Perl to be invoked with a wrong PERL5LIB
(pointing to root's profile), leading to "Can't locate strict.pm".
 2015-06-15 11:56:51 +02:00
Eelco Dolstra abca7a87da Cleanup 2015-06-12 18:15:58 +02:00
Eelco Dolstra f06ec78859 Handle building from a dirty Git tree 2015-06-12 18:15:58 +02:00
Eelco Dolstra 18e0a62e24 Disable 32-bit builds again 
They're failing consistently:

http://hydra.nixos.org/job/hydra/master/build.i686-linux/all
 2015-06-12 15:35:14 +02:00
Eelco Dolstra dc446c3980 Start of single-process hydra-queue-runner 2015-05-28 17:39:29 +02:00
Eelco Dolstra c93ff1e817 Doh 2015-05-26 15:25:21 +02:00
Chris Warburton d64ba15357 Allow running on i686-linux 2015-05-17 18:41:27 +01:00
Eelco Dolstra 8523130ebb Use Email::MIME instead of Email::Simple 
Email::Simple cannot handle non-ASCII characters.

Fixes #191.
 2014-11-19 14:45:46 +01:00
Eelco Dolstra d28ee3fd3f Remove Term::Size::Any dependency 
This is now propagated by Catalyst::Runtime.
 2014-11-12 13:49:04 +01:00
Eelco Dolstra eb2965071e Depend on Term::Size::Any 
This shuts up a warning from Catalyst.
 2014-11-06 15:36:46 +01:00
Eelco Dolstra 594fb7a009 Disable the S3 backup test 
This test has never succeeded
(http://hydra.nixos.org/job/hydra/master/tests.s3backup.x86_64-linux)
so until somebody fixes it, there is not much point in building it.
 2014-09-24 15:21:01 +02:00
Eelco Dolstra 4727165832 Use pkgconfig to find Nix 2014-09-18 12:24:05 +02:00
Eelco Dolstra 1306291c6c Handle utf-8 properly 
Fixes errors like:

  Caught exception in engine "Wide character in syswrite at /nix/store/498lwsrn5kkdh1q8kn3vcpd3457w6m7a-hydra-perl-deps/lib/perl5/site_perl/5.16.3/Starman/Server.pm line 547."

Note that these errors didn't happen if the database encoding was set
to SQL_ASCII (which was the case for hydra.nixos.org, explaining why
it didn't get these errors). However, now the encoding must be
UTF8. To change it, do:

  update pg_database set encoding = pg_char_to_encoding('UTF8') where datname = 'hydra';
 2014-05-01 16:33:25 +02:00
Eelco Dolstra c92410c147 Use hydra-module.nix in the tests 2013-11-06 18:10:52 +01:00
Eelco Dolstra bb2976693c Use hydra-init / hydra-create-user in the tests 2013-11-06 16:12:07 +01:00
Eelco Dolstra 0fd6bb4e3f Drop building on i686-linux 
Nobody cares.
 2013-11-05 00:09:33 +01:00