Commit graph

193 commits

Author SHA1 Message Date
Graham Christensen 3a04f28459
hydra: upgrade nixpkgs to 19.09 2019-09-24 16:51:18 -04:00
Graham Christensen 937e165328
export a /prometheus endpoint
Currently only shows per-machine build times
2019-09-24 16:50:52 -04:00
Nikola Knezevic a9d744d2a3 Add Term::Size::Any
This removes a supper annoying set of messages that polute the logs:

    Aug 30 09:00:30 xxx.compute.internal hydra-server[957]: Trouble trying to detect your terminal size, looking at $ENV{COLUMNS}
    Aug 30 09:00:30 xxx.compute.internal hydra-server[957]: Term::Size::Any is not installed, can't autodetect terminal column width
2019-08-30 11:58:24 +02:00
Nikola Knezevic 06bdc8f85c Added the InfluxDBNotification plugin including a NixOS test
This adds a `InfluxDBNotification` plugin which is configured as:

```
<influxdb>
  url = http://127.0.0.1:8086
  db = hydra
</influxdb>
```

which will write a notification for every finished job to the
configured database in InfluxDB looking like:

```
hydra_build_status,cached=false,job=job,jobset=default,project=sample,repo=default,result=success,status=success,system=x86_64-linux build_id="1",build_status=0i,closure_size=584i,duration=0i,main_build_id="1",queued=0i,size=168i 1564156212
```
2019-07-26 17:47:03 +02:00
Antoine Eiche c620bc2be0 test.api: use Hydra perl dependencies to run api-test.pl
The test was failing with:

    machine# Can't locate LWP/UserAgent.pm in @INC (you may need to install the LWP::UserAgent module)...
2019-06-21 16:57:12 +02:00
Graham Christensen bcb1c2916d release: bump to 19.03 2019-04-26 15:14:27 +02:00
Bas van Dijk e9d032bfe9 Fix Linux build on Darwin
Without this patch running the following on MacOS:

  nix-build release.nix -A build.x86_64-linux

results in the following error during the configuration phase (note that Nix
should be configured with a x86_64-linux build machine):

  building '/nix/store/jb6ca1gmplyb69ayd43z7fb0y9npxd53-hydra-0.1.2581.8b5948f4cf12424c04df67a6eb136c9846fb2cfd.drv' on 'ssh://my-linux-build-machine'...
  ...
  checking whether /nix/store/s6bhdppx66bkgf741vk4d29hgsj1h1zp-hydra-perl-deps/bin/nix-store is recent enough... ./configure: line 16254: /nix/store/s6bhdppx66bkgf741vk4d29hgsj1h1zp-hydra-perl-deps/bin/nix-store: cannot execute binary file: Exec format error
  no
  configure: error: `/nix/store/s6bhdppx66bkgf741vk4d29hgsj1h1zp-hydra-perl-deps/bin/nix-store' doesn't support `--timeout'; please use a newer version.
  build time elapsed:  0m1.624s 0m1.774s 0m9.366s 0m6.110s
  builder for '/nix/store/jb6ca1gmplyb69ayd43z7fb0y9npxd53-hydra-0.1.2581.8b5948f4cf12424c04df67a6eb136c9846fb2cfd.drv' failed with exit code 1

This problem is that the `nix` dependency of hydra is selected from a nixpkgs
set configured with a default `system` parameter,
i.e. `builtin.currentSystem`. This means that the hydra derivation which is
build for and on Linux depends on the nix derivation build for Darwin.

The fix is to select nix from the nixpkgs set configured with a system specified
by the user.
2019-03-15 11:27:16 +01:00
Michael Bishop 3ef253c65f
also add CatalystXRoleApplicator
f7fe61207b
2018-10-30 15:11:01 +01:00
Michael Bishop 6d693b6b2a
fix hydra build on nixpkgs master
copied from c501958535
2018-10-30 14:50:22 +01:00
Eelco Dolstra 8d26144121
Fix building against nix master 2018-10-30 14:41:21 +01:00
Eelco Dolstra 4dca8fe14d
Gradually increase maxHeapSize 2018-08-07 11:47:53 +02:00
Eelco Dolstra 1a899890ac
Use nixUnstable 2018-07-31 21:19:10 +02:00
Eelco Dolstra b11789f399
Add boost dependency 2018-06-05 13:24:23 +02:00
Eelco Dolstra 821f8bf5a0
release.nix: Use fetchGit to get Nixpkgs 2018-06-05 13:24:23 +02:00
Eelco Dolstra 691f7e168c
Fix building on Nixpkgs 17.09 2018-05-10 15:11:24 +02:00
Eelco Dolstra cb43da06f3
Use Nix 2.0 and default GCC 2018-03-07 09:52:18 +01:00
Eelco Dolstra 3ec73c93e2
Remove unused dependency on aws-sdk-cpp 2018-02-23 16:47:24 +01:00
Eelco Dolstra 5216b7eb2c
Don't rely on ${LOGNAME:-foo} evaluating to a string 2017-11-14 15:11:14 +01:00
Eelco Dolstra d9cbf8cf01
Use PostgreSQL 9.5
9.2 is no longer in Nixpkgs.
2017-09-22 15:18:22 +02:00
spacefrogg 7e9b6acbce Fix perl search path; avoid reference to build path (#471)
Fixes #460
2017-08-10 14:13:00 +02:00
Eelco Dolstra 8abc595d1c
preHook -> shellHook
Adding the build directory to $PERL5LIB causes it to end up in the
wrapper scripts, which is bad.
2017-05-04 15:25:22 +02:00
Eelco Dolstra 468b048fa2
Revert "Move build logic for the core hydra package into its own default.nix."
This reverts commit 949e5865c6. This
makes release.nix harder to read/maintain IMHO. There already is a Nix
expression for Hydra in Nixpkgs that can be used for this purpose.
2017-04-12 13:10:01 +02:00
Shea Levy 949e5865c6 Move build logic for the core hydra package into its own default.nix.
This allows users to build hydra using a normal mkDerivation, package overrides, etc.
2017-04-11 13:07:46 -04:00
Eelco Dolstra a0376a92e5
Add dependency on nix-perl
Fixes #459.
2017-04-11 15:48:22 +02:00
Eelco Dolstra a366f362e1 Use latest nixUnstable 2017-02-03 14:39:18 +01:00
Eelco Dolstra aef048b3cb
Bump Nix dependency 2016-11-16 17:52:26 +01:00
Eelco Dolstra cb5e438a08 Bump Nix
Fixes #398.
2016-11-09 19:15:13 +01:00
Eelco Dolstra 2b790ceefa Remove the pretty-printed log
It didn't work anymore due to the removal of nix-log2xml. And in any
case XSL is the work of the devil.
2016-10-27 14:43:17 +02:00
Eelco Dolstra 3b84d4711b Bump Nix 2016-10-26 15:10:56 +02:00
Eelco Dolstra 881d3bc017 Bump Nix 2016-10-21 19:19:28 +02:00
Eelco Dolstra e0b2921ff2 Concurrent hydra-evaluator
This rewrites the top-level loop of hydra-evaluator in C++. The Perl
stuff is moved into hydra-eval-jobset. (Rewriting the entire evaluator
would be nice but is a bit too much work.) The new version has some
advantages:

* It can run multiple jobset evaluations in parallel.

* It uses PostgreSQL notifications so it doesn't have to poll the
  database. So if a jobset is triggered via the web interface or from
  a GitHub / Bitbucket webhook, evaluation of the jobset will start
  almost instantaneously (assuming the evaluator is not at its
  concurrency limit).

* It imposes a timeout on evaluations. So if e.g. hydra-eval-jobset
  hangs connecting to a Mercurial server, it will eventually be
  killed.
2016-10-14 14:22:12 +02:00
Eelco Dolstra 303718dbd6 Doh 2016-10-10 15:26:50 +02:00
Eelco Dolstra 0b5a7472be Add a job for the manual 2016-10-10 15:26:18 +02:00
Eelco Dolstra 633724cc8a services.hydra -> services.hydra-dev
Since Nixpkgs 16.09 contains a Hydra module, we were getting a
conflict. I guess we should have a way to override modules in
Nixpkgs...
2016-10-10 11:29:15 +02:00
Eelco Dolstra 2f0d2f8d91 Bump Nix 2016-10-10 11:18:17 +02:00
Eelco Dolstra fb0d2d2dda Build against the bleeding edge of Nix 2016-10-06 17:26:12 +02:00
Eelco Dolstra 934829a1a9 Remove NetStatsd since it's also in Nixpkgs 2016-10-06 14:50:59 +02:00
Domen Kožar 23bf99b8e2 Use pixz instead of bzip2 for compressing nars
According to following two benchmarks:

- https://www.rootusers.com/gzip-vs-bzip2-vs-xz-performance-comparison/
- http://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO

xz has better compression ratio than bzip2 at lowest compression rate.

https://github.com/vasi/pixz has been chosen as it can scale compressing
over multiple cores linearly.

We're using this in snabblab for a month now and it has improved CPU
wise the main Hydra server.
2016-08-12 11:39:07 +02:00
Alexander Ried 2c78a92ac5 release.nix: add missing dependency on nix 2016-05-09 18:35:24 +02:00
Alexander Ried 3f5ccfc7f3 release.nix use makeBinPath to construct PATH
makeBinPath takes care to use the correct output.

nix-repl> lib.makeSearchPath "bin" [pkgs.nix]
"/nix/store/zpp83pr21ihxwsr15l6mkzwkr49zj71d-nix-1.11.2-dev/bin"

nix-repl> lib.makeBinPath [pkgs.nix]
"/nix/store/9n8c3g541qn43yjjs94f1a0m69wp8scg-nix-1.11.2/bin"
2016-04-29 04:23:54 +02:00
Eelco Dolstra aa4c1fb1ab Fix version 2016-03-22 17:26:50 +01:00
Eelco Dolstra aba2356932 Restore path in nix-shell 2016-03-22 16:59:05 +01:00
Eelco Dolstra 1332463b02 Don't wrap C++ programs 2016-03-22 13:35:09 +01:00
Eelco Dolstra e624652dd8 Use patched aws-sdk-cpp 2016-03-22 13:11:30 +01:00
Eelco Dolstra a727643286 inNixShell considered harmful 2016-03-22 13:10:37 +01:00
Eelco Dolstra 74426e6820 Simplify running nix-shell
This also removes building a separate source tarball or building a PDF
manual since it's unlikely anybody cares.
2016-03-22 12:53:28 +01:00
Eelco Dolstra 2d40888e2e Add an S3-backed binary cache store 2016-02-18 16:18:50 +01:00
Eelco Dolstra 993647d1e3 Use Google's verifier 2016-01-14 12:54:47 +01:00
Eelco Dolstra 5a580b1bb2 Add support for logging in via a Google account
The required configuration in hydra.conf:

  enable_google_login = 1
  google_client_id = 238429sdjkds....apps.googleusercontent.com

and optionally persona_allowed_domains to restrict to one or more
domains.
2016-01-13 17:32:52 +01:00
Eelco Dolstra 1b1fdb3f8d Revert "Don't wrap hydra-queue-runner / hydra-eval-jobs"
This reverts commit f8ee98f455.
2015-10-31 15:32:19 +01:00