Commit graph

106 commits

Author SHA1 Message Date
John Ericson c5f37eca91 Reorganize hydra modules 2024-01-25 14:55:07 -05:00
Rick van Schijndel a084e204ae
systemd: hydra-queue-runner: wait for network.target too
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2023-03-07 21:56:20 +01:00
Rick van Schijndel 65c1249227 systemd: hydra-queue-runner: wait for network-online
This prevents eval errors when a machine is just started and the network isn't yet online.
I'm running hydra on a laptop and the network takes a bit of time to come online (WLAN),
so it's nice if the evaluator starts only when the network actually goes online.

Otherwise an error like this can happen on the first eval(s):

```
error fetching latest change from git repo at `https://github.com/nixos/nixpkgs.git':
fatal: unable to access 'https://github.com/nixos/nixpkgs.git/': Could not resolve host: github.com
```
2023-02-16 19:24:53 +01:00
Jörg Thalheim 940c0d7d97 fix deprecated chown syntax 2022-09-29 21:22:13 +02:00
Eelco Dolstra d61702f86d Merge remote-tracking branch 'origin/master' into update-nix 2022-09-06 16:14:23 +02:00
Amanda Cameron 9656f16509
Update hydra-module.nix to use newer nixos options
nix.trustedUsers is deprecated as of 22.05, and since the nix.extraOptions config is just doing something similar, I moved that to the new nix.settings as well
2022-07-22 10:35:38 -04:00
Eelco Dolstra 8dd1daac8a Update to Nix 2.10 2022-07-12 14:45:30 +02:00
Naïm Favier 5e3374cb86
Prepare for nixos-search integration 2022-03-31 12:55:15 +02:00
Cole Helbling 9c4e6f78e7 hydra-module: don't bzip2 runcommand-logs 2022-01-31 12:55:36 -08:00
Cole Helbling c8f3943d03 hydra-module: log files don't have a .drv extension 2022-01-28 13:07:11 -08:00
Cole Helbling 4a441b54ce hydra-module: /var/lib/hydra -> ${baseDir} 2022-01-28 13:03:15 -08:00
Janne Heß 4cb5e6cd94 RunCommand: Capture the output of the commands 2022-01-28 13:00:17 -08:00
Graham Christensen 9708f6bf6f Fetch the tracker HTML from the config file, remove HYDRA_TRACKER
The indentation in the hydra.conf makes it possible to include multi-line
strings without it being likely that the contents of the tracker
is mis-parsed or interrupts tho config parser.

It isn't impossible / foolproof probably, but it shouldn't be likely.
2022-01-05 15:27:21 -05:00
Maximilian Bosch dd2ce84fe5
hydra-module: don't use createHome = true; to create Hydra's base-dir
In NixOS, the user generation script was changed to set the permissions `0700`
to a home-directory that's specified in the `users.users`-submodule with
`createHome` being set to `true`[1].

However, the home-directory of `hydra` is also the base directory of other services using
other users (e.g. `hydra-queue-runner`). With permissions being `0700`, processes with
such a user cannot traverse into `/var/lib/hydra` and thus not into subdirectories.

I guess that this issue was kind of hidden because `hydra-init.service` ensures
proper permissions[2]. However, if `hydra-init.service` is not restarted on a
system-activation, the permissions of `/var/lib/hydra` will be set back to `0700`
by the activation script that runs on each activation.

This has lead to errors like this in `hydra-queue-runner` on my Hydra:

```
Sep 20 09:11:30 hydra hydra-queue-runner[306]: error (ignored): error: cannot unlink '/var/lib/hydra/build-logs/7h/dssz03gazrkqzfmlr5cprd0dvkg4db-squashfs.img.drv': Permission denied
Sep 20 09:11:30 hydra hydra-queue-runner[306]: error (ignored): error: cannot unlink '/var/lib/hydra/build-logs/b9/350vd8jpv1f86i312c9pkdcd2z56aw-squashfs.img.drv': Permission denied
Sep 20 09:11:30 hydra hydra-queue-runner[306]: error (ignored): error: cannot unlink '/var/lib/hydra/build-logs/kz/vlq4v9a1rylcp4fsqqav3lcjgskky4-squashfs.img.drv': Permission denied
Sep 20 09:11:30 hydra hydra-queue-runner[306]: error (ignored): error: cannot unlink '/var/lib/hydra/build-logs/xd/hkjnbbr9jp7364pkn8zpk9v8xapj2c-nix-2.4pre20210917_37cc50f.drv': Permission denied
Sep 20 09:11:30 hydra hydra-queue-runner[306]: error (ignored): error: cannot unlink '/var/lib/hydra/build-logs/zn/9df7225fl8p7iavqqfvlyay4rf0msw-nix-2.4pre20210917_37cc50f.drv': Permission denied
Sep 20 09:11:30 hydra hydra-queue-runner[306]: possibly transient failure building ‘/nix/store/7hdssz03gazrkqzfmlr5cprd0dvkg4db-squashfs.img.drv’ on ‘roflmayr’: error: creating directory '/var/lib/hydra/build-logs': Permission denied
Sep 20 09:11:30 hydra hydra-queue-runner[306]: will retry ‘/nix/store/7hdssz03gazrkqzfmlr5cprd0dvkg4db-squashfs.img.drv’ after 543s
```

Because of that, I decided to remove the `createHome = true;` setting and instead used
`systemd-tmpfiles`[3] which can not only ensure that certain directories
exist, but also proper permissions.

With this change, we can also get rid of the manual setup in
`hydra-init.service` since `systemd-tmpfiles` will be executed by
`switch-to-configuration` before *any* systemd service gets started. On
startup, `systemd-tmpfiles-setup.service` is invoked within
`sysinit.target` being reached, so when `hydra-init.service` gets called
in `multi-user.target`, the structure already exists.

[1] fa0d499dbf
[2] 3cec908738/hydra-module.nix (L260-L262)
[3] https://www.freedesktop.org/software/systemd/man/systemd-tmpfiles.html
2021-09-22 00:08:15 +02:00
Maximilian Bosch 302f5e0de9
Set isSystemUser = true; for each statically declared user
This is needed to evaluate the module on 21.05.
2021-05-24 14:47:30 +02:00
Eelco Dolstra a53f6657ee
Remove gc-check-reachability setting
It no longer exists on nix master.
2021-04-20 13:35:39 +02:00
Cole Helbling fe70160008
module: append application_name to HYDRA_DBI
This will make it easier to track specifically where queries are being
made from (assuming a `log_line_prefix` that includes `%a` in the
postgres configuration).
2021-03-03 16:02:52 -08:00
Kevin Quick 3e73a2f709
Fix issue #614: restart queue/evaluator on sufficient disk space avai… (#777)
* Fix issue #614: restart queue/evaluator on sufficient disk space available.

* Only try to stop the service if it is currently running.

* Use named variables and added restarting message.
2020-07-27 15:46:57 -04:00
Nikola Knezevic e5c757e219 Remove SSL_CERT_FILE
This was slated to be removed with Nix 16.03. We're now in 20.03, so
it's good to clean that up a bit.
2020-06-05 18:53:16 +02:00
Eelco Dolstra 0ccf36ca3b
Merge remote-tracking branch 'origin/master' into flake 2019-09-24 19:03:18 +02:00
Eelco Dolstra 4223202c99
Merge pull request #432 from LumiGuide/better-buildMachinesFiles-default
Only set buildMachinesFiles when nix.buildMachines is defined
2019-09-24 19:00:32 +02:00
Eelco Dolstra 66d71689ec
flake.nix improvements
The hydra flake now exports a Nixpkgs overlay and a NixOS module.
2019-09-17 17:25:59 +02:00
Eelco Dolstra e2537f741b
Restart hydra-notify 2019-08-13 18:18:24 +02:00
Eelco Dolstra c7861b85c4
Add hydra-notify service 2019-08-13 18:18:24 +02:00
Antoine Eiche 8a0a5ec3a3 Create extension pg_trgm in the NixOS module
The creation of the `pg_trgm` extension needs superuser power. So,
this patch makes the extension creation in the Hydra NixOS module when
a local database is used.

If it is not possible to create this extension (remote database for
instance with nosuperuser), the creation of the `pg_trgm` index is
skipped (this index speedup queries on builds.drvpath) and warnings
are emitted:

    initialising the Hydra database schema...
    WARNING:  Can not create extension pg_trgm: permission denied to create extension "pg_trgm"
    WARNING:  HINT: Temporary provide superuser role to your Hydra Postgresql user and run the script src/sql/upgrade-57.sql
    WARNING:  The pg_trgm index on builds.drvpath has been skipped (slower complex queries on builds.drvpath)

This allows to keep smooth migrations: the migration process doesn't
require a manual step (but this manual step is recommended on big
remote databases).
2019-07-25 16:19:33 +02:00
Eelco Dolstra a71e6301d2
Fix Hydra VM tests 2018-07-31 21:49:05 +02:00
Eelco Dolstra f39c13b2ff hydra-server: Set XDG_CACHE_HOME, other Nix barfs 2017-10-18 15:33:46 +02:00
Eelco Dolstra 3e7938c61a
hydra-module: Include "=" in hydra.conf
hydra-queue-runner is more anal about the format than Catalyst, so it
was misinterpreting gc_roots_dir.
2017-10-18 14:23:10 +02:00
Domen Kožar f220f45d29
hydra-module.nix: restart daemons on config change 2017-06-27 16:01:06 +02:00
michael bishop 6e4a7a30f7
fix github pulls json to be deterministic 2017-06-21 09:35:26 -03:00
Domen Kožar 81fee456c2
fix #425: parsable config by hydra-queue-runner 2017-04-10 14:59:58 +02:00
Eelco Dolstra 285754aff6
hydra-evaluator improvements
* The "Jobset" page now shows when evaluations are in progress (rather
  than just pending).

* Restored the ability to do a single evaluation from the command line
  by doing "hydra-evaluator <project> <jobset>".

* Fix some consistency issues between jobset status in PostgreSQL and
  in hydra-evaluator. In particular, "lastCheckedTime" was never
  updated internally.
2017-03-15 16:59:57 +01:00
Bas van Dijk 4efd078977 Only set buildMachinesFiles when nix.buildMachines is defined 2016-12-11 15:35:38 +01:00
Bas van Dijk cba7e7e0e2 Allow hydra-eval-jobset to be found by the hydra-evaluator
Without this I got the following error in my journal:

  Oct 25 22:42:29 mymachine hydra-evaluator[4085]: starting evaluation of jobset ‘myproject:.jobsets’
  Oct 25 22:42:29 mymachine hydra-evaluator[4085]: timeout: failed to run command ‘hydra-eval-jobset’: No such file or directory
  Oct 25 22:42:29 mymachine hydra-evaluator[4085]: evaluation of jobset ‘myproject:.jobsets’ finished with status 32512
2016-10-25 22:55:32 +02:00
Eelco Dolstra 633724cc8a services.hydra -> services.hydra-dev
Since Nixpkgs 16.09 contains a Hydra module, we were getting a
conflict. I guess we should have a way to override modules in
Nixpkgs...
2016-10-10 11:29:15 +02:00
Eelco Dolstra ee2e9f5335 Update to reflect BinaryCacheStore changes
BinaryCacheStore no longer implements buildPaths() and ensurePath(),
so we need to use copyPath() / copyClosure().
2016-10-07 20:23:05 +02:00
Eric Sagnes dedcb76bed module: add default to buildMachinesFiles (#372) 2016-08-20 19:50:12 +02:00
Domen Kožar 23bf99b8e2 Use pixz instead of bzip2 for compressing nars
According to following two benchmarks:

- https://www.rootusers.com/gzip-vs-bzip2-vs-xz-performance-comparison/
- http://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO

xz has better compression ratio than bzip2 at lowest compression rate.

https://github.com/vasi/pixz has been chosen as it can scale compressing
over multiple cores linearly.

We're using this in snabblab for a month now and it has improved CPU
wise the main Hydra server.
2016-08-12 11:39:07 +02:00
Domen Kožar 6bcbabb4df Merge pull request #307 from groxxda/fix/trivial
Trivial fixes (readme, comment, missing dependency in release.nix)
2016-05-31 09:59:07 +01:00
Nikolay Amiantov 6f10cd297b hydra-module: fix useSubstitutes configuration option 2016-05-26 14:16:47 +03:00
Alexander Ried 51d21b9b91 hydra-module.nix: fix warning about deprecated types.list 2016-05-09 19:09:33 +02:00
Peter Simons 3ca0e50e01 hydra-module.nix: add 'useSubstitutes' option to control use of binary caches
Gives users who want to use binary caches a way to do so without messing
with module internals. Fixes https://github.com/NixOS/hydra/issues/297.
2016-05-06 10:30:54 +02:00
Alexander Ried 5777f246a5 hydra-module: fix debug attribute 2016-04-29 04:21:05 +02:00
Eelco Dolstra 7dad867498 Add smtpHost option 2016-04-13 16:46:24 +02:00
Eelco Dolstra fd20c726e1 Remove ssmtp from $PATH again
See 10cad61231.
2016-04-13 16:33:38 +02:00
Sander van der Burg cbd2e3a50d Fix problem with delegating builds to localhost due to nix-store not being in the PATH 2016-04-06 14:16:04 +00:00
Eelco Dolstra 74dfcc84e9 Make NIX_REMOTE_SYSTEMS configurable 2016-03-25 15:41:38 +01:00
Eelco Dolstra 32fa392146 Fix hydra-queue-runner PATH 2016-03-23 12:35:55 +01:00
Eelco Dolstra f09b92e289 Remove another obsolete SSL variable 2016-03-02 15:03:54 +01:00
Eelco Dolstra ad035b5227 hydra-queue-runner: Enable core dumps 2016-02-28 14:09:04 +01:00