Commit graph

3033 commits

Author SHA1 Message Date
Maximilian Bosch 2f9d422172
Fix build against latest Nix 2020-04-07 13:55:38 +02:00
Eelco Dolstra 4cabb37ebd
Merge pull request #730 from NixOS/flake
Flake support
2020-04-07 11:18:38 +02:00
Eelco Dolstra 62e6c65e68
Merge pull request #703 from kquick/libpqxx_undeprecate
Update libpqxx usage to move away from deprecated API interactions.
2020-04-01 22:04:46 +02:00
Kevin Quick a055796ef5
Merge branch 'master' into libpqxx_undeprecate 2020-04-01 11:54:41 -07:00
Eelco Dolstra 2d092a6fbc
Merge pull request #702 from kquick/fix_api_push
Handle case where jobset has no defined errormsg for api/jobsets
2020-04-01 13:09:05 +02:00
Eelco Dolstra 8564089186
Merge pull request #707 from basvandijk/fix-aggregate-status
Fix printing aggregate status
2020-04-01 13:04:53 +02:00
Eelco Dolstra 6c0bb909df
Merge pull request #704 from basvandijk/sleep-after-catch-exception
hydra-queue-runner: sleep 5s after handling an exception
2020-04-01 13:04:31 +02:00
Eelco Dolstra 9668bc01da
Merge pull request #726 from knl/document-slack-notifications-plugin
Document SlackNotifications plugin
2020-04-01 13:04:00 +02:00
Eelco Dolstra 681d84da0f Merge remote-tracking branch 'origin/master' into flake 2020-03-31 22:36:36 +02:00
Eelco Dolstra 53e1294825 Fix Mercurial tests
In Nixpkgs 20.03, Mercurial fails if PYTHONPATH is set:

  $ hg
  Traceback (most recent call last):
    File "/nix/store/q7s856v6nw4dffdrm9k3w38qs35i8kr3-mercurial-5.2.2/bin/..hg-wrapped-wrapped", line 37, in <module>
      dispatch.run()
    File "/nix/store/bffdy7q3wi3qinflnvbdkigqj39zzynd-python3-3.7.6/lib/python3.7/importlib/util.py", line 245, in __getattribute__
      self.__spec__.loader.exec_module(self)
    File "<frozen importlib._bootstrap_external>", line 728, in exec_module
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "/nix/store/q7s856v6nw4dffdrm9k3w38qs35i8kr3-mercurial-5.2.2/lib/python3.7/site-packages/mercurial/dispatch.py", line 10, in <module>
      import difflib
    File "/nix/store/bffdy7q3wi3qinflnvbdkigqj39zzynd-python3-3.7.6/lib/python3.7/difflib.py", line 1084, in <module>
      import re
    File "/nix/store/bffdy7q3wi3qinflnvbdkigqj39zzynd-python3-3.7.6/lib/python3.7/re.py", line 143, in <module>
      class RegexFlag(enum.IntFlag):
  AttributeError: module 'enum' has no attribute 'IntFlag'

(cherry picked from commit 4009d4295e)
2020-03-31 22:19:19 +02:00
Eelco Dolstra 9727892b61 Don't spam the journal with hydra-queue-runner status dumps
(cherry picked from commit 15ae932488)
2020-03-31 22:19:07 +02:00
Eelco Dolstra ccd046ca3d Keep track of the number of unsupported steps
(cherry picked from commit 45ffe578b6)
2020-03-31 22:19:03 +02:00
Eelco Dolstra 4417f9f260 Abort unsupported build steps
If we don't see machine that supports a build step for
'max_unsupported_time' seconds, the step is aborted. The default is 0,
which is appropriate for Hydra installations that don't provision
missing machines dynamically.

(cherry picked from commit f5cdbfe21d)
2020-03-31 22:19:01 +02:00
Eelco Dolstra 68a59f34a0 Remove tabs
(cherry picked from commit 5b731004da)
2020-03-31 22:18:58 +02:00
Eelco Dolstra 8b5900ac33 Remove outdated email address
(cherry picked from commit 22a65d0269)
2020-03-31 22:18:46 +02:00
Eelco Dolstra d1f367f657
Add default.nix, shell.nix for compatibility 2020-03-31 16:07:46 +02:00
Eelco Dolstra 4009d4295e
Fix Mercurial tests
In Nixpkgs 20.03, Mercurial fails if PYTHONPATH is set:

  $ hg
  Traceback (most recent call last):
    File "/nix/store/q7s856v6nw4dffdrm9k3w38qs35i8kr3-mercurial-5.2.2/bin/..hg-wrapped-wrapped", line 37, in <module>
      dispatch.run()
    File "/nix/store/bffdy7q3wi3qinflnvbdkigqj39zzynd-python3-3.7.6/lib/python3.7/importlib/util.py", line 245, in __getattribute__
      self.__spec__.loader.exec_module(self)
    File "<frozen importlib._bootstrap_external>", line 728, in exec_module
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "/nix/store/q7s856v6nw4dffdrm9k3w38qs35i8kr3-mercurial-5.2.2/lib/python3.7/site-packages/mercurial/dispatch.py", line 10, in <module>
      import difflib
    File "/nix/store/bffdy7q3wi3qinflnvbdkigqj39zzynd-python3-3.7.6/lib/python3.7/difflib.py", line 1084, in <module>
      import re
    File "/nix/store/bffdy7q3wi3qinflnvbdkigqj39zzynd-python3-3.7.6/lib/python3.7/re.py", line 143, in <module>
      class RegexFlag(enum.IntFlag):
  AttributeError: module 'enum' has no attribute 'IntFlag'
2020-03-31 13:33:11 +02:00
Eelco Dolstra f6119f2b19
perlDeps: Use lib.closePropagation
Apparentely, buildEnv in 20.03 no longer respects
propagated-build-inputs.

Note that the use of a library function (closePropagation) seems
fundamentally wrong to me - propagated-build-inputs should be used at
runtime, not at evaluation time.
2020-03-31 13:21:01 +02:00
Eelco Dolstra 22a65d0269
Remove outdated email address 2020-03-31 12:58:24 +02:00
Eelco Dolstra 810075cd3d
Update to 20.03 2020-03-31 12:55:35 +02:00
Eelco Dolstra 05bf278971
flake.lock: Update
Flake input changes:

* Updated 'nix': 'github:NixOS/nix/4ad5826a182f8eeb4cbc61e27d2128b08050b0d4' -> 'github:NixOS/nix/3e7aab81ce6787e01ea4ced5af1fc6a84e523762'
* Updated 'nix/nixpkgs': 'github:edolstra/nixpkgs/43e283790fbe330b723ac44b584824cf6d85c1ee' -> 'github:NixOS/nixpkgs/b88ff468e9850410070d4e0ccd68c7011f15b2be'
* Updated 'nixpkgs': 'github:edolstra/nixpkgs/d959778e0934c4c93896d509a65de798909c7cdd' -> 'github:NixOS/nixpkgs/b0c285807d6a9f1b7562ec417c24fa1a30ecc31a'
2020-03-31 12:55:07 +02:00
Samuel Dionne-Riel 7889aef792
hydra-eval-jobs: Mirror eval errors in STDERR
Otherwise, errors will not be shown to end-users, which makes debugging
long evals pretty much impossible.

(cherry picked from commit 76299b9174)
2020-03-31 12:48:33 +02:00
Eelco Dolstra 082dbdb922
Merge pull request #729 from samueldr/fix/error-transmission
hydra-eval-jobs: Mirror eval errors in STDERR
2020-03-31 09:04:32 +02:00
Samuel Dionne-Riel 76299b9174 hydra-eval-jobs: Mirror eval errors in STDERR
Otherwise, errors will not be shown to end-users, which makes debugging
long evals pretty much impossible.
2020-03-30 16:11:44 -04:00
Eelco Dolstra a7540b141d
Merge pull request #727 from NixOS/fail-unsupported
Abort unsupported build steps
2020-03-27 09:45:11 +01:00
Eelco Dolstra 15ae932488
Don't spam the journal with hydra-queue-runner status dumps 2020-03-26 15:30:37 +01:00
Eelco Dolstra 45ffe578b6
Keep track of the number of unsupported steps 2020-03-26 15:27:13 +01:00
Eelco Dolstra f5cdbfe21d
Abort unsupported build steps
If we don't see machine that supports a build step for
'max_unsupported_time' seconds, the step is aborted. The default is 0,
which is appropriate for Hydra installations that don't provision
missing machines dynamically.
2020-03-26 15:00:04 +01:00
Eelco Dolstra 5b731004da
Remove tabs 2020-03-26 14:59:52 +01:00
Nikola Knezevic 1bee6e3d8a Add debug logging
This will help us track potential problems with the plugin.
2020-03-26 13:27:44 +01:00
Nikola Knezevic 986fde8888 Refactor code
Extract the conditions before the loop, as they do not change due to channel
definition.
2020-03-26 13:27:44 +01:00
Nikola Knezevic 956f009672 Add documentation for SlackNotification plugin 2020-03-26 13:27:44 +01:00
Eelco Dolstra d0688b93e1
Merge remote-tracking branch 'origin/master' into flake 2020-03-26 11:44:11 +01:00
Graham Christensen 12cc46cdb3
fixup: hydra-init: correct reference to hydra-backill-ids 2020-03-24 11:22:14 -04:00
Eelco Dolstra c85ee736a4
Merge pull request #724 from Ma27/different-icons-for-timeout-and-loglimit
Use the stopsign as icon for timeouts and exceeded log-limits
2020-03-23 09:39:04 +01:00
Maximilian Bosch 73694087a0
Use the stopsign as icon for timeouts and exceeded log-limits
When I browse failed builds in a jobset-eval on Hydra, I regularly
mistake actual build-failures with temporary issues like timeouts (that
probably disappear at the next eval).

To prevent this kind of issue, I figured that using the stopsign-svg for
builds with timeouts or exceeded log-limits is a reasonable choice for
the following reasons:

* A user can now distinguish between actual build-errors (like
  compilation-failures or oversized outputs) and (usually) temporary issues
  (like a bloated log or a timeout).

* The stopsign is also used for aborted jobs that are shown in a
  different tab and can't be confused with timeouts for that reason.
2020-03-21 21:08:33 +01:00
Eelco Dolstra 5ae8b54d25 README.md: Link to the latest manual
(cherry picked from commit 455611b0b4e97290a858687142805babc9929673)
2020-03-13 21:33:22 +01:00
Eelco Dolstra be0ec2d223
Remove style.css
https://github.com/NixOS/nix/issues/3392
2020-03-13 16:17:48 +01:00
Nikola Knezevic 01ae944c80 Add host tag to InfluxDB metrics
This should help us discern machines in environments with multiple Hydra deployments.
2020-03-12 14:23:07 +01:00
Eelco Dolstra 101a9b3797
Run 'make check'
This got lost when I converted from releaseTools.nixBuild to
stdenv.mkDerivation. Doh!
2020-03-04 15:42:51 +01:00
Eelco Dolstra 4b5bb4e760
Merge remote-tracking branch 'origin/master' into flake 2020-03-04 15:28:23 +01:00
Eelco Dolstra 123bee1db5
Restore job type checking 2020-03-04 15:16:26 +01:00
Eelco Dolstra 69a6f3448a
Fix calling job functions
Fixes #718.
2020-03-04 15:16:04 +01:00
Eelco Dolstra 3cc1deb125
Merge pull request #721 from grahamc/one-by-one
hydra-evaluator: add a 'ONE_AT_A_TIME' evaluator style
2020-03-04 08:44:43 +01:00
Eelco Dolstra e6e4aa54c0
Merge pull request #722 from grahamc/fix-declarative-jobsets
Fix declarative jobsets due to `nix` command being hidden
2020-03-04 08:41:13 +01:00
Graham Christensen 994430b94b
treewide: allow nix command 2020-03-03 22:52:20 -05:00
Graham Christensen 117b9ecef1
Nix.pm: readNixFile: pass «--experimental-features nix-command»
Declarative jobsets were broken by the Nix update, causing
nix cat-file to break silently.

This commit restores declarative jobsets, based on top of a commit
making it easier to see what broke.
2020-03-03 22:36:21 -05:00
Graham Christensen 113a312f67
handleDeclarativeJobsetBuild: handle errors from readNixFile 2020-03-03 22:32:13 -05:00
Graham Christensen 5fae9d96a2
hydra-evaluator: add a 'ONE_AT_A_TIME' evaluator style
In the past, jobsets which are automatically evaluated are evaluated
regularly, on a schedule. This schedule means a new evaluation is
created every checkInterval seconds (assuming something changed.)

This model works well for architectures where our build farm can
easily keep up with demand.

This commit adds a new type of evaluation, called ONE_AT_A_TIME, which
only schedules a new evaluation if the previous evaluation of the
jobset has no unfinished builds.

This model of evaluation lets us have 'low-tier' architectures.

For example, we could now have a jobset for ARMv7l builds, where
the buildfarm only has a single, underpowered ARMv7l builder.
Configuring that jobset as ONE_AT_A_TIME will create an evaluation
and then won't schedule another evaluation until every job of
the existing evaluation is complete.

This way, the cache will have a complete collection of pre-built
software for some commits, but the underpowered architecture will
never become backlogged in ancient revisions.
2020-03-03 19:28:44 -05:00
Graham Christensen eaa65f51f4
hydra-evaluator: make the logic of the scheduler easier to read 2020-03-03 18:17:21 -05:00