Commit graph

2769 commits

Author SHA1 Message Date
Eelco Dolstra cd234f6a14
Merge pull request #529 from bennofs/feat-all-builds
feat: add /eval/<id>/builds endpoint
2018-11-19 17:36:19 +01:00
Eelco Dolstra f9870b5fce
Merge pull request #612 from input-output-hk/devops-1126
[DEVOPS-1126] throttle github status calls to remain under api ratelimits
2018-11-12 10:09:38 +01:00
Michael Bishop 7916c6b185 [DEVOPS-1126] throttle github status calls to remain under api ratelimits 2018-11-09 12:06:37 -04:00
Eelco Dolstra 1c44de1779
Merge pull request #567 from phile314/bitbucket_pulls
Add BitBucket pull request support
2018-11-06 10:48:37 +01:00
Antoine Eiche 0d2a2d8923 Add json output for the search API endpoint
This commit also add a test of this feature.

Note the search JSON output doesn't contain any jobs because they can
not be exported to JSON yet.

The JSON output on a search query matching a build looks like:

```
{
  "builds": [
    {
      "buildoutputs": {
        "out": {
          "path": "/nix/store/wdag3pznrvqk01byk989irg7rq3q2a2c-job"
        }
      },
      "finished": 0,
      "releasename": null,
      "starttime": null,
      "project": "sample",
      "buildproducts": {},
      "timestamp": 1541007629,
      "buildstatus": null,
      "nixname": "job",
      "drvpath": "/nix/store/n9zqndn7j7nyr6gg3bmxvw26cfmdwv2n-job.drv",
      "job": "job",
      "id": 1,
      "stoptime": null,
      "priority": 100,
      "system": "x86_64-linux",
      "jobsetevals": [
        1
      ],
      "jobset": "default",
      "buildmetrics": {}
    }
  ],
  "projects": [],
  "jobsets": [],
  "buildsdrv": []
}
```
2018-11-01 09:31:15 +01:00
Antoine Eiche dd1e4a92c0 api-test: make created project and jobset visible 2018-10-31 18:51:02 +01:00
Rob Vermaas adf59a3959
Merge pull request #604 from andir/single-build-restart
allow users with 'restart-jobs' role to restart individual builds
2018-10-31 12:26:35 +01:00
Michael Bishop 3ef253c65f
also add CatalystXRoleApplicator
f7fe61207b
2018-10-30 15:11:01 +01:00
Michael Bishop 6d693b6b2a
fix hydra build on nixpkgs master
copied from c501958535
2018-10-30 14:50:22 +01:00
Eelco Dolstra 8d26144121
Fix building against nix master 2018-10-30 14:41:21 +01:00
Antoine Eiche 68aad22d19 Add GitlabPulls input plugin 2018-10-15 14:45:26 +02:00
Andreas Rammhold 63a294d4ca
allow users with 'restart-jobs' role to restart individual builds 2018-10-04 21:59:42 +02:00
Eelco Dolstra e0f204f3da
Merge pull request #599 from tomberek/19.03_fixes
submodule fix
2018-10-03 09:20:02 +02:00
Tom Bereknyei 8bffbb7928
evalSettings deprecated, submodule fix 2018-10-03 00:45:42 -04:00
Eelco Dolstra 0bc548ee2d
Merge pull request #595 from kquick/darcs_cache_dir
Fix darcs input to use darcs-specific SCM cache dir.
2018-09-10 10:51:08 +02:00
Kevin Quick 35bcab74ed Fix darcs input to use darcs-specific SCM cache dir.
Currently re-using the git cache dir which could cause overlap problems.
2018-09-09 22:04:32 -07:00
Eelco Dolstra b7c42a38a3
Merge pull request #575 from Ma27/fix-tab-jobset-view
jobset view: jump back to the proper tab when displaying all failed jobsets
2018-09-06 21:11:59 +02:00
Eelco Dolstra b15288bea4
Merge pull request #590 from Taneb/one-runcommand
Allow for precisely one instance of RunCommand plugin
2018-08-21 17:00:23 +02:00
Nathan van Doorn a77954be4d Allow for precisely one instance of RunCommand plugin 2018-08-21 15:52:41 +01:00
Eelco Dolstra b542b73ced hydra-eval-jobs: Reinitialize Boehm GC in the child
This is necessary to prevent hangs in the garbage collector.
2018-08-10 20:25:43 +02:00
Eelco Dolstra 7876cf677c Increase maxHeapSize more gradually 2018-08-10 19:23:31 +02:00
Eelco Dolstra 4dca8fe14d
Gradually increase maxHeapSize 2018-08-07 11:47:53 +02:00
Eelco Dolstra e122d3bef3
RunCommand: Return metrics as a float
Apparently, DBIx::Class doesn't handle columns with type 'double
precision' properly.
2018-08-02 12:31:28 +02:00
Eelco Dolstra 6db2cbf094
Add a plugin to execute arbitrary commands when a build finishes
The plugin can be configured using one or more <runcommand> sections
in hydra.conf, e.g.

<runcommand>
  command = echo Build finished
</runcommand>

Optionally, the command can be executed for specific
projects/jobsets/jobs:

  job = patchelf:master:tarball

or

  job = patchelf:*:*

The default is *:*:*.

The command is executed with the environment variable $HYDRA_JSON
pointing to a JSON file containing info about the build, e.g.

  {
    "build": 3772978,
    "buildStatus": 0,
    "drvPath": "/nix/store/9y4h1fyx9pl3ic08i2f09239b90x1lww-patchelf-tarball-0.8pre894_ed92f9f.drv",
    "event": "buildFinished",
    "finished": 1,
    "job": "tarball",
    "jobset": "master",
    "metrics": [
      {
        "name": "random1",
        "unit": null,
        "value": "20282"
      },
      {
        "name": "random2",
        "unit": "KiB",
        "value": "6664"
      }
    ],
    "outputs": [
      {
        "name": "out",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f"
      }
    ],
    "products": [
      {
        "defaultPath": "",
        "fileSize": 148216,
        "name": "patchelf-0.8pre894_ed92f9f.tar.gz",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f/tarballs/patchelf-0.8pre894_ed92f9f.tar.gz",
        "productNr": 4,
        "sha1hash": "9f27d18382436a7f743f6c2f6ad66e1b536ab4c8",
        "sha256hash": "b04faef2916c411f10711b58ea26965df7cb860ca33a87f1e868051b874c44b3",
        "subtype": "source-dist",
        "type": "file"
      },
      {
        "defaultPath": "",
        "fileSize": 121279,
        "name": "patchelf-0.8pre894_ed92f9f.tar.bz2",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f/tarballs/patchelf-0.8pre894_ed92f9f.tar.bz2",
        "productNr": 3,
        "sha1hash": "7a664841fb779dec19023be6a6121e0398067b7c",
        "sha256hash": "c81e36099893f541a11480f869fcdebd2fad3309900519065c8745f614dd024a",
        "subtype": "source-dist",
        "type": "file"
      },
      {
        "defaultPath": "README",
        "fileSize": null,
        "name": "",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f",
        "productNr": 2,
        "sha1hash": null,
        "sha256hash": null,
        "subtype": "readme",
        "type": "doc"
      },
      {
        "defaultPath": "",
        "fileSize": 6230,
        "name": "README",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f/README",
        "productNr": 1,
        "sha1hash": "dc6bb09093183ab52d7e6a35b72d179869bd6fbf",
        "sha256hash": "5371aee9de0216b3ea2d5ea869da9d5ee441b99156a99055e7e11e7a705f7920",
        "subtype": "readme",
        "type": "doc"
      }
    ],
    "project": "patchelf",
    "startTime": 1533137091,
    "stopTime": 1533137094,
    "timestamp": 1533136076
  }

So for example, the following command:

  command = echo Build $(jq -r .build $HYDRA_JSON) \($(jq -r .project $HYDRA_JSON):$(jq -r .jobset $HYDRA_JSON):$(jq -r .job $HYDRA_JSON)\) finished, metrics: $(jq -r '.metrics[].value' $HYDRA_JSON).

will print

  Build 3772978 (patchelf:master:tarball) finished, metrics: 20282 6664.
2018-08-01 19:43:50 +02:00
Eelco Dolstra 8e17a413f5
hydra-notify step-finished: Don't barf if the step has no log file 2018-08-01 17:17:46 +02:00
Eelco Dolstra dca6c5a983
hydra-eval-jobs: Filter ANSI escape sequences 2018-08-01 17:06:32 +02:00
Eelco Dolstra 1898e31ed8
hydra-eval-jobset: Respect email_notification 2018-08-01 16:56:53 +02:00
Eelco Dolstra a71e6301d2
Fix Hydra VM tests 2018-07-31 21:49:05 +02:00
Eelco Dolstra 1a899890ac
Use nixUnstable 2018-07-31 21:19:10 +02:00
Eelco Dolstra 162d671c48
Revert "Fix broken build.x86_64-linux job (#573)"
This reverts commit c2e2b42877, which
breaks building with nixUnstable.
2018-07-31 10:23:41 +02:00
Maximilian Bosch 37d35ac51e
jobset view: jump back to the proper tab when displaying all failed jobsets
When I press "n builds omitted" I get back to the first tab of a jobset.
This is extremely counter-intuitive, instead this notice should link to
the currently opened tab.
2018-07-15 17:52:14 +02:00
Jacob Mitchell c2e2b42877 Fix broken build.x86_64-linux job (#573)
The job has been failing since https://hydra.nixos.org/eval/1461286
with the following error:

    hydra-eval-jobs.cc:278:17: error: 'evalSettings' was not declared in this scope
                     evalSettings.restrictEval = true;
                     ^~~~~~~~~~~~

This is likely due to a typo in 0882519 where that line and the
corresponding comment were moved, and `settings` was changed in that
one place to `evalSettings`.

I reproduced the error by running `nix-build release.nix -A
build.x86_64-linux` on my machine, and this small change fixes it.
2018-07-07 19:15:01 +01:00
Kevin Quick e523e8a643 Additional helpful information in error messages. 2018-06-29 17:41:23 -07:00
Philipp Hausmann f738f6feb3 Add BitBucket pull request support 2018-06-22 11:22:54 +02:00
Rob Vermaas 879333fd3c
Make /api/push-github public for private Hydra instances. This makes it possible to use webhooks to trigger evaluations for private Hydra instances. 2018-06-14 17:23:01 +02:00
Eelco Dolstra cff52ab151
Merge pull request #556 from AmineChikhaoui/machine-types-stats
add some more statistics about runnable/running builds per arch/system features
2018-06-12 14:02:57 +02:00
AmineChikhaoui 298e99449b
add metrics for hydra notifications. 2018-06-12 13:00:39 +01:00
AmineChikhaoui 99e6c19672
add some more statistics about runnable/running builds per arch/system
features.
2018-06-12 13:00:00 +01:00
Eelco Dolstra b11789f399
Add boost dependency 2018-06-05 13:24:23 +02:00
Eelco Dolstra 0882519b10
hydra-eval-jobs: Ugly hackery to reduce memory usage
You can now set 'evaluator_max_heap_size' to make hydra-eval-jobs
restart itself if the Boehm heap exceeds the specified size.

For example, with 'evaluator_max_heap_size = 256000000',

  $ hydra-eval-jobs '<nixpkgs/pkgs/top-level/release.nix>' -I nixpkgs=channel:nixos-17.09

has a max RSS of .56 GiB rather than 4.7 GiB.

Unfortunately it doesn't help much for the NixOS jobsets because of
the "tested" job which requires a huge amount of memory all by itself.
2018-06-05 13:24:23 +02:00
Eelco Dolstra 821f8bf5a0
release.nix: Use fetchGit to get Nixpkgs 2018-06-05 13:24:23 +02:00
Eelco Dolstra 39eca8bef1
Fix build against Nix master 2018-06-05 13:24:23 +02:00
Eelco Dolstra 7f6d52f4c3
Merge pull request #562 from grahamc/141-redux
Allow channels to be usefully imported by other nix expressions.
2018-05-25 18:30:15 +02:00
Shea Levy 70eb67fe5b
Allow channels to be usefully imported by other nix expressions. 2018-05-25 11:22:20 -04:00
Eelco Dolstra 4013e83e3f
Merge pull request #560 from Synthetica9/nixpkgs-40521-hydra
Remove "bashism"
2018-05-17 12:03:19 +02:00
Patrick Hilhorst d27732bdf0
Remove "bashism"
Replace `bash <(...)` with `... | bash`

See also: NixOS/nixpkgs#40521
2018-05-16 23:56:40 +02:00
Eelco Dolstra 09a4c9fad2
hydra-evaluator: Allow setting the maximum number of concurrent evaluations 2018-05-16 14:22:00 +02:00
Eelco Dolstra 4e27796eba
Allow setting GC_INITIAL_HEAP_SIZE for hydra-eval-jobs
This cannot be done in the hydra-evaluator systemd unit, since then
every other Nix process (e.g. hydra-evaluator and nix-prefetch-*) will
also allocate the specified heap size, probably leading to OOM.
2018-05-16 14:14:53 +02:00
Eelco Dolstra 691f7e168c
Fix building on Nixpkgs 17.09 2018-05-10 15:11:24 +02:00
Eelco Dolstra 215b724e44
Fix passing arguments to the reproduction script 2018-05-10 15:11:08 +02:00