lix-project/hydra
3
0
Fork 6
Code Issues Pull requests Packages Projects Releases Wiki Activity
2716 commits 1 branch 0 tags 20 MiB
Commit graph

2228 commits

Author SHA1 Message Date
Samuel Dionne-Riel 9986053e73 Controllers: allows lazy tabs to return custom errors. 2018-12-01 13:40:41 -05:00
Antoine Eiche d9253543e4 plugin/GitLabPulls: support for using a personal access token (PAT) 
In order to access protected or private repositories. Using the target
repository URL along with the merge-request ref instead of the source
repository url and branch is necessary to avoid running into issues if
the source repository is not actually accessible to the user Hydra is
authenticating as.

Thanks Alexei Robyn for this patch.
 2018-11-20 16:27:40 +01:00
Rob Vermaas 14d5577bf8 Add duration to Slack notification. 2018-11-20 14:57:50 +01:00
Eelco Dolstra cd234f6a14
Merge pull request #529 from bennofs/feat-all-builds 
feat: add /eval/<id>/builds endpoint
 2018-11-19 17:36:19 +01:00
Michael Bishop 7916c6b185 [DEVOPS-1126] throttle github status calls to remain under api ratelimits 2018-11-09 12:06:37 -04:00
Eelco Dolstra 1c44de1779
Merge pull request #567 from phile314/bitbucket_pulls 
Add BitBucket pull request support
 2018-11-06 10:48:37 +01:00
Antoine Eiche 0d2a2d8923 Add json output for the search API endpoint 
This commit also add a test of this feature.

Note the search JSON output doesn't contain any jobs because they can
not be exported to JSON yet.

The JSON output on a search query matching a build looks like:

```
{
  "builds": [
    {
      "buildoutputs": {
        "out": {
          "path": "/nix/store/wdag3pznrvqk01byk989irg7rq3q2a2c-job"
        }
      },
      "finished": 0,
      "releasename": null,
      "starttime": null,
      "project": "sample",
      "buildproducts": {},
      "timestamp": 1541007629,
      "buildstatus": null,
      "nixname": "job",
      "drvpath": "/nix/store/n9zqndn7j7nyr6gg3bmxvw26cfmdwv2n-job.drv",
      "job": "job",
      "id": 1,
      "stoptime": null,
      "priority": 100,
      "system": "x86_64-linux",
      "jobsetevals": [
        1
      ],
      "jobset": "default",
      "buildmetrics": {}
    }
  ],
  "projects": [],
  "jobsets": [],
  "buildsdrv": []
}
```
 2018-11-01 09:31:15 +01:00
Rob Vermaas adf59a3959
Merge pull request #604 from andir/single-build-restart 
allow users with 'restart-jobs' role to restart individual builds
 2018-10-31 12:26:35 +01:00
Eelco Dolstra 8d26144121
Fix building against nix master 2018-10-30 14:41:21 +01:00
Antoine Eiche 68aad22d19 Add GitlabPulls input plugin 2018-10-15 14:45:26 +02:00
Andreas Rammhold 63a294d4ca
allow users with 'restart-jobs' role to restart individual builds 2018-10-04 21:59:42 +02:00
Tom Bereknyei 8bffbb7928
evalSettings deprecated, submodule fix 2018-10-03 00:45:42 -04:00
Kevin Quick 35bcab74ed Fix darcs input to use darcs-specific SCM cache dir. 
Currently re-using the git cache dir which could cause overlap problems.
 2018-09-09 22:04:32 -07:00
Eelco Dolstra b7c42a38a3
Merge pull request #575 from Ma27/fix-tab-jobset-view 
jobset view: jump back to the proper tab when displaying all failed jobsets
 2018-09-06 21:11:59 +02:00
Nathan van Doorn a77954be4d Allow for precisely one instance of RunCommand plugin 2018-08-21 15:52:41 +01:00
Eelco Dolstra b542b73ced hydra-eval-jobs: Reinitialize Boehm GC in the child 
This is necessary to prevent hangs in the garbage collector.
 2018-08-10 20:25:43 +02:00
Eelco Dolstra 7876cf677c Increase maxHeapSize more gradually 2018-08-10 19:23:31 +02:00
Eelco Dolstra 4dca8fe14d
Gradually increase maxHeapSize 2018-08-07 11:47:53 +02:00
Eelco Dolstra e122d3bef3
RunCommand: Return metrics as a float 
Apparently, DBIx::Class doesn't handle columns with type 'double
precision' properly.
 2018-08-02 12:31:28 +02:00
Eelco Dolstra 6db2cbf094
Add a plugin to execute arbitrary commands when a build finishes 
The plugin can be configured using one or more <runcommand> sections
in hydra.conf, e.g.

<runcommand>
  command = echo Build finished
</runcommand>

Optionally, the command can be executed for specific
projects/jobsets/jobs:

  job = patchelf:master:tarball

or

  job = patchelf:*:*

The default is *:*:*.

The command is executed with the environment variable $HYDRA_JSON
pointing to a JSON file containing info about the build, e.g.

  {
    "build": 3772978,
    "buildStatus": 0,
    "drvPath": "/nix/store/9y4h1fyx9pl3ic08i2f09239b90x1lww-patchelf-tarball-0.8pre894_ed92f9f.drv",
    "event": "buildFinished",
    "finished": 1,
    "job": "tarball",
    "jobset": "master",
    "metrics": [
      {
        "name": "random1",
        "unit": null,
        "value": "20282"
      },
      {
        "name": "random2",
        "unit": "KiB",
        "value": "6664"
      }
    ],
    "outputs": [
      {
        "name": "out",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f"
      }
    ],
    "products": [
      {
        "defaultPath": "",
        "fileSize": 148216,
        "name": "patchelf-0.8pre894_ed92f9f.tar.gz",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f/tarballs/patchelf-0.8pre894_ed92f9f.tar.gz",
        "productNr": 4,
        "sha1hash": "9f27d18382436a7f743f6c2f6ad66e1b536ab4c8",
        "sha256hash": "b04faef2916c411f10711b58ea26965df7cb860ca33a87f1e868051b874c44b3",
        "subtype": "source-dist",
        "type": "file"
      },
      {
        "defaultPath": "",
        "fileSize": 121279,
        "name": "patchelf-0.8pre894_ed92f9f.tar.bz2",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f/tarballs/patchelf-0.8pre894_ed92f9f.tar.bz2",
        "productNr": 3,
        "sha1hash": "7a664841fb779dec19023be6a6121e0398067b7c",
        "sha256hash": "c81e36099893f541a11480f869fcdebd2fad3309900519065c8745f614dd024a",
        "subtype": "source-dist",
        "type": "file"
      },
      {
        "defaultPath": "README",
        "fileSize": null,
        "name": "",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f",
        "productNr": 2,
        "sha1hash": null,
        "sha256hash": null,
        "subtype": "readme",
        "type": "doc"
      },
      {
        "defaultPath": "",
        "fileSize": 6230,
        "name": "README",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f/README",
        "productNr": 1,
        "sha1hash": "dc6bb09093183ab52d7e6a35b72d179869bd6fbf",
        "sha256hash": "5371aee9de0216b3ea2d5ea869da9d5ee441b99156a99055e7e11e7a705f7920",
        "subtype": "readme",
        "type": "doc"
      }
    ],
    "project": "patchelf",
    "startTime": 1533137091,
    "stopTime": 1533137094,
    "timestamp": 1533136076
  }

So for example, the following command:

  command = echo Build $(jq -r .build $HYDRA_JSON) \($(jq -r .project $HYDRA_JSON):$(jq -r .jobset $HYDRA_JSON):$(jq -r .job $HYDRA_JSON)\) finished, metrics: $(jq -r '.metrics[].value' $HYDRA_JSON).

will print

  Build 3772978 (patchelf:master:tarball) finished, metrics: 20282 6664.
 2018-08-01 19:43:50 +02:00
Eelco Dolstra 8e17a413f5
hydra-notify step-finished: Don't barf if the step has no log file 2018-08-01 17:17:46 +02:00
Eelco Dolstra dca6c5a983
hydra-eval-jobs: Filter ANSI escape sequences 2018-08-01 17:06:32 +02:00
Eelco Dolstra 1898e31ed8
hydra-eval-jobset: Respect email_notification 2018-08-01 16:56:53 +02:00
Eelco Dolstra 162d671c48
Revert "Fix broken build.x86_64-linux job (#573)" 
This reverts commit c2e2b42877, which
breaks building with nixUnstable.
 2018-07-31 10:23:41 +02:00
Maximilian Bosch 37d35ac51e
jobset view: jump back to the proper tab when displaying all failed jobsets 
When I press "n builds omitted" I get back to the first tab of a jobset.
This is extremely counter-intuitive, instead this notice should link to
the currently opened tab.
 2018-07-15 17:52:14 +02:00
Jacob Mitchell c2e2b42877 Fix broken build.x86_64-linux job (#573) 
The job has been failing since https://hydra.nixos.org/eval/1461286
with the following error:

    hydra-eval-jobs.cc:278:17: error: 'evalSettings' was not declared in this scope
                     evalSettings.restrictEval = true;
                     ^~~~~~~~~~~~

This is likely due to a typo in 0882519 where that line and the
corresponding comment were moved, and `settings` was changed in that
one place to `evalSettings`.

I reproduced the error by running `nix-build release.nix -A
build.x86_64-linux` on my machine, and this small change fixes it.
 2018-07-07 19:15:01 +01:00
Kevin Quick e523e8a643 Additional helpful information in error messages. 2018-06-29 17:41:23 -07:00
Philipp Hausmann f738f6feb3 Add BitBucket pull request support 2018-06-22 11:22:54 +02:00
Rob Vermaas 879333fd3c
Make /api/push-github public for private Hydra instances. This makes it possible to use webhooks to trigger evaluations for private Hydra instances. 2018-06-14 17:23:01 +02:00
AmineChikhaoui 298e99449b
add metrics for hydra notifications. 2018-06-12 13:00:39 +01:00
AmineChikhaoui 99e6c19672
add some more statistics about runnable/running builds per arch/system 
features.
 2018-06-12 13:00:00 +01:00
Eelco Dolstra 0882519b10
hydra-eval-jobs: Ugly hackery to reduce memory usage 
You can now set 'evaluator_max_heap_size' to make hydra-eval-jobs
restart itself if the Boehm heap exceeds the specified size.

For example, with 'evaluator_max_heap_size = 256000000',

  $ hydra-eval-jobs '<nixpkgs/pkgs/top-level/release.nix>' -I nixpkgs=channel:nixos-17.09

has a max RSS of .56 GiB rather than 4.7 GiB.

Unfortunately it doesn't help much for the NixOS jobsets because of
the "tested" job which requires a huge amount of memory all by itself.
 2018-06-05 13:24:23 +02:00
Eelco Dolstra 39eca8bef1
Fix build against Nix master 2018-06-05 13:24:23 +02:00
Shea Levy 70eb67fe5b
Allow channels to be usefully imported by other nix expressions. 2018-05-25 11:22:20 -04:00
Patrick Hilhorst d27732bdf0
Remove "bashism" 
Replace `bash <(...)` with `... | bash`

See also: NixOS/nixpkgs#40521
 2018-05-16 23:56:40 +02:00
Eelco Dolstra 09a4c9fad2
hydra-evaluator: Allow setting the maximum number of concurrent evaluations 2018-05-16 14:22:00 +02:00
Eelco Dolstra 4e27796eba
Allow setting GC_INITIAL_HEAP_SIZE for hydra-eval-jobs 
This cannot be done in the hydra-evaluator systemd unit, since then
every other Nix process (e.g. hydra-evaluator and nix-prefetch-*) will
also allocate the specified heap size, probably leading to OOM.
 2018-05-16 14:14:53 +02:00
Eelco Dolstra 215b724e44
Fix passing arguments to the reproduction script 2018-05-10 15:11:08 +02:00
Michael Bishop 4015bca225
fix a stray newline in a path 2018-03-16 13:04:48 -03:00
Eelco Dolstra 85524b2d0f
Disable jobset deletion 
This is a good way to make Hydra hang. (E.g. we had a deletion of
nixos:gcc-7 running for > 12 hours and blocking UPDATE statements from
hydra-queue-runner.) Generally it's better to just disable/hide an old
jobset anyway.
 2018-03-13 14:05:55 +01:00
Eelco Dolstra 02888105a8
Add a hydra.conf option to enable email notification 
Note that it's disabled by default.
 2018-03-12 14:04:23 +01:00
Eelco Dolstra c0fac52872
Add some debug code 2018-03-07 10:23:43 +01:00
Eelco Dolstra 5a1f2a50e5
Handle derivations with system type 'builtin' 
Fixes #540.
 2018-03-07 10:22:35 +01:00
Eelco Dolstra 68afa2bf6f
Dump more system info in /queue-runner-status 2018-03-07 10:06:56 +01:00
Philipp Hausmann 3cf2ffe367 [ Fix #531 ] Trim github pr plugin store paths. 
nix-store --add returns a trailing LF which caused wrong include
paths for the github PR plugin JSON input.
 2018-02-12 17:59:56 +01:00
Graham Christensen aeca462de0
Typo: on the build view page: Yu -> You 2018-02-08 12:37:21 -05:00
Eelco Dolstra fd463ff403
Remove nix-install-package support 
'nix-install-package' no longer exists in Nix 1.12.

Closes #268.
 2018-01-15 14:27:58 +01:00
Benno Fünfstück 3ab5d299c1 feat: add /eval/<id>/builds endpoint 
This endpoint allows efficient retrieval of all the builds in an
evaluation, without making a request for each single build.
 2018-01-14 01:29:37 +01:00
Graham Christensen 83a48d2d4f
Add a restart-jobs role 
Frequently users want Hydra access just to restart jobs. However,
prior to this commit the only way to grant that access was by giving
them full Admin access which isn't necessarily what we want to do.

By having a restart-jobs role, we can grant this privilege to users
who are known to the community and want to help, but aren't long-time
members.

I haven't tested this commit, but it looks good to me...
 2017-12-30 08:28:23 -05:00
Eelco Dolstra 8913c682cf
Doh 2017-12-25 15:02:17 +01:00
Eelco Dolstra 0d79f16fd5
git-export -> source 2017-12-25 14:31:30 +01:00
Eelco Dolstra 7ccec0900b Fix links to steps in the machines page 2017-12-07 16:20:23 +01:00
Eelco Dolstra e9670641ec
Distinguish build step states 
The web interface now shows whether a build step is connecting,
copying inputs/outputs, building, etc.
 2017-12-07 15:35:31 +01:00
Eelco Dolstra 457483ba0e
Don't lock the BuildSteps table when inserting 
Instead, optimistically insert a row and retry if there is a conflict.
 2017-12-07 14:41:29 +01:00
Eelco Dolstra 2660c3cc54
Remove redundant check 2017-12-05 16:00:18 +01:00
AmineChikhaoui 52e0ebcfed
hide disabled jobsets by default. 2017-12-05 14:07:58 +01:00
Eelco Dolstra 364e21919a
hydra-eval-{jobs,jobset}: Pass file name as <...> 2017-11-28 16:51:00 +01:00
Eelco Dolstra b7bc4384b7
Fix serving plain text files 2017-11-21 17:55:52 +01:00
Eelco Dolstra 5986aae704
Add option eval_substituter 
When using the "build" or "sysbuild" jobset input types in conjunction
with a binary cache store, the evaluator needs to be able to fetch
store paths from the binary cache. Typical usage:

  store_uri = s3://nix-test-cache?secret-key=...
  eval_substituter = s3://nix-test-cache

Also, the public key of the binary cache must be added to
binary-cache-public-keys in nix.conf, otherwise the local nix-daemon
won't allow the store paths to be copied over.
 2017-11-20 16:12:20 +01:00
Eelco Dolstra 7857f83251
Serve directories and symlinks correctly 2017-11-14 17:17:07 +01:00
Eelco Dolstra 100249c066
Use git rev-list --count 2017-11-14 16:18:54 +01:00
Eelco Dolstra 2cdc84f34f
Fix build against Nix master 
Also, remove support in hydra-eval-jobs for multiple jobset input
alternatives. The web interface hasn't supported this in a long
time. Thus we can use the regular "--arg" handler.
 2017-10-26 13:10:14 +02:00
Eelco Dolstra eef0d8861b
Remove test line 2017-10-19 13:17:29 +02:00
Eelco Dolstra b04dc6c76e
Fix root creation when the root already exists but is owned by another user 2017-10-19 12:28:38 +02:00
Eelco Dolstra b904f25247 Hide contents links 2017-10-18 15:57:43 +02:00
Eelco Dolstra d8d71ea4a3 hydra-server: Use server_store_uri instead of store_uri, if set 
The store_uri may contain parameters that cause Nix to fail when
running under the hydra-server user (e.g. the signing key).
 2017-10-18 15:33:55 +02:00
Eelco Dolstra 62c6cca937
Make one-click installs work for binary cache stores 
This requires setting binary_cache_public_uri in hydra.conf.
 2017-10-18 14:09:28 +02:00
Eelco Dolstra 44ef743d84
Hide some things that require a local store 2017-10-18 13:44:41 +02:00
Eelco Dolstra 40a9b70df2
Shut up warning 2017-10-18 13:39:39 +02:00
Eelco Dolstra 30943dd217
Detect MIME type 2017-10-18 13:38:34 +02:00
Eelco Dolstra 42fbde0383
Use "nix cat-store" to serve files from the Nix store 
This makes downloading/viewing build results work with binary cache
stores. For good performance, this should be used in conjunction with
ca580bec35,
i.e. you should set store_uri to something like

  s3://my-cache?local-nar-cache=/tmp/nar-cache

to cache NARs between requests.
 2017-10-18 12:48:31 +02:00
Eelco Dolstra bc60fccf78
Remove remaining references to store_mode etc. 2017-10-18 12:23:07 +02:00
Eelco Dolstra cc64e51f75
USER -> LOGNAME for consistency 
Don't remember why we use LOGNAME. Also ensure that it's set.
 2017-10-18 11:23:00 +02:00
Will Dietz c81594f470 hydra-queue-runner: ensure roots directory exists 
Fixes #513
 2017-10-17 13:04:56 -05:00
Eelco Dolstra 45b138373b
hydra-queue-runner: Write GC roots for outputs paths 
We lost this behaviour somewhere. So build outputs could be GC'ed when
running the collector with --option gc-keep-outputs false.
 2017-10-12 18:55:38 +02:00
Remy Goldschmidt eb7631fb9d hydra-create-user now has --password-hash option (#504) 
When creating a Hydra user with the `hydra-create-user` command, you can now
provide a SHA1 password hash with the `--password-hash` flag. This is useful for
the upcoming work on Fully Declarative Hydra, since the end user should not have
to specify plaintext passwords in their `configuration.nix` file.
 2017-10-08 11:55:51 +01:00
Rob Vermaas b645b03536
Fix string comparison 2017-09-25 11:54:55 +00:00
Eelco Dolstra 27103398c9
Make maxLogSize configurable 2017-09-22 15:23:58 +02:00
Rob Vermaas 2318baaade
Add 'readonly_ips' option, to allow readonly access for certain IPs when 'private' option is enabled. 2017-09-22 11:36:49 +00:00
Eelco Dolstra b828224fee
Periodically close RemoteStore connections 
This prevents an accumulation of temproots. See
89dc62c174.
 2017-09-14 18:16:33 +02:00
Eelco Dolstra 6517446c34
Update to latest nixUnstable 2017-09-14 17:22:48 +02:00
Eelco Dolstra d676c91bd5
Return Builds.drvpath in the JSON API 
E.g.

  curl -H 'Accept: application/json' http://127.0.0.4/build/3772858 | jq -r .drvpath
  /nix/store/97vla79lavlcsk8milskr06rfm34bskx-patchelf-0.8pre843_f4522f4.drv
 2017-09-12 18:42:18 +02:00
Eelco Dolstra 4af97c57f5
Acquire the send lock only while actually sending 
Thus, we no longer hold the send lock while substituting missing paths
on the build machine. This is a good thing in particular for macOS
builders which have a tendency to hang forever in curl downloads.
 2017-09-01 16:28:49 +02:00
Eelco Dolstra 50ab80caf2
Don't wait forever to acquire the send lock 2017-09-01 15:29:06 +02:00
Eelco Dolstra b405837d4d Merge pull request #474 from zraexy/patch-1 
Make all content of rotated table header clickable
 2017-08-22 22:37:23 +02:00
Eelco Dolstra 337a72ef4d
Fix displaying errors from JSON requests 2017-08-17 15:32:46 +02:00
montag451 48609664a5 Fixes NixOS/hydra#402 (#424) 2017-08-10 14:16:24 +02:00
Eelco Dolstra 7c976d2aec
hydra-queue-runner: Make build notification more reliable 
Previously, when hydra-queue-runner was restarted, any pending "build
finished" notifications were lost. Now hydra-queue-runner marks
finished but unnotified builds in the database and uses that to run
pending notifications at startup.
 2017-07-26 15:17:51 +02:00
Will Dietz 719df63190 queue-monitor: never move lastBuildId forward without processing jobs. 2017-07-25 20:05:37 -05:00
Eelco Dolstra e117d85c2a
hydra-queue-runner: Set a thread title for the builder threads 
This should make debugging slightly easier.
 2017-07-25 15:59:41 +02:00
Shea Levy a6d9201947 GithubStatus: Allow config to specify the integration description and context 2017-07-24 15:44:10 -04:00
Eelco Dolstra e78b9fd4ee
hydra-queue-runner: Allow concurrent notifications 
The queue runner can now run up to ‘max-concurrent-notifications’ in
parallel (default is 2). This is useful when some hydra-notify
invocations can take a long time to complete (e.g. because they need
to compress a giant build log) and we don't want this to block all
other notifications.
 2017-07-24 16:35:34 +02:00
Eelco Dolstra bba383bf1b
hydra-queue-runner: Keep some notification statistics 2017-07-24 16:26:44 +02:00
Eelco Dolstra f46a21e16e
Slight cleanup 2017-07-21 17:22:11 +02:00
Eelco Dolstra dc5e0b120a
Fix a race that can cause hydra-queue-runner to ignore newly added builds 
As @dtzWill discovered, with the concurrent hydra-evaluator, there can
be multiple active transactions adding builds to the database. As a
result, builds can become visible in a non-monotonically increasing
order, breaking the queue monitor's assumption that build IDs only go
up.

The fix is to have hydra-eval-jobset provide the lowest build ID it
just added in the builds_added notification, and have the queue
monitor check from there.

Fixes #496.
 2017-07-21 14:34:48 +02:00
Eelco Dolstra b0432c7762
Sync with nixUnstable 2017-07-21 13:09:08 +02:00
Eelco Dolstra 6fc851d376 Improve erorr message 2017-07-17 14:10:34 +02:00
Eelco Dolstra 66ae66024e Sync with latest Nix 2017-07-17 11:38:58 +02:00
Eelco Dolstra 064fd20b6f Merge pull request #494 from Chakerbh/master 
Use build id for bitbucket Key
 2017-07-11 17:41:22 +02:00
Chaker Benhamed c5666095a1 Use build id for bitbucket Key 2017-07-11 16:13:56 +01:00
Eelco Dolstra f95516a82b Merge pull request #466 from cyraxjoe/configurable-git-input 
Allow to configure the timeout value for the GitInput plugin in different places.
 2017-07-07 11:03:06 +02:00
Chaker Benhamed 8da3ca6e94 Skip the input if the emailResponsible field is disabled 2017-07-05 15:51:20 +01:00
Chaker Benhamed fd8cb39a11 Add BitBucket plugin 
This plugin will post to the build status system in BitBucket. In order
to use it you need to add to ExtraConfig
<bitbucket>
username = bitbucket_username
password = bitbucket_password
</bitbucket>
You can use an application password https://blog.bitbucket.org/2016/06/06/app-passwords-bitbucket-cloud/
 2017-07-04 10:07:25 +01:00
Eelco Dolstra 803833aba7 Disable the build-queued hook 
This can take an excessive amount of time. For example, on
hydra.nixos.org, a call to hydra-notify takes 0.7s even if there are
no plugins. So for an eval with ~45K new builds, the calls to
hydra-notify add up to about 9 hours.

The proper fix would be to pass a list of build IDs, or an eval ID.
 2017-06-22 15:09:04 +02:00
Shea Levy 207d2dd10c GithubStatus: Don't send a pending status for already-finished builds 
Fixes #488
 2017-06-21 14:51:09 -04:00
michael bishop 6e4a7a30f7
fix github pulls json to be deterministic 2017-06-21 09:35:26 -03:00
Eelco Dolstra f0d24af940
Fix "Evaluation pending since..." message even though the jobset has been evaluated 2017-06-12 14:15:00 +02:00
Shea Levy dfb3a52f67 Fix build-queued plugin hook 2017-05-24 11:12:48 -04:00
Shea Levy c40077ece1 Fix tests 2017-05-24 10:00:38 -04:00
Shea Levy 582c399420 Add buildQueued plugin hook 2017-05-24 09:45:31 -04:00
Shea Levy 3d044d2ec2 Github status API: Pull authorization from github_authorization by default 2017-05-24 08:53:29 -04:00
Shea Levy 08708e899a GithubPulls: Add 'revision' to the input 2017-05-24 07:33:06 -04:00
Shea Levy c6f0753435 Add GithubPulls input plugin. 
This can be used with declarative projects to build PRs.

The github_authorization section should contain verbatim Authorization header contents keyed by repo owner for private repos
 2017-05-24 05:31:27 -04:00
Eelco Dolstra 941665044e
/api/scmdiff: Set correct MIME type 
It was set to application/json but should be text/plain.
 2017-05-05 15:58:38 +02:00
Joel Rivera a39ffba042 Include the project and jobset names in the configuration blocks 
to set the timeout for a specific input in the GitInput plugin.

Remove the comments about using inputs with spaces.
 2017-04-28 21:43:35 -05:00
Eelco Dolstra 1f94f03699
Fix build 2017-04-26 15:11:12 +02:00
Shea Levy 86aa851d8d Merge branch 'circleci-plugin' 2017-04-24 11:28:09 -04:00
zraexy b3c4c4ef0d Make all content of rotated table header clickable 2017-04-20 15:58:17 -08:00
Shea Levy 4c26cf0c81 getHydraConfig: Allow multiple includes 2017-04-20 08:00:09 -04:00
Shea Levy f4b4ee5751 Add CircleCI plugin. 2017-04-19 15:20:09 -04:00
Eelco Dolstra cc85208fe4
Fix build 2017-04-18 20:50:18 +02:00
Joel Rivera af2f8122e3 Allow to configure the timeout value for the GitInput plugin in different places. 
1. From the hydra configuration file.

    The configuration is loaded from the "git-input" block.
    Currently only the "timeout" variable is been looked up in the file.

    <git-input>
      # general timeout
      timeout = 400
      <input-name>
        # specific timeout for a particular input name
        timeout = 400
      </input-name>

      # use quotes when the input name has spaces
      <"foot with spaces">
        # specific timeout for a particular input name
        timeout = 400
      </"foo with spaces">
    </git-input>

2. As an argument in the input value after the repo url and branch (and after the deepClone if is defined)
   "timeout=<value>"

The preference on which value is used:

 1. input value
 2. Block with the name of the input in the <git-input> block
 3. "timeout" inside the <git-input> block
 4. Default value of 600 seconds. (original hard-coded value)

The code is generalized for more values to be configured, it might be too much
for a single value on a single plugin.
 2017-04-13 19:10:53 -05:00
Shea Levy fed37ea640 nix-prefetch-git: Only remove .git, not e.g. .gitkeep (#454) 2017-04-12 21:08:04 +02:00
Eelco Dolstra c2c2633e50
Fix link to cached build log 
Fixes #462.
 2017-04-11 14:25:48 +02:00
Eelco Dolstra 433b94ee68
Fix Mercurial diff 
It only showed the log messages of $rev1 and $rev2. Now it shows the
range $rev1::$rev2, excluding $rev1.
 2017-04-11 13:39:54 +02:00
Eelco Dolstra 585d3274e3
Better error message if jobset doesn't evaluate to an attrset 2017-04-11 13:01:15 +02:00
Shea Levy 7e3f12fe10 Merge branch 'decl-project-remote-store' 2017-04-10 11:36:41 -04:00
Eelco Dolstra 426aea1236
hydra-queue-runner: Allow multiple concurrent daemon connections 2017-04-06 18:50:53 +02:00
Eelco Dolstra 5810042a3b
Periodically clear Store's path info cache 
Otherwise the queue runner can consider paths as valid that have been
garbage-collected since the first time it queried them.
 2017-04-06 17:20:23 +02:00
Eelco Dolstra 8364f4ec70
Upload log files to the right location 
We were mixing up builds and steps. So for example

  https://cache.nixos.org/log/2w66a98iqbjdppc5s2b8qvhi3gprvy45-freecell-solver-4.8.0.drv

at the moment contains the log for
/nix/store/442r9d5ihbcpgq8q9dhijhvhlmplzp96-perl-namespace-autoclean-0.28.drv
because the latter is a step in http://hydra.nixos.org/build/51300420.
Oops.
 2017-04-06 13:05:30 +02:00
Eelco Dolstra fec895a642
hydra-server: Support logs in S3 2017-04-05 17:55:56 +02:00
Eelco Dolstra 4f11cf45dc
Fix build cancellation 
We nowadays ignore SIGINT, so the sshd child process inherited this
and ignored SIGINT as well.
 2017-04-05 11:01:57 +02:00
Shea Levy a738f826e8 declarative projects: Pull jobset spec build from the remote store 
Fixes #443
 2017-04-01 10:38:49 -04:00
Will Dietz 90870f1416 NixExprs.pm: encode default.nix as utf-8 to fix missing chars/data. (#455) 
cc #254
 2017-04-01 11:18:51 +02:00
Eelco Dolstra 147ba3ca31
Set proper charset on log files 2017-03-31 18:00:08 +02:00
Shea Levy ec1c8efd43 Merge branch 'eval-no-build-hook' 2017-03-30 13:19:37 -04:00
Eelco Dolstra 8771f7f913 Merge pull request #382 from shlevy/cached-build-notifications 
Send BuildFinished notifications on cached build results.
 2017-03-29 18:52:20 +02:00
Eelco Dolstra 57bc0eaead
hydra-queue-runner: Limit concurrent database connections 
Adding a 96-core aarch64 build machine to the build farm caused the
potential number of database connections to increase a lot, so we
started hitting the Postgres connection limit.
 2017-03-21 11:53:46 +01:00
Shea Levy 3b5fd2d857 hydra-eval-jobs: Disable the build hook. 
For some reason this causes an unexpected EOF when doing import-from-derivation.
 2017-03-20 12:57:05 -04:00
Eelco Dolstra 150228d7de
Upload build logs to the binary cache 2017-03-15 16:59:57 +01:00
Eelco Dolstra 7e6486e694
Move log compression to a plugin 2017-03-15 16:59:57 +01:00
Eelco Dolstra 285754aff6
hydra-evaluator improvements 
* The "Jobset" page now shows when evaluations are in progress (rather
  than just pending).

* Restored the ability to do a single evaluation from the command line
  by doing "hydra-evaluator <project> <jobset>".

* Fix some consistency issues between jobset status in PostgreSQL and
  in hydra-evaluator. In particular, "lastCheckedTime" was never
  updated internally.
 2017-03-15 16:59:57 +01:00
Eelco Dolstra 242ea8009f
hydra-evaluator: Don't use the "timeout" command 
"timeout" does not propagate PR_SET_PDEATHSIG, so it prevents
hydra-eval-jobset from being killed when hydra-evaluator quits.
 2017-03-15 16:59:56 +01:00
Eelco Dolstra d1afb42f12
Supress debug message 2017-03-15 16:59:56 +01:00
Eelco Dolstra 73900e9f5f Fix std::stoi exception 2017-03-08 15:07:52 +01:00
Eelco Dolstra edebdf33f0
hydra-queue-runner: Handle SIGINT 2017-03-03 12:41:00 +01:00
Eelco Dolstra 500c27e4d5
Add hydra.conf option "nar_buffer_size" to configure memoryTokens limit 
It defaults to half the physical RAM.
 2017-03-03 12:37:27 +01:00
Rob Vermaas fd754d678e
Do not trigger eval on jobset change when check interval is 0 (disabled). 2017-02-23 09:15:26 +00:00
Eelco Dolstra 9f6afb3375
Lock paths in the scm cache 
This is necessary now that hydra-evaluator runs multiple evaluations
in parallel, to prevent corruption of Git/Mercurial clones.
 2017-02-21 18:12:52 +01:00
Eelco Dolstra 52753e49c2
hydra-evaluator: Handle SIGINT 
This makes "systemctl stop hydra-evaluator" work promptly again.
 2017-02-21 17:54:31 +01:00