Commit graph

2366 commits

Author SHA1 Message Date
Alexander Ried 2b7fd6e509 readme: update hacking section 2016-05-12 09:45:51 +02:00
Alexander Ried 51d21b9b91 hydra-module.nix: fix warning about deprecated types.list 2016-05-09 19:09:33 +02:00
Alexander Ried 2c78a92ac5 release.nix: add missing dependency on nix 2016-05-09 18:35:24 +02:00
Eelco Dolstra 615c10bcc4 Merge pull request #277 from shlevy/declarative-project
Enable declarative projects.
2016-05-09 16:16:20 +02:00
Shea Levy aa7cc6d599 Declarative projects: Include the fetch info for the declarative input in the project eval 2016-05-09 08:54:27 -04:00
Shea Levy 4392d3e21d Enable declarative projects.
This allows fully declarative project specifications. This is best
illustrated by example:

* I create a new project, setting the declarative spec file to
  "spec.json" and the declarative input to a git repo pointing
  at git://github.com/shlevy/declarative-hydra-example.git
* hydra creates a special ".jobsets" jobset alongside the project
* Just before evaluating the ".jobsets" jobset, hydra fetches
  declarative-hydra-example.git, reads spec.json as a jobset spec,
  and updates the jobset's configuration accordingly:
{
    "enabled": 1,
    "hidden": false,
    "description": "Jobsets",
    "nixexprinput": "src",
    "nixexprpath": "default.nix",
    "checkinterval": 300,
    "schedulingshares": 100,
    "enableemail": false,
    "emailoverride": "",
    "keepnr": 3,
    "inputs": {
        "src": { "type": "git", "value": "git://github.com/shlevy/declarative-hydra-example.git", "emailresponsible": false },
        "nixpkgs": { "type": "git", "value": "git://github.com/NixOS/nixpkgs.git release-16.03", "emailresponsible": false }
    }
}
* When the "jobsets" job of the ".jobsets" jobset completes, hydra
  reads its output as a JSON representation of a dictionary of
  jobset specs and creates a jobset named "master" configured
  accordingly (In this example, this is the same configuration as
  .jobsets itself, except using release.nix instead of default.nix):
{
    "enabled": 1,
    "hidden": false,
    "description": "js",
    "nixexprinput": "src",
    "nixexprpath": "release.nix",
    "checkinterval": 300,
    "schedulingshares": 100,
    "enableemail": false,
    "emailoverride": "",
    "keepnr": 3,
    "inputs": {
        "src": { "type": "git", "value": "git://github.com/shlevy/declarative-hydra-example.git", "emailresponsible": false },
        "nixpkgs": { "type": "git", "value": "git://github.com/NixOS/nixpkgs.git release-16.03", "emailresponsible": false }
    }
}
2016-05-09 08:54:27 -04:00
Peter Simons 3ca0e50e01 hydra-module.nix: add 'useSubstitutes' option to control use of binary caches
Gives users who want to use binary caches a way to do so without messing
with module internals. Fixes https://github.com/NixOS/hydra/issues/297.
2016-05-06 10:30:54 +02:00
Eelco Dolstra 995f3b21db Merge pull request #303 from groxxda/fix/searchpath
release.nix use makeBinPath to construct PATH
2016-04-29 12:58:07 +02:00
Eelco Dolstra fa335bf090 Merge pull request #302 from groxxda/fix/debugging
hydra-module: fix debug attribute
2016-04-29 12:57:45 +02:00
Eelco Dolstra a2a9ff64b5 Merge pull request #300 from groxxda/patch-1
Update link to hydra nixexpression
2016-04-29 12:57:16 +02:00
Alexander Ried 3f5ccfc7f3 release.nix use makeBinPath to construct PATH
makeBinPath takes care to use the correct output.

nix-repl> lib.makeSearchPath "bin" [pkgs.nix]
"/nix/store/zpp83pr21ihxwsr15l6mkzwkr49zj71d-nix-1.11.2-dev/bin"

nix-repl> lib.makeBinPath [pkgs.nix]
"/nix/store/9n8c3g541qn43yjjs94f1a0m69wp8scg-nix-1.11.2/bin"
2016-04-29 04:23:54 +02:00
Alexander Ried 5777f246a5 hydra-module: fix debug attribute 2016-04-29 04:21:05 +02:00
Eelco Dolstra b829be29b6 Lazy load Google and Persona login stuff 2016-04-28 17:13:02 +02:00
Eelco Dolstra 0c4e8b13a1 Update jquery and jquery-ui and use SRI checking 2016-04-28 15:13:54 +02:00
Alexander Ried 5232557b1a Update link to hydra nixexpression 2016-04-21 02:36:59 +02:00
Eelco Dolstra b50a105ca7 S3BinaryCacheStore: Use disk cache 2016-04-20 15:29:40 +02:00
Eelco Dolstra b2b978eda0 Fix JSON output for builds 2016-04-18 11:44:59 +02:00
Eelco Dolstra afb86638cd Updates for negative .narinfo caching 2016-04-15 15:39:20 +02:00
Eelco Dolstra 177bf25d64 Queue monitor: Bail out earlier if a step has failed previously
Currently, the hydra.nixos.org queue contains 1000s of Darwin builds
that all depend on a stdenv-darwin that previously failed. However,
before, first createStep() would construct a dependency graph for each
build, then getQueuedBuilds() would discover that one of the steps had
failed previously and discard all those steps. Since the graph
construction involves a lot of uncached calls to isValidPath(), this
took several seconds per build.

Now createStep() detects the previous failure right away and bails
out.
2016-04-15 14:32:16 +02:00
Eelco Dolstra 2f0f7406d4 Merge pull request #292 from shlevy/git-rev-test
Add eval test for specific git revision
2016-04-14 20:05:00 +02:00
Eelco Dolstra ef72569cc3 Merge pull request #280 from shlevy/github-status-api
Add a plugin to interact with the github status API.
2016-04-14 20:03:45 +02:00
Eelco Dolstra 7dad867498 Add smtpHost option 2016-04-13 16:46:24 +02:00
Eelco Dolstra d6f188a01a Typo 2016-04-13 16:45:40 +02:00
Eelco Dolstra fd20c726e1 Remove ssmtp from $PATH again
See 10cad61231.
2016-04-13 16:33:38 +02:00
Eelco Dolstra b1e36b550c max-output-size -> max_output_size
To be consistent with other Catalyst/Hydra config option names.
2016-04-13 16:30:52 +02:00
Eelco Dolstra 077ed3f571 Periodically clear orphaned build steps
These are build steps that remain "busy" in the database even though
they have finished, because they couldn't be updated (e.g. due to a
PostgreSQL connection problem). To prevent them from showing up as
busy in the "Machine status" page, we now periodically purge them.
2016-04-13 16:30:52 +02:00
Eelco Dolstra f3f661bac1 Reuse build products / metrics stored in the database
Previously, if the queue monitor thread encounters a build that Hydra
has previously built, it downloaded the output paths from the binary
cache, just to determine the build products and metrics. This is very
inefficient. In particular, when doing something like merging
nixpkgs:staging into nixpkgs:master, the queue monitor thread will be
locked up for a long time fetching files from S3, causing the build
farm to be mostly idle.

Of course this is entirely unnecessary, since the build
products/metrics are already in the Hydra database. So now we just
look up a previous build with the same output path, and copy the
products/metrics.
2016-04-13 16:30:52 +02:00
Eelco Dolstra a7755678fe Drop unused BuildProducts.description column 2016-04-13 16:30:52 +02:00
Eelco Dolstra 8c7edb1005 Fix narrowing conversion 2016-04-13 16:30:52 +02:00
Eelco Dolstra 00c78440b1 Disambiguate "marking build as succeeded" message 2016-04-13 16:30:52 +02:00
Eelco Dolstra ad834343b5 Fix build against current Nix master 2016-04-13 16:30:52 +02:00
Shea Levy 93f9846425 git-rev: Remove superfluous git tag 2016-04-12 16:30:42 -04:00
Shea Levy 10e036238c Add eval test for specific git revision 2016-04-12 16:27:11 -04:00
Shea Levy ae24e7fb31 GithubStatus: Include the full job name and build ID in the context.
Build ID can be omitted by setting excludeBuildFromContext in config
2016-04-12 15:19:05 -04:00
Shea Levy 0d0b925af1 Add a plugin to interact with the github status API.
Mutliple <githubstatus> sections are possible:
* jobs: regexp for jobs to match
* inputs: the input which corresponds to the github repo/rev whose
  status we want to report. Can be repeated
* authorization: Verbatim contents of the Authorization header. See
  https://developer.github.com/v3/#authentication.
2016-04-12 14:42:01 -04:00
Shea Levy 9b37cb89ae Add buildStarted plugin hook 2016-04-12 14:42:01 -04:00
Eelco Dolstra 49f94bac3a Merge pull request #291 from elitak/typo
typo
2016-04-06 21:07:21 +02:00
Eric Litak 17035661f2 typo 2016-04-06 11:28:32 -07:00
Eelco Dolstra 68b636c1f2 Merge pull request #290 from svanderburg/master
Fix problem with delegating builds to localhost due to nix-store not being in the PATH
2016-04-06 16:21:07 +02:00
Sander van der Burg cbd2e3a50d Fix problem with delegating builds to localhost due to nix-store not being in the PATH 2016-04-06 14:16:04 +00:00
Eelco Dolstra 74dfcc84e9 Make NIX_REMOTE_SYSTEMS configurable 2016-03-25 15:41:38 +01:00
Eelco Dolstra ed88bbaac0 Set Vary to Accept
Otherwise, the browser may mix up HTML and JSON responses if it has
requested both. For example, hitting the back button to return to a
job metric page will show a JSON response, because that was the last
thing the browser fetched for that URL.

This requires Catalyst::Action::Rest >= 1.20.
2016-03-25 14:48:12 +01:00
Eelco Dolstra 32adc53070 Add tooltips to metrics showing the exact value of the data point 2016-03-25 14:32:36 +01:00
Eelco Dolstra 3e2911803d Add link to metrics 2016-03-25 13:57:17 +01:00
Eelco Dolstra dab16fb26b Lazy load the metrics tab 2016-03-25 13:49:06 +01:00
Eelco Dolstra 7a72f64e5e Move chart code to common.js 2016-03-25 13:33:10 +01:00
Eelco Dolstra dc2010eafc Fix rendering of metrics with dots in their name 2016-03-25 13:24:43 +01:00
Eelco Dolstra ef63dd77e3 Fix metric alignment 2016-03-25 12:08:18 +01:00
Eelco Dolstra 759bd38ef2 Sort metrics by name 2016-03-25 11:56:25 +01:00
Eelco Dolstra 32fa392146 Fix hydra-queue-runner PATH 2016-03-23 12:35:55 +01:00