From 243291f9bdbacc4362a606bb0cdf4952edbb928f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20Thalheim?= Date: Wed, 29 Sep 2021 23:39:15 +0200 Subject: [PATCH] add basic documentation for all plugins Co-authored-by: Pablo Ovelleiro Corral Co-authored-by: Graham Christensen --- doc/manual/src/SUMMARY.md | 2 + doc/manual/src/plugins/README.md | 276 ++++++++++++++++++ .../src/plugins/declarative-projects.md | 143 +++++++++ doc/manual/src/projects.md | 143 +-------- 4 files changed, 423 insertions(+), 141 deletions(-) create mode 100644 doc/manual/src/plugins/README.md create mode 100644 doc/manual/src/plugins/declarative-projects.md diff --git a/doc/manual/src/SUMMARY.md b/doc/manual/src/SUMMARY.md index ab60d0d8..b315aeff 100644 --- a/doc/manual/src/SUMMARY.md +++ b/doc/manual/src/SUMMARY.md @@ -5,6 +5,8 @@ - [Configuration](configuration.md) - [Creating and Managing Projects](projects.md) - [Hydra jobs](./jobs.md) +- [Plugins](./plugins/README.md) + - [Declarative Projects](./plugins/declarative-projects.md) - [Using the external API](api.md) - [Monitoring Hydra](./monitoring/README.md) diff --git a/doc/manual/src/plugins/README.md b/doc/manual/src/plugins/README.md new file mode 100644 index 00000000..25ea6e3a --- /dev/null +++ b/doc/manual/src/plugins/README.md @@ -0,0 +1,276 @@ +# Plugins + +This chapter describes all plugins present in Hydra. + +### Inputs + +Hydra supports the following inputs: + +- Bazaar input +- Darcs input +- Git input +- Mercurial input +- Path input + +## Bitbucket pull requests + +Create jobs based on open bitbucket pull requests. + +### Configuration options + +- `bitbucket_authorization.` + +## Bitbucket status + +Sets Bitbucket CI status. + +### Configuration options + +- `enable_bitbucket_status` +- `bitbucket.username` +- `bitbucket.password` + +## CircleCI Notification + +Sets CircleCI status. + +### Configuration options + +- `circleci.[].jobs` +- `circleci.[].vcstype` +- `circleci.[].token` + +## Compress build logs + +Compresses build logs after a build with bzip2. + +### Configuration options + +- `compress_build_logs` + +Enable log compression + +### Example + +```xml +compress_build_logs = 1 +``` + +## Coverity Scan + +Uploads source code to [coverity scan](https://scan.coverity.com). + +### Configuration options + +- `coverityscan.[].jobs` +- `coverityscan.[].project` +- `coverityscan.[].email` +- `coverityscan.[].token` +- `coverityscan.[].scanurl` + +## Email notification + +Sends email notification if build status changes. + +### Configuration options + +- `email_notification` + +## Gitea status + +Sets Gitea CI status + +### Configuration options + +- `gitea_authorization.` + +## GitHub pulls + +Create jobs based on open GitHub pull requests + +### Configuration options + +- `github_authorization.` + +## Github refs + +Hydra plugin for retrieving the list of references (branches or tags) from +GitHub following a certain naming scheme. + +### Configuration options + +- `github_endpoint` +- `github_authorization.` + +## Github status + +Sets GitHub CI status. + +### Configuration options + +- `githubstatus.[].jobs` + +Regular expression for jobs to match in the format `project:jobset:job`. +Defaults to `*:*:*`. + +- `githubstatus.[].excludeBuildFromContext` + +Don't include the build's ID in the status. + +- `githubstatus.[].context` + +Context shown in the status + +- `githubstatus.[].useShortContext` + +Renames `continuous-integration/hydra` to `ci/hydra` and removes the PR suffix +from the name. Useful to see the full path in GitHub for long job names. + +- `githubstatus.[].description` + +Description shown in the status. Defaults to `Hydra build # of +` + +- `githubstatus.[].inputs` + +The input which corresponds to the github repo/rev whose +status we want to report. Can be repeated. + +- `githubstatus.[].authorization` + +Verbatim contents of the Authorization header. See +[GitHub documentation](https://developer.github.com/v3/#authentication) for +details. + +### Example + +```xml + + jobs = test:pr:build + inputs = src + authorization = Basic notgivingyoumypasswordosorry + excludeBuildFromContext = 1 + +``` + +## GitLab pulls + +Create jobs based on open gitlab pull requests. + +### Configuration options + +- `gitlab_authorization.` + +## Gitlab status + +Sets Gitlab CI status. + +### Configuration options + +- `gitlab_authorization.` + +## HipChat notification + +Sends hipchat chat notifications when a build finish. + +### Configuration options + +- `hipchat.[].jobs` +- `hipchat.[].builds` +- `hipchat.[].token` +- `hipchat.[].notify` + +## InfluxDB notification + +Writes InfluxDB events when a builds finished. + +### Configuration options + +- `influxdb.url` +- `influxdb.db` + +## Run command + +Runs a shell command when the build is finished. + +### Configuration options: + +- `runcommand.[].job` + +Regular expression for jobs to match in the format `project:jobset:job`. +Defaults to `*:*:*`. + +- `runcommand.[].command` + +Command to run. Can use the `$HYDRA_JSON` environment variable to access +information about the build. + +### Example + +```xml + + job = myProject:*:* + command = cat $HYDRA_JSON > /tmp/hydra-output + +``` + +## S3 backup + +Upload nars and narinfos to S3 storage. + +### Configuration options + +- `s3backup.[].jobs` +- `s3backup.[].compression_type` +- `s3backup.[].name` +- `s3backup.[].prefix` + +## Slack notification + +Sending Slack notifications about build results. + +### Configuration options + +- `slack.[].jobs` +- `slack.[].force` +- `slack.[].url` + + +## SoTest + +Scheduling hardware tests to SoTest controller + +This plugin submits tests to a SoTest controller for all builds that contain +two products matching the subtypes "sotest-binaries" and "sotest-config". + +Build products are declared by the file "nix-support/hydra-build-products" +relative to the root of a build, in the following format: + +``` + file sotest-binaries /nix/store/…/binaries.zip + file sotest-config /nix/store/…/config.yaml +``` + +### Configuration options + +- `sotest.[].uri` + +URL of the controller, defaults to `https://opensource.sotest.io` + +- `sotest.[].authfile` + +File containing `username:password` + +- `sotest.[].priority` + +Optional priority setting. + +### Example + +```xml + + uri = https://sotest.example + authfile = /var/lib/hydra/sotest.auth + priority = 1 + + ``` diff --git a/doc/manual/src/plugins/declarative-projects.md b/doc/manual/src/plugins/declarative-projects.md new file mode 100644 index 00000000..12dfed18 --- /dev/null +++ b/doc/manual/src/plugins/declarative-projects.md @@ -0,0 +1,143 @@ +## Declarative Projects + +Hydra supports declaratively configuring a project\'s jobsets. This +configuration can be done statically, or generated by a build job. + +> **Note** +> +> Hydra will treat the project\'s declarative input as a static definition +> if and only if the spec file contains a dictionary of dictionaries. If +> the value of any key in the spec is not a dictionary, it will treat the +> spec as a generated declarative spec. + +### Static, Declarative Projects + +Hydra supports declarative projects, where jobsets are configured from a +static JSON document in a repository. + +To configure a static declarative project, take the following steps: + +1. Create a Hydra-fetchable source like a Git repository or local path. + +2. In that source, create a file called `spec.json`, and add the + specification for all of the jobsets. Each key is jobset and each + value is a jobset\'s specification. For example: + + ``` {.json} + { + "nixpkgs": { + "enabled": 1, + "hidden": false, + "description": "Nixpkgs", + "nixexprinput": "nixpkgs", + "nixexprpath": "pkgs/top-level/release.nix", + "checkinterval": 300, + "schedulingshares": 100, + "enableemail": false, + "emailoverride": "", + "keepnr": 3, + "inputs": { + "nixpkgs": { + "type": "git", + "value": "git://github.com/NixOS/nixpkgs.git master", + "emailresponsible": false + } + } + }, + "nixos": { + "enabled": 1, + "hidden": false, + "description": "NixOS: Small Evaluation", + "nixexprinput": "nixpkgs", + "nixexprpath": "nixos/release-small.nix", + "checkinterval": 300, + "schedulingshares": 100, + "enableemail": false, + "emailoverride": "", + "keepnr": 3, + "inputs": { + "nixpkgs": { + "type": "git", + "value": "git://github.com/NixOS/nixpkgs.git master", + "emailresponsible": false + } + } + } + } + ``` + +3. Create a new project, and set the project\'s declarative input type, + declarative input value, and declarative spec file to point to the + source and JSON file you created in step 2. + +Hydra will create a special jobset named `.jobsets`. When the `.jobsets` +jobset is evaluated, this static specification will be used for +configuring the rest of the project\'s jobsets. + + +### Generated, Declarative Projects + +Hydra also supports generated declarative projects, where jobsets are +configured automatically from specification files instead of being +managed through the UI. A jobset specification is a JSON object +containing the configuration of the jobset, for example: + +``` {.json} + { + "enabled": 1, + "hidden": false, + "description": "js", + "nixexprinput": "src", + "nixexprpath": "release.nix", + "checkinterval": 300, + "schedulingshares": 100, + "enableemail": false, + "emailoverride": "", + "keepnr": 3, + "inputs": { + "src": { "type": "git", "value": "git://github.com/shlevy/declarative-hydra-example.git", "emailresponsible": false }, + "nixpkgs": { "type": "git", "value": "git://github.com/NixOS/nixpkgs.git release-16.03", "emailresponsible": false } + } + } + +``` + +To configure a declarative project, take the following steps: + +1. Create a jobset repository in the normal way (e.g. a git repo with a + `release.nix` file, any other needed helper files, and taking any + kind of hydra input), but without adding it to the UI. The nix + expression of this repository should contain a single job, named + `jobsets`. The output of the `jobsets` job should be a JSON file + containing an object of jobset specifications. Each member of the + object will become a jobset of the project, configured by the + corresponding jobset specification. + +2. In some hydra-fetchable source (potentially, but not necessarily, + the same repo you created in step 1), create a JSON file containing + a jobset specification that points to the jobset repository you + created in the first step, specifying any needed inputs + (e.g. nixpkgs) as necessary. + +3. In the project creation/edit page, set declarative input type, + declarative input value, and declarative spec file to point to the + source and JSON file you created in step 2. + +Hydra will create a special jobset named `.jobsets`, which whenever +evaluated will go through the steps above in reverse order: + +1. Hydra will fetch the input specified by the declarative input type + and value. + +2. Hydra will use the configuration given in the declarative spec file + as the jobset configuration for this evaluation. In addition to any + inputs specified in the spec file, hydra will also pass the + `declInput` argument corresponding to the input fetched in step 1 and + the `projectName` argument containing the project\'s name. + +3. As normal, hydra will build the jobs specified in the jobset + repository, which in this case is the single `jobsets` job. When + that job completes, hydra will read the created jobset + specifications and create corresponding jobsets in the project, + disabling any jobsets that used to exist but are not present in the + current spec. diff --git a/doc/manual/src/projects.md b/doc/manual/src/projects.md index f23063cc..95174f1b 100644 --- a/doc/manual/src/projects.md +++ b/doc/manual/src/projects.md @@ -307,149 +307,10 @@ used to change the way Hello and *all* its dependencies--including the C library and compiler used to build it--are built. See the Nixpkgs manual for more. -Declarative projects +Declarative Projects -------------------- -Hydra supports declaratively configuring a project\'s jobsets. This -configuration can be done statically, or generated by a build job. - -> **Note** -> -> Hydra will treat the project\'s declarative input as a static definition -> if and only if the spec file contains a dictionary of dictionaries. If -> the value of any key in the spec is not a dictionary, it will treat the -> spec as a generated declarative spec. - -### Static, Declarative Projects - -Hydra supports declarative projects, where jobsets are configured from a -static JSON document in a repository. - -To configure a static declarative project, take the following steps: - -1. Create a Hydra-fetchable source like a Git repository or local path. - -2. In that source, create a file called `spec.json`, and add the - specification for all of the jobsets. Each key is jobset and each - value is a jobset\'s specification. For example: - - ``` {.json} - { - "nixpkgs": { - "enabled": 1, - "hidden": false, - "description": "Nixpkgs", - "nixexprinput": "nixpkgs", - "nixexprpath": "pkgs/top-level/release.nix", - "checkinterval": 300, - "schedulingshares": 100, - "enableemail": false, - "emailoverride": "", - "keepnr": 3, - "inputs": { - "nixpkgs": { - "type": "git", - "value": "git://github.com/NixOS/nixpkgs.git master", - "emailresponsible": false - } - } - }, - "nixos": { - "enabled": 1, - "hidden": false, - "description": "NixOS: Small Evaluation", - "nixexprinput": "nixpkgs", - "nixexprpath": "nixos/release-small.nix", - "checkinterval": 300, - "schedulingshares": 100, - "enableemail": false, - "emailoverride": "", - "keepnr": 3, - "inputs": { - "nixpkgs": { - "type": "git", - "value": "git://github.com/NixOS/nixpkgs.git master", - "emailresponsible": false - } - } - } - } - ``` - -3. Create a new project, and set the project\'s declarative input type, - declarative input value, and declarative spec file to point to the - source and JSON file you created in step 2. - -Hydra will create a special jobset named `.jobsets`. When the `.jobsets` -jobset is evaluated, this static specification will be used for -configuring the rest of the project\'s jobsets. - -### Generated, Declarative Projects - -Hydra also supports generated declarative projects, where jobsets are -configured automatically from specification files instead of being -managed through the UI. A jobset specification is a JSON object -containing the configuration of the jobset, for example: - -``` {.json} - { - "enabled": 1, - "hidden": false, - "description": "js", - "nixexprinput": "src", - "nixexprpath": "release.nix", - "checkinterval": 300, - "schedulingshares": 100, - "enableemail": false, - "emailoverride": "", - "keepnr": 3, - "inputs": { - "src": { "type": "git", "value": "git://github.com/shlevy/declarative-hydra-example.git", "emailresponsible": false }, - "nixpkgs": { "type": "git", "value": "git://github.com/NixOS/nixpkgs.git release-16.03", "emailresponsible": false } - } - } - -``` - -To configure a declarative project, take the following steps: - -1. Create a jobset repository in the normal way (e.g. a git repo with a - `release.nix` file, any other needed helper files, and taking any - kind of hydra input), but without adding it to the UI. The nix - expression of this repository should contain a single job, named - `jobsets`. The output of the `jobsets` job should be a JSON file - containing an object of jobset specifications. Each member of the - object will become a jobset of the project, configured by the - corresponding jobset specification. - -2. In some hydra-fetchable source (potentially, but not necessarily, - the same repo you created in step 1), create a JSON file containing - a jobset specification that points to the jobset repository you - created in the first step, specifying any needed inputs - (e.g. nixpkgs) as necessary. - -3. In the project creation/edit page, set declarative input type, - declarative input value, and declarative spec file to point to the - source and JSON file you created in step 2. - -Hydra will create a special jobset named `.jobsets`, which whenever -evaluated will go through the steps above in reverse order: - -1. Hydra will fetch the input specified by the declarative input type - and value. - -2. Hydra will use the configuration given in the declarative spec file - as the jobset configuration for this evaluation. In addition to any - inputs specified in the spec file, hydra will also pass the - `declInput` argument corresponding to the input fetched in step 1 and - the `projectName` argument containing the project\'s name. - -3. As normal, hydra will build the jobs specified in the jobset - repository, which in this case is the single `jobsets` job. When - that job completes, hydra will read the created jobset - specifications and create corresponding jobsets in the project, - disabling any jobsets that used to exist but are not present in the - current spec. +see this [chapter](./plugins/declarative-projects.md) Email Notifications -------------------