Commit graph

72 commits

Author SHA1 Message Date
Graham Christensen 71453dd341 Expose Prometheus metrics at /metrics
Exposes metrics:

* http_request_duration_seconds_bucket
* http_request_size_bytes_bucket
* http_response_size_bytes_bucket
* http_requests_total

with labels of action and controller to help identify popular
endpoints and their performance characteristics.
2021-06-10 20:02:42 -04:00
Graham Christensen db6bad108b Use PrometheusTinyShared 2021-06-10 17:47:11 -04:00
Graham Christensen 1652631b21 Prometheus: init Tiny, TinyShared 2021-06-10 17:47:11 -04:00
Graham Christensen c6670b805f init: HashSharedMem for Prometheus 2021-06-10 17:47:11 -04:00
Graham Christensen b1fcc0c25b init DataRandom perl package for Prometheus stats 2021-06-10 17:47:11 -04:00
Graham Christensen 015d8298cb flake.nix: drop the API test 2021-04-28 18:32:23 -04:00
Maximilian Bosch 963085a04a
Add AuthenSASL to perl deps; improve email docs
The addition of AuthenSASL seems to be necessary to properly
authenticate against an SMTP server. Without this I got errors
such as

    error with Hydra::Plugin::EmailNotification=HASH(0x6ad0128)->buildFinished: SMTP auth requires MIME::Base64 and Authen::SASL
2021-04-26 18:33:57 +02:00
Graham Christensen 1da70030b7 Users: transparently upgrade passwords to Argon2
Passwords that are sha1 will be transparently upgraded to argon2,
and future comparisons will use Argon2

Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:35:11 -04:00
Graham Christensen 29620df85e Passwords: check in constant time
The default password comparison logic does not use
constant time validation. Switching to constant time
offers a meager improvement by removing a timing
oracle.

A prepatory step in moving to Argon2id password storage, since we'll need this change anyway after
for validating existing passwords.

Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:34:56 -04:00
Maximilian Bosch 3bc5252c80
Get rid of now obsolte LD_PRELOAD hack 2021-04-03 09:56:15 +02:00
Maximilian Bosch 2beb1f5405
Replace TestHTTPMockServer with python script
This seems to work fine in a `nix build`-sandbox as it doesn't depend on
`getprotobyname`.
2021-04-03 01:07:07 +02:00
Maximilian Bosch d16bf5b8cd
Implement yath-test for the new Gitea plugin 2021-04-02 19:11:25 +02:00
Maximilian Bosch f9f5ab2fb1
Make gitea public URL configurable
Otherwise, it will be obtained from the jobset input that contains the
URL to the git repo to build.
2021-03-30 23:01:36 +02:00
Maximilian Bosch eecea56131
Implement VM-test for gitea plugin 2021-03-30 22:35:39 +02:00
Cole Helbling 025be052b7
tests: move to t, allow yath test from root
By moving the tests subdirectory to t, we gain the ability to run `yath
test` with no arguments from inside `nix develop` in the root of the
the repo.

(`nix develop` is necessary in order to set the proper env vars for
`yath` to find our test libraries.)
2021-03-05 09:49:06 -08:00
Graham Christensen 9ddc6e355f
flake: add TestPostgreSQL for per-test DBs 2021-02-23 21:12:02 -05:00
Graham Christensen bfcc50f978
Merge pull request #877 from immae/mdbook
Convert documentation to markdown / mdbook
2021-02-23 20:16:33 -05:00
Ismaël Bouya 9d916877fb
Add markdown files for documentation
projects.xml and declarative-projects.xml were merged with xmllint, and
then I ran that to convert files
for i in *.xml; do pandoc -s -f docbook -t markdown $i -o ${i/xml/md}; done
2021-02-24 01:07:00 +01:00
Graham Christensen 45d9a22d73
flake.nix: add perlPackages until they're available from nixpkgs
These packages were added to Nixpkgs in https://github.com/NixOS/nixpkgs/pull/113702.
2021-02-19 17:06:49 -05:00
Graham Christensen 2240035e20
Run tests with yath
This will let us run tests in parallel, and creates a more Perl-standard
test development experience.
2021-02-19 17:04:19 -05:00
Eelco Dolstra 525a229dac Convert validate-openapi to a Hydra job 2021-01-03 18:47:05 +01:00
Eelco Dolstra 79d34ed7c9 Follow nix/nixpkgs 2020-10-20 12:21:06 +02:00
Eelco Dolstra 90a2f8eb2b Merge branch 'master' of github.com:NixOS/hydra into master 2020-10-20 12:19:53 +02:00
Eelco Dolstra 338d946620 Remove nix from hydra-perl-deps
Also add nix to passthru. This makes it easier to override nix in 'nix
develop', e.g.

  $ nix develop \
    --redirect .#hydraJobs.build.x86_64-linux.nix ~/Dev/nix/outputs/out \
    --redirect .#hydraJobs.build.x86_64-linux.nix.dev ~/Dev/nix/outputs/dev
2020-10-20 12:15:18 +02:00
Andreas Rammhold 458b0ceb85
ldap: Use hashed passwords in the test
We should lead with good example and when users copy code from this repo
it should not incentivize them to use plain text passwords.
2020-10-12 17:18:37 +02:00
Andreas Rammhold 656b0c27b3
Update to NixOS 20.09
This allows us to make use of the openldap modules for hashed passwords.
In 20.03 the openldap expression didn't yet build those modules.
2020-10-12 17:18:01 +02:00
Andreas Rammhold ee60f4ccd9 tests: fix invalid YAML
Somwhere while polishing the initial PR I forgot to add that colo.
2020-09-12 19:57:24 +02:00
Andreas Rammhold f229da352e
LDAP add test for roles and multiple users 2020-09-10 17:09:32 +02:00
Andreas Rammhold cfc01e2518
LDAP: add VM test to flake.nix
In this newly added test an OpenLDAP server will provide one user
(called `user`) and it will be attempted to login as that said user.
Also logging in with any other password must fail.
2020-09-10 17:09:32 +02:00
Andreas Rammhold b8c19337b6
LDAP: add the required packages to the perlPackage via the overlay
Nixpkgs doesn't currently provide these required packages. In order to
use this feature without waiting for a newer release of NixOS/Nixpkgs
thes have been packages inline.
2020-09-10 16:47:33 +02:00
edef b9ff7b2671 include perlPackages.YAML in buildInputs 2020-09-09 13:02:49 +02:00
Nikola Knezevic fceaed2b24 Make PathInput plugin cache validity configurable
PathInput plugin keeps a cache of path evaluations. This cache is simple, and
path is not checked more than once every N seconds, where N=30. The caching is
there to avoid expensive calls to `nix-store --add`.

This change makes the validity period configurable. The main use case is
`api-test.pl` which was implemented wrong for a while, as the invocation of
`hydra-eval-jobset` would return the previous evaluation, claiming there are no
changes. The test has been fixed to check better for a new evaluation.
2020-06-04 12:26:47 +02:00
Eelco Dolstra af9f635a5a
Use PostgreSQL 11
This is what we use for hydra.nixos.org.
2020-05-27 17:34:14 +02:00
Tobias Pflug 31262f14fb Address PR comments:
- scripts -> foreman
- drop runHydra
- drop devShell
- move postgresql to buildInputs
2020-05-15 13:48:21 +02:00
Tobias Pflug d9d58b7055 Use pg_ctl for readiness check
Using `pg_ctl status` is more reliable than relying checking an
open port via netcat.
2020-05-15 12:37:16 +02:00
Tobias Pflug 07a4465260 Use custom ports for postgres/hydra
Use custom ports so hydra and postgres can run in environments where
the default ports are in use already.
2020-05-15 12:37:16 +02:00
Andreas Rammhold eb06a435ab Add devShell for faster feedback
This adds a `devShell` which unlike `runHydra` doesn't start hydra
automatically and doesn't receive hydra as build input. It is better
suited for interactive development cycles:

```
$ nix-shell -A devShell
$ ./bootstrap
$ configurePhase
$ make
$ # hack hack hack
$ foreman start
  # test test test
  <C-c>
$ # hack hack hack
```
2020-05-15 12:37:16 +02:00
Andreas Rammhold d1237c315d Add runHydra shell
runHyda automatically starts hydra and postgres:

```
$ nix-shell -A runHydra
```

The shell receives hydra from the working copy as buildInput.
Running hydra, queue-runner, evaluator and postgres is managed
by foreman (https://github.com/ddollar/foreman) and configured
in `Procfile`.
2020-05-15 12:37:16 +02:00
Eelco Dolstra cb06f0af47
flake.nix: Remove edition field 2020-05-12 14:03:06 +02:00
Nikola Knezevic 3efa8223c9 Convert all tests to the Python DSL
Since Perl-based NixOS tests will be deprecated in nixpkgs 20.09, this
change migrates all tests to the new Python-based NixOS tests.
2020-05-11 22:05:30 +02:00
Maximilian Bosch 16a8b4d086
Re-add DBDSQLite since it's needed by the Perl-bindings of Nix 2020-04-16 00:42:40 +02:00
Maximilian Bosch efcbc08686
Get rid of dependency to SQLite
SQLite isn't properly supported by Hydra for a few years now[1], but
Hydra still depends on it. Apart from a slightly bigger closure this can
cause confusion by users since Hydra picks up SQLite rather than
PostgreSQL by default if HYDRA_DBI isn't configured properly[2]

[1] 78974abb69
[2] https://logs.nix.samueldr.com/nixos-dev/2020-04-10#3297342;
2020-04-16 00:42:40 +02:00
Eelco Dolstra 400a0fbd14
lastModified -> lastModifiedDate 2020-04-09 15:44:53 +02:00
Eelco Dolstra f6119f2b19
perlDeps: Use lib.closePropagation
Apparentely, buildEnv in 20.03 no longer respects
propagated-build-inputs.

Note that the use of a library function (closePropagation) seems
fundamentally wrong to me - propagated-build-inputs should be used at
runtime, not at evaluation time.
2020-03-31 13:21:01 +02:00
Eelco Dolstra 810075cd3d
Update to 20.03 2020-03-31 12:55:35 +02:00
Eelco Dolstra 101a9b3797
Run 'make check'
This got lost when I converted from releaseTools.nixBuild to
stdenv.mkDerivation. Doh!
2020-03-04 15:42:51 +01:00
Eelco Dolstra 6f1d68bda4
Revert "hydra-eval-jobs -> nix eval-hydra-jobs"
This reverts commit 345512a6d0.
2020-02-19 20:36:52 +01:00
Eelco Dolstra 4b9cdc4933 Remove boehmgc dependency
This is now propagated by Nix.
2020-02-15 21:56:35 +01:00
Eelco Dolstra 345512a6d0
hydra-eval-jobs -> nix eval-hydra-jobs 2020-02-15 15:59:34 +01:00
Eelco Dolstra 23c9ca3e94
Remove hydra-eval-guile-jobs
This hasn't been used in a long time (Guix uses its own CI system),
and it probably doesn't work anymore.
2020-02-15 15:59:34 +01:00