Commit graph

78 commits

Author SHA1 Message Date
Your Name
5d0ad5f649 hydra-notify: initial scratch take of prometheus events 2021-08-24 10:56:12 -04:00
Your Name
098e48d055 Flake: add perlcritic deps 2021-08-19 16:17:49 -04:00
Your Name
194eda28b7 flake.nix sort hydra-perl-deps inputs 2021-08-18 15:10:55 -04:00
Your Name
d311a0d002 hydraJobs.tests.notifications: fixup file reference 2021-08-16 15:51:51 -04:00
Nathan van Doorn
b63dfb7ad4 Rename version to version.txt
<version> is a standard header with C++20 which could cause issues if a library checks it exists then imports it

Because we have the root of this repo in the include path, it'd see that <version> exists (with, e.g., __has_include), and then try to include it as a header

But because it's just a file that says 0.1, this would fail

This happens with libpqxx 7
2021-07-05 19:47:58 +01:00
Amanda Cameron
b0f8a207fe
flake: stdenv.lib -> lib
Nixpkgs on unstable has removed `stdenv.lib` as they've been warning for a while now. This removes the extra references to it in the flake.nix

I'm not entirely sure if this is right, but I figured it was trivial enough to give a quick try using the GH Editor while I was waiting for a job to finish
2021-06-17 16:50:38 -04:00
Graham Christensen
71453dd341 Expose Prometheus metrics at /metrics
Exposes metrics:

* http_request_duration_seconds_bucket
* http_request_size_bytes_bucket
* http_response_size_bytes_bucket
* http_requests_total

with labels of action and controller to help identify popular
endpoints and their performance characteristics.
2021-06-10 20:02:42 -04:00
Graham Christensen
db6bad108b Use PrometheusTinyShared 2021-06-10 17:47:11 -04:00
Graham Christensen
1652631b21 Prometheus: init Tiny, TinyShared 2021-06-10 17:47:11 -04:00
Graham Christensen
c6670b805f init: HashSharedMem for Prometheus 2021-06-10 17:47:11 -04:00
Graham Christensen
b1fcc0c25b init DataRandom perl package for Prometheus stats 2021-06-10 17:47:11 -04:00
Graham Christensen
015d8298cb flake.nix: drop the API test 2021-04-28 18:32:23 -04:00
963085a04a
Add AuthenSASL to perl deps; improve email docs
The addition of AuthenSASL seems to be necessary to properly
authenticate against an SMTP server. Without this I got errors
such as

    error with Hydra::Plugin::EmailNotification=HASH(0x6ad0128)->buildFinished: SMTP auth requires MIME::Base64 and Authen::SASL
2021-04-26 18:33:57 +02:00
Graham Christensen
1da70030b7 Users: transparently upgrade passwords to Argon2
Passwords that are sha1 will be transparently upgraded to argon2,
and future comparisons will use Argon2

Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:35:11 -04:00
Graham Christensen
29620df85e Passwords: check in constant time
The default password comparison logic does not use
constant time validation. Switching to constant time
offers a meager improvement by removing a timing
oracle.

A prepatory step in moving to Argon2id password storage, since we'll need this change anyway after
for validating existing passwords.

Co-authored-by: Graham Christensen <graham@grahamc.com>
2021-04-15 11:34:56 -04:00
3bc5252c80
Get rid of now obsolte LD_PRELOAD hack 2021-04-03 09:56:15 +02:00
2beb1f5405
Replace TestHTTPMockServer with python script
This seems to work fine in a `nix build`-sandbox as it doesn't depend on
`getprotobyname`.
2021-04-03 01:07:07 +02:00
d16bf5b8cd
Implement yath-test for the new Gitea plugin 2021-04-02 19:11:25 +02:00
f9f5ab2fb1
Make gitea public URL configurable
Otherwise, it will be obtained from the jobset input that contains the
URL to the git repo to build.
2021-03-30 23:01:36 +02:00
eecea56131
Implement VM-test for gitea plugin 2021-03-30 22:35:39 +02:00
Cole Helbling
025be052b7
tests: move to t, allow yath test from root
By moving the tests subdirectory to t, we gain the ability to run `yath
test` with no arguments from inside `nix develop` in the root of the
the repo.

(`nix develop` is necessary in order to set the proper env vars for
`yath` to find our test libraries.)
2021-03-05 09:49:06 -08:00
Graham Christensen
9ddc6e355f
flake: add TestPostgreSQL for per-test DBs 2021-02-23 21:12:02 -05:00
Graham Christensen
bfcc50f978
Merge pull request #877 from immae/mdbook
Convert documentation to markdown / mdbook
2021-02-23 20:16:33 -05:00
Ismaël Bouya
9d916877fb
Add markdown files for documentation
projects.xml and declarative-projects.xml were merged with xmllint, and
then I ran that to convert files
for i in *.xml; do pandoc -s -f docbook -t markdown $i -o ${i/xml/md}; done
2021-02-24 01:07:00 +01:00
Graham Christensen
45d9a22d73
flake.nix: add perlPackages until they're available from nixpkgs
These packages were added to Nixpkgs in https://github.com/NixOS/nixpkgs/pull/113702.
2021-02-19 17:06:49 -05:00
Graham Christensen
2240035e20
Run tests with yath
This will let us run tests in parallel, and creates a more Perl-standard
test development experience.
2021-02-19 17:04:19 -05:00
Eelco Dolstra
525a229dac Convert validate-openapi to a Hydra job 2021-01-03 18:47:05 +01:00
Eelco Dolstra
79d34ed7c9 Follow nix/nixpkgs 2020-10-20 12:21:06 +02:00
Eelco Dolstra
90a2f8eb2b Merge branch 'master' of github.com:NixOS/hydra into master 2020-10-20 12:19:53 +02:00
Eelco Dolstra
338d946620 Remove nix from hydra-perl-deps
Also add nix to passthru. This makes it easier to override nix in 'nix
develop', e.g.

  $ nix develop \
    --redirect .#hydraJobs.build.x86_64-linux.nix ~/Dev/nix/outputs/out \
    --redirect .#hydraJobs.build.x86_64-linux.nix.dev ~/Dev/nix/outputs/dev
2020-10-20 12:15:18 +02:00
Andreas Rammhold
458b0ceb85
ldap: Use hashed passwords in the test
We should lead with good example and when users copy code from this repo
it should not incentivize them to use plain text passwords.
2020-10-12 17:18:37 +02:00
Andreas Rammhold
656b0c27b3
Update to NixOS 20.09
This allows us to make use of the openldap modules for hashed passwords.
In 20.03 the openldap expression didn't yet build those modules.
2020-10-12 17:18:01 +02:00
Andreas Rammhold
ee60f4ccd9 tests: fix invalid YAML
Somwhere while polishing the initial PR I forgot to add that colo.
2020-09-12 19:57:24 +02:00
Andreas Rammhold
f229da352e
LDAP add test for roles and multiple users 2020-09-10 17:09:32 +02:00
Andreas Rammhold
cfc01e2518
LDAP: add VM test to flake.nix
In this newly added test an OpenLDAP server will provide one user
(called `user`) and it will be attempted to login as that said user.
Also logging in with any other password must fail.
2020-09-10 17:09:32 +02:00
Andreas Rammhold
b8c19337b6
LDAP: add the required packages to the perlPackage via the overlay
Nixpkgs doesn't currently provide these required packages. In order to
use this feature without waiting for a newer release of NixOS/Nixpkgs
thes have been packages inline.
2020-09-10 16:47:33 +02:00
b9ff7b2671 include perlPackages.YAML in buildInputs 2020-09-09 13:02:49 +02:00
Nikola Knezevic
fceaed2b24 Make PathInput plugin cache validity configurable
PathInput plugin keeps a cache of path evaluations. This cache is simple, and
path is not checked more than once every N seconds, where N=30. The caching is
there to avoid expensive calls to `nix-store --add`.

This change makes the validity period configurable. The main use case is
`api-test.pl` which was implemented wrong for a while, as the invocation of
`hydra-eval-jobset` would return the previous evaluation, claiming there are no
changes. The test has been fixed to check better for a new evaluation.
2020-06-04 12:26:47 +02:00
Eelco Dolstra
af9f635a5a
Use PostgreSQL 11
This is what we use for hydra.nixos.org.
2020-05-27 17:34:14 +02:00
Tobias Pflug
31262f14fb Address PR comments:
- scripts -> foreman
- drop runHydra
- drop devShell
- move postgresql to buildInputs
2020-05-15 13:48:21 +02:00
Tobias Pflug
d9d58b7055 Use pg_ctl for readiness check
Using `pg_ctl status` is more reliable than relying checking an
open port via netcat.
2020-05-15 12:37:16 +02:00
Tobias Pflug
07a4465260 Use custom ports for postgres/hydra
Use custom ports so hydra and postgres can run in environments where
the default ports are in use already.
2020-05-15 12:37:16 +02:00
Andreas Rammhold
eb06a435ab Add devShell for faster feedback
This adds a `devShell` which unlike `runHydra` doesn't start hydra
automatically and doesn't receive hydra as build input. It is better
suited for interactive development cycles:

```
$ nix-shell -A devShell
$ ./bootstrap
$ configurePhase
$ make
$ # hack hack hack
$ foreman start
  # test test test
  <C-c>
$ # hack hack hack
```
2020-05-15 12:37:16 +02:00
Andreas Rammhold
d1237c315d Add runHydra shell
runHyda automatically starts hydra and postgres:

```
$ nix-shell -A runHydra
```

The shell receives hydra from the working copy as buildInput.
Running hydra, queue-runner, evaluator and postgres is managed
by foreman (https://github.com/ddollar/foreman) and configured
in `Procfile`.
2020-05-15 12:37:16 +02:00
Eelco Dolstra
cb06f0af47
flake.nix: Remove edition field 2020-05-12 14:03:06 +02:00
Nikola Knezevic
3efa8223c9 Convert all tests to the Python DSL
Since Perl-based NixOS tests will be deprecated in nixpkgs 20.09, this
change migrates all tests to the new Python-based NixOS tests.
2020-05-11 22:05:30 +02:00
16a8b4d086
Re-add DBDSQLite since it's needed by the Perl-bindings of Nix 2020-04-16 00:42:40 +02:00
efcbc08686
Get rid of dependency to SQLite
SQLite isn't properly supported by Hydra for a few years now[1], but
Hydra still depends on it. Apart from a slightly bigger closure this can
cause confusion by users since Hydra picks up SQLite rather than
PostgreSQL by default if HYDRA_DBI isn't configured properly[2]

[1] 78974abb69
[2] https://logs.nix.samueldr.com/nixos-dev/2020-04-10#3297342;
2020-04-16 00:42:40 +02:00
Eelco Dolstra
400a0fbd14
lastModified -> lastModifiedDate 2020-04-09 15:44:53 +02:00
Eelco Dolstra
f6119f2b19
perlDeps: Use lib.closePropagation
Apparentely, buildEnv in 20.03 no longer respects
propagated-build-inputs.

Note that the use of a library function (closePropagation) seems
fundamentally wrong to me - propagated-build-inputs should be used at
runtime, not at evaluation time.
2020-03-31 13:21:01 +02:00