the-distro/infra
9
0
Fork 5
Code Issues 40 Pull requests 6 Packages Projects 1 Releases Wiki Activity
352 commits 26 branches 0 tags 2.8 MiB
Commit graph

155 commits

Author SHA1 Message Date
Luke Granger-Brown 84cfbdb050 feat: check formatting and validity of alerts 
Fixes #94.
 2024-10-07 20:00:54 +00:00
Luke Granger-Brown 6a8f49f180 feat(gerrit): add some basic theming 
This is based on some of the preliminary colour work done by @ckie in
the the-distro/floral.systems repo.
 2024-10-07 19:27:13 +00:00
Luke Granger-Brown de085155a6 fix: update paths to floral secrets to secrets/floral/ 2024-10-07 15:48:05 +00:00
Luke Granger-Brown 2001012325 feat(uptime-kuma): status.forkos.org should point at the ForkOS page 2024-10-07 15:47:33 +00:00
raito fbf26302b6 hotfix(lix): use build01 features for build02 remote builder 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-10-07 15:26:27 +02:00
raito b56b8963a2 feat: introduce Buildbot multi-tenancy 
This shares the same expression to deploy the Buildbot.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-10-06 11:28:29 +02:00
raito 1e421889e4 feat(monitoring): add static label for tenancy 
So we can distinguish easily things in the dashboards.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-10-06 11:10:16 +02:00
raito 002db9a78f feat: introduce tenant-specific extra build capacity 
At Lix, we have few aarch64-linux and aarch64-darwin systems we use to
boost our CI.

This is a module to handle tenant-specific extra build capacity without
it leaking over the rest of the deployment.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-10-06 11:09:23 +02:00
raito 92560708b8 feat: multi-tenant secrets 
Lix may have its own secrets and we want to maintain a certain
generalization level on the NixOS modules, so we can decorrelate which
secret we select dynamically by having a simple tenancy hierarchy
system.

This unfortunately requires to rewrite all call sites with a floral
prefix until we migrate them to the simple internal secret module which
is aware of this.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-10-06 08:10:44 +00:00
raito a2eecd1886 feat(buildbot): disable manhole debugging 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-10-06 07:59:56 +00:00
raito b5d412a5ba feat: adopt new version of Buildbot with incoming ref data 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-10-06 07:59:56 +00:00
raito 1a862b2b0f hotfix: add the path to the stateless uptime kuma's password file 
Forgotten in the previous merge.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-10-05 16:33:38 +02:00
Ilya K 5582a0a29b Fix Hydra exporter crash loop nonsense 2024-10-01 19:27:13 +03:00
Ilya K 4ddf87fa8e Add new metric to Hydra exporter 2024-10-01 19:27:05 +03:00
Kiara Grouwstra b291caac46 feat(monitoring): add uptime-kuma for status page, fixes #97 
Adds a service for a status page using
[`uptime-kuma`](https://uptime.kuma.pet/).
 2024-10-01 16:13:23 +00:00
Ilya K e2c6550796 Hydra metrics 
Yoink the nixos org exporter, rewrite most of it, deploy
 2024-10-01 19:06:26 +03:00
Janik Haag d462e8ca9c
feat(gerrit): run git-gc-preserve on a daily timer 2024-09-18 22:27:57 +02:00
raito 94d1881e10 feat(gerrit): add git-gc-preserve script 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-09-02 11:05:54 +02:00
raito a14f496db8 fix(channel-scripts): fix RUST_LOG=info 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-31 20:03:11 +02:00
raito c2ad3d6d26 fix(channel-scripts): push OTLP properly now 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-31 19:56:46 +02:00
raito 9a04ef909b feat(nixpkgs): run oxidized channel scripts 
We don't need weird Perl scripts where we are going. Here's a streaming
channel-scripts deployment with plenty of bells, including OTLP.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-31 19:32:23 +02:00
Ilya K c1712dc1fa Set up tempo 2024-08-31 15:05:30 +03:00
raito 8073ae6942 feat(s3-revproxy): tune the cache-control 
Adopt the original values from the Perl script.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-31 00:52:13 +02:00
raito c38e9b482f feat(web): provide a directory listing via s3-revproxy 
Thanks to Jade Lovelace who built all this machinery for Lix initially.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-31 00:29:18 +02:00
raito 024b431cbc feat(grafana): plug jsonnet-based dashboards in provisioning 
Add the gerrit dashboards as an example.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-24 16:32:21 +02:00
Ilya K aef541829e Fix pyroscope datasource 2024-08-24 11:39:25 +03:00
raito 1fc15526d7 fix(pyroscope): add the gRPC endpoint as proxy as well 
This is not documented but necessary for Alloy to operate.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-24 10:33:49 +02:00
raito 2544adba8e fix(gerrit): setup Alloy & Pyroscope more according to the docs 
Still not working due to "unimplemented: error 404 not found" at push
time, but it's really unclear now why this occur.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-24 08:45:20 +02:00
raito 4f4a25a5ad feat(gerrit): push pyroscope profiling to Pyroscope 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-23 22:37:33 +02:00
raito 702867cd62 feat(pyroscope): add push API & reverse proxy 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-23 21:04:22 +02:00
raito 7cde6e92ae feat(grafana): add Pyroscope datasource 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-23 21:04:11 +02:00
raito ac7815321a feat(pyroscope): add secrets and storage 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-23 20:58:08 +02:00
raito db46b01ae9 feat(monitoring): add pyroscope to the infrastructure 
Vendored for the time being.
See https://cl.forkos.org/c/nixpkgs/+/181 for upstreaming properly.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-23 20:43:00 +02:00
raito c380f29937 fix(grafana): remove the global pgsql module dependency for now 
We should re-introduce it once things are a bit scoped out.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-23 20:43:00 +02:00
raito 5dc6165c2e feat(gerrit): add git in the environment to perform git-native clones 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-23 20:43:00 +02:00
raito 58c0dd3d2e feat(public): add listmonk instance on news.forkos.org 
To prepare for public communications and updates.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-21 16:45:12 +02:00
raito 8c35dfa8e0 fix(gerrit): tinker a bit with gerrit defaults for transfer & caching 
We had some issues in the past with too many packfiles and timeout
during transfers, let's try to provide a bit of relief in bad scenarios.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-21 16:31:16 +02:00
Yureka cfc24abfe1 adjust hydra-gc numbers 
for the new ssds
 2024-08-20 12:08:49 +02:00
Pierre Bourdon f938fcb24e
hydra: increase git operations timeout 2024-08-16 17:44:45 +02:00
Pierre Bourdon d3e053809c
hydra: log_prefix needs to be / terminated 2024-08-16 09:25:46 +02:00
Pierre Bourdon e2a990c982
hydra: listen on 127.0.0.1 instead of localhost 
For some cursed reasons, the latter doesn't work on build-coord:

Aug 16 07:06:22 build-coord hydra-server[109560]: Resolved [localhost]:3000 to [::1]:3000, IPv6
Aug 16 07:06:22 build-coord hydra-server[109560]: Resolved [localhost]:3000 to [127.0.0.1]:3000, IPv4
Aug 16 07:06:22 build-coord hydra-server[109560]: Binding to TCP port 3000 on host ::1 with IPv6
Aug 16 07:06:22 build-coord hydra-server[109560]: Binding to TCP port 3000 on host 127.0.0.1 with IPv4
Aug 16 07:06:22 build-coord hydra-server[109560]: 2024/08/16-07:06:22 Can't connect to TCP port 3000 on 127.0.0.1 [Invalid argument]
 2024-08-16 09:20:49 +02:00
Pierre Bourdon c33326f836
hydra: switch to using mTLS instead of local peer auth 2024-08-16 08:19:18 +02:00
Pierre Bourdon 0dd333c573
postgres: add mTLS support 
New client certs can be minted via the provided script, which is meant
to be run on the postgres server (where the CA private key is
conveniently deployed).
 2024-08-16 07:59:12 +02:00
Pierre Bourdon 29babfc5c4
Revert "Partial revert "Add Grapevine Matrix server and matrix-hookshot"" 
This reverts commit 17c342b33e.

Grapevine's use of IFD was fixed upstream.
 2024-08-15 16:22:22 +02:00
Pierre Bourdon 90325344a3
Reserve builder-11 for build coordination, rename to build-coord 2024-08-13 19:12:36 +02:00
Pierre Bourdon 17c342b33e
Partial revert "Add Grapevine Matrix server and matrix-hookshot" 
This partially reverts commit d2f3ca5624.

Said commit requires IFD to eval, which is generally unwanted, and is
currently forbidden on Hydra (imo: rightfully so, we should try to
properly separate evals from builds).

The services/ file for grapevine is kept but will not work without the
flake.nix change reapplied.
 2024-08-13 00:35:10 +02:00
raito 84efd0976d feat(alerts): add a sync failed too often alert 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-09 16:25:34 +02:00
raito e2f5a7b0e4 feat(alerts): add basic postgresql alerts 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-09 16:06:34 +02:00
raito 7388de79c4 feat(alerts): add some basic "host & hardware" alerts 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-08-09 16:06:34 +02:00
Ilya K f8cad42b5c Set up alertmanager-hookshot-adapter 2024-08-09 14:03:56 +00:00