Commit graph

141 commits

Author SHA1 Message Date
Pierre Bourdon f2c2bc5ab6
hydra: output machine host key as base64 in the generated machines.conf 2024-07-10 02:16:45 +02:00
Pierre Bourdon f214da9228
hydra: add hydra to nix trusted-users 2024-07-10 02:03:33 +02:00
Luke Granger-Brown 82db8f7f1e gerrit01: some more tuning
* flip off proxy_buffering again
* enable REVWALK_USE_PRIORITY_QUEUE
* enable delta compression, because that's not a bottleneck and it's
  nicer on bandwidth
2024-07-10 00:27:36 +01:00
raito 9988811be5 hydra: unplug the EPYC
thank you for your testing services

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-10 01:13:10 +02:00
raito 2308870aa5 builders: add a nice tag to deploy all of them at once
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-10 00:59:31 +02:00
raito 645ad7d062 builders: add builder user
currently hardcoded to hydra's coordinator public key

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-10 00:55:25 +02:00
raito a30c1f7d78 hydra: wire up new builders
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-10 00:45:02 +02:00
Yureka eb21cb6916 add baremetal builders 2024-07-10 00:35:01 +02:00
raito 3828721e4f services/netbox: enable OIDC via Lix SSO
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-09 02:45:58 +02:00
Luke Granger-Brown 8a9ff8c40d services/gerrit: migrate to Gerrit from the-distro/nix-gerrit flake 2024-07-08 23:30:59 +01:00
Pierre Bourdon 7f46e5d9a4
services: add ofborg, currently running rabbitmq only 2024-07-08 23:55:11 +02:00
Ilya K b55475c12e Fix up the rest of the dashboards 2024-07-08 11:43:57 +03:00
Ilya K 9f0e601d84 Scrape grafana/loki/mimir own metrics 2024-07-08 10:25:15 +03:00
Ilya K 209f71c63a Update node_exporter dashboard for new metrics structure 2024-07-08 10:16:37 +03:00
Ilya K 563e0685d4 Metrics fixups
- fix grafana-agent config format
- rekey metrics-push-password for fodwatch
2024-07-08 10:01:25 +03:00
emily 8d2a367e92 grafana-agent: make bagel.monitoring.grafana-agent.exporters an attrset
This allows us to use multiple jobs, one for each additional exporter,
and set their `job_name` accordingly.

`job_name` is exported as `job` label on the resulting metrics.
This allows us to quickly get an understanding what metrics of an
exporter are actually available by simply filtering all metrics by
`{job="$jobname"}`
2024-07-08 09:34:26 +03:00
emily db8c831c2f grafana-agent: set hostname label on all metrics
This is handy to quickly see all metrics exported by a node, without
having to mangle with the already existing `instance` label.

`hostname` is essentially a variant of `instance` but without ports.
2024-07-08 09:34:26 +03:00
Ilya K ba0d50624d Switch to push metrics with Grafana Agent 2024-07-08 09:34:24 +03:00
Ilya K 40ba3c4ae7 Prepare for remote push metrics 2024-07-08 09:33:59 +03:00
Ilya K 346a74eabc Wire up Grafana to Alertmanager 2024-07-08 09:33:59 +03:00
Ilya K e8e262c6a4 Enable Mimir Alertmanager, add example alert
Still TODO: actually connect it to Matrix
2024-07-08 09:33:59 +03:00
Pierre Bourdon caa1fce74e
hydra: move to hydra.forkos.org 2024-07-07 23:53:21 +02:00
Ilya K 5b0f3c4541 Split node_exporter and cadvisor config, disable cadvisor for nodes that are themselves containers 2024-07-05 20:06:43 +03:00
raito b319b02f07 fix: remove custom logging format for Gerrit
This way, we get picked up by the LGTM stack exporter machinery.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-05 18:52:38 +02:00
Ilya K 2441d18f17 Add Loki + Promtail setup 2024-07-05 16:10:31 +00:00
Ilya K 03cb9c390c Add postgres exporter 2024-07-05 16:10:31 +00:00
Ilya K 42f8ad8fa4 Add nginx log exporter 2024-07-05 16:10:31 +00:00
Ilya K 63b31e98cf Add Grafana/Prometheus/Mimir minimal setup
More later, Loki also later.
2024-07-05 16:10:31 +00:00
Pierre Bourdon 34a29552da
hydra: update the epyc.infra.newtype.fr public host key 2024-07-05 16:43:29 +02:00
raito 0b01e9a99f gerrit01: those who finetune even further
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-05 12:23:44 +02:00
raito 6c237e8d40 gerrit01: make it go brrr on https clone
proxy_buffering was the root cause.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-04 14:42:49 +02:00
Pierre Bourdon e387fffd66
hydra: add i686-linux support to the remote builder because nixpkgs bootstrap relies on it, even on x86_64 2024-07-04 13:44:59 +02:00
raito 182e55c35f gerrit01: rename to cl.forkos.org
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-03 10:58:49 +02:00
raito 98a33e4300 gerrit01: init
With:

- A package hierarchy
- A source-based Gerrit deployment

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-01 21:22:36 +02:00
raito e3f3c87c0d meta01: init
Includes:

- Raito VM module
- Raito proxy aware NGINX module
- Base server module
- Sysadmin module
- New SSH keys
- Netbox module

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-01 19:40:37 +02:00
Pierre Bourdon be5c6f0656
postgres: fix permissions on the dataDir, it refuses 0770 2024-06-24 21:45:17 +02:00
Pierre Bourdon 2ed6f92ed8
postgres: bump max connections count 2024-06-24 21:45:17 +02:00
Pierre Bourdon cb6e5b1652
hydra: actually use version from flake 2024-06-24 21:45:17 +02:00
Pierre Bourdon 73aecaef41
hydra: provide S3 and SSH credentials (via agenix) 2024-06-24 20:59:19 +02:00
Pierre Bourdon 04bd33e32c
infra: add agenix, add s3 credentials 2024-06-24 18:03:20 +02:00
Pierre Bourdon 91beb0eddc
bagel-box: add postgres+hydra 2024-06-24 18:03:20 +02:00