danderson/infra
0
0
Fork 0
forked from the-distro/infra
Code Pull requests Activity
205 commits 17 branches 0 tags 1.2 MiB
Commit graph

205 commits

This branch
This branch
All branches
Author SHA1 Message Date
Ilya K e608b92e4f Add htop and btop to default machine config 2024-07-10 15:01:09 +03:00
Ilya K e84b362b7a Allow 12 hour of backfill for metrics 
This is somewhat experimental and may explode, but we'll see, I guess
 2024-07-10 14:59:09 +03:00
Ilya K 9e7e6d42ab Make nginx/loki/mimir go fast 2024-07-10 14:55:28 +03:00
Pierre Bourdon f2c2bc5ab6
hydra: output machine host key as base64 in the generated machines.conf 2024-07-10 02:16:45 +02:00
Pierre Bourdon f214da9228
hydra: add hydra to nix trusted-users 2024-07-10 02:03:33 +02:00
Pierre Bourdon 0e24c18815
tf/hydra: add a test staging-small nixpkgs jobset 2024-07-10 01:57:12 +02:00
Pierre Bourdon 2ac0a599fb
bagel-box: fix missing toString 2024-07-10 01:51:31 +02:00
raito 0ccf0b023e Merge pull request 'hydra: moar power' (#51) from moar-builders into main 
Reviewed-on: the-distro/infra#51
Reviewed-by: Luke Granger-Brown <lukegb@noreply.git.lix.systems>
 2024-07-09 23:32:38 +00:00
Luke Granger-Brown 82db8f7f1e gerrit01: some more tuning 
* flip off proxy_buffering again
* enable REVWALK_USE_PRIORITY_QUEUE
* enable delta compression, because that's not a bottleneck and it's
  nicer on bandwidth
 2024-07-10 00:27:36 +01:00
raito 7b7a645cfc hydra: moar power 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-07-10 01:27:35 +02:00
Yureka 39d2352bbc general quality of life improvements 2024-07-09 23:26:12 +00:00
Yureka a7d21e96a0 add global hardening options 2024-07-09 23:26:12 +00:00
raito c51676a560 Merge pull request 'hydra: unplug the EPYC' (#49) from unplug-epyc into main 
Reviewed-on: the-distro/infra#49
 2024-07-09 23:26:08 +00:00
raito 9988811be5 hydra: unplug the EPYC 
thank you for your testing services

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-07-10 01:13:10 +02:00
Pierre Bourdon afaf49eb97
secrets: rekey 2024-07-10 01:05:05 +02:00
Pierre Bourdon bc8ef7b5fc
ssh-keys: remove raito's key which is too NSA'd for agenix 2024-07-10 01:04:48 +02:00
Pierre Bourdon 61e8048445
sysadmin: remove pwru, does not build on latest nixpkgs 2024-07-10 01:01:27 +02:00
Pierre Bourdon 2ebb0e82e8
flake.lock: Update 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/9693852a2070b398ee123a329e68f0dab5526681' (2024-06-22)
  → 'github:NixOS/nixpkgs/ab82a9612aa45284d4adf69ee81871a389669a9e' (2024-07-07)
 2024-07-10 01:01:17 +02:00
raito 664fa033aa Merge pull request 'hydra: wire up new builders' (#47) from hydra-wire-up into main 
Reviewed-on: the-distro/infra#47
 2024-07-09 23:00:27 +00:00
raito 2308870aa5 builders: add a nice tag to deploy all of them at once 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-07-10 00:59:31 +02:00
raito f9f955214f ssh-keys: add raito to secrets set 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-07-10 00:59:22 +02:00
raito 90e54d7292 terraform: add DNS records for VPN-GW & builders 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-07-10 00:55:42 +02:00
raito 645ad7d062 builders: add builder user 
currently hardcoded to hydra's coordinator public key

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-07-10 00:55:25 +02:00
raito a30c1f7d78 hydra: wire up new builders 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-07-10 00:45:02 +02:00
Yureka eb21cb6916 add baremetal builders 2024-07-10 00:35:01 +02:00
Yureka 62af42fc97 init wob-vpn-gw host 2024-07-09 23:42:20 +02:00
Yureka 7396107bf4 add a shim to provide nixosConfigurations from colmena hive 2024-07-09 10:49:29 +02:00
Yureka c0e1d05b3c admins: add yuka 2024-07-09 10:34:30 +02:00
raito 3828721e4f services/netbox: enable OIDC via Lix SSO 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-07-09 02:45:58 +02:00
Luke Granger-Brown 8a9ff8c40d services/gerrit: migrate to Gerrit from the-distro/nix-gerrit flake 2024-07-08 23:30:59 +01:00
raito 48579e8818 feat: add gdb to sysadmin tooling 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-07-08 22:10:06 +00:00
raito 8fe33b4e46 feat: add perf, pwru and various sysadmin tools to bagel-box 
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-07-08 22:10:06 +00:00
Luke Granger-Brown d4e9dcc2a6 admins: provision lukegb 
hello I can be trusted with your infrastructure
 2024-07-08 21:55:41 +00:00
Pierre Bourdon 7f46e5d9a4
services: add ofborg, currently running rabbitmq only 2024-07-08 23:55:11 +02:00
raito 512cfdb43e fix: downgrade mina sshd due to broken PQC algorithm 
https://cl.tvl.fyi/c/depot/+/11965

This breaks it with "ssh_dispatch_run_fatal: Connection to
2a01:4f8:242:5b21:0:feed:edef:beef port 29418: incorrect signature"

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
 2024-07-08 15:59:31 +02:00
raito 82395ec8ce Merge pull request 'pkgs/gerrit: update to 3.10.0' (#34) from upgrade-gerrit-differently into main 
Reviewed-on: the-distro/bagel-infra#34
 2024-07-08 12:21:21 +00:00
Ilya K 82e074881f DNS: clean up a bit, add root level record for future Matrix shenanigans 2024-07-08 13:54:15 +03:00
Ilya K b55475c12e Fix up the rest of the dashboards 2024-07-08 11:43:57 +03:00
Ilya K 9f0e601d84 Scrape grafana/loki/mimir own metrics 2024-07-08 10:25:15 +03:00
Ilya K 209f71c63a Update node_exporter dashboard for new metrics structure 2024-07-08 10:16:37 +03:00
Ilya K 563e0685d4 Metrics fixups 
- fix grafana-agent config format
- rekey metrics-push-password for fodwatch
 2024-07-08 10:01:25 +03:00
emily 8d2a367e92 grafana-agent: make bagel.monitoring.grafana-agent.exporters an attrset 
This allows us to use multiple jobs, one for each additional exporter,
and set their `job_name` accordingly.

`job_name` is exported as `job` label on the resulting metrics.
This allows us to quickly get an understanding what metrics of an
exporter are actually available by simply filtering all metrics by
`{job="$jobname"}`
 2024-07-08 09:34:26 +03:00
emily db8c831c2f grafana-agent: set hostname label on all metrics 
This is handy to quickly see all metrics exported by a node, without
having to mangle with the already existing `instance` label.

`hostname` is essentially a variant of `instance` but without ports.
 2024-07-08 09:34:26 +03:00
Ilya K ba0d50624d Switch to push metrics with Grafana Agent 2024-07-08 09:34:24 +03:00
Ilya K 40ba3c4ae7 Prepare for remote push metrics 2024-07-08 09:33:59 +03:00
Ilya K 346a74eabc Wire up Grafana to Alertmanager 2024-07-08 09:33:59 +03:00
Ilya K e8e262c6a4 Enable Mimir Alertmanager, add example alert 
Still TODO: actually connect it to Matrix
 2024-07-08 09:33:59 +03:00
Luke Granger-Brown dd6ee53bfe pkgs/gerrit: update to 3.10.0 
This does a bit more than advertised, since this also switches to a
different set of Bazel package building infrastructure that I'm hoping
will be more extensible than buildBazelPackage as it exists in nixpkgs
today.

In any case, the FOD here _seems_ to be much more stable than that
previously produced by the old approach, but no promises :)
 2024-07-08 02:44:05 +01:00
Pierre Bourdon 5ebd71e4d5
tf/hydra: change Hydra URL 2024-07-08 00:01:24 +02:00
Pierre Bourdon 2700ac5efc
tf/dns: fix hydra CNAME 2024-07-08 00:01:14 +02:00