Commit graph

188 commits

Author SHA1 Message Date
raito cf98ed80dc chore: introduce finer-grained baremetal management
for multiple roles such as storage or builders.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-12-23 21:30:38 +01:00
emily f593645cde
feat(forgejo): add robots.txt 2024-12-23 21:22:11 +01:00
emily e930a17b0b
fix(forgejo): lower cache.last_commit TTL to limit size of the cache
We really don't want to cache them for a year, which is the default.

Yes, computing them may be expensive, but not worth a multi-gigabyte
redis database that takes minutes to load into RAM on service (re)start.
2024-12-18 17:03:49 +01:00
emily 4e87e35bb5
feat(forgejo): offload custom forgejo package into its own repository
It has been a recurring issue that flake lockfile bumps in this repo
here make the forgejo patches no longer apply.

The dedicated repository (nix-forgejo) solves this by not overriding the
existing forgejo derivation from nixpkgs but rather having its own.

Additionally, nix-forgejo pins and uses a "known good" nixpkgs revision
itself, unless `pkgs` is passed on import.

So if issues should arise after a flake bump, we can use that revision
by modifying our import statement, or we can rollback the nix-forgejo
revision itself.

Moving forgejo out of tree also makes iterating on it a lot easier and
opens a lot of other possibilities :)
2024-12-18 03:39:37 +01:00
raito f4588aff2b feat: listen on Gerrit events and rewrite them as generic VCS events
This introduces the private SSH key for Gerrit event streaming.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-12-16 01:25:53 +01:00
raito 90038e80a2 fix: do not propagate rabbitmq-password to all nodes
This was a mistake.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-12-16 00:25:54 +01:00
raito 665a750e35 chore: fix vhost and username for ofborg
Username and vhost creation are out of band and manual.

$ cd /var/lib/rabbitmq
$ sudo -u rabbitmq rabbitmqctl create_user ofborg $pwd
$ sudo -u rabbitmq rabbitmqctl set_permissions ofborg '.*' '.*' '.*'

Here's a simple way to reproduce that setup on the RabbitMQ server.

Doing better will require the Vault server which will come soon anyway.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-12-15 18:19:49 +01:00
raito df0bd6b4eb feat: introduce statcheck worker
Status & checks RPC & event queue.

The status & checks is set by the rest of OfBorg, the web service needs
to be exposed.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-12-15 16:51:45 +01:00
raito c007bbeeb9 feat: introduce ofborg gerrit streamer
This pipes events from Gerrit into the whole AMQP broker and enable all
the system to react to VCS changes.

We need a filter to transform raw Gerrit events into ofBorg specific
events that we will continue to send in the system.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-12-15 16:51:45 +01:00
raito 4fe922bcd0 feat: introduce ofborg mass rebuilder
With Gerrit support.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-12-15 16:51:45 +01:00
raito adb78e633c feat: introduce ofborg pastebin service
The web service is not available yet.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-12-15 16:51:45 +01:00
raito 8fa0e5abe3 feat: introduce ofborg stats
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-12-15 16:51:45 +01:00
raito 47b713ca58 feat: introduce ofborg builder
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-12-15 16:51:45 +01:00
raito 436882c3eb fix(services/vault): proxy pass to the local vault server web port
Oopsie, forgot that commit.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-12-15 16:51:11 +01:00
raito 14f5bc10a1 chore(pkgs/openbao): 2.0.2 -> 2.1.0
https://openbao.org/docs/release-notes/2-1-0/
https://openbao.org/docs/release-notes/2-0-0/#203
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-12-15 16:42:48 +01:00
raito 2c4e60760f feat: introduce a Vault module for secrets management
Via a fork of the Linux Foundation, called OpenBao.

The module supports high availability but we only have one node for now.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-12-14 21:49:44 +01:00
raito dc23bb7054 feat: introduce awareness module for WAN addresses
Introduce a data-only module to perform abstraction on the deployment,
we use it for WAN for now.

The usecase is service discovery for simple cases.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-12-14 21:47:53 +01:00
raito 84899b48ea feat(channel-scripts): support push to git and automatic cleanup of failed streaming
Now, we won't pile a bunch of failed streaming attempts and this will
automatically push to git.

Credentials are left to be done for the push to actually work.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-12-14 19:06:47 +01:00
raito 112f60afd1 feat(ows): support moving away onewaysync
We are running into too many out of disk space situations with OWS on
the main disk.

This way, we can reuse the Gerrit disk for all that data, which
hopefully, is quite shared with Gerrit.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-12-14 14:42:22 +00:00
Yureka 2e7a702c28 adjust builder assignments 2024-12-09 10:28:20 +01:00
Ilya K cae763e8dd ofborg: enable nginx for certs 2024-11-17 14:48:08 +03:00
Ilya K 54e8282aac fix: use promtool to verify rules, fix format 2024-11-12 23:21:30 +03:00
Ilya K b08330c42b fix: allow insecure netbox 2024-11-12 22:52:11 +03:00
raito f56576d644 fix: add util-linux in git-gc-preserve
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-10-28 12:07:54 +01:00
raito 02f8bc7ca4 chore(o11y): filter by tenancy on node_exporter
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-10-22 16:57:37 +02:00
raito e0c029ba43 fix: make all buildbot postgres faster
work_mem was effectively absurdly low.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-10-22 16:57:24 +02:00
raito b1f4674da0 chore: add tenancy in postgres
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-10-22 16:57:06 +02:00
raito 226eacdeec chore: add tenancy in node_exporter
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-10-22 16:57:06 +02:00
Ilya K 14935c5e92 fix: update grapevine config 2024-10-21 16:31:26 +03:00
raito bee402fecc fix: ensure that pg_stat_statements is always created as an ext
Otherwise, we will have issues with this exporter.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-10-21 14:33:18 +02:00
raito 3efdd0f6c9 fix: disable gitiles on gerrit01
It is generating too much traffic and CPU load for no good reason.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-10-20 11:24:58 +02:00
raito 8c0c7b517f feat: block automatically crawlers if the blocker is enabled
This help us getting rid of useless traffic by crawlers.

It is enabled for gerrit01 which is suffering the most from this.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-10-19 19:12:10 +02:00
raito e3129fec51 fix(buildbot): fix CORS properly
wildcards are not allowed in the headers.
We need to include credentials as well.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-10-18 23:22:51 +00:00
Luke Granger-Brown 84cfbdb050 feat: check formatting and validity of alerts
Fixes #94.
2024-10-07 20:00:54 +00:00
Luke Granger-Brown 6a8f49f180 feat(gerrit): add some basic theming
This is based on some of the preliminary colour work done by @ckie in
the the-distro/floral.systems repo.
2024-10-07 19:27:13 +00:00
Luke Granger-Brown de085155a6 fix: update paths to floral secrets to secrets/floral/ 2024-10-07 15:48:05 +00:00
Luke Granger-Brown 2001012325 feat(uptime-kuma): status.forkos.org should point at the ForkOS page 2024-10-07 15:47:33 +00:00
raito fbf26302b6 hotfix(lix): use build01 features for build02 remote builder
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-10-07 15:26:27 +02:00
raito b56b8963a2 feat: introduce Buildbot multi-tenancy
This shares the same expression to deploy the Buildbot.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-10-06 11:28:29 +02:00
raito 1e421889e4 feat(monitoring): add static label for tenancy
So we can distinguish easily things in the dashboards.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-10-06 11:10:16 +02:00
raito 002db9a78f feat: introduce tenant-specific extra build capacity
At Lix, we have few aarch64-linux and aarch64-darwin systems we use to
boost our CI.

This is a module to handle tenant-specific extra build capacity without
it leaking over the rest of the deployment.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-10-06 11:09:23 +02:00
raito 92560708b8 feat: multi-tenant secrets
Lix may have its own secrets and we want to maintain a certain
generalization level on the NixOS modules, so we can decorrelate which
secret we select dynamically by having a simple tenancy hierarchy
system.

This unfortunately requires to rewrite all call sites with a floral
prefix until we migrate them to the simple internal secret module which
is aware of this.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-10-06 08:10:44 +00:00
raito a2eecd1886 feat(buildbot): disable manhole debugging
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-10-06 07:59:56 +00:00
raito b5d412a5ba feat: adopt new version of Buildbot with incoming ref data
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-10-06 07:59:56 +00:00
raito 1a862b2b0f hotfix: add the path to the stateless uptime kuma's password file
Forgotten in the previous merge.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-10-05 16:33:38 +02:00
Ilya K 5582a0a29b Fix Hydra exporter crash loop nonsense 2024-10-01 19:27:13 +03:00
Ilya K 4ddf87fa8e Add new metric to Hydra exporter 2024-10-01 19:27:05 +03:00
Kiara Grouwstra b291caac46 feat(monitoring): add uptime-kuma for status page, fixes #97
Adds a service for a status page using
[`uptime-kuma`](https://uptime.kuma.pet/).
2024-10-01 16:13:23 +00:00
Ilya K e2c6550796 Hydra metrics
Yoink the nixos org exporter, rewrite most of it, deploy
2024-10-01 19:06:26 +03:00
Janik Haag d462e8ca9c
feat(gerrit): run git-gc-preserve on a daily timer 2024-09-18 22:27:57 +02:00