Grafana monitoring #345

New issue

Open

opened 2024-11-12 19:42:47 +00:00 by RaitoBezarius · 2 comments

RaitoBezarius commented

2024-11-12 19:42:47 +00:00

(Migrated from github.com)

Set up monitoring for:

Last time evaluation ran (if it's too old, raise a warning)
Last time successful delta ran (if it's too old, raise a warning), alternatively just pick it from the database latest dump ingested
Last time matching ran (if it's too old, raise a warning)

Set up dashboards for:

PGSQL disk size
Query performance on the PGSQL (classical PG dashboards)

Set up monitoring for: - Last time evaluation ran (if it's too old, raise a warning) - Last time successful delta ran (if it's too old, raise a warning), alternatively just pick it from the database latest dump ingested - Last time matching ran (if it's too old, raise a warning) Set up dashboards for: - PGSQL disk size - Query performance on the PGSQL (classical PG dashboards)

fricklerhandwerk commented

2024-12-09 09:15:51 +00:00

(Migrated from github.com)

Discussed with @Erethon:

Will put perf metrics to prometheus.nixos.org for now
NixOS Infra Matrix had a history of being spammed with alerts, may want to consider our own instance for maintenance for production deployment
- Or send alerts somewhere else

Discussed with @Erethon: - Will put perf metrics to prometheus.nixos.org for now - NixOS Infra Matrix had a history of being spammed with alerts, may want to consider our own instance for maintenance for production deployment - Or send alerts somewhere else

Erethon commented

2024-12-12 18:53:18 +00:00

(Migrated from github.com)

Since last night we have metrics from prometheus node exporter on the NixOS foundation Prometheus. The node metrics can be found here.

After the demo we can see what other metrics we want (Postgres exporter is easy to add, as well as custom queries).

Since last night we have metrics from prometheus node exporter on the NixOS foundation Prometheus. The node metrics can be found [here](https://grafana.nixos.org/d/rYdddlPWk/node-exporter-full?orgId=1&from=now-24h&to=now&timezone=browser&var-datasource=default&var-job=node&var-node=tracker.security.nixos.org:9100&var-diskdevices=%5Ba-z%5D%2B%7Cnvme%5B0-9%5D%2Bn%5B0-9%5D%2B%7Cmmcblk%5B0-9%5D%2B&refresh=1m). After the demo we can see what other metrics we want (Postgres exporter is easy to add, as well as custom queries).