Commit graph

5 commits

Author SHA1 Message Date
Thomas Draebing 7088daaa31 Add option to use vault to manage key used for encryption
Using a local PGP-key for encryption of the secrets in the configuration
is not very secure and makes it hard to rotate and distribute the
key. Sops provides the option to use managed services for this
purpose, e.g. HashiCorp Vault.

This change adds the option to use HashiCorp Vault, when using the
provided python scripts to encrypt the config file.

Change-Id: I7683fbfdbed00506c3bca264ac8565f48bc5ea73
2022-05-09 06:59:40 +00:00
Thomas Draebing 6b75c12831 Rewrite the scripts in python
The scripts were written in bash. Using bash became quite unwieldy.

Python by nature can deal well with yaml and is thus better suited
in dealing with the yaml-based configuration files. This change
rewrites the original scripts staying as close as possible to the
original ones.

Right now, the python scripts call subprocesses a lot to work with
the tools, which were already used before. At least for yaml-
templating there may be better tools that have a python integration,
which could be used in the future.

Change-Id: Ida16318445a05dcfdada9c7a56a391e4827f02e7
2020-04-16 14:25:50 +02:00
Thomas Draebing eb4e6ea191 Use object store to store chunks created by Loki
The chunks created by Loki were stored in a persistent volume. This
does not scale well, since volumes cannot easily be resized in
Kubernetes. Also, at least the ext4-filesystem had issues, when large
numbers of logs were saved. These issues are due to the dir_index as
discussed in [1].

An object store provides a more scalable and cheaper solution. Loki
supports S3 as an object storage and also other object stores that
understand the S3 API like Ceph or OpenStack Swift.

[1] https://github.com/grafana/loki/issues/1502

Change-Id: Id55095c3b6659f40708712c1a494753dbcab7686
2020-03-24 16:01:34 +01:00
Thomas Draebing 8daaa2695f Add basic dev documentation
Change-Id: I6de025c38fa87d4b70bdd4d8eaf261ced97716f2
2020-03-11 15:23:19 +01:00
Thomas Draebing be862d863e Move internal project to open source
This change adds the current status of a project that aims to create
a simple monitoring setup to monitor Gerrit servers, which was developed
internally at SAP.

The project provides an opinionated and basic configuration for helm
charts that can be used to install Loki, Prometheus and Grafana on a
Kubernetes cluster. Scripts to easily apply the configuration and
install the whole setup are provided as well.

The contributions so far were done by (with number of commits)

  80  Thomas Draebing
  11  Matthias Sohn
   2  Saša Živkov

Change-Id: I8045780446edfb3c0dc8287b8f494505e338e066
2020-03-11 15:23:19 +01:00