lix-project/buildbot-nix
15
0
Fork 0
Code Issues 13 Pull requests 7 Packages Projects Releases Wiki Activity
294 commits 17 branches 0 tags 2.7 MiB
Commit graph

294 commits

This branch
This branch
All branches
Author SHA1 Message Date
jade d2ad4745c1 Remove --accept-flake-config 
This is a cursed option that is free root for anyone who puts hacks into
flake.nix. We don't actually use `nixConfig` in Lix, so we can just
delete this thing.

Fixes: #11
 2024-05-06 19:08:23 -07:00
raito 3876a30117 feat: support Prometheus exports 
We package a quite old plugin for Buildbot: https://github.com/claws/buildbot-prometheus
Ideally, we should probably vendor it and maintain it ourselves.

There seems to be no protection against the metrics endpoint for
Buildbot, this is not a big deal given that the CI is public.

Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-05-06 14:26:32 +02:00
Qyriad ea4b9ce350 Merge pull request 'feat: support Gerrit in Buildbot' (#1) from gerrit into main 
Reviewed-on: #1
 2024-04-30 19:41:58 +00:00
eldritch horrors 131fc792f7 allow worker counts to be set per arch 2024-04-05 15:13:11 +02:00
eldritch horrors daa84f4169 never build on the coordinator 
for such cases just add the coordinator as a remote builder.
 2024-04-05 14:12:15 +02:00
eldritch horrors 3717bfab04 automatically cancel outdated builds 2024-03-28 03:52:13 +01:00
puck 2eaee8f62b Fix marking jobs as successful if they never finish evaluating. 2024-03-18 00:07:34 +00:00
eldritch horrors d394f35f55 use one scheduler and worker set per arch 
and an additional set for generic tasks like error reporting. this
prevents hol blocking for underutilized arches when at least one arch is
blocking, as usually happens to us with aarch64-linux.
 2024-03-15 14:47:49 +01:00
eldritch horrors 5e50a858d7 revert to stable web ui 
the react-based ui is too slow for our needs, janky, the log viewer
doesn't work quite right (breaking after ~600 lines of logs viewed),
loses updates to sub-builds, and just blanks its entire screen when a
build finishes. the old ui doesn't do that.
 2024-03-15 14:40:23 +01:00
eldritch horrors a9ce436201 fix system builds with binary cache disabled 2024-03-15 13:17:25 +01:00
raito 8d36ac1d90 feat: signing key 
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-12 01:27:46 +01:00
raito 6118daa0a4 feat: binary cache 
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-12 01:27:46 +01:00
puck e9b3b38bbf Skip scheduling cached builds; improve reporter message 2024-03-11 15:05:15 +00:00
puck de02c833d0 Merge pull request 'better gerrit integration' (#5) from better-gerrit into gerrit 
Reviewed-on: #5
 2024-03-11 07:57:50 -06:00
eldritch horrors 5cdef7efb6 fix status reporting to gerrit 
also adjust labels from split verified to single verified, split labels
were only useful during the pre-ci hours
 2024-03-11 14:44:09 +01:00
eldritch horrors 51f7b52149 pre-filter drv_info into all_deps 
otherwise failure reporting is *enormous* with the entirety of a full
derivation info dump in there
 2024-03-11 13:07:35 +01:00
eldritch horrors 13a67b483a fix interrupt() 
can't interrupt with things to interrupt. this is technically duplicated
information but keeping parts of the code close to Trigger seems useful.
 2024-03-11 13:05:12 +01:00
eldritch horrors 9933971ab0 re-enable the gerrit status reporter 2024-03-11 09:06:29 +01:00
eldritch horrors 29a2ef63e2 show hydra job count in trigger step 
previously we immediately triggered all jobs, now we no longer do.
showing the total count at least somewhere is nice to have a rough
indication of how much longer a build may still need to run.
 2024-03-11 09:05:28 +01:00
puck 9a15348984 Fix up a few loose ends 2024-03-11 08:08:55 +01:00
puck 4d73275123 Add build result tracking, schedule newly available builds 2024-03-11 08:08:53 +01:00
puck 28ca39af25 WIP: Replace Trigger with custom logic 2024-03-11 08:06:37 +01:00
eldritch horrors e9874c3d98 wip: dependency-tracked build triggering 2024-03-11 07:53:56 +01:00
eldritch horrors f869b52a8d use build-local gc-root directory 
without this two builds can interfere with each other if:

  - builds 1 and 2 start
  - build 1 is starved of workers
  - build 2 finishes, removes the shared gcroots directory
  - gc runs
  - build 1 schedules more builds whose .drvs have now been removed

using a dedicated directory for each build fixes this.

we now also need to set alwaysRun on the cleanup command or we risk
littering the system with stale gc roots when a build fails.
 2024-03-11 06:48:41 +01:00
eldritch horrors 156e6e3dea remove skipped-builds builder 
run all of them on the normal build worker. this significantly
simplifies the overall scheduler/builder config and removes a
triplication of possible builds paths.
 2024-03-11 06:27:32 +01:00
eldritch horrors 753df8e340 remove cachix 
we aren't using it and it's somewhat in the way of our efforts to
improve scheduling and stuff.
 2024-03-11 06:26:39 +01:00
eldritch horrors 0b2545b036 remove unused GitWithRetry 2024-03-11 06:26:39 +01:00
eldritch horrors fdfeef8ad4 remove retry logic 
retries don't help us very much, in fact they mostly hurt by repeating
builds that failed for non-transient reasons. retries could help with
workers dropping while running a build, but those rare cases are better
to restart manually than to pend at least twice the ci time for commits
that simply do not build cleanly.
 2024-03-11 06:26:38 +01:00
puck e1dfa0e545 Remove cachix from dependencies 
The rest of the Cachix infrastructure is intact. For now.
 2024-03-10 13:03:16 +00:00
puck ec2ef903ab use .#hydraJobs rather than .#checks 2024-03-08 23:28:49 +00:00
puck c487ada514 review callback: Set labels, render list of failed checks in reporter 2024-03-07 04:40:58 +00:00
raito 18d537e5d4 nix/coordinator(reporting): re-enable reviewCB, disable summaryCB, skip evaluations 
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-04 20:50:12 +01:00
raito 322944f8d1 nix/coordinator(reporting): disable startCB and reviewCB 
excessive noise.

Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-04 20:37:13 +01:00
raito 9a67a1f2cc nix/coordinator: enable reporting to Gerrit 
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-03 02:58:25 +01:00
raito 2bcc0ee4e9 nix/coordinator: hardcode remaining github properties to lix 
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-02 23:22:36 +01:00
raito 3895b90493 nix/coordinator: disable reporter 
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-02 23:07:36 +01:00
raito 670a1d1281 nix/coordinator: fix repourl 
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-02 23:07:36 +01:00
raito e92a2225f7 nix/coordinator: use a special key for service account 
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-02 23:07:33 +01:00
raito 94e3d7aeda nix/coordinator: support specific private SSH keys 
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-02 23:07:33 +01:00
raito 9f98533dd7 nix/worker: rename master → coordinator 
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-02 22:09:56 +01:00
raito 5a8ab145e6 buildbot: add repourl 
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-02 22:09:56 +01:00
raito 7ad9c1a378 nix/coordinator: introduce OAuth2 client secret 
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-02 22:09:56 +01:00
raito 81bd57ffac nix/coordinator: instantiate the GerritNixConfigurator 
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-02 22:09:56 +01:00
raito 101612eb70 nix/coordinator: fix various errors 
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-02 22:09:56 +01:00
raito 7ace78107c buildbot: further Gerritification 
- `BuildTrigger` still depends on `github` properties.

Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-02 22:09:56 +01:00
raito 5f7b3e0cdb buildbot: instantiate fully LixSystemsOAuth2 
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-02 22:09:56 +01:00
raito beea96da2c nix/coordinator: simplify the module 
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-02 18:45:19 +01:00
raito 329d9dd6d4 nix/coordinator: rename it into what this really is 
No need to use legacy names.

Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-02 18:45:19 +01:00
raito acfd225e6d buildbot/__init__.py: rework the file for gerrit usecase, our oauth2 instance 
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2024-03-02 18:45:19 +01:00
github-actions[bot] 7a44c9f71c flake.lock: Update 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:Nixos/nixpkgs/6d8aba54f305eb12a57d92ad4eaeb42049961f19' (2024-02-24)
  → 'github:Nixos/nixpkgs/96990c69fd59e807cee8ff874d0526c70dffdf8f' (2024-02-28)
• Updated input 'treefmt-nix':
    'github:numtide/treefmt-nix/36fd6923c122a983bc3915692e6cb3ff341ef083' (2024-02-23)
  → 'github:numtide/treefmt-nix/e497a9ddecff769c2a7cbab51e1ed7a8501e7a3a' (2024-02-25)
 2024-02-29 01:34:25 +00:00