Commit graph

267 commits

Author SHA1 Message Date
c49e591ae5 chore(nix-builders): remove legacy system field
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-19 15:59:39 +02:00
7cafac3125 chore(dataclass): use default_factory
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-19 15:48:07 +02:00
4ad5459da8 chore(security): clean up accept-flake-config mistakes
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-18 12:15:38 +02:00
f5499f5922 chore(builders): localize builders specification like Hydra does
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-18 12:15:38 +02:00
1cb13868bc chore(entrypoint): hydraJobs → buildbotJobs
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-18 12:12:10 +02:00
8286c1028b chore(systemd): add ssh in the path
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-17 17:25:51 +02:00
3821a116f5 chore(reporters): make it 3.11+ (and 4.0) compatible!
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-17 17:22:34 +02:00
8a1b11f8b1 fix(gerrit): make buildbot able to read the priv ssh key
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-17 16:56:21 +02:00
4e5f0cb44b chore(auth): further generalize authn
So that it's possible to plug another OAuth2 instance.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-17 16:46:20 +02:00
164ba3b014 chore(origins): expose in a cuter way allowed origins
Worked around in our original deployment, here's a nicer way to set it.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
2024-07-17 16:33:04 +02:00
d5e3345097 chore(schedule): generalize source
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-06 21:03:58 +02:00
8dd3d67735 chore(review-callback): generalize the event project name
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-06 20:26:42 +02:00
98b5d1ed27 chore(nix-eval): generalize the builds_scheduler_group by project
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-06 20:26:28 +02:00
fd058a5022 chore(nix): make the target attribute a constant
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-06 20:11:12 +02:00
f647440f86 chore(gerrit): put the gerrit configuration in one place and generate repo URLs templates
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-06 20:10:52 +02:00
b417cd4519 chore(canceller): generalize it to any project
Just iterate over all project names.

Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-06 19:57:12 +02:00
87a9b9df01 chore(auth): generalize authentication method to internals of NixOS module
This makes it easier to make it configurable, this is step 1.

Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-06 19:52:55 +02:00
914c28af91 chore(*): cleanup unused code
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-06 19:39:35 +02:00
2a7b42ef6b chore(gerrit): offer projects configuration and factor out private SSH keys
Previously, we needed to hardcode the URL for private SSH keys,
this is cleaned up and we can iterate over each project for its
configuration.

Configuration is at deployment time.

Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-06 19:39:16 +02:00
f221ab324e chore(web): remove outputsPath option
It was relying on GitHub stuff which we don't have and is not an option
we want to support.

If we wanted to do it, we would rather use S3 directly.

Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-06 19:37:46 +02:00
7c92e8315e chore(nix): factor out the Gerrit configuration to the Nix module
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-06 18:40:38 +02:00
7b2fc73c78 chore(statuses): clarify why we don't use {start, summary}CB
Instead of just commenting them out.

Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-06 18:27:38 +02:00
6543b8e414 chore(flake): rename the description
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-06 18:26:48 +02:00
5fe3d87802 chore(pyproject): add authorship information
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-06 18:26:36 +02:00
fef4c0d808 chore(*): cleanup buildbot-effects
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-06 18:26:08 +02:00
ea4b9ce350 Merge pull request 'feat: support Gerrit in Buildbot' (#1) from gerrit into main
Reviewed-on: #1
2024-04-30 19:41:58 +00:00
131fc792f7 allow worker counts to be set per arch 2024-04-05 15:13:11 +02:00
daa84f4169 never build on the coordinator
for such cases just add the coordinator as a remote builder.
2024-04-05 14:12:15 +02:00
3717bfab04 automatically cancel outdated builds 2024-03-28 03:52:13 +01:00
2eaee8f62b Fix marking jobs as successful if they never finish evaluating. 2024-03-18 00:07:34 +00:00
d394f35f55 use one scheduler and worker set per arch
and an additional set for generic tasks like error reporting. this
prevents hol blocking for underutilized arches when at least one arch is
blocking, as usually happens to us with aarch64-linux.
2024-03-15 14:47:49 +01:00
5e50a858d7 revert to stable web ui
the react-based ui is too slow for our needs, janky, the log viewer
doesn't work quite right (breaking after ~600 lines of logs viewed),
loses updates to sub-builds, and just blanks its entire screen when a
build finishes. the old ui doesn't do that.
2024-03-15 14:40:23 +01:00
a9ce436201 fix system builds with binary cache disabled 2024-03-15 13:17:25 +01:00
8d36ac1d90 feat: signing key
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-12 01:27:46 +01:00
6118daa0a4 feat: binary cache
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-12 01:27:46 +01:00
e9b3b38bbf Skip scheduling cached builds; improve reporter message 2024-03-11 15:05:15 +00:00
de02c833d0 Merge pull request 'better gerrit integration' (#5) from better-gerrit into gerrit
Reviewed-on: #5
2024-03-11 07:57:50 -06:00
5cdef7efb6 fix status reporting to gerrit
also adjust labels from split verified to single verified, split labels
were only useful during the pre-ci hours
2024-03-11 14:44:09 +01:00
51f7b52149 pre-filter drv_info into all_deps
otherwise failure reporting is *enormous* with the entirety of a full
derivation info dump in there
2024-03-11 13:07:35 +01:00
13a67b483a fix interrupt()
can't interrupt with things to interrupt. this is technically duplicated
information but keeping parts of the code close to Trigger seems useful.
2024-03-11 13:05:12 +01:00
9933971ab0 re-enable the gerrit status reporter 2024-03-11 09:06:29 +01:00
29a2ef63e2 show hydra job count in trigger step
previously we immediately triggered all jobs, now we no longer do.
showing the total count at least somewhere is nice to have a rough
indication of how much longer a build may still need to run.
2024-03-11 09:05:28 +01:00
9a15348984 Fix up a few loose ends 2024-03-11 08:08:55 +01:00
4d73275123 Add build result tracking, schedule newly available builds 2024-03-11 08:08:53 +01:00
28ca39af25 WIP: Replace Trigger with custom logic 2024-03-11 08:06:37 +01:00
e9874c3d98 wip: dependency-tracked build triggering 2024-03-11 07:53:56 +01:00
f869b52a8d use build-local gc-root directory
without this two builds can interfere with each other if:

  - builds 1 and 2 start
  - build 1 is starved of workers
  - build 2 finishes, removes the shared gcroots directory
  - gc runs
  - build 1 schedules more builds whose .drvs have now been removed

using a dedicated directory for each build fixes this.

we now also need to set alwaysRun on the cleanup command or we risk
littering the system with stale gc roots when a build fails.
2024-03-11 06:48:41 +01:00
156e6e3dea remove skipped-builds builder
run all of them on the normal build worker. this significantly
simplifies the overall scheduler/builder config and removes a
triplication of possible builds paths.
2024-03-11 06:27:32 +01:00
753df8e340 remove cachix
we aren't using it and it's somewhat in the way of our efforts to
improve scheduling and stuff.
2024-03-11 06:26:39 +01:00
0b2545b036 remove unused GitWithRetry 2024-03-11 06:26:39 +01:00