Commit graph

144 commits

Author SHA1 Message Date
51f7b52149 pre-filter drv_info into all_deps
otherwise failure reporting is *enormous* with the entirety of a full
derivation info dump in there
2024-03-11 13:07:35 +01:00
13a67b483a fix interrupt()
can't interrupt with things to interrupt. this is technically duplicated
information but keeping parts of the code close to Trigger seems useful.
2024-03-11 13:05:12 +01:00
9933971ab0 re-enable the gerrit status reporter 2024-03-11 09:06:29 +01:00
29a2ef63e2 show hydra job count in trigger step
previously we immediately triggered all jobs, now we no longer do.
showing the total count at least somewhere is nice to have a rough
indication of how much longer a build may still need to run.
2024-03-11 09:05:28 +01:00
9a15348984 Fix up a few loose ends 2024-03-11 08:08:55 +01:00
4d73275123 Add build result tracking, schedule newly available builds 2024-03-11 08:08:53 +01:00
28ca39af25 WIP: Replace Trigger with custom logic 2024-03-11 08:06:37 +01:00
e9874c3d98 wip: dependency-tracked build triggering 2024-03-11 07:53:56 +01:00
f869b52a8d use build-local gc-root directory
without this two builds can interfere with each other if:

  - builds 1 and 2 start
  - build 1 is starved of workers
  - build 2 finishes, removes the shared gcroots directory
  - gc runs
  - build 1 schedules more builds whose .drvs have now been removed

using a dedicated directory for each build fixes this.

we now also need to set alwaysRun on the cleanup command or we risk
littering the system with stale gc roots when a build fails.
2024-03-11 06:48:41 +01:00
156e6e3dea remove skipped-builds builder
run all of them on the normal build worker. this significantly
simplifies the overall scheduler/builder config and removes a
triplication of possible builds paths.
2024-03-11 06:27:32 +01:00
753df8e340 remove cachix
we aren't using it and it's somewhat in the way of our efforts to
improve scheduling and stuff.
2024-03-11 06:26:39 +01:00
0b2545b036 remove unused GitWithRetry 2024-03-11 06:26:39 +01:00
fdfeef8ad4 remove retry logic
retries don't help us very much, in fact they mostly hurt by repeating
builds that failed for non-transient reasons. retries could help with
workers dropping while running a build, but those rare cases are better
to restart manually than to pend at least twice the ci time for commits
that simply do not build cleanly.
2024-03-11 06:26:38 +01:00
ec2ef903ab use .#hydraJobs rather than .#checks 2024-03-08 23:28:49 +00:00
c487ada514 review callback: Set labels, render list of failed checks in reporter 2024-03-07 04:40:58 +00:00
18d537e5d4 nix/coordinator(reporting): re-enable reviewCB, disable summaryCB, skip evaluations
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-04 20:50:12 +01:00
322944f8d1 nix/coordinator(reporting): disable startCB and reviewCB
excessive noise.

Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-04 20:37:13 +01:00
9a67a1f2cc nix/coordinator: enable reporting to Gerrit
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-03 02:58:25 +01:00
2bcc0ee4e9 nix/coordinator: hardcode remaining github properties to lix
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-02 23:22:36 +01:00
3895b90493 nix/coordinator: disable reporter
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-02 23:07:36 +01:00
670a1d1281 nix/coordinator: fix repourl
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-02 23:07:36 +01:00
94e3d7aeda nix/coordinator: support specific private SSH keys
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-02 23:07:33 +01:00
5a8ab145e6 buildbot: add repourl
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-02 22:09:56 +01:00
7ace78107c buildbot: further Gerritification
- `BuildTrigger` still depends on `github` properties.

Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-02 22:09:56 +01:00
5f7b3e0cdb buildbot: instantiate fully LixSystemsOAuth2
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-02 22:09:56 +01:00
acfd225e6d buildbot/__init__.py: rework the file for gerrit usecase, our oauth2 instance
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-02 18:45:19 +01:00
3c1e2c0e76 cleanup drv gcroots after a build 2024-01-15 13:17:19 +00:00
c9e850ff32 only refresh github projects in timer and sort them
This way errors are more visible to users on the first running.

Co-authored-by: zimbatm <zimbatm@zimbatm.com>
2024-01-08 11:45:01 +00:00
f69a37a69e relax concurrent builds
So far we cannot have builds running concurrently, whereas we only would
need to be strict about nix evaluation to stay below memory limits.
2024-01-05 08:49:54 +01:00
b2467c4ac5 typecheck twisted 2024-01-01 15:38:28 +00:00
6882c81e90 fixup logger warning call 2024-01-01 08:05:12 +01:00
acdab58765 worker: fix basedir 2023-12-27 10:25:28 +00:00
645c7fd0cc fix workername formatting 2023-12-27 10:25:28 +00:00
8b9c060dbe fix new linting errors 2023-12-26 22:36:40 +01:00
35079f89e7 apply ruff 2023-12-26 19:50:34 +01:00
f56e43267d expose cachix options explictly 2023-12-24 07:10:01 +00:00
232a617fe1 let nix handle the silent timeout of builds 2023-12-22 14:40:31 +01:00
a13cd78a1c drop prometheus exporter again
User can add this themself
2023-12-08 11:11:48 +00:00
48ea2f7db4 GithubAuth: increase apiVersion to 4 2023-12-08 11:11:48 +00:00
cbe4ce0571 move extraConfig into configurator
extraConfig can be only set once. By moving it to the configurator we allow users to set this option themself.
2023-12-08 11:11:48 +00:00
dd6eacc4c4 allow all members in an org to restart/cancel/trigger builds 2023-12-08 11:11:48 +00:00
ed8a6c0333 git: fix retry message 2023-12-04 10:20:27 +01:00
590f31eb6f retry git clonning errors 2023-12-03 15:10:16 +00:00
b33d7e5a3e drop nix_update_flake_config pipeline
This is better offloaded to tools like dependabot/renovatebot.
I will probably work on integrate the latter one.
2023-12-03 15:10:16 +00:00
49f97876d5 fix status_name and virtual_builder_name 2023-11-20 09:07:45 +01:00
9d80f5f2c7 move skipped builds to local worker 2023-11-18 10:13:16 +01:00
392329ea71 fix conditionals when skipping cached builds 2023-11-18 07:02:34 +01:00
37372c90da skip cached builds 2023-11-17 18:04:27 +01:00
5b4ddd014c make evalWorkerCount configurable 2023-11-12 07:23:35 +01:00
c477a14d1a fix github avatar authentication 2023-11-12 07:23:23 +01:00
zowoq
8dbd6a2101 prefix scheduler names with project.id 2023-11-11 15:10:13 +00:00
1719bf1810 make eval lock exclusive
looks like maxCount doesn't quite work
2023-11-11 15:10:13 +00:00
Jörg Thalheim
4d71870239
Merge pull request #27 from Mic92/fixes
Also pass GITHUB_TOKEN to AvatarGithub
2023-11-10 17:24:47 +01:00
524c8fd150 buildbot-nix: limit evaluation to one job per worker 2023-11-10 14:30:09 +01:00
zowoq
2a565a8d5a pad worker numbering 2023-11-10 13:49:24 +01:00
abdff1d927 also pass GITHUB_TOKEN to AvatarGithub 2023-11-09 08:52:46 +01:00
f7573d02bc make sure update-flake has a github project field set 2023-11-09 08:52:46 +01:00
30b8d7b090 Configurator: make sure all config values have defaults set before hand 2023-11-09 08:52:46 +01:00
5b3f986a29 buildbot_nix: update comment
This doesn't invoke nix-instantiate anymore, but
`nix-eval-jobs --flake .#checks`.
2023-11-08 15:31:14 +02:00
0e52cdf81f buildbot: set treeStableTimer
Otherwise we build every commit
2023-11-04 12:14:53 +01:00
ecaafe61b4 make nginx output paths optional 2023-11-04 11:21:36 +01:00
zowoq
3551cb6bef remove extra slash from webhook url 2023-11-04 13:47:55 +10:00
25fe8ed9ae fix jobs filter 2023-10-27 11:45:54 +02:00
Jörg Thalheim
9f2ec133e1 use rstrip to remove trailing newlines 2023-10-27 09:11:40 +00:00
9ad17c646b fmt 2023-10-27 09:11:40 +00:00
c7b360eb16 pass project name for forced builds 2023-10-27 09:11:39 +00:00
2edf6860b0 allow github projects with "." in their name 2023-10-27 09:11:39 +00:00
eac9acc041 only activate github auth if no other is active 2023-10-27 09:11:39 +00:00
44486f6633 skip github projects without sufficient perms 2023-10-27 09:11:39 +00:00
a758cb41ca fix: also set repos if not cached 2023-10-27 09:11:38 +00:00
f14619f0df strip read secrets 2023-10-27 09:11:38 +00:00
4ae3ef7ca7 make buildbot_nix a python module 2023-10-27 09:11:38 +00:00
b042eb8011 apply nix fmt 2023-10-16 18:41:52 +01:00
14556fd91f disable periodic flake updates for now 2023-10-15 11:31:07 +02:00
a553fdfa34 use nix-eval-jobs from nixpkgs 2023-10-15 11:31:07 +02:00
2409fc9c91 reload github projects every 12 hours 2023-10-15 11:15:54 +02:00
776a6ea499 update flakes with a random delay 2023-10-15 08:36:33 +02:00
b5681b1ea0 remove incorrect skip if path exists
the path might exists also the build failed
2023-10-13 00:38:57 +02:00
e82d3a3d91 add option to reload github projects
fix mypy
2023-10-12 23:58:59 +02:00
74fb30f6ff also install webhooks secrets 2023-10-12 21:05:37 +02:00
54bcb08fae also create webhooks automatically 2023-10-12 17:50:43 +02:00
cb35c312c1 make topic filter configurable 2023-10-12 15:54:28 +02:00
dd95055cb6 make per-project gcroots 2023-10-12 15:54:28 +02:00
e90d7e151e fix doStepIf 2023-10-12 12:14:44 +02:00
08feafa5d4 drop automerge feature 2023-10-09 16:09:19 +02:00
94c75f9631 switch to using configurators option from nixpkgs fork again 2023-10-09 15:51:05 +02:00
2488ef4424 switch to configurators class 2023-09-18 17:01:07 +02:00
5915bf4ffa ruff: also lint upgrade rules 2023-09-18 15:43:51 +02:00
cdde629ff3 drop irc notifications 2023-09-17 06:46:19 +02:00
2afa25ca27 master: move DB_URL to global variable 2023-09-15 09:10:35 +02:00
aeffb167fe worker: move global variables to config class 2023-09-15 09:09:41 +02:00
97f1870c15 add first working configuration 2023-09-10 11:16:33 +00:00
dc190c7635 improve master module 2023-09-10 08:53:04 +00:00
6406f34ebe first commit; very broken 2023-09-10 08:11:56 +00:00