lix-project/hydra
4
0
Fork 6
Code Issues 2 Pull requests Packages Projects Releases Wiki Activity

WIP: hydra-queue-runner: do not assume reservation is available when build step throws unexpectedly #1

Closed
raito wants to merge 1 commit from raito/hydra:rsv-nullpo into main
pull from: raito/hydra:rsv-nullpo
merge into: lix-project:main
Conversation 6 Commits 1 Files changed 1 +8 -2
raito commented 2024-07-08 15:05:02 +00:00
Owner
Copy link

reservation might be null when an uncaught exception has been thrown.
We copy the data we need to report the error.

Not sure if this is the right way to go about this fix.

`reservation` might be null when an uncaught exception has been thrown. We copy the data we need to report the error. Not sure if this is the right way to go about this fix.
raito added 1 commit 2024-07-08 15:05:07 +00:00 
`reservation` might be null when an uncaught exception has been thrown.
We copy the data we need to report the error.

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
raito commented 2024-07-08 15:28:49 +00:00
Author
Owner
Copy link

I don't think it passes all the tests, one is failing (flakiness, idk?).

I don't think it passes all the tests, one is failing (flakiness, idk?).
raito commented 2024-07-08 15:33:58 +00:00
Author
Owner
Copy link

And it's the LDAP test…

And it's the LDAP test…
delroth commented 2024-07-08 19:18:42 +00:00
Member
Copy link

Yeah that looks correct - a while back I made it so machines drop their reservations early to not hog build slots while copying stuff around, and I'm guessing we've been encountering exceptions that happen on the "copying outputs" step?

Yeah that looks correct - a while back I made it so machines drop their reservations early to not hog build slots while copying stuff around, and I'm guessing we've been encountering exceptions that happen on the "copying outputs" step?
delroth commented 2024-07-08 19:21:00 +00:00
Member
Copy link

(See 5c3e508e55)

(See 5c3e508e55d99b6d9e50aee48b8bf514e15ad006)
delroth reviewed 2024-07-08 19:21:37 +00:00
src/hydra-queue-runner/builder.cc Outdated
@ -40,2 +41,4 @@
res = doBuildStep(destStore, reservation, activeStep);
} catch (std::exception & e) {
if (!reservation) {
printMsg(lvlError, "machine '%s' has been released unexpectedly", machineSshName);
delroth commented 2024-07-08 19:21:37 +00:00
Member
Copy link

Not unexpectedly. See my other comments.

Not unexpectedly. See my other comments.
raito commented 2024-07-09 00:47:02 +00:00
Author
Owner
Copy link

Yep, makes sense. Will change it, to a lvlTalkative maybe?

Yep, makes sense. Will change it, to a `lvlTalkative` maybe?
raito marked this conversation as resolved
raito force-pushed rsv-nullpo from a8a5dea411 to 1f9d8dc0a6 2024-07-09 01:12:42 +00:00 Compare
lukegb approved these changes 2024-07-09 01:16:08 +00:00
raito commented 2024-07-09 01:16:46 +00:00
Author
Owner
Copy link

The LDAP test still fails deterministcally, I have no idea why.

The LDAP test still fails deterministcally, I have no idea why.
delroth commented 2024-07-13 04:12:52 +00:00
Member
Copy link

Fixed in fb9e29d4.

Fixed in fb9e29d4.
delroth closed this pull request 2024-07-13 04:12:52 +00:00
Some checks failed
Test / tests (pull_request) Has been cancelled
Details

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No reviewers
lukegb
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: lix-project/hydra#1
No description provided.
Delete branch "raito/hydra:rsv-nullpo"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?