functional2: attach debugging tools #850

New issue

Open

opened 2025-06-08 20:14:12 +00:00 by raito · 2 comments

raito commented

2025-06-08 20:14:12 +00:00

Owner

Summary

When I'm debugging some tests, I would like to be able to attach common diagnostic tools (e.g. strace, ltrace, perf, gdb, rr, dump the mount namespace view (findmnt -T)) to some of the inner processes.

Motivation

just test --gdb is mostly useless because it's not aware this needs to be attached to some of the inner processes.

When debugging the problem with sandbox = true for why-depends, I looked into it while comparing strace reports on the execve syscall, then inspected the mount namespaces views and realized… there was no sandbox for functional.

Selecting the right target

A functional2 test can probably run multiple processes which are equally target for debugging, in general, I care about attaching the first failing process.

It might be desireable to attach things like the daemon.

Roadmap

Among top priorities, rr, gdb and strace are going to be the most useful IMHO.

The way I imagine this should work is that I should be able to run functional2 with some special options and it will prepend strace in the argv of some of the tests I care about, how to select the right process requires some UI design.

cc @commentatorforall

### Summary When I'm debugging some tests, I would like to be able to attach common diagnostic tools (e.g. strace, ltrace, perf, gdb, rr, dump the mount namespace view (`findmnt -T`)) to some of the inner processes. ### Motivation `just test --gdb` is mostly useless because it's not aware this needs to be attached to some of the inner processes. When debugging the problem with `sandbox = true` for `why-depends`, I looked into it while comparing `strace` reports on the `execve` syscall, then inspected the mount namespaces views and realized… there was no sandbox for functional. ### Selecting the right target A functional2 test can probably run multiple processes which are equally target for debugging, in general, I care about attaching the first failing process. It might be desireable to attach things like the daemon. ### Roadmap Among top priorities, `rr`, `gdb` and `strace` are going to be the most useful IMHO. The way I imagine this should work is that I should be able to run functional2 with some special options and it will prepend `strace` in the argv of some of the tests I care about, how to select the right process requires some UI design. cc @commentatorforall

piegames added this to the functional2 project

2025-06-08 20:14:33 +00:00

raito added the

devx

testing

labels

2025-06-08 20:14:33 +00:00

commentatorforall commented

2025-06-08 20:19:52 +00:00

Member

as you would like to have multiple debugger options, i'd argue that the most simple way would be to put the selection into either an environment variable or into a flag for pytest.

the latter is probably more ideal, as one probably doesn't want to run pytest via meson as that would make it execute in parallel which brings a ton of debugging issues with it (as there are multiple pytest subprocesses already) and would make attach selection likely impossible to pull off

as you would like to have multiple debugger options, i'd argue that the most simple way would be to put the selection into either an environment variable or into a flag for pytest. the latter is probably more ideal, as one probably doesn't want to run pytest via meson as that would make it execute in parallel which brings a ton of debugging issues with it (as there are multiple pytest subprocesses already) and would make attach selection likely impossible to pull off

helle self-assigned this

2025-06-08 20:53:43 +00:00

helle commented

2025-06-08 21:05:17 +00:00

Member

I have some ideas for the UI, but let me double check the UX.

Cases:

When invoked in a specific way, run all tests, and on the first failed test, rerun it with the debug tooling specified attached.
Run a specific test, with the debug tooling specified attached.

Tools are attached to potentially multiple processes and selecting the correct ones should ideally be as implicit as possible.

Tooling:
Both interactive and logging (where?, a file?, stdout and stderr are already getting output from nix and pytest, which probably is undesirable to have fully mixed in with say strace output) tooling should be usable, good handling for at least rr, gdb and strace should be provided.

Notes:
It may be relatively hard to do the "rerun" of case 1 internally, it however may be more trivial to implement to rerun with an option that enables "debug the first test that failed the previous run" (there is a cache that stores this information.

I have some ideas for the UI, but let me double check the UX. Cases: 1. When invoked in a specific way, run all tests, and on the first failed test, rerun it with the debug tooling specified attached. 2. Run a specific test, with the debug tooling specified attached. Tools are attached to potentially multiple processes and selecting the correct ones should ideally be as implicit as possible. Tooling: Both interactive and logging (where?, a file?, `stdout` and `stderr` are already getting output from nix and pytest, which probably is undesirable to have fully mixed in with say strace output) tooling should be usable, good handling for at least `rr`, `gdb` and `strace` should be provided. Notes: It may be relatively hard to do the "rerun" of case 1 internally, it however may be more trivial to implement to rerun with an option that enables "debug the first test that failed the previous run" (there is a cache that stores this information.