Print time taken to copy things to store when verbose logging #653

New issue

Open

opened 2025-01-31 11:18:13 +00:00 by teofilc · 3 comments

teofilc commented

2025-01-31 11:18:13 +00:00

Motivation

I find that excessive copying to the store is often the cause of Lix evaluation slowness. But AFAIK we don't have great tools for debugging this stuff. I often just end up staring at the verbose logs and trying to catch the "copying ... to the store" lines that hang around for the longest. Or I dump the logs and see their sizes.

Solution

It would be nice if, when logging verbosely, Lix emitted information about how long each path took to be copied to the store.

Describe alternatives you've considered

We could have a more general opentelemetry-esque tracing output from evaluation, or other forms of tracing/profiling, but that seems like a lot more work.

Additional context

This is my first issue on this tracker :)

## Motivation I find that excessive copying to the store is often the cause of Lix evaluation slowness. But AFAIK we don't have great tools for debugging this stuff. I often just end up staring at the verbose logs and trying to catch the "copying ... to the store" lines that hang around for the longest. Or I dump the logs and see their sizes. ## Solution It would be nice if, when logging verbosely, Lix emitted information about how long each path took to be copied to the store. ## Describe alternatives you've considered We could have a more general opentelemetry-esque tracing output from evaluation, or other forms of tracing/profiling, but that seems like a lot more work. ## Additional context This is my first issue on this tracker :)

jade added the

labels

2025-01-31 16:32:42 +00:00

jade commented

2025-01-31 16:37:07 +00:00

Owner

Well, on the plus side, we want to implement rust tracing in lix. It's just hard. It would be maybe plausible, perhaps easy even, to fix up https://github.com/lf-/nix-otel but that's a bit of a distraction and my goal is to get tracing through to the finish line instead. It's approximately my next priority.

Well, on the plus side, we *want* to implement rust tracing in lix. It's just hard. It would be maybe plausible, perhaps easy even, to fix up https://github.com/lf-/nix-otel but that's a bit of a distraction and my goal is to get tracing through to the finish line instead. It's approximately my next priority.

teofilc commented

2025-01-31 17:19:38 +00:00

Author

That's really exciting!

lix-bot commented

2025-04-01 12:02:56 +00:00

Member

This issue was mentioned on Gerrit on the following CLs:

comment in cl/2925 ("builtins.path: log slow paths to copy")

This issue was mentioned on Gerrit on the following CLs: * comment in [cl/2925](https://gerrit.lix.systems/c/lix/+/2925) ("builtins.path: log slow paths to copy")