Rework `/etc/nix/machines` definitions #854

New issue

Open

opened 2025-06-09 16:09:50 +00:00 by raito · 2 comments

raito commented

2025-06-09 16:09:50 +00:00

Owner

The current /etc/nix/machines format is brittle and hard to work with:

It relies on a fragile, space-separated line-based format with positional arguments.
It only supports a very specific (and undocumented) format for the SSH public key.
There's no way to define groups of machines to easily enable or disable sets of builders.
There is no nix doctor-like command to validate or diagnose misconfigurations short for nix store ping --store ssh://... which doesn't tell too much about the negotiated feature sets.
It lacks explicit naming of features or capabilities per machine; there's no clear distinction between required and optional fields.

This makes it error-prone even for the most experienced Nix users.

Describe the solution you'd like

Introduce a structured format for remote builder configuration, such as JSON, TOML, or even a Nix expression. For instance:

Support /etc/nix/machines.json or /etc/nix/machines.d/*.toml
Each builder entry should be a map with named fields (host, sshUser, sshKey, system, maxJobs, etc.).
Support grouping of builders (e.g., group = "slow-arm-builders";), with ways to enable/disable them via CLI or configuration.
Provide a nix doctor remote-builders [--group XYZ] [--machine NAME] [--simulate-build-drv XYZ] command to validate SSH access, parse errors, missing keys, and system mismatches.

Describe alternatives you've considered

Keeping the current format and layering a linter/doctor on top.
Writing a wrapper that generates the current format from a higher-level definition in Nix or JSON.

## Is your feature request related to a problem? Please describe. The current `/etc/nix/machines` format is brittle and hard to work with: * It relies on a fragile, space-separated line-based format with positional arguments. * It only supports a very specific (and undocumented) format for the SSH public key. * There's no way to define groups of machines to easily enable or disable sets of builders. * There is no `nix doctor`-like command to validate or diagnose misconfigurations short for `nix store ping --store ssh://...` which doesn't tell too much about the negotiated feature sets. * It lacks explicit naming of features or capabilities per machine; there's no clear distinction between required and optional fields. This makes it error-prone even for the most experienced Nix users. ## Describe the solution you'd like Introduce a structured format for remote builder configuration, such as JSON, TOML, or even a Nix expression. For instance: * Support `/etc/nix/machines.json` or `/etc/nix/machines.d/*.toml` * Each builder entry should be a map with named fields (`host`, `sshUser`, `sshKey`, `system`, `maxJobs`, etc.). * Support grouping of builders (e.g., `group = "slow-arm-builders";`), with ways to enable/disable them via CLI or configuration. * Provide a `nix doctor remote-builders [--group XYZ] [--machine NAME] [--simulate-build-drv XYZ]` command to validate SSH access, parse errors, missing keys, and system mismatches. ## Describe alternatives you've considered * Keeping the current format and layering a linter/doctor on top. * Writing a wrapper that generates the current format from a higher-level definition in Nix or JSON.

👍 3

raito added the

E/easy

Area/remote-builds

Topic/Large Scale Installations

labels

2025-06-09 16:10:26 +00:00

jade commented

2025-06-10 22:31:39 +00:00

Owner

I think this is probably subsumed by "rewrite the config system to toml" which I think we have a bug somewhere for...

raito commented

2025-06-11 18:15:19 +00:00

Author

Owner

I think this is probably orthogonal to rewriting the whole config system to TOML, we can already just start by taking care of the machines format independently.

Nonetheless, achieving the full features mentioned here is blocked on proper RPC because changing builders is a trusted setting, it can only be cleared and this leads to a bunch of stupid things.

I think this is probably orthogonal to rewriting the whole config system to TOML, we can already just start by taking care of the machines format independently. Nonetheless, achieving the full features mentioned here is blocked on proper RPC because changing builders is a trusted setting, it can only be cleared and this leads to a bunch of stupid things.