puck/lix
0
0
Fork 0
forked from lix-project/lix
Code Pull requests Activity

ssh-ng: also store build logs to make them accessible by nix log

Browse source 
Right now when building a derivation remotely via

    $ nix build -j0 -f . hello -L --builders 'ssh://builder'

it's possible later to read through the entire build-log by running
`nix log -f . hello`. This isn't possible however when using `ssh-ng`
rather than `ssh`.

The reason for that is that there are two different ways to transfer
logs in Nix through e.g. an SSH tunnel (that are used by `ssh`/`ssh-ng`
respectively):

* `ssh://` receives its logs from the fd pointing to `builderOut`. This
  is directly passed to the "log-sink" (and to the logger on each `\n`),
  hence `nix log` works here.
* `ssh-ng://` however expects JSON-like messages (i.e. `@nix {log data
  in here}`) and passes it directly to the logger without doing anything
  with the `logSink`. However it's certainly possible to extract
  log-lines from this format as these have their own message-type in the
  JSON payload (i.e. `resBuildLogLine`).

  This is basically what I changed in this patch: if the code-path for
  `builderOut` is not reached and a `logSink` is initialized, the
  message was successfully processed by the JSON logger (i.e. it's in
  the expected format) and the line is of the expected type (i.e.
  `resBuildLogLine`), the line will be written to the log-sink as well.

Closes #5079
This commit is contained in:
Maximilian Bosch 2022-01-31 22:05:28 +01:00
parent b592359c56
commit 7a04839ea5
No known key found for this signature in database
GPG key ID: 091DBF4D1FC46B8E
1 changed files with 10 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

13
src/libstore/build/derivation-goal.cc
View file

@ -1175,10 +1175,10 @@ bool DerivationGoal::isReadDesc(int fd)
return fd == hook->builderOut.readSide.get();
}
void DerivationGoal::handleChildOutput(int fd, std::string_view data)
{
if (isReadDesc(fd))
auto isWrittenToLog = isReadDesc(fd);
if (isWrittenToLog)
{
logSize += data.size();
if (settings.maxLogSize && logSize > settings.maxLogSize) {
@ -1207,7 +1207,14 @@ void DerivationGoal::handleChildOutput(int fd, std::string_view data)
if (hook && fd == hook->fromHook.readSide.get()) {
for (auto c : data)
if (c == '\n') {
handleJSONLogMessage(currentHookLine, worker.act, hook->activities, true);
auto s = handleJSONLogMessage(currentHookLine, worker.act, hook->activities, true);
if (s && !isWrittenToLog && logSink) {
auto json = nlohmann::json::parse(std::string(currentHookLine, 5));
if (json["type"] == resBuildLogLine) {
auto f = json["fields"];
(*logSink)((f.size() > 0 ? f.at(0).get<std::string>() : "") + "\n");
}
}
currentHookLine.clear();
} else
currentHookLine += c;