Support remote-building to macOS hosts (#714)

* Support remote-building to macOS hosts

Our README has long featured a snippet to add to the zshenv, with a
caevat that it might behave strangely if you're writing a script with an
empty PATH.

It is pretty straightforward to eliminate those caveats while still
providing remote building for Nix to macOS hosts.

Co-authored-by: Ana Hobden <operator@hoverbear.org>
This commit is contained in:
Graham Christensen 2023-11-10 14:02:19 -05:00 committed by GitHub
parent 6ca2c68e31
commit dac0adca28
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
5 changed files with 127 additions and 48 deletions

View file

@ -10,7 +10,7 @@ A fast, friendly, and reliable tool to help you use Nix with Flakes everywhere.
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
```
The `nix-installer` has successfully completed over 500,000 installs in a number of environments, including [Github Actions](#as-a-github-action):
The `nix-installer` has successfully completed over 1,000,000 installs in a number of environments, including [Github Actions](#as-a-github-action):
| Platform | Multi User | `root` only | Maturity |
|------------------------------|:------------------:|:-----------:|:-----------------:|
@ -265,49 +265,6 @@ This is especially useful when using the installer in non-interactive scripts.
While `nix-installer` tries to provide a comprehensive and unquirky experience, there are unfortunately some issues which may require manual intervention or operator choices.
### Using MacOS remote SSH builders, Nix binaries are not on `$PATH`
When connecting to a Mac remote SSH builder users may sometimes see this error:
```bash
$ nix store ping --store "ssh://$USER@$HOST"
Store URL: ssh://$USER@$HOST
zsh:1: command not found: nix-store
error: cannot connect to '$USER@$HOST'
```
The way MacOS populates the `PATH` environment differs from other environments. ([Some background](https://gist.github.com/Linerre/f11ad4a6a934dcf01ee8415c9457e7b2))
There are two possible workarounds for this:
* **(Preferred)** Update the remote builder URL to include the `remote-program` parameter pointing to `nix-store`. For example:
```bash
nix store ping --store "ssh://$USER@$HOST?remote-program=/nix/var/nix/profiles/default/bin/nix-store"
```
If you are unsure where the `nix-store` binary is located, run `which nix-store` on the remote.
* Update `/etc/zshenv` on the remote so that `zsh` populates the Nix path for every shell, even those that are neither *interactive* or *login*:
```bash
# Nix
if [ -e '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh' ]; then
. '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh'
fi
# End Nix
```
<details>
<summary>This strategy has some behavioral caveats, namely, <code>$PATH</code> may have unexpected contents</summary>
For example, if `$PATH` gets unset then a script invoked, `$PATH` may not be as empty as expected:
```bash
$ cat example.sh
#! /bin/zsh
echo $PATH
$ PATH= ./example.sh
/Users/ephemeraladmin/.nix-profile/bin:/nix/var/nix/profiles/default/bin:
```
This strategy results in Nix's paths being present on `$PATH` twice and may have a minor impact on performance.
</details>
### Using MacOS after removing `nix` while `nix-darwin` was still installed, network requests fail
If `nix` was previously uninstalled without uninstalling `nix-darwin` first, users may experience errors similar to this:
@ -470,6 +427,7 @@ Subtle differences in the shell implementations and tool used in the scripts mak
The Determinate Nix installer has numerous advantages:
* survives macOS upgrades
* keeping an installation receipt for easy uninstallation
* offering users a chance to review an accurate, calculated install plan
* having 'planners' which can create appropriate install plans for complicated targets
@ -478,6 +436,7 @@ The Determinate Nix installer has numerous advantages:
* supporting a expanded test suite including 'curing' cases
* supporting SELinux and OSTree based distributions without asking users to make compromises
* operating as a single, static binary with external dependencies such as `openssl`, only calling existing system tools (like `useradd`) where necessary
* As a MacOS remote build target, ensures `nix` is not absent from path
It has been wonderful to collaborate with other participants in the Nix Installer Working Group and members of the broader community. The working group maintains a [foundation owned fork of the installer](https://github.com/nixos/experimental-nix-installer/).

View file

@ -0,0 +1,96 @@
use crate::action::base::{create_or_insert_into_file, CreateOrInsertIntoFile};
use crate::action::{Action, ActionDescription, ActionError, ActionTag, StatefulAction};
use std::path::Path;
use tracing::{span, Instrument, Span};
const PROFILE_NIX_FILE_SHELL: &str = "/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh";
/**
Configure macOS's zshenv to load the Nix environment when ForceCommand is used.
This enables remote building, which requires `ssh host nix` to work.
*/
#[derive(Debug, serde::Deserialize, serde::Serialize, Clone)]
pub struct ConfigureRemoteBuilding {
create_or_insert_into_file: StatefulAction<CreateOrInsertIntoFile>,
}
impl ConfigureRemoteBuilding {
#[tracing::instrument(level = "debug", skip_all)]
pub async fn plan() -> Result<StatefulAction<Self>, ActionError> {
let shell_buf = format!(
r#"
# Set up Nix only on SSH connections
# See: https://github.com/DeterminateSystems/nix-installer/pull/714
if [ -e '{PROFILE_NIX_FILE_SHELL}' ] && [ -n "${{SSH_CONNECTION}}" ] && [ "${{SHLVL}}" -eq 1 ]; then
. '{PROFILE_NIX_FILE_SHELL}'
fi
# End Nix
"#
);
let create_or_insert_into_file = CreateOrInsertIntoFile::plan(
Path::new("/etc/zshenv"),
None,
None,
0o644,
shell_buf.to_string(),
create_or_insert_into_file::Position::Beginning,
)
.await
.map_err(Self::error)?;
Ok(Self {
create_or_insert_into_file,
}
.into())
}
}
#[async_trait::async_trait]
#[typetag::serde(name = "configure_remote_building")]
impl Action for ConfigureRemoteBuilding {
fn action_tag() -> ActionTag {
ActionTag("configure_remote_building")
}
fn tracing_synopsis(&self) -> String {
"Configuring zsh to support using Nix in non-interactive shells".to_string()
}
fn tracing_span(&self) -> Span {
span!(tracing::Level::DEBUG, "configure_remote_building",)
}
fn execute_description(&self) -> Vec<ActionDescription> {
vec![ActionDescription::new(
self.tracing_synopsis(),
vec!["Update `/etc/zshenv` to import Nix".to_string()],
)]
}
#[tracing::instrument(level = "debug", skip_all)]
async fn execute(&mut self) -> Result<(), ActionError> {
let span = tracing::Span::current().clone();
self.create_or_insert_into_file
.try_execute()
.instrument(span)
.await
.map_err(Self::error)?;
Ok(())
}
fn revert_description(&self) -> Vec<ActionDescription> {
vec![ActionDescription::new(
"Remove the Nix configuration from zsh's non-login shells".to_string(),
vec!["Update `/etc/zshenv` to no longer import Nix".to_string()],
)]
}
#[tracing::instrument(level = "debug", skip_all)]
async fn revert(&mut self) -> Result<(), ActionError> {
self.create_or_insert_into_file.try_revert().await?;
Ok(())
}
}

View file

@ -2,6 +2,7 @@
*/
pub(crate) mod bootstrap_launchctl_service;
pub(crate) mod configure_remote_building;
pub(crate) mod create_apfs_volume;
pub(crate) mod create_fstab_entry;
pub(crate) mod create_nix_hook_service;
@ -16,6 +17,7 @@ pub(crate) mod set_tmutil_exclusions;
pub(crate) mod unmount_apfs_volume;
pub use bootstrap_launchctl_service::BootstrapLaunchctlService;
pub use configure_remote_building::ConfigureRemoteBuilding;
pub use create_apfs_volume::CreateApfsVolume;
pub use create_nix_hook_service::CreateNixHookService;
pub use create_nix_volume::{CreateNixVolume, NIX_VOLUME_MOUNTD_DEST};

View file

@ -38,9 +38,23 @@ impl CommandExecute for Repair {
if let Err(err) = reconfigure.try_execute().await {
println!("{:#?}", err);
Ok(ExitCode::FAILURE)
} else {
return Ok(ExitCode::FAILURE);
}
// TODO: Using `cfg` based on OS is not a long term solution.
// Make this read the planner from the `/nix/receipt.json` to determine which tasks to run.
#[cfg(target_os = "macos")]
{
let mut reconfigure = crate::action::macos::ConfigureRemoteBuilding::plan()
.await
.map_err(PlannerError::Action)?
.boxed();
if let Err(err) = reconfigure.try_execute().await {
println!("{:#?}", err);
return Ok(ExitCode::FAILURE);
}
}
Ok(ExitCode::SUCCESS)
}
}
}

View file

@ -12,7 +12,9 @@ use crate::{
action::{
base::RemoveDirectory,
common::{ConfigureInitService, ConfigureNix, CreateUsersAndGroups, ProvisionNix},
macos::{CreateNixHookService, CreateNixVolume, SetTmutilExclusions},
macos::{
ConfigureRemoteBuilding, CreateNixHookService, CreateNixVolume, SetTmutilExclusions,
},
StatefulAction,
},
execute_command,
@ -166,6 +168,12 @@ impl Planner for Macos {
.map_err(PlannerError::Action)?
.boxed(),
);
plan.push(
ConfigureRemoteBuilding::plan()
.await
.map_err(PlannerError::Action)?
.boxed(),
);
if self.settings.modify_profile {
plan.push(