Teach the installer how to substitute from a real binary cache #13

Open
opened 2024-05-31 22:46:07 +00:00 by jade · 4 comments
Owner

All these tarballs of Nix stores are very silly. We should just teach the installer to substitute things from a cache normally.

All these tarballs of Nix stores are very silly. We should just teach the installer to substitute things from a cache normally.
Member

Seems like installer knows how to substitute now.

// Install `nix` itself into the store
execute_command(
Command::new(nix_pkg.join("bin/nix-env"))
.process_group(0)
.arg("-i")
.arg(&nix_pkg)
.stdin(std::process::Stdio::null())
.env(
"HOME",
dirs::home_dir()
.ok_or_else(|| Self::error(SetupDefaultProfileError::NoRootHome))?,
)
.env(
"NIX_SSL_CERT_FILE",
nss_ca_cert_pkg.join("etc/ssl/certs/ca-bundle.crt"),
), /* This is apparently load bearing... */
)
.await
.map_err(Self::error)?;
// Install `nix` itself into the store
execute_command(
Command::new(nix_pkg.join("bin/nix-env"))
.process_group(0)
.arg("-i")
.arg(&nss_ca_cert_pkg)
.stdin(std::process::Stdio::null())
.env(
"HOME",
dirs::home_dir()
.ok_or_else(|| Self::error(SetupDefaultProfileError::NoRootHome))?,
)
.env(
"NIX_SSL_CERT_FILE",
nss_ca_cert_pkg.join("etc/ssl/certs/ca-bundle.crt"),
), /* This is apparently load bearing... */
)
.await
.map_err(Self::error)?;

Seems like installer knows how to substitute now. https://git.lix.systems/lix-project/lix-installer/src/commit/be67a8a4e2bfc449e139cbf4e23096434140d02e/src/action/base/setup_default_profile.rs#L166-L204
Member

This issue was mentioned on Gerrit on the following CLs:

  • commit message in cl/1642 ("Support downloading nix from substituters")
<!-- GERRIT_LINKBOT: {"cls": [{"backlink": "https://gerrit.lix.systems/c/lix-installer/+/1642", "number": 1642, "kind": "commit message"}], "cl_meta": {"1642": {"change_title": "Support downloading nix from substituters"}}} --> This issue was mentioned on Gerrit on the following CLs: * commit message in [cl/1642](https://gerrit.lix.systems/c/lix-installer/+/1642) ("Support downloading nix from substituters")
Author
Owner

you are mistaken. that is not substitution. the comment sucks, but it is not substitution. the store path already exists in the target store since it was unpacked into there (afaiu) and the store was told it was there with the load-db thing. all that command does is puts it into the profile.

proper substitution gets us signature verification and overall makes the installer workflow (and builds of nixes in general) less special: you can just give it whatever store path for a nix that exists in a binary cache, no matter if a tarball was produced, and it will just work: it allows us to potentially stop producing the bootstrap tarball altogether.

this is part of an overall view that the installer should eventually not need to exist as a separate special entity: the installer would at that point just be an option to a statically built nix.

you are mistaken. that is not substitution. the comment sucks, but it is not substitution. the store path already exists in the target store since it was unpacked into there (afaiu) and the store was told it was there with the load-db thing. all that command does is puts it into the profile. proper substitution gets us signature verification and overall makes the installer workflow (and builds of nixes in general) less special: you can just give it whatever store path for a nix that exists in a binary cache, no matter if a tarball was produced, and it will just work: it allows us to potentially stop producing the bootstrap tarball altogether. this is part of an overall view that the installer should eventually not need to exist as a separate special entity: the installer would at that point just be an option to a statically built nix.
Member

this is part of an overall view that the installer should eventually not need to exist as a separate special entity: the installer would at that point just be an option to a statically built nix.

Hmm, I'm not sure about that. Installer can do a lot of heavy lifting for detecting environment and handling upgrades. Well, it's not a pressing issue, we can decide if installer even needs to exist later.

For substitution - I think we need to have RUst bindings for Lix or a substitution spec first. I'm inclined to think that implementing substitution (or a subset of it) from scratch in installer code is a bad idea.

I do agree that the way installer currently works is pretty stupid, though. But until the substitution layer is reusable (either as a spec or as bindings) - isn't tarball "good enough"? The substitution part is basically replaced with "trust me bro and trust my CI bro". Which is not great, but it's also better than nothing, or than reimplementing substitution blindly.

Let me know what you think about it!

> this is part of an overall view that the installer should eventually not need to exist as a separate special entity: the installer would at that point just be an option to a statically built nix. Hmm, I'm not sure about that. Installer can do a lot of heavy lifting for detecting environment and handling upgrades. Well, it's not a pressing issue, we can decide if installer even needs to exist later. For substitution - I think we need to have RUst bindings for Lix or a substitution spec first. I'm inclined to think that implementing substitution (or a subset of it) from scratch in installer code is a bad idea. I do agree that the way installer currently works is pretty stupid, though. But until the substitution layer is reusable (either as a spec or as bindings) - isn't tarball "good enough"? The substitution part is basically replaced with "trust me bro and trust my CI bro". Which is not great, but it's also better than nothing, or than reimplementing substitution blindly. Let me know what you think about it!
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: lix-project/lix-installer#13
No description provided.