lix-project/lix

Fork 31

Allow client to pass environment variables to FOD builder #590

New issue

Open

opened 2024-12-03 08:33:23 +00:00 by szlend · 3 comments

szlend commented

2024-12-03 08:33:23 +00:00

Currently there's no decent way of passing secrets to builders that need to be able to download files from sources that require authentication.

Current methods:

builtins.fetchTree/builtins.fetchGit - client-side fetching, blocks evaluation, restricted to git/github/gitlab etc..
pkgs.fetch* - FOD builder fetching, requires credentials on the filesystem (netrc)

The ideal solution would be #558, however that's still pretty far away.

Describe the solution you'd like

Nix 2.19 introduced a new experimental feature that I actually quite like, called configurable-impure-env (PR). This allows you to pass impure environmental variables to the daemon. I think it's a pretty decent simple solution until we can do #558.

Example usage:

NIX_CONIFG="impure-env = TOKEN=$(fetch-token-from-key-store)" nix build .#foo

Open questions:

Should this be an experimental feature?
Should impure-env require a user to be trusted?
- If the user does not need to be trusted, we probably want some sort of env var filtering so users can't just override stuff like HTTP_PROXY.
Better CLI UX? Something like --impure-env FOO=bar (set explicitly) / --impure-env FOO (inherit from env)
- Could be a future improvement

Describe alternatives you've considered

We could just not do anything until #558 and keep passing credentials through netrc.

Additional context

Happy to port this from Nix 2.19 myself if maintainers agree and the way we wanna implement this is clear.

## Is your feature request related to a problem? Please describe. Currently there's no decent way of passing secrets to builders that need to be able to download files from sources that require authentication. Current methods: - `builtins.fetchTree`/`builtins.fetchGit` - client-side fetching, blocks evaluation, restricted to git/github/gitlab etc.. - `pkgs.fetch*` - FOD builder fetching, requires credentials on the filesystem (`netrc`) The ideal solution would be #558, however that's still pretty far away. ## Describe the solution you'd like Nix 2.19 introduced a new experimental feature that I actually quite like, called `configurable-impure-env` ([PR](https://github.com/NixOS/nix/pull/8830)). This allows you to pass impure environmental variables to the daemon. I think it's a pretty decent simple solution until we can do #558. Example usage: ``` NIX_CONIFG="impure-env = TOKEN=$(fetch-token-from-key-store)" nix build .#foo ``` Open questions: - Should this be an experimental feature? - Should `impure-env` require a user to be trusted? - If the user does not need to be trusted, we probably want some sort of env var filtering so users can't just override stuff like `HTTP_PROXY`. - Better CLI UX? Something like `--impure-env FOO=bar` (set explicitly) / `--impure-env FOO` (inherit from env) - Could be a future improvement ## Describe alternatives you've considered We could just not do anything until #558 and keep passing credentials through `netrc`. ## Additional context Happy to port this from Nix 2.19 myself if maintainers agree and the way we wanna implement this is clear.

raito commented

2024-12-03 15:16:37 +00:00

Owner

Hello there,

I discussed this with few team members in private who are working in adjacent areas. While I agree / recognize that netrc is insufficient, we need more documentation / examples of where netrc fell short (scenarios, usecases, etc.) so that we can keep our priority on this topic updated and clear. Most of the workloads I work with private environments are, today, already possible with the current netrc capabilities, albeit with Nix code and sandbox impure paths.

Now, onto the feature, one problem with this sort of thing is what are the various policies we should apply, explain, advise to users with multi-tenant contexts (single tenant context is a special case of multi-tenancy here.)

For example, consider that you have a community builder as a remote builder, and you pass a GitHub authentication token as an impure environment variable, any operators of that community builder who are logging or watching for environments over there will be able to capture it, constituting a dangerous scenario.

More than an experimental feature, I think this is a dangerous-feature, because it is a super well-designed assault rifle pointed to the feet of our users. Now, with netrc, the potential for leakage also exist but is better understood with the experience we have with netrc, arguably also netrc forces people to design custom solutions based on sandbox impure paths which needs to be provisioned uniformly on any remote builder or environments you care about.

So we are inclined towards implementing:

client-side building first and foremost
providing policy hooks to decide whether to carry pluggable authentication data on remote builders in non-client contexts (if the scheduling decides so)
designing a pluggable authentication protocol

Some of the architectural work that is being conducted right now should provide the building blocks to achieve this, without paying the cost that CppNix is self-inflicting themselves with, all that technical debt.

Nonetheless, as you remarked, this will take a bit of time.

The position I would aim for is that I would prefer to see more energy spent on implementing the correct path rather than deal with impure-env that might be a distraction which solves a tiny surface of the problem statement.

That being said, I do believe that helping the scenario where you use impure sandbox paths which are shaped as a socket that you can ask for secrets (whether you put a Vault proxy behind or just a simple socat that sends back your secret) can be made more secure by helping the sandbox prove to the socket who they are. I'd like to see more towards identity-aware build contexts which can help for more than just obtaining temporary secrets to do things, but also signing, etc.

Note that this is my opinion as a core team member, @jade who partaked in the thread may have a different one.

Hello there, I discussed this with few team members in private who are working in adjacent areas. While I agree / recognize that netrc is insufficient, we need more documentation / examples of where netrc fell short (scenarios, usecases, etc.) so that we can keep our priority on this topic updated and clear. Most of the workloads I work with private environments are, today, already possible with the current netrc capabilities, albeit with Nix code and sandbox impure paths. Now, onto the feature, one problem with this sort of thing is what are the various policies we should apply, explain, advise to users with multi-tenant contexts (single tenant context is a special case of multi-tenancy here.) For example, consider that you have a community builder as a remote builder, and you pass a GitHub authentication token as an impure environment variable, any operators of that community builder who are logging or watching for environments over there will be able to capture it, constituting a dangerous scenario. More than an experimental feature, I think this is a `dangerous-feature`, because it is a super well-designed assault rifle pointed to the feet of our users. Now, with netrc, the potential for leakage also exist but is better understood with the experience we have with netrc, arguably also netrc forces people to design custom solutions based on sandbox impure paths which needs to be provisioned uniformly on any remote builder or environments you care about. So we are inclined towards implementing: - client-side building first and foremost - providing policy hooks to decide whether to carry pluggable authentication data on remote builders in non-client contexts (if the scheduling decides so) - designing a pluggable authentication protocol Some of the architectural work that is being conducted right now should provide the building blocks to achieve this, without paying the cost that CppNix is self-inflicting themselves with, all that technical debt. Nonetheless, as you remarked, this will take a bit of time. The position I would aim for is that I would prefer to see more energy spent on implementing the correct path rather than deal with impure-env that might be a distraction which solves a tiny surface of the problem statement. That being said, I do believe that helping the scenario where you use impure sandbox paths which are shaped as a socket that you can ask for secrets (whether you put a Vault proxy behind or just a simple `socat` that sends back your secret) can be made more secure by helping the sandbox prove to the socket who they are. I'd like to see more towards identity-aware build contexts which can help for more than just obtaining temporary secrets to do things, but also signing, etc. Note that this is my opinion as a core team member, @jade who partaked in the thread may have a different one.

szlend commented

2024-12-03 16:12:53 +00:00

Author

That being said, I do believe that helping the scenario where you use impure sandbox paths which are shaped as a socket that you can ask for secrets (whether you put a Vault proxy behind or just a simple socat that sends back your secret) can be made more secure by helping the sandbox prove to the socket who they are. I'd like to see more towards identity-aware build contexts which can help for more than just obtaining temporary secrets to do things, but also signing, etc.

Exactly what I was looking into before discovering configurable-impure-env.

However I couldn't figure out how to prove the build process that's requesting credentials is who it says it is. Without that I felt like it was just smoke and mirrors really, so I just went with the simpler solution.

> That being said, I do believe that helping the scenario where you use impure sandbox paths which are shaped as a socket that you can ask for secrets (whether you put a Vault proxy behind or just a simple socat that sends back your secret) can be made more secure by helping the sandbox prove to the socket who they are. I'd like to see more towards identity-aware build contexts which can help for more than just obtaining temporary secrets to do things, but also signing, etc. Exactly what I was looking into before discovering `configurable-impure-env`. However I couldn't figure out how to prove the build process that's requesting credentials is who it says it is. Without that I felt like it was just smoke and mirrors really, so I just went with the simpler solution.

raito commented

2024-12-03 22:50:30 +00:00

Owner

However I couldn't figure out how to prove the build process that's requesting credentials is who it says it is. Without that I felt like it was just smoke and mirrors really, so I just went with the simpler solution.

This is something we could provide in our sandboxes, but we need probably a bit of help on how to design this "right" to ensure that we don't forget anyone for their workloads.

> However I couldn't figure out how to prove the build process that's requesting credentials is who it says it is. Without that I felt like it was just smoke and mirrors really, so I just went with the simpler solution. This is something we could provide in our sandboxes, but we need probably a bit of help on how to design this "right" to ensure that we don't forget anyone for their workloads.

No milestone

No project

No assignees

2 participants

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: lix-project/lix#590

No description provided.