S3 binary cache no respecting PROXY env variables #433

New issue

Closed

opened 2024-07-05 17:32:14 +00:00 by itzwam · 2 comments

itzwam commented

2024-07-05 17:32:14 +00:00

Describe the bug

In the original nix we have the same bug

when running nix build airgapped with a PROXY used to call internet all aws-sdk-cpp requests are done PROXY-less, so the host tries to join internet directly, which fails

Steps To Reproduce

block internet traffic except to the proxy host
allow anything on the proxy
try to build / fetch some cached biary pkgs

Expected behavior

uses cache corectly

`nix --version` output

tried with Nix officiel and Lix beta / rc version

Additional context

not sure if this is a too much breaking change or if it can be implemented easily, so I made this issue to have more context / feedback on this behavior

AWS SDK added a ClientConfig option called allowSystemProxy, which let curl use PROXYs env variables

can be "fixed" by setting the option to True somewhere near there :

 ref<Aws::Client::ClientConfiguration> S3Helper::makeConfig(
     const std::string & region,
     const std::string & scheme,
     const std::string & endpoint)
 {
     initAWS();
     auto res = make_ref<Aws::Client::ClientConfiguration>();
     res->region = region;
     if (!scheme.empty()) {
         res->scheme = Aws::Http::SchemeMapper::FromString(scheme.c_str());
     }
     if (!endpoint.empty()) {
         res->endpointOverride = endpoint;
     }
     res->requestTimeoutMs = 600 * 1000;
     res->connectTimeoutMs = 5 * 1000;
     res->retryStrategy = std::make_shared<RetryStrategy>();
     res->caFile = settings.caFile;
     return res;
 }

## Describe the bug In the original nix we have the same bug when running nix build airgapped with a PROXY used to call internet all aws-sdk-cpp requests are done PROXY-less, so the host tries to join internet directly, which fails ## Steps To Reproduce block internet traffic except to the proxy host allow anything on the proxy try to build / fetch some cached biary pkgs ## Expected behavior uses cache corectly ## `nix --version` output tried with Nix officiel and Lix beta / rc version ## Additional context not sure if this is a too much breaking change or if it can be implemented easily, so I made this issue to have more context / feedback on this behavior AWS SDK added a ClientConfig option called allowSystemProxy, which let curl use PROXYs env variables can be "fixed" by setting the option to True somewhere near there : https://git.lix.systems/lix-project/lix/src/commit/d00edfb28d0a52d9acd392c582a43f98e773cf4c/src/libstore/s3-binary-cache-store.cc#L125-L144

itzwam added the

bug

label 2024-07-05 17:32:14 +00:00

jade commented

2024-08-18 21:52:51 +00:00

Owner

Didn't notice this report. As far as I know the fact that we are not setting allowSystemProxy is a bug. I'll go submit a change to just set that; I cannot imagine a good reason that the proxy should not be used for s3.

lix-bot commented

2024-08-18 21:57:56 +00:00

Member

This issue was mentioned on Gerrit on the following CLs:

commit message in cl/1788 ("fix: use http proxy for s3 access")

This issue was mentioned on Gerrit on the following CLs: * commit message in [cl/1788](https://gerrit.lix.systems/c/lix/+/1788) ("fix: use http proxy for s3 access")