lix/doc/manual/packages/s3-substituter.xml

192 lines
5.8 KiB
XML
Raw Normal View History

2018-08-01 15:03:32 +00:00
<?xml version="1.0" encoding="utf-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:xi="http://www.w3.org/2001/XInclude"
version="5.0"
xml:id="ssec-s3-substituter">
<title>Serving a Nix store via S3</title>
2018-08-01 15:03:32 +00:00
<para>Nix has built-in support for storing and fetching store paths
from Amazon S3 and S3-compatible services. This uses the same
2018-08-01 15:03:32 +00:00
<emphasis>binary</emphasis> cache mechanism that Nix usually uses to
fetch prebuilt binaries from <uri>cache.nixos.org</uri>.</para>
<para>The following options can be specified as URL parameters to
the S3 URL:</para>
<variablelist>
<varlistentry><term><literal>profile</literal></term>
<listitem>
<para>
The name of the AWS configuration profile to use. By default
Nix will use the <literal>default</literal> profile.
</para>
</listitem>
</varlistentry>
<varlistentry><term><literal>region</literal></term>
<listitem>
<para>
The region of the S3 bucket. <literal>useast-1</literal> by
default.
</para>
<para>
If your bucket is not in <literal>useast-1</literal>, you
should always explicitly specify the region parameter.
</para>
</listitem>
</varlistentry>
<varlistentry><term><literal>endpoint</literal></term>
<listitem>
<para>
The URL to your S3-compatible service, for when not using
Amazon S3. Do not specify this value if you're using Amazon
S3.
</para>
<note><para>This endpoint must support HTTPS and will use
path-based addressing instead of virtual host based
addressing.</para></note>
</listitem>
</varlistentry>
2018-12-07 22:48:35 +00:00
<varlistentry><term><literal>scheme</literal></term>
<listitem>
<para>
The scheme used for S3 requests, <literal>https</literal>
(default) or <literal>http</literal>. This option allows you to
disable HTTPS for binary caches which don't support it.
</para>
<note><para>HTTPS should be used if the cache might contain
sensitive information.</para></note>
</listitem>
</varlistentry>
</variablelist>
2018-08-01 15:03:32 +00:00
<para>In this example we will use the bucket named
<literal>example-nix-cache</literal>.</para>
2018-08-01 15:03:32 +00:00
<section xml:id="ssec-s3-substituter-anonymous-reads">
<title>Anonymous Reads to your S3-compatible binary cache</title>
<para>If your binary cache is publicly accessible and does not
require authentication, the simplest and easiest way to use Nix with
your S3 compatible binary cache is to use the HTTP URL for that
cache.</para>
<para>For AWS S3 the binary cache URL for example bucket will be
exactly <uri>https://example-nix-cache.s3.amazonaws.com</uri> or
<uri>s3://example-nix-cache</uri>. For S3 compatible binary caches,
consult that cache's documentation.</para>
2018-08-01 15:03:32 +00:00
<para>Your bucket will need the following bucket policy:</para>
<programlisting><![CDATA[
2018-08-01 15:03:32 +00:00
{
"Id": "DirectReads",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowDirectReads",
"Action": [
"s3:GetObject",
"s3:GetBucketLocation"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::example-nix-cache",
"arn:aws:s3:::example-nix-cache/*"
],
"Principal": "*"
}
]
2018-08-01 15:03:32 +00:00
}
]]></programlisting>
2018-08-01 15:03:32 +00:00
</section>
<section xml:id="ssec-s3-substituter-authenticated-reads">
<title>Authenticated Reads to your S3 binary cache</title>
<para>For AWS S3 the binary cache URL for example bucket will be
exactly <uri>s3://example-nix-cache</uri>.</para>
2018-08-01 15:03:32 +00:00
<para>Nix will use the <link
2019-06-20 12:47:45 +00:00
xlink:href="https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/credentials.html">default
2018-08-01 15:03:32 +00:00
credential provider chain</link> for authenticating requests to
Amazon S3.</para>
<para>Nix supports authenticated reads from Amazon S3 and S3
compatible binary caches.</para>
2018-08-01 15:03:32 +00:00
<para>Your bucket will need a bucket policy allowing the desired
users to perform the <literal>s3:GetObject</literal> and
<literal>s3:GetBucketLocation</literal> action on all objects in the
bucket. The anonymous policy in <xref
linkend="ssec-s3-substituter-anonymous-reads" /> can be updated to
have a restricted <literal>Principal</literal> to support
this.</para>
2018-08-01 15:03:32 +00:00
</section>
<section xml:id="ssec-s3-substituter-authenticated-writes">
<title>Authenticated Writes to your S3-compatible binary cache</title>
<para>Nix support fully supports writing to Amazon S3 and S3
compatible buckets. The binary cache URL for our example bucket will
be <uri>s3://example-nix-cache</uri>.</para>
2018-08-01 15:03:32 +00:00
<para>Nix will use the <link
2019-06-20 12:47:45 +00:00
xlink:href="https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/credentials.html">default
2018-08-01 15:03:32 +00:00
credential provider chain</link> for authenticating requests to
Amazon S3.</para>
<para>Your account will need the following IAM policy to
upload to the cache:</para>
<programlisting><![CDATA[
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "UploadToCache",
"Effect": "Allow",
"Action": [
"s3:AbortMultipartUpload",
"s3:GetBucketLocation",
"s3:GetObject",
"s3:ListBucket",
"s3:ListBucketMultipartUploads",
"s3:ListMultipartUploadParts",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::example-nix-cache",
"arn:aws:s3:::example-nix-cache/*"
]
}
]
}
]]></programlisting>
</section>
<section><title>Examples</title>
<para>To upload with a specific credential profile for Amazon S3:</para>
<screen>
nix copy --to 's3://example-nix-cache?profile=cache-upload&amp;region=eu-west-2' nixpkgs.hello
</screen>
<para>To upload to an S3-compatible binary cache:</para>
<screen>
nix copy --to 's3://example-nix-cache?profile=cache-upload&amp;scheme=https&amp;endpoint=minio.example.com' nixpkgs.hello
</screen>
2018-08-01 15:03:32 +00:00
</section>
2018-08-01 15:03:32 +00:00
</section>