From 51cbeec49ae78b0109dbed2e9efd641415853aa7 Mon Sep 17 00:00:00 2001 From: Graham Christensen Date: Thu, 27 Sep 2018 16:54:20 -0400 Subject: [PATCH] Update docs to describe how s3:// URLS does in fact support endpoint, region, and profile for upload --- doc/manual/packages/s3-substituter.xml | 196 +++++++++++++------------ 1 file changed, 104 insertions(+), 92 deletions(-) diff --git a/doc/manual/packages/s3-substituter.xml b/doc/manual/packages/s3-substituter.xml index bcd91cfdb..ea654392c 100644 --- a/doc/manual/packages/s3-substituter.xml +++ b/doc/manual/packages/s3-substituter.xml @@ -12,8 +12,49 @@ from Amazon S3 and S3 compatible services. This uses the same binary cache mechanism that Nix usually uses to fetch prebuilt binaries from cache.nixos.org. +The following options can be specified as URL parameters to +the S3 URL: + + + profile + + + The name of the AWS configuration profile to use. By default + Nix will use the default profile. + + + + + region + + + The region of the S3 bucket. us–east-1 by + default. + + + + If your bucket is not in us–east-1, you + should always explicitly specify the region parameter. + + + + + endpoint + + + The URL to your S3-compatible service, for when not using + Amazon S3. Do not specify this value if you're using Amazon + S3. + + This endpoint must support HTTPS and will use + path-based addressing instead of virtual host based + addressing. + + + + In this example we will use the bucket named -example-bucket. +example-nix-cache.
Anonymous Reads to your S3-compatible binary cache @@ -24,65 +65,56 @@ fetch prebuilt binaries from cache.nixos.org. cache. For AWS S3 the binary cache URL for example bucket will be - exactly https://example-bucket.s3.amazonaws.com. For S3 - compatible binary caches ago have to consult your software's - documentation. + exactly https://example-nix-cache.s3.amazonaws.com or + s3://example-nix-cache. For S3 compatible binary caches, + consult that cache's documentation. Your bucket will need the following bucket policy: - - - +]]>
Authenticated Reads to your S3 binary cache For AWS S3 the binary cache URL for example bucket will be - exactly s3://example-bucket. + exactly s3://example-nix-cache. Nix will use the default credential provider chain for authenticating requests to Amazon S3. - Nix supports authenticated writes to S3 compatible binary - caches but only supports Authenticated reads from Amazon S3. - Additionally, the following limitations are in place for - authenticated reads: - - - The bucket must actually be hosted by Amazon S3 and - not an S3 compatible - service. - - The bucket must be within the - us-east-1 region. - - The Amazon credentials, if stored in a credential - profile, must be stored in the default - profile. - + Nix supports authenticated reads from Amazon S3 and S3 + compatible binary caches. Your bucket will need a bucket policy allowing the desired - users to perform the s3:GetObject action on all - objects in the bucket. + users to perform the s3:GetObject and + s3:GetBucketLocation action on all objects in the + bucket. The anonymous policy in can be updated to + have a restricted Principal to support + this.
@@ -91,69 +123,49 @@ fetch prebuilt binaries from cache.nixos.org. Nix support fully supports writing to Amazon S3 and S3 compatible buckets. The binary cache URL for our example bucket will - be s3://example-bucket. + be s3://example-nix-cache. Nix will use the default credential provider chain for authenticating requests to Amazon S3. - The following options can be specified as URL parameters to - the S3 URL: - - profile - - - The name of the AWS configuration profile to use. By default - Nix will use the default profile. - - - + Your account will need the following IAM policy to + upload to the cache: - region - - - The region of the S3 bucket. us–east-1 by - default. - - - + - endpoint - - - The URL to your S3-compatible service, for when not using - Amazon S3. Do not specify this value if you're using Amazon - S3. - - This endpoint must support HTTPS and will use - path-based addressing instead of virtual host based - addressing. - - - - Uploading with non-default credential profile for Amazon S3 - nix copy --to ssh://machine nixpkgs.hello s3://example-bucket?profile=cache-upload + Uploading with a specific credential profile for Amazon S3 + nix copy --to 's3://example-nix-cache?profile=cache-upload&region=eu-west-2' nixpkgs.hello Uploading to an S3-Compatible Binary Cache - nix copy --to ssh://machine nixpkgs.hello s3://example-bucket?profile=cache-upload&endpoint=minio.example.com + nix copy --to 's3://example-nix-cache?profile=cache-upload&endpoint=minio.example.com' nixpkgs.hello - - The user writing to the bucket will need to perform the - following actions against the bucket: - - - s3:ListBucket - s3:GetBucketLocation - s3:ListObjects - s3:GetObject - s3:PutObject - s3:ListBucketMultipartUploads - s3:CreateMultipartUpload - s3:ListMultipartUploadParts - s3:AbortMultipartUpload -