lix-project/lix
20
57
Fork 31
Code Issues 349 Code Review (Gerrit) Projects 3 Releases Wiki Activity

releng: fix upload of multiarch images to forgejo

Browse source 
Forgejo appears to immediately delete registry content that is
overwritten. This means that we are forced to delete our previous
workaround of making a temporary tag and use a new, more absurd
workaround of making an entire temporary image that we basically only
need to create to get its hash.

However, on the plus side, the new workaround doesn't create garbage
tags to begin with, which means that we don't have to deal with GitHub
not implementing the standardized tag delete endpoint and instead
only implementing a proprietary one.

Upstream-Bug: https://github.com/containers/skopeo/issues/2354
Change-Id: I220e7ce9a17fd230c38882f12c009a166dcc9336
This commit is contained in:
jade 2024-06-13 17:04:07 -07:00
parent 7be0d237e0
commit 16ea19ced8
2 changed files with 22 additions and 13 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

29
releng/docker.xsh
View file

@ -19,6 +19,7 @@ def check_all_logins(env: RelengEnvironment):
check_login(target) check_login(target)
def check_login(target: DockerTarget): def check_login(target: DockerTarget):
log.info('Checking login for %s', target.registry_name)
skopeo login @(target.registry_name()) skopeo login @(target.registry_name())
def upload_docker_images(target: DockerTarget, paths: list[Path]): def upload_docker_images(target: DockerTarget, paths: list[Path]):
@ -43,7 +44,23 @@ def upload_docker_images(target: DockerTarget, paths: list[Path]):
for path in paths: for path in paths:
digest_file = tmp / (path.name + '.digest') digest_file = tmp / (path.name + '.digest')
inspection = json.loads($(skopeo inspect docker-archive:@(path))) tmp_image = tmp / 'tmp-image.tar.gz'
# insecure-policy: we don't have any signature policy, we are just uploading an image
#
# Absurd: we copy it into an OCI image first so we can get the hash
# we need to upload it untagged, because skopeo has no "don't tag
# this" option.
# The reason for this is that forgejo's container registry throws
# away old versions of tags immediately, so we cannot use a temp
# tag, and it *does* reduce confusion to not upload tags that
# should not be used.
#
# Workaround for: https://github.com/containers/skopeo/issues/2354
log.info('skopeo copy to temp oci-archive %s', tmp_image)
skopeo --insecure-policy copy --format oci --all --digestfile @(digest_file) docker-archive:@(path) oci-archive:@(tmp_image)
inspection = json.loads($(skopeo inspect oci-archive:@(tmp_image)))
docker_arch = inspection['Architecture'] docker_arch = inspection['Architecture']
docker_os = inspection['Os'] docker_os = inspection['Os']
@ -51,21 +68,13 @@ def upload_docker_images(target: DockerTarget, paths: list[Path]):
log.info('Pushing image %s for %s to %s', path, docker_arch, target.registry_path) log.info('Pushing image %s for %s to %s', path, docker_arch, target.registry_path)
# insecure-policy: we don't have any signature policy, we are just uploading an image
# We upload to a junk tag, because otherwise it will upload to `latest`, which is undesirable
skopeo --insecure-policy copy --format oci --digestfile @(digest_file) docker-archive:@(path) docker://@(target.registry_path):temp
digest = digest_file.read_text().strip() digest = digest_file.read_text().strip()
skopeo --insecure-policy copy --preserve-digests --all oci-archive:@(tmp_image) f'docker://{target.registry_path}@{digest}'
# skopeo doesn't give us the manifest size directly, so we just ask the registry # skopeo doesn't give us the manifest size directly, so we just ask the registry
metadata = reg.image_info(target.registry_path, digest) metadata = reg.image_info(target.registry_path, digest)
manifests.append(OCIIndexItem(metadata=metadata, architecture=docker_arch, os=docker_os)) manifests.append(OCIIndexItem(metadata=metadata, architecture=docker_arch, os=docker_os))
# delete the temp tag, which we only have to create because of skopeo
# limitations anyhow (it seems to not have a way to say "don't tag it, find
# your checksum and put it there")
# FIXME: this is not possible because GitHub only has a proprietary API for it. amazing. 11/10.
# reg.delete_tag(target.registry_path, 'temp')
log.info('Pushed images to %r, building a bigger and more menacing manifest from %r with metadata %r', target, manifests, meta) log.info('Pushed images to %r, building a bigger and more menacing manifest from %r with metadata %r', target, manifests, meta)
# send the multiarch manifest to each tag # send the multiarch manifest to each tag

6
releng/docker_assemble.py
View file

@ -49,8 +49,8 @@ if DEBUG_REQUESTS:
# fix that. Thus, a little bit of homebrew containers code. # fix that. Thus, a little bit of homebrew containers code.
# #
# Essentially what we are doing in here is splatting a bunch of images into the # Essentially what we are doing in here is splatting a bunch of images into the
# registry without tagging them (except as "temp", due to podman issues), then # registry without tagging them (with a silly workaround to skopeo issues),
# simply sending a new composite manifest ourselves. # then simply sending a new composite manifest ourselves.
DockerArchitecture = Literal['amd64'] | Literal['arm64'] DockerArchitecture = Literal['amd64'] | Literal['arm64']
MANIFEST_MIME = 'application/vnd.oci.image.manifest.v1+json' MANIFEST_MIME = 'application/vnd.oci.image.manifest.v1+json'
@ -276,7 +276,7 @@ class AuthState:
'Authorization': 'Basic ' + creds 'Authorization': 'Basic ' + creds
}).json() }).json()
token = resp['token'] token = resp['token']
self.token_cache[service] = token self.token_cache[authority] = token
return token return token
def find_credential_for(self, image_path: str): def find_credential_for(self, image_path: str):