This uses skopeo to not think about docker daemons. I, however, noticed
that the docker image we had would have totally terrible cache hits, so
I rewrote it.
Fixes: lix-project/lix#252
Change-Id:
|
||
---|---|---|
.. | ||
__init__.py | ||
__main__.py | ||
cli.py | ||
create_release.xsh | ||
docker.xsh | ||
environment.py | ||
keys.py | ||
README.md | ||
release-jobs.nix | ||
version.py |
Release engineering
This directory contains the release engineering scripts for Lix.
Release process
Prerequisites
- FIXME: validation via misc tests in nixpkgs, among other things? What other validation do we need before we can actually release?
- Have a release post ready to go as a PR on the website repo.
- No release-blocker bugs.
Process
The following process can be done either against the staging environment or the live environment.
For staging, the buckets are staging-releases
, staging-cache
, etc.
FIXME: obtainment of signing key for signing cache paths?
First, we prepare the release. python -m releng prepare
is used for this.
- Gather everything in
doc/manual/rl-next
and put it indoc/manual/src/release-notes/rl-MAJOR.md
.
Then we tag the release with python -m releng tag
:
- Git HEAD is detached.
officialRelease = true
is set inflake.nix
, this is committed, and a release is tagged.- The tag is merged back into the last branch (either
main
for new releases orrelease-MAJOR
for maintenance releases) withgit merge -s ours VERSION
creating a history link but ignoring the tree of the release tag. - Git HEAD is once again detached onto the release tag.
Then, we build the release artifacts with python -m releng build
:
- Source tarball is generated with
git archive
, then checksummed. - Manifest for
nix upgrade-nix
is produced and put ins3://releases
at/manifest.nix
and/lix/lix-VERSION
. - Release is built:
hydraJobs.binaryTarball
jobs are built, and joined into a derivation that depends on all of them and adds checksum files. This and the sources go intos3://releases/lix/lix-VERSION
.
At this point we have a release/artifacts
and release/manual
directory
which are ready to publish, and tags ready for publication. No keys are
required to do this part.
Next, we do the publication with python -m releng upload
:
-
Artifacts for this release are uploaded:
- s3://releases/manifest.nix, changing the default version of Lix for
nix upgrade-nix
. - s3://releases/lix/lix-VERSION/ gets the following contents
- Binary tarballs
- Docs:
manual/
(FIXME: should we actually do this? what about putting it on docs.lix.systems? I think doing both is correct, since the Web site should not be an archive of random old manuals) - Docs as tarball in addition to web.
- Source tarball
- Docker image (FIXME: upload to forgejo registry and github registry in the future)
- s3://docs/manual/lix/MAJOR
- s3://docs/manual/lix/stable
- s3://releases/manifest.nix, changing the default version of Lix for
-
The tag is uploaded to the remote repo.
-
Manually build the installer using the scripts in the installer repo and upload.
FIXME: This currently requires a local Apple Macintosh® aarch64 computer, but we could possibly automate doing it from the aarch64-darwin builder.
-
Manually Push the main/release branch directly to gerrit.
-
If this is a new major release, branch-off to
release-MAJOR
and push that branch directly to gerrit as well (FIXME: special creds for doing this as a service account so we don't need to have the gerrit perms to shoot ourselves in the foot by default because pushing to main is bad?).FIXME: automate branch-off to
release-*
branch. -
Manually (FIXME?) switch back to the release branch, which now has the correct revision.
-
Post!!
- Merge release blog post to lix-website.
- Toot about it! https://chaos.social/@lix_project
- Tweet about it! https://twitter.com/lixproject
Installer
The installer is cross-built to several systems from a Mac using
build-all.xsh
and upload-to-lix.xsh
in the installer repo (FIXME: currently
at least; maybe this should be moved here?) .
It installs a binary tarball (FIXME: it should be taught to substitute from
cache instead)
from some URL; this is the hydraJobs.binaryTarball
. The default URLs differ
by architecture and are configured here.
Infrastructure summary
- releases.lix.systems (
s3://releases
):- Each release gets a directory: https://releases.lix.systems/lix/lix-2.90-beta.1
- Binary tarballs:
nix-2.90.0-beta.1-x86_64-linux.tar.xz
, fromhydraJobs.binaryTarball
- Manifest:
manifest.nix
, an attrset of the store paths by architecture.
- Binary tarballs:
- Manifest for
nix upgrade-nix
to the latest release at/manifest.nix
.
- Each release gets a directory: https://releases.lix.systems/lix/lix-2.90-beta.1
- cache.lix.systems (
s3://cache
):- Receives all artifacts for released versions of Lix; is a plain HTTP binary cache.
- install.lix.systems (
s3://install
):~ » aws s3 ls s3://install/lix/ PRE lix-2.90-beta.0/ PRE lix-2.90-beta.1/ PRE lix-2.90.0pre20240411/ PRE lix-2.90.0pre20240412/ 2024-05-05 18:59:11 6707344 lix-installer-aarch64-darwin 2024-05-05 18:59:16 7479768 lix-installer-aarch64-linux 2024-05-05 18:59:14 7982208 lix-installer-x86_64-darwin 2024-05-05 18:59:17 8978352 lix-installer-x86_64-linux ~ » aws s3 ls s3://install/lix/lix-2.90-beta.1/ 2024-05-05 18:59:01 6707344 lix-installer-aarch64-darwin 2024-05-05 18:59:06 7479768 lix-installer-aarch64-linux 2024-05-05 18:59:03 7982208 lix-installer-x86_64-darwin 2024-05-05 18:59:07 8978352 lix-installer-x86_64-linux