lehmanator/lix
0
0
Fork 0
forked from lix-project/lix
Code Pull requests Activity
lix/releng/README.md

143 lines
6.4 KiB
Markdown
Raw Normal View History

Put into place initial release engineering This can release x86_64-linux binaries to staging, with ephemeral keys. I think it's good enough to review at least at this point, so we don't keep adding more stuff to it to make it harder to review. Change-Id: Ie95e8f35d1252f5d014e819566f170b30eda152e
2024-05-31 23:35:13 +00:00
# Release engineering
This directory contains the release engineering scripts for Lix.
## Release process
### Prerequisites
* FIXME: validation via misc tests in nixpkgs, among other things? What other
validation do we need before we can actually release?
* Have a release post ready to go as a PR on the website repo.
* No [release-blocker bugs][release-blockers].
[release-blockers]: https://git.lix.systems/lix-project/lix/issues?q=&type=all&sort=&labels=145&state=open&milestone=0&project=0&assignee=0&poster=0
### Process
The following process can be done either against the staging environment or the
live environment.
For staging, the buckets are `staging-releases`, `staging-cache`, etc.
FIXME: obtainment of signing key for signing cache paths?
First, we prepare the release. `python -m releng prepare` is used for this.
* Gather everything in `doc/manual/rl-next` and put it in
`doc/manual/src/release-notes/rl-MAJOR.md`.
Then we tag the release with `python -m releng tag`:
* Git HEAD is detached.
releng: move officialRelease to version.json This was causing a few bits of suffering downstream, in particular, in the NixOS module, which, after this change, can have the `officialRelease` stuff in *it* completely deleted since we now have correct defaulting in package.nix for it. It also eliminates some automated editing of Nix files, which is certainly always welcome to eliminate. Fixes: https://git.lix.systems/lix-project/lix/issues/406 Change-Id: Id12f3018cff4633e379dbfcbe26b7bc84922bdaf
2024-07-23 21:25:18 +00:00
* `"official_release": true` is set in `version.json`, this is committed, and a
Put into place initial release engineering This can release x86_64-linux binaries to staging, with ephemeral keys. I think it's good enough to review at least at this point, so we don't keep adding more stuff to it to make it harder to review. Change-Id: Ie95e8f35d1252f5d014e819566f170b30eda152e
2024-05-31 23:35:13 +00:00
release is tagged.
* The tag is merged back into the last branch (either `main` for new releases
or `release-MAJOR` for maintenance releases) with `git merge -s ours VERSION`
creating a history link but ignoring the tree of the release tag.
* Git HEAD is once again detached onto the release tag.
Then, we build the release artifacts with `python -m releng build`:
* Source tarball is generated with `git archive`, then checksummed.
* Manifest for `nix upgrade-nix` is produced and put in `s3://releases` at
`/manifest.nix` and `/lix/lix-VERSION`.
* Release is built: `hydraJobs.binaryTarball` jobs are built, and joined into a
derivation that depends on all of them and adds checksum files. This and the
sources go into `s3://releases/lix/lix-VERSION`.
At this point we have a `release/artifacts` and `release/manual` directory
which are ready to publish, and tags ready for publication. No keys are
required to do this part.
Next, we do the publication with `python -m releng upload`:
* Artifacts for this release are uploaded:
* s3://releases/manifest.nix, changing the default version of Lix for
`nix upgrade-nix`.
* s3://releases/lix/lix-VERSION/ gets the following contents
* Binary tarballs
releng: clarify/update docs, add instructions after tag This is not a proper fix for the confusion that can happen about how the tags are supposed to be used. For a proper fix, we need to do https://git.lix.systems/lix-project/lix/issues/439 and implement worktrees such that the user never sees the git state anymore. Change-Id: I7b543967f522cede486e42684b48cad47da95429
2024-08-08 03:31:02 +00:00
* Docs: `manual/`, primarily as an archive of old manuals
Put into place initial release engineering This can release x86_64-linux binaries to staging, with ephemeral keys. I think it's good enough to review at least at this point, so we don't keep adding more stuff to it to make it harder to review. Change-Id: Ie95e8f35d1252f5d014e819566f170b30eda152e
2024-05-31 23:35:13 +00:00
* Docs as tarball in addition to web.
* Source tarball
releng: clarify/update docs, add instructions after tag This is not a proper fix for the confusion that can happen about how the tags are supposed to be used. For a proper fix, we need to do https://git.lix.systems/lix-project/lix/issues/439 and implement worktrees such that the user never sees the git state anymore. Change-Id: I7b543967f522cede486e42684b48cad47da95429
2024-08-08 03:31:02 +00:00
* Docker image
Put into place initial release engineering This can release x86_64-linux binaries to staging, with ephemeral keys. I think it's good enough to review at least at this point, so we don't keep adding more stuff to it to make it harder to review. Change-Id: Ie95e8f35d1252f5d014e819566f170b30eda152e
2024-05-31 23:35:13 +00:00
* s3://docs/manual/lix/MAJOR
* s3://docs/manual/lix/stable
* The tag is uploaded to the remote repo.
* **Manually** build the installer using the scripts in the installer repo and upload.
FIXME: This currently requires a local Apple Macintosh® aarch64 computer, but
we could possibly automate doing it from the aarch64-darwin builder.
* **Manually** Push the main/release branch directly to gerrit.
* If this is a new major release, branch-off to `release-MAJOR` and push *that* branch
directly to gerrit as well (FIXME: special creds for doing this as a service
account so we don't need to have the gerrit perms to shoot ourselves in the
foot by default because pushing to main is bad?).
FIXME: automate branch-off to `release-*` branch.
* **Manually** (FIXME?) switch back to the release branch, which now has the
correct revision.
releng: clarify/update docs, add instructions after tag This is not a proper fix for the confusion that can happen about how the tags are supposed to be used. For a proper fix, we need to do https://git.lix.systems/lix-project/lix/issues/439 and implement worktrees such that the user never sees the git state anymore. Change-Id: I7b543967f522cede486e42684b48cad47da95429
2024-08-08 03:31:02 +00:00
* Deal with the external systems (see sections below).
Put into place initial release engineering This can release x86_64-linux binaries to staging, with ephemeral keys. I think it's good enough to review at least at this point, so we don't keep adding more stuff to it to make it harder to review. Change-Id: Ie95e8f35d1252f5d014e819566f170b30eda152e
2024-05-31 23:35:13 +00:00
* Post!!
* Merge release blog post to [lix-website].
* Toot about it! https://chaos.social/@lix_project
* Tweet about it! https://twitter.com/lixproject
[lix-website]: https://git.lix.systems/lix-project/lix-website
### Installer
releng: clarify/update docs, add instructions after tag This is not a proper fix for the confusion that can happen about how the tags are supposed to be used. For a proper fix, we need to do https://git.lix.systems/lix-project/lix/issues/439 and implement worktrees such that the user never sees the git state anymore. Change-Id: I7b543967f522cede486e42684b48cad47da95429
2024-08-08 03:31:02 +00:00
The installer is cross-built to several systems from a Mac using `build-all.xsh` and `upload-to-lix.xsh` in the installer repo (FIXME: currently at least; maybe this should be moved here?).
It installs a binary tarball (FIXME: [it should be taught to substitute from cache instead][installer-substitute]) from some URL; this is the `hydraJobs.binaryTarball`.
The default URLs differ by architecture and are [configured here][tarball-urls].
Put into place initial release engineering This can release x86_64-linux binaries to staging, with ephemeral keys. I think it's good enough to review at least at this point, so we don't keep adding more stuff to it to make it harder to review. Change-Id: Ie95e8f35d1252f5d014e819566f170b30eda152e
2024-05-31 23:35:13 +00:00
releng: clarify/update docs, add instructions after tag This is not a proper fix for the confusion that can happen about how the tags are supposed to be used. For a proper fix, we need to do https://git.lix.systems/lix-project/lix/issues/439 and implement worktrees such that the user never sees the git state anymore. Change-Id: I7b543967f522cede486e42684b48cad47da95429
2024-08-08 03:31:02 +00:00
To automatically do the file changes for a new version, run `python3 set_version.py NEW_VERSION`, and submit the result for review.
Put into place initial release engineering This can release x86_64-linux binaries to staging, with ephemeral keys. I think it's good enough to review at least at this point, so we don't keep adding more stuff to it to make it harder to review. Change-Id: Ie95e8f35d1252f5d014e819566f170b30eda152e
2024-05-31 23:35:13 +00:00
[installer-substitute]: https://git.lix.systems/lix-project/lix-installer/issues/13
[tarball-urls]: https://git.lix.systems/lix-project/lix-installer/src/commit/693592ed10d421a885bec0a9dd45e87ab87eb90a/src/settings.rs#L14-L28
releng: clarify/update docs, add instructions after tag This is not a proper fix for the confusion that can happen about how the tags are supposed to be used. For a proper fix, we need to do https://git.lix.systems/lix-project/lix/issues/439 and implement worktrees such that the user never sees the git state anymore. Change-Id: I7b543967f522cede486e42684b48cad47da95429
2024-08-08 03:31:02 +00:00
### Web site
The website has various release-version dependent pieces.
You can update them with `python3 update_version.py NEW_VERSION`, which will regenerate the affected page sources.
These need the release to have been done first as they need hashes for tarballs and such.
### NixOS module
The NixOS module has underdeveloped releng in it.
Currently you have to do the whole branch-off dance manually to a `release-VERSION` branch and update the tarball URLs to point to the release versions manually.
FIXME: this should be unified with the `set_version.py` work in `lix-installer` and probably all the releng kept in here, or kept elsewhere.
Related: https://git.lix.systems/lix-project/lix/issues/439
Put into place initial release engineering This can release x86_64-linux binaries to staging, with ephemeral keys. I think it's good enough to review at least at this point, so we don't keep adding more stuff to it to make it harder to review. Change-Id: Ie95e8f35d1252f5d014e819566f170b30eda152e
2024-05-31 23:35:13 +00:00
## Infrastructure summary
* releases.lix.systems (`s3://releases`):
* Each release gets a directory: https://releases.lix.systems/lix/lix-2.90-beta.1
* Binary tarballs: `nix-2.90.0-beta.1-x86_64-linux.tar.xz`, from `hydraJobs.binaryTarball`
* Manifest: `manifest.nix`, an attrset of the store paths by architecture.
* Manifest for `nix upgrade-nix` to the latest release at `/manifest.nix`.
* cache.lix.systems (`s3://cache`):
* Receives all artifacts for released versions of Lix; is a plain HTTP binary cache.
* install.lix.systems (`s3://install`):
```
~ » aws s3 ls s3://install/lix/
PRE lix-2.90-beta.0/
PRE lix-2.90-beta.1/
PRE lix-2.90.0pre20240411/
PRE lix-2.90.0pre20240412/
2024-05-05 18:59:11 6707344 lix-installer-aarch64-darwin
2024-05-05 18:59:16 7479768 lix-installer-aarch64-linux
2024-05-05 18:59:14 7982208 lix-installer-x86_64-darwin
2024-05-05 18:59:17 8978352 lix-installer-x86_64-linux
~ » aws s3 ls s3://install/lix/lix-2.90-beta.1/
2024-05-05 18:59:01 6707344 lix-installer-aarch64-darwin
2024-05-05 18:59:06 7479768 lix-installer-aarch64-linux
2024-05-05 18:59:03 7982208 lix-installer-x86_64-darwin
2024-05-05 18:59:07 8978352 lix-installer-x86_64-linux
```
Copy permalink