Deploy channel scripts #8
Labels
No labels
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: the-distro/infra#8
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
#91 is a first step, it takes care of the 2nd item of the previous message more or less.
Plan to rewrite the Perl script, here's what it's doing:
(1) Dry run feature
(2) Configuration file (state directory for nixpkgs, S3 targets, Hydra URL) -- currently hardcoded
(3) Protection against spurious channel rollback
(4) Idempotency (pushing twice the same release has no effect)
S3 operations
It writes the following in the root of the release directory:
src-url
: evaluation URLgit-revision
: git revisionbinary-cache-url
: the binary cache URL associated (hardcoded to cno)store-paths.xz
if available, otherwisestore-paths
by manual fetching (?)If it's a NixOS channel:
nixexprs.tar.xz
by downloading thenixos.channel
productpackages.json.br
by downloading thenixpkgs.tarball
product of typejson-br
options.json.br
by downloading thenixos.options
product of typejson-br
nixos.iso_minimal.aarch64-linux
nixos.iso_minimal.x86_64-linux
If it's not small: add
nixos.iso_plasma(5|6).(aarch64|x86_64)-linux
.Add the OVA appliance as well (
nixos.ova.x86_64-linux
).Otherwise (nixpkgs, etc.): add
nixexprs.tar.xz
from thetarball
product, of typesource-dist
andpackages.json.br
from thetarball
product of typejson-br
.If it's a NixOS channel, generate the SQLite database and add it to
nixexprs.tar.xz
in a mutable way (https://github.com/NixOS/nixos-channel-scripts/blob/master/mirror-nixos-branch.pl#L247-L256).Write an HTML page for that particular release.
Finally, upload all the contents with some Content-Encoding fiddling and checksums computation which are added to the HTML page at the end to the S3 release bucket.
Git operations
git remote update origin <&2 && git push origin $rev:refs/heads/$channelName &>2
is performed.S3 redirection operations
Redirections are added from the channel name to the release prefix name for:
/
/nixexprs.tar.xz
/git-revision
/packages.json.br
/store-paths.xz
If the channel is NixOS-type:
/options.json.br
is addedlatest-$type-$arch.iso
Proposed rewrite in Rust is to separate the concerns and get incremental feature parity:
(1) A configuration file to centralize all interesting parameters
(2) Implement each download and store operations with a work queue design
(3) Drop all the legacy cruft (compression, etc, etc.)
(4) Drop the SQLite database mutable generation stuff for now
(5) Implement the full S3 uploading process
(6) Git operations
For now, I suggest to put aside the S3 redirections.
The rewrite in Rust is moving at a steady pace here: https://git.lix.systems/the-distro/channel-scripts/src/branch/rust.
I'm taking the lock because I have almost all the context now.