CI system for the distro
Find a file
Daiderd Jordan 08717c5cf2
Merge pull request #496 from LnL7/converted-to-draft
handle converted_to_draft events
2020-05-23 00:32:37 +02:00
.github Add .github/FUNDING.yml 2019-06-01 11:20:08 -04:00
doc Add a sample-command workflow 2017-11-06 12:38:23 -05:00
log-api The log API grows new features.. 2018-03-11 11:19:07 -04:00
nix apply carnix patches 2020-04-29 23:00:53 +02:00
ofborg handle converted_to_draft events 2020-05-23 00:16:46 +02:00
ofborg-simple-build work around carnix workspace issue 2020-04-25 14:28:29 +02:00
php Update to current nixpkgs-unstable 2020-02-21 17:10:45 +01:00
.gitignore ignore nix patches 2018-02-09 19:33:29 -05:00
.travis.yml Remove nix1 2020-04-12 11:04:40 -07:00
Cargo.lock lapin: 1.0.0-beta3 -> 1.0.0-beta4 2020-05-11 20:48:27 +02:00
Cargo.nix lapin: 1.0.0-beta3 -> 1.0.0-beta4 2020-05-11 20:48:27 +02:00
Cargo.toml move simple-build to a separate project 2020-04-24 22:17:12 +02:00
CODE_OF_CONDUCT.md Create CODE_OF_CONDUCT.md 2017-11-29 19:21:27 -05:00
config.public.json config.public.json: add zowoq to trusted users for darwin access 2020-05-21 08:59:02 +10:00
crates-io.list lapin: 1.0.0-beta3 -> 1.0.0-beta4 2020-05-11 20:48:27 +02:00
crates-io.nix lapin: 1.0.0-beta3 -> 1.0.0-beta4 2020-05-11 20:48:27 +02:00
default.nix Pepper in allowing old openssl 2020-05-19 23:19:00 -04:00
example.config.json Document my experience with re-setting-up a builder 2018-03-25 23:58:58 +02:00
factoids.toml factoids: Adds !stateVersion 2018-04-21 18:22:48 -04:00
LICENSE Add a license 2017-11-24 08:51:55 -05:00
README.md README: "Not you: team." 2020-05-20 10:20:07 -07:00
release.nix Pepper in allowing old openssl 2020-05-19 23:19:00 -04:00
service.nix borg service example 2017-11-04 12:24:18 -04:00
shell.nix Pepper in allowing old openssl 2020-05-19 23:19:00 -04:00

ofborg

Guidelines

  1. Review the code of all PRs before triggering the bot on them.
  2. Be gentle; try not to run mass rebuilds or massive builds (like Chromium) on it.

Automatic Building

All users will have their PRs automatically trigger builds if their commits follow the well-defined format of Nixpkgs. Specifically: prefixing the commit title with the package attribute. This includes package bumps as well as other changes.

Example commit titles and the builds they will start:

Message Automatic Build
vim: 1.0.0 -> 2.0.0 vim
vagrant: Fix dependencies for version 2.0.2 vagrant
python36Packages.requests,python27Packages.requests: 1.0.0 -> 2.0.0 python36Packages.requests, python27Packages.requests
python{2,3}Packages.requests: 1.0.0 -> 2.0.0 nothing

When opening a PR with multiple commits, ofborg creates a single build job for all detected packages. If multiple commits get pushed to a PR one-by-one, each detected package will get a separate build job.

If the title of a PR begins with WIP:, contains [WIP] anywhere, or has the 2.status: work-in-progress label, its packages are not built automatically. Note: Marking a PR as a draft does not prevent automatic builds.

Commands

The comment parser is line-based, so commentary can be interwoven with instructions for ofborg.

  1. To trigger the bot, the line must start with @ofborg (case insensitive).
    • Note: GitHub will not suggest @ofborg to you, but it will work all the same. When in doubt, preview your comment and verify that @ofborg links to https://github.com/ofborg/.
  2. To use multiple commands, separate them with whitespace. For examples, see the "Multiple Commands" section.

test

@ofborg test list of tests

This will run nix-build ./nixos/release.nix -A tests.list -A tests.of -A tests.tests from the root of the Nixpkgs checkout.

Tests will run on all allowed machines. For more information, see the "Trusted Users" section.

eval

@ofborg eval

See "How does ofborg call nix-instantiate?" for what command(s) this will run.

Note: Every PR automatically evaluates both upon creation and when the commits change. There is no reason to run eval on a PR unless the evaluation failed for weird reasons or master was previously broken.

build

@ofborg build list of attrs

This will run nix-build ./default.nix -A list -A of -A attrs from the root of the Nixpkgs checkout (see also "How does ofborg call nix-build?").

Builds will run on all allowed machines. For more information, see the "Trusted Users" section.

Multiple Commands

You can use multiple commands in a variety ways. Here are some valid combinations:

  • @ofborg build list of attrs
    @ofborg eval
    
  • @ofborg build list of attrs @ofborg eval
    
  • looks good to me!
    @ofborg eval
    @ofborg build list of attrs
    
  • @ofborg eval
    @ofborg build list of attrs
    looks good to me!
    
  • @ofborg build list of attrs
    @ofborg test list of attrs
    
  • This will build list, of, attrs, looks, good, to, and me! (which is probably not what you want):

    @ofborg build list of attrs looks good to me!
    

Trusted Users

Trusted users have their builds and tests executed on all available platforms, including those without good sandboxing. Because this exposes the host to a higher risk of security issues, the trusted users list consists of only well-known, trusted members of the community.

At the time of writing, trusted users have their builds and tests run on these platforms:

  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin

See config.public.json for a list of all trusted users.

How does ofborg call nix-build?

ofborg runs builds with a command similar to the following:

$ HOME=/homeless-shelter NIX_PATH=ofborg-nixpkgs-pr=$(pwd) nix-build ./default.nix \
    -A hello \
    --no-out-link \
    --keep-going \
    --option restrict-eval true \ 
    --option build-timeout 1800 \ 
    --argstr system thesystem \
    --show-trace

How does ofborg call nix-instantiate?

ofborg runs NixOS evals with a command similar to the following:

$ HOME=/homeless-shelter NIX_PATH=ofborg-nixpkgs-pr=$(pwd) nix-instantiate ./nixos/release.nix \
    -A manual \
    --option restrict-eval true \
    --option build-timeout 1800 \
    --argstr system thesystem \
    --show-trace

ofborg runs Nixpkgs evals with a command similar to the following:

$ HOME=/homeless-shelter NIX_PATH=ofborg-nixpkgs-pr=$(pwd) nix-instantiate ./pkgs/top-level/release.nix \
    -A manual \
    --option restrict-eval true \
    --option build-timeout 1800 \
    --argstr system thesystem \
    --show-trace

Running meta checks locally

To run the meta checks, you will need the outpaths.nix file. You can acquire this file and run the checks themselves like so:

$ curl -o outpaths.nix https://raw.githubusercontent.com/NixOS/ofborg/released/ofborg/src/outpaths.nix
$ GC_INITIAL_HEAP_SIZE=4g nix-env -f ./outpaths.nix -qaP --no-name --out-path --arg checkMeta true > out-paths

Hacking

$ git clone https://github.com/NixOS/ofborg/
$ cd ofborg
$ nix-shell ./shell.nix
$ cd ofborg # enter the subdirectory with Rust code
# make your changes
$ cargo build
$ cargo check
$ cargo test

To test whether or not Continuous Integration will pass with your changes, you can run the following commands from the root of your checkout:

$ nix-shell --run checkPhase -A mozilla-rust-overlay # checks rustfmt and clippy
$ nix-shell --run checkPhase # runs the test suite
$ nix-build -A ofborg.rs # build ofborg

Currently there is no easy way to set up a test instance of ofborg. If cargo check and cargo test both succeed, feel free to Pull Request your changes. Make sure to format your code with cargo fmt and check for additional warnings with cargo clippy. If you added, removed, or updated the dependencies, also be sure to update Carnix by running ./nix/update-carnix.sh.

To disable warnings as errors, run your command with an empty RUSTFLAGS. For example:

$ RUSTFLAGS= cargo clippy

This will override the default of -D warnings set in shell.nix, which tells Rust to error if it detects any warnings.

Running a builder

If you want to run a builder of your own, check out the wiki page on operating a builder.