Commit graph

1397 commits

Author SHA1 Message Date
Eelco Dolstra 14d4624dc2
Merge pull request #1215 from knedlsepp/fix-hydra-with-latest-git
scmdiff: Hardcode `--git-dir`
2022-11-24 12:04:23 +01:00
Maximilian Bosch d3fe4ffbf6
Job: expose closuresize and size (output size in the UI) as prometheus metrics 2022-09-22 10:47:22 +02:00
Janne Heß 371402c3c1
Drop the HipChat plugin
https://en.wikipedia.org/wiki/HipChat says:
> Following this, HipChat and Stride customers were migrated to the
> Slack group collaboration platform in a transition that was completed by
> February 2019.
2022-08-20 19:16:43 +02:00
Marco Rebhan a58e2f1a64
Use libmagic for better output MIME detection 2022-08-04 22:34:58 +02:00
Janne Heß e05118171b
Merge pull request #1229 from helsinki-systems/fix/nix-cat-store
replace nix cat-store with nix store cat
2022-07-01 13:28:27 +02:00
ajs124 bb1f04ed86 AddBuilds: fix declarative jobsets with dynamic runcommand enabled
$project->{enable_dynamic_run_command} is undefined
2022-06-30 01:49:30 +02:00
ajs124 bab671124d replace nix cat-store with nix store cat
the former was deprecated in favor of the latter
2022-06-30 00:24:09 +02:00
Josef Kemetmüller b6ea85a601 scmdiff: Hardcode --git-dir
The newest version of git refuses to work on repositories not owned by
the current user. This leads to issues with the /api/scmdiff endpoint:

May 27 11:16:05 myhydra hydra-server[923698]: fatal: unsafe repository ('/var/lib/hydra/scm/git/57ea036ec7ecd85c8dd085e02ecc6f12dd5c079a6203d16aea49f586cadfb2be' is owned by someone else)
May 27 11:16:05 myhydra hydra-server[923698]: To add an exception for this directory, call:
May 27 11:16:05 myhydra hydra-server[923698]:         git config --global --add safe.directory /var/lib/hydra/scm/git/57ea036ec7ecd85c8dd085e02ecc6f12dd5c079a6203d16aea49f586cadfb2be
May 27 11:16:05 myhydra hydra-server[923701]: warning: Not a git repository. Use --no-index to compare two paths outside a working tree
May 27 11:16:05 myhydra hydra-server[923701]: usage: git diff --no-index [<options>] <path> <path>

I used the same solution that was used in NixOS/nix#6440.

Fixes #1214
2022-05-27 12:12:44 +02:00
Maximilian Bosch a8b590014b
Fix email notifications for jobsets w/git-inputs
I started to wonder quite recently why Hydra doesn't send email
notifications anymore to me. I saw the following issue in the log of
`hydra-notify.service`:

    May 22 11:57:29 hydra 9bik0bxyxbrklhx6lqwifd6af8kj84va-hydra-notify[1887289]: fatal: unsafe repository ('/var/lib/hydra/scm/git/3e70c16c266ef70dc4198705a688acccf71e932878f178277c9ac47d133cc663' is owned by someone else)
    May 22 11:57:29 hydra 9bik0bxyxbrklhx6lqwifd6af8kj84va-hydra-notify[1887289]: To add an exception for this directory, call:
    May 22 11:57:29 hydra 9bik0bxyxbrklhx6lqwifd6af8kj84va-hydra-notify[1887289]:         git config --global --add safe.directory /var/lib/hydra/scm/git/3e70c16c266ef70dc4198705a688acccf71e932878f178277c9ac47d133cc663
    May 22 11:57:29 hydra 9bik0bxyxbrklhx6lqwifd6af8kj84va-hydra-notify[1886654]: error running build_finished hooks: command `git log --pretty=format:%H%x09%an%x09%ae%x09%at b0c30a7557685d25a8ab3f34fdb775e66db0bc4c..eaf28389fcebc2beca13a802f79b2cca6e9ca309 --git-dir=.git' failed with e>

This is also a problem because of Git's fix for CVE-2022-24765[1], so I
applied the same fix as for Nix[2], by using `--git-dir` which skips the
code-path for the ownership-check[3].

[1] https://lore.kernel.org/git/xmqqv8veb5i6.fsf@gitster.g/
[2] https://github.com/NixOS/nix/pull/6440
[3] To quote `git(1)`:
    > Specifying the location of the ".git" directory using this option
    > (or GIT_DIR environment variable) turns off the repository
    > discovery that tries to find a directory with ".git" subdirectory
2022-05-22 14:14:14 +02:00
Ulrik Strid 3c71be5b5b GithubPulls: Don't fail on missing Link 2022-05-18 08:14:00 +02:00
Graham Christensen 5c90edd19f
Merge pull request #1103 from DeterminateSystems/runcommand/dynamic
Dynamic RunCommand
2022-04-19 10:09:47 -04:00
Graham Christensen 27ddde1e9e dynamic runcommand: print a notice on the build page if it is disabled 2022-02-11 15:04:54 -05:00
Cole Helbling a22a8fa62d AddBuilds: reject declarative jobsets with dynamic runcommand enabled if disabled elsewhere 2022-02-11 14:35:52 -05:00
Cole Helbling 928ba9e854 Controller/{Jobset,Project}: error when enabling dynamic runcommand but it's disabled elsewhere 2022-02-11 14:35:52 -05:00
Graham Christensen 71c06f2ce7 LDAP normalization errors: note that the error came while normalizing the roles. 2022-02-11 10:55:27 -05:00
Graham Christensen f07fb7d279 LDAP support: include BC support for the YAML based loading
Includes a refactoring of the configuration loader.
2022-02-11 10:49:38 -05:00
Janne Heß 61d74a7194 Redo LDAP config in the main configuration and add role mappings 2022-02-11 10:49:38 -05:00
Graham Christensen 9ae7c8bddc Hydra::Helper::Exec add an expectOkay which dies with stdout / stderr on exit 2022-02-09 20:56:10 -05:00
Graham Christensen 845e6d4760 captureStdoutStderr*: move to Hydra::Helper::Exec which helps avoid some environment variable fixation problems 2022-02-09 14:28:50 -05:00
Graham Christensen 517dce285a eval_added event: change interface to traceID\tjobsetID\tevaluationID
I was not going to break the interface until I noticed
the current implementation uses the string literal \t.
2022-02-08 09:51:35 -05:00
Graham Christensen d512e6220f eval_failed event: change interface to traceID\tjobsetID
I was not going to break the interface until I noticed the other eval_* events used literal \ts
2022-02-08 09:51:35 -05:00
Graham Christensen 2597fa8c11 eval_cached event: change interface to traceID\tjobsetID\tevaluationID
I was not going to break the interface until I noticed
the current implementation uses the string literal \t.
2022-02-08 09:51:35 -05:00
Graham Christensen c30f084f32 eval_started event: change interface to traceID\tjobsetID
I was not going to break the interface until I noticed
the current implementation uses the string literal \t.
2022-02-08 09:51:35 -05:00
Graham Christensen 8a18326f2b Sort notification classes / events 2022-02-07 16:08:27 -05:00
Graham Christensen d8b56f022d RunCommand: print a warning if the hook isn't run because the project / jobset doens't have it enabled 2022-02-01 10:58:54 -05:00
Graham Christensen 3aa2393091 Jobsets: add a supportsDynamicRunCommand which also checks the project's dynamic runcommand support 2022-02-01 10:58:54 -05:00
Graham Christensen daa6864a58 Project result: add a supportsDynamicRunCommand helper 2022-02-01 10:58:54 -05:00
Graham Christensen bc1630bd27 fixup! RunCommand: Add a WIP execution of dynamic commands 2022-02-01 10:58:54 -05:00
Graham Christensen 8a96f07f58 Project: enable enabling dynamic runcommand per project 2022-02-01 10:58:54 -05:00
Graham Christensen 1affb1cfb1 jobset API: expose and check the enable_dynamic_run_command 2022-02-01 10:58:54 -05:00
Graham Christensen 726ea80e99 HTTP/Jobset: support setting / reading enable_dynamic_run_command 2022-02-01 10:58:54 -05:00
Graham Christensen 1802bd0113 Declarative Jobs: add support for the enable_dynamic_run_command flag 2022-02-01 10:58:54 -05:00
Graham Christensen 0810f5debc finish making the dynamic hooks only run on project & jobset agreement 2022-02-01 10:58:54 -05:00
Graham Christensen aef11685a0 regenerate schema files after adding the flag to the projects 2022-02-01 10:58:54 -05:00
Graham Christensen 3cce0c5ef6 Only run dynamic runcommand hooks if the jobset enables them 2022-02-01 10:57:30 -05:00
Graham Christensen 97a1d2d1d4 Jobsets: add enable_dynamic_run_command 2022-02-01 10:57:30 -05:00
Graham Christensen 216d8bee35 DynamicRunCommand: don't run if the build failed 2022-02-01 10:57:30 -05:00
Graham Christensen 1a30a0c2f1 Dynamic RunCommand: validate that the job's out exists, is a file (or points to a file) which is executable. 2022-02-01 10:57:30 -05:00
Graham Christensen e7f68045f4 DynamicRunCommand: pull out the function determining if a build is
eligible for execution under dynamic run commands.
2022-02-01 10:57:30 -05:00
Graham Christensen e56c49333f RunCommand: Add a WIP execution of dynamic commands
This in-progress feature will run a dynamically generated set of
buildFinished hooks, which must be nested under the `runCommandHook.*`
attribute set. This implementation is not very good, with some to-dos:

1. Only run if the build succeeded
2. Verify the output is named $out and that it is an executable file
   (or a symlink to a file)
3. Require the jobset itself have a flag enabling the feature, since
   this feature can be a bit dangerous if various people of different
   trust levels can create the jobs.
2022-02-01 10:57:30 -05:00
Graham Christensen ea311a0eb4 RunCommand: enable the plugin if dynamicruncommand is set 2022-02-01 10:57:30 -05:00
Graham Christensen 85b842e0ac
Merge pull request #1137 from DeterminateSystems/runcommand-logs
Store and display the output of RunCommands
2022-01-31 16:26:31 -05:00
Cole Helbling 8c67e32480 RunCommand: ensure we reset the umask 2022-01-31 12:55:36 -08:00
Cole Helbling 61189ecca9 Helper/Nix: constructRunCommandLogPath: verify uuid is valid
This shouldn't be possible normally, but it is possible to:

    $db->resultset('RunCommandLogs')->new({ uuid => "../etc/passwd" });

if you have access to the `$db`.
2022-01-31 08:58:33 -08:00
Cole Helbling e381751564 Helper/Nix: constructRunCommandLogPath: return undef in case of an error
This allows us to give a web request to an invalid UUID a 404.
2022-01-31 08:58:33 -08:00
Cole Helbling 8eab7b8543 Helper/Nix: constructRunCommandLogPath: take RunCommandLog as input
This way we ensure that it actually exists in the database, rather than
blindly trusting user-generated input.
2022-01-31 08:58:33 -08:00
Cole Helbling 3594ba942a Controller/Build: use showLog in view_runcommandlog
This also adds the `runcommandlog` object to the stash so that we can
access its uuid as well as command run in order to display more useful
and specific information on the webpage.
2022-01-31 08:58:33 -08:00
Cole Helbling 1d0076408b Controller/Build: pass log_uri to showLog in place of drvPath
This way, we can reuse the `showLog` sub for other things, such as
`view_runcommandlog` (which doesn't have a drvPath attached).
2022-01-31 08:58:33 -08:00
Cole Helbling ff390e89a6 Controller/Build: remove unused parameter from showLog 2022-01-31 08:58:33 -08:00
Cole Helbling fc3cf4ecb2 RunCommandLogs: identify and access via uuid
Using a sha1 of the command combined with the build ID is not a
particularly good or unique identifier:

* A build could fail, be restarted, and then succeed -- assuming no
configuration changes, the sha1 hash of the command as well as the build
ID will be the same. This would lead to an overwritten log file.

* Allowing user input to influence filenames is not the best of ideas.
2022-01-31 08:58:33 -08:00