Commit graph

558 commits

Author SHA1 Message Date
Eelco Dolstra 77e37c1ea3 Don't add a nix-build build product unless $out is a directory 2013-05-28 18:43:05 +02:00
Eelco Dolstra 1f1615e80b Support revision control systems via plugins 2013-05-25 15:36:58 -04:00
Eelco Dolstra 5e5b344527 Make latest/latest-for more efficient
Getting all matching rows from the database is expensive.
2013-05-23 12:18:38 -04:00
Eelco Dolstra 7b5e92868b Remove unused Jobs columns
Actually, we can *almost* get rid of the Jobs table entirely...
2013-05-23 12:12:19 -04:00
Eelco Dolstra 57b2bb0674 Let Builds.timestamp refer to the time the build was added
Previously, for scheduled builds, "timestamp" contained the time the
build was added to the queue, while for finished builds, it was the
time the build finished.  Now it's always the former.
2013-05-23 10:45:49 -04:00
Eelco Dolstra e09622eb29 Include # of failed builds in Subject 2013-05-21 18:30:05 -04:00
Eelco Dolstra 444b03bad2 Omit the system type if it's part of the job name 2013-05-15 11:43:06 +02:00
Eelco Dolstra ac6a152ec5 Improve email notifications status messages
Add "Failed with output", and shorten "Failed with non-zero exit code"
to "Failed".
2013-05-15 11:31:37 +02:00
Eelco Dolstra 184d23dc41 Remove debug line 2013-05-09 15:10:30 +02:00
Eelco Dolstra 26b3da5d13 Fix tests
http://hydra.nixos.org/build/4902041
2013-05-09 14:29:04 +02:00
Eelco Dolstra de89c566f8 Use Template::Toolkit to generate email messages
Also, make the messages much shorter by not including build details.
2013-05-09 14:23:15 +02:00
Eelco Dolstra b378d94087 If multiple dependent builds fail, send a single email 2013-05-09 11:39:58 +02:00
Eelco Dolstra 44b8d6f449 Use OO-style plugins
This is mostly so we don't have to pass around common parameters like
"db" and "config", and we don't have to check for the existence of
methods.

A plugin now looks like this:

  package Hydra::Plugin::TwitterNotification;

  use parent 'Hydra::Plugin';

  sub buildFinished {
      my ($self, $build, $dependents) = @_;
      print STDERR "tweeting about build ", $build->id, "\n";
      # Send tweet...
      # Hydra database is $self->{db}.
  }
2013-05-08 18:35:24 +02:00
Eelco Dolstra 1d8bb0764b Add a plugin mechanism
You can now add plugins to Hydra by writing a module called
Hydra::Plugin::<whatever> and putting it in Perl's search path.  The
only plugin operation currently supported in buildFinished, called
when hydra-build has finished doing a build.

For instance, a Twitter notification plugin would look like this:

  package Hydra::Plugin::TwitterNotification;

  sub buildFinished {
      my ($self, $db, $config, $build, $dependents) = @_;
      print STDERR "tweeting about build ", $build->id, "\n";
      # send tweet...
  }

  1;
2013-05-08 17:30:30 +02:00
Eelco Dolstra 507e5bb190 Drop unused "disabled" columns 2013-05-03 16:39:17 +02:00
Eelco Dolstra e96916533f Allow a per-jobset check interval
This allows checking a jobset (say) at most once a day.  It's also
possible to disable polling by setting the interval to 0.  This is
useful for jobsets that use push notification or are manually
evaluated.
2013-05-02 17:55:47 +02:00
Shea Levy b8ea428bb1 Allow manually scheduling disabled jobsets for evaluation
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-05-02 11:21:43 -04:00
Eelco Dolstra 3003bb1c99 Don't fall thru to Hydra::View::NARInfo
This caused exceptions like:

  Caught exception in Hydra::Controller::Build->view_build "writing to file: Broken pipe at /nix/store/ihdb3widsq1dk7sbl5vqjxfcxb5ypad4-hydra-0.1pre1297-8158093/libexec/hydra/lib/Hydra/Controller/Build.pm line 59."

because the connection to the Nix daemon would be terminated due to a
protocol violation (calling queryPathInfo with an empty string).
2013-04-30 16:23:19 +02:00
Eelco Dolstra 45a1bb9926 Remove unnecessary parentheses in SQL query
These make SQLite 3.7.16.2 crash.

http://hydra.nixos.org/build/4815315
2013-04-29 21:38:57 +02:00
Eelco Dolstra 71eaa115f8 Ellipsize long jobset/job names to prevent bootstrap navbar breakage 2013-04-26 17:51:05 +02:00
Rob Vermaas fd7acd9bbf When searching a path / drv path, use separate queries for both options to prevent seq. scan. 2013-04-26 08:51:49 -04:00
Eelco Dolstra 591aee0793 getPrimaryBuildsForView: Return all view results
Returning only the first 20 results can cause NixOS/Nixpkgs channel
generation to fail, if the first 20 view results correspond to
evaluations that haven't finished yet.  Then URLs like
/view/nixos/tested/latest-finished will return 500 rather than the
latest finished view.
2013-04-26 13:21:41 +02:00
Rob Vermaas bfc1b0e76a Allow searching for store and drv paths. Only does a complete match to reduce load on database. 2013-04-25 09:57:30 -04:00
Eelco Dolstra 2c7a479994 Minor tweak 2013-04-23 15:39:05 +02:00
Eelco Dolstra 72a44cfcaa Make negative narinfo responses much smaller 2013-04-23 15:33:58 +02:00
Eelco Dolstra 7528a9c7e7 Machine status: Don't show removed machines anymore
This requires a sequential scan on the BuildSteps table, which by now
takes > 8s.
2013-04-23 15:20:24 +02:00
Eelco Dolstra 9f027b22b1 Allow users to reproduce builds on their own systems
You can now do:

  bash <(curl http://hydra-server/build/1238757/reproduce)

to download and execute a script that reproduces a Hydra build
locally.  This script fetches all inputs (e.g. Git repositories) and
then invokes nix-build.

The downloaded sources are stored in /tmp/build-<buildid> and reused
between invocations of the script.

Any additional command line options are passed to nix-build.  So

  bash <(curl http://hydra-server/build/1238757/reproduce) --run-env

will drop you in a shell where you can interactively hack on the
build, e.g.

  $ source $stdenv/setup
  $ set +e
  $ unpackPhase
  $ cd $sourceRoot
  $ configurePhase
  $ emacs foo.c &
  $ make

and so on.
2013-04-04 17:30:07 +02:00
Eelco Dolstra 48bb9c48ed Security: Prevent shell injection attack in rendering build contents 2013-04-03 00:13:03 +02:00
Eelco Dolstra e7926e046b Security: Improve checking of build products
Build product paths cannot reference locations outside of the Nix
store.  We previously disallowed paths from being symlinks, but this
didn't take into account that parent path elements can be symlinks as
well.  So a build product /nix/store/bla.../foo/passwd, with
/nix/store/bla.../foo being a symlink to /etc, would still work.

So now we check all paths encountered during path resolution.
Symlinks are allowed again so long as they point to the Nix store.
2013-04-02 23:32:04 +02:00
Hydra 16dbec332d Postgresql 8.x needs as for column. 2013-04-01 20:18:00 -04:00
Eelco Dolstra ee5b762919 Disable account creation for now 2013-03-28 11:56:12 +01:00
Rob en Lizi Vermaas 52a9f1a14e Use get_column for 'as' column (max_stoptime). 2013-03-24 18:50:17 +01:00
Sander van der Burg 03189bf62b Implemented quoted strings support in hydra-build-products to allow file names with spaces + testcase 2013-03-20 18:05:21 +01:00
Shea Levy 94382bd8f0 Admin controller: Use PathPart, not Path
Chaining paths only works properly when PathPart is used. Before this
fix, the affected URIs bypassed the top-level 'admin' sub.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-03-14 19:22:02 -04:00
Eelco Dolstra c96e427ec0 Log segfaults from the evaluator 2013-03-14 17:39:24 +01:00
Eelco Dolstra fcd511c4de Revert "Remove now-unused SystemTypes table"
This reverts commit 71d020735b.
Unfortunately there are still some cases where we need to set Hydra's
concurrency separately.  (Ideally, Hydra would start *all* queued
builds in parallel and let Nix take care of everything...)
2013-03-05 17:42:16 +01:00
Shea Levy 71d020735b Remove now-unused SystemTypes table 2013-03-04 17:46:37 -05:00
Shea Levy 233e485a55 Remove the BuildMachines and BuildMachinesSystemTypes tables 2013-03-04 16:32:16 -05:00
Shea Levy 232170e301 Remove all entry points to modify machines 2013-03-04 16:05:50 -05:00
Shea Levy 170c7c98d0 machines: Read /etc/nix.machines instead of using the BuildMachines table 2013-03-04 16:03:04 -05:00
Shea Levy f9426f365b machine-status: Read /etc/nix.machines instead of using the BuildMachines table 2013-03-04 15:37:20 -05:00
Eelco Dolstra a77161e40a Allow users to edit their own settings
Also, don't use the flash anymore for going back to the referer.
2013-03-04 15:25:23 +01:00
Eelco Dolstra e8cbcb50ac Add user registration 2013-02-27 18:33:47 +01:00
Eelco Dolstra d1c2771ab8 Ensure that each eval is compared to the previous one from the same jobset
Also, reduce the number of database hits.
2013-02-26 19:08:39 +01:00
Eelco Dolstra 56a62c3d88 Make jobset deletion consistent with project deletion 2013-02-26 17:36:49 +01:00
Eelco Dolstra d596b58991 "Evaluate" command: push the jobset to the front of the eval queue
Don't let hydra-server call hydra-evaluate.  That will probably just
timeout, get killed, etc.
2013-02-26 16:56:19 +01:00
Eelco Dolstra eea3846754 After editing a jobset, push it to the front of the evaluation queue 2013-02-26 16:10:36 +01:00
Eelco Dolstra 4b70be6753 Fix same system inputs 2013-02-26 15:51:56 +01:00
Eelco Dolstra 1e338e7958 api/push: Don't barf on non-existent jobsets 2013-02-26 01:46:19 +01:00
Eelco Dolstra f501648cd2 Add simpler push support for GitHub
You can use the URL

  http://<hydra-server>/api/push-github

as GitHub's WebHook URL.  Hydra will automatically trigger an
evaluation of all affected jobsets.
2013-02-26 01:14:50 +01:00
Eelco Dolstra 11d87c6f82 Use query_params instead of params
Otherwise crazy things happen with POST requests from GitHub.
2013-02-26 00:38:18 +01:00
Eelco Dolstra 5e6896b2d9 Turn prepared statements back on
We once turned these off (in commit
abe71a767b) because they caused the
PostgreSQL query optimizer to use very suboptimal plans.  However,
PostgreSQL 9.2 has supposedly fixed this:

  http://www.postgresql.org/docs/9.2/static/release-9-2.html

So let's try again.
2013-02-25 21:20:52 +01:00
Eelco Dolstra 42d2015357 Support push notification of repository changes
External machines can now notify Hydra that it should check a
repository by sending a GET or PUSH request to /api/push, providing a
list of jobsets to be checked and/or a list of repository URLs.  In
the latter case, all jobsets that have any of the specified
repositories as an input will be checked.

For instance, you can configure GitHub or BitBucket to send a request
to the URL

  http://hydra.example.org/api/push?repos=git://github.com/NixOS/nixpkgs.git

to trigger evaluation of all jobsets that have
git://github.com/NixOS/nixpkgs.git as an input, or to the URL

  http://hydra.example.org/api/push?jobsets=patchelf:trunk,nixpkgs:trunk

to trigger evaluation of just the specified jobsets.
2013-02-25 21:10:32 +01:00
Eelco Dolstra 65db523e46 Fix duplicate function name 2013-02-25 19:01:43 +01:00
Eelco Dolstra 6658419f69 Disallow build products that are symlinks
Otherwise you can do

  ln -s /etc/passwd $out/foo
  echo "file misc $out/foo" >> $out/nix-support/hydra-build-products

and get Hydra to serve its /etc/passwd file.
2013-02-23 16:28:44 +01:00
Eelco Dolstra aa7ddeb8e9 Use hashFile instead of nix-hash
Calling a shell command is a security hole if $path contains special
characters (e.g. "${foo}").  Observed in
http://hydra.nixos.org/build/4041321.
2013-02-23 16:09:34 +01:00
Eelco Dolstra 08dc6650e1 Search: Limit the number of results 2013-02-22 16:41:42 +01:00
Eelco Dolstra 4d4ea8c754 Search: Mark disabled projects/jobsets/jobs 2013-02-22 16:21:50 +01:00
Eelco Dolstra f3f6e8a4e3 Search: Don't return hidden projects/jobsets 2013-02-22 15:56:29 +01:00
Eelco Dolstra 07daff32b8 Add a search feature
This allows searching for projects, jobsets or jobs by name or
description.
2013-02-22 15:45:10 +01:00
Eelco Dolstra 4343781181 Clear failed builds etc.: Redirect back to the referrer
Would be better to do this AJAX style though.
2013-02-22 14:31:43 +01:00
Eelco Dolstra 629fe6f998 Jobset page: Load the jobs and status tabs on demand
This makes the jobset page much smaller and faster.  (E.g. for
nixpkgs:trunk, this page was ~2.5 MB.)
2013-02-22 14:29:12 +01:00
Eelco Dolstra f2de374f28 Set a 500 HTTP status on errors 2013-02-22 14:27:38 +01:00
Eelco Dolstra 098368c3e3 Job page: Remove "Latest builds (latest evaluation)"
It was kind of confusing since there are two other "latest builds"
tabs/sections.
2013-02-22 12:53:10 +01:00
Eelco Dolstra 6dbb948f1f Job page: show queued as well as running builds 2013-02-22 12:52:11 +01:00
Eelco Dolstra d08a391b43 Jobset page: Show last evaluation time 2013-02-22 11:37:35 +01:00
Eelco Dolstra c9ac7746b2 Allow showing all evaluations that contain a given build 2013-02-21 18:49:57 +01:00
Eelco Dolstra 6afe528570 Remove the related builds tab
You can now just click on the evaluation link on the first tab to see
all builds in the same jobset.  This also makes rendering build pages
quite a bit faster for jobsets like Nixpkgs.
2013-02-21 18:43:16 +01:00
Eelco Dolstra c6fe9f00bf Also show number of evaluations of which a build is a part 2013-02-21 18:42:18 +01:00
Eelco Dolstra edbe531ccc On build pages, provide a link to the build's first eval 2013-02-21 18:34:34 +01:00
Eelco Dolstra d8370142c2 Remove garbage from Users.pm 2013-02-21 18:20:55 +01:00
Eelco Dolstra ba68826860 Add a page to show the latest evaluations for the entire server 2013-02-21 17:27:17 +01:00
Eelco Dolstra 767cab6cd2 Correctly redirect to the referring page when logging in 2013-02-21 14:38:18 +01:00
Eelco Dolstra fec45d1f07 Remove dead code 2013-02-21 13:45:11 +01:00
Eelco Dolstra dd5eb51a84 Put job status on a separate tab
Also, show the "last checked time" on the initial tab.
2013-02-21 13:30:14 +01:00
Eelco Dolstra 8db83d2017 Turn hiding/unhiding a jobset into a checkbox 2013-02-21 02:36:51 +01:00
Eelco Dolstra 547e41263d Split viewing and editing a jobset 2013-02-21 02:33:57 +01:00
Eelco Dolstra 9c7b416f8b Make the hide/unhide actions a checkbox in the project settings
Also use proper bootstrap layout for the project settings form.
2013-02-21 01:45:39 +01:00
Eelco Dolstra cea17fd57e Show releases in a tab on the project page 2013-02-21 01:23:42 +01:00
Eelco Dolstra d9f6e662d8 Split viewing and editing a project 2013-02-21 01:12:57 +01:00
Eelco Dolstra 85e3f69cdd Remove debug statement 2013-02-20 18:22:06 +01:00
Eelco Dolstra 36a1015444 Show the runtime/build dependencies in the build page 2013-02-20 17:58:27 +01:00
Eelco Dolstra 7da2398f27 Allow non-admin users to see the machine status page 2013-02-20 16:40:09 +01:00
Eelco Dolstra 82daf4f8ca Show dependencies as a tree rather than a list 2013-02-14 15:53:34 +01:00
Eelco Dolstra 4e67665b7f Builds page: Remove redundant tab titles 2013-02-14 13:23:54 +01:00
Eelco Dolstra 1160e17bfc Add missing files
Doh!
2013-02-14 11:53:09 +01:00
Eelco Dolstra 2b3c72efeb Use getMainOutput 2013-02-13 18:36:01 +01:00
Eelco Dolstra eb9d0c8992 Security: Also check paths in the web server 2013-02-13 18:34:33 +01:00
Eelco Dolstra cd83b8e3aa Security: Ensure that a build product refers to the Nix store 2013-02-13 18:26:00 +01:00
Eelco Dolstra 10882a1ffd Add multiple output support
This requires turning the outPath columns in the Builds and BuildSteps
tables into separate tables, and so requires a schema upgrade.
2013-02-13 16:49:28 +00:00
Eelco Dolstra e555c7bb36 Automatically chomp all [%...%] directives
This makes the use of [%-...-%] unnecessary.
2013-02-05 14:00:09 +01:00
Eelco Dolstra 58dd49e645 Fix handling of IPC::Run::run exit status
Turns out that the exit status is returned in $?, not as the return
value of run().  So our error checking was completely bogus.
2013-01-23 15:56:28 +01:00
Eelco Dolstra e0de5de91d Fix some warnings 2013-01-23 12:41:57 +00:00
Eelco Dolstra 30e5185acf Remove the logfile and logSize columns from the database
It's pointless to store these, since Nix knows where the logs are.
Also handle (in fact require) Nix's new log storage scheme.  Also some
cleanups in the build page.
2013-01-22 22:48:02 +01:00
Eelco Dolstra 67aefde62c Remove trailing whitespace 2013-01-22 14:41:02 +01:00
Eelco Dolstra aa28ffe3a8 Die tabs die 2013-01-22 14:09:37 +01:00
Eelco Dolstra 486b6037a9 Use $c->config 2013-01-22 14:06:12 +01:00
Eelco Dolstra f188fe5683 hydra-evaluator: Don't require $HYDRA_CONFIG 2013-01-22 13:19:28 +01:00
Rob Vermaas d4e49773c5 Allow dashes in attrNames 2013-01-11 12:16:21 +01:00
Eelco Dolstra 64399991a7 Give Hydra binary caches a low priority
We only want to hit a Hydra binary cache if static caches don't
provide a binary.
2012-11-06 17:13:17 +01:00