Commit graph

954 commits

Author SHA1 Message Date
Eelco Dolstra 7b5b434014 Fix getResponsibleAuthors 2014-11-19 15:22:30 +01:00
Eelco Dolstra 0d28e9372b Remove dead code 2014-11-19 14:59:36 +01:00
Eelco Dolstra 2d46741840 Fix message 2014-11-19 14:52:42 +01:00
Eelco Dolstra 8523130ebb Use Email::MIME instead of Email::Simple
Email::Simple cannot handle non-ASCII characters.

Fixes #191.
2014-11-19 14:45:46 +01:00
Eelco Dolstra 41bc918382 Fix robots.txt 2014-11-18 14:55:52 +01:00
Rob Vermaas d5db1d3bc1 Revert "Make evaluation fail with proper error when a input of type build is not available."
This reverts commit a1b5020562.
2014-11-18 11:13:34 +01:00
Rob Vermaas fd36d79581 Fix Changes section on build page. Now shows the changes of the first evaluation a build was part of compared to the previous one. Also separated previous builds from the changes section, as that was confusing sometimes. 2014-11-18 11:00:28 +01:00
Rob Vermaas a1b5020562 Make evaluation fail with proper error when a input of type build is not available. 2014-11-18 11:00:28 +01:00
Eelco Dolstra c503876a7e Don't cache nix-cache-info very long 2014-11-06 15:30:24 +01:00
Eelco Dolstra c250407f3c Set Expires headers for Hydra's binary cache
This allows caching by reverse proxies.
2014-11-06 14:31:48 +01:00
Rob Vermaas 55eb55478d Allow HipChat notification on all builds, in stead of status changes only. 2014-10-21 07:50:12 +00:00
Eelco Dolstra 1b12daa032 Move the build time chart to the job page 2014-10-07 12:59:09 +02:00
Eelco Dolstra 0bb027e633 Move the store path size chart to the job page
The history is a property of the job, not individual builds.
2014-10-07 11:53:17 +02:00
Eelco Dolstra 5f474b252c Add a chart to the job pages showing the closure size over time 2014-10-07 11:23:15 +02:00
Eelco Dolstra 3687c9c30b Validate build IDs 2014-10-01 17:05:39 +02:00
Eelco Dolstra 3f51ac6603 Partially revert 1c20cfdf24
Not clear why update-dbix messed up the Jobsets and Projects
classes. This will probably happen again in the future...
2014-10-01 15:28:18 +02:00
Eelco Dolstra 9b38b5f134 Remove the longDescription field
It's not useful and takes up a lot of space.
2014-09-30 15:44:09 +02:00
Eelco Dolstra f4acc9a522 Create Builds with iscurrent set
This should eliminate a subsequent update.
2014-09-30 15:44:09 +02:00
Eelco Dolstra 5b4de2dee6 hydra-evaluator: Reduce verbosity 2014-09-30 15:44:08 +02:00
Eelco Dolstra 09a96c642a hydra-eval-jobs: Use JSON instead of XML
XML::Simple is pretty slow - reading the output for the Nixpkgs jobset
takes half a minute or so. JSON is pretty much instantaneous.
2014-09-30 15:44:08 +02:00
Eelco Dolstra 1c20cfdf24 Drop the errorMsg column in the Jobs table
We're not using it anywhere.
2014-09-29 19:46:11 +02:00
Eelco Dolstra 2054759fdf Status page: Show running builds rather than build steps
The active build steps list was redundant, because the "Machine
status" page also shows them. So instead show active builds.
2014-09-25 20:17:33 +02:00
Eelco Dolstra 23a9fea26c Remove the ability to add multiple jobset input alternatives
Now each jobset input has exactly one value. (Actually, adding
multiple values was already broken, so all this does is clean up the
UI...)
2014-09-25 16:43:17 +02:00
Eelco Dolstra 01f4037d6f hydra-eval-jobs: Don't keep track of used inputs
We no longer store this in the database, so it's not necessary for
hydra-eval-jobs to do this.
2014-09-25 13:38:43 +02:00
Eelco Dolstra 6284fd540d Disallow multiple jobs with the same name
This has been deprecated since a8db329839.

Issue #60.
2014-09-24 18:12:59 +02:00
Eelco Dolstra 748c3409b4 Don't maintain BuildInputs anymore
We don't need to record inputs per build anymore because we have
JobsetEvalInputs now.
2014-09-06 19:06:07 +02:00
Eelco Dolstra d6b61f4925 More charset hackery 2014-08-23 16:39:20 +02:00
Sander van der Burg ede4a10353 Compose release name for an evaluation if none has been defined 2014-08-23 16:20:58 +02:00
Eelco Dolstra 319e053368 Hopefully fix UTF-8 chars in Hipchat notification messages 2014-08-18 14:01:39 +02:00
Eelco Dolstra 28b528c1ab Ensure proper escaping of /build/.../contents pages 2014-08-17 23:43:22 +02:00
Eelco Dolstra f50066ab6f Fix encoding of Hydra logs 2014-08-17 23:43:01 +02:00
Eelco Dolstra 42b23133e8 Fix UTF-8 handling of log files 2014-08-13 18:53:29 +02:00
Eelco Dolstra a2b27c7cf2 Preserve whitespace in string inputs 2014-08-13 17:25:08 +02:00
Eelco Dolstra 9334b84d6f Fix registerRoot
Fixes #175.
2014-08-13 16:29:00 +02:00
Eelco Dolstra 69e3aa0438 Write Hydra roots as regular files instead of symlinks
Note that this requires at least NixOS/Nix@1c208f2b7e.
2014-08-01 17:24:55 +02:00
Ronny Esterluss 7351752066 fixed string comparison 2014-07-31 22:18:28 +02:00
Tino Breddin 5ebc2e5419 only checkout the target branch when topgit is used 2014-07-31 22:18:28 +02:00
Ronny Esterluss 55aaf807eb added clean command 2014-07-31 21:53:25 +02:00
Ronny Esterluss ee840be313 forcing branch switch 2014-07-31 21:53:25 +02:00
Eelco Dolstra 6b88be040e Make restartBuilds faster 2014-07-18 00:03:26 +02:00
Rob Vermaas 268f273a3e Add value and dependency fields to eval json output. 2014-05-08 17:08:31 +02:00
Eelco Dolstra da73eb5f8a Include dependency column in JSON output 2014-05-08 16:36:15 +02:00
Rob Vermaas 1c92d760fa Merge pull request #161 from thoughtpolice/coverity
hydra: add Coverity Scan plugin
2014-05-03 17:46:35 +02:00
Austin Seipp 4166974657 hydra: add Coverity Scan plugin
This adds a Hydra plugin for users to submit their open source projects
to the Coverity Scan system for analysis.

First, add a <coverityscan> section to your Hydra config, including the
access token, project name, and email, and a regex specifying jobs to
upload:

    <coverityscan>
      project = testrix
      jobs    = foobar:.*:coverity.*
      email   = aseipp@pobox.com
      token   = ${builtins.readFile ./coverity-token}
    </coverityscan>

This will upload the scan results for any job whose name matches
'coverity.*' in any jobset in the Hydra 'foobar' project, for the
Coverity Scan project named 'testrix'.

Note that one upload will occur per job matched by the regular
expression - so be careful with how many builds you upload.

The jobs which are matched by the jobs specification must have a file in
their output path of the form:

  $out/tarballs/...-cov-int.(xz|lzma|zip|bz2|tgz)

The file must have the 'cov-int' directory produced by `cov-build` in
the root.

(You can also output something into
$out/nix-support/hydra-build-products for the Hydra UI.)

This file will be found in the store, and uploaded to the service
directly using your access credentials. Note the exact extension: don't
use .tar.xz, only use .xz specifically.

Signed-off-by: Austin Seipp <aseipp@pobox.com>
2014-05-02 14:15:25 -05:00
Eelco Dolstra 1306291c6c Handle utf-8 properly
Fixes errors like:

  Caught exception in engine "Wide character in syswrite at /nix/store/498lwsrn5kkdh1q8kn3vcpd3457w6m7a-hydra-perl-deps/lib/perl5/site_perl/5.16.3/Starman/Server.pm line 547."

Note that these errors didn't happen if the database encoding was set
to SQL_ASCII (which was the case for hydra.nixos.org, explaining why
it didn't get these errors). However, now the encoding must be
UTF8. To change it, do:

  update pg_database set encoding = pg_char_to_encoding('UTF8') where datname = 'hydra';
2014-05-01 16:33:25 +02:00
Eelco Dolstra 67c79a16a8 Remove unnecessary join against releases table
This gets rid of the warning:

  DBIx::Class::Storage::DBI::select_single(): Query returned more than one row.  SQL that returns multiple rows is DEPRECATED for ->find and ->single at /home/eelco/Dev/hydra/src/script/../lib/Hydra/Controller/Project.pm line 15
2014-05-01 13:03:34 +02:00
Eelco Dolstra 6c6ce7a781 Fix cloning jobsets 2014-04-30 09:26:53 +02:00
Eelco Dolstra 4d9edd0ca8 Also redirect /job requests that refer to a renamed jobset 2014-04-24 11:05:16 +02:00
Eelco Dolstra 23416f9960 Do a permanent redirect 2014-04-24 10:52:46 +02:00
Eelco Dolstra affec8881d Remove support for views
They're replaced by aggregates, which are declarative, faster, and
have a better interface.
2014-04-23 23:22:44 +02:00
Eelco Dolstra 5e0d10a392 When renaming a jobset, add a redirect from the old name 2014-04-23 23:15:22 +02:00
Eelco Dolstra be63c50560 Show whether a job still exists
In the dashboard and on the job page, indicate whether the job appears
in the latest jobset eval. That way, the user gets some indication if
a job has accidentally disappeared (e.g. due to an evaluation error).
2014-04-08 18:50:37 +02:00
Ludovic Courtès ccdb6f2b23 Honor $NIX_STORE_DIR. 2014-02-28 14:09:12 +01:00
Your Name f409d2b4f6 Allow notify option to be set for hipchat notification. (https://www.hipchat.com/docs/api/method/rooms/message) 2014-02-28 09:27:33 +00:00
Shea Levy b29d7b1091 Only email responsible authors when the build has changed 2014-02-26 11:05:46 -05:00
Eelco Dolstra 8dcef7702f Add jobset/job menus on buid log pages 2014-02-26 11:49:28 +01:00
Eelco Dolstra 66b8c1a9e0 Return 410 Gone (rather than 500) if an output is no longer available 2014-02-26 11:38:02 +01:00
Rob Vermaas 6327edd63f Add a convenient way to get logs of a path/drv. Requested by phreedom. 2014-02-19 10:21:59 +00:00
Hydra d4d9896f9f Re-add the history tab (store size and build time), now as lazy-loaded tab on build page. 2014-01-28 11:55:19 +01:00
Rob Vermaas ac177d0984 Add nixexprpath/nixexprinput to GET of jobset (API). 2014-01-23 12:44:02 +01:00
Eelco Dolstra 492fe121ea Jobset eval page: Fix handling of aborted/cancelled builds
The fix in 85d51074 was not quite right.
2014-01-15 16:06:47 +01:00
Eelco Dolstra cad2c07a35 Fix broken redirects 2014-01-14 18:17:33 +01:00
Eelco Dolstra 6572869596 clear-queue-non-current: Work around PostgreSQL query optimizer stupidity 2014-01-14 18:14:10 +01:00
Rob Vermaas bd7b6fc401 Allow Hydra to run as a private instance by requiring a login.
Use the following in your hydra.conf to make your instance a
private Hydra instance (public is the default):

  private 1

Currently, this will not allow you to use the API, channels
and the binary cache when running in private mode. We will add
solutions for these functionalities later.
2014-01-10 11:04:28 +01:00
Rob Vermaas 3e1a98668f Allow configuring a set of domains to allow logins from Persona.
E.g. add the following to only allow Persona logins from email.com and gmail.com email addresses.

  persona_allowed_domains email.com,gmail.com
2014-01-09 13:31:02 +01:00
Eelco Dolstra a598fe7e81 Allow Hydra's binary cache to be signed
This requires adding the following to hydra.conf:

  binary_cache_key_name = <key-name>
  binary_cache_private_key_file = <path-to-private-key>

e.g.

  binary_cache_key_name = hydra.nixos.org-1
  binary_cache_private_key_file = /home/hydra/cache-key.sec
2014-01-08 15:19:17 +01:00
Shea Levy de26b55afe User.pm: Handle params from JSON properly
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-12-12 09:38:57 -05:00
Rob Vermaas f25f8dbfa3 Higher timeout for prefetching bazaar inputs. 2013-12-11 17:34:45 +01:00
Eelco Dolstra dd4e57fb0c Allow passing a specific build as an input
Fixes #62.
2013-11-11 21:36:26 +00:00
Eelco Dolstra 8f104396ec Support passing a jobset evaluation as an input
All successful, non-garbage-collected builds in the evaluation are
passed in a attribute set.  So if you declare a Hydra input named
‘foo’ of type ‘eval’, you get a set with members ‘foo.<jobname>’.  For
instance, if you passed a Nixpkgs eval as an input named ‘nixpkgs’,
then you could get the Firefox build for x86_64-linux as
‘nixpkgs.firefox.x86_64-linux’.

Inputs of type ‘eval’ can be specified in three ways:

* As the number of the evaluation.

* As a jobset identifier (‘<project>:<jobset>’), which will yield the
  latest finished evaluation of that jobset.  Note that there is no
  guarantee that any job in that evaluation has succeeded, so it might
  not be very useful.

* As a job identifier (‘<project>:<jobset>:<job>’), which will yield
  the latest finished evaluation of that jobset in which <job>
  succeeded.  In conjunction with aggregate jobs, this allows you to
  make sure that the evaluation contains the desired builds.
2013-11-11 21:17:22 +00:00
Eelco Dolstra 4fb3d67f49 Make /login give a proper JSON response again 2013-11-06 16:12:07 +01:00
Eelco Dolstra 60e7e4fbe9 Remove obsolete/broken timeline feature 2013-11-06 15:20:30 +01:00
Eelco Dolstra 33f01da507 Fix hydra_logo setting 2013-11-06 15:10:26 +01:00
Eelco Dolstra 3c45992182 Make the logo configurable via hydra.conf 2013-11-06 14:40:30 +01:00
Eelco Dolstra b44baf3746 Add a flag to enable Persona support
It's disabled by default.  To enable, add the following to hydra.conf:

  enable_persona = 1
2013-11-06 14:38:36 +01:00
Eelco Dolstra 63100b0ee8 Merge branch 'master' into persona 2013-11-06 12:35:36 +01:00
Shea Levy f1c187198c Stupid capitalization scheme
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-11-05 12:39:55 -05:00
Shea Levy 2e1c07c5f7 Argh
ALWAYS RUN THE TESTS

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-11-05 12:31:36 -05:00
Eelco Dolstra ae85d2df23 Add a "My jobsets" tab to the dashboard 2013-11-05 16:05:29 +01:00
Shea Levy d5574230e4 typo
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-11-05 09:57:15 -05:00
Shea Levy 217ba725a3 Restore accidentally-removed fields to the Builds API
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-11-05 09:43:32 -05:00
Shea Levy 385ceaff4d Show buildinput and buildproduct information in the Builds API
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-11-05 09:42:05 -05:00
Eelco Dolstra 589271bd72 In the "My jobs" tab, ignore disabled projects/jobsets 2013-11-05 14:53:52 +01:00
Eelco Dolstra fc827a003a Be paranoid about the Persona email address 2013-11-05 14:40:40 +01:00
Eelco Dolstra b53bd868fb Add a "My jobs" tab to the dashboard
It shows all current jobs of which the user is a maintainer.
2013-11-05 14:35:49 +01:00
Eelco Dolstra 3c981bed02 Slight cleanup in the Persona sign in code 2013-11-05 14:10:20 +01:00
Eelco Dolstra bdd6e99fbd Clean up user editing 2013-11-05 14:02:04 +01:00
Eelco Dolstra bd927eba66 Make sign in a modal dialog box rather than a separate page 2013-11-05 13:13:02 +01:00
Eelco Dolstra 299ac07cbc Re-enable adding new users via the web interface
Also unify user creation and user editing.
2013-11-05 12:41:10 +01:00
Eelco Dolstra 36c961a376 Store the account type ("hydra" or "persona") explicitly in the database 2013-11-05 11:46:05 +01:00
Eelco Dolstra b54cfbf032 Merge branch 'master' into persona
Conflicts:
	src/lib/Hydra/Helper/CatalystUtils.pm
	src/root/layout.tt
	src/root/topbar.tt
	src/root/user.tt
2013-11-05 11:11:48 +01:00
Eelco Dolstra a04c117eb6 Revert "Remove wacky "sysbuild" filtering"
This reverts commit 2d7e106d29.

Unfortunately some jobsets still depend on this behaviour.  They could
probably do something like "assert system == input.system; ..." but
changing them all is undesirable.
2013-11-01 18:30:36 +01:00
Eelco Dolstra cbd56824a2 Use /etc/nix/machines 2013-10-30 11:21:23 +01:00
Eelco Dolstra a919b503fb Restore api/get-info 2013-10-29 19:53:23 +01:00
Shea Levy 5da3dcfd57 Make the outputs' outpaths available via the Build JSON API
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-29 14:41:48 -04:00
Shea Levy 4eef090a45 Flesh out the API a bit
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-29 13:54:11 -04:00
Shea Levy b51b8cddf4 Add in missing needed join
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-25 14:06:52 -04:00
Shea Levy 8e82ba0093 Get rid of old ToJSON code path
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-25 11:09:11 -04:00
Shea Levy a89096ec5e Clean up root controller
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-25 11:09:11 -04:00
Shea Levy 5d3df69a0b Cleanup build serialization
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-25 11:09:11 -04:00
Shea Levy e04477003e Cleanup eval serialization
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-25 11:09:11 -04:00
Shea Levy 917660442e Add eager fetching of relations and enable that for jobset->jobsetinputs
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-25 11:09:11 -04:00
Shea Levy ec6568f9b7 Fix jobset input handling in the API
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-25 11:09:06 -04:00
Shea Levy 04d8adaad3 Cleanup Jobset serialization
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-24 11:39:40 -04:00
Shea Levy 78dcd3e619 Cleanup Project model
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-24 11:39:40 -04:00
Shea Levy ee8275cfdb Start api cleanup with the User model
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-24 11:39:40 -04:00
Eelco Dolstra f551de21f7 Fix creating jobsets in a RESTful way 2013-10-24 17:35:58 +02:00
Eelco Dolstra c719bebcb4 Remove redundant newlines 2013-10-17 11:20:27 +02:00
Shea Levy 806af3fe68 Fix typos
I could've sworn I tested this...

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-16 08:57:48 -04:00
Shea Levy 0db950931a Merge remote-tracking branch 'upstream/who-broke-builds' into upstream-master
Include information about who changed the build status in notification
emails, and enable optional per-input notification of said committers.

Conflicts due to two branches modifying the database schema.

Signed-off-by: Shea Levy <shea@shealevy.com>

Conflicts:
	src/lib/Hydra/Schema/Jobsets.pm
	src/sql/upgrade-23.sql
2013-10-15 09:49:20 -04:00
Eelco Dolstra 2127d133cd Add a dashboard
Currently the dashboard allows users to get a quick overview of the
status of jobs they're interested in, but more will be added,
e.g. viewing all your jobsets or all jobs of which you're a
maintainer.
2013-10-14 20:07:26 +02:00
Eelco Dolstra 09b5679ee7 Clean up the authorization code a bit 2013-10-14 18:01:04 +02:00
Eelco Dolstra c4e39d4769 Add one-shot jobsets
There are jobsets that are evaluated only once, that is, after they've
been evaluated, they're disabled automatically.  This is primarily
useful for doing releases: for instance, doing an evaluation with
"officialRelease" set to "true" should be done only once.
2013-10-11 12:01:52 +02:00
Shea Levy 26470f1656 Check all inputs for blame but only email selected inputs
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-08 14:47:24 -04:00
Shea Levy 804617f075 Email responsible authors if requested
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-08 13:24:49 -04:00
Shea Levy f8b80c99c2 Include who-broke-the-build information in notification emails
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-08 13:24:49 -04:00
Shea Levy 272d9e235d Remove unused assignment
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-08 13:24:49 -04:00
Shea Levy 2c90857689 getResponsibleAuthors: Respect checkResponsible
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-08 13:24:49 -04:00
Shea Levy 58ad3b4b6c Enable setting checkresponsible in the edit jobset form
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-08 13:24:49 -04:00
Shea Levy 3e1f930928 Enable setting emailresponsible in the edit jobset form
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-08 13:24:49 -04:00
Shea Levy 3e4a4e3761 Propagate checkresponsible from JobsetInput to BuildInput
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-08 13:24:49 -04:00
Shea Levy 07157f8125 Update Schema classes
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-08 13:24:49 -04:00
Shea Levy 7913701d54 Factor a getResponsibleAuthors helper out of the HipChatNotification
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-08 13:24:49 -04:00
Eelco Dolstra 0ec03aa0f4 Handle builds with multiple outputs correctly in Hydra channels 2013-10-07 17:06:17 +02:00
Eelco Dolstra 5ccff14f6b In Hydra channels, show only packages matching the user's system type
Fixes NixOS/nix#169.
2013-10-07 14:53:27 +02:00
Eelco Dolstra 5294a0a8a0 Register restarted derivations as GC roots 2013-10-04 17:11:42 +02:00
Eelco Dolstra 052bab169d Add a jobset eval action to restart all aborted/cancelled builds 2013-10-04 17:01:47 +02:00
Eelco Dolstra 85d51074b9 Jobset eval page: Show all aborted builds under the "aborted jobs" tab
Previously some might be included under the "new jobs" tab
2013-10-04 16:36:22 +02:00
Eelco Dolstra aa49b128a8 Remove the "clone build" feature 2013-10-04 15:43:51 +02:00
Eelco Dolstra 7818bb75ed Add an action to cancel all builds in a jobset eval 2013-10-04 15:40:43 +02:00
Eelco Dolstra 3e54f0a6ab Fix an uninitialized value warning 2013-10-04 14:47:30 +02:00
Eelco Dolstra 550bf210fe Use more flash messages 2013-10-03 19:54:22 +02:00
Eelco Dolstra 720c3892a3 Use delete instead of delete_all
DBIC's delete_all method fetches all rows separately, which is slow.
2013-10-03 19:42:44 +02:00
Eelco Dolstra f32077b5e8 Simplify jobset cloning
We can just show the normal "edit jobset" page for the original jobset
and then do a PUT request to create a new jobset.

Also simplified updating the jobset inputs.  We can just delete all of
them and recreate them from the user parameters.  That's safe because
it's done in a transaction.
2013-10-03 19:28:25 +02:00
Eelco Dolstra 232f46c750 Use the REST API in the web interface for editing jobsets 2013-10-03 18:49:37 +02:00
Eelco Dolstra 851c3329d0 Implement DELETE for jobsets and use it in the web interface 2013-10-03 17:54:40 +02:00
Eelco Dolstra ae13dd3e67 Use the REST API in the web interface for editing projects 2013-10-03 17:23:41 +02:00
Eelco Dolstra 8355dc57ee Ensure proper encoding of error messages 2013-10-03 17:20:00 +02:00
Eelco Dolstra ed132cf1fc Remove obsolete JobStatus source 2013-10-03 15:05:53 +02:00
Eelco Dolstra c7a039bd05 Remove the Errors page
The same info is readily available under the most recent jobset eval
page.
2013-10-03 15:00:28 +02:00
Eelco Dolstra 64d617a6b2 Remove already disabled jobstatus code 2013-10-03 14:56:07 +02:00
Eelco Dolstra adcdfcde6b Remove some obsolete JSON APIs 2013-10-03 14:50:56 +02:00
Eelco Dolstra c8e5faf81e Clean up error handling 2013-10-03 14:45:23 +02:00
Eelco Dolstra 4ddb173ca1 Use notFound instead of status_not_found 2013-10-03 14:16:21 +02:00
Eelco Dolstra 2166c478ef Fix and clean up editing project settings 2013-10-03 14:05:10 +02:00
Eelco Dolstra 9f3b47c963 Allow only project owners to delete projects / jobsets
Giant gaping security hole.
2013-10-03 13:06:16 +02:00
Eelco Dolstra 77fd72070f Fix showing the project's display name 2013-10-03 10:17:53 +00:00
Eelco Dolstra cc1fcf657c Show aborted/cancelled builds separately in jobset eval pages 2013-10-03 01:54:42 +02:00
Eelco Dolstra a5cfae078e Remove the Build menu from the top bar
It's now a dropdown menu in the tabs thingy, which subsumes the
"Reproduce locally" button.  This makes the actions in the menu a bit
more visible, IMHO.
2013-10-02 19:10:00 +02:00
Eelco Dolstra f50477141d Add an input type "nix" for passing arbitrary Nix expressions 2013-09-30 12:03:25 +02:00
Eelco Dolstra d46ebeea99 Distinguish between permanent evaluation errors and transient input errors
Fixes #112.
2013-09-25 16:21:16 +02:00
Eelco Dolstra e1c9e28589 Handle UTF-8 characters in eval error messages 2013-09-25 15:51:03 +02:00
Eelco Dolstra a8db329839 Warn against multiple jobs with the same name 2013-09-25 15:30:59 +02:00
Eelco Dolstra 2d7e106d29 Remove wacky "sysbuild" filtering
It's none of our business if a jobset wants to return a build that
uses a build for another system as an input...
2013-09-25 01:00:20 +02:00
Eelco Dolstra 550d6b79b1 Allow dots in jobset names (like "release-13.09") 2013-09-24 15:15:44 +02:00
Shea Levy 1a313ad566 Allow dashes in jobset input names
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-09-23 17:53:35 -04:00
Shea Levy 98c9559bf2 Delay initializing the s3 client until it's needed to silence warnings
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-09-22 18:48:37 -04:00
Eelco Dolstra 4ed877360b hydra-queue-runner: Improved scheduling
Each jobset now has a "scheduling share" that determines how much of
the build farm's time it is entitled to.  For instance, if a jobset
has 100 shares and the total number of shares of all jobsets is 1000,
it's entitled to 10% of the build farm's time.  When there is a free
build slot for a given system type, the queue runner will select the
jobset that is furthest below its scheduling share over a certain time
window (currently, the last day).  Withing that jobset, it will pick
the build with the highest priority.

So meta.schedulingPriority now only determines the order of builds
within a jobset, not between jobsets.  This makes it much easier to
prioritise one jobset over another (e.g. nixpkgs:trunk over
nixpkgs:stdenv).
2013-09-21 14:57:01 +00:00
Shea Levy 74388353b5 Add a plugin for backing up builds in s3
In your hydra config, you can add an arbitrary number of <s3config>
sections, with the following options:

* name (required): Bucket name
* jobs (required): A regex to match job names (in project:jobset:job
  format) that should be backed up to this bucket
* compression_type: bzip2 (default), xz, or none
* prefix: String to prepend to all hydra-created s3 keys (if this is
  meant to represent a directory, you should include the trailing slash,
  e.g. "cache/"). Default "".

After each build with an output (i.e. successful or failed-with-output
builds), the output path and its closure are uploaded to the bucket as
.nar files, with corresponding .narinfos to enable use as a binary
cache.

This plugin requires that s3 credentials be available. It uses
Net::Amazon::S3, which as of this commit the nixpkgs version can
retrieve s3 credentials from the AWS_ACCESS_KEY_ID and
AWS_SECRET_ACCESS_KEY environment variables, or from ec2 instance
metadata when using an IAM role.

This commit also adds a hydra-s3-backup-collect-garbage program, which
uses hydra's gc roots directory to determine which paths are live, and
then deletes all files except nix-cache-info and any .nar or .narinfo
files corresponding to live paths. hydra-s3-backup-collect-garbage
respects the prefix configuration option, so it won't delete anything
outside of the hierarchy you give it, and it has the same credential
requirements as the plugin. Probably a timer unit running the garbage
collection periodically should be added to hydra-module.nix

Note that two of the added tests fail, due to a bug in the interaction
between Net::Amazon::S3 and fake-s3. Those behaviors work against real
s3 though, so I'm committing this even with the broken tests.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-09-18 18:32:58 +02:00
Eelco Dolstra e4cbd057e2 Fix an uninitialized value warning 2013-09-18 12:51:51 +00:00
Eelco Dolstra 3b70a807b5 Allow setting keepnr to 0 2013-09-18 11:27:37 +02:00
Eelco Dolstra c667b9112d Don't trigger evaluation of disabled jobsets 2013-09-18 11:25:52 +02:00
Petr Rockai 0041c336d1 DarcsInput: Avoid a chdir. 2013-09-07 14:47:49 +02:00
Petr Rockai d31e4469bb Put a 5-second CPU time limit on the log processing pipeline. 2013-09-07 13:28:53 +02:00
Petr Rockai 1e1a1f0838 Also allow : (colon) in pathCompRE. 2013-09-07 13:28:53 +02:00
Petr Rockai bb38bdfe35 Allow : and + in build product filenames. 2013-09-07 13:28:53 +02:00
Petr Rockai dcf386cfed Make getDrvLogPath work with both bucketed and non-bucketed nix logs. 2013-09-07 13:28:53 +02:00
Petr Rockai bb8059e50c Add support for fetching URLs as inputs. 2013-09-07 13:28:53 +02:00
Petr Rockai 5a35912956 Add support for darcs repositories. 2013-09-07 13:28:51 +02:00
Shea Levy 788c2bebaa Not every buildstep has a defined drvpath
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-09-03 14:43:08 -04:00
Eelco Dolstra 04e6eda4c7 Remove unused function 2013-09-03 20:29:52 +02:00
Eelco Dolstra 614f07755a Don't show logs bigger than 64 MB 2013-08-30 20:44:04 +00:00
Eelco Dolstra 6655ada236 Serve raw uncompressed logs directly 2013-08-30 18:11:03 +00:00
Eelco Dolstra fd7e37ef89 Try harder to find build logs
Due to the fixed-output derivation hashing scheme, there can be
multiple derivations of the same output path.  But build logs are
indexed by derivation path.  Thus, we may not be able to find the
log of a build or build step using its derivation.  So as a fallback,
Hydra now looks for other derivations with the same output paths.
2013-08-30 13:53:25 +00:00
Eelco Dolstra 07747b7e88 Allow filtering jobs in eval pages 2013-08-28 17:40:51 +02:00
Eelco Dolstra d0bcaa6284 Allow comparing an eval against the jobset one day/week/month before 2013-08-28 17:40:51 +02:00
Eelco Dolstra 58a6fdc5ed Remove the job status pages
They're mostly redundant since there is a faster "jobs" tab on
the jobset pages now.  The only thing the latter lacks is the
ability to see status change times, but those are quite expensive
to compute, and are visible on build pages if you really need them.
2013-08-28 17:40:51 +02:00
Eelco Dolstra 42c4ef856f Fix incorrect removed jobs in eval pages
PostgreSQL and Perl have different sort orders, in particular when
comparing job names such as "aspell.x86_64-linux" and
"aspellDicts.cs.i686-freebsd".  This confused the evaluation
comparison code, causing some jobs to appear as "removed".
So now we do all the sorting in Perl.

Fixes #105.
2013-08-28 17:40:50 +02:00
Eelco Dolstra d886ff9973 Integrate the "Job status" and "All jobs" tabs
The job status tabs now has a toggle to show inactive jobs, rendering
the "All jobs" tab redundant.
2013-08-28 17:40:50 +02:00
Eelco Dolstra 9002b69c2d By default, show the first 250 jobs only
This is particularly useful for the Nixpkgs jobsets, which now have
~24K jobs.
2013-08-28 17:40:50 +02:00
Eelco Dolstra 410060ec8a Jobset page: Add a new tab to show job status in a matrix 2013-08-28 17:40:50 +02:00
Eelco Dolstra e69c7ac360 "limit" -> "rows" 2013-08-28 17:40:50 +02:00
Eelco Dolstra 7685596aa8 Transpose the aggregate constituents table
This way, it grows vertically rather than horizontally.  Horizontal
may be more "logical", but this is more practical.
2013-08-28 17:40:50 +02:00
Eelco Dolstra 7725038821 On aggregate job pages, show a matrix showing all the constituent builds 2013-08-28 17:40:50 +02:00
Eelco Dolstra a57957df84 Handle job aliases in AggregateConstituents
Aggregate constituents are derivations.  However there can be multiple
builds in an evaluation that have the same derivation, i.e. they can
alias each other (e.g. "emacs", "emacs24" and "emacs24Packages.emacs"
in Nixpkgs).  Previously we picked a build arbitrarily for the
AggregateConstituents table.  Now we pick the one with the shortest
name (e.g. "emacs").
2013-08-27 11:48:02 +02:00
Rob Vermaas a98075f386 HipChat notification: do not include latest commits of all inputs in 'who-broke-the-build' list. Use only committers from inputs that have actually changed since previous build. 2013-08-26 11:06:10 +00:00
Eelco Dolstra 02cba75610 Add an action to download a specific output of a build as a .nar.bz2
E.g. http://hydra/build/3515983/output/out downloads the output named
"out" as a bzip2-compressed NAR.
2013-08-21 14:30:38 +02:00
Eelco Dolstra edb88ef452 Remove unused ActiveJobs source 2013-08-20 15:22:46 +02:00
Eelco Dolstra 6264995198 Remove the jobs status page
The per-system presentation doesn't make much sense any more given
issue #60.  It should be replaced by (say) a grid showing each job per
evaluation.
2013-08-16 17:16:15 +02:00
Eelco Dolstra e54c361a95 Remove per-platform links from the job page
Having different builds within a job is obsolete (issue #60), one
should have different job per platform (e.g. build.x86_64-linux).
2013-08-16 16:39:42 +02:00
Eelco Dolstra a9c6f522e6 clear_queue_non_current: Don't use isCurrent 2013-08-16 16:38:09 +02:00
Eelco Dolstra 72a0fa6ec5 Sort constituents by job name 2013-08-15 03:28:21 +02:00
Eelco Dolstra d92d83a82a Fix broken redirect when editing a release 2013-08-15 02:57:36 +02:00
Eelco Dolstra 1776d9118f Rename aggregate members to constituents 2013-08-15 02:33:10 +02:00
Eelco Dolstra 81322de94e Show aggregate members 2013-08-15 00:30:19 +02:00
Eelco Dolstra d58142b3f0 Store aggregate members in the database
For presentation purposes, we need to know what builds are part of an
aggregate build.  So at evaluation time, look at the "members"
attribute, find the corresponding builds in the eval, and create a
mapping in the AggregateMembers table.
2013-08-14 01:59:29 +02:00
Eelco Dolstra c27f4bbaf5 Add a redirect from an eval to a named member
E.g. /eval/798867/job/tarball redirects to the build with job name
"tarball".
2013-08-13 01:56:45 +02:00
Eelco Dolstra a25ea193f7 Add a /eval action to /build to go from a build to the latest finished eval 2013-08-13 01:43:50 +02:00
Eelco Dolstra 4792256e43 Add some more fields to the JSON output of /build 2013-08-12 22:57:53 +02:00
Eelco Dolstra bef263c930 Add a ‘latest-finished’ action
It redirects to the latest successful build from a finished
evaluation.  This is mostly useful for the Nixpkgs/NixOS mirroring
script, which need the latest finished evaluation in which some
aggregate job (such as ‘tested’ in NixOS) succeeded.
2013-08-12 22:17:04 +02:00
Eelco Dolstra 452c8e36d1 Materialize the number of finished builds
The NrBuilds table tracks the value of ‘select count(*) from Builds
where finished = 0’, keeping it up to date via a trigger.  This is
necessary to make the /all page fast, since otherwise it needs to do a
sequential scan on the Builds table.
2013-08-12 20:19:10 +02:00
Eelco Dolstra 182f725612 Don't pass an undefined input 2013-08-12 18:15:11 +02:00
Eelco Dolstra d96df42c03 GitInput.pm: Don't do a chdir to the Git clone
Doing a chdir in the parent is evil.  For instance, we had Hydra core
dumps ending up in the cloned directory.  Therefore, the function
‘run’ allows doing a chdir in the child.  The function ‘grab’ returns
the child's stdout and throws an exception if the child fails.
2013-08-12 17:46:26 +02:00
Rob Vermaas 90eedcf256 HipChat notification: add support for Mercurial inputs for determining who might have broken the build. 2013-08-07 08:53:32 +00:00
Shea Levy 96e987bbfa Use inputTypes from plugins to determine valid input types
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-29 17:42:49 -04:00
Shea Levy 687ca429c3 Pass project and jobset to fetchInput
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-29 15:33:22 -04:00
Shea Levy f231c23b75 Only serialize JSON and HTML, not the C::C::REST defaults
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-26 14:25:25 -04:00
Shea Levy eab13d8736 ToJSON is automatically called recursively
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-26 13:54:07 -04:00
Shea Levy d6b23272e3 Don't try to serialize if there's nothing to serialize
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-26 12:04:27 -04:00
Shea Levy f7bcf9fc19 The field is actually enabled, not disabled
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-25 17:59:13 -04:00
Shea Levy 1404d33005 Show when a Project's jobsets are disabled
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-25 17:48:28 -04:00
Rob Vermaas 513c030268 Do not use local clone for Bazaar inputs. 2013-07-22 20:42:17 +02:00
Eelco Dolstra 6574d125c7 Get rid of a warning in the HipChat plugin 2013-07-19 14:43:04 +02:00
Ludovic Courtès 2d5e06918b Hydra::View::Plain: Explicitly set the response body.
This fixes a bug with Catalyst 1.39 whereby a raw hash table would
erroneously be returned for /nix-cache-info.
2013-07-12 16:53:48 +02:00
Ludovic Courtès 438d7f7c5c Reply 404 for requests for non-existent .narinfo. 2013-07-12 16:52:40 +02:00
Eelco Dolstra d071bbfb28 Fix Hipchat notification 2013-07-12 15:04:13 +02:00
Shea Levy 7a0f80f016 Include the email override list in the Jobset serialization
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-11 11:01:36 -04:00
Eelco Dolstra 486a844866 Fix audience URL 2013-07-09 14:05:04 +02:00
Eelco Dolstra 1db9381473 Fix legacy login 2013-07-09 13:55:44 +02:00
Eelco Dolstra 476203d0bb Prevent Persona users from setting a password 2013-07-09 13:01:56 +02:00
Eelco Dolstra 5fecf8e385 Show sign in as success 2013-07-09 12:07:18 +02:00
Eelco Dolstra 12ca755bea Don't redirect to /login if authentication is required
We can just serve the login page in place.
2013-07-09 12:02:15 +02:00
Eelco Dolstra c08fc6ce1e Add basic Persona support
This allows users to sign in to Hydra using Mozilla Persona accounts.
When a user first sign in, a row in the Users table for the given
Persona identity (an email address) is created automatically.

To do: figure out how to deal with legacy accounts.
2013-07-08 23:54:40 +02:00
Shea Levy efd011fbc3 Revert "Don't call buildFinished after we already know it failed"
I don't understand perl strings.

This reverts commit b2f6be9686.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-08 14:30:46 -04:00
Shea Levy b2f6be9686 Don't call buildFinished after we already know it failed
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-08 13:52:51 -04:00
Shea Levy 166d56088f Call buildFinished when a cached build is added
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-08 13:35:34 -04:00
Shea Levy b46f2134e0 Revert "Emable overriding the email template."
This doesn't work as planned, will re-work.

This reverts commit 304f8a5714.
2013-07-05 18:10:50 -04:00
Shea Levy a168f43515 Revert "Handle HYDRA_EMAIL_TEMPLATES that are paths"
This doesn't work as planned, will re-work.

This reverts commit d2ce7747a0.
2013-07-05 18:10:21 -04:00
Shea Levy d2ce7747a0 Handle HYDRA_EMAIL_TEMPLATES that are paths
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-05 17:22:51 -04:00
Shea Levy 304f8a5714 Emable overriding the email template.
The HYDRA_EMAIL_TEMPLATE env var can be set to any value understood by
https://metacpan.org/module/Template#process-template-vars-output-options
as the $template argument.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-05 16:29:09 -04:00
Shea Levy be3ca0dd84 Fix jobset update
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-05 14:44:29 -04:00
Shea Levy 93a14ccf32 Check if we want to send email for a build before iterating through its users
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-05 14:06:10 -04:00
Shea Levy 002ac9ef63 Merge in the first bits of the API work
The catalyst-action-rest branch from shlevy/hydra was an exploration of
using Catalyst::Action::REST to create a JSON API for hydra. This commit
merges in the best bits from that experiment, with the goal that further
API endpoints can be added incrementally.

In addition to migrating more endpoints, there is potential for
improvement in what's already been done:
* The web interface can be updated to use the same non-GET endpoints as
  the JSON interface (using x-tunneled-method) instead of having a
  separate endpoint
* The web rendering should use the $c->stash->{resource} data structure
  where applicable rather than putting the same data in two places in
  the stash
* Which columns to render for each endpoint is a completely debatable
  question
* Hydra::Component::ToJSON should turn has_many relations that have
  strings as their primary keys into objects instead of arrays

Fixes NixOS/hydra#98

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-02 14:00:46 -04:00
Eelco Dolstra d18fc4fc38 Include names of committers in HipChat notifications
HipChat notification messages now say which committers were
responsible, e.g.

  Job patchelf:trunk:tarball: Failed, probably due to 2 commits by Eelco Dolstra
2013-07-02 13:54:18 +02:00
Eelco Dolstra fe030331b5 Revert "Don't compress already-compressed files."
This reverts commit 190bffd846.
2013-07-01 18:46:18 +02:00
Eelco Dolstra 11414b0447 Revert "Doh"
This reverts commit f1f938512d.
2013-07-01 18:46:14 +02:00
Rob Vermaas f1f938512d Doh 2013-07-01 18:12:33 +02:00
Rob Vermaas 9d7c14b342 Merge pull request #103 from civodul/nar-uncompressed
Don't compress already-compressed files.
2013-07-01 08:16:51 -07:00
Eelco Dolstra 9939ddf22c Add a plugin for HipChat notification
This plugin sends notification of build failure or success to a
HipChat room, if the status differs from the last build.

The plugin can be configured by adding one or more of these stanzas to
hydra.conf:

  <hipchat>
    jobs = (patchelf|nixops):.*:.*
    room = 1234
    token = 39ab2198fe...
  </hipchat>

Here "jobs" is a regular expression against which the fully qualified
job name of the build is matched (so for instance
"nixops:master:tarball" will match the stanza above).
2013-06-27 18:56:31 +02:00
Eelco Dolstra 698d6eac89 Eliminate duplicate getPreviousBuild functions
And move some stuff into CatalystUtils for use by other plugins.
2013-06-27 18:08:00 +02:00
Eelco Dolstra a5436be2ce After editing a project/jobset, redirect back to the config tab 2013-06-25 01:00:59 +02:00
Eelco Dolstra afc90b36db Don't print to stdout 2013-06-24 22:22:43 +00:00
Ludovic Courtès 190bffd846 Don't compress already-compressed files.
Fixes <https://github.com/NixOS/hydra/issues/102>.
2013-06-23 15:23:16 +02:00
Eelco Dolstra 51f0185797 Allow _ in release names 2013-06-18 16:00:24 +02:00
Eelco Dolstra 1c30a777ad Fix project/jobset deletion 2013-06-11 16:57:22 +02:00
Eelco Dolstra f4ecf27def Fix Perl warning about $path masking earlier definition 2013-06-02 23:32:09 +02:00
Eelco Dolstra 77e37c1ea3 Don't add a nix-build build product unless $out is a directory 2013-05-28 18:43:05 +02:00
Eelco Dolstra 1f1615e80b Support revision control systems via plugins 2013-05-25 15:36:58 -04:00
Eelco Dolstra 5e5b344527 Make latest/latest-for more efficient
Getting all matching rows from the database is expensive.
2013-05-23 12:18:38 -04:00
Eelco Dolstra 7b5e92868b Remove unused Jobs columns
Actually, we can *almost* get rid of the Jobs table entirely...
2013-05-23 12:12:19 -04:00