lix-project/hydra
3
0
Fork 6
Code Issues Pull requests Packages Projects Releases Wiki Activity
3848 commits 2 branches 0 tags 20 MiB
Commit graph

2812 commits

Author SHA1 Message Date
Eelco Dolstra 2e3899ed27 Don't set Expires header for logs of unfinished builds/steps 2015-08-12 12:22:14 +02:00
Eelco Dolstra 576dc0c120 For completeness, re-implement meta.schedulingPriority 2015-08-12 12:05:43 +02:00
Eelco Dolstra b7965df928 Load the queue in order of global priority 2015-08-11 02:14:34 +02:00
Eelco Dolstra 97f11baa8d Revive jobset scheduling 
(I.e. taking the jobset scheduling share into account.)
 2015-08-11 01:31:56 +02:00
Shea Levy 1705ca41e7 Remove unneeded camelcase 2015-08-10 13:59:22 -04:00
Shea Levy 163e696813 Copy-paste error 2015-08-10 13:52:40 -04:00
Shea Levy 882b6b3377 Pass a build's drv path as a store path 2015-08-10 13:48:09 -04:00
Shea Levy ce5ffa9fba Only pass the drv path if it is still valid 2015-08-10 13:47:39 -04:00
Eelco Dolstra 08739a2a5a Allow bumping an entire evaluation to the front of the queue 
This is mostly useful if we need to prioritize (say) a nixos-stable
eval that has some critical security bug fix.
 2015-08-10 18:57:59 +02:00
Eelco Dolstra eb13007fe6 Allow build to be bumped to the front of the queue via the web interface 
Builds now have a "Bump up" action. This will cause the queue runner
to prioritise the steps of the build above all other steps.
 2015-08-10 16:19:47 +02:00
Eelco Dolstra 27182c7c1d Start steps in order of ascending build ID 2015-08-10 16:19:47 +02:00
Shea Levy 2a240e458e Pass along drvPath and outputName for inputs that are previous builds. 
This allows importing the .drv and getting the same store paths as if the
input had been passed in as nix expressions defining a proper derivation.
 2015-08-10 08:02:10 -04:00
Eelco Dolstra 52ec971154 Don't render machine name if not applicable to step 2015-08-10 12:54:56 +02:00
Eelco Dolstra 593850b956 Fix potential race in dispatcher wakeup 2015-08-10 12:54:55 +02:00
Eelco Dolstra 6a1c950e94 Unindent 2015-08-10 11:33:22 +02:00
Eelco Dolstra f21b88e388 Remove superfluous check 2015-08-07 04:20:34 +02:00
Eelco Dolstra f1fbf8c605 Fix race in finishing builds that have been cancelled 2015-08-07 04:18:48 +02:00
Eelco Dolstra 90c462a222 Merge remote-tracking branch 'origin/master' into build-ng 
Conflicts:
	hydra-module.nix
 2015-08-04 14:30:22 +02:00
Shea Levy 7b7f189b3e Whoops, misread signature 2015-08-04 08:12:24 -04:00
Shea Levy a5b986cf33 Only pass the first value of each input in the inputs argument 
Multiple alts are on their way out
 2015-08-04 08:10:47 -04:00
Shea Levy 4d967dd17a hydra-eval-jobs: Pass all inputs as 'inputs' arg. 
If there is no input named 'inputs', hydra-eval-jobs now passes in a set
of lists, where each attribute corresponds to an input defined in the
jobset specification and each list element is a different input alt, as
an argument named 'inputs'.

Among other things, this allows for generic hydra expressions to be
shared amongst projects with similar structures but different sets of
specific inputs.
 2015-08-04 07:54:24 -04:00
Shea Levy d450d08929 buildInputToString: Use inputType attribute instead of type attribute 2015-08-04 06:50:04 -04:00
Eelco Dolstra 6f04299411 Allow downloading file by filename 
For instance, instead of

  http://.../build/3770911/download/4/source.tar.gz

you can get

  http://.../build/3770911/download/source.tar.gz

This is useful when file names are constant across builds but product
numbers are not.
 2015-07-31 17:50:22 +02:00
Shea Levy 07634e8862 buildInputToString: Pass along the input's type and urr 2015-07-31 09:47:44 -04:00
Eelco Dolstra ff3f5eb4d8 Fix remote building on Nix 1.10 2015-07-31 03:41:55 +02:00
Eelco Dolstra 5b9a288123 Workaround for RemoteStore not supporting cmdBuildDerivation yet 2015-07-31 03:39:20 +02:00
Eelco Dolstra 706746d4bf Fix build 2015-07-31 01:45:16 +02:00
Eelco Dolstra 4d26546d3c Add support for tracking custom metrics 
Builds can now emit metrics that Hydra will store in its database and
render as time series via flot charts. Typical applications are to
keep track of performance indicators, coverage percentages, artifact
sizes, and so on.

For example, a coverage build can emit the coverage percentage as
follows:

  echo "lineCoverage $pct %" > $out/nix-support/hydra-metrics

Graphs of all metrics for a job can be seen at

  http://.../job/<project>/<jobset>/<job>#tabs-charts

Specific metrics are also visible at

  http://.../job/<project>/<jobset>/<job>/metric/<metric>

The latter URL also allows getting the data in JSON format (e.g. via
"curl -H 'Accept: application/json'").
 2015-07-31 00:57:30 +02:00
Eelco Dolstra 9afaf322b3 Use latest DBIx::Class::Schema::Loader 2015-07-30 16:55:11 +02:00
aszlig 36cbdd1bd8
common.tt: Properly generate URL for /api/scmdiff. 
If Hydra isn't hosted on https://example.com/ but something like
https://example.com/hydra/, the URL for /api/scmdiff would have ended up
on /api/scmdiff rather than /hydra/api/scmdiff.

This is because we didn't use the URI resolver from the controller,
hence we're using it now to build up the whole URL including the query
string.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
 2015-07-22 12:00:15 +02:00
Eelco Dolstra c18fb0ad74 Temporarily disable machines after a connection failure 2015-07-21 15:58:47 +02:00
Eelco Dolstra 7e026d35f7 Split hydra-queue-runner.cc more 2015-07-21 15:14:17 +02:00
Eelco Dolstra 5370be9f52 hydra-queue-runner: Use cmdBuildDerivation 
See 1511aa9f48 and eda2f36c2a.
 2015-07-21 01:54:24 +02:00
Eelco Dolstra 62b1b095ad Handle IP addresses as machine names 2015-07-21 01:54:24 +02:00
Rob Vermaas fbd48f5684 Make cloning of jobsets work again. 
(cherry picked from commit 706551453800fb3a4bb3d8b681dad157a8cea98b)
 2015-07-14 09:07:10 +00:00
Eelco Dolstra 3ded87329d Keep track of how many threads are waiting 2015-07-10 19:10:14 +02:00
Eelco Dolstra 06d75699a7 Fix restarting a build 2015-07-10 16:56:53 +02:00
Eelco Dolstra 7f865a30d5 hydra-evaluator: Fix input change check 
Because inputs were processed in random order by inputsToArgs, the
inputs hash could be different every time, leading to unnecessary
re-evaluations.
 2015-07-10 16:44:06 +02:00
Eelco Dolstra 3e7bbec40b hydra-evaluator: Send statistics to statsd 2015-07-10 16:40:50 +02:00
Eelco Dolstra 5919e911db Don't show how long a machine has been idle 
Without an index on (machine, stoptime desc), this requires a
sequential scan. And adding a whole index for this seems
overkill. (Possibly the queue runner could maintain this info more
efficiently.)
 2015-07-10 15:41:57 +02:00
Eelco Dolstra 3bb9e17e5c IndexJobsetEvalsOnJobsetId: Only index jobsets with new builds 2015-07-10 15:41:57 +02:00
Eelco Dolstra b09f7e0989 Add page showing latest build steps 2015-07-10 15:41:57 +02:00
Eelco Dolstra 0da08df4eb Stream logs if possible and remove size limit 2015-07-08 19:05:17 +02:00
Eelco Dolstra d8acaf2181 Index BuildSteps on propagatedFrom 
This significantly speeds up deleting Builds, since it removes the
need for a sequential scan on BuildSteps.
 2015-07-08 12:12:44 +02:00
Eelco Dolstra bbee81efae Use triggers for all notifications on Builds table changes 2015-07-08 12:05:32 +02:00
Eelco Dolstra 89fb723ace Notify the queue runner when a build is deleted 2015-07-08 11:43:35 +02:00
Eelco Dolstra 95c4294560 Allow cancelling builds marked as busy 
Note that if there are active build *steps*, this won't cancel them.
 2015-07-07 14:08:46 +02:00
Eelco Dolstra 35b7c4f82b Allow only 1 thread to send a closure to a given machine at the same time 
This prevents a race where multiple threads see that machine X is
missing path P, and start sending it concurrently. Nix handles this
correctly, but it's still wasteful (especially for the case where P ==
GHC).

A more refined scheme would be to have per machine, per path locks.
 2015-07-07 14:06:48 +02:00
Eelco Dolstra 16696a4aee Namespace cleanup 2015-07-07 10:29:43 +02:00
Eelco Dolstra 63745b8e25 Move buildRemote() into State 2015-07-07 10:25:33 +02:00
Eelco Dolstra df29527531 Refactor 2015-07-07 10:17:21 +02:00
Eelco Dolstra dd4f6e695e Merge branch 'master' into build-ng 2015-07-06 17:17:51 +02:00
Eelco Dolstra ccf6e6062c Store full Mercurial revision hashes 2015-07-06 17:17:17 +02:00
Eelco Dolstra 309ef5baa9 Merge branch 'master' into build-ng 2015-07-06 15:57:09 +02:00
Eelco Dolstra b85e9ef1cd Support using Git revisions as branch names 2015-07-06 15:56:24 +02:00
Eelco Dolstra b03de925cb Allow a jobset to be created from an evaluation 
Fixes #150.
 2015-07-06 15:56:20 +02:00
Eelco Dolstra dffb629b8a Unify Hydra's NixOS module with the one used for hydra.nixos.org 
In particular, the queue runner and web server now run under different
UIDs.
 2015-07-02 01:01:44 +02:00
Eelco Dolstra 3e0f5f664a GitInput plugin: Don't clone during getCommits 
This doesn't work if hydra-queue-runner has no write access to the scm
directory, and in any case races with the evaluator.
 2015-07-02 00:44:40 +02:00
Eelco Dolstra ae52fc7f61 Remove display of queue runner log file (it no longer exists) 2015-07-02 00:18:33 +02:00
Eelco Dolstra e35b704d80 Drop the 5 minute minimum interval between triggered evals 2015-07-01 14:45:39 +02:00
Eelco Dolstra 85a1ce99c9 Only include Persona JS when Persona is enabled 2015-07-01 14:24:18 +02:00
Eelco Dolstra 3c665dac82 Remove superfluous HYDRA_LOGO environment variable 2015-07-01 11:34:19 +02:00
Eelco Dolstra 7e6135a8c6 Don't repeat links to build step logs 
Hydra only stores the last log for a particular derivation, so only
show log links for the last one.
 2015-06-30 00:27:31 +02:00
Eelco Dolstra 2ece42b2b9 Support preferLocalBuild 
Derivations with "preferLocalBuild = true" can now be executed on
specific machines (typically localhost) by setting the mandary system
features field to include "local". For example:

  localhost x86_64-linux,i686-linux - 10 100 - local

says that "localhost" can *only* do builds with "preferLocalBuild =
true". The speed factor of 100 will make the machine almost always win
over other machines.
 2015-06-30 00:20:19 +02:00
Eelco Dolstra 008d610467 getQueuedBuilds(): Don't catch errors while loading a build from the queue 
Otherwise we never recover from reset daemon connections, e.g.

  hydra-queue-runner[16106]: while loading build 599369: cannot start daemon worker: reading from file: Connection reset by peer
  hydra-queue-runner[16106]: while loading build 599236: writing to file: Broken pipe
  ...

The error is now handled queueMonitor(), causing the next call to
queueMonitorLoop() to create a new connection.
 2015-06-26 21:06:35 +02:00
Eelco Dolstra f5e5a1b96e Don't wake up the queue runner for cached evals 2015-06-26 20:59:14 +02:00
Eelco Dolstra 401f5bdce2 Add a unit for hydra-send-stats 2015-06-26 15:24:12 +02:00
Eelco Dolstra 9a041f9a36 Restart builds failed due to unsupported system type 2015-06-26 11:28:38 +02:00
Eelco Dolstra 2f4676bd97 JSONObject doesn't handle 64-bit integers 2015-06-25 16:59:48 +02:00
Eelco Dolstra c54a04688e Fix email sender address when notification_sender is not set 2015-06-25 16:49:01 +02:00
Eelco Dolstra c6fcce3b3b Moar stats 2015-06-25 16:47:39 +02:00
Eelco Dolstra 18a3c3ff1c Update "make check" for the new queue runner 
Also, if the machines file contains an entry for localhost, then run
"nix-store --serve" directly, without going through SSH.
 2015-06-25 16:47:39 +02:00
Eelco Dolstra 32210905d8 Automatically reload $NIX_REMOTE_SYSTEMS when it changes 
Otherwise, you'd have to restart the queue runner to add or remove
machines.
 2015-06-25 16:47:25 +02:00
Eelco Dolstra 1a0e1eb5a0 More stats 2015-06-24 13:19:27 +02:00
Eelco Dolstra 3f8891b6ff Fix incorrect debug message 2015-06-23 17:53:15 +02:00
Eelco Dolstra 62219adaf3 Send queue runner stats to statsd 
This is currently done by a separate program that periodically
calls "hydra-queue-runner --status". Eventually, I'll do this
in the queue runner directly.

Fixes #220.
 2015-06-23 14:56:43 +02:00
Eelco Dolstra af5cbe97aa createStep(): Cache finished derivations 
This gets rid of a lot of redundant calls to readDerivation().
 2015-06-23 03:25:31 +02:00
Eelco Dolstra 681f63a382 Typo 2015-06-23 02:15:11 +02:00
Eelco Dolstra 524ee295e0 Fix sending notifications in the successful case 2015-06-23 02:13:06 +02:00
Eelco Dolstra 4db7c51b5c Rate-limit the number of threads copying closures at the same time 
Having a hundred threads doing I/O at the same time is bad on magnetic
disks because of the excessive disk seeks. So allow only 4 threads to
copy closures in parallel.
 2015-06-23 01:49:14 +02:00
Eelco Dolstra a317d24b29 hydra-queue-runner: Send build notifications 
Since our notification plugins are written in Perl, sending
notification from C++ requires a small Perl helper named
‘hydra-notify’.
 2015-06-23 00:14:49 +02:00
Eelco Dolstra 5312e1209b Keep per-machine stats 2015-06-22 17:11:17 +02:00
Eelco Dolstra d06366e7cf Remove obsolete comment 2015-06-22 16:59:50 +02:00
Eelco Dolstra e069ee960e Doh 2015-06-22 16:58:40 +02:00
Eelco Dolstra e32ee3d5b9 Remove hydra-build and the old hydra-queue-runner 2015-06-22 15:43:15 +02:00
Eelco Dolstra 41ba7418e2 hydra-queue-runner: More stats 2015-06-22 15:34:33 +02:00
Eelco Dolstra 62b53a0a47 Guard against concurrent invocations of hydra-queue-runner 2015-06-22 14:24:03 +02:00
Eelco Dolstra fbd7c02217 Periodically dump/log status 2015-06-22 14:15:43 +02:00
Eelco Dolstra 4f4141e1db Add command ‘hydra-queue-runner --status’ to show current status 2015-06-22 14:06:44 +02:00
Eelco Dolstra 44a2b74f5a Keep track of the number of build steps that are being built 
(As opposed to being in the closure copying stage.)
 2015-06-22 11:23:00 +02:00
Eelco Dolstra fed71d3fe9 Move "created" field into Step::State 2015-06-22 11:07:52 +02:00
Eelco Dolstra 90a08db241 hydra-queue-runner: Fix assertion failure 2015-06-22 10:59:07 +02:00
Eelco Dolstra d744362e4a hydra-queue-runner: Fix segfault sorting machines by load 
While sorting machines by load, the load of a machine
(machine->currentJobs) can be changed by other threads. If that
happens, the comparator is no longer a proper ordering, in which case
std::sort() can segfault. So we now make a copy of currentJobs before
sorting.
 2015-06-21 16:21:42 +02:00
Eelco Dolstra a0eff6fc15 Fix machine selection 2015-06-19 17:45:26 +02:00
Eelco Dolstra 81abb6e166 Improve parsing of hydra-build-products 2015-06-19 17:20:20 +02:00
Eelco Dolstra e13477bdf2 Robustness 2015-06-19 16:35:49 +02:00
Eelco Dolstra f196967c43 Don't create a propagated build step to the same build 2015-06-19 15:33:37 +02:00
Eelco Dolstra 7afc61691b Doh 2015-06-19 15:27:49 +02:00
Eelco Dolstra 133d298e26 Asynchronously compress build logs 2015-06-19 15:06:12 +02:00
Eelco Dolstra 8e408048e2 Create build step for non-top-level cached failures 
This fixes the missing build step on failures like

  http://hydra.nixos.org/build/23222231
 2015-06-19 11:33:15 +02:00
Eelco Dolstra 77c8bfd392 Improve logging for aborts 2015-06-19 10:37:22 +02:00
Eelco Dolstra 8db1ae2855 Less verbosity 2015-06-18 17:43:13 +02:00
Eelco Dolstra 89b629eeb1 Fix finishing steps that are not top-level of any build 2015-06-18 17:37:35 +02:00
Eelco Dolstra 9cdbff2fdf Handle concurrent finishing of the same build 
There is a slight possibility that the queue monitor and a builder
thread simultaneously decide to mark a build as finished. That's fine,
as long as we ensure the DB update is idempotent (as ensured by doing
"update Builds set finished = 1 ... where finished = 0").
 2015-06-18 17:12:51 +02:00
Eelco Dolstra 948473c909 Fix race between the queue monitor and the builder threads 2015-06-18 16:30:28 +02:00
Eelco Dolstra 9c03b11ca8 Simplify retry handling 2015-06-18 14:51:50 +02:00
Eelco Dolstra e039f5f840 Create failed build steps for cached failures 2015-06-18 04:35:37 +02:00
Eelco Dolstra 92ea800cfb Set finishedInDB in a few more places 2015-06-18 04:19:21 +02:00
Eelco Dolstra 47367451c7 hydra-queue-runner: Set isCachedBuild 2015-06-18 03:28:58 +02:00
Eelco Dolstra 8257812d0a Acquire exclusive table lock earlier 2015-06-18 02:44:29 +02:00
Eelco Dolstra 69be3cfe93 hydra-queue-runner: Handle status queries on the main thread 
Doing it on the queue monitor thread was problematic because
processing the queue can take a while.
 2015-06-18 01:57:01 +02:00
Eelco Dolstra a40ca6b76e hydra-queue-runner: Improve dispatcher 
We now take the machine speed factor into account, just like
build-remote.pl.
 2015-06-18 01:52:20 +02:00
Eelco Dolstra 3855131185 hydra-queue-runner: Improve SSH flags 2015-06-18 00:50:48 +02:00
Eelco Dolstra f57d0b0c54 hydra-queue-runner: Maintain count of active build steps 2015-06-18 00:24:56 +02:00
Eelco Dolstra 59dae60558 hydra-queue-runner: More stats 2015-06-17 22:38:12 +02:00
Eelco Dolstra ec8e8edc86 hydra-queue-runner: Handle $HYDRA_DBI 2015-06-17 22:11:01 +02:00
Eelco Dolstra 4d9c74335d Add forgotten file 2015-06-17 21:39:28 +02:00
Eelco Dolstra ce9e859a9c hydra-queue-runner: Implement --unlock 2015-06-17 21:35:20 +02:00
Eelco Dolstra ca48818b30 Fix remote building 2015-06-17 17:28:59 +02:00
Eelco Dolstra 11be780948 Handle failure with output 2015-06-17 17:11:42 +02:00
Eelco Dolstra b1a75c7f63 getQueuedBuilds(): Handle dependent builds first 
If a build A depends on a derivation that is the top-level derivation
of some build B, then we should process B before A (meaning we
shouldn't make the derivation runnable before B has been
added). Otherwise, the derivation will be "accounted" to A rather than
B (so the build step will show up in the wrong build).
 2015-06-17 14:46:02 +02:00
Eelco Dolstra c6d504edbb Handle SSH hosts without a @ 2015-06-17 13:49:18 +02:00
Eelco Dolstra 745efce828 hydra-queue-runner: Implement timeouts 
Also, keep track of timeouts in the database as a distinct build
status.
 2015-06-17 13:32:33 +02:00
Eelco Dolstra 2da4987bc2 Don't lock the CPU 2015-06-17 11:48:38 +02:00
Eelco Dolstra b91a616520 Automatically retry aborted builds 
Aborted builds are now put back on the runnable queue and retried
after a certain time interval (currently 60 seconds for the first
retry, then tripled on each subsequent retry).
 2015-06-17 11:45:20 +02:00
Eelco Dolstra e02654b3a0 Prefer cached failure over unsupported system type 2015-06-16 18:00:39 +02:00
Eelco Dolstra a984c0badc Merge branch 'master' into build-ng 2015-06-15 18:21:07 +02:00
Eelco Dolstra 42e7301c08 Add status dump facility 
Doing

  $ psql hydra -c 'notify dump_status'

will cause hydra-queue-runner to dump some internal status info on
stderr.
 2015-06-15 18:20:14 +02:00
Eelco Dolstra dd104f14ea Make the queue monitor more robust, and better debug output 2015-06-15 16:54:52 +02:00
Eelco Dolstra 147eb4fd15 Support requiredSystemFeatures 2015-06-15 16:33:50 +02:00
Eelco Dolstra 508ab7f8a2 Tweak build steps 2015-06-15 15:48:05 +02:00
Eelco Dolstra 21aaa0596b Fail builds with previously failed steps early 2015-06-15 15:31:42 +02:00
Eelco Dolstra c00bf7cd1a Check non-runnable steps for unsupported system type 2015-06-15 15:13:03 +02:00
Eelco Dolstra 5019fceb20 Add a error type for "unsupported system type" 2015-06-15 15:07:04 +02:00
Eelco Dolstra 541fbd62cc Immediately abort builds that require an unsupported system type 2015-06-15 14:51:49 +02:00
Eelco Dolstra f9cd5adae8 Queue monitor: Get only the fields we need 2015-06-11 18:09:50 +02:00
Eelco Dolstra c974fb893b Support cancelling builds 2015-06-11 18:07:45 +02:00
Eelco Dolstra c08883966c Use PostgreSQL notifications for queue events 
Hydra-queue-runner now no longer polls the queue periodically, but
instead sleeps until it receives a notification from PostgreSQL about
a change to the queue (build added, build cancelled or build
restarted).

Also, for the "build added" case, we now only check for builds with an
ID greater than the previous greatest ID. This is much more efficient
if the queue is large.
 2015-06-11 17:41:59 +02:00
Eelco Dolstra 672bbb1c67 hydra-update-gc-roots: Get Builds and BuildOutputs in the same query 
This greatly reduces the number of roundtrips to the database.
 2015-06-11 14:10:21 +02:00
Eelco Dolstra 9989a90e99 Keep the most recent successful build of current jobs 
Fixes #140.
 2015-06-11 14:10:21 +02:00
Eelco Dolstra d72a88b562 Don't try to handle SIGINT 
It just makes things unnecessarily complicated. We can just exit
without cleaning anything up, since the only thing to do is unmark
builds and build steps as busy. But we can do that by having systemd
call "hydra-queue-runner --unlock" from ExecStopPost.
 2015-06-10 15:55:46 +02:00
Eelco Dolstra a4fb93c119 Lock builds for a shorter amount of time 2015-06-10 15:36:21 +02:00
Eelco Dolstra 6d738a31bf Keep track of failed paths in the Hydra database 
I.e. don't use Nix's failed paths feature anymore. Easier to keep
everything in one place.
 2015-06-10 14:57:16 +02:00
Eelco Dolstra c68036f8b0 Pass ssh key 2015-06-10 14:57:07 +02:00
Eelco Dolstra 7dd1f0097e Finish copyClosure 2015-06-09 16:03:41 +02:00
Eelco Dolstra c93aa92563 Create BuildSteps race-free 
If multiple threads create a step for the same build, they could get
the same "max(stepnr)" and allocate conflicting new step numbers. So
lock the BuildSteps table while doing this. We could use a different
isolation level, but this is easier.
 2015-06-09 15:03:20 +02:00
Eelco Dolstra 61d4060522 Record the machine used for a build step 2015-06-09 14:57:49 +02:00
Eelco Dolstra 08633508da Fix colspan 2015-06-09 14:42:02 +02:00
Eelco Dolstra ca1fbdd058 Mark builds as busy 2015-06-09 14:31:43 +02:00
Eelco Dolstra 8b12ac1f6d Basic remote building 
This removes the need for Nix's build-remote.pl.

Build logs are now written to $HYDRA_DATA/build-logs because
hydra-queue-runner doesn't have write permission to /nix/var/log.
 2015-06-09 14:21:21 +02:00
Eelco Dolstra 3a6cb2f270 Implement a database connection pool 2015-05-29 20:55:13 +02:00
Eelco Dolstra 214b95706c On SIGINT, shut down the builder threads 
Note that they don't get interrupted at the moment (so on SIGINT, any
running builds will need to finish first).
 2015-05-29 20:02:15 +02:00
Eelco Dolstra e778821940 Make concurrency more robust 2015-05-29 17:14:20 +02:00
Eelco Dolstra 8640e30787 Very basic multi-threaded queue runner 2015-05-29 01:31:12 +02:00
Eelco Dolstra 604fdb908f Pass null values to libpqxx properly 2015-05-28 19:06:17 +02:00
Eelco Dolstra dc446c3980 Start of single-process hydra-queue-runner 2015-05-28 17:39:29 +02:00
Eelco Dolstra ff8cb93fe7 reproduce.tt: Wrap script to prevent problems piping into bash 2015-05-26 17:11:29 +02:00
Eelco Dolstra c32a244bd7 reproduce.tt: Add --print-flags option 
Useful when scripting stuff (e.g. if you want to do a nix-instantiate
instead of a nix-build).
 2015-05-26 16:36:08 +02:00
Eelco Dolstra f5ecb7a81a reproduce.tt: Use -I instead of $NIX_PATH 2015-05-26 16:07:00 +02:00
Eelco Dolstra b8a0956443 reproduce.tt: Add option to fetch but not build 2015-05-26 16:01:37 +02:00
Eelco Dolstra 91ecee0e5d Fix reproduce script 2015-05-26 15:54:38 +02:00
Eelco Dolstra 1ab0f96079 Rename c -> hydra-eval-jobs 2015-05-26 13:58:01 +02:00
Eelco Dolstra e003665146 Split timeSpent query into 2 separate queries, as postgresql isn't able to figure out a decent query plan. With 120k jobs in queue, this makes some queries go from 100s to 1-2s. 2015-05-01 12:28:29 +02:00
Eelco Dolstra d9ab964203 UTF-8 fix 2015-04-14 15:20:56 +02:00
Eelco Dolstra 01cd6397cb Better error message 2015-04-14 15:16:24 +02:00
Eelco Dolstra 5a61aefe4f Use Nix's restricted evaluation mode 2015-04-14 15:16:00 +02:00
Eelco Dolstra 5d8a563e90 Merge pull request #204 from aszlig/tail-improvements 
Improvements of tail-reload output
 2015-04-14 12:44:36 +02:00
Eelco Dolstra 63306aaf5a hydra-evaluator: Add some debug code 2015-04-09 17:35:04 +02:00
Eelco Dolstra a2dc92d871 Die tabs die 2015-04-09 17:22:10 +02:00
Eelco Dolstra 9e664cf8b0 Fix not-null constraint violation inserting build step 
DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR:  null value in column "machine" violates not-null constraint
 2015-04-07 13:34:00 +02:00
aszlig 7a48ad375a
plain-reload: Scroll down at document load. 
When visiting the tail-reload page, for a short amount of time the
"unscrolled" version is shown. To circumvent that, let's scroll down
immediately at the first possibility to fill the gap between the loading
of the document and the first AJAX request coming in.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
 2015-04-03 00:00:27 +02:00
aszlig c251598241
plain-reload: Constrain tail output to 50 lines. 
There are quite a lot of build outputs which have lines with a length
exceeding the width of the taillog <pre/> and thus visually produce more
lines than 50. This causes the tail "box" to change height frequently
and to get to the bottom you need to scroll down.

We now set a fixed line-height to 120% of the font size and cap the
maximum height based on that value (50 * 1.2 = 60). It's probably not
nice to override the line-height, but max-lines is currently only
available using browser-specific property names. But after all it's just
for the tail output, if people complain about the line-height, we can
still change it :-)

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
 2015-04-02 17:29:48 +02:00
aszlig be051bcadc
plain-reload.tt: Properly escape tail content. 
We're just implicitly escaping the tail content by not using .load() but
explicitly setting the text content using .text(), so that escaping
isn't needed on our side.

This should get rid of a few formatting errors and possibly XSS if
someone manages to place JS code in the tail of a build and manages to
lurk a user to that tail output.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
 2015-04-02 16:55:59 +02:00
Eelco Dolstra aec0a35114 Fix duplicate row-link 2015-03-24 14:31:27 +01:00
Rob Vermaas 125dd80dd8 Respect hideJobset in renderBuildListBody. 2015-03-24 08:38:40 +00:00
Rob Vermaas cad82493b0 Restore build and eval IDs. 2015-03-23 15:11:38 +00:00
Rob Vermaas 91e7e6f6ab Use DAG range to determine commits between mercurial revisions. Previously it would show also commits in other branches. 2015-03-23 13:52:06 +00:00
Eelco Dolstra 3b5e84cc73 Tweak debug output 2015-03-23 12:59:41 +01:00
Eelco Dolstra 479632efa3 Call initGC() 2015-03-19 20:16:38 +01:00
Eelco Dolstra 4c0e3e4703 Debug tweak 2015-03-18 22:03:55 +01:00
Eelco Dolstra ad2b7646ac Don't show missing paths in logs 2015-03-04 15:44:04 +01:00
Eelco Dolstra 522bcea8fc Hide build IDs from build lists 
Like eval IDs, build IDs don't convey useful information.

Also, make the job name link to the build rather than the job. When
people click on a build, they expect to go to the build page, not the
job page.
 2015-02-26 13:28:14 +01:00
Eelco Dolstra 62805dd73c Queue: Remove the scheduling priority 
Scheduling is mostly based on jobset shares these days. So showing and
sorting by priority just wastes space and gives the incorrect
impression that Hydra executes builds in the order shown on the queue
page.
 2015-02-26 13:16:25 +01:00
Eelco Dolstra 7f6faee08a Show some eval stats 2015-02-26 13:11:33 +01:00
Eelco Dolstra f835ae492f Likewise remove eval IDs from eval lists 2015-02-26 13:06:53 +01:00
Eelco Dolstra c04c8c2bf1 Jobset job tab: Show eval date rather than eval ID 
Eval IDs carry no useful information, so it's better to show the date
of the eval. If the date is recent, a relative date is given (e.g. "3h
ago").
 2015-02-26 13:01:01 +01:00
Eelco Dolstra fa1e043526 Make input value wider 
Fixes #187.
 2015-02-25 17:42:47 +01:00
Eelco Dolstra adc72d2409 Record which build a failed build step was propagated from 2015-02-25 16:42:32 +01:00
Eelco Dolstra 48af914e28 When propagating failure, propagate the duration and machine 
Previously the duration would just show as "0" and you would have to
search for the original build to get the duration.
 2015-02-25 16:21:54 +01:00
Eelco Dolstra 2f946bdfd7 Render machine correctly if it doesn't contain @ 2015-02-25 16:21:29 +01:00
Eelco Dolstra 9282bcb95f Show eval input changes in deterministic order 2015-02-25 13:13:12 +01:00
Eelco Dolstra 7c048cbbcf Allow a different state directory for hydra-server 
This is necessary if hydra-server runs under a different uid than the
other Hydra processes.
 2015-02-24 11:40:24 +01:00
Eelco Dolstra 00e03b9db5 For consistency, Use base32 hashes in manifests 2015-02-19 12:44:52 +01:00
Eelco Dolstra 69adcd4be1 Update binary cache signing to Nix 1.9 2015-02-17 20:15:37 +01:00
Eelco Dolstra ea815d55b8 More robot exclusions 2015-01-13 13:50:13 +01:00
Eelco Dolstra 80116b648e Exclude robots from all of /build 2015-01-13 13:45:39 +01:00
Eelco Dolstra a12135fc51 Don't use Perl's -w flag 2014-12-12 17:39:52 +01:00
Eelco Dolstra 6a0b9a3476 hydra-build: Handle new trace messages 
Now build step duration no longer includes network overhead (i.e.
time to copy closures to the build machine).
 2014-12-12 17:20:50 +01:00
Eelco Dolstra c0ca5489e1 Don't use given/when 
These give warnings in Perl >= 5.18:

  given is experimental at /home/hydra/src/hydra/src/lib/Hydra/Helper/CatalystUtils.pm line 241.
  when is experimental at /home/hydra/src/hydra/src/lib/Hydra/Helper/CatalystUtils.pm line 242.
  ...
 2014-12-12 11:27:17 +01:00
Shea Levy e56e743f97 S3Backup: Create a proper nar 2014-12-10 23:06:52 -05:00
Shea Levy 208bbbb4b0 s3backup: Create temporary file in temp dir 2014-12-10 22:06:32 -05:00
Domen Kožar 8de15ce38d make sure status images always display for a list of builds 2014-12-03 13:54:22 +01:00
Eelco Dolstra 9e00d98d34 Shut up some Perl 5.20 warnings 2014-11-25 00:27:52 +01:00
Eelco Dolstra 5e265e6739 Tweak blame message 2014-11-19 15:24:31 +01:00
Eelco Dolstra 7b5b434014 Fix getResponsibleAuthors 2014-11-19 15:22:30 +01:00
Eelco Dolstra 0d28e9372b Remove dead code 2014-11-19 14:59:36 +01:00
Eelco Dolstra 2d46741840 Fix message 2014-11-19 14:52:42 +01:00
Eelco Dolstra 8523130ebb Use Email::MIME instead of Email::Simple 
Email::Simple cannot handle non-ASCII characters.

Fixes #191.
 2014-11-19 14:45:46 +01:00
Eelco Dolstra 41bc918382 Fix robots.txt 2014-11-18 14:55:52 +01:00
Rob Vermaas d5db1d3bc1 Revert "Make evaluation fail with proper error when a input of type build is not available." 
This reverts commit a1b5020562.
 2014-11-18 11:13:34 +01:00
Rob Vermaas fd36d79581 Fix Changes section on build page. Now shows the changes of the first evaluation a build was part of compared to the previous one. Also separated previous builds from the changes section, as that was confusing sometimes. 2014-11-18 11:00:28 +01:00
Rob Vermaas a1b5020562 Make evaluation fail with proper error when a input of type build is not available. 2014-11-18 11:00:28 +01:00
Eelco Dolstra c503876a7e Don't cache nix-cache-info very long 2014-11-06 15:30:24 +01:00
Eelco Dolstra c250407f3c Set Expires headers for Hydra's binary cache 
This allows caching by reverse proxies.
 2014-11-06 14:31:48 +01:00
Rob Vermaas 55eb55478d Allow HipChat notification on all builds, in stead of status changes only. 2014-10-21 07:50:12 +00:00
Eelco Dolstra aad3a11eca Build page: Show output / closure sizes 2014-10-07 18:12:00 +02:00
Eelco Dolstra 83d4930101 Reorder 2014-10-07 17:16:41 +02:00
Eelco Dolstra 52ec8abf97 Update flot to 0.8.3 2014-10-07 13:35:31 +02:00
Eelco Dolstra dfe5325490 Scale the Y axis to the visible points 2014-10-07 13:10:10 +02:00
Eelco Dolstra 1b12daa032 Move the build time chart to the job page 2014-10-07 12:59:09 +02:00
Eelco Dolstra 0bb027e633 Move the store path size chart to the job page 
The history is a property of the job, not individual builds.
 2014-10-07 11:53:17 +02:00
Eelco Dolstra 5f474b252c Add a chart to the job pages showing the closure size over time 2014-10-07 11:23:15 +02:00
Eelco Dolstra 3687c9c30b Validate build IDs 2014-10-01 17:05:39 +02:00
Eelco Dolstra 5a7efc0469 Fix aggregate handling 2014-10-01 15:34:05 +02:00
Eelco Dolstra 3f51ac6603 Partially revert 1c20cfdf24 
Not clear why update-dbix messed up the Jobsets and Projects
classes. This will probably happen again in the future...
 2014-10-01 15:28:18 +02:00
Eelco Dolstra 9b38b5f134 Remove the longDescription field 
It's not useful and takes up a lot of space.
 2014-09-30 15:44:09 +02:00
Eelco Dolstra f4acc9a522 Create Builds with iscurrent set 
This should eliminate a subsequent update.
 2014-09-30 15:44:09 +02:00
Eelco Dolstra 5b4de2dee6 hydra-evaluator: Reduce verbosity 2014-09-30 15:44:08 +02:00
Eelco Dolstra 09a96c642a hydra-eval-jobs: Use JSON instead of XML 
XML::Simple is pretty slow - reading the output for the Nixpkgs jobset
takes half a minute or so. JSON is pretty much instantaneous.
 2014-09-30 15:44:08 +02:00
Eelco Dolstra d9a5143fcb Use partial indexes 
There is no point in indexing rows with common column values like
"finished = 1", since those are the majority of the table. Only the
exceptions ("finished = 0") are interesting. Having smaller tables
should make updates/insertions faster.
 2014-09-30 15:44:08 +02:00
Eelco Dolstra 773c7e89cf Get rid of some obsolete indexes 2014-09-30 15:44:08 +02:00
Eelco Dolstra 1c20cfdf24 Drop the errorMsg column in the Jobs table 
We're not using it anywhere.
 2014-09-29 19:46:11 +02:00
Eelco Dolstra beb94d5fb3 Machine status page: Add link to build step 2014-09-25 20:24:55 +02:00
Eelco Dolstra 2054759fdf Status page: Show running builds rather than build steps 
The active build steps list was redundant, because the "Machine
status" page also shows them. So instead show active builds.
 2014-09-25 20:17:33 +02:00
Eelco Dolstra 23a9fea26c Remove the ability to add multiple jobset input alternatives 
Now each jobset input has exactly one value. (Actually, adding
multiple values was already broken, so all this does is clean up the
UI...)
 2014-09-25 16:43:17 +02:00
Eelco Dolstra c28e7ac692 Fix XML wellformedness 2014-09-25 16:06:07 +02:00
Eelco Dolstra b04c8adbd5 build.tt: Show jobset eval inputs instead of build inputs 
We're not recording build inputs anymore, so we now just show all
inputs of the build's first evaluation.
 2014-09-25 14:49:09 +02:00
Eelco Dolstra 01f4037d6f hydra-eval-jobs: Don't keep track of used inputs 
We no longer store this in the database, so it's not necessary for
hydra-eval-jobs to do this.
 2014-09-25 13:38:43 +02:00
Eelco Dolstra 6284fd540d Disallow multiple jobs with the same name 
This has been deprecated since a8db329839.

Issue #60.
 2014-09-24 18:12:59 +02:00
Eelco Dolstra 0d5a38a40b Really fix hydra-eval-jobs 
Fixes #183.
 2014-09-24 15:55:19 +02:00
Eelco Dolstra b11cb37044 Handle derivations without a system attribute 2014-09-22 16:53:40 +02:00
Eelco Dolstra eb2f3eb8d4 Fix build 2014-09-22 16:47:02 +02:00
Eelco Dolstra 4727165832 Use pkgconfig to find Nix 2014-09-18 12:24:05 +02:00
Eelco Dolstra c1bf3bb0f2 hydra-eval-jobs: Add --dry-run option 2014-09-12 14:30:01 +02:00
Eelco Dolstra 748c3409b4 Don't maintain BuildInputs anymore 
We don't need to record inputs per build anymore because we have
JobsetEvalInputs now.
 2014-09-06 19:06:07 +02:00
aszlig 9c7f303255 Use mktemp for tempdir creation in prefetchers. 
This incorporates the following two commits from <nixpkgs>:

NixOS/nixpkgs@f83af95f8a
NixOS/nixpkgs@5e7a1cf955

Hydra was the original reason why I was fixing tempdir creation in the
first place. Seeing that Hydra ships its own versions of these scripts,
we need to patch them here as well.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
 2014-08-29 12:40:11 +02:00
Eelco Dolstra 2f7e928f4e Bump default silent timeout to 2 hours 2014-08-25 11:35:56 +02:00
Eelco Dolstra d6b61f4925 More charset hackery 2014-08-23 16:39:20 +02:00
Sander van der Burg ede4a10353 Compose release name for an evaluation if none has been defined 2014-08-23 16:20:58 +02:00
Eelco Dolstra 319e053368 Hopefully fix UTF-8 chars in Hipchat notification messages 2014-08-18 14:01:39 +02:00
Eelco Dolstra 2ffceef875 Tweak 2014-08-17 23:54:12 +02:00
Eelco Dolstra 28b528c1ab Ensure proper escaping of /build/.../contents pages 2014-08-17 23:43:22 +02:00
Eelco Dolstra f50066ab6f Fix encoding of Hydra logs 2014-08-17 23:43:01 +02:00
Eelco Dolstra a80bfceaca Remove timeout detection hack 2014-08-17 19:26:03 +02:00
Eelco Dolstra 42b23133e8 Fix UTF-8 handling of log files 2014-08-13 18:53:29 +02:00
Eelco Dolstra a2b27c7cf2 Preserve whitespace in string inputs 2014-08-13 17:25:08 +02:00
Eelco Dolstra 9334b84d6f Fix registerRoot 
Fixes #175.
 2014-08-13 16:29:00 +02:00
Eelco Dolstra 3520315fea Fix build and handling of string inputs starting with a dash 
Fixes #176.
 2014-08-13 16:24:26 +02:00
Eelco Dolstra 69e3aa0438 Write Hydra roots as regular files instead of symlinks 
Note that this requires at least NixOS/Nix@1c208f2b7e.
 2014-08-01 17:24:55 +02:00
Ronny Esterluss 7351752066 fixed string comparison 2014-07-31 22:18:28 +02:00
Tino Breddin 5ebc2e5419 only checkout the target branch when topgit is used 2014-07-31 22:18:28 +02:00
Ronny Esterluss 55aaf807eb added clean command 2014-07-31 21:53:25 +02:00
Ronny Esterluss ee840be313 forcing branch switch 2014-07-31 21:53:25 +02:00
Eelco Dolstra 6b88be040e Make restartBuilds faster 2014-07-18 00:03:26 +02:00
Eelco Dolstra 365de86ead Fix hydra-update-gc-roots 2014-07-16 23:20:58 +02:00
Eelco Dolstra fb5f01097b Fix race between hydra-eval-jobs and hydra-update-gc-roots 
If hydra-eval-jobs creates a new root, and hydra-update-gc-roots runs
before hydra-evaluator has had a chance to add the corresponding build
to the database, then hydra-update-gc-roots will remove the root. If
subsequently the Nix garbage collector kicks in, it may remove the
build's .drv file before the build is performed. Since evaluation of
the Nixpkgs and NixOS jobsets nowadays takes a lot of time (e.g. an
hour), the probability of this happening is fairly high.

The quick fix is not to delete roots that are less than a day old. So
long as evaluation doesn't take longer than a day, this should be fine
;-)

Fixes #166.
 2014-07-14 13:18:07 +02:00
Shea Levy 114f8a26ee Fix build for new nixUnstable 2014-06-16 12:24:17 -04:00
Rob Vermaas 268f273a3e Add value and dependency fields to eval json output. 2014-05-08 17:08:31 +02:00
Eelco Dolstra da73eb5f8a Include dependency column in JSON output 2014-05-08 16:36:15 +02:00
Rob Vermaas 1c92d760fa Merge pull request #161 from thoughtpolice/coverity 
hydra: add Coverity Scan plugin
 2014-05-03 17:46:35 +02:00
Austin Seipp 4166974657 hydra: add Coverity Scan plugin 
This adds a Hydra plugin for users to submit their open source projects
to the Coverity Scan system for analysis.

First, add a <coverityscan> section to your Hydra config, including the
access token, project name, and email, and a regex specifying jobs to
upload:

    <coverityscan>
      project = testrix
      jobs    = foobar:.*:coverity.*
      email   = aseipp@pobox.com
      token   = ${builtins.readFile ./coverity-token}
    </coverityscan>

This will upload the scan results for any job whose name matches
'coverity.*' in any jobset in the Hydra 'foobar' project, for the
Coverity Scan project named 'testrix'.

Note that one upload will occur per job matched by the regular
expression - so be careful with how many builds you upload.

The jobs which are matched by the jobs specification must have a file in
their output path of the form:

  $out/tarballs/...-cov-int.(xz|lzma|zip|bz2|tgz)

The file must have the 'cov-int' directory produced by `cov-build` in
the root.

(You can also output something into
$out/nix-support/hydra-build-products for the Hydra UI.)

This file will be found in the store, and uploaded to the service
directly using your access credentials. Note the exact extension: don't
use .tar.xz, only use .xz specifically.

Signed-off-by: Austin Seipp <aseipp@pobox.com>
 2014-05-02 14:15:25 -05:00
Eelco Dolstra 1306291c6c Handle utf-8 properly 
Fixes errors like:

  Caught exception in engine "Wide character in syswrite at /nix/store/498lwsrn5kkdh1q8kn3vcpd3457w6m7a-hydra-perl-deps/lib/perl5/site_perl/5.16.3/Starman/Server.pm line 547."

Note that these errors didn't happen if the database encoding was set
to SQL_ASCII (which was the case for hydra.nixos.org, explaining why
it didn't get these errors). However, now the encoding must be
UTF8. To change it, do:

  update pg_database set encoding = pg_char_to_encoding('UTF8') where datname = 'hydra';
 2014-05-01 16:33:25 +02:00
Eelco Dolstra 67c79a16a8 Remove unnecessary join against releases table 
This gets rid of the warning:

  DBIx::Class::Storage::DBI::select_single(): Query returned more than one row.  SQL that returns multiple rows is DEPRECATED for ->find and ->single at /home/eelco/Dev/hydra/src/script/../lib/Hydra/Controller/Project.pm line 15
 2014-05-01 13:03:34 +02:00
Eelco Dolstra 6c6ce7a781 Fix cloning jobsets 2014-04-30 09:26:53 +02:00
Eelco Dolstra 4d9edd0ca8 Also redirect /job requests that refer to a renamed jobset 2014-04-24 11:05:16 +02:00
Eelco Dolstra 23416f9960 Do a permanent redirect 2014-04-24 10:52:46 +02:00
Eelco Dolstra affec8881d Remove support for views 
They're replaced by aggregates, which are declarative, faster, and
have a better interface.
 2014-04-23 23:22:44 +02:00
Eelco Dolstra 5e0d10a392 When renaming a jobset, add a redirect from the old name 2014-04-23 23:15:22 +02:00
Eelco Dolstra be63c50560 Show whether a job still exists 
In the dashboard and on the job page, indicate whether the job appears
in the latest jobset eval. That way, the user gets some indication if
a job has accidentally disappeared (e.g. due to an evaluation error).
 2014-04-08 18:50:37 +02:00
Ludovic Courtès 4471cae07e hydra-eval-guile-jobs: Add "This file is part of Hydra". 2014-04-08 18:22:21 +02:00
Ludovic Courtès cc46456598 hydra-eval-guile-jobs: Register derivations as GC roots. 
* src/script/hydra-eval-guile-jobs.in (register-gc-root): New
  procedure.
  (job-evaluations->sxml): Add #:gc-roots-dir parameter.  Call
  'register-gc-root'.
  (job-evaluations->xml): Add #:gc-roots-dir parameter; pass it to
  'job-evaluations->sxml'.
  (eval-guile-jobs): Warn when --gc-roots-dir isn't passed.  Pass
  GC-ROOTS-DIR to 'job-evaluations->xml'.
 2014-04-08 18:22:21 +02:00
Ludovic Courtès e3e8c1bc74 hydra-eval-guile-jobs: Honor $NIX_STORE_DIR. 
* src/script/hydra-eval-guile-jobs.in (strip-store-path): Honor
  $NIX_STORE_DIR.
 2014-04-08 18:22:21 +02:00
Eelco Dolstra 4cf0d7c4b3 hydra-update-gc-roots: Keep derivations of failed builds 
By keeping the derivations of failed builds in the most recent
evaluations, we ensure that failed builds can be restarted.
 2014-04-08 17:54:11 +02:00
Eelco Dolstra 4aa01e021e Fix building against the latest Nix 2014-04-08 17:08:09 +02:00
Shea Levy d087aeb0d9 Fix adding jobset inputs 2014-03-06 08:22:59 -05:00
Shea Levy 024a5e7d75 edit-jobset.tt: Retain old inputs with dashes in their name 
Fixes #149
 2014-03-06 07:50:49 -05:00
Eelco Dolstra 2d8f2503c5 Build against latest Nix 2014-03-05 11:11:59 +01:00
Ludovic Courtès ccdb6f2b23 Honor $NIX_STORE_DIR. 2014-02-28 14:09:12 +01:00
Your Name f409d2b4f6 Allow notify option to be set for hipchat notification. (https://www.hipchat.com/docs/api/method/rooms/message) 2014-02-28 09:27:33 +00:00
Your Name 421950aede Use minutes in build history charts. 2014-02-28 09:16:45 +00:00
Ludovic Courtès 61448ca2bd guile: Raise default absolute timeout to from 2h to 20h. 2014-02-27 22:19:52 +01:00
Shea Levy b29d7b1091 Only email responsible authors when the build has changed 2014-02-26 11:05:46 -05:00
Eelco Dolstra 8dcef7702f Add jobset/job menus on buid log pages 2014-02-26 11:49:28 +01:00
Eelco Dolstra 66b8c1a9e0 Return 410 Gone (rather than 500) if an output is no longer available 2014-02-26 11:38:02 +01:00
Rob Vermaas 6327edd63f Add a convenient way to get logs of a path/drv. Requested by phreedom. 2014-02-19 10:21:59 +00:00
Eelco Dolstra 73a0cb57c5 Jobset evals page: Don't show nrFailed if it's zero 2014-02-18 11:08:12 +01:00
Hydra d4d9896f9f Re-add the history tab (store size and build time), now as lazy-loaded tab on build page. 2014-01-28 11:55:19 +01:00
Rob Vermaas ac177d0984 Add nixexprpath/nixexprinput to GET of jobset (API). 2014-01-23 12:44:02 +01:00
Eelco Dolstra 2398f2ce49 Spelling fix 2014-01-15 16:09:02 +01:00
Eelco Dolstra 492fe121ea Jobset eval page: Fix handling of aborted/cancelled builds 
The fix in 85d51074 was not quite right.
 2014-01-15 16:06:47 +01:00
Eelco Dolstra cad2c07a35 Fix broken redirects 2014-01-14 18:17:33 +01:00
Eelco Dolstra 6572869596 clear-queue-non-current: Work around PostgreSQL query optimizer stupidity 2014-01-14 18:14:10 +01:00
Eelco Dolstra 62809f484f hydra-eval-jobs: Fix building against the latest unstable Nix 2014-01-12 17:37:56 +01:00
Rob Vermaas 1d392f64c6 Fix typo. 2014-01-10 15:20:19 +01:00
Rob Vermaas bd7b6fc401 Allow Hydra to run as a private instance by requiring a login. 
Use the following in your hydra.conf to make your instance a
private Hydra instance (public is the default):

  private 1

Currently, this will not allow you to use the API, channels
and the binary cache when running in private mode. We will add
solutions for these functionalities later.
 2014-01-10 11:04:28 +01:00
Rob Vermaas 3e1a98668f Allow configuring a set of domains to allow logins from Persona. 
E.g. add the following to only allow Persona logins from email.com and gmail.com email addresses.

  persona_allowed_domains email.com,gmail.com
 2014-01-09 13:31:02 +01:00
Eelco Dolstra a598fe7e81 Allow Hydra's binary cache to be signed 
This requires adding the following to hydra.conf:

  binary_cache_key_name = <key-name>
  binary_cache_private_key_file = <path-to-private-key>

e.g.

  binary_cache_key_name = hydra.nixos.org-1
  binary_cache_private_key_file = /home/hydra/cache-key.sec
 2014-01-08 15:19:17 +01:00
Shea Levy de26b55afe User.pm: Handle params from JSON properly 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-12-12 09:38:57 -05:00
Rob Vermaas f25f8dbfa3 Higher timeout for prefetching bazaar inputs. 2013-12-11 17:34:45 +01:00
Shea Levy a92a57f3b0 hydra-evaluator: When evaluating a single jobset, exit with a failure code if evaluation fails 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-11-27 14:29:05 -05:00
Eelco Dolstra dd4e57fb0c Allow passing a specific build as an input 
Fixes #62.
 2013-11-11 21:36:26 +00:00
Eelco Dolstra 8f104396ec Support passing a jobset evaluation as an input 
All successful, non-garbage-collected builds in the evaluation are
passed in a attribute set.  So if you declare a Hydra input named
‘foo’ of type ‘eval’, you get a set with members ‘foo.<jobname>’.  For
instance, if you passed a Nixpkgs eval as an input named ‘nixpkgs’,
then you could get the Firefox build for x86_64-linux as
‘nixpkgs.firefox.x86_64-linux’.

Inputs of type ‘eval’ can be specified in three ways:

* As the number of the evaluation.

* As a jobset identifier (‘<project>:<jobset>’), which will yield the
  latest finished evaluation of that jobset.  Note that there is no
  guarantee that any job in that evaluation has succeeded, so it might
  not be very useful.

* As a job identifier (‘<project>:<jobset>:<job>’), which will yield
  the latest finished evaluation of that jobset in which <job>
  succeeded.  In conjunction with aggregate jobs, this allows you to
  make sure that the evaluation contains the desired builds.
 2013-11-11 21:17:22 +00:00
Eelco Dolstra 7b35e4d0de Add a link to the latest finished eval of a jobset 2013-11-11 14:49:39 +01:00
Eelco Dolstra a0fdf8d2ba Use local copy of the Persona sign in button 2013-11-06 22:20:57 +01:00
Eelco Dolstra 4fb3d67f49 Make /login give a proper JSON response again 2013-11-06 16:12:07 +01:00
Eelco Dolstra 60e7e4fbe9 Remove obsolete/broken timeline feature 2013-11-06 15:20:30 +01:00
Eelco Dolstra e8d92445d7 Keep sessions for a week 2013-11-06 15:15:35 +01:00
Eelco Dolstra 5723fda7fc Keep session state across hydra-server restarts 2013-11-06 15:14:31 +01:00
Eelco Dolstra 33f01da507 Fix hydra_logo setting 2013-11-06 15:10:26 +01:00
Eelco Dolstra df1a33c63f Fix layout at all page sizes 2013-11-06 14:50:21 +01:00
Eelco Dolstra 3c45992182 Make the logo configurable via hydra.conf 2013-11-06 14:40:30 +01:00
Eelco Dolstra b44baf3746 Add a flag to enable Persona support 
It's disabled by default.  To enable, add the following to hydra.conf:

  enable_persona = 1
 2013-11-06 14:38:36 +01:00
Eelco Dolstra d118a575e1 Use page-header on the front page 2013-11-06 14:11:56 +01:00
Eelco Dolstra 8928c35ecd Slight layout tweak 2013-11-06 14:07:43 +01:00
Eelco Dolstra 55f9d23933 Add a command `hydra-create-user' for managing user accounts 2013-11-06 13:36:29 +01:00
Eelco Dolstra 3315d1ea51 Remove obsolete hydra-control script 2013-11-06 11:42:49 +00:00
Eelco Dolstra 7a18e5f0c1 Remove unused file 2013-11-06 11:42:04 +00:00
Eelco Dolstra 63100b0ee8 Merge branch 'master' into persona 2013-11-06 12:35:36 +01:00
Eelco Dolstra 27a740d4b1 Don't show an empty projects list 2013-11-06 12:15:11 +01:00
Eelco Dolstra a0b58b6b62 Fix typo that caused empty news item to show up 2013-11-06 12:15:11 +01:00
Eelco Dolstra ecadcef642 Prevent a division by zero in hydra-queue-runner 
Fixes #131.
 2013-11-06 12:15:11 +01:00
Shea Levy f1c187198c Stupid capitalization scheme 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-11-05 12:39:55 -05:00
Shea Levy 2e1c07c5f7 Argh 
ALWAYS RUN THE TESTS

Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-11-05 12:31:36 -05:00
aszlig cfd4843290 topbar: Include "Create project" in menu. 
For users who only have the "create-projects" role, actually display the
item in the menu as the only option.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
 2013-11-05 17:03:31 +01:00
aszlig f6a83bcc5b user: Fix spelling of create-projects role. 
This actually is right in the manual but displayed incorrectly on the
web interface.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
 2013-11-05 17:03:31 +01:00
Eelco Dolstra 8a0200edd2 Doh 2013-11-05 16:13:09 +01:00
Eelco Dolstra ae85d2df23 Add a "My jobsets" tab to the dashboard 2013-11-05 16:05:29 +01:00
Shea Levy d5574230e4 typo 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-11-05 09:57:15 -05:00
Shea Levy 217ba725a3 Restore accidentally-removed fields to the Builds API 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-11-05 09:43:32 -05:00
Shea Levy 385ceaff4d Show buildinput and buildproduct information in the Builds API 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-11-05 09:42:05 -05:00
Eelco Dolstra 589271bd72 In the "My jobs" tab, ignore disabled projects/jobsets 2013-11-05 14:53:52 +01:00
Eelco Dolstra 27b795f46e Doh 2013-11-05 14:46:20 +01:00
Eelco Dolstra fc827a003a Be paranoid about the Persona email address 2013-11-05 14:40:40 +01:00
Eelco Dolstra b53bd868fb Add a "My jobs" tab to the dashboard 
It shows all current jobs of which the user is a maintainer.
 2013-11-05 14:35:49 +01:00
Eelco Dolstra 3c981bed02 Slight cleanup in the Persona sign in code 2013-11-05 14:10:20 +01:00
Eelco Dolstra bdd6e99fbd Clean up user editing 2013-11-05 14:02:04 +01:00
Eelco Dolstra bd927eba66 Make sign in a modal dialog box rather than a separate page 2013-11-05 13:13:02 +01:00
Eelco Dolstra 299ac07cbc Re-enable adding new users via the web interface 
Also unify user creation and user editing.
 2013-11-05 12:41:10 +01:00
Eelco Dolstra 36c961a376 Store the account type ("hydra" or "persona") explicitly in the database 2013-11-05 11:46:05 +01:00
Eelco Dolstra 14b9b201e1 Don't show a password reset button for Persona accounts 2013-11-05 11:18:44 +01:00
Eelco Dolstra b54cfbf032 Merge branch 'master' into persona 
Conflicts:
	src/lib/Hydra/Helper/CatalystUtils.pm
	src/root/layout.tt
	src/root/topbar.tt
	src/root/user.tt
 2013-11-05 11:11:48 +01:00
Eelco Dolstra 32b6ddf772 hydra-eval-jobs: Support meta.license being a list 
TODO: PostgreSQL has an array column type, we should use it for the
license and maintainers columns.
 2013-11-04 22:50:32 +01:00
Eelco Dolstra 647d6bc3f1 Force creation of a new uncached eval if jobs have been removed 
Previously we only checked if jobs had been added.  We should probably
rename the "hasNewBuilds" field in the database.
 2013-11-01 19:24:52 +01:00
Eelco Dolstra a04c117eb6 Revert "Remove wacky "sysbuild" filtering" 
This reverts commit 2d7e106d29.

Unfortunately some jobsets still depend on this behaviour.  They could
probably do something like "assert system == input.system; ..." but
changing them all is undesirable.
 2013-11-01 18:30:36 +01:00
Eelco Dolstra cbd56824a2 Use /etc/nix/machines 2013-10-30 11:21:23 +01:00
Eelco Dolstra a919b503fb Restore api/get-info 2013-10-29 19:53:23 +01:00
Shea Levy 5da3dcfd57 Make the outputs' outpaths available via the Build JSON API 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-29 14:41:48 -04:00
Shea Levy 4eef090a45 Flesh out the API a bit 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-29 13:54:11 -04:00
Shea Levy b51b8cddf4 Add in missing needed join 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-25 14:06:52 -04:00
Shea Levy 8e82ba0093 Get rid of old ToJSON code path 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-25 11:09:11 -04:00
Shea Levy a89096ec5e Clean up root controller 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-25 11:09:11 -04:00
Shea Levy 5d3df69a0b Cleanup build serialization 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-25 11:09:11 -04:00
Shea Levy e04477003e Cleanup eval serialization 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-25 11:09:11 -04:00
Shea Levy 917660442e Add eager fetching of relations and enable that for jobset->jobsetinputs 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-25 11:09:11 -04:00
Shea Levy ec6568f9b7 Fix jobset input handling in the API 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-25 11:09:06 -04:00
Shea Levy 04d8adaad3 Cleanup Jobset serialization 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-24 11:39:40 -04:00
Shea Levy 78dcd3e619 Cleanup Project model 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-24 11:39:40 -04:00
Shea Levy ee8275cfdb Start api cleanup with the User model 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-24 11:39:40 -04:00
Eelco Dolstra f551de21f7 Fix creating jobsets in a RESTful way 2013-10-24 17:35:58 +02:00
Eelco Dolstra c719bebcb4 Remove redundant newlines 2013-10-17 11:20:27 +02:00
Shea Levy 806af3fe68 Fix typos 
I could've sworn I tested this...

Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-16 08:57:48 -04:00
Shea Levy 0db950931a Merge remote-tracking branch 'upstream/who-broke-builds' into upstream-master 
Include information about who changed the build status in notification
emails, and enable optional per-input notification of said committers.

Conflicts due to two branches modifying the database schema.

Signed-off-by: Shea Levy <shea@shealevy.com>

Conflicts:
	src/lib/Hydra/Schema/Jobsets.pm
	src/sql/upgrade-23.sql
 2013-10-15 09:49:20 -04:00
Eelco Dolstra 2127d133cd Add a dashboard 
Currently the dashboard allows users to get a quick overview of the
status of jobs they're interested in, but more will be added,
e.g. viewing all your jobsets or all jobs of which you're a
maintainer.
 2013-10-14 20:07:26 +02:00
Eelco Dolstra 09b5679ee7 Clean up the authorization code a bit 2013-10-14 18:01:04 +02:00
Eelco Dolstra 86e9abeb15 Use c.req.captures 2013-10-14 17:43:31 +02:00
Eelco Dolstra 854d419b2c Use redirectJSON 2013-10-14 17:35:14 +02:00
Eelco Dolstra c4e39d4769 Add one-shot jobsets 
There are jobsets that are evaluated only once, that is, after they've
been evaluated, they're disabled automatically.  This is primarily
useful for doing releases: for instance, doing an evaluation with
"officialRelease" set to "true" should be done only once.
 2013-10-11 12:01:52 +02:00
Eelco Dolstra f592ce0026 Fix extreme slowness in hydra-queue-runner 
If there are builds in the queue that depend on another scheduled
build, then hydra-queue-runner will start the dependency first and
block the dependent builds.  This is implemented in
findBuildDependencyInQueue.  However, if there are tens of thousands
of such dependent builds, since each call to
findBuildDependencyInQueue may take a second or so, hydra-queue-runner
will spend hours just deciding which builds *not* to do.  Thus very
little progress is made.

So now, when a build is started, we immediately check which builds are
"blocked" by it (i.e. depend on it), and remove such builds from
consideration.
 2013-10-11 10:54:02 +02:00
Eelco Dolstra a49457b2fd Don't break inside durations 2013-10-10 12:43:27 +02:00
Shea Levy 26470f1656 Check all inputs for blame but only email selected inputs 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-08 14:47:24 -04:00
Shea Levy 804617f075 Email responsible authors if requested 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-08 13:24:49 -04:00
Shea Levy f8b80c99c2 Include who-broke-the-build information in notification emails 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-08 13:24:49 -04:00
Shea Levy 272d9e235d Remove unused assignment 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-08 13:24:49 -04:00
Shea Levy 2c90857689 getResponsibleAuthors: Respect checkResponsible 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-08 13:24:49 -04:00
Shea Levy 58ad3b4b6c Enable setting checkresponsible in the edit jobset form 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-08 13:24:49 -04:00
Shea Levy 3e1f930928 Enable setting emailresponsible in the edit jobset form 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-08 13:24:49 -04:00
Shea Levy 3e4a4e3761 Propagate checkresponsible from JobsetInput to BuildInput 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-08 13:24:49 -04:00
Shea Levy 07157f8125 Update Schema classes 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-08 13:24:49 -04:00
Shea Levy 6342464110 Add DB columns for when to notify responsible committers and which inputs should be checked 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-08 13:24:49 -04:00
Shea Levy 7913701d54 Factor a getResponsibleAuthors helper out of the HipChatNotification 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-10-08 13:24:49 -04:00
Rob Vermaas 20f1bf215a Make actions dropdown easier to find by making the dropdown title bold 2013-10-08 13:32:46 +02:00
Eelco Dolstra 0ec03aa0f4 Handle builds with multiple outputs correctly in Hydra channels 2013-10-07 17:06:17 +02:00
Eelco Dolstra 5ccff14f6b In Hydra channels, show only packages matching the user's system type 
Fixes NixOS/nix#169.
 2013-10-07 14:53:27 +02:00
Eelco Dolstra 5294a0a8a0 Register restarted derivations as GC roots 2013-10-04 17:11:42 +02:00
Eelco Dolstra 052bab169d Add a jobset eval action to restart all aborted/cancelled builds 2013-10-04 17:01:47 +02:00
Eelco Dolstra 85d51074b9 Jobset eval page: Show all aborted builds under the "aborted jobs" tab 
Previously some might be included under the "new jobs" tab
 2013-10-04 16:36:22 +02:00
Eelco Dolstra e334ff541d Jobset eval page: Show the number of builds in each tab 2013-10-04 16:35:56 +02:00
Eelco Dolstra aa49b128a8 Remove the "clone build" feature 2013-10-04 15:43:51 +02:00
Eelco Dolstra 7818bb75ed Add an action to cancel all builds in a jobset eval 2013-10-04 15:40:43 +02:00
Eelco Dolstra 3e54f0a6ab Fix an uninitialized value warning 2013-10-04 14:47:30 +02:00
Eelco Dolstra ee5b655535 Maintain the order of the input alternatives 2013-10-03 20:03:57 +02:00
Eelco Dolstra 550bf210fe Use more flash messages 2013-10-03 19:54:22 +02:00
Eelco Dolstra 383bc62803 Restore link to clearvcscache 2013-10-03 19:43:21 +02:00
Eelco Dolstra 720c3892a3 Use delete instead of delete_all 
DBIC's delete_all method fetches all rows separately, which is slow.
 2013-10-03 19:42:44 +02:00
Eelco Dolstra f32077b5e8 Simplify jobset cloning 
We can just show the normal "edit jobset" page for the original jobset
and then do a PUT request to create a new jobset.

Also simplified updating the jobset inputs.  We can just delete all of
them and recreate them from the user parameters.  That's safe because
it's done in a transaction.
 2013-10-03 19:28:25 +02:00
Eelco Dolstra 232f46c750 Use the REST API in the web interface for editing jobsets 2013-10-03 18:49:37 +02:00
Eelco Dolstra 63062f7bba Instead hard breaks in multi-line error messages 2013-10-03 18:05:37 +02:00
Eelco Dolstra 851c3329d0 Implement DELETE for jobsets and use it in the web interface 2013-10-03 17:54:40 +02:00
Eelco Dolstra ae13dd3e67 Use the REST API in the web interface for editing projects 2013-10-03 17:23:41 +02:00
Eelco Dolstra 8355dc57ee Ensure proper encoding of error messages 2013-10-03 17:20:00 +02:00
Eelco Dolstra acdeaa1d80 Fix tabs 2013-10-03 15:14:55 +02:00
Eelco Dolstra 97c76f34df Remove more dead code 2013-10-03 15:14:40 +02:00
Eelco Dolstra ed132cf1fc Remove obsolete JobStatus source 2013-10-03 15:05:53 +02:00
Eelco Dolstra c7a039bd05 Remove the Errors page 
The same info is readily available under the most recent jobset eval
page.
 2013-10-03 15:00:28 +02:00
Eelco Dolstra 64d617a6b2 Remove already disabled jobstatus code 2013-10-03 14:56:07 +02:00
Eelco Dolstra adcdfcde6b Remove some obsolete JSON APIs 2013-10-03 14:50:56 +02:00
Eelco Dolstra c8e5faf81e Clean up error handling 2013-10-03 14:45:23 +02:00
Eelco Dolstra 4ddb173ca1 Use notFound instead of status_not_found 2013-10-03 14:16:21 +02:00
Eelco Dolstra 2166c478ef Fix and clean up editing project settings 2013-10-03 14:05:10 +02:00
Eelco Dolstra b1f7096935 Restore old findBuildDependencyInQueue behaviour 2013-10-03 13:08:32 +02:00
Eelco Dolstra b1a26e6caa Revert "Add a dependency_lookup configuration option to enable (slow) dependency lookup in queue. This behaviour was disabled temporarily in accefbb79 due to slowness in very large queues, but some people might be dependent on it, so it is configurable until the previous behaviour is implemented more efficiently." 
This reverts commit 24f5a6b15f.
 2013-10-03 13:07:32 +02:00
Eelco Dolstra 9f3b47c963 Allow only project owners to delete projects / jobsets 
Giant gaping security hole.
 2013-10-03 13:06:16 +02:00
Eelco Dolstra 7a5ec00ea1 Fix a "Argument isn't numeric" warning if there are not jobsets 2013-10-03 13:04:20 +02:00
Eelco Dolstra 77fd72070f Fix showing the project's display name 2013-10-03 10:17:53 +00:00
Rob Vermaas 24f5a6b15f Add a dependency_lookup configuration option to enable (slow) dependency lookup in queue. This behaviour was disabled temporarily in accefbb79 due to slowness in very large queues, but some people might be dependent on it, so it is configurable until the previous behaviour is implemented more efficiently. 2013-10-03 09:09:18 +00:00
Eelco Dolstra cc1fcf657c Show aborted/cancelled builds separately in jobset eval pages 2013-10-03 01:54:42 +02:00
Eelco Dolstra 9801cb76db Add an action menu to evaluations 2013-10-03 01:34:17 +02:00
Eelco Dolstra 4fa2821eea Move more actions from the top bar 2013-10-03 01:17:52 +02:00
Eelco Dolstra a5cfae078e Remove the Build menu from the top bar 
It's now a dropdown menu in the tabs thingy, which subsumes the
"Reproduce locally" button.  This makes the actions in the menu a bit
more visible, IMHO.
 2013-10-02 19:10:00 +02:00
Eelco Dolstra f50477141d Add an input type "nix" for passing arbitrary Nix expressions 2013-09-30 12:03:25 +02:00
Eelco Dolstra 4dd1197d89 Fix uninitialized value warning 2013-09-30 10:01:09 +00:00
Eelco Dolstra af2b0c8bad Remove dead code 2013-09-30 11:57:38 +02:00
Eelco Dolstra baafe77489 Fix HTML error 
From Mats Erik Andersson.
 2013-09-30 11:18:48 +02:00
Eelco Dolstra d46ebeea99 Distinguish between permanent evaluation errors and transient input errors 
Fixes #112.
 2013-09-25 16:21:16 +02:00
Eelco Dolstra e1c9e28589 Handle UTF-8 characters in eval error messages 2013-09-25 15:51:03 +02:00
Eelco Dolstra a8db329839 Warn against multiple jobs with the same name 2013-09-25 15:30:59 +02:00
aszlig 0b202580ce templates: Use uri_for to reference static paths. 
This commit is provided by (zsh syntax):

sed -i 's|/static[^"]*|[% c.uri_for("&") %]|;s/\[% size %\]/${size}/' **/*.tt

And the reason for this change is to make it easier to change the base
path with headers like X-Request-Base to be served within a URI prefix,
especially when behind a reverse proxy.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
 2013-09-25 14:37:18 +02:00
Eelco Dolstra a2491f76a4 Use the same start/stop time for the build steps as for the build 2013-09-25 01:00:20 +02:00
Eelco Dolstra f037a318e3 *headdesk* 
DBIC::Class helpfully doesn't warn you when you're matching against
unselected columns.  So this query actually returned all builds...
 2013-09-25 01:00:20 +02:00
Eelco Dolstra 2d7e106d29 Remove wacky "sysbuild" filtering 
It's none of our business if a jobset wants to return a build that
uses a build for another system as an input...
 2013-09-25 01:00:20 +02:00
Rob Vermaas b1e29e50a7 Only send email notification of evaluation error when the evaluation error has changed. Fixes #121. 2013-09-24 12:01:57 -04:00
Eelco Dolstra 550d6b79b1 Allow dots in jobset names (like "release-13.09") 2013-09-24 15:15:44 +02:00
Shea Levy 1a313ad566 Allow dashes in jobset input names 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-09-23 17:53:35 -04:00
Shea Levy 6d5a3d0580 Derivations with multiple outputs break the 'link name is store path' assumption 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-09-22 21:26:59 -04:00
Shea Levy 98c9559bf2 Delay initializing the s3 client until it's needed to silence warnings 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-09-22 18:48:37 -04:00
Eelco Dolstra 77dbf55abb hydra-queue-runner: Tweaked the selection method 
Pick the jobset that has used the smallest fraction of its share,
rather than the jobset furthest below its share in absolute terms.
This gives jobsets with a small share a quicker start (but they
will also run out of their share quicker).
 2013-09-21 19:54:58 +00:00
Eelco Dolstra cf43c605cd hydra-queue-runner: Cache the lookup of time spent per jobset 2013-09-21 19:54:46 +00:00
Eelco Dolstra 4cdf1a270d hydra-queue-runner: Set the start time properly 2013-09-21 19:38:02 +00:00
Eelco Dolstra 52ce662710 hydra-queue-runner: Don't kill builds we just started 2013-09-21 20:51:43 +02:00
Eelco Dolstra accefbb798 hydra-queue-runner: Disable findBuildDependencyInQueue for now 
It's way too slow.
 2013-09-21 20:35:02 +02:00
Eelco Dolstra 24c757845d Fix lazy tab loading 2013-09-21 19:25:01 +02:00
Eelco Dolstra 9602499c1c hydra-evaluator: Do the actual work in a subprocess 
This should get rid of the slow memory leaks exhibited by
hydra-evaluator.
 2013-09-21 15:49:27 +00:00
Eelco Dolstra 4ed877360b hydra-queue-runner: Improved scheduling 
Each jobset now has a "scheduling share" that determines how much of
the build farm's time it is entitled to.  For instance, if a jobset
has 100 shares and the total number of shares of all jobsets is 1000,
it's entitled to 10% of the build farm's time.  When there is a free
build slot for a given system type, the queue runner will select the
jobset that is furthest below its scheduling share over a certain time
window (currently, the last day).  Withing that jobset, it will pick
the build with the highest priority.

So meta.schedulingPriority now only determines the order of builds
within a jobset, not between jobsets.  This makes it much easier to
prioritise one jobset over another (e.g. nixpkgs:trunk over
nixpkgs:stdenv).
 2013-09-21 14:57:01 +00:00
Eelco Dolstra 7efe793ee6 Fix the Darcs schema upgrade 2013-09-19 12:00:29 +00:00
Shea Levy 74388353b5 Add a plugin for backing up builds in s3 
In your hydra config, you can add an arbitrary number of <s3config>
sections, with the following options:

* name (required): Bucket name
* jobs (required): A regex to match job names (in project:jobset:job
  format) that should be backed up to this bucket
* compression_type: bzip2 (default), xz, or none
* prefix: String to prepend to all hydra-created s3 keys (if this is
  meant to represent a directory, you should include the trailing slash,
  e.g. "cache/"). Default "".

After each build with an output (i.e. successful or failed-with-output
builds), the output path and its closure are uploaded to the bucket as
.nar files, with corresponding .narinfos to enable use as a binary
cache.

This plugin requires that s3 credentials be available. It uses
Net::Amazon::S3, which as of this commit the nixpkgs version can
retrieve s3 credentials from the AWS_ACCESS_KEY_ID and
AWS_SECRET_ACCESS_KEY environment variables, or from ec2 instance
metadata when using an IAM role.

This commit also adds a hydra-s3-backup-collect-garbage program, which
uses hydra's gc roots directory to determine which paths are live, and
then deletes all files except nix-cache-info and any .nar or .narinfo
files corresponding to live paths. hydra-s3-backup-collect-garbage
respects the prefix configuration option, so it won't delete anything
outside of the hierarchy you give it, and it has the same credential
requirements as the plugin. Probably a timer unit running the garbage
collection periodically should be added to hydra-module.nix

Note that two of the added tests fail, due to a bug in the interaction
between Net::Amazon::S3 and fake-s3. Those behaviors work against real
s3 though, so I'm committing this even with the broken tests.

Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-09-18 18:32:58 +02:00
Eelco Dolstra 4705af48b8 hydra-build: Hack to handle timeouts 2013-09-18 13:06:35 +00:00
Eelco Dolstra e4cbd057e2 Fix an uninitialized value warning 2013-09-18 12:51:51 +00:00
Eelco Dolstra e54b536bb7 hydra-update-gc-roots: Don't keep the most recent successful view result 
Views are deprecated.
 2013-09-18 11:12:33 +00:00
Eelco Dolstra 2845d46d21 hydra-update-gc-roots: Keep more evals 
We now keep *all* unfinished evaluations of a jobset, in addition to
the <keepnr> most recent finished evaluations.

The main motivation is to ensure that mirror-{nixos,nixpkgs} work
properly: if building an evaluation takes too long, some of its builds
may already have been garbage-collected by the time the others finish.
 2013-09-18 11:10:10 +00:00
Eelco Dolstra 3b70a807b5 Allow setting keepnr to 0 2013-09-18 11:27:37 +02:00
Eelco Dolstra c667b9112d Don't trigger evaluation of disabled jobsets 2013-09-18 11:25:52 +02:00
Eelco Dolstra c14a333062 Revive release button on eval page 2013-09-17 16:39:50 +02:00
Rob Vermaas 597fd827b1 Merge pull request #81 from mornfall/master 
DARCS support
 2013-09-14 06:18:48 -07:00
Eelco Dolstra 3f68076577 hydra-build: Don't send a giant query to the database 
We had Postgres barfing with this error:

  DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR: stack depth limit exceeded

because the ‘drvpath => [ @dependentDrvs ]’ in failDependents can
cause a query of unbounded size.  (In this specific case there was a
failure of Bison, which has > 10000 dependent derivations.)  So now we
just get all scheduled builds from the DB.
 2013-09-10 11:01:29 +00:00
Eelco Dolstra 35aad40692 Kill builds that produce more than 64 MiB of log output 2013-09-10 10:33:55 +00:00
Petr Rockai 0041c336d1 DarcsInput: Avoid a chdir. 2013-09-07 14:47:49 +02:00
Petr Rockai d31e4469bb Put a 5-second CPU time limit on the log processing pipeline. 2013-09-07 13:28:53 +02:00
Petr Rockai 1e1a1f0838 Also allow : (colon) in pathCompRE. 2013-09-07 13:28:53 +02:00
Petr Rockai bb38bdfe35 Allow : and + in build product filenames. 2013-09-07 13:28:53 +02:00
Petr Rockai dcf386cfed Make getDrvLogPath work with both bucketed and non-bucketed nix logs. 2013-09-07 13:28:53 +02:00
Petr Rockai bb8059e50c Add support for fetching URLs as inputs. 2013-09-07 13:28:53 +02:00
Petr Rockai 5a35912956 Add support for darcs repositories. 2013-09-07 13:28:51 +02:00
Eelco Dolstra 5078730cb5 Use evalFile() instead of parseExprFromFile() 2013-09-06 15:17:27 +00:00
Shea Levy 788c2bebaa Not every buildstep has a defined drvpath 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-09-03 14:43:08 -04:00
Eelco Dolstra 04e6eda4c7 Remove unused function 2013-09-03 20:29:52 +02:00
Eelco Dolstra d914845ad7 Use a distinct icon for cancelled builds 2013-09-03 17:37:47 +02:00
Eelco Dolstra e8fb659d63 Fix some wellformedness issues 
Also fixed some duplicate IDs reported by Mats Erik Andersson.
 2013-09-03 17:35:21 +02:00
Eelco Dolstra 614f07755a Don't show logs bigger than 64 MB 2013-08-30 20:44:04 +00:00
Eelco Dolstra 6655ada236 Serve raw uncompressed logs directly 2013-08-30 18:11:03 +00:00
Eelco Dolstra fd7e37ef89 Try harder to find build logs 
Due to the fixed-output derivation hashing scheme, there can be
multiple derivations of the same output path.  But build logs are
indexed by derivation path.  Thus, we may not be able to find the
log of a build or build step using its derivation.  So as a fallback,
Hydra now looks for other derivations with the same output paths.
 2013-08-30 13:53:25 +00:00
Eelco Dolstra 559a98cfee Use a different icon for aborted builds 2013-08-28 17:40:51 +02:00
Eelco Dolstra 2f7071ee9d Tabify the eval page 
The different types of types (new, removed, newly failing, and so
on) now each have their own tab.
 2013-08-28 17:40:51 +02:00
Eelco Dolstra 07747b7e88 Allow filtering jobs in eval pages 2013-08-28 17:40:51 +02:00
Eelco Dolstra d0bcaa6284 Allow comparing an eval against the jobset one day/week/month before 2013-08-28 17:40:51 +02:00
Eelco Dolstra ccc09c565e Sort jobsets by name 2013-08-28 17:40:51 +02:00
Eelco Dolstra 58a6fdc5ed Remove the job status pages 
They're mostly redundant since there is a faster "jobs" tab on
the jobset pages now.  The only thing the latter lacks is the
ability to see status change times, but those are quite expensive
to compute, and are visible on build pages if you really need them.
 2013-08-28 17:40:51 +02:00
Eelco Dolstra 42c4ef856f Fix incorrect removed jobs in eval pages 
PostgreSQL and Perl have different sort orders, in particular when
comparing job names such as "aspell.x86_64-linux" and
"aspellDicts.cs.i686-freebsd".  This confused the evaluation
comparison code, causing some jobs to appear as "removed".
So now we do all the sorting in Perl.

Fixes #105.
 2013-08-28 17:40:50 +02:00
Eelco Dolstra d886ff9973 Integrate the "Job status" and "All jobs" tabs 
The job status tabs now has a toggle to show inactive jobs, rendering
the "All jobs" tab redundant.
 2013-08-28 17:40:50 +02:00
Eelco Dolstra 9002b69c2d By default, show the first 250 jobs only 
This is particularly useful for the Nixpkgs jobsets, which now have
~24K jobs.
 2013-08-28 17:40:50 +02:00
Eelco Dolstra 10d959a9ff Show visual feedback that the filter is being applied 2013-08-28 17:40:50 +02:00
Eelco Dolstra 11acf0be28 Load a tab only once 2013-08-28 17:40:50 +02:00
Eelco Dolstra 410060ec8a Jobset page: Add a new tab to show job status in a matrix 2013-08-28 17:40:50 +02:00
Eelco Dolstra 82e073d043 Show build status 2 and 5 in the same way 
(Status 5 is obsolete.)
 2013-08-28 17:40:50 +02:00
Eelco Dolstra e69c7ac360 "limit" -> "rows" 2013-08-28 17:40:50 +02:00
Eelco Dolstra 7685596aa8 Transpose the aggregate constituents table 
This way, it grows vertically rather than horizontally.  Horizontal
may be more "logical", but this is more practical.
 2013-08-28 17:40:50 +02:00
Eelco Dolstra 7725038821 On aggregate job pages, show a matrix showing all the constituent builds 2013-08-28 17:40:50 +02:00
Rob Vermaas bf42392fe4 Fix typo. 2013-08-27 15:12:41 +02:00
Eelco Dolstra a57957df84 Handle job aliases in AggregateConstituents 
Aggregate constituents are derivations.  However there can be multiple
builds in an evaluation that have the same derivation, i.e. they can
alias each other (e.g. "emacs", "emacs24" and "emacs24Packages.emacs"
in Nixpkgs).  Previously we picked a build arbitrarily for the
AggregateConstituents table.  Now we pick the one with the shortest
name (e.g. "emacs").
 2013-08-27 11:48:02 +02:00
Rob Vermaas a98075f386 HipChat notification: do not include latest commits of all inputs in 'who-broke-the-build' list. Use only committers from inputs that have actually changed since previous build. 2013-08-26 11:06:10 +00:00
Eelco Dolstra 02cba75610 Add an action to download a specific output of a build as a .nar.bz2 
E.g. http://hydra/build/3515983/output/out downloads the output named
"out" as a bzip2-compressed NAR.
 2013-08-21 14:30:38 +02:00
Eelco Dolstra edb88ef452 Remove unused ActiveJobs source 2013-08-20 15:22:46 +02:00
Eelco Dolstra 14e418cafa Don't show bogus last-checked times 2013-08-16 18:26:01 +02:00
Eelco Dolstra 6264995198 Remove the jobs status page 
The per-system presentation doesn't make much sense any more given
issue #60.  It should be replaced by (say) a grid showing each job per
evaluation.
 2013-08-16 17:16:15 +02:00
Eelco Dolstra e54c361a95 Remove per-platform links from the job page 
Having different builds within a job is obsolete (issue #60), one
should have different job per platform (e.g. build.x86_64-linux).
 2013-08-16 16:39:42 +02:00
Eelco Dolstra a9c6f522e6 clear_queue_non_current: Don't use isCurrent 2013-08-16 16:38:09 +02:00
Eelco Dolstra 46f8b25c1f Keep builds that failed with output 
The user may want to look at the output, so they shouldn't be
GC'ed right away.
 2013-08-16 16:36:06 +02:00
Eelco Dolstra d16738e130 hydra-update-gc-roots: Keep the most recent evaluations 
We now keep all builds in the N most recent evaluations of a jobset,
rather than the N most recent builds of every job.  Note that this
means that typically fewer builds will be kept (since jobs may be
unchanged across evaluations).
 2013-08-16 16:21:30 +02:00
Eelco Dolstra 8e1ade4422 Fix display of non-aggregate builds 2013-08-15 13:57:47 +02:00
Eelco Dolstra 242072bbd6 Hide the views tab for project that don't have them 
Views are obsolete (replaced by the declarative "aggregate" build
mechanism) so we don't want people creating new ones.
 2013-08-15 13:54:23 +02:00
Eelco Dolstra c9a0e12804 Hide project/jobset in constituent list 2013-08-15 03:35:18 +02:00
Eelco Dolstra 72a0fa6ec5 Sort constituents by job name 2013-08-15 03:28:21 +02:00
Eelco Dolstra 06c74085b5 Make "Add to release" a modal dialog 2013-08-15 03:07:20 +02:00
Eelco Dolstra d92d83a82a Fix broken redirect when editing a release 2013-08-15 02:57:36 +02:00
Eelco Dolstra 1776d9118f Rename aggregate members to constituents 2013-08-15 02:33:10 +02:00
Eelco Dolstra e4141afcc9 On the build page, show how many aggregate constituents failed 
(Also, renamed aggregate "member" to "constituent", since "member" is
rather vague.)
 2013-08-15 02:17:06 +02:00
Eelco Dolstra 81322de94e Show aggregate members 2013-08-15 00:30:19 +02:00
Eelco Dolstra d58142b3f0 Store aggregate members in the database 
For presentation purposes, we need to know what builds are part of an
aggregate build.  So at evaluation time, look at the "members"
attribute, find the corresponding builds in the eval, and create a
mapping in the AggregateMembers table.
 2013-08-14 01:59:29 +02:00
Eelco Dolstra c27f4bbaf5 Add a redirect from an eval to a named member 
E.g. /eval/798867/job/tarball redirects to the build with job name
"tarball".
 2013-08-13 01:56:45 +02:00
Eelco Dolstra a25ea193f7 Add a /eval action to /build to go from a build to the latest finished eval 2013-08-13 01:43:50 +02:00
Eelco Dolstra fcdca0d4de Fix some XML wellformedness issues 2013-08-13 00:41:37 +02:00
Eelco Dolstra 4792256e43 Add some more fields to the JSON output of /build 2013-08-12 22:57:53 +02:00
Eelco Dolstra bef263c930 Add a ‘latest-finished’ action 
It redirects to the latest successful build from a finished
evaluation.  This is mostly useful for the Nixpkgs/NixOS mirroring
script, which need the latest finished evaluation in which some
aggregate job (such as ‘tested’ in NixOS) succeeded.
 2013-08-12 22:17:04 +02:00
Eelco Dolstra 84acccb3ea Index builds on stop time 
This is necessary to make the /all page fast, since it sorts builds on
descending stop time.
 2013-08-12 20:19:10 +02:00
Eelco Dolstra 452c8e36d1 Materialize the number of finished builds 
The NrBuilds table tracks the value of ‘select count(*) from Builds
where finished = 0’, keeping it up to date via a trigger.  This is
necessary to make the /all page fast, since otherwise it needs to do a
sequential scan on the Builds table.
 2013-08-12 20:19:10 +02:00
Eelco Dolstra 182f725612 Don't pass an undefined input 2013-08-12 18:15:11 +02:00
Eelco Dolstra d96df42c03 GitInput.pm: Don't do a chdir to the Git clone 
Doing a chdir in the parent is evil.  For instance, we had Hydra core
dumps ending up in the cloned directory.  Therefore, the function
‘run’ allows doing a chdir in the child.  The function ‘grab’ returns
the child's stdout and throws an exception if the child fails.
 2013-08-12 17:46:26 +02:00
Rob Vermaas 90eedcf256 HipChat notification: add support for Mercurial inputs for determining who might have broken the build. 2013-08-07 08:53:32 +00:00
Shea Levy 96e987bbfa Use inputTypes from plugins to determine valid input types 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-07-29 17:42:49 -04:00
Shea Levy 687ca429c3 Pass project and jobset to fetchInput 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-07-29 15:33:22 -04:00
Shea Levy f231c23b75 Only serialize JSON and HTML, not the C::C::REST defaults 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-07-26 14:25:25 -04:00
Shea Levy eab13d8736 ToJSON is automatically called recursively 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-07-26 13:54:07 -04:00
Shea Levy d6b23272e3 Don't try to serialize if there's nothing to serialize 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-07-26 12:04:27 -04:00
Shea Levy f7bcf9fc19 The field is actually enabled, not disabled 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-07-25 17:59:13 -04:00
Shea Levy 1404d33005 Show when a Project's jobsets are disabled 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-07-25 17:48:28 -04:00
Rob Vermaas 513c030268 Do not use local clone for Bazaar inputs. 2013-07-22 20:42:17 +02:00
Eelco Dolstra 6574d125c7 Get rid of a warning in the HipChat plugin 2013-07-19 14:43:04 +02:00
Ludovic Courtès 2d5e06918b Hydra::View::Plain: Explicitly set the response body. 
This fixes a bug with Catalyst 1.39 whereby a raw hash table would
erroneously be returned for /nix-cache-info.
 2013-07-12 16:53:48 +02:00
Ludovic Courtès 438d7f7c5c Reply 404 for requests for non-existent .narinfo. 2013-07-12 16:52:40 +02:00
Eelco Dolstra db3647aa15 Set the character set 
Cherry-picked from the persona branch.
 2013-07-12 15:04:13 +02:00
Eelco Dolstra d071bbfb28 Fix Hipchat notification 2013-07-12 15:04:13 +02:00
Shea Levy 7a0f80f016 Include the email override list in the Jobset serialization 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-07-11 11:01:36 -04:00
Eelco Dolstra ccd2292458 Link to both the Persona and legacy sign in 2013-07-09 14:39:28 +02:00
Eelco Dolstra 486a844866 Fix audience URL 2013-07-09 14:05:04 +02:00
Eelco Dolstra 1db9381473 Fix legacy login 2013-07-09 13:55:44 +02:00
Eelco Dolstra 9aa44fa944 Some escaping 2013-07-09 13:55:29 +02:00
Eelco Dolstra 895f21cfad Show a user's roles in the preferences page 2013-07-09 13:07:04 +02:00
Eelco Dolstra 476203d0bb Prevent Persona users from setting a password 2013-07-09 13:01:56 +02:00
Eelco Dolstra 5fecf8e385 Show sign in as success 2013-07-09 12:07:18 +02:00
Eelco Dolstra 08f3bb78c4 Use the alert class for flash messages 2013-07-09 12:05:25 +02:00
Eelco Dolstra 12ca755bea Don't redirect to /login if authentication is required 
We can just serve the login page in place.
 2013-07-09 12:02:15 +02:00
Eelco Dolstra 13f0b54c49 Set the character set 2013-07-09 00:23:48 +02:00
Eelco Dolstra c08fc6ce1e Add basic Persona support 
This allows users to sign in to Hydra using Mozilla Persona accounts.
When a user first sign in, a row in the Users table for the given
Persona identity (an email address) is created automatically.

To do: figure out how to deal with legacy accounts.
 2013-07-08 23:54:40 +02:00
Shea Levy efd011fbc3 Revert "Don't call buildFinished after we already know it failed" 
I don't understand perl strings.

This reverts commit b2f6be9686.

Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-07-08 14:30:46 -04:00
Shea Levy b2f6be9686 Don't call buildFinished after we already know it failed 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-07-08 13:52:51 -04:00
Shea Levy 166d56088f Call buildFinished when a cached build is added 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-07-08 13:35:34 -04:00
Shea Levy b46f2134e0 Revert "Emable overriding the email template." 
This doesn't work as planned, will re-work.

This reverts commit 304f8a5714.
 2013-07-05 18:10:50 -04:00
Shea Levy a168f43515 Revert "Handle HYDRA_EMAIL_TEMPLATES that are paths" 
This doesn't work as planned, will re-work.

This reverts commit d2ce7747a0.
 2013-07-05 18:10:21 -04:00
Shea Levy d2ce7747a0 Handle HYDRA_EMAIL_TEMPLATES that are paths 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-07-05 17:22:51 -04:00
Shea Levy 304f8a5714 Emable overriding the email template. 
The HYDRA_EMAIL_TEMPLATE env var can be set to any value understood by
https://metacpan.org/module/Template#process-template-vars-output-options
as the $template argument.

Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-07-05 16:29:09 -04:00
Shea Levy be3ca0dd84 Fix jobset update 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-07-05 14:44:29 -04:00
Shea Levy 93a14ccf32 Check if we want to send email for a build before iterating through its users 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-07-05 14:06:10 -04:00
Shea Levy 002ac9ef63 Merge in the first bits of the API work 
The catalyst-action-rest branch from shlevy/hydra was an exploration of
using Catalyst::Action::REST to create a JSON API for hydra. This commit
merges in the best bits from that experiment, with the goal that further
API endpoints can be added incrementally.

In addition to migrating more endpoints, there is potential for
improvement in what's already been done:
* The web interface can be updated to use the same non-GET endpoints as
  the JSON interface (using x-tunneled-method) instead of having a
  separate endpoint
* The web rendering should use the $c->stash->{resource} data structure
  where applicable rather than putting the same data in two places in
  the stash
* Which columns to render for each endpoint is a completely debatable
  question
* Hydra::Component::ToJSON should turn has_many relations that have
  strings as their primary keys into objects instead of arrays

Fixes NixOS/hydra#98

Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-07-02 14:00:46 -04:00
Eelco Dolstra d18fc4fc38 Include names of committers in HipChat notifications 
HipChat notification messages now say which committers were
responsible, e.g.

  Job patchelf:trunk:tarball: Failed, probably due to 2 commits by Eelco Dolstra
 2013-07-02 13:54:18 +02:00
Eelco Dolstra 7e11d01abf Remove tabs 2013-07-02 11:37:16 +02:00
Eelco Dolstra fe030331b5 Revert "Don't compress already-compressed files." 
This reverts commit 190bffd846.
 2013-07-01 18:46:18 +02:00
Eelco Dolstra 11414b0447 Revert "Doh" 
This reverts commit f1f938512d.
 2013-07-01 18:46:14 +02:00
Rob Vermaas f1f938512d Doh 2013-07-01 18:12:33 +02:00
Rob Vermaas 9d7c14b342 Merge pull request #103 from civodul/nar-uncompressed 
Don't compress already-compressed files.
 2013-07-01 08:16:51 -07:00
Eelco Dolstra 9939ddf22c Add a plugin for HipChat notification 
This plugin sends notification of build failure or success to a
HipChat room, if the status differs from the last build.

The plugin can be configured by adding one or more of these stanzas to
hydra.conf:

  <hipchat>
    jobs = (patchelf|nixops):.*:.*
    room = 1234
    token = 39ab2198fe...
  </hipchat>

Here "jobs" is a regular expression against which the fully qualified
job name of the build is matched (so for instance
"nixops:master:tarball" will match the stanza above).
 2013-06-27 18:56:31 +02:00
Eelco Dolstra 698d6eac89 Eliminate duplicate getPreviousBuild functions 
And move some stuff into CatalystUtils for use by other plugins.
 2013-06-27 18:08:00 +02:00
Eelco Dolstra c65f40e146 Add common.js to the tarball 2013-06-25 12:48:25 +02:00
Eelco Dolstra 611f402a70 Move common Javascript code into a separate file 2013-06-25 01:16:28 +02:00
Eelco Dolstra a5436be2ce After editing a project/jobset, redirect back to the config tab 2013-06-25 01:00:59 +02:00
Eelco Dolstra 68a03bc674 When a tab is selected, change the fragment id 
And when a page is visited, active the tab identified by the fragment
id.  This allows linking to a specific tab.

Fixes #93.
 2013-06-25 00:55:14 +02:00
Eelco Dolstra afc90b36db Don't print to stdout 2013-06-24 22:22:43 +00:00
Ludovic Courtès 190bffd846 Don't compress already-compressed files. 
Fixes <https://github.com/NixOS/hydra/issues/102>.
 2013-06-23 15:23:16 +02:00
Eelco Dolstra 51f0185797 Allow _ in release names 2013-06-18 16:00:24 +02:00
Eelco Dolstra 98a105fe69 hydra-build: Give a nicer error message if the derivation is gone 2013-06-14 11:01:53 +00:00
Eelco Dolstra cceab7308b hydra-queue-runner: Handle restarted builds whose derivation is gone 
Restarted builds whose derivation has been garbage-collected in the
meantime caused hydra-queue-runner to get stuck in a loop saying:

Jun 14 11:54:25 lucifer hydra-queue-runner[31844]: system type `x86_64-darwin': 0 active, 2 allowed, started 2 builds
Jun 14 11:54:25 lucifer hydra-queue-runner[31844]: {UNKNOWN}: path `/nix/store/wcizsch2garjlvs4pswrar47i1hwjaia-inconsolata.drv' is not valid at
/nix/store/ypkdm4v13yrk941rvp8h0y425a5ww6nm-hydra-0.1pre1353-40debf1/bin/.hydra-queue-runner-wrapped line 51. at
/nix/store/kjpsc2zdaxnd44azxyw60f2px839m1cd-hydra-perl-deps/lib/perl5/site_perl/5.16.2/Catalyst/Model/DBIC/Schema.pm line 501
 2013-06-14 11:00:05 +00:00
Eelco Dolstra 1c30a777ad Fix project/jobset deletion 2013-06-11 16:57:22 +02:00
Eelco Dolstra 40debf1515 hydra-queue-runner: Don't unlock builds we just started 
This happens if the previous iteration took more than 60 seconds.
Then the queue runner may think that builds failed to start properly
and unlock them, e.g.

build 5264936 pid 19248 died, unlocking
build 5264951 pid 19248 died, unlocking
build 5257073 pid 19248 died, unlocking
...
 2013-06-07 20:15:37 +00:00
Eelco Dolstra 5d9b7c6ab2 Speed up findBuildDependencyInQueue 
This was taking a long time due to the giant SQL query.

Issue #99.
 2013-06-07 20:15:32 +00:00
Eelco Dolstra 8e36343b62 hydra-queue-runner: Start as many builds as possible on each iteration 
Because we don't start a build if a dependency is already building,
it's possible that some or all of the $extraAllowed highest-priority
builds in the queue are not eligible.  E.g. with $extraAllowed = 32,
we might start only 3 builds even though there are thousands in the
queue.  The fix is to try all queued builds until $extraAllowed have
been started.

Issue #99.
 2013-06-07 20:15:20 +00:00
Eelco Dolstra 2974fea1a7 Use https://ajax.googleapis.com if we're on https 2013-06-07 15:45:37 +02:00
Eelco Dolstra f4ecf27def Fix Perl warning about $path masking earlier definition 2013-06-02 23:32:09 +02:00
Eelco Dolstra 8e246a835a Don't do camel case in filenames 2013-06-02 23:30:46 +02:00
Shea Levy 591acda6f9 reproduce.tt: Save the revCount when reproducing more than once 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-05-30 14:06:32 -04:00
Shea Levy 8e716ff062 reproduce.tt: Create the checkout dir before an hg clone 
For some reason, hg clone from a local (path-based) repo will fail if
the parent directory of the destination directory doesn't exist (though
it succeeds when cloning from an http repo).

Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-05-30 13:31:53 -04:00
Eelco Dolstra 1277a8fc1c Don't break certain table columns 
For instance, if the "timestamp" column becomes too small, then every
timestamp is spread out across two lines, doubling the size of the
table.
 2013-05-29 14:58:44 +02:00
Eelco Dolstra 47c49cf3d4 Show links on products of type "doc" 2013-05-29 11:39:23 +02:00
Eelco Dolstra 77e37c1ea3 Don't add a nix-build build product unless $out is a directory 2013-05-28 18:43:05 +02:00
Eelco Dolstra 1f1615e80b Support revision control systems via plugins 2013-05-25 15:36:58 -04:00
Eelco Dolstra e0c411220e built.tt: Handle the case where the original build is not known 
And remove some parentheses.
 2013-05-24 12:43:31 -04:00
Eelco Dolstra 9ac363d32a Fill in starttime/stoptime for cached builds 2013-05-24 12:43:02 -04:00
Petr Rockai 6c0577b226 Respect X-Request-Base header coming from a frontend proxy. 2013-05-23 13:07:40 -04:00
Eelco Dolstra 5e5b344527 Make latest/latest-for more efficient 
Getting all matching rows from the database is expensive.
 2013-05-23 12:18:38 -04:00
Eelco Dolstra 7b5e92868b Remove unused Jobs columns 
Actually, we can *almost* get rid of the Jobs table entirely...
 2013-05-23 12:12:19 -04:00
Eelco Dolstra b1f89bf92d Ensure that stoptime and starttime are set for finished builds 2013-05-23 12:05:12 -04:00
Eelco Dolstra 57b2bb0674 Let Builds.timestamp refer to the time the build was added 
Previously, for scheduled builds, "timestamp" contained the time the
build was added to the queue, while for finished builds, it was the
time the build finished.  Now it's always the former.
 2013-05-23 10:45:49 -04:00
Eelco Dolstra e09622eb29 Include # of failed builds in Subject 2013-05-21 18:30:05 -04:00
Shea Levy 4b1a8384db reproduce.tt: Set revCount for hg repos 
The revision counting changes depending on which revision is cloned
initially, so clone the default branch first and then checkout the
required revision to match hydra's revCount.

Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-05-16 11:24:17 -04:00
Rob Vermaas 666f398e98 Always show the links related to the build product. 2013-05-16 11:29:54 +00:00
Eelco Dolstra 444b03bad2 Omit the system type if it's part of the job name 2013-05-15 11:43:06 +02:00
Eelco Dolstra ac6a152ec5 Improve email notifications status messages 
Add "Failed with output", and shorten "Failed with non-zero exit code"
to "Failed".
 2013-05-15 11:31:37 +02:00
Rob Vermaas 43785dfca9 Merge pull request #85 from peti/dont-clutter-system-log-with-debug-messages 
hydra-queue-runner: don't clutter the system log with debug message
 2013-05-10 14:52:13 -07:00
Eelco Dolstra 3939974df8 Set build status to 1 if the primary build failed 2013-05-10 00:51:45 +02:00
Eelco Dolstra 102359bf44 Add separate build step status codes for cached failures and timeouts 2013-05-09 22:13:01 +00:00
Eelco Dolstra a6d8566faf If a build aborts, mark any remaining active build steps as aborted 
See e.g. http://hydra.nixos.org/build/4915744.

P.S. existing active build steps of finished builds can be marked as
aborted by running:

update buildsteps set busy = 0, status = 4
  where (build, stepnr) in
    (select s.build, s.stepnr from buildsteps s join builds b on s.build = b.id where b.finished = 1 and s.busy = 1);
 2013-05-09 18:03:34 +02:00
Eelco Dolstra 7b78596412 Install plugins 2013-05-09 17:02:16 +02:00
Eelco Dolstra 184d23dc41 Remove debug line 2013-05-09 15:10:30 +02:00
Eelco Dolstra 3ca7b84297 Don't float the search bar to the right in collapsed mode 2013-05-09 14:36:51 +02:00
Eelco Dolstra 26b3da5d13 Fix tests 
http://hydra.nixos.org/build/4902041
 2013-05-09 14:29:04 +02:00
Eelco Dolstra de89c566f8 Use Template::Toolkit to generate email messages 
Also, make the messages much shorter by not including build details.
 2013-05-09 14:23:15 +02:00
Eelco Dolstra b378d94087 If multiple dependent builds fail, send a single email 2013-05-09 11:39:58 +02:00
Eelco Dolstra 038db3abeb Pass failing dependent builds to buildFinished 2013-05-08 18:54:01 +02:00
Eelco Dolstra 44b8d6f449 Use OO-style plugins 
This is mostly so we don't have to pass around common parameters like
"db" and "config", and we don't have to check for the existence of
methods.

A plugin now looks like this:

  package Hydra::Plugin::TwitterNotification;

  use parent 'Hydra::Plugin';

  sub buildFinished {
      my ($self, $build, $dependents) = @_;
      print STDERR "tweeting about build ", $build->id, "\n";
      # Send tweet...
      # Hydra database is $self->{db}.
  }
 2013-05-08 18:35:24 +02:00
Eelco Dolstra a420a33f66 Use "can" to check whether a plugin supports buildFinished 2013-05-08 17:57:48 +02:00
Eelco Dolstra 1d8bb0764b Add a plugin mechanism 
You can now add plugins to Hydra by writing a module called
Hydra::Plugin::<whatever> and putting it in Perl's search path.  The
only plugin operation currently supported in buildFinished, called
when hydra-build has finished doing a build.

For instance, a Twitter notification plugin would look like this:

  package Hydra::Plugin::TwitterNotification;

  sub buildFinished {
      my ($self, $db, $config, $build, $dependents) = @_;
      print STDERR "tweeting about build ", $build->id, "\n";
      # send tweet...
  }

  1;
 2013-05-08 17:30:30 +02:00
Eelco Dolstra f447c7d9db Add separation between news items 2013-05-07 17:04:42 +02:00
Eelco Dolstra a84db1c7cc Use extra-binary-caches 2013-05-07 15:57:20 +02:00
Eelco Dolstra 9947415ef0 Remove Twitter notification support 
Turns out Twitter is not an ideal medium for sending build
notifications :-)
 2013-05-03 18:33:11 +02:00
Eelco Dolstra f762d111f1 If a build step fail, immediately fail all queued builds that depend on it 
This prevents unnecessary work, but it's mostly a refactoring to
support combining notification emails.
 2013-05-03 18:30:13 +02:00
Eelco Dolstra 507e5bb190 Drop unused "disabled" columns 2013-05-03 16:39:17 +02:00
Eelco Dolstra 906b129f6a Fix findBuildDependencyInQueue 
Previously this function didn't actually have a lot of effect.  If a
build A had a dependency B, Hydra would start B first.  But on the
next scan through the queue, it would start A anyway, because of the
"busy => 0" restriction.

Now the queue runner won't start a build if a dependency is already
running.  (This is not necessarily optimal, since the build may have
other dependencies that don't correspond to a build in the queue but
could run.  One day we'll start all Hydra builds in parallel...)

Also, for performance, use computeFSClosure instead of "nix-store
-qR".  And don't bother with topological sorting because it didn't
have an effect anyway since the database returns dependencies in
arbitrary order.
 2013-05-03 16:27:26 +02:00
Eelco Dolstra 9e72c64eff Restore the links to the contents pages 
These were accidentally lost in
80e0e7c84a.
 2013-05-03 14:13:14 +02:00
Eelco Dolstra e96916533f Allow a per-jobset check interval 
This allows checking a jobset (say) at most once a day.  It's also
possible to disable polling by setting the interval to 0.  This is
useful for jobsets that use push notification or are manually
evaluated.
 2013-05-02 17:55:47 +02:00
Eelco Dolstra edaeb892e7 reproduce.tt: Use color 2013-05-02 17:55:47 +02:00
Eelco Dolstra a468a28840 reproduce.tt: Check for the presence of commands like nix-build 2013-05-02 17:55:47 +02:00
Eelco Dolstra 659898bd67 reproduce.tt: Add --help flag 
And a --dir flag to override the directory where inputs etc. are
stored.
 2013-05-02 17:55:47 +02:00
Shea Levy adb9ad83dd hydra-evaluator: Respect triggers of disabled jobsets 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-05-02 11:32:25 -04:00
Shea Levy b8ea428bb1 Allow manually scheduling disabled jobsets for evaluation 
Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-05-02 11:21:43 -04:00
Eelco Dolstra 3003bb1c99 Don't fall thru to Hydra::View::NARInfo 
This caused exceptions like:

  Caught exception in Hydra::Controller::Build->view_build "writing to file: Broken pipe at /nix/store/ihdb3widsq1dk7sbl5vqjxfcxb5ypad4-hydra-0.1pre1297-8158093/libexec/hydra/lib/Hydra/Controller/Build.pm line 59."

because the connection to the Nix daemon would be terminated due to a
protocol violation (calling queryPathInfo with an empty string).
 2013-04-30 16:23:19 +02:00
Eelco Dolstra 45a1bb9926 Remove unnecessary parentheses in SQL query 
These make SQLite 3.7.16.2 crash.

http://hydra.nixos.org/build/4815315
 2013-04-29 21:38:57 +02:00
Eelco Dolstra 8158093884 Add correct bootstrap-2.3.1.zip 2013-04-29 09:54:01 +02:00
Rob Vermaas 7fe63854fb Replace bootstrap tarball 2013-04-28 07:06:24 +02:00
Eelco Dolstra 80e0e7c84a Use popovers for the product list help/details 2013-04-26 17:51:06 +02:00
Eelco Dolstra ea4c88bda1 Use a popover to show how to use the build reproduction script 2013-04-26 17:51:06 +02:00
Eelco Dolstra 7bd1fab268 Use the default navbar style for readability 2013-04-26 17:51:06 +02:00
Eelco Dolstra 74262dd9c4 Remove project/jobset/job/build id from the navbar 
They take up too much space, and they're visible in the page title
anyway.  But include them as a nav-header in the submenus.
 2013-04-26 17:51:05 +02:00
Eelco Dolstra ac2bdaf1c5 Remove tablesorter 
We haven't used it for a while now.
 2013-04-26 17:51:05 +02:00
Eelco Dolstra 4a2cb5e206 Update bootstrap to 2.3.1 2013-04-26 17:51:05 +02:00
Eelco Dolstra bbc3e25d5a Use bootstrap's pager (instead of pagination) class 2013-04-26 17:51:05 +02:00
Eelco Dolstra 002793bf2d makeLink -> menuItem 2013-04-26 17:51:05 +02:00
Eelco Dolstra 77a0fd9488 Remove unused file 2013-04-26 17:51:05 +02:00
Eelco Dolstra 3058ce2ca3 Don't do <tt> on input type 2013-04-26 17:51:05 +02:00
Eelco Dolstra 71eaa115f8 Ellipsize long jobset/job names to prevent bootstrap navbar breakage 2013-04-26 17:51:05 +02:00
Rob Vermaas fd7acd9bbf When searching a path / drv path, use separate queries for both options to prevent seq. scan. 2013-04-26 08:51:49 -04:00
Eelco Dolstra 591aee0793 getPrimaryBuildsForView: Return all view results 
Returning only the first 20 results can cause NixOS/Nixpkgs channel
generation to fail, if the first 20 view results correspond to
evaluations that haven't finished yet.  Then URLs like
/view/nixos/tested/latest-finished will return 500 rather than the
latest finished view.
 2013-04-26 13:21:41 +02:00
Rob Vermaas bfc1b0e76a Allow searching for store and drv paths. Only does a complete match to reduce load on database. 2013-04-25 09:57:30 -04:00
Eelco Dolstra 2c7a479994 Minor tweak 2013-04-23 15:39:05 +02:00
Eelco Dolstra 72a44cfcaa Make negative narinfo responses much smaller 2013-04-23 15:33:58 +02:00
Eelco Dolstra 54b03527cb Remove some redundant SQL queries 2013-04-23 15:23:08 +02:00
Eelco Dolstra 7528a9c7e7 Machine status: Don't show removed machines anymore 
This requires a sequential scan on the BuildSteps table, which by now
takes > 8s.
 2013-04-23 15:20:24 +02:00
Eelco Dolstra b3612b8a81 Fix showing build steps in the machine status page 
This was broken because of the comparison

  step.machine.match('@(.*)').0 == m.key

where m.key contains the SSH user.

Also some style tweaks.
 2013-04-23 14:24:48 +02:00
Ludovic Courtès bd5454d5d9 hydra-eval-guile-jobs: Use `--fresh-auto-compile'. 
* src/script/hydra-eval-guile-jobs.in (main): Run `guile' with
  `--fresh-auto-compile'.
 2013-04-13 18:59:42 +02:00
Peter Simons ccbe75781b hydra-queue-runner: don't clutter the system log with debug message 
Avoid the frequently printed

  hydra-queue-runner[10293]: system type `x86_64-linux': 2 active, 2 allowed, starting 0 builds

message. That information is only interesting when some build are
actually started.
 2013-04-11 10:56:40 +02:00
Eelco Dolstra 9f027b22b1 Allow users to reproduce builds on their own systems 
You can now do:

  bash <(curl http://hydra-server/build/1238757/reproduce)

to download and execute a script that reproduces a Hydra build
locally.  This script fetches all inputs (e.g. Git repositories) and
then invokes nix-build.

The downloaded sources are stored in /tmp/build-<buildid> and reused
between invocations of the script.

Any additional command line options are passed to nix-build.  So

  bash <(curl http://hydra-server/build/1238757/reproduce) --run-env

will drop you in a shell where you can interactively hack on the
build, e.g.

  $ source $stdenv/setup
  $ set +e
  $ unpackPhase
  $ cd $sourceRoot
  $ configurePhase
  $ emacs foo.c &
  $ make

and so on.
 2013-04-04 17:30:07 +02:00
Eelco Dolstra 48bb9c48ed Security: Prevent shell injection attack in rendering build contents 2013-04-03 00:13:03 +02:00
Eelco Dolstra 841e03ecba Hack to render empty lines in logs correctly 2013-04-02 23:40:19 +02:00
Eelco Dolstra e7926e046b Security: Improve checking of build products 
Build product paths cannot reference locations outside of the Nix
store.  We previously disallowed paths from being symlinks, but this
didn't take into account that parent path elements can be symlinks as
well.  So a build product /nix/store/bla.../foo/passwd, with
/nix/store/bla.../foo being a symlink to /etc, would still work.

So now we check all paths encountered during path resolution.
Symlinks are allowed again so long as they point to the Nix store.
 2013-04-02 23:32:04 +02:00
Eelco Dolstra 94984270b0 Pretty log: Number lines and make them linkable 2013-04-02 11:25:46 +02:00
Eelco Dolstra b8369a6a5f Put build status in front of the notification mail subject 2013-04-02 10:40:50 +02:00
Hydra 16dbec332d Postgresql 8.x needs as for column. 2013-04-01 20:18:00 -04:00
Eelco Dolstra 61405ec58a hydra-eval-jobs: Use function argument default values 2013-03-29 01:14:24 +01:00
Eelco Dolstra d734b435fa Remove the link for creating views since they're deprecated 2013-03-29 00:59:48 +01:00
Eelco Dolstra ee5b762919 Disable account creation for now 2013-03-28 11:56:12 +01:00
Ludovic Courtès f3aa48978e hydra-eval-guile-jobs: Allow use of the installed Guix. 
* src/script/hydra-eval-guile-jobs.in: Leave GUILE_LOAD_PATH and
  GUILE_LOAD_COMPILED_PATH unchanged.
 2013-03-27 00:07:57 +01:00
Ludovic Courtès 9a9f37f209 hydra-eval-guile-jobs: Adjust to multiple-output format. 
This is a followup to commit
10882a1ffd ("Add multiple output
support").

* src/script/hydra-eval-guile-jobs.in (job-evaluations->sxml): Return
  several `output' tags in the body, and remove the `outPath' attribute
  of `job'.
 2013-03-27 00:07:57 +01:00
Ludovic Courtès 9da89dd32b hydra-eval-guile-jobs: Adjust to <license> objects. 
* src/script/hydra-eval-guile-jobs.in (job-evaluations->sxml): Output
  the license name, not the external representation of <license>
  objects.
 2013-03-27 00:07:57 +01:00
Peter Simons 5ffc925ae7 hydra-queue-{runner,evaluator}: don't clutter the system log with debug messages 2013-03-26 11:03:15 +01:00
Rob en Lizi Vermaas 52a9f1a14e Use get_column for 'as' column (max_stoptime). 2013-03-24 18:50:17 +01:00
Sander van der Burg 03189bf62b Implemented quoted strings support in hydra-build-products to allow file names with spaces + testcase 2013-03-20 18:05:21 +01:00
Eelco Dolstra 50434d76c2 Fix rendering of jobset inputs 2013-03-19 16:14:47 +01:00
Eelco Dolstra c1aefc4c41 Update bootstrap to 2.0.3 
This makes the navbar collapse properly on small screens.
 2013-03-19 00:15:12 +01:00
Ludovic Courtès 93f4a26058 Don't always include "Hydra" in the logo part. 
This partially reverts 67986b03fa.
 2013-03-15 11:23:00 +01:00
Shea Levy 94382bd8f0 Admin controller: Use PathPart, not Path 
Chaining paths only works properly when PathPart is used. Before this
fix, the affected URIs bypassed the top-level 'admin' sub.

Signed-off-by: Shea Levy <shea@shealevy.com>
 2013-03-14 19:22:02 -04:00
Eelco Dolstra c96e427ec0 Log segfaults from the evaluator 2013-03-14 17:39:24 +01:00
Shea Levy 152c392198 Whoops 2013-03-07 10:52:31 -05:00
Eelco Dolstra 6d131719be Respect SystemTypes if defined 2013-03-05 18:01:44 +01:00
Eelco Dolstra fcd511c4de Revert "Remove now-unused SystemTypes table" 
This reverts commit 71d020735b.
Unfortunately there are still some cases where we need to set Hydra's
concurrency separately.  (Ideally, Hydra would start *all* queued
builds in parallel and let Nix take care of everything...)
 2013-03-05 17:42:16 +01:00
Eelco Dolstra 5b6b9d37f0 Add a button to compare a jobset eval against other jobsets 
So now "?compare=<jobset name>" is no longer a hidden feature!

P.S. Encountered this wonderful TemplateToolkit brainfuck again: if
you want to get the number of rows in (say) project.jobsets, you can't
say "project.jobsets.size".  That will *usually* give the right
result, except that if there is only one row in project.jobsets, it
will evaluate to 3.  Instead you have to use "project.jobsets_rs.count".
 2013-03-05 17:41:50 +01:00
Eelco Dolstra fe7e0ff1f4 Show inputs and input changes on the jobset eval page 2013-03-05 17:41:50 +01:00
Shea Levy 71d020735b Remove now-unused SystemTypes table 2013-03-04 17:46:37 -05:00
Shea Levy d764c135ce hydra-queue-runner: Use nix.machines instead of the SystemTypes table to determine how many build jobs are allowed per system type. 
Note that on machines that support multiple system types, EACH system type gets the full number of build slots, which is almost certainly not what we want.
 2013-03-04 17:44:19 -05:00
Shea Levy 233e485a55 Remove the BuildMachines and BuildMachinesSystemTypes tables 2013-03-04 16:32:16 -05:00
Shea Levy 232170e301 Remove all entry points to modify machines 2013-03-04 16:05:50 -05:00
Shea Levy 170c7c98d0 machines: Read /etc/nix.machines instead of using the BuildMachines table 2013-03-04 16:03:04 -05:00
Shea Levy f9426f365b machine-status: Read /etc/nix.machines instead of using the BuildMachines table 2013-03-04 15:37:20 -05:00
Eelco Dolstra a77161e40a Allow users to edit their own settings 
Also, don't use the flash anymore for going back to the referer.
 2013-03-04 15:25:23 +01:00
Eelco Dolstra f831287d4b Add font for the captcha 
Taken from http://www.thibault.org/fonts/staypuft/.
 2013-03-04 12:16:13 +01:00
Eelco Dolstra e8cbcb50ac Add user registration 2013-02-27 18:33:47 +01:00
Eelco Dolstra 180068605a Active build steps: Make the rows clickable 2013-02-26 19:12:10 +01:00
Eelco Dolstra d1c2771ab8 Ensure that each eval is compared to the previous one from the same jobset 
Also, reduce the number of database hits.
 2013-02-26 19:08:39 +01:00
Eelco Dolstra 1007735364 Render links to changed input builds 
Also, use get_column('dependency') instead of dependency.id to prevent
an expensive row fetch.
 2013-02-26 18:55:59 +01:00
Eelco Dolstra 56a62c3d88 Make jobset deletion consistent with project deletion 2013-02-26 17:36:49 +01:00
Eelco Dolstra d596b58991 "Evaluate" command: push the jobset to the front of the eval queue 
Don't let hydra-server call hydra-evaluate.  That will probably just
timeout, get killed, etc.
 2013-02-26 16:56:19 +01:00
Eelco Dolstra eea3846754 After editing a jobset, push it to the front of the evaluation queue 2013-02-26 16:10:36 +01:00
Eelco Dolstra 96e24fcf24 Tweaks 2013-02-26 16:04:33 +01:00
Eelco Dolstra 5462be23d5 Don't show duration of unfinished builds 2013-02-26 16:01:00 +01:00
Eelco Dolstra 2a3e832fc4 Add some margin about h3 elements 2013-02-26 16:00:46 +01:00
Eelco Dolstra 4b70be6753 Fix same system inputs 2013-02-26 15:51:56 +01:00
Eelco Dolstra 5a0054f38c Fix adding jobset inputs 2013-02-26 15:45:34 +01:00
Eelco Dolstra 1e338e7958 api/push: Don't barf on non-existent jobsets 2013-02-26 01:46:19 +01:00
Eelco Dolstra f501648cd2 Add simpler push support for GitHub 
You can use the URL

  http://<hydra-server>/api/push-github

as GitHub's WebHook URL.  Hydra will automatically trigger an
evaluation of all affected jobsets.
 2013-02-26 01:14:50 +01:00
Eelco Dolstra 11d87c6f82 Use query_params instead of params 
Otherwise crazy things happen with POST requests from GitHub.
 2013-02-26 00:38:18 +01:00
Eelco Dolstra 5e6896b2d9 Turn prepared statements back on 
We once turned these off (in commit
abe71a767b) because they caused the
PostgreSQL query optimizer to use very suboptimal plans.  However,
PostgreSQL 9.2 has supposedly fixed this:

  http://www.postgresql.org/docs/9.2/static/release-9-2.html

So let's try again.
 2013-02-25 21:20:52 +01:00
Eelco Dolstra f51b93da03 Remove debug line 2013-02-25 21:18:29 +01:00
Eelco Dolstra 42d2015357 Support push notification of repository changes 
External machines can now notify Hydra that it should check a
repository by sending a GET or PUSH request to /api/push, providing a
list of jobsets to be checked and/or a list of repository URLs.  In
the latter case, all jobsets that have any of the specified
repositories as an input will be checked.

For instance, you can configure GitHub or BitBucket to send a request
to the URL

  http://hydra.example.org/api/push?repos=git://github.com/NixOS/nixpkgs.git

to trigger evaluation of all jobsets that have
git://github.com/NixOS/nixpkgs.git as an input, or to the URL

  http://hydra.example.org/api/push?jobsets=patchelf:trunk,nixpkgs:trunk

to trigger evaluation of just the specified jobsets.
 2013-02-25 21:10:32 +01:00
Eelco Dolstra ddcb9f1d5d Handle the case where a jobset has never been evaluated 2013-02-25 19:38:11 +00:00
Eelco Dolstra 65db523e46 Fix duplicate function name 2013-02-25 19:01:43 +01:00
Eelco Dolstra 24de044c55 hydra-evaluator: Always pick the jobset that hasn't been evaluated longest 2013-02-25 18:47:54 +01:00
Eelco Dolstra c30cee55dd Send ETag and Expires headers 2013-02-25 18:18:05 +01:00
Eelco Dolstra fb643cd971 Install bootbox 2013-02-23 17:01:43 +01:00
Eelco Dolstra 6658419f69 Disallow build products that are symlinks 
Otherwise you can do

  ln -s /etc/passwd $out/foo
  echo "file misc $out/foo" >> $out/nix-support/hydra-build-products

and get Hydra to serve its /etc/passwd file.
 2013-02-23 16:28:44 +01:00
Eelco Dolstra aa7ddeb8e9 Use hashFile instead of nix-hash 
Calling a shell command is a security hole if $path contains special
characters (e.g. "${foo}").  Observed in
http://hydra.nixos.org/build/4041321.
 2013-02-23 16:09:34 +01:00
Eelco Dolstra 67986b03fa Fix logo size 
Also, always include the text "Hydra" in the navbar.
 2013-02-23 14:31:30 +00:00
Eelco Dolstra 77fa6d2646 Set mouse pointer to indicate you can click on rows 2013-02-22 18:13:18 +01:00
Eelco Dolstra c0f9c9b4aa Make some more tables clickable 2013-02-22 18:11:29 +01:00
Eelco Dolstra 9422c9d2a7 Optimise clickable rows 
Set a click handler on the table instead of on every row.  This should
be faster on large tables.  Also, it's easier to use: you just set the
clickable-rows class on the table, and the row-link class on the <a>
element that contains the "main" link of the row.
 2013-02-22 18:05:04 +01:00
Eelco Dolstra bd7be1cb84 Remove obsolete CSS classes 2013-02-22 17:11:10 +01:00
Eelco Dolstra f5a2859ce2 Show current query in the search box 2013-02-22 16:44:39 +01:00
Eelco Dolstra 08dc6650e1 Search: Limit the number of results 2013-02-22 16:41:42 +01:00
Eelco Dolstra 4d4ea8c754 Search: Mark disabled projects/jobsets/jobs 2013-02-22 16:21:50 +01:00
Eelco Dolstra f3f6e8a4e3 Search: Don't return hidden projects/jobsets 2013-02-22 15:56:29 +01:00
Eelco Dolstra 07daff32b8 Add a search feature 
This allows searching for projects, jobsets or jobs by name or
description.
 2013-02-22 15:45:10 +01:00
Eelco Dolstra 4343781181 Clear failed builds etc.: Redirect back to the referrer 
Would be better to do this AJAX style though.
 2013-02-22 14:31:43 +01:00
Eelco Dolstra 629fe6f998 Jobset page: Load the jobs and status tabs on demand 
This makes the jobset page much smaller and faster.  (E.g. for
nixpkgs:trunk, this page was ~2.5 MB.)
 2013-02-22 14:29:12 +01:00
Eelco Dolstra f2de374f28 Set a 500 HTTP status on errors 2013-02-22 14:27:38 +01:00
Eelco Dolstra 961feedf4e Show an error message if tab loading fails 2013-02-22 14:17:14 +01:00
Eelco Dolstra 47d2ee0dab Generalize lazy tabs 2013-02-22 13:36:15 +01:00
Eelco Dolstra e2e973578f Some unnecessary job names 2013-02-22 13:17:07 +01:00
Eelco Dolstra 5d3494011f Job page: Put latest builds on top 
Also, rename the "Latest builds" tabs to prevent confusion.
 2013-02-22 13:00:40 +01:00
Eelco Dolstra 098368c3e3 Job page: Remove "Latest builds (latest evaluation)" 
It was kind of confusing since there are two other "latest builds"
tabs/sections.
 2013-02-22 12:53:10 +01:00
Eelco Dolstra 6dbb948f1f Job page: show queued as well as running builds 2013-02-22 12:52:11 +01:00
Eelco Dolstra 1643d90d92 Job page: Remove channel tab 2013-02-22 12:00:32 +01:00
Eelco Dolstra 0f10a37894 Fix indentation 2013-02-22 11:55:06 +01:00
Eelco Dolstra d08a391b43 Jobset page: Show last evaluation time 2013-02-22 11:37:35 +01:00
Eelco Dolstra c9ac7746b2 Allow showing all evaluations that contain a given build 2013-02-21 18:49:57 +01:00
Eelco Dolstra 6afe528570 Remove the related builds tab 
You can now just click on the evaluation link on the first tab to see
all builds in the same jobset.  This also makes rendering build pages
quite a bit faster for jobsets like Nixpkgs.
 2013-02-21 18:43:16 +01:00
Eelco Dolstra c6fe9f00bf Also show number of evaluations of which a build is a part 2013-02-21 18:42:18 +01:00
Eelco Dolstra edbe531ccc On build pages, provide a link to the build's first eval 2013-02-21 18:34:34 +01:00
Eelco Dolstra d8370142c2 Remove garbage from Users.pm 2013-02-21 18:20:55 +01:00
Eelco Dolstra c22d93438e Fix adding jobset inputs 2013-02-21 18:10:06 +01:00
Eelco Dolstra ba68826860 Add a page to show the latest evaluations for the entire server 2013-02-21 17:27:17 +01:00
Eelco Dolstra 767cab6cd2 Correctly redirect to the referring page when logging in 2013-02-21 14:38:18 +01:00
Eelco Dolstra 26fa9ea0ea Add links to the project/jobset edit pages in the logical place 2013-02-21 14:19:34 +01:00
Eelco Dolstra fec45d1f07 Remove dead code 2013-02-21 13:45:11 +01:00
Eelco Dolstra 36032e4bed Make the info tables less compressed 2013-02-21 13:42:44 +01:00
Eelco Dolstra dd5eb51a84 Put job status on a separate tab 
Also, show the "last checked time" on the initial tab.
 2013-02-21 13:30:14 +01:00
Eelco Dolstra 5099375549 Indicate disabled/hidden projects/jobsets in the overviews 2013-02-21 11:57:48 +01:00
Eelco Dolstra 8db83d2017 Turn hiding/unhiding a jobset into a checkbox 2013-02-21 02:36:51 +01:00
Eelco Dolstra 547e41263d Split viewing and editing a jobset 2013-02-21 02:33:57 +01:00
Eelco Dolstra 9c7b416f8b Make the hide/unhide actions a checkbox in the project settings 
Also use proper bootstrap layout for the project settings form.
 2013-02-21 01:45:39 +01:00
Eelco Dolstra 3924780eac Tweaks 2013-02-21 01:26:29 +01:00
Eelco Dolstra cea17fd57e Show releases in a tab on the project page 2013-02-21 01:23:42 +01:00
Eelco Dolstra d9f6e662d8 Split viewing and editing a project 2013-02-21 01:12:57 +01:00
Eelco Dolstra 03aee6726f Add a menu item for deleting jobsets 
Also add bootbox for sexier dialog boxes.
 2013-02-21 00:47:21 +01:00
Eelco Dolstra 176b8cab7e Use a proper spinning wheel icon 2013-02-20 23:57:33 +01:00
Eelco Dolstra b5273eed7a Don't generate clashing IDs 2013-02-20 18:26:26 +01:00
Eelco Dolstra d1e983dfc6 All/recent builds -> Latest builds 2013-02-20 18:23:49 +01:00
Eelco Dolstra 85e3f69cdd Remove debug statement 2013-02-20 18:22:06 +01:00
Eelco Dolstra 78250581ed There is only one channel, so drop the "(latest)" suffix 2013-02-20 18:20:16 +01:00
Eelco Dolstra af18808ca7 Dependency tree: Revert to "repeated" links 2013-02-20 18:15:42 +01:00
Eelco Dolstra 4552117e51 Load the dependency tabs on demand 2013-02-20 18:13:20 +01:00
Eelco Dolstra 36a1015444 Show the runtime/build dependencies in the build page 2013-02-20 17:58:27 +01:00
Eelco Dolstra 11c09e4c5c Always show the build duration (using the cached build if necessary) 2013-02-20 17:07:20 +01:00
Eelco Dolstra aebefda6ab Remove unnecessary Javascript code 
Tabs are activated automatically.
 2013-02-20 17:03:11 +01:00
Eelco Dolstra b5d59c5c63 Fix indentation of build.tt 2013-02-20 17:02:07 +01:00
Eelco Dolstra f544c63607 Formatting tweaks 2013-02-20 16:47:57 +01:00
Eelco Dolstra 7da2398f27 Allow non-admin users to see the machine status page 2013-02-20 16:40:09 +01:00
Eelco Dolstra 656b1bd45a Group some menu items together under a "Status" menu 2013-02-20 16:24:54 +01:00
Eelco Dolstra c63812afa5 Fix indentation 
Also move flashing into layout.tt.
 2013-02-20 16:14:30 +01:00
Eelco Dolstra ea9862fdc1 Give every page a consistent title 2013-02-20 15:54:33 +01:00
Eelco Dolstra 5c317fa0e9 Remove unused file 2013-02-20 15:42:40 +01:00
Eelco Dolstra 84602f9ee6 Ensure that the first tab is selected 2013-02-20 14:29:35 +01:00
Eelco Dolstra 3df9665f60 Compatibility with latest bootstrap 2013-02-20 14:06:17 +01:00
Eelco Dolstra c2e5bec1e1 Evil CSS hackery to prevent link targets from being under the navbar 
http://stackoverflow.com/questions/9047703/fixed-position-navbar-obscures-anchors
 2013-02-20 13:56:41 +01:00
Eelco Dolstra f550f1a1de Revert to jQuery 1.8.3 
1.9 removes the toggle() function which is actually useful (and used
by us).
 2013-02-20 01:49:01 +01:00
Eelco Dolstra 1feca4b105 Turn repeated dependencies into hyperlinks 2013-02-20 01:28:29 +01:00
Eelco Dolstra 18da3ac397 Render the dependencies as a tree 2013-02-20 01:02:18 +01:00
Eelco Dolstra 41d7d63339 Simplify the tree implementation a bit more 2013-02-20 00:54:03 +01:00
Eelco Dolstra 1a992d170d Factor out the tree rendering 2013-02-19 17:22:28 +01:00
Eelco Dolstra 4266632eb0 Simplify the log tree implementation a bit 
In particular use the :last-child selector, which is non-standard but
supported by all major browsers.
 2013-02-19 17:14:02 +01:00
Eelco Dolstra 8c37f46955 Update jquery/jqueryui 2013-02-19 15:22:15 +01:00
Eelco Dolstra 7b4d4f1cd7 Move <style> to hydra.css 2013-02-19 14:39:50 +01:00
Eelco Dolstra 22ed3c4847 Fix some XML well-formedness problems 2013-02-14 18:03:12 +01:00
Eelco Dolstra 5620e3c386 Fix indentation and get rid of some unnecessary whitespace in the output 2013-02-14 17:51:09 +01:00
Eelco Dolstra 2fa1a7efa7 Get rid of unnecessary [%- and -%] tags 2013-02-14 16:51:42 +01:00
Eelco Dolstra 82daf4f8ca Show dependencies as a tree rather than a list 2013-02-14 15:53:34 +01:00
Eelco Dolstra 4e67665b7f Builds page: Remove redundant tab titles 2013-02-14 13:23:54 +01:00
Eelco Dolstra 1160e17bfc Add missing files 
Doh!
 2013-02-14 11:53:09 +01:00
Eelco Dolstra 2b3c72efeb Use getMainOutput 2013-02-13 18:36:01 +01:00
Eelco Dolstra eb9d0c8992 Security: Also check paths in the web server 2013-02-13 18:34:33 +01:00
Eelco Dolstra cd83b8e3aa Security: Ensure that a build product refers to the Nix store 2013-02-13 18:26:00 +01:00
Eelco Dolstra 10882a1ffd Add multiple output support 
This requires turning the outPath columns in the Builds and BuildSteps
tables into separate tables, and so requires a schema upgrade.
 2013-02-13 16:49:28 +00:00
Eelco Dolstra 799e5437bd Remove unnecessary "Page generated" line 2013-02-05 14:08:20 +01:00
Eelco Dolstra e555c7bb36 Automatically chomp all [%...%] directives 
This makes the use of [%-...-%] unnecessary.
 2013-02-05 14:00:09 +01:00
Eelco Dolstra 58dd49e645 Fix handling of IPC::Run::run exit status 
Turns out that the exit status is returned in $?, not as the return
value of run().  So our error checking was completely bogus.
 2013-01-23 15:56:28 +01:00
Eelco Dolstra e0de5de91d Fix some warnings 2013-01-23 12:41:57 +00:00
Eelco Dolstra 66c107364b Add missing file 
Doh!
 2013-01-23 12:49:44 +01:00
Eelco Dolstra f3d6643724 Omit the small build status icon on the build page 
It's kinda pointless next to the big-ass icon.
 2013-01-22 23:05:02 +01:00
Eelco Dolstra ecdbce1a61 Handle active build steps of aborted builds properly 2013-01-22 23:01:29 +01:00
Eelco Dolstra 30e5185acf Remove the logfile and logSize columns from the database 
It's pointless to store these, since Nix knows where the logs are.
Also handle (in fact require) Nix's new log storage scheme.  Also some
cleanups in the build page.
 2013-01-22 22:48:02 +01:00
Eelco Dolstra 67aefde62c Remove trailing whitespace 2013-01-22 14:41:02 +01:00
Eelco Dolstra aa28ffe3a8 Die tabs die 2013-01-22 14:09:37 +01:00
Eelco Dolstra 486b6037a9 Use $c->config 2013-01-22 14:06:12 +01:00
Eelco Dolstra f188fe5683 hydra-evaluator: Don't require $HYDRA_CONFIG 2013-01-22 13:19:28 +01:00
Eelco Dolstra fc39034772 Whitespace 2013-01-22 13:19:08 +01:00
Eelco Dolstra f3a274657d Trailing whitespace 2013-01-22 12:05:33 +01:00
Rob Vermaas d4e49773c5 Allow dashes in attrNames 2013-01-11 12:16:21 +01:00
Eelco Dolstra 64399991a7 Give Hydra binary caches a low priority 
We only want to hit a Hydra binary cache if static caches don't
provide a binary.
 2012-11-06 17:13:17 +01:00
Eelco Dolstra 2906116fda Log time required by a request 2012-11-05 17:37:23 +01:00
Eelco Dolstra 9ba1fd6952 Turn off overly verbose log messages by default 
The "Request took ...s" messages are rather spammy, so turn them off
by default.  You can set CATALYST_DEBUG=1 if you need them.
 2012-11-05 17:21:09 +01:00
Ludovic Courtès 183078131a Capture the path to `guile', when available. 2012-10-17 16:25:49 +02:00
Ludovic Courtès f27ae1d566 Add support for Guile & Guix. 2012-10-17 16:23:00 +02:00
Rob Vermaas 4ade8c5a1b Add dollar to pathCompRE, which allows files with dollars in them to be served by Hydra, e.g. sphinx generated docs often has these types of files. 2012-10-12 13:48:11 +02:00
Eelco Dolstra 5fa688acf5 Don't ignore assertion failures in getDerivation() 2012-10-04 15:24:25 -04:00
Eelco Dolstra a4ff9b0d08 hydra-eval-jobs: Don't go into an infinite recursion 
The function getDerivation() can return false if its argument is a
derivation.  This happens if evaluating the name or system attribute
triggers an assertion.  In that case, we shouldn't recurse into the
attributes of the derivation.
 2012-10-04 14:31:47 -04:00
Rob Vermaas f4a24b94ee Fallback for git fetch, which allows specific gits revisions to be used as build inputs. 2012-09-11 22:31:49 +02:00
Rob Vermaas c57242c76d Show build status for up to 100 jobs in stead of 50. 2012-09-10 12:15:31 +02:00
Rob Vermaas 396a356198 Add create-user action that was mostly present already. 2012-08-06 00:02:14 +02:00
Rob Vermaas 90c9b5dd60 Do not send emails when build is cancelled/aborted. Also, ignore aborted/cancelled builds in comparing to previous build. 2012-08-06 00:00:07 +02:00
Eelco Dolstra aa8bf564e2 Set the Content-Type properly 2012-08-01 18:00:55 +00:00
Eelco Dolstra 13688e364f Add a binary-cache-url file to channels to advertise the binary cache URL 2012-08-01 16:31:25 +00:00
Eelco Dolstra fad2a084fa Provide the nix-cache-info file required by the binary cache substituter 2012-07-30 20:26:34 +00:00
Eelco Dolstra d15dd6bac9 Include the URL of the binary cache in .nixpkg files 2012-07-30 20:26:11 +00:00
Eelco Dolstra 74fb2b37ce Add more spacing between products 2012-07-19 00:33:02 +02:00
Eelco Dolstra 00155aa7b1 Add an icon for binary distributions 2012-07-19 00:16:22 +02:00
Eelco Dolstra 67528698e6 Allow creating a release from an evaluation 2012-07-18 23:46:41 +02:00
Eelco Dolstra 2ab4c7d597 Update queryPathInfo calls 2012-07-18 23:14:45 +02:00
Eelco Dolstra da960b507d Use queryPathFromHashPart to look up binary cache requests efficiently 2012-07-18 23:14:17 +02:00
Rob Vermaas 544f2f2503 Removing tarball as option for build input; does not exist. 2012-07-13 13:06:55 +02:00
Eelco Dolstra 1b3cf68b77 Generate *.narinfo files on the fly to support the binary cache substituter 2012-07-02 20:09:45 +02:00
Eelco Dolstra fe2dab6fe8 Fix some warnings about undefined values 2012-07-02 15:18:30 +00:00
Rob Vermaas d37c2c239c export logContents 2012-06-26 12:09:05 +02:00
Rob Vermaas a2f56ce574 * Read logs using logContents function in stead of handling it everywhere separately. 2012-06-26 12:00:18 +02:00
Rob Vermaas 0701cd0da8 * Fix getBuildLog for bzip2'd files. 2012-06-26 08:04:35 +02:00
Rob Vermaas 2aa52131ac * Nix now stores logs by default as bzip2, make sure the build page uncompresses before showing. 2012-06-25 15:05:16 +02:00
Eelco Dolstra ab4f0911ce Fix rendering of Subversion inputs in the Evaluations table 
Broken since commit 75277311ac.

Fixes #30.
 2012-06-04 14:55:24 +00:00
Eelco Dolstra fdf441a8b7 Set utf-8 encoding on raw log pages 2012-05-25 14:27:56 +00:00
Rob Vermaas 75277311ac Different icons (darker red) for cancelled builds. Fixes #25. 2012-05-25 04:14:36 +02:00
Eelco Dolstra d1da9a8823 Escape the log tail to ensure we produce correct HTML 2012-05-23 20:14:51 +00:00
Eelco Dolstra fdc516289c Doh - clear $NIX_PATH properly 2012-05-23 14:44:23 -04:00
Rob Vermaas 97d41a2102 Allow project members to create jobsets, regression introduced by bootstrap make-over. Currently all logged in users see these options, however, only on performing the action the privileges are checked. Need to change that in the future. 2012-05-18 10:09:53 +02:00
Rob Vermaas da206fdfac Add time to evaluation error tab. Fixes #21. 2012-05-18 10:06:55 +02:00
Rob Vermaas b591c443b0 Keep builds of disabled jobsets that are not hidden. Fixes #18. 2012-05-11 09:11:07 +02:00
Eelco Dolstra 8538953868 Don't write one character at a time, it's inefficient 2012-05-10 12:02:41 -04:00
Eelco Dolstra 48effa116c Delete obsolete CSS files 2012-05-07 15:12:56 -04:00
Eelco Dolstra 7d3c1852ed "r. " -> "r" in showing revisions 2012-05-04 17:12:45 -04:00
Rob Vermaas 0fdcd3176b Add numbered revision id of mercurial as revCount. 2012-05-01 09:15:52 +02:00
Eelco Dolstra 8f85fe67d4 Revert sorting build products 
Sorting the build products list fails if there are fewer than 3 build
products.  This is because of extreme Template Toolkit brain damage:
it treats arrays of 0-2 elements differently in some magic way.  See
e.g.

  http://mail.template-toolkit.org/pipermail/templates/2011-April/011510.html
 2012-04-30 22:19:38 +02:00
Rob Vermaas df71a86183 Do not delete builds when clearing the queue. Fixes #8. 2012-04-30 22:13:53 +02:00
Eelco Dolstra b52018e61a Use a span instead of a div in the pretty logs 
Fixes #6.
 2012-04-30 21:32:53 +02:00
Rob Vermaas eb51065a16 Making pretty printed log look normal again. Issue #6 2012-04-30 20:22:03 +02:00
Rob Vermaas 437e59c4a8 When checking if build is already built, check for potential 'failed with result'. Fixes issue #7. 2012-04-30 19:47:21 +02:00
Eelco Dolstra 854513be8e I should test first 2012-04-30 17:57:13 +02:00
Eelco Dolstra dcc570f454 Set the build status properly for failing local builds 
If a build has ‘preferLocalBuilds = true’ (or we're not using remote
building), and the build has a non-permanent failure, then the build
status should be "Aborted" rather than "Failed".  This is denoted by
an exit status of 100 from nix-store.
 2012-04-30 17:15:35 +02:00
Eelco Dolstra 4a6f798ef4 Fix Mercurial input fetching if a cached input has been GC'ed 
Fixes Hydra/76.
 2012-04-30 16:21:28 +02:00
Eelco Dolstra 1033178728 Ensure build products are in sorted order 2012-04-27 11:23:38 +02:00
Eelco Dolstra 29b40e6418 Fix the duration on Build pages 2012-04-27 10:54:49 +02:00
Rob Vermaas 4a1a2203ac I should sleep... grmbl 2012-04-26 09:37:48 +02:00
Rob Vermaas 3c465c3661 Revert nix-prefetch-git changes merged in earlier from Merge request #3, which broke git describe functionality used in GNU jobsets. 2012-04-26 09:31:00 +02:00
Rob Vermaas 6d5567bb16 Use git fetch -fu origin +$branch:$branch to fetch all changes for branch. 2012-04-26 08:36:40 +02:00
Rob Vermaas be3748e1ec In case of deep clone, make sure there is local branch. 2012-04-25 13:49:28 +02:00
Rob Vermaas 7da48b5220 remove origin from git pull call, to get rid of fatal: fetch --all does not take a repository argument error 2012-04-25 10:17:22 +02:00
Rob Vermaas b49e138e9c Merge pull request #3 from nbp/candidates/fetch-git 
Optimize fetch-git.
 2012-04-24 10:18:28 -07:00
Rob Vermaas 966cc22131 Fix query for certain postgresql versions. 2012-04-22 08:30:48 +02:00
Rob Vermaas b0596c214a Merge branch 'master' of github.com:NixOS/hydra 2012-04-18 11:24:12 +02:00
Rob Vermaas cc38d0352f Strip all whitespace from revision from subversion call. Patch provided by Petr Ročkai 2012-04-18 11:22:26 +02:00
Eelco Dolstra f8c914297f Fix path to the session data 2012-04-17 22:25:32 +02:00
Eelco Dolstra 51b920c875 Bootstrapify the Hydra forms (except the project and jobset edit pages) 
Plus lots of other tweaks.
 2012-04-17 16:53:11 +02:00
Eelco Dolstra 8f31935ffa Handle the case where there are no builds and no previous eval 2012-04-17 12:32:44 +02:00
Eelco Dolstra db09760e8c Move the "Cancel build" button to the menu 2012-04-17 11:39:08 +02:00
Eelco Dolstra ea4aba83c3 Proper styling for flash message 2012-04-17 09:36:10 +00:00
Eelco Dolstra 634d8c092f Use <h3> for running/failed build steps 2012-04-17 09:35:37 +00:00
Eelco Dolstra 896a47d950 Clear nrSucceeded when restarting a build 2012-04-17 09:34:35 +00:00
Eelco Dolstra d350b935f2 Add validation for project and jobset names 2012-04-17 08:53:00 +00:00
Rob Vermaas 90e0ba2a7a Revert changeset 4399683fa0, make jobset options available to project members as well 2012-04-17 09:49:27 +02:00
Ludovic Courtès 4399683fa0 Reinstate the edit/clone links on the jobset page. 
They were removed in commit 17d30cd179.
 2012-04-16 16:47:23 +02:00
Rob Vermaas 922bad7306 Also remove dep. graph from robots.txt. 2012-04-16 15:25:14 +02:00
Rob Vermaas 334135448e Remove buildtime and runtime dependency graph. It is useless on normal derivations. 2012-04-16 15:23:07 +02:00
Rob Vermaas 28b6ff3ed6 Add maximum nest level for input changes on build page. 2012-04-16 13:59:29 +02:00
Eelco Dolstra 1cf2a1592a Add missing upgrade-*.sql files 2012-04-15 23:23:01 +00:00
Eelco Dolstra ec87ad2bf2 Missing part of aaacf9eda3 2012-04-15 22:57:10 +00:00
Eelco Dolstra aaacf9eda3 Speed up the listing of jobset evals by caching some info in the DB 
This denormalises the database, but it's the best we can do until
PostgreSQL has materialised views.
 2012-04-16 00:46:19 +02:00
Eelco Dolstra 034d50069b Speed up the jobset eval list a bit 2012-04-15 22:11:20 +00:00
Eelco Dolstra 573a333bc9 Cleanup 2012-04-15 21:49:54 +00:00
Eelco Dolstra 5793baa3cd Doh 2012-04-15 21:35:46 +00:00
Eelco Dolstra 1e228267be Only show "Running build steps" if there are running build steps 2012-04-15 20:12:35 +00:00
Eelco Dolstra 2c6a38760b Consistent capitalisation 2012-04-15 20:10:22 +00:00
Eelco Dolstra 0daba6bb89 In the jobset evals lists, show what inputs changed between consecutive evals 2012-04-15 20:06:42 +00:00
Eelco Dolstra b9824ca422 Cleanup 2012-04-15 18:47:22 +00:00
Eelco Dolstra fd50ac1d4e Store the inputs of each evaluation in the database 
Achtung: this requires a schema upgrade via "hydra-init".
 2012-04-15 18:36:36 +00:00
Eelco Dolstra 12dd78d889 Change the "Available" link to "pretty" 
"Available" is confusing in this context.  "Pretty" denotes that this
is the pretty-printed log.
 2012-04-15 18:51:34 +02:00
Eelco Dolstra e7fa54e48e Fix the spacing around some <h3> headers 2012-04-15 18:48:29 +02:00
Eelco Dolstra 80705c8a20 hydra-init: show SQL commands being executed 2012-04-15 18:34:32 +02:00
Eelco Dolstra bf757f687d Remove close button on news items since it's not persistent, so kinda useless 2012-04-15 18:31:47 +02:00
Eelco Dolstra a633e4b361 Make the jobset eval rows clickable 2012-04-15 18:21:56 +02:00
Eelco Dolstra 6b88640457 Remove the "There are evaluation errors!", instead emphasize the presence of the tab more 2012-04-15 18:17:19 +02:00
Eelco Dolstra 101e60b03d Nicer link to the rest of the jobset evals 2012-04-15 18:08:51 +02:00
Eelco Dolstra 794602d7ce Drop unused "tag" column 
It was intended for CVS inputs but never used.
 2012-04-15 12:42:46 +00:00
Nicolas Pierron 19d9955e89 Optimize fetch-git. 2012-04-14 18:17:35 -07:00
Rob Vermaas ccc5d38976 Move Hide/Unhide links for project to toplevel menu. 2012-04-13 16:52:25 +02:00
Rob Vermaas 7f858928df Fix hide/unhide links. 2012-04-13 16:36:45 +02:00
Rob Vermaas fbb857bf80 Do not load charts lazily for now. 2012-04-13 14:50:48 +02:00
Rob Vermaas 320632f7f4 Renamed running to started in queue. 2012-04-13 12:55:26 +02:00
Rob Vermaas f339f5ae85 Fix evaluate link. 2012-04-13 12:42:58 +02:00
Rob Vermaas 516a644a7f Merge branch 'master' into bootstrap 2012-04-13 11:55:49 +02:00
Rob Vermaas 293c7d3507 update bootstrap.zip 2012-04-13 11:54:40 +02:00
Rob Vermaas bb4f470348 Remove default logo, replaced by text for now. Hide template in jobset edit. 2012-04-13 11:47:05 +02:00
Rob Vermaas f63e9a63b4 Replace last 5 builds on jobset page with last 5 evaluations. Moved hide/unhide and evaluate options to jobset topmenu. 2012-04-13 11:26:37 +02:00
Rob Vermaas 11bf848b09 Move channel link to top menu of Project and Jobset. 2012-04-13 10:56:02 +02:00
Rob Vermaas bf197f0d77 Remove pagers on the bottom of paginated lists. 2012-04-13 10:43:56 +02:00
Rob Vermaas b81aa89260 Replace old logo with Nix/NixOS logo. 2012-04-13 10:33:11 +02:00
Rob Vermaas 1e7f9c8a15 Fix pager prev/next button. 2012-04-13 10:23:27 +02:00
Rob Vermaas 6b65712045 Restructure build menu 2012-04-13 10:19:15 +02:00
Rob Vermaas 9a1f38e06e Remove comments, re-add logo. 2012-04-13 10:05:02 +02:00
Rob Vermaas 4f5327d7ed Added running icon in queue in stead of background color. 2012-04-12 22:10:16 +02:00
Rob Vermaas 8aaa0a1a8a Move 'keep build' and 'restart' to toplevel menu 2012-04-12 21:50:06 +02:00
Rob Vermaas 089a9c836f Machine status page now looks much better and more consistent with rest of Hydra 2012-04-12 21:42:21 +02:00
Rob Vermaas 6853d0398a Move 'clone build' and 'restart build' to top menu 2012-04-12 21:41:31 +02:00
Rob Vermaas c630433b61 Move 'clone build' to top menu 2012-04-12 21:40:57 +02:00
Rob Vermaas 7886666b0b Replace ugly link with nice small buttons 2012-04-12 21:03:59 +02:00
Rob Vermaas 72785f35aa Some overriding of bootstrap default css 2012-04-12 21:03:25 +02:00
Rob Vermaas 06d6f5b5f7 remove header on build page 2012-04-12 20:19:52 +02:00
Rob Vermaas 5748262517 Add bootstrap.zip 2012-04-12 20:13:04 +02:00
Rob Vermaas 17d30cd179 Using twitter bootstrap for more consistent looks for Hydra 2012-04-12 20:12:07 +02:00
Ludovic Courtès 6e62d9b7b5 Allow use of a single Git clone for several branches. 
This reinstates commit 53329ecc61, but
uses "git pull --all", instead of "git fetch --all", so that local
tracking branches get updated too.
 2012-04-12 16:55:33 +02:00
Rob Vermaas ed23a6e6d3 Add branch to local clone path of git repositories for now. 2012-04-10 10:06:12 +02:00
Rob Vermaas 79211d5efe Revert "Use git fetch --all to update local clone in stead of git pull. Also, do not use --branch in initial clone." 
This reverts commit 53329ecc61.
 2012-04-10 10:04:22 +02:00
Rob Vermaas 53329ecc61 Use git fetch --all to update local clone in stead of git pull. Also, do not use --branch in initial clone. 2012-04-05 15:49:48 +02:00
Eelco Dolstra 8b88ce19c2 Huuuge speedup in the Hydra evaluator 
The check to see whether a build had been scheduled in a previous
evaluation took about 200 ms for the nixpkgs:trunk jobset.  Given
that it has more than 15000 builds, this added up to a lot.  Now
it takes 0.2 ms per build.
 2012-04-04 17:29:03 +02:00
Eelco Dolstra 1cdc023427 Propagate URI parameters 2012-04-04 13:17:22 +02:00
Eelco Dolstra e9dd3158a0 Propagate URI parameters 
This allows URIs like

  http://hydra.nixos.org/jobset/nixpkgs/trunk/latest-eval?compare=stdenv

to do the right thing.
 2012-04-04 12:56:49 +02:00
Eelco Dolstra 05d03ee153 Don't use colspan=0, it doesn't work in all browsers 2012-04-04 10:33:00 +00:00
Eelco Dolstra f52ca0c588 Add a redirect to the latest view result for which the underlying evaluation has finished completely 
This will be useful for the Nixpkgs channel mirror script:

  http://hydra.nixos.org/view/nixpkgs/unstable/latest-finished/channel

is the channel containing the latest, consistent, tested set of builds.
 2012-04-03 17:45:03 +02:00
Eelco Dolstra bd93ca9316 Add a redirect to the evaluation page of a view result 
E.g.

  http://hydra.nixos.org/view/nixpkgs/unstable/latest/eval

Thus this is the channel representing the latest Nixpkgs for which the
elements of the view "unstable" succeeded:

  http://hydra.nixos.org/view/nixpkgs/unstable/latest/eval/channel
 2012-04-03 15:45:48 +02:00
Eelco Dolstra 2c677ec71a Always record inputs passed through -I in the BuildInputs table 
We currently have no way to determine if the Nix evaluator used a
specific -I input, so we need to record all of them as inputs.
 2012-04-03 10:10:45 +00:00
Eelco Dolstra 30e2b9046a Cleanup 2012-04-03 11:28:59 +02:00
Eelco Dolstra db2b2b6134 Add a redirect to the latest finished jobset evaluation 
The action .../jobset/<project>/<jobset>/latest-eval redirects to the
latest evaluation of the jobset that has no unfinished builds.  Thus,
for instance,

  http://hydra.nixos.org/jobset/nixpkgs/trunk/latest-eval/channel

is the channel containing the latest consistent set of Nixpkgs builds.
 2012-04-03 01:30:54 +02:00
Eelco Dolstra 262f068e26 Support obtaining the contents of an evaluation as a channel 
E.g. http://hydra.nixos.org/eval/552513/channel

The advantage of such a channel is that all builds come from the same
evaluation, and thus are consistent.
 2012-04-02 23:27:17 +02:00
Eelco Dolstra 3e57cbfeb8 User color instead of a big animated GIF to indicate whether a build is running 2012-04-02 21:07:49 +02:00
Eelco Dolstra 69e600da89 Allow comparing an evaluation with an arbitrary other evaluation 
The URI parameter "compare=..." can denote either an arbitrary
evaluation ID, or the name of a jobset in the same project.  In the
latter case, the comparison is made against the latest completed
evaluation of the specified jobset.
 2012-04-02 20:40:59 +02:00
Eelco Dolstra 2f9153c640 Prevent multiple builds with the same (job, outPath) tuple from being added 
This happened in a pathological case in Nixpkgs: the "grub" job is
evaluated for i686-linux and x86_64-linux, but in the latter case it
returns the same derivation as in the former case.  So only one build
should be added.
 2012-04-02 15:56:29 +00:00
Eelco Dolstra 9d4cb29e08 Show queued builds in jobset evaluation pages 2012-04-02 15:23:42 +00:00
Eelco Dolstra 427e5e8db8 Show new and removed jobs 2012-04-02 16:45:05 +02:00
Eelco Dolstra 662cdf0421 Add support for viewing jobset evaluations 2012-04-02 16:11:22 +02:00
Eelco Dolstra d33fbad438 Unpack jQuery etc. in the build tree for easier testing 2012-04-02 11:15:48 +00:00
Eelco Dolstra 0aae52bfcc In build tables, hide the project, jobset and/or job name if they're constant 2012-04-02 13:12:39 +02:00
Eelco Dolstra d8f8143cc2 No wonder our disk was filling up 2012-03-26 17:13:50 +02:00
Rob Vermaas 4d4e1986e8 remove delete jobset button until we actually have implemented the action properly 2012-03-26 14:39:50 +02:00
Eelco Dolstra e1768cae86 Don't barf if the SQLite DB is missing 
This prevented hydra-init from starting.
 2012-03-19 03:57:11 +00:00
Ludovic Courtès 3846c2407d Fix URL in the "password changed" email. 2012-03-13 18:30:57 +01:00
Ludovic Courtès 2b68e41ee4 Actually add SQL files to the distribution. 2012-03-13 17:49:18 +01:00
Ludovic Courtès ace94192ac Update the list of SQL files to be installed. 
This fixes `hydra-init', which would otherwise not find `upgrade-*.sql',
thus skipping the update altogether.
 2012-03-13 17:45:14 +01:00
Eelco Dolstra dae5032c1c I'm stupid 2012-03-13 13:33:19 +01:00
Eelco Dolstra c844b9f09a Remove the global Job status and Errors links 2012-03-13 13:31:36 +01:00
Eelco Dolstra 161d7418f8 Doh 2012-03-13 13:30:41 +01:00
Eelco Dolstra 1f268d2d43 Update isCurrent properly 2012-03-13 13:09:10 +01:00
Eelco Dolstra 179b012a8e Open the DB using Hydra::Model::DB->new 
This gets rid of the openHydraDB function and ensures that we
open the database in a consistent way.

Also drop the PostgreSQL sequence hacks.  They don't seem to be
necessary anymore.
 2012-03-13 12:10:19 +01:00
Eelco Dolstra 13f4636436 hydra-evaluator: handle the case where there is no previous jobset eval 2012-03-12 21:13:28 +01:00
Eelco Dolstra a4cda5e337 Don't use the Switch module 
It was removed in Perl 5.14.
 2012-03-12 20:47:30 +01:00
Eelco Dolstra 87e4d43848 Fix the jobset unchanged check 
When checking whether the jobset is unchanged, we need to compare with
the previous JobsetEval regardless of whether it had new builds.
Otherwise we'll keep adding new JobsetEval rows.
 2012-03-12 20:47:30 +01:00
Eelco Dolstra 27619f18e2 Re-enable the "Related Builds" tab 
However, there is a "limit 1" on the JobsetEval search to prevent a
sequential scan of the Builds table.
 2012-03-12 20:47:30 +01:00
Eelco Dolstra abe71a767b Disable prepared statements completely 
Because of the way DBIx::Class does prepared statements, even
innocuous queries such

  $c->model('DB::Builds)->search({finished => 0})

can be extremely slow.  This is because DBIx::Class prepares a
PostgreSQL statement

  select ... from Builds where finished = ?

and since Builds is very large and there is a large fraction of rows
with "finished = 1", the PostgreSQL query planner decides to implement
this query with a sequential scan of the Builds table (despite the
existence of an index on "finished"), which is extremely slow.  It
would be nice if we could tell DBIx::Class that constants should be
part of the prepared statement, i.e.

  select ... from Builds where finished = 0

but AFAIK we can't.
 2012-03-12 20:47:30 +01:00
Eelco Dolstra 2d1cf73974 Speed up channel processing 
In particular the /pkg action is now O(lg n) instead of O(n) in the
number of packages in the channel, and listing the channel contents
no longer requires calling isValidPath() on all packages.

Derivations (and thus build time dependencies) are no longer included
in the channel, because they're not GC roots.  Thus they could
disappear unexpectedly.
 2012-03-12 20:47:30 +01:00
Eelco Dolstra 9f10c0f9c0 Fix a reference to BuildResultInfo 2012-03-12 20:47:30 +01:00
Eelco Dolstra 07b3dffd20 Reduce I/O in build listings by only fetching required columns 
Columns such as "longDescription" can be large, so fetching them
when they're not needed is wasteful.
 2012-03-12 20:47:30 +01:00
Eelco Dolstra 6526d4a65f hydra-build: only send email if the status differs from the previous build 
This isn't perfect because it doesn't handle the case where a
previous build hasn't finished yet.  But at least it won't send mail
for old builds that fail while a newer build has already succeeded.
 2012-03-12 20:47:29 +01:00
Eelco Dolstra 47f877c5bb Evaluator cleanups 
* Don't use isCurrent anymore; instead look up builds in the previous
  jobset evaluation.  (The isCurrent field is still maintained because
  it's still used in some other places.)

* To determine whether to perform an evaluation, compare the hash of
  the current inputs with the inputs of the previous jobset
  evaluation, rather than checking if there was ever an evaluation
  with those inputs.  This way, if the inputs of an evaluation change
  back to a previous state, we get a new jobset evaluation in the
  database (and thus the latest jobset evaluation correctly represents
  the latest state of the jobset).

* Improve performance by removing some unnecessary operations and
  adding an index.
 2012-03-12 20:47:29 +01:00
Eelco Dolstra bc82a82593 Fix a race condition in hydra-update-gc-roots 
Since it read the actual roots after determining the set of desired
roots, there was a possibility that it would delete roots added by
hydra-evaluator or hydra-build while hydra-update-gc-roots was
running.  This could cause a derivation to be garbage-collected before
the build was performed, for instance.  Now the actual roots are read
first, so any root added after that time is not deleted.
 2012-03-12 20:47:29 +01:00
Eelco Dolstra 29d5a02b94 Speed up hydra-update-gc-roots 
The hydra-update-gc-roots script is taking around 95 minutes on our
Hydra instance (though a lot of that is I/O wait).  This patch
significantly reduces the number of database queries.  In particular,
the N most recent successful builds for each job in a jobset are now
determined in a single query.  Also, it removes the calls to
readlink().
 2012-03-12 20:47:29 +01:00
Eelco Dolstra 5be004c999 Drop the merged tables after conversion 2012-03-12 20:47:29 +01:00
Eelco Dolstra 8d65ab6158 Don't use a prepared statement for the active build steps query 
Prepared statements are sometimes much slower than unprepared
statements, because the planner doesn't have access to the query
parameters.  This is the case for the active build steps query (in
/status), where a prepared statement is three orders of magnitude
slower.  So disable the use of prepared statements in this case.

(Since the query parameters are constant here, it would be nicer if we
could tell DBIx::Class to prepare a statement with those parameters
fixed.  But I don't know an easy way to do so.)
 2012-03-12 20:47:29 +01:00
Eelco Dolstra 68a867da67 Merge the BuildResultInfo table into the Builds table 2012-03-12 20:47:29 +01:00
Eelco Dolstra 25334715f8 Merge the BuildSchedulingInfo table into the Builds table 
This simplifies the code and improves performance since it reduces
the number of joins.
 2012-03-12 20:47:29 +01:00
Eelco Dolstra 19fe4b9b4a Add hydra-init to the tarball 2012-03-05 15:35:13 +01:00
Eelco Dolstra 541238030d Provide a command ‘hydra-init’ to initialise/upgrade the database 
For schema upgrades, hydra-init executes the files
src/sql/upgrade-<N>.sql, each of which upgrades the schema from
version N-1 to N.  The upgrades are wrapped in a transaction.
 2012-02-28 20:16:16 +01:00
Eelco Dolstra da26294fdb Don't install hydra-create 
It doesn't do anything useful for users.
 2012-02-28 15:33:28 +01:00
Eelco Dolstra b13664308c Whitespace 2012-02-28 15:27:44 +01:00
Eelco Dolstra 3d824b3a3a Indentation 2012-02-21 17:59:36 +01:00
Eelco Dolstra 578530dac2 Don't show duplicate builds in "Related builds" 
Since a build may be a member of multiple jobset evaluations, we need
to do a "select distinct" here.  But maybe we should only show builds
from a single evaluation (e.g. the most recent), since showing builds
from several may be confusing.
 2012-02-21 17:16:32 +01:00
Rob Vermaas 2cf9b148eb 50 for related builds might be a kind of low, will start with 100 2012-02-16 17:00:59 +01:00
Rob Vermaas 924d94cafd * minor performance tweaks 
* added related builds tab which show related builds (builds that are in same evaluations), shows max 50
 2012-02-16 16:56:01 +01:00
Rob Vermaas 355135b6e8 add links in admin page to clear queue (all non-running builds, and all non-running old (non-current) builds) 2012-02-16 15:31:12 +01:00
Rob Vermaas bb6ef9c2ed accidentally committed an = too little, which caused all unknown values to result in 'not building', which was not the intention 2012-02-16 11:14:22 +01:00
Rob Vermaas 739a03393c allow null values for jobs, meaning it should build 2012-02-16 11:03:22 +01:00
Ludovic Courtès 1f448673f0 hydra-build: Do not send email following an abortion. 2012-02-14 11:48:44 +01:00
Eelco Dolstra a3cc2ce796 Remove redundant dot in status emails 
E.g.:

  This is to let you know that Hydra build 1998534 of job patchelf:trunk:deb_ubuntu1110x86_64 is 'Success'..
 2012-02-07 14:51:47 +01:00
Ludovic Courtès 5644c1c7d9 hydra-build: Add system info to the subject and extra headers. 2012-01-31 18:27:33 +01:00
Rob Vermaas 8d78a688c7 is not in hydra-server's scope 2012-01-28 21:03:44 +01:00
Rob Vermaas 1c0cd99c3e remove old line 2012-01-28 20:55:33 +01:00
Rob Vermaas fded9e62af logdiff now support bzipped logs for logdiff 2012-01-28 20:47:01 +01:00
Rob Vermaas ae491f7223 use base_uri config in password reset email 2012-01-28 07:18:10 +01:00
Rob Vermaas 01f07be8f2 fix; resetting password used to send email to logged in user (typically admin), now to the user whose password is resetted 2012-01-28 07:14:40 +01:00
Rob Vermaas 1840a842d3 fix build links in input diff's, showed only current build in each link 2012-01-22 09:03:57 +01:00
Rob Vermaas 9b4f09ba83 allow dots (.) in job names when used as build input 2012-01-06 23:04:48 +01:00
Rob Vermaas 203a9b9ad7 make filesize field for buildproducts a bigint to support big files 2011-12-10 14:48:33 -05:00
Eelco Dolstra 9fe8c43437 Pass the abbreviated Git revision 
The abbreviated Git revision hash (e.g. "267480b") is typically
contained in ‘gitTag’ as well, but the latter can contain other
elements as well, e.g., the delta to the closest tag.  That may
be undesirable in version strings, so this is an alternative.
 2011-12-06 23:04:15 +01:00
Eelco Dolstra b1da85140d Fix sysbuild input type handling 2011-12-05 17:13:20 +01:00
Eelco Dolstra e9c88783ce Pass additional attributes for Git inputs 
The ‘revCount’ attribute is the number of commits in the history
of the revision.  This is useful if you need a monotonically
increasing version number.

The ‘gitTag’ attribute is the output of ‘git describe’, e.g.
‘v1.0.4-14-g2414721’ to indicate that the current revision is 14
commits after the tag ‘v1.0.4’.
 2011-12-05 15:13:31 +01:00
Eelco Dolstra 9032c55aa6 Keep track of the database schema version 
The singleton table SchemaVersion contains the current version
of the Hydra database schema.  This can be used to upgrade the
schema on the fly.

Also reran the DBIx::Class schema loader.
 2011-12-05 14:29:29 +01:00
Eelco Dolstra 3b9a62c964 Pass inputs to release expressions using -I 
This means that you can now write (for instance)

  with <nixpkgs> { ... };

in release.nix, rather than

  { nixpkgs } :

  with nixpkgs { ... };
 2011-12-05 13:08:43 +01:00
Eelco Dolstra 9d4b2253ea Add missing / 2011-12-04 22:53:07 +01:00
Eelco Dolstra 52406e108b Handle missing "build" inputs 2011-12-04 22:36:55 +01:00
Eelco Dolstra 65ea71578f Cleanup 
Don't propagate $type unnecessarily.
 2011-12-04 22:18:23 +01:00
Eelco Dolstra 6c3f878026 Die TABs die 2011-12-04 22:05:43 +01:00
Eelco Dolstra 29846d7f2f Do incremental SVN checkouts 
In hydra-evaluator, reuse an SVN working copy between runs (similar to
what we do with Git and other input types).  This reduces network
traffic in the common case.

Also, don't use nix-prefetch-svn.  It doesn't do anything useful.
 2011-12-04 22:05:43 +01:00
Eelco Dolstra 692a11fd3b Indentation 2011-12-04 22:05:43 +01:00
Ludovic Courtès 6c4e83d454 More command renaming. 2011-12-02 15:58:26 +01:00
Rob Vermaas 41cd5d2838 fix missing import of Nix::Store for isValidPath 2011-12-01 20:46:02 -05:00
Eelco Dolstra be1935f66a Rename hydra_eval_jobs to hydra-eval-jobs 2011-11-30 18:14:48 +01:00
Eelco Dolstra 92536c2e3c Random indentation fix 2011-11-30 18:14:48 +01:00
Eelco Dolstra 82d17a2d0b Rename hydra_*.pl to hydra-* 
The underscores are ugly and the .pl extension is an implementation
detail that shouldn't be visible to the outside.

Also, get rid of the *.in files.  It's not really necessary to
generate them.  And I was always modifying the wrong file.
 2011-11-30 18:14:48 +01:00
Rob Vermaas 787ed37cb3 git branches can share local clones 2011-11-30 11:40:35 -05:00
Eelco Dolstra 2c328e4b20 Forgot to change Nix to Nix::Store in one place 2011-11-30 17:22:31 +01:00
Eelco Dolstra c613b885f2 Use the new Nix Perl bindings 2011-11-30 15:25:28 +01:00
Eelco Dolstra 5333c67bf9 Prevent an unnecessary SQL query for every package in the NixExprs view 2011-11-30 14:27:46 +01:00
Eelco Dolstra c8d572a0dc Remove the "all" channel 
The "all" channel fundamentally doesn't scale, because it needs
to fetch N builds from the database (where N is potentially a very
large number), then check whether they are still valid.  And it's
not very useful anyway.
 2011-11-30 12:07:47 +01:00
Eelco Dolstra d37c4cfe86 Remove unnecessary whitespace in the HTML output 2011-11-29 20:15:42 +01:00
Eelco Dolstra 69e9a91bcb Fix a huge performance regression in the jobstatus page 
The renderBuildStatusIcon function was looking at the
resultInfo and schedulingInfo of each build, causing
a SQL query for every build in the result.
 2011-11-29 19:55:49 +01:00
Eelco Dolstra afef502bb6 Fix indentation 2011-11-29 19:04:40 +01:00
Eelco Dolstra 88b4c4f55d Add another index to speed up the jobstatus page 2011-11-29 18:48:11 +01:00
Eelco Dolstra eca51da45c Print out an Apache-style access log 2011-11-29 18:09:46 +01:00
Eelco Dolstra 8d0ea54fa9 Speed up the /jobset overview pages a lot 2011-11-29 18:09:04 +01:00
Eelco Dolstra 3b550a8086 Minor style changes 2011-11-29 18:08:23 +01:00
Eelco Dolstra 0f639021d0 Add an index to speed up the /project/<name> pages 2011-11-29 17:07:37 +01:00
Eelco Dolstra 3685ffd464 Whitespace 2011-11-29 15:57:47 +01:00
Eelco Dolstra ee67821d1f Disable Catalyst's Debug flag 
It can be re-enabled by setting $HYDRA_DEBUG to 1.
 2011-11-25 16:44:18 +01:00
Eelco Dolstra ec32e833e6 Install the file in the Hydra/Script directory 2011-11-22 17:34:23 +01:00
Eelco Dolstra 43a9fdc891 Remove the gray container around the main page div 2011-11-22 17:30:13 +01:00
Rob Vermaas 0d81e23b46 remove reference to non-existing nix-common.css 2011-11-22 11:10:51 -05:00
Eelco Dolstra 109e5807c2 Use CatalystX::Script::Server::Starman 
This allows us to pass --max_server and similar options
to raise the number of workers from the default (5).
 2011-11-22 16:44:32 +01:00
Rob Vermaas 15cac00906 Merge branch 'master' of github.com:NixOS/hydra 2011-11-19 21:40:00 -05:00
Rob Vermaas 2db4462735 fix annoying behaviour in webkit browsers when middle-clicking (open in new tab) on a build, it used to open the build both in new tab as current tab 2011-11-19 21:38:40 -05:00
Eelco Dolstra 6c4ad48be7 Gratuitous whitespace. 2011-11-18 20:35:27 +01:00
Rob Vermaas 43198e9353 used wrong builds collection for determining if we need to show current build section in job page 2011-11-16 15:21:53 -05:00
Rob Vermaas c8127b887f use newer icons consistently 2011-11-16 13:29:40 -05:00
Rob Vermaas 98fadb53e4 use newer icons consistently 2011-11-16 13:03:48 -05:00
Rob Vermaas f4204df39a always show latest 10 builds on job page 2011-11-16 10:36:18 -05:00
Rob Vermaas 1395717245 recognize tgz in build products contents page 2011-11-16 10:32:32 -05:00
Rob Vermaas d9ba24b216 add link to build on tail/-reload log pages 2011-11-10 12:22:27 -05:00
Rob Vermaas fd9b32bef7 show all related jobset/job links in navbar for the tail/-reload logs 2011-11-10 12:19:05 -05:00
Rob Vermaas 3eaa615281 point log link for running build step to the automatically reloading log page in stead of the 'useless' pretty log page 2011-11-10 11:52:42 -05:00
Rob Vermaas 41c649a2fd remove .hg-archival.txt in nix-prefetch-hg 2011-11-03 19:44:05 -04:00
Rob Vermaas d14f7c7720 fix mercurial diffs, should change to cloned repo first 2011-11-01 21:36:46 -04:00
Rob Vermaas 13df3e4191 fix nrbuilds query 2011-11-01 21:35:02 -04:00
Rob Vermaas ab81ce689d add nix-prefetch- scripts for now, were externals in svn 2011-10-28 22:12:51 -04:00
Rob Vermaas 9ce95d31f1 change content type of logdiff to ->response->content_type('application/json'); 2011-10-12 13:36:20 +00:00
Rob Vermaas a2a187b36a if no previous successful build, do not show changes section 2011-10-12 12:43:31 +00:00
Ludovic Courtès c329bece61 Handle xz-compressed files. 2011-10-10 21:00:54 +00:00
Eelco Dolstra 5f93e6437f * Update to Catalyst 5.9. 2011-10-07 13:45:23 +00:00
Rob Vermaas 6d81bafae4 check for revisions 2011-10-05 19:08:45 +00:00
Rob Vermaas 52c2b31711 no branch info available yet in scmdiff, so disable for now 2011-10-05 18:45:19 +00:00
Rob Vermaas 6ca9501632 make default branch default, remove quotes 2011-10-04 22:29:36 +00:00
Rob Vermaas 9374cfcb34 add support for mercurial tags/revs 2011-10-04 20:03:43 +00:00
Ludovic Courtès 96b4db1e99 Allow Subversion inputs to specify a revision number. 2011-10-04 16:15:22 +00:00
Rob Vermaas 998665aa0f add log diff to compare logs to previous builds 2011-09-26 14:47:55 +00:00
Rob Vermaas ef802279ed use build timestamp as last-modified header on download 2011-09-15 15:22:00 +00:00
Eelco Dolstra 2a18d8aac9 * Fix hydra_eval_jobs. 2011-09-15 08:34:08 +00:00
Eelco Dolstra d8a7ca67f4 * Start of a JSON API to get information about a specific build. 
E.g. http://server/build/1341335/api/get-info returns a JSON 
  record containing information about the build.
 2011-09-15 08:27:17 +00:00
Eelco Dolstra d72538e641 * Remove empty file. 2011-09-11 20:17:59 +00:00
Rob Vermaas f34ec401ba fix for buildsteps starting at 0, probably something changed in catalyst 2011-09-08 22:48:38 +00:00
Rob Vermaas 203716597d set threshold to show status on jobset page to a more usable size 2011-08-29 17:56:38 +00:00
Rob Vermaas 5d3f5f9fca hydra: when no external url is given for diff's of git/hg input, revert to a diff done on a local available clone 2011-08-25 14:50:31 +00:00
Eelco Dolstra c51af99ecf * Catalyst now escapes slashes to %2f, which broke defaultUriForProduct. 2011-08-19 16:23:01 +00:00
Eelco Dolstra 9f78a942bc 2011-08-19 16:09:14 +00:00
Eelco Dolstra d49ccadfea * Build fix. 2011-08-19 15:43:23 +00:00
Eelco Dolstra 5e359374a1 * Fix indentation. 2011-08-19 15:13:34 +00:00
Eelco Dolstra 3d6a8fa632 * Slight cleanup. 2011-08-16 14:51:19 +00:00
Rob Vermaas b75ddbc11c allow _'s in first character of filenames 2011-07-25 23:19:24 +00:00
Ludovic Courtès 337ba26aa6 Add links missing from the admin page. 2011-07-25 09:38:41 +00:00
Rob Vermaas 8101874b83 grr, typo 2011-07-18 15:03:15 +00:00
Rob Vermaas 360b4a74bf remove buildonlylatest propery, which was committed by accident 2011-07-17 12:02:20 +00:00
Ludovic Courtès dcbdd0ad26 hydra_build.pl: Honor `$build->timeout'. 2011-07-04 13:55:02 +00:00
Rob Vermaas 07f1d85518 remove unintentional committed line 2011-06-10 12:56:49 +00:00
Rob Vermaas 3824aeaab8 change regexp for ascii escapes in log 2011-06-10 10:53:59 +00:00
Rob Vermaas 5d166150f9 using backquote as argument resulted in only first line as first argument to removeAsciiEscapes 2011-06-10 10:48:51 +00:00
Rob Vermaas 48b435c2de remove ascii escapes from log in tail page and emails 2011-06-10 09:53:15 +00:00
Rob Vermaas 259d163ec9 clean up build page for build that failed with result 2011-06-10 07:36:30 +00:00
Ludovic Courtès b4dd17c8c8 Tweak CSS for img/logo. 
This makes it easy to choose a larger logo, while still occupying as
much vertical space as the original logo.
 2011-06-09 14:49:48 +00:00
Rob Vermaas c4130a8161 Hydra/64: Leaving number of builds empty gives DBIx error 2011-05-16 08:03:06 +00:00
Eelco Dolstra e080af3daa * Revert r27114, it seems to break stuff. 2011-05-09 08:57:19 +00:00
Eelco Dolstra 334c71cd07 * Use default values of job function arguments if they're not 
specified in Hydra.
 2011-05-03 09:21:27 +00:00
Eelco Dolstra adbba3bbcb * me.id -> build.id. 2011-04-20 08:57:55 +00:00
Rob Vermaas 2f215aac4b add tracker html code via HYDRA_TRACKER 2011-04-19 12:00:54 +00:00
Rob Vermaas a8b3ebd514 make logo configurable via HYDRA_LOGO env var 2011-04-18 08:21:27 +00:00
Rob Vermaas d4c96a47c9 qualify order by column 2011-04-18 08:10:10 +00:00
Rob Vermaas fca3019c7b qualify ordery_by clauses when necessary, remove unnecessary order_by's, reported by Ludo, resulted in errors in sqlite 2011-04-01 07:40:06 +00:00
Rob Vermaas f3f2c5683f Hydra/62 fix, broken nar download 2011-03-23 13:03:40 +00:00
Ludovic Courtès 7b1d87fde9 fetchInputGit: Populate TopGit branches when on a topic branch. 2011-03-22 22:21:19 +00:00
Rob Vermaas 1df83d52f8 fix bug reported by ludo (log in, non-admin) on project overview page 2011-03-22 08:51:15 +00:00
Rob Vermaas d719b0480f hydra: run nix-prefetch-git on local clone of git repo 2011-03-16 16:03:01 +00:00
Rob Vermaas 8f35d03bca use NIX_*_DIR env vars when defined 2011-03-16 13:08:12 +00:00
Rob Vermaas 413d49342d fixed bug when updating user when no roles were selected, changed change-password link 2011-03-15 14:57:05 +00:00
Rob Vermaas edbfc781d0 qualify id column 2011-03-15 13:54:43 +00:00
Rob Vermaas 2849279f80 do not send email for builds with status 'aborted' 2011-03-15 12:12:56 +00:00
Rob Vermaas 03d12ca68e Hydra/57: Unknown failure -> Failed 2011-03-15 11:51:00 +00:00
Rob Vermaas 53c48e4d7d qualify id column in queries 2011-03-14 14:05:32 +00:00
Rob Vermaas 436373c165 added store path size graph 2011-03-11 08:50:42 +00:00
Ludovic Courtès f57f91f9ff Change fetchInputGit' to do git clone -b BRANCH' instead of `git clone'. 
Suggested by Vladimír Čunát <vcunat@gmail.com>.
 2011-03-10 15:06:54 +00:00
Rob Vermaas 92d547f4e5 do not check username, as you are not allowed to change it 2011-03-08 07:23:28 +00:00
Rob Vermaas c9929761bd hydra: cannot change username, also update fullname 2011-03-07 16:46:46 +00:00
Rob Vermaas 2bbbfd7a61 remove fk constraint jobsets.nixExprInput -> JobsetInputs.name, would only work on postgresql in postpone-fk-checking mode (at end of transaction in stead of through the transaction) which is not the default 2011-03-07 16:07:16 +00:00
Rob Vermaas 107a879adc if no emailoverride specified, use empty string 2011-03-07 16:02:23 +00:00
Rob Vermaas 5c187a0640 if no emailoverride specified, use empty string 2011-03-07 15:55:37 +00:00
Rob Vermaas 560f41a7e5 if HYDRA_CONFIG is not set, take default 2011-03-07 15:49:48 +00:00
Rob Vermaas 586f131ebd use HYDRA_DATA/hydra.conf as default location for hydra.conf 2011-03-07 15:06:32 +00:00
Rob Vermaas d4e9f5383b Hydra/56: handle failed builds with result only at build level, not buildsteps 2011-03-07 13:55:43 +00:00
Rob Vermaas f95ded45a5 revert commit 26130, use hardcoded default value for HYDRA_DATA (/var/lib/hydra) 2011-03-07 12:30:27 +00:00
Rob Vermaas 0439e7efb7 do not install .in file 2011-03-07 08:04:10 +00:00
Ludovic Courtès bf927b3180 Set a default value for `getHydraPath'. 2011-03-02 15:17:03 +00:00
Rob Vermaas b59ca01e40 pass svn/bzr revisions as integers 2011-02-10 10:35:26 +00:00
Rob Vermaas c2a7de45a5 properly pass bazaar inputs 2011-02-09 12:54:29 +00:00
Rob Vermaas 98824f2aaf hydra: nix-prefetch-bzr missing in tarball 2011-02-09 12:44:46 +00:00
Eelco Dolstra 2931689dcf * Ran update-dbix. 2011-02-09 10:19:36 +00:00
Rob Vermaas e7c203a3b4 typo 2011-02-08 13:37:50 +00:00
Rob Vermaas c645fa55ff initial bzr support 2011-02-08 13:11:08 +00:00
Rob Vermaas e17fec61b2 added help, removed nix closure from 'failed build with output' productlist 2011-02-02 12:16:57 +00:00
Rob Vermaas a561c09f2c forgot to commit a file 2011-02-02 09:07:17 +00:00
Rob Vermaas 559ab9c97a automatic reload of tail log when build is running 2011-02-02 09:00:52 +00:00
Rob Vermaas 9aeaef80fc hydra: current build -> this build 2011-01-17 10:06:11 +00:00
Rob Vermaas 6e213fab97 hydra: add nix-prefetch-* to tarball 2011-01-14 14:11:08 +00:00
Rob Vermaas 2892c92569 move nix-prefetch-* into tarball 2011-01-14 14:04:46 +00:00
Ludovic Courtès 18e23e90e2 hydra: Clarify the dependency on BDW-GC. 2011-01-14 12:53:54 +00:00
Rob Vermaas ceffcae5e0 ignore this 2011-01-14 08:27:05 +00:00
Rob Vermaas ff506fb3fd hydra: description at changes section 2011-01-14 08:22:32 +00:00
Rob Vermaas 5015c1bcd4 hydra: description at changes section 2011-01-14 08:19:37 +00:00
Rob Vermaas 88dc824479 make changes section more compact/readable 2011-01-14 08:11:21 +00:00
Rob Vermaas 7e69512701 hydra: refine error marking 2011-01-12 08:48:25 +00:00
Rob Vermaas c4d0ee44f0 hydra: change icon for positive failures 2011-01-04 16:08:19 +00:00
Rob Vermaas 80e9bb01f0 hydra: missing plain.tt 2011-01-04 12:55:59 +00:00
Rob Vermaas f35c9a4574 hydra: put contents links into html, add links to files from contents page 2011-01-04 12:50:59 +00:00
Rob Vermaas 039d1bed21 hydra: add support for succeedOnFailure feature in stdenv 2010-12-22 14:55:05 +00:00
Rob Vermaas 496c22dc0f hydra: remove old clear eval cache option 2010-12-20 09:53:01 +00:00
Rob Vermaas 11ad35443b hydra: make size columns bigger (integer -> bigint) 2010-12-08 10:14:57 +00:00
Rob Vermaas 6bb569387a hydra: remove debug statement 2010-12-07 13:26:31 +00:00
Rob Vermaas 4fdd011b2d hydra: fixed missing argument to restartbuild function 2010-12-07 13:25:29 +00:00
Rob Vermaas cd7742f610 hydra: added some user admin 2010-12-03 09:40:25 +00:00
Rob Vermaas 6d6f43fa0c hydra: factored out build restart and 2010-12-03 08:40:34 +00:00
Rob Vermaas 02856780cf hydra: show machine in buildsteps 2010-11-29 09:10:10 +00:00
Eelco Dolstra a75a12e819 * Include the system type in manifests as a hint for the 
binary patch generator.
 2010-11-26 14:34:58 +00:00
Rob Vermaas 0cee7c0909 hydra: handle case where build to keep set to 0 2010-11-23 09:05:09 +00:00
Eelco Dolstra 91fe70522f * Doh. Remove debug statement. 2010-11-22 12:20:04 +00:00
Eelco Dolstra b4ce69ed8d * Index the Builds table on drvPath to prevent a full table scan 
in findBuildDependencyInQueue in hydra_queue_runner.
 2010-11-19 15:49:55 +00:00
Eelco Dolstra a93e272364 * Add a NarSize field to Hydra manifests. This allows nix-env 
to predict how much disk space a package will require.
* Compute the output / closure size using the info stored in the
  Nix database (rather than doing a slow "du").
 2010-11-19 15:44:20 +00:00
Eelco Dolstra 738712fca0 * Merge the GC branch. 2010-11-19 11:01:31 +00:00
Rob Vermaas dcdbb1d814 hydra: store logfile/output path/closure size 2010-11-11 11:03:50 +00:00
Rob Vermaas c9ca8d6c63 add link to status page on queue page 2010-11-09 13:25:04 +00:00
Rob Vermaas 0f1c580731 hydra: order machines by enabled/disabled, add green and red color for enabled/disabled 2010-10-21 14:37:03 +00:00