snaakey/hydra
0
0
Fork 0
forked from lix-project/hydra
Code Pull requests Activity
1174 commits 1 branch 0 tags 20 MiB
Commit graph

435 commits

Author SHA1 Message Date
Eelco Dolstra 2d1cf73974 Speed up channel processing 
In particular the /pkg action is now O(lg n) instead of O(n) in the
number of packages in the channel, and listing the channel contents
no longer requires calling isValidPath() on all packages.

Derivations (and thus build time dependencies) are no longer included
in the channel, because they're not GC roots.  Thus they could
disappear unexpectedly.
 2012-03-12 20:47:30 +01:00
Eelco Dolstra 9f10c0f9c0 Fix a reference to BuildResultInfo 2012-03-12 20:47:30 +01:00
Eelco Dolstra 07b3dffd20 Reduce I/O in build listings by only fetching required columns 
Columns such as "longDescription" can be large, so fetching them
when they're not needed is wasteful.
 2012-03-12 20:47:30 +01:00
Eelco Dolstra 47f877c5bb Evaluator cleanups 
* Don't use isCurrent anymore; instead look up builds in the previous
  jobset evaluation.  (The isCurrent field is still maintained because
  it's still used in some other places.)

* To determine whether to perform an evaluation, compare the hash of
  the current inputs with the inputs of the previous jobset
  evaluation, rather than checking if there was ever an evaluation
  with those inputs.  This way, if the inputs of an evaluation change
  back to a previous state, we get a new jobset evaluation in the
  database (and thus the latest jobset evaluation correctly represents
  the latest state of the jobset).

* Improve performance by removing some unnecessary operations and
  adding an index.
 2012-03-12 20:47:29 +01:00
Eelco Dolstra 8d65ab6158 Don't use a prepared statement for the active build steps query 
Prepared statements are sometimes much slower than unprepared
statements, because the planner doesn't have access to the query
parameters.  This is the case for the active build steps query (in
/status), where a prepared statement is three orders of magnitude
slower.  So disable the use of prepared statements in this case.

(Since the query parameters are constant here, it would be nicer if we
could tell DBIx::Class to prepare a statement with those parameters
fixed.  But I don't know an easy way to do so.)
 2012-03-12 20:47:29 +01:00
Eelco Dolstra 68a867da67 Merge the BuildResultInfo table into the Builds table 2012-03-12 20:47:29 +01:00
Eelco Dolstra 25334715f8 Merge the BuildSchedulingInfo table into the Builds table 
This simplifies the code and improves performance since it reduces
the number of joins.
 2012-03-12 20:47:29 +01:00
Eelco Dolstra 541238030d Provide a command ‘hydra-init’ to initialise/upgrade the database 
For schema upgrades, hydra-init executes the files
src/sql/upgrade-<N>.sql, each of which upgrades the schema from
version N-1 to N.  The upgrades are wrapped in a transaction.
 2012-02-28 20:16:16 +01:00
Eelco Dolstra b13664308c Whitespace 2012-02-28 15:27:44 +01:00
Eelco Dolstra 3d824b3a3a Indentation 2012-02-21 17:59:36 +01:00
Eelco Dolstra 578530dac2 Don't show duplicate builds in "Related builds" 
Since a build may be a member of multiple jobset evaluations, we need
to do a "select distinct" here.  But maybe we should only show builds
from a single evaluation (e.g. the most recent), since showing builds
from several may be confusing.
 2012-02-21 17:16:32 +01:00
Rob Vermaas 2cf9b148eb 50 for related builds might be a kind of low, will start with 100 2012-02-16 17:00:59 +01:00
Rob Vermaas 924d94cafd * minor performance tweaks 
* added related builds tab which show related builds (builds that are in same evaluations), shows max 50
 2012-02-16 16:56:01 +01:00
Rob Vermaas 355135b6e8 add links in admin page to clear queue (all non-running builds, and all non-running old (non-current) builds) 2012-02-16 15:31:12 +01:00
Rob Vermaas 8d78a688c7 is not in hydra-server's scope 2012-01-28 21:03:44 +01:00
Rob Vermaas 1c0cd99c3e remove old line 2012-01-28 20:55:33 +01:00
Rob Vermaas fded9e62af logdiff now support bzipped logs for logdiff 2012-01-28 20:47:01 +01:00
Rob Vermaas ae491f7223 use base_uri config in password reset email 2012-01-28 07:18:10 +01:00
Rob Vermaas 01f07be8f2 fix; resetting password used to send email to logged in user (typically admin), now to the user whose password is resetted 2012-01-28 07:14:40 +01:00
Rob Vermaas 9b4f09ba83 allow dots (.) in job names when used as build input 2012-01-06 23:04:48 +01:00
Eelco Dolstra 9fe8c43437 Pass the abbreviated Git revision 
The abbreviated Git revision hash (e.g. "267480b") is typically
contained in ‘gitTag’ as well, but the latter can contain other
elements as well, e.g., the delta to the closest tag.  That may
be undesirable in version strings, so this is an alternative.
 2011-12-06 23:04:15 +01:00
Eelco Dolstra b1da85140d Fix sysbuild input type handling 2011-12-05 17:13:20 +01:00
Eelco Dolstra e9c88783ce Pass additional attributes for Git inputs 
The ‘revCount’ attribute is the number of commits in the history
of the revision.  This is useful if you need a monotonically
increasing version number.

The ‘gitTag’ attribute is the output of ‘git describe’, e.g.
‘v1.0.4-14-g2414721’ to indicate that the current revision is 14
commits after the tag ‘v1.0.4’.
 2011-12-05 15:13:31 +01:00
Eelco Dolstra 9032c55aa6 Keep track of the database schema version 
The singleton table SchemaVersion contains the current version
of the Hydra database schema.  This can be used to upgrade the
schema on the fly.

Also reran the DBIx::Class schema loader.
 2011-12-05 14:29:29 +01:00
Eelco Dolstra 3b9a62c964 Pass inputs to release expressions using -I 
This means that you can now write (for instance)

  with <nixpkgs> { ... };

in release.nix, rather than

  { nixpkgs } :

  with nixpkgs { ... };
 2011-12-05 13:08:43 +01:00
Eelco Dolstra 9d4b2253ea Add missing / 2011-12-04 22:53:07 +01:00
Eelco Dolstra 52406e108b Handle missing "build" inputs 2011-12-04 22:36:55 +01:00
Eelco Dolstra 65ea71578f Cleanup 
Don't propagate $type unnecessarily.
 2011-12-04 22:18:23 +01:00
Eelco Dolstra 6c3f878026 Die TABs die 2011-12-04 22:05:43 +01:00
Eelco Dolstra 29846d7f2f Do incremental SVN checkouts 
In hydra-evaluator, reuse an SVN working copy between runs (similar to
what we do with Git and other input types).  This reduces network
traffic in the common case.

Also, don't use nix-prefetch-svn.  It doesn't do anything useful.
 2011-12-04 22:05:43 +01:00
Eelco Dolstra 692a11fd3b Indentation 2011-12-04 22:05:43 +01:00
Ludovic Courtès 6c4e83d454 More command renaming. 2011-12-02 15:58:26 +01:00
Rob Vermaas 41cd5d2838 fix missing import of Nix::Store for isValidPath 2011-12-01 20:46:02 -05:00
Eelco Dolstra be1935f66a Rename hydra_eval_jobs to hydra-eval-jobs 2011-11-30 18:14:48 +01:00
Eelco Dolstra 92536c2e3c Random indentation fix 2011-11-30 18:14:48 +01:00
Rob Vermaas 787ed37cb3 git branches can share local clones 2011-11-30 11:40:35 -05:00
Eelco Dolstra c613b885f2 Use the new Nix Perl bindings 2011-11-30 15:25:28 +01:00
Eelco Dolstra 5333c67bf9 Prevent an unnecessary SQL query for every package in the NixExprs view 2011-11-30 14:27:46 +01:00
Eelco Dolstra c8d572a0dc Remove the "all" channel 
The "all" channel fundamentally doesn't scale, because it needs
to fetch N builds from the database (where N is potentially a very
large number), then check whether they are still valid.  And it's
not very useful anyway.
 2011-11-30 12:07:47 +01:00
Eelco Dolstra 69e9a91bcb Fix a huge performance regression in the jobstatus page 
The renderBuildStatusIcon function was looking at the
resultInfo and schedulingInfo of each build, causing
a SQL query for every build in the result.
 2011-11-29 19:55:49 +01:00
Eelco Dolstra 3b550a8086 Minor style changes 2011-11-29 18:08:23 +01:00
Eelco Dolstra 3685ffd464 Whitespace 2011-11-29 15:57:47 +01:00
Eelco Dolstra 109e5807c2 Use CatalystX::Script::Server::Starman 
This allows us to pass --max_server and similar options
to raise the number of workers from the default (5).
 2011-11-22 16:44:32 +01:00
Rob Vermaas f4204df39a always show latest 10 builds on job page 2011-11-16 10:36:18 -05:00
Rob Vermaas 1395717245 recognize tgz in build products contents page 2011-11-16 10:32:32 -05:00
Rob Vermaas d14f7c7720 fix mercurial diffs, should change to cloned repo first 2011-11-01 21:36:46 -04:00
Rob Vermaas 13df3e4191 fix nrbuilds query 2011-11-01 21:35:02 -04:00
Rob Vermaas 9ce95d31f1 change content type of logdiff to ->response->content_type('application/json'); 2011-10-12 13:36:20 +00:00
Ludovic Courtès c329bece61 Handle xz-compressed files. 2011-10-10 21:00:54 +00:00
Rob Vermaas 6d81bafae4 check for revisions 2011-10-05 19:08:45 +00:00
Rob Vermaas 52c2b31711 no branch info available yet in scmdiff, so disable for now 2011-10-05 18:45:19 +00:00
Rob Vermaas 6ca9501632 make default branch default, remove quotes 2011-10-04 22:29:36 +00:00
Rob Vermaas 9374cfcb34 add support for mercurial tags/revs 2011-10-04 20:03:43 +00:00
Ludovic Courtès 96b4db1e99 Allow Subversion inputs to specify a revision number. 2011-10-04 16:15:22 +00:00
Rob Vermaas 998665aa0f add log diff to compare logs to previous builds 2011-09-26 14:47:55 +00:00
Rob Vermaas ef802279ed use build timestamp as last-modified header on download 2011-09-15 15:22:00 +00:00
Eelco Dolstra d8a7ca67f4 * Start of a JSON API to get information about a specific build. 
E.g. http://server/build/1341335/api/get-info returns a JSON 
  record containing information about the build.
 2011-09-15 08:27:17 +00:00
Rob Vermaas 203716597d set threshold to show status on jobset page to a more usable size 2011-08-29 17:56:38 +00:00
Rob Vermaas 5d3f5f9fca hydra: when no external url is given for diff's of git/hg input, revert to a diff done on a local available clone 2011-08-25 14:50:31 +00:00
Eelco Dolstra c51af99ecf * Catalyst now escapes slashes to %2f, which broke defaultUriForProduct. 2011-08-19 16:23:01 +00:00
Eelco Dolstra 5e359374a1 * Fix indentation. 2011-08-19 15:13:34 +00:00
Eelco Dolstra 3d6a8fa632 * Slight cleanup. 2011-08-16 14:51:19 +00:00
Rob Vermaas b75ddbc11c allow _'s in first character of filenames 2011-07-25 23:19:24 +00:00
Rob Vermaas 8101874b83 grr, typo 2011-07-18 15:03:15 +00:00
Rob Vermaas 360b4a74bf remove buildonlylatest propery, which was committed by accident 2011-07-17 12:02:20 +00:00
Rob Vermaas 07f1d85518 remove unintentional committed line 2011-06-10 12:56:49 +00:00
Rob Vermaas 3824aeaab8 change regexp for ascii escapes in log 2011-06-10 10:53:59 +00:00
Rob Vermaas 5d166150f9 using backquote as argument resulted in only first line as first argument to removeAsciiEscapes 2011-06-10 10:48:51 +00:00
Rob Vermaas 48b435c2de remove ascii escapes from log in tail page and emails 2011-06-10 09:53:15 +00:00
Rob Vermaas 259d163ec9 clean up build page for build that failed with result 2011-06-10 07:36:30 +00:00
Rob Vermaas c4130a8161 Hydra/64: Leaving number of builds empty gives DBIx error 2011-05-16 08:03:06 +00:00
Eelco Dolstra adbba3bbcb * me.id -> build.id. 2011-04-20 08:57:55 +00:00
Rob Vermaas 2f215aac4b add tracker html code via HYDRA_TRACKER 2011-04-19 12:00:54 +00:00
Rob Vermaas a8b3ebd514 make logo configurable via HYDRA_LOGO env var 2011-04-18 08:21:27 +00:00
Rob Vermaas d4c96a47c9 qualify order by column 2011-04-18 08:10:10 +00:00
Rob Vermaas fca3019c7b qualify ordery_by clauses when necessary, remove unnecessary order_by's, reported by Ludo, resulted in errors in sqlite 2011-04-01 07:40:06 +00:00
Rob Vermaas f3f2c5683f Hydra/62 fix, broken nar download 2011-03-23 13:03:40 +00:00
Ludovic Courtès 7b1d87fde9 fetchInputGit: Populate TopGit branches when on a topic branch. 2011-03-22 22:21:19 +00:00
Rob Vermaas 1df83d52f8 fix bug reported by ludo (log in, non-admin) on project overview page 2011-03-22 08:51:15 +00:00
Rob Vermaas d719b0480f hydra: run nix-prefetch-git on local clone of git repo 2011-03-16 16:03:01 +00:00
Rob Vermaas 8f35d03bca use NIX_*_DIR env vars when defined 2011-03-16 13:08:12 +00:00
Rob Vermaas 413d49342d fixed bug when updating user when no roles were selected, changed change-password link 2011-03-15 14:57:05 +00:00
Rob Vermaas edbfc781d0 qualify id column 2011-03-15 13:54:43 +00:00
Rob Vermaas 53c48e4d7d qualify id column in queries 2011-03-14 14:05:32 +00:00
Ludovic Courtès f57f91f9ff Change fetchInputGit' to do git clone -b BRANCH' instead of `git clone'. 
Suggested by Vladimír Čunát <vcunat@gmail.com>.
 2011-03-10 15:06:54 +00:00
Rob Vermaas 92d547f4e5 do not check username, as you are not allowed to change it 2011-03-08 07:23:28 +00:00
Rob Vermaas c9929761bd hydra: cannot change username, also update fullname 2011-03-07 16:46:46 +00:00
Rob Vermaas 107a879adc if no emailoverride specified, use empty string 2011-03-07 16:02:23 +00:00
Rob Vermaas 5c187a0640 if no emailoverride specified, use empty string 2011-03-07 15:55:37 +00:00
Rob Vermaas 586f131ebd use HYDRA_DATA/hydra.conf as default location for hydra.conf 2011-03-07 15:06:32 +00:00
Rob Vermaas f95ded45a5 revert commit 26130, use hardcoded default value for HYDRA_DATA (/var/lib/hydra) 2011-03-07 12:30:27 +00:00
Ludovic Courtès bf927b3180 Set a default value for `getHydraPath'. 2011-03-02 15:17:03 +00:00
Rob Vermaas b59ca01e40 pass svn/bzr revisions as integers 2011-02-10 10:35:26 +00:00
Rob Vermaas c2a7de45a5 properly pass bazaar inputs 2011-02-09 12:54:29 +00:00
Eelco Dolstra 2931689dcf * Ran update-dbix. 2011-02-09 10:19:36 +00:00
Rob Vermaas e7c203a3b4 typo 2011-02-08 13:37:50 +00:00
Rob Vermaas c645fa55ff initial bzr support 2011-02-08 13:11:08 +00:00
Rob Vermaas 559ab9c97a automatic reload of tail log when build is running 2011-02-02 09:00:52 +00:00
Rob Vermaas f35c9a4574 hydra: put contents links into html, add links to files from contents page 2011-01-04 12:50:59 +00:00
Rob Vermaas 496c22dc0f hydra: remove old clear eval cache option 2010-12-20 09:53:01 +00:00
Rob Vermaas 6bb569387a hydra: remove debug statement 2010-12-07 13:26:31 +00:00
Rob Vermaas 4fdd011b2d hydra: fixed missing argument to restartbuild function 2010-12-07 13:25:29 +00:00
Rob Vermaas cd7742f610 hydra: added some user admin 2010-12-03 09:40:25 +00:00
Rob Vermaas 6d6f43fa0c hydra: factored out build restart and 2010-12-03 08:40:34 +00:00
Eelco Dolstra a75a12e819 * Include the system type in manifests as a hint for the 
binary patch generator.
 2010-11-26 14:34:58 +00:00
Eelco Dolstra a93e272364 * Add a NarSize field to Hydra manifests. This allows nix-env 
to predict how much disk space a package will require.
* Compute the output / closure size using the info stored in the
  Nix database (rather than doing a slow "du").
 2010-11-19 15:44:20 +00:00
Rob Vermaas dcdbb1d814 hydra: store logfile/output path/closure size 2010-11-11 11:03:50 +00:00
Rob Vermaas 0f1c580731 hydra: order machines by enabled/disabled, add green and red color for enabled/disabled 2010-10-21 14:37:03 +00:00
Rob Vermaas 431656546e hydra: disallow robots.txt for root and project all builds page 2010-10-19 08:34:13 +00:00
Rob Vermaas 42750364de hydra: show all running builds on job page, in stead of only showing the 'current' builds that are running 2010-10-18 12:06:59 +00:00
Rob Vermaas df921a4fd0 hydra: add comment that file is generated by hydra to /etc/nix.machines 2010-10-18 11:05:36 +00:00
Rob Vermaas a391c78aaf hydra: buildmachine disabled by default, add some default constraints 2010-10-13 12:51:29 +00:00
Rob Vermaas ef1efa80e4 hydra: missing property 2010-10-13 12:35:34 +00:00
Rob Vermaas 398993f688 hydra: add some admin for adding/enabling/etc build machines 2010-10-13 12:32:57 +00:00
Rob Vermaas 0491ea3455 hydra: catalyst plain view doesn;t work with data => 0, so put some whitespace around it 2010-10-08 10:45:40 +00:00
Rob Vermaas 673ef76020 fix redirect after login 2010-09-29 11:01:48 +00:00
Rob Vermaas 822ca9c29e hydra: in jobstatus pages, allow maintainer query argument to show only the packages you are maintaining 2010-09-08 06:37:19 +00:00
Rob Vermaas 526ffb7eef hydra: missing argument 2010-09-07 15:12:41 +00:00
Rob Vermaas 05ebf34441 hydra: do not use refs/heads while checking for latest commit in branch 2010-09-07 14:56:53 +00:00
Rob Vermaas d3f0c3f67d hydra: export addBuildProducts 2010-09-07 12:31:11 +00:00
Rob Vermaas 38d50806b9 hydra: if evaluator sees cached build, also add the buildproducts 2010-09-07 11:29:52 +00:00
Rob Vermaas 6d74064999 add /status to robots.txt 2010-09-03 09:17:54 +00:00
Rob Vermaas 4c63ccb6bc hydra: moves jobsetOverview sub 2010-09-03 09:17:52 +00:00
Rob Vermaas ce34776992 added some json responses 2010-09-02 12:21:56 +00:00
Rob Vermaas 79168ae722 hydra: missing uses 2010-09-02 09:00:06 +00:00
Rob Vermaas c5ceece093 hydra: moved getbuildlog 2010-09-02 08:56:29 +00:00
Rob Vermaas d7487ab0d9 hydra: at evaluation, check if path is already built, and mark as built in stead of adding to the queue. 2010-09-01 10:50:57 +00:00
Eelco Dolstra fbeb5abc69 2010-08-31 15:37:50 +00:00
Eelco Dolstra 0caf68c638 * Disambiguate. 2010-08-31 15:30:20 +00:00
Eelco Dolstra f6715fa0ef * Added a status page that shows all the currently executing build steps. 
* Store the system type in the BuildSteps table.
* Don't query the queue size when serving static pages.  This prevents
  two unnecessary database queries per request.
 2010-08-31 15:27:46 +00:00
Eelco Dolstra 2a69745a88 * Store the name of the machine that performed a build step in the 
BuildSteps table.
 2010-08-31 14:08:59 +00:00
Rob Vermaas c174998a0c hydra: when restarting a build, remove possible failed dependencies from the failed-cache. 2010-08-11 06:43:59 +00:00
Rob Vermaas f7ce960b96 hydra: make sure viewing logs works when logs are bz2'd 2010-08-10 13:11:41 +00:00
Rob Vermaas 9d9bf8b264 hydra: make nr of build to keep configurable per jobset 2010-08-10 06:48:45 +00:00
Rob Vermaas 23886606d0 Hydra/40: Last 5 builds on job page incorrect 2010-08-02 12:32:15 +00:00
Rob Vermaas dda064bc8c hydra: nar was replaced, so also move in robots.txt 2010-07-30 10:17:47 +00:00
Rob Vermaas 0802559b03 hydra: do not perform git clone every time. in stead work on local clone and pull 2010-07-28 12:48:29 +00:00
Rob Vermaas 812689e523 use local clone to limit bandwidth usage 2010-07-27 18:06:47 +00:00
Rob Vermaas 6d028a8d43 missing file 2010-07-27 16:24:21 +00:00
Rob Vermaas 005e088637 missing case for hg input type 2010-07-27 16:17:06 +00:00
Rob Vermaas 125cf1dc5b missing case for hg input type 2010-07-27 16:09:35 +00:00
Rob Vermaas 06e0e80591 bug fix 2010-07-27 11:21:21 +00:00
Rob Vermaas d6131fba50 initial support for mercurial 2010-07-27 11:14:24 +00:00
Rob Vermaas caa0f1a255 Hydra: In case of failure, also show changes between last succesful build and the first failed build after this 2010-07-14 07:31:14 +00:00
Rob Vermaas 33ef55815e Hydra/33: 'Sign in' should return to the referring page 2010-07-06 07:27:55 +00:00
Eelco Dolstra e35f0005e6 * Move NARs from the NixChannel controller to the Root controller and 
drop the "/nix/store".  I.e. instead of

    http://hydra.nixos.org/jobset/nixpkgs/trunk/channel/latest/nar/nix/store/99djmii6l4yw9gn07zxqabh8z58pp84c-libmspack-0.0.20040308alpha.drv 

  we now have

    http://hydra.nixos.org/nar/99djmii6l4yw9gn07zxqabh8z58pp84c-libmspack-0.0.20040308alpha.drv 

  The main reason is that this is much faster, since we don't need to
  get all the channel data (which isn't used anyway for NAR
  generation).  This speeds up downloading a NAR from the Nixpkgs
  channel by > 2 seconds.

* Drop "Hydra::View::" from view names to get rid of an ugly warning.
 2010-06-22 12:00:19 +00:00
Eelco Dolstra dfd8b4406d 2010-06-21 23:48:25 +00:00
Rob Vermaas 5f7ebaad37 added hide feature for project/jobset 2010-06-04 14:43:28 +00:00
Rob Vermaas b25761d7b2 hydra: added missing fields to query 2010-06-03 09:17:24 +00:00
Eelco Dolstra e2e4b49b98 * Pass `-j 1' to hydra_eval_jobs to ensure that it can make progress 
even when all the remote build slots are in use.  The evaluator can
  cause builds if Nix expressions import derivations (e.g. in
  pkgs/build-support/vm to compute the RPM/Deb closures).  If there
  are no free build slots, the evaluator can hang for a long time.
 2010-06-01 11:17:02 +00:00