Commit graph

1140 commits

Author SHA1 Message Date
Rob Vermaas 1904f82f06 add openssl to deps 2012-04-01 18:15:00 +02:00
Eelco Dolstra d8f8143cc2 No wonder our disk was filling up 2012-03-26 17:13:50 +02:00
Rob Vermaas 4d4e1986e8 remove delete jobset button until we actually have implemented the action properly 2012-03-26 14:39:50 +02:00
Eelco Dolstra e1768cae86 Don't barf if the SQLite DB is missing
This prevented hydra-init from starting.
2012-03-19 03:57:11 +00:00
Ludovic Courtès 3846c2407d Fix URL in the "password changed" email. 2012-03-13 18:30:57 +01:00
Ludovic Courtès d0f18a1d6f Add dependency on Switch. 2012-03-13 18:12:37 +01:00
Ludovic Courtès 2b68e41ee4 Actually add SQL files to the distribution. 2012-03-13 17:49:18 +01:00
Ludovic Courtès ace94192ac Update the list of SQL files to be installed.
This fixes `hydra-init', which would otherwise not find `upgrade-*.sql',
thus skipping the update altogether.
2012-03-13 17:45:14 +01:00
Eelco Dolstra dae5032c1c I'm stupid 2012-03-13 13:33:19 +01:00
Eelco Dolstra c844b9f09a Remove the global Job status and Errors links 2012-03-13 13:31:36 +01:00
Eelco Dolstra 161d7418f8 Doh 2012-03-13 13:30:41 +01:00
Eelco Dolstra fa62c8b7f2 Fix tests 2012-03-13 13:18:40 +01:00
Eelco Dolstra 1f268d2d43 Update isCurrent properly 2012-03-13 13:09:10 +01:00
Eelco Dolstra 179b012a8e Open the DB using Hydra::Model::DB->new
This gets rid of the openHydraDB function and ensures that we
open the database in a consistent way.

Also drop the PostgreSQL sequence hacks.  They don't seem to be
necessary anymore.
2012-03-13 12:10:19 +01:00
Eelco Dolstra 914c5e248a Use <...> syntax 2012-03-13 11:03:32 +01:00
Eelco Dolstra 93efae0c23 Fix "make check" 2012-03-12 21:13:50 +01:00
Eelco Dolstra 13f4636436 hydra-evaluator: handle the case where there is no previous jobset eval 2012-03-12 21:13:28 +01:00
Eelco Dolstra 3f1bbf7ef4 Better fix 2012-03-12 21:06:42 +01:00
Eelco Dolstra 259afadbab Set $PERL5LIB properly in hydra-init in "make check" 2012-03-12 20:56:41 +01:00
Eelco Dolstra a4cda5e337 Don't use the Switch module
It was removed in Perl 5.14.
2012-03-12 20:47:30 +01:00
Eelco Dolstra 87e4d43848 Fix the jobset unchanged check
When checking whether the jobset is unchanged, we need to compare with
the previous JobsetEval regardless of whether it had new builds.
Otherwise we'll keep adding new JobsetEval rows.
2012-03-12 20:47:30 +01:00
Eelco Dolstra 27619f18e2 Re-enable the "Related Builds" tab
However, there is a "limit 1" on the JobsetEval search to prevent a
sequential scan of the Builds table.
2012-03-12 20:47:30 +01:00
Eelco Dolstra abe71a767b Disable prepared statements completely
Because of the way DBIx::Class does prepared statements, even
innocuous queries such

  $c->model('DB::Builds)->search({finished => 0})

can be extremely slow.  This is because DBIx::Class prepares a
PostgreSQL statement

  select ... from Builds where finished = ?

and since Builds is very large and there is a large fraction of rows
with "finished = 1", the PostgreSQL query planner decides to implement
this query with a sequential scan of the Builds table (despite the
existence of an index on "finished"), which is extremely slow.  It
would be nice if we could tell DBIx::Class that constants should be
part of the prepared statement, i.e.

  select ... from Builds where finished = 0

but AFAIK we can't.
2012-03-12 20:47:30 +01:00
Eelco Dolstra 2d1cf73974 Speed up channel processing
In particular the /pkg action is now O(lg n) instead of O(n) in the
number of packages in the channel, and listing the channel contents
no longer requires calling isValidPath() on all packages.

Derivations (and thus build time dependencies) are no longer included
in the channel, because they're not GC roots.  Thus they could
disappear unexpectedly.
2012-03-12 20:47:30 +01:00
Eelco Dolstra 9f10c0f9c0 Fix a reference to BuildResultInfo 2012-03-12 20:47:30 +01:00
Eelco Dolstra 07b3dffd20 Reduce I/O in build listings by only fetching required columns
Columns such as "longDescription" can be large, so fetching them
when they're not needed is wasteful.
2012-03-12 20:47:30 +01:00
Eelco Dolstra 6526d4a65f hydra-build: only send email if the status differs from the previous build
This isn't perfect because it doesn't handle the case where a
previous build hasn't finished yet.  But at least it won't send mail
for old builds that fail while a newer build has already succeeded.
2012-03-12 20:47:29 +01:00
Eelco Dolstra 47f877c5bb Evaluator cleanups
* Don't use isCurrent anymore; instead look up builds in the previous
  jobset evaluation.  (The isCurrent field is still maintained because
  it's still used in some other places.)

* To determine whether to perform an evaluation, compare the hash of
  the current inputs with the inputs of the previous jobset
  evaluation, rather than checking if there was ever an evaluation
  with those inputs.  This way, if the inputs of an evaluation change
  back to a previous state, we get a new jobset evaluation in the
  database (and thus the latest jobset evaluation correctly represents
  the latest state of the jobset).

* Improve performance by removing some unnecessary operations and
  adding an index.
2012-03-12 20:47:29 +01:00
Eelco Dolstra bc82a82593 Fix a race condition in hydra-update-gc-roots
Since it read the actual roots after determining the set of desired
roots, there was a possibility that it would delete roots added by
hydra-evaluator or hydra-build while hydra-update-gc-roots was
running.  This could cause a derivation to be garbage-collected before
the build was performed, for instance.  Now the actual roots are read
first, so any root added after that time is not deleted.
2012-03-12 20:47:29 +01:00
Eelco Dolstra 29d5a02b94 Speed up hydra-update-gc-roots
The hydra-update-gc-roots script is taking around 95 minutes on our
Hydra instance (though a lot of that is I/O wait).  This patch
significantly reduces the number of database queries.  In particular,
the N most recent successful builds for each job in a jobset are now
determined in a single query.  Also, it removes the calls to
readlink().
2012-03-12 20:47:29 +01:00
Eelco Dolstra 5be004c999 Drop the merged tables after conversion 2012-03-12 20:47:29 +01:00
Eelco Dolstra 8d65ab6158 Don't use a prepared statement for the active build steps query
Prepared statements are sometimes much slower than unprepared
statements, because the planner doesn't have access to the query
parameters.  This is the case for the active build steps query (in
/status), where a prepared statement is three orders of magnitude
slower.  So disable the use of prepared statements in this case.

(Since the query parameters are constant here, it would be nicer if we
could tell DBIx::Class to prepare a statement with those parameters
fixed.  But I don't know an easy way to do so.)
2012-03-12 20:47:29 +01:00
Eelco Dolstra 68a867da67 Merge the BuildResultInfo table into the Builds table 2012-03-12 20:47:29 +01:00
Eelco Dolstra 25334715f8 Merge the BuildSchedulingInfo table into the Builds table
This simplifies the code and improves performance since it reduces
the number of joins.
2012-03-12 20:47:29 +01:00
Eelco Dolstra 19fe4b9b4a Add hydra-init to the tarball 2012-03-05 15:35:13 +01:00
Eelco Dolstra 541238030d Provide a command ‘hydra-init’ to initialise/upgrade the database
For schema upgrades, hydra-init executes the files
src/sql/upgrade-<N>.sql, each of which upgrades the schema from
version N-1 to N.  The upgrades are wrapped in a transaction.
2012-02-28 20:16:16 +01:00
Eelco Dolstra 918fc5e6df Manual: Remove tabs, indent consistently 2012-02-28 15:38:47 +01:00
Eelco Dolstra da26294fdb Don't install hydra-create
It doesn't do anything useful for users.
2012-02-28 15:33:28 +01:00
Eelco Dolstra b13664308c Whitespace 2012-02-28 15:27:44 +01:00
Eelco Dolstra 3d824b3a3a Indentation 2012-02-21 17:59:36 +01:00
Eelco Dolstra 578530dac2 Don't show duplicate builds in "Related builds"
Since a build may be a member of multiple jobset evaluations, we need
to do a "select distinct" here.  But maybe we should only show builds
from a single evaluation (e.g. the most recent), since showing builds
from several may be confusing.
2012-02-21 17:16:32 +01:00
Rob Vermaas 2cf9b148eb 50 for related builds might be a kind of low, will start with 100 2012-02-16 17:00:59 +01:00
Rob Vermaas 924d94cafd * minor performance tweaks
* added related builds tab which show related builds (builds that are in same evaluations), shows max 50
2012-02-16 16:56:01 +01:00
Rob Vermaas 355135b6e8 add links in admin page to clear queue (all non-running builds, and all non-running old (non-current) builds) 2012-02-16 15:31:12 +01:00
Rob Vermaas bb6ef9c2ed accidentally committed an = too little, which caused all unknown values to result in 'not building', which was not the intention 2012-02-16 11:14:22 +01:00
Rob Vermaas 739a03393c allow null values for jobs, meaning it should build 2012-02-16 11:03:22 +01:00
Ludovic Courtès 1f448673f0 hydra-build: Do not send email following an abortion. 2012-02-14 11:48:44 +01:00
Eelco Dolstra a3cc2ce796 Remove redundant dot in status emails
E.g.:

  This is to let you know that Hydra build 1998534 of job patchelf:trunk:deb_ubuntu1110x86_64 is 'Success'..
2012-02-07 14:51:47 +01:00
Ludovic Courtès 5644c1c7d9 hydra-build: Add system info to the subject and extra headers. 2012-01-31 18:27:33 +01:00
Rob Vermaas 8d78a688c7 is not in hydra-server's scope 2012-01-28 21:03:44 +01:00