Commit graph

1259 commits

Author SHA1 Message Date
Eelco Dolstra 8b88ce19c2 Huuuge speedup in the Hydra evaluator
The check to see whether a build had been scheduled in a previous
evaluation took about 200 ms for the nixpkgs:trunk jobset.  Given
that it has more than 15000 builds, this added up to a lot.  Now
it takes 0.2 ms per build.
2012-04-04 17:29:03 +02:00
Eelco Dolstra 1cdc023427 Propagate URI parameters 2012-04-04 13:17:22 +02:00
Eelco Dolstra e9dd3158a0 Propagate URI parameters
This allows URIs like

  http://hydra.nixos.org/jobset/nixpkgs/trunk/latest-eval?compare=stdenv

to do the right thing.
2012-04-04 12:56:49 +02:00
Eelco Dolstra 05d03ee153 Don't use colspan=0, it doesn't work in all browsers 2012-04-04 10:33:00 +00:00
Eelco Dolstra 7734de5817 Add OpenSSL as a dependency because Nix needs it
Actually the real problem is that a) Nix doesn't propagate the OpenSSL
dependency; b) Nix shouldn't link against OpenSSL by default anyway.
2012-04-04 11:43:14 +02:00
Eelco Dolstra f52ca0c588 Add a redirect to the latest view result for which the underlying evaluation has finished completely
This will be useful for the Nixpkgs channel mirror script:

  http://hydra.nixos.org/view/nixpkgs/unstable/latest-finished/channel

is the channel containing the latest, consistent, tested set of builds.
2012-04-03 17:45:03 +02:00
Eelco Dolstra bd93ca9316 Add a redirect to the evaluation page of a view result
E.g.

  http://hydra.nixos.org/view/nixpkgs/unstable/latest/eval

Thus this is the channel representing the latest Nixpkgs for which the
elements of the view "unstable" succeeded:

  http://hydra.nixos.org/view/nixpkgs/unstable/latest/eval/channel
2012-04-03 15:45:48 +02:00
Eelco Dolstra 2c677ec71a Always record inputs passed through -I in the BuildInputs table
We currently have no way to determine if the Nix evaluator used a
specific -I input, so we need to record all of them as inputs.
2012-04-03 10:10:45 +00:00
Eelco Dolstra 30e2b9046a Cleanup 2012-04-03 11:28:59 +02:00
Eelco Dolstra db2b2b6134 Add a redirect to the latest finished jobset evaluation
The action .../jobset/<project>/<jobset>/latest-eval redirects to the
latest evaluation of the jobset that has no unfinished builds.  Thus,
for instance,

  http://hydra.nixos.org/jobset/nixpkgs/trunk/latest-eval/channel

is the channel containing the latest consistent set of Nixpkgs builds.
2012-04-03 01:30:54 +02:00
Eelco Dolstra 262f068e26 Support obtaining the contents of an evaluation as a channel
E.g. http://hydra.nixos.org/eval/552513/channel

The advantage of such a channel is that all builds come from the same
evaluation, and thus are consistent.
2012-04-02 23:27:17 +02:00
Eelco Dolstra 3e57cbfeb8 User color instead of a big animated GIF to indicate whether a build is running 2012-04-02 21:07:49 +02:00
Eelco Dolstra 69e600da89 Allow comparing an evaluation with an arbitrary other evaluation
The URI parameter "compare=..." can denote either an arbitrary
evaluation ID, or the name of a jobset in the same project.  In the
latter case, the comparison is made against the latest completed
evaluation of the specified jobset.
2012-04-02 20:40:59 +02:00
Eelco Dolstra 2f9153c640 Prevent multiple builds with the same (job, outPath) tuple from being added
This happened in a pathological case in Nixpkgs: the "grub" job is
evaluated for i686-linux and x86_64-linux, but in the latter case it
returns the same derivation as in the former case.  So only one build
should be added.
2012-04-02 15:56:29 +00:00
Eelco Dolstra 9d4cb29e08 Show queued builds in jobset evaluation pages 2012-04-02 15:23:42 +00:00
Eelco Dolstra 427e5e8db8 Show new and removed jobs 2012-04-02 16:45:05 +02:00
Eelco Dolstra 662cdf0421 Add support for viewing jobset evaluations 2012-04-02 16:11:22 +02:00
Eelco Dolstra d33fbad438 Unpack jQuery etc. in the build tree for easier testing 2012-04-02 11:15:48 +00:00
Eelco Dolstra 0aae52bfcc In build tables, hide the project, jobset and/or job name if they're constant 2012-04-02 13:12:39 +02:00
Rob Vermaas 1904f82f06 add openssl to deps 2012-04-01 18:15:00 +02:00
Eelco Dolstra d8f8143cc2 No wonder our disk was filling up 2012-03-26 17:13:50 +02:00
Rob Vermaas 4d4e1986e8 remove delete jobset button until we actually have implemented the action properly 2012-03-26 14:39:50 +02:00
Eelco Dolstra e1768cae86 Don't barf if the SQLite DB is missing
This prevented hydra-init from starting.
2012-03-19 03:57:11 +00:00
Ludovic Courtès 3846c2407d Fix URL in the "password changed" email. 2012-03-13 18:30:57 +01:00
Ludovic Courtès d0f18a1d6f Add dependency on Switch. 2012-03-13 18:12:37 +01:00
Ludovic Courtès 2b68e41ee4 Actually add SQL files to the distribution. 2012-03-13 17:49:18 +01:00
Ludovic Courtès ace94192ac Update the list of SQL files to be installed.
This fixes `hydra-init', which would otherwise not find `upgrade-*.sql',
thus skipping the update altogether.
2012-03-13 17:45:14 +01:00
Eelco Dolstra dae5032c1c I'm stupid 2012-03-13 13:33:19 +01:00
Eelco Dolstra c844b9f09a Remove the global Job status and Errors links 2012-03-13 13:31:36 +01:00
Eelco Dolstra 161d7418f8 Doh 2012-03-13 13:30:41 +01:00
Eelco Dolstra fa62c8b7f2 Fix tests 2012-03-13 13:18:40 +01:00
Eelco Dolstra 1f268d2d43 Update isCurrent properly 2012-03-13 13:09:10 +01:00
Eelco Dolstra 179b012a8e Open the DB using Hydra::Model::DB->new
This gets rid of the openHydraDB function and ensures that we
open the database in a consistent way.

Also drop the PostgreSQL sequence hacks.  They don't seem to be
necessary anymore.
2012-03-13 12:10:19 +01:00
Eelco Dolstra 914c5e248a Use <...> syntax 2012-03-13 11:03:32 +01:00
Eelco Dolstra 93efae0c23 Fix "make check" 2012-03-12 21:13:50 +01:00
Eelco Dolstra 13f4636436 hydra-evaluator: handle the case where there is no previous jobset eval 2012-03-12 21:13:28 +01:00
Eelco Dolstra 3f1bbf7ef4 Better fix 2012-03-12 21:06:42 +01:00
Eelco Dolstra 259afadbab Set $PERL5LIB properly in hydra-init in "make check" 2012-03-12 20:56:41 +01:00
Eelco Dolstra a4cda5e337 Don't use the Switch module
It was removed in Perl 5.14.
2012-03-12 20:47:30 +01:00
Eelco Dolstra 87e4d43848 Fix the jobset unchanged check
When checking whether the jobset is unchanged, we need to compare with
the previous JobsetEval regardless of whether it had new builds.
Otherwise we'll keep adding new JobsetEval rows.
2012-03-12 20:47:30 +01:00
Eelco Dolstra 27619f18e2 Re-enable the "Related Builds" tab
However, there is a "limit 1" on the JobsetEval search to prevent a
sequential scan of the Builds table.
2012-03-12 20:47:30 +01:00
Eelco Dolstra abe71a767b Disable prepared statements completely
Because of the way DBIx::Class does prepared statements, even
innocuous queries such

  $c->model('DB::Builds)->search({finished => 0})

can be extremely slow.  This is because DBIx::Class prepares a
PostgreSQL statement

  select ... from Builds where finished = ?

and since Builds is very large and there is a large fraction of rows
with "finished = 1", the PostgreSQL query planner decides to implement
this query with a sequential scan of the Builds table (despite the
existence of an index on "finished"), which is extremely slow.  It
would be nice if we could tell DBIx::Class that constants should be
part of the prepared statement, i.e.

  select ... from Builds where finished = 0

but AFAIK we can't.
2012-03-12 20:47:30 +01:00
Eelco Dolstra 2d1cf73974 Speed up channel processing
In particular the /pkg action is now O(lg n) instead of O(n) in the
number of packages in the channel, and listing the channel contents
no longer requires calling isValidPath() on all packages.

Derivations (and thus build time dependencies) are no longer included
in the channel, because they're not GC roots.  Thus they could
disappear unexpectedly.
2012-03-12 20:47:30 +01:00
Eelco Dolstra 9f10c0f9c0 Fix a reference to BuildResultInfo 2012-03-12 20:47:30 +01:00
Eelco Dolstra 07b3dffd20 Reduce I/O in build listings by only fetching required columns
Columns such as "longDescription" can be large, so fetching them
when they're not needed is wasteful.
2012-03-12 20:47:30 +01:00
Eelco Dolstra 6526d4a65f hydra-build: only send email if the status differs from the previous build
This isn't perfect because it doesn't handle the case where a
previous build hasn't finished yet.  But at least it won't send mail
for old builds that fail while a newer build has already succeeded.
2012-03-12 20:47:29 +01:00
Eelco Dolstra 47f877c5bb Evaluator cleanups
* Don't use isCurrent anymore; instead look up builds in the previous
  jobset evaluation.  (The isCurrent field is still maintained because
  it's still used in some other places.)

* To determine whether to perform an evaluation, compare the hash of
  the current inputs with the inputs of the previous jobset
  evaluation, rather than checking if there was ever an evaluation
  with those inputs.  This way, if the inputs of an evaluation change
  back to a previous state, we get a new jobset evaluation in the
  database (and thus the latest jobset evaluation correctly represents
  the latest state of the jobset).

* Improve performance by removing some unnecessary operations and
  adding an index.
2012-03-12 20:47:29 +01:00
Eelco Dolstra bc82a82593 Fix a race condition in hydra-update-gc-roots
Since it read the actual roots after determining the set of desired
roots, there was a possibility that it would delete roots added by
hydra-evaluator or hydra-build while hydra-update-gc-roots was
running.  This could cause a derivation to be garbage-collected before
the build was performed, for instance.  Now the actual roots are read
first, so any root added after that time is not deleted.
2012-03-12 20:47:29 +01:00
Eelco Dolstra 29d5a02b94 Speed up hydra-update-gc-roots
The hydra-update-gc-roots script is taking around 95 minutes on our
Hydra instance (though a lot of that is I/O wait).  This patch
significantly reduces the number of database queries.  In particular,
the N most recent successful builds for each job in a jobset are now
determined in a single query.  Also, it removes the calls to
readlink().
2012-03-12 20:47:29 +01:00
Eelco Dolstra 5be004c999 Drop the merged tables after conversion 2012-03-12 20:47:29 +01:00