Fix findBuildDependencyInQueue

Previously this function didn't actually have a lot of effect.  If a
build A had a dependency B, Hydra would start B first.  But on the
next scan through the queue, it would start A anyway, because of the
"busy => 0" restriction.

Now the queue runner won't start a build if a dependency is already
running.  (This is not necessarily optimal, since the build may have
other dependencies that don't correspond to a build in the queue but
could run.  One day we'll start all Hydra builds in parallel...)

Also, for performance, use computeFSClosure instead of "nix-store
-qR".  And don't bother with topological sorting because it didn't
have an effect anyway since the database returns dependencies in
arbitrary order.
This commit is contained in:
Eelco Dolstra 2013-05-03 16:18:27 +02:00
parent 9e72c64eff
commit 906b129f6a

View file

@ -48,23 +48,15 @@ sub unlockDeadBuilds {
} }
# Given a build, return an arbitrary queued build on which this build
# depends; or undef if no such build exists.
sub findBuildDependencyInQueue { sub findBuildDependencyInQueue {
my ($build) = @_; my ($build) = @_;
my $drvpath = $build->drvpath; my @deps = grep { /\.drv$/ && $_ ne $build->drvpath } computeFSClosure(0, 0, $build->drvpath);
my @paths = reverse(split '\n', `nix-store -qR $drvpath`); return unless scalar @deps > 0;
return $db->resultset('Builds')->search(
my $depBuild; { drvpath => [ @deps ], finished => 0, enabled => 1, disabled => 0 },
my @drvs = (); { join => ['project'], rows => 1 })->single;
foreach my $path (@paths) {
push @drvs, $path if $path =~ /\.drv$/ && $path ne $drvpath;
}
return unless scalar @drvs > 0;
($depBuild) = $db->resultset('Builds')->search(
{ drvpath => [ @drvs ], finished => 0, busy => 0, enabled => 1, disabled => 0 },
{ join => ['project'], rows => 1 } ) ;
return $depBuild;
} }
@ -116,8 +108,15 @@ sub checkBuilds {
"starting ", scalar(@builds), " builds\n"; "starting ", scalar(@builds), " builds\n";
foreach my $build (@builds) { foreach my $build (@builds) {
my $depbuild = findBuildDependencyInQueue($build); # Find a dependency of $build that has no queued
$build = $depbuild if defined $depbuild; # dependencies itself. This isn't strictly necessary,
# but it ensures that Nix builds are done as part of
# their corresponding Hydra builds, rather than as a
# dependency of some other Hydra build.
while (my $dep = findBuildDependencyInQueue($build)) {
$build = $dep;
}
next if $build->busy;
my $logfile = getcwd . "/logs/" . $build->id; my $logfile = getcwd . "/logs/" . $build->id;
mkdir(dirname $logfile); mkdir(dirname $logfile);