The catalyst-action-rest branch from shlevy/hydra was an exploration of
using Catalyst::Action::REST to create a JSON API for hydra. This commit
merges in the best bits from that experiment, with the goal that further
API endpoints can be added incrementally.
In addition to migrating more endpoints, there is potential for
improvement in what's already been done:
* The web interface can be updated to use the same non-GET endpoints as
the JSON interface (using x-tunneled-method) instead of having a
separate endpoint
* The web rendering should use the $c->stash->{resource} data structure
where applicable rather than putting the same data in two places in
the stash
* Which columns to render for each endpoint is a completely debatable
question
* Hydra::Component::ToJSON should turn has_many relations that have
strings as their primary keys into objects instead of arrays
FixesNixOS/hydra#98
Signed-off-by: Shea Levy <shea@shealevy.com>
Previously, for scheduled builds, "timestamp" contained the time the
build was added to the queue, while for finished builds, it was the
time the build finished. Now it's always the former.
You can now do:
bash <(curl http://hydra-server/build/1238757/reproduce)
to download and execute a script that reproduces a Hydra build
locally. This script fetches all inputs (e.g. Git repositories) and
then invokes nix-build.
The downloaded sources are stored in /tmp/build-<buildid> and reused
between invocations of the script.
Any additional command line options are passed to nix-build. So
bash <(curl http://hydra-server/build/1238757/reproduce) --run-env
will drop you in a shell where you can interactively hack on the
build, e.g.
$ source $stdenv/setup
$ set +e
$ unpackPhase
$ cd $sourceRoot
$ configurePhase
$ emacs foo.c &
$ make
and so on.
Build product paths cannot reference locations outside of the Nix
store. We previously disallowed paths from being symlinks, but this
didn't take into account that parent path elements can be symlinks as
well. So a build product /nix/store/bla.../foo/passwd, with
/nix/store/bla.../foo being a symlink to /etc, would still work.
So now we check all paths encountered during path resolution.
Symlinks are allowed again so long as they point to the Nix store.
External machines can now notify Hydra that it should check a
repository by sending a GET or PUSH request to /api/push, providing a
list of jobsets to be checked and/or a list of repository URLs. In
the latter case, all jobsets that have any of the specified
repositories as an input will be checked.
For instance, you can configure GitHub or BitBucket to send a request
to the URL
http://hydra.example.org/api/push?repos=git://github.com/NixOS/nixpkgs.git
to trigger evaluation of all jobsets that have
git://github.com/NixOS/nixpkgs.git as an input, or to the URL
http://hydra.example.org/api/push?jobsets=patchelf:trunk,nixpkgs:trunk
to trigger evaluation of just the specified jobsets.
Otherwise you can do
ln -s /etc/passwd $out/foo
echo "file misc $out/foo" >> $out/nix-support/hydra-build-products
and get Hydra to serve its /etc/passwd file.
You can now just click on the evaluation link on the first tab to see
all builds in the same jobset. This also makes rendering build pages
quite a bit faster for jobsets like Nixpkgs.
It's pointless to store these, since Nix knows where the logs are.
Also handle (in fact require) Nix's new log storage scheme. Also some
cleanups in the build page.
This happened in a pathological case in Nixpkgs: the "grub" job is
evaluated for i686-linux and x86_64-linux, but in the latter case it
returns the same derivation as in the former case. So only one build
should be added.
In particular the /pkg action is now O(lg n) instead of O(n) in the
number of packages in the channel, and listing the channel contents
no longer requires calling isValidPath() on all packages.
Derivations (and thus build time dependencies) are no longer included
in the channel, because they're not GC roots. Thus they could
disappear unexpectedly.
* Don't use isCurrent anymore; instead look up builds in the previous
jobset evaluation. (The isCurrent field is still maintained because
it's still used in some other places.)
* To determine whether to perform an evaluation, compare the hash of
the current inputs with the inputs of the previous jobset
evaluation, rather than checking if there was ever an evaluation
with those inputs. This way, if the inputs of an evaluation change
back to a previous state, we get a new jobset evaluation in the
database (and thus the latest jobset evaluation correctly represents
the latest state of the jobset).
* Improve performance by removing some unnecessary operations and
adding an index.
Since a build may be a member of multiple jobset evaluations, we need
to do a "select distinct" here. But maybe we should only show builds
from a single evaluation (e.g. the most recent), since showing builds
from several may be confusing.
