Commit graph

132 commits

Author SHA1 Message Date
Cole Helbling 387fe8005a
tests/input-types: split out scminputs into individual tests
This makes the test faster (by removing it and replacing it with a
`TestScmInput` module that exports the `testScmInput` subroutine). Now,
all the input tests can be run in parallel.

Some of the `tests/jobs/*-update.sh` scripts were "broken" (e.g. tests
failed for various reasons on my machine), so I fixed those up as well.

Co-authored-by: gustavderdrache <gustavderdrache@gmail.com>
2021-03-04 13:58:48 -08:00
Graham Christensen a756614fa1
RunCommand: pass homepage, description, license, system, and nixname 2021-02-24 16:13:09 -05:00
Graham Christensen 3fda37f65a
RunCommand: Test 2021-02-24 13:43:25 -05:00
Graham Christensen b2520267a9
Test setup: support arbitrary hydra config 2021-02-24 11:44:16 -05:00
Graham Christensen cccdc70162
input-types.t: don't litter ./tests/ 2021-02-24 07:01:32 -05:00
Graham Christensen 611d7b71f2
input-types: use is() for test comparisons 2021-02-24 07:01:32 -05:00
Graham Christensen 371826f931
Tests: build-products: use is for good errors on failures 2021-02-24 07:01:32 -05:00
Graham Christensen 0df9c68422
Relocate the final evalutation tests to a build-products specific test. 2021-02-24 07:01:31 -05:00
Graham Christensen 2776ae6c78
Move tests for SCM inputs in to its own .t 2021-02-24 07:01:31 -05:00
Graham Christensen c8df544046
evaluate-dependent-jobsets: clean up test to be more clear 2021-02-24 07:01:31 -05:00
Graham Christensen 0b693ad8e8
Use is in evaluate-dependent-jobsets 2021-02-24 07:01:31 -05:00
Graham Christensen 9590bababc
Split out dependent tests in to its own .t 2021-02-24 07:01:25 -05:00
Graham Christensen 62b2880dfc
Give each test its own Nix directories
Otherwise we risk tripping over confusing statuses where a build is
"done" and "Cached", but we were expecting to run it.
2021-02-23 21:43:54 -05:00
Graham Christensen fe1f2f0806
Create an ephemeral PostgreSQL database per test 2021-02-23 21:12:06 -05:00
Graham Christensen b15d8edab1
tests: delete set-up.pl / tear-down.pl
We'll set these up on a per-test basis.
2021-02-23 21:12:06 -05:00
Graham Christensen 2240035e20
Run tests with yath
This will let us run tests in parallel, and creates a more Perl-standard
test development experience.
2021-02-19 17:04:19 -05:00
Graham Christensen bd99052a6f
tests: create database with the utf-8 locale
Otherwise tests may fail with wide character errors.
2021-01-21 17:08:05 -05:00
Nikola Knezevic fceaed2b24 Make PathInput plugin cache validity configurable
PathInput plugin keeps a cache of path evaluations. This cache is simple, and
path is not checked more than once every N seconds, where N=30. The caching is
there to avoid expensive calls to `nix-store --add`.

This change makes the validity period configurable. The main use case is
`api-test.pl` which was implemented wrong for a while, as the invocation of
`hydra-eval-jobset` would return the previous evaluation, claiming there are no
changes. The test has been fixed to check better for a new evaluation.
2020-06-04 12:26:47 +02:00
Eelco Dolstra 8adb433e3b
Remove the Jobs table
This table has been superfluous for a long time.
2020-05-27 20:09:36 +02:00
Maximilian Bosch efcbc08686
Get rid of dependency to SQLite
SQLite isn't properly supported by Hydra for a few years now[1], but
Hydra still depends on it. Apart from a slightly bigger closure this can
cause confusion by users since Hydra picks up SQLite rather than
PostgreSQL by default if HYDRA_DBI isn't configured properly[2]

[1] 78974abb69
[2] https://logs.nix.samueldr.com/nixos-dev/2020-04-10#3297342;
2020-04-16 00:42:40 +02:00
Eelco Dolstra 53e1294825 Fix Mercurial tests
In Nixpkgs 20.03, Mercurial fails if PYTHONPATH is set:

  $ hg
  Traceback (most recent call last):
    File "/nix/store/q7s856v6nw4dffdrm9k3w38qs35i8kr3-mercurial-5.2.2/bin/..hg-wrapped-wrapped", line 37, in <module>
      dispatch.run()
    File "/nix/store/bffdy7q3wi3qinflnvbdkigqj39zzynd-python3-3.7.6/lib/python3.7/importlib/util.py", line 245, in __getattribute__
      self.__spec__.loader.exec_module(self)
    File "<frozen importlib._bootstrap_external>", line 728, in exec_module
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "/nix/store/q7s856v6nw4dffdrm9k3w38qs35i8kr3-mercurial-5.2.2/lib/python3.7/site-packages/mercurial/dispatch.py", line 10, in <module>
      import difflib
    File "/nix/store/bffdy7q3wi3qinflnvbdkigqj39zzynd-python3-3.7.6/lib/python3.7/difflib.py", line 1084, in <module>
      import re
    File "/nix/store/bffdy7q3wi3qinflnvbdkigqj39zzynd-python3-3.7.6/lib/python3.7/re.py", line 143, in <module>
      class RegexFlag(enum.IntFlag):
  AttributeError: module 'enum' has no attribute 'IntFlag'

(cherry picked from commit 4009d4295e)
2020-03-31 22:19:19 +02:00
Nikola Knezevic 06bdc8f85c Added the InfluxDBNotification plugin including a NixOS test
This adds a `InfluxDBNotification` plugin which is configured as:

```
<influxdb>
  url = http://127.0.0.1:8086
  db = hydra
</influxdb>
```

which will write a notification for every finished job to the
configured database in InfluxDB looking like:

```
hydra_build_status,cached=false,job=job,jobset=default,project=sample,repo=default,result=success,status=success,system=x86_64-linux build_id="1",build_status=0i,closure_size=584i,duration=0i,main_build_id="1",queued=0i,size=168i 1564156212
```
2019-07-26 17:47:03 +02:00
danbst 4171ab4c4f tests: change postgresql socket dir to /tmp
In https://github.com/NixOS/nixpkgs/pull/57677 default postgresql socket directory was changed
to `/run/postgresql`, which doesn't exist (and can't be created) in
Nix build environment. We'll use /tmp as socket dir explicitly then.

Fixes build failure https://hydra.nixos.org/build/91221682

Cc @aszlig @edolstra
2019-03-29 23:46:38 +02:00
Graham Christensen 215ca5da9c
Merge pull request #607 from nlewo/json-search
Add JSON search API endpoint
2019-03-18 15:08:32 -05:00
Graham Christensen 4160132278 tests: disable sandbox
Prior, tests would all fail to build, causing, roughly, the following
error (roughly, because I added some debug log messages :)):

    ok 68 - Evaluating jobs/build-products.nix should result in 2 builds
    Queue runner stderr: using 4185024512 bytes for the NAR buffer
    locking path '/build/source/tests/data/queue-runner/lock'
    lock acquired on '/build/source/tests/data/queue-runner/lock.lock'
    warning: unknown setting 'max-connection-age'
    warning: unknown setting 'max-connections'
    dispatcher woken up
    dispatcher woken up
    dispatcher sleeping for 7674380800s
    adding new machine ‘localhost’
    dispatcher woken up
    checking the queue for builds > 0...
    dispatcher sleeping for 7674380800s
    sending notification about build 1
    loading build 18 (tests:build-products:simple)
    considering derivation ‘/build/source/tests/nix/store/24h0i450d4k00a4jhhk6r7qpqdvzskw6-build-product-simple.drv’
    sending notification about build 2
    creating build step ‘/build/source/tests/nix/store/24h0i450d4k00a4jhhk6r7qpqdvzskw6-build-product-simple.drv’
    added build 18 (top-level step /build/source/tests/nix/store/24h0i450d4k00a4jhhk6r7qpqdvzskw6-build-product-simple.drv, 1 new steps)
    got 1 new runnable steps from 1 new builds
    step ‘/build/source/tests/nix/store/24h0i450d4k00a4jhhk6r7qpqdvzskw6-build-product-simple.drv’ is now runnable
    dispatcher woken up
    dispatcher sleeping for 7674380800s
    performing step ‘/build/source/tests/nix/store/24h0i450d4k00a4jhhk6r7qpqdvzskw6-build-product-simple.drv’ 1 times on ‘localhost’ (needed by build 18
    and 0 others)
    sending closure of ‘/build/source/tests/nix/store/24h0i450d4k00a4jhhk6r7qpqdvzskw6-build-product-simple.drv’ to ‘localhost’
    building ‘/build/source/tests/nix/store/24h0i450d4k00a4jhhk6r7qpqdvzskw6-build-product-simple.drv’ on ‘localhost’
    killing process 10462
    marking build 18 as failed
    finishing build step ‘/build/source/tests/nix/store/24h0i450d4k00a4jhhk6r7qpqdvzskw6-build-product-simple.drv’

    ok 69 - Build 'simple' from jobs/build-products.nix should exit with code 0
    ok 70 - newbuild->finished was '1' instead of 1
    not ok 71 - newbuild->buildstatus was '1' instead of 0
    not ok 72 - Build 'simple' from jobs/build-products.nix should have buildstatus 0
    Can't call method "name" on an undefined value at ./evaluation-tests.pl line 173.
    FAIL: evaluation-tests.pl
2019-02-01 04:31:11 +01:00
Antoine Eiche 0d2a2d8923 Add json output for the search API endpoint
This commit also add a test of this feature.

Note the search JSON output doesn't contain any jobs because they can
not be exported to JSON yet.

The JSON output on a search query matching a build looks like:

```
{
  "builds": [
    {
      "buildoutputs": {
        "out": {
          "path": "/nix/store/wdag3pznrvqk01byk989irg7rq3q2a2c-job"
        }
      },
      "finished": 0,
      "releasename": null,
      "starttime": null,
      "project": "sample",
      "buildproducts": {},
      "timestamp": 1541007629,
      "buildstatus": null,
      "nixname": "job",
      "drvpath": "/nix/store/n9zqndn7j7nyr6gg3bmxvw26cfmdwv2n-job.drv",
      "job": "job",
      "id": 1,
      "stoptime": null,
      "priority": 100,
      "system": "x86_64-linux",
      "jobsetevals": [
        1
      ],
      "jobset": "default",
      "buildmetrics": {}
    }
  ],
  "projects": [],
  "jobsets": [],
  "buildsdrv": []
}
```
2018-11-01 09:31:15 +01:00
Antoine Eiche dd1e4a92c0 api-test: make created project and jobset visible 2018-10-31 18:51:02 +01:00
Shea Levy c40077ece1 Fix tests 2017-05-24 10:00:38 -04:00
Eelco Dolstra 81ac547d2b
Move most of AddBuilds to hydra-eval-jobset
Having this stuff in a separate module is a remnant of the time when
hydra-server could add builds to the queue directly. This is no longer
the case.
2017-02-21 17:20:48 +01:00
Eelco Dolstra cc866bff1e Fix API tests
POST requests require a referrer now.

http://hydra.nixos.org/build/42775011
2016-10-27 13:15:09 +02:00
Eelco Dolstra e0b2921ff2 Concurrent hydra-evaluator
This rewrites the top-level loop of hydra-evaluator in C++. The Perl
stuff is moved into hydra-eval-jobset. (Rewriting the entire evaluator
would be nice but is a bit too much work.) The new version has some
advantages:

* It can run multiple jobset evaluations in parallel.

* It uses PostgreSQL notifications so it doesn't have to poll the
  database. So if a jobset is triggered via the web interface or from
  a GitHub / Bitbucket webhook, evaluation of the jobset will start
  almost instantaneously (assuming the evaluator is not at its
  concurrency limit).

* It imposes a timeout on evaluations. So if e.g. hydra-eval-jobset
  hangs connecting to a Mercurial server, it will eventually be
  killed.
2016-10-14 14:22:12 +02:00
Eelco Dolstra e94ef4627c Tests: Clear $NIX_REMOTE_SYSTEMS 2016-10-06 17:19:16 +02:00
Eelco Dolstra f2724d088d Test: Show error output from hydra-queue-runner on timeout 2016-10-06 17:18:59 +02:00
Shea Levy 93f9846425 git-rev: Remove superfluous git tag 2016-04-12 16:30:42 -04:00
Shea Levy 10e036238c Add eval test for specific git revision 2016-04-12 16:27:11 -04:00
Eelco Dolstra b1ce76c2b4 Fix test
nix-support/failed is supposed to be a file, not a directory.
2016-02-26 19:54:32 +01:00
Eelco Dolstra 48d6f0de2a Fix the API test 2015-08-12 16:07:19 +02:00
Eelco Dolstra b5815e2aa6 Run PostgreSQL during "make check" 2015-06-25 16:47:39 +02:00
Eelco Dolstra 18a3c3ff1c Update "make check" for the new queue runner
Also, if the machines file contains an entry for localhost, then run
"nix-store --serve" directly, without going through SSH.
2015-06-25 16:47:39 +02:00
Eelco Dolstra d6354cbe1f Fix Perl error in tests.api
Doing "su hydra" causes Perl to be invoked with a wrong PERL5LIB
(pointing to root's profile), leading to "Can't locate strict.pm".
2015-06-15 11:56:51 +02:00
Eelco Dolstra c93ff1e817 Doh 2015-05-26 15:25:21 +02:00
Eelco Dolstra f53e3c738d Fix test
http://hydra.nixos.org/build/14852570
2014-09-26 13:47:42 +02:00
Eelco Dolstra a052bba1bb Fix test 2014-04-24 16:08:04 +02:00
Eelco Dolstra c92410c147 Use hydra-module.nix in the tests 2013-11-06 18:10:52 +01:00
Eelco Dolstra 64cbdddea8 Test whether incorrect passwords are rejected 2013-11-06 16:12:08 +01:00
Eelco Dolstra 8c2acdd294 make clean: Delete darcs-repo 2013-11-06 14:29:55 +00:00
Shea Levy 5da3dcfd57 Make the outputs' outpaths available via the Build JSON API
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-29 14:41:48 -04:00
Shea Levy 4eef090a45 Flesh out the API a bit
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-29 13:54:11 -04:00
Shea Levy 5d3df69a0b Cleanup build serialization
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-25 11:09:11 -04:00
Shea Levy e04477003e Cleanup eval serialization
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-10-25 11:09:11 -04:00