lix-project/hydra
3
0
Fork 6
Code Issues 2 Pull requests 1 Packages Projects Releases Wiki Activity
3710 commits 3 branches 0 tags 20 MiB
Commit graph

3710 commits

This branch
This branch
All branches
Author SHA1 Message Date
Graham Christensen ba96a13407 Record metrics when getting the closure to localhost 2022-01-21 15:38:05 -05:00
Graham Christensen 7e9e82398d build-remote: copy missing paths from the binary cache to localhost 
In a Hydra instance I saw:

    possibly transient failure building ‘/nix/store/X.drv’ on ‘localhost’:
      dependency '/nix/store/Y' of '/nix/store/Y.drv' does not exist,
      and substitution is disabled

This is confusing because the Hydra in question does have substitution enabled.

This instance uses:

  keep-outputs = true
  keep-derivations = true

and an S3 binary cache which is not configured as a substituter in the nix.conf.

It appears this instance encountered a situation where store path Y was built
and present in the binary cache, and Y.drv was GC rooted on the instance,
however Y was not on the host.

When Hydra would try to build this path locally, it would look in the binary
cache to see if it was cached:

    (nix)
    439      bool valid = isValidPathUncached(storePath);
    440
    441      if (diskCache && !valid)
    442          // FIXME: handle valid = true case.
    443          diskCache->upsertNarInfo(getUri(), hashPart, 0);
    444
    445      return valid;

Since it was cached, the store path was considered Valid.

The queue monitor would then not put this input in for substitution, because
the path is valid:

    (hydra)
    470          if (!destStore->isValidPath(*i.second.path(*localStore, step->drv->name, i.first))) {
    471              valid = false;
    472              missing.insert_or_assign(i.first, i.second);
    473          }

Hydra appears to correctly handle the case of missing paths that need
to be substituted from the binary cache already, but since most
Hydra instances use `keep-outputs` *and* all paths in the binary cache
originate from that machine, it is not common for a path to be cached
and not GC rooted locally.

I'll run Hydra with this patch for a while and see if we run in to the
problem again.

A big thanks to John Ericson who helped debug this particular issue.
 2022-01-21 15:26:45 -05:00
Graham Christensen 952f629b7c Test the queue runner in the scenario where a dependency is available in the cache but GC'd locally, where we're building locally 2022-01-21 15:26:45 -05:00
Graham Christensen 5c3e48fd0d CliRunners: decode UTF8 before printing stderr/stdout 
Fixes yath output. It used to say:

    step â is now runnable

Now it says:

    step ‘/run...2ipqz6hbc41m4c5w5bkq-dependent-job.drv’ is now runnable
 2022-01-21 15:24:33 -05:00
Graham Christensen e4407f8c93 HydraTestContext: expose the nix state dir 2022-01-21 15:12:10 -05:00
Graham Christensen e351054f61
Merge pull request #1129 from DeterminateSystems/fixup-argon2 
Fixup argon2 instructions in hydra-create-user
 2022-01-21 13:01:37 -05:00
Graham Christensen 61325853a6
Merge pull request #1132 from DeterminateSystems/ldap-role-match 
LDAP support: require the prefix 'hydra_' to match documentation
 2022-01-21 12:58:35 -05:00
Graham Christensen a888a57baf tests.ldap: verify the hydra_ prefix is required 2022-01-21 12:46:02 -05:00
Graham Christensen 8ba4ae461e
Merge pull request #1131 from kenranunderscore/fix-invalid-yaml 
Fix invalid YAML in documentation
 2022-01-21 11:03:28 -05:00
Graham Christensen b8f72d7ff2 LDAP support: require the prefix 'hydra_' to match documentation 2022-01-21 10:48:04 -05:00
Graham Christensen 4945306a2b hydra-create-user: make docs about using --password-hash better 2022-01-21 10:39:22 -05:00
Johannes Maier 4476aba5f7 Fix invalid YAML in documentation 2022-01-21 16:38:59 +01:00
Graham Christensen 12d0d0c176 hydra-create-user: use test_context over test_init 2022-01-21 10:37:48 -05:00
Graham Christensen bb9864bad7 hydra-create-user.t: rename the various users based on their password type 2022-01-21 09:12:47 -05:00
Graham Christensen ed1b532b74
Merge pull request #1093 from DeterminateSystems/builds-jobset-project 
Builds: drop the jobset and project columns
 2022-01-17 10:11:55 -05:00
Graham Christensen 8c50cd06e4 machines: ensure the jobset name is present 2022-01-15 17:11:08 -05:00
Graham Christensen c8dc6a9419 Plugins: get project and jobset information from the project and jobset tables 2022-01-15 15:58:02 -05:00
Graham Christensen 9dc40e0816 evaluator: don't save project, jobset on builds 2022-01-15 15:58:02 -05:00
Graham Christensen c539deea99 builds: add a build->project func to get the project via the jobset 2022-01-15 15:58:02 -05:00
Graham Christensen f120909547 builds: drop project, jobset columns 
Indexes were haphazardly dropped.
 2022-01-15 15:58:02 -05:00
Graham Christensen 1caff3a250
Merge pull request #1117 from DeterminateSystems/project-jobset/queue-runner 
queue-runner: track jobsets by ID
 2022-01-15 15:57:14 -05:00
Graham Christensen 9671d4d135
Merge pull request #1119 from DeterminateSystems/project-jobset/update-gc-roots 
Project jobset: update-gc-roots
 2022-01-15 15:57:06 -05:00
Graham Christensen f1a608ac35
Merge pull request #1118 from DeterminateSystems/project-jobset/declarativejobsets 
Project jobset: Update DeclarativeJobsets to not use the project or jobsets columns on Bulids
 2022-01-15 15:56:48 -05:00
Graham Christensen 7544d4ff47 hydra-update-gc-roots: get project and jobset information from the project and jobset tables 2022-01-15 14:26:45 -05:00
Graham Christensen 52dda56b99 hydra-update-gc-roots: add a very basic test validating successful execution 2022-01-15 14:20:11 -05:00
Graham Christensen 72c3110002 queue-runner: track jobsets by ID 2022-01-15 14:06:00 -05:00
Graham Christensen 17c6bd4fd8 DeclarativeJobsets: get the jobset name from the jobset table 2022-01-15 13:46:32 -05:00
Graham Christensen b2cdde0901 DeclarativeJobsets: test basic functionality 2022-01-15 13:46:32 -05:00
Graham Christensen eef633c1cc tests: create a declarative project spec and the autoconfig machinery 2022-01-15 13:46:32 -05:00
Graham Christensen 9d3b14dd43
Merge pull request #1116 from DeterminateSystems/project-jobset/input-type-build 
Project jobset: update input type "build"
 2022-01-15 12:48:20 -05:00
Graham Christensen 8c3c573953 hydra-eval-jobset: fixup old reference to project / jobset columns 2022-01-15 12:32:16 -05:00
Graham Christensen 2abcd84931
Merge pull request #1115 from DeterminateSystems/project-jobset/builds-json-repr 
Project jobset: update builds json repr
 2022-01-15 12:09:45 -05:00
Graham Christensen d91593e93a
Merge pull request #1114 from DeterminateSystems/project-jobset/delete-project 
Projects: delete: delete all builds first
 2022-01-14 23:04:55 -05:00
Graham Christensen 6bb9adc1a5 Builds: get the project and jobset names from the their tables 2022-01-14 22:45:26 -05:00
Graham Christensen cb68629417 test a Build's json representation 2022-01-14 21:19:48 -05:00
Graham Christensen f4c4b496d8 Projects: delete: delete all builds first 
Deleting jobsets first would fail because buildmetrics has an FK
to the jobset. However, the jobset / project relationship is not
marked as CASCADE.

Deleting all the builds automatically cascades to delete
buildmetrics, so deleting the relevant builds first, then deleting
the jobset solves it.
 2022-01-14 20:37:55 -05:00
Graham Christensen 0044622198 Projects: test deleting 2022-01-14 20:36:52 -05:00
Graham Christensen 9cc3e4264d
Merge pull request #1113 from DeterminateSystems/project-jobset/steps 
Project jobset: update /steps
 2022-01-14 17:00:52 -05:00
Graham Christensen 31c6c26121 common.tt: fixup refs 2022-01-14 16:49:58 -05:00
Graham Christensen 8ae2daece7 root: test /steps and /evals 2022-01-14 16:49:58 -05:00
Graham Christensen 3a03784af6
Merge pull request #1112 from DeterminateSystems/project-jobset/search 
Project jobset: update search
 2022-01-14 16:49:43 -05:00
Graham Christensen c7c4759600 search: fix references to jobset / project info 2022-01-14 16:38:25 -05:00
Graham Christensen 0b33550871 search.tt: fixup project and jobset reference 2022-01-14 16:38:25 -05:00
Graham Christensen e0e8840d8f Test /search 2022-01-14 16:38:25 -05:00
Graham Christensen 42a871e413
Merge pull request #1111 from DeterminateSystems/project-jobset/queue-summary-machines 
Project jobset columns: fixup /queue-summary and /machines
 2022-01-14 15:34:43 -05:00
Graham Christensen cc8104fa46
Merge pull request #1110 from DeterminateSystems/project-jobset/api-buildToHash 
Project jobset: api: update endpoints to stop using build.{project,jobset}
 2022-01-14 15:27:13 -05:00
Graham Christensen 8a663f2cf8 machines: fixup refs 2022-01-14 15:23:19 -05:00
Graham Christensen bdccad573c machines: test 2022-01-14 15:23:19 -05:00
Graham Christensen c945529f05 queue summary: fix refs 2022-01-14 15:23:19 -05:00
Graham Christensen da516f70a4 queue summary: test 2022-01-14 15:23:19 -05:00