Cole Helbling
4a441b54ce
hydra-module: /var/lib/hydra -> ${baseDir}
2022-01-28 13:03:15 -08:00
Cole Helbling
14090fbb86
runcommand-log.tt: init
2022-01-28 13:03:15 -08:00
Janne Heß
796ce165d4
RunCommand: Allow displaying command output
2022-01-28 13:03:15 -08:00
Janne Heß
4cb5e6cd94
RunCommand: Capture the output of the commands
2022-01-28 13:00:17 -08:00
Graham Christensen
001539c3d2
Merge pull request #1127 from NixOS/grahamc-patch-1
...
Create a bug report issue template
2022-01-25 09:57:17 -05:00
Graham Christensen
0a5f625746
Update bug_report.md
2022-01-24 20:36:08 -05:00
Graham Christensen
f6e86efc9f
Merge pull request #1091 from Ma27/ssh-remote-store-location
...
hydra-queue-runner: support store URIs declaring an alternate store location
2022-01-24 14:10:54 -05:00
Graham Christensen
3a4ea6e563
Merge pull request #1124 from obsidiansystems/simplify--closure-of-path-set
...
simplify, `computeFSClosure` can take a set now
2022-01-24 14:09:35 -05:00
Graham Christensen
bb68b56f61
Merge pull request #1133 from helsinki-systems/doc/config-format
...
doc: Document the file format of the config
2022-01-21 20:49:18 -05:00
Graham Christensen
c280692f91
Merge pull request #1126 from DeterminateSystems/build-localhost-paths
...
build-remote: copy missing paths from the binary cache to localhost
2022-01-21 16:16:33 -05:00
Graham Christensen
44cd890ae3
Merge pull request #1130 from DeterminateSystems/prompt-password
...
hydra-create-user: support prompting for password
2022-01-21 15:38:39 -05:00
Graham Christensen
ba96a13407
Record metrics when getting the closure to localhost
2022-01-21 15:38:05 -05:00
Graham Christensen
7e9e82398d
build-remote: copy missing paths from the binary cache to localhost
...
In a Hydra instance I saw:
possibly transient failure building ‘/nix/store/X.drv’ on ‘localhost’:
dependency '/nix/store/Y' of '/nix/store/Y.drv' does not exist,
and substitution is disabled
This is confusing because the Hydra in question does have substitution enabled.
This instance uses:
keep-outputs = true
keep-derivations = true
and an S3 binary cache which is not configured as a substituter in the nix.conf.
It appears this instance encountered a situation where store path Y was built
and present in the binary cache, and Y.drv was GC rooted on the instance,
however Y was not on the host.
When Hydra would try to build this path locally, it would look in the binary
cache to see if it was cached:
(nix)
439 bool valid = isValidPathUncached(storePath);
440
441 if (diskCache && !valid)
442 // FIXME: handle valid = true case.
443 diskCache->upsertNarInfo(getUri(), hashPart, 0);
444
445 return valid;
Since it was cached, the store path was considered Valid.
The queue monitor would then not put this input in for substitution, because
the path is valid:
(hydra)
470 if (!destStore->isValidPath(*i.second.path(*localStore, step->drv->name, i.first))) {
471 valid = false;
472 missing.insert_or_assign(i.first, i.second);
473 }
Hydra appears to correctly handle the case of missing paths that need
to be substituted from the binary cache already, but since most
Hydra instances use `keep-outputs` *and* all paths in the binary cache
originate from that machine, it is not common for a path to be cached
and not GC rooted locally.
I'll run Hydra with this patch for a while and see if we run in to the
problem again.
A big thanks to John Ericson who helped debug this particular issue.
2022-01-21 15:26:45 -05:00
Graham Christensen
952f629b7c
Test the queue runner in the scenario where a dependency is available in the cache but GC'd locally, where we're building locally
2022-01-21 15:26:45 -05:00
Graham Christensen
5c3e48fd0d
CliRunners: decode UTF8 before printing stderr/stdout
...
Fixes yath output. It used to say:
step â is now runnable
Now it says:
step ‘/run...2ipqz6hbc41m4c5w5bkq-dependent-job.drv’ is now runnable
2022-01-21 15:24:33 -05:00
Graham Christensen
e4407f8c93
HydraTestContext: expose the nix state dir
2022-01-21 15:12:10 -05:00
Janne Heß
56308dbb05
doc: Document the file format of the config
2022-01-21 20:48:50 +01:00
Graham Christensen
da1af1ce68
Docs: use hydra-create-user --password-prompt
2022-01-21 13:05:12 -05:00
Graham Christensen
e351054f61
Merge pull request #1129 from DeterminateSystems/fixup-argon2
...
Fixup argon2 instructions in hydra-create-user
2022-01-21 13:01:37 -05:00
Graham Christensen
61325853a6
Merge pull request #1132 from DeterminateSystems/ldap-role-match
...
LDAP support: require the prefix 'hydra_' to match documentation
2022-01-21 12:58:35 -05:00
Graham Christensen
0eeced7f08
hydra-create-user: Warn that creating users with a plaintext password is deprecated
2022-01-21 12:56:15 -05:00
Graham Christensen
98928a4125
fixups
2022-01-21 12:52:06 -05:00
Graham Christensen
a888a57baf
tests.ldap: verify the hydra_ prefix is required
2022-01-21 12:46:02 -05:00
Graham Christensen
76fbde6d6b
Set noecho when reading passwords
2022-01-21 11:11:09 -05:00
Graham Christensen
8ba4ae461e
Merge pull request #1131 from kenranunderscore/fix-invalid-yaml
...
Fix invalid YAML in documentation
2022-01-21 11:03:28 -05:00
Graham Christensen
b8f72d7ff2
LDAP support: require the prefix 'hydra_' to match documentation
2022-01-21 10:48:04 -05:00
Graham Christensen
bb893d0bd5
hydra-create-user: support prompting for passwords
...
I'm not sure this is a good implementation as-is. It does work,
but the password gets echo'd to the screen. I tried to use IO::Prompt
but IO::Prompt really seems to want to read the password from ARGV.
2022-01-21 10:40:56 -05:00
Graham Christensen
3a6c25489c
Hydra::Helper::Nix: expose a captureStdoutStderrWithStdin, make it available in tests
2022-01-21 10:40:06 -05:00
Graham Christensen
d4fe7e55dd
Hydra::Helper::Nix: sort exported functions
2022-01-21 10:40:06 -05:00
Graham Christensen
4945306a2b
hydra-create-user: make docs about using --password-hash better
2022-01-21 10:39:22 -05:00
Johannes Maier
4476aba5f7
Fix invalid YAML in documentation
2022-01-21 16:38:59 +01:00
Graham Christensen
12d0d0c176
hydra-create-user: use test_context over test_init
2022-01-21 10:37:48 -05:00
Graham Christensen
bb9864bad7
hydra-create-user.t: rename the various users based on their password type
2022-01-21 09:12:47 -05:00
Graham Christensen
46b8f7cce8
Create a bug report issue template
2022-01-20 20:10:28 -05:00
John Ericson
e7a1ae87aa
simplify, computeFSClosure
can take a set now
2022-01-20 14:53:01 -05:00
Graham Christensen
ed1b532b74
Merge pull request #1093 from DeterminateSystems/builds-jobset-project
...
Builds: drop the jobset and project columns
2022-01-17 10:11:55 -05:00
Graham Christensen
8c50cd06e4
machines: ensure the jobset name is present
2022-01-15 17:11:08 -05:00
Graham Christensen
c8dc6a9419
Plugins: get project and jobset information from the project and jobset tables
2022-01-15 15:58:02 -05:00
Graham Christensen
9dc40e0816
evaluator: don't save project, jobset on builds
2022-01-15 15:58:02 -05:00
Graham Christensen
c539deea99
builds: add a build->project func to get the project via the jobset
2022-01-15 15:58:02 -05:00
Graham Christensen
f120909547
builds: drop project, jobset columns
...
Indexes were haphazardly dropped.
2022-01-15 15:58:02 -05:00
Graham Christensen
1caff3a250
Merge pull request #1117 from DeterminateSystems/project-jobset/queue-runner
...
queue-runner: track jobsets by ID
2022-01-15 15:57:14 -05:00
Graham Christensen
9671d4d135
Merge pull request #1119 from DeterminateSystems/project-jobset/update-gc-roots
...
Project jobset: update-gc-roots
2022-01-15 15:57:06 -05:00
Graham Christensen
f1a608ac35
Merge pull request #1118 from DeterminateSystems/project-jobset/declarativejobsets
...
Project jobset: Update DeclarativeJobsets to not use the project or jobsets columns on Bulids
2022-01-15 15:56:48 -05:00
Graham Christensen
7544d4ff47
hydra-update-gc-roots: get project and jobset information from the project and jobset tables
2022-01-15 14:26:45 -05:00
Graham Christensen
52dda56b99
hydra-update-gc-roots: add a very basic test validating successful execution
2022-01-15 14:20:11 -05:00
Graham Christensen
72c3110002
queue-runner: track jobsets by ID
2022-01-15 14:06:00 -05:00
Graham Christensen
17c6bd4fd8
DeclarativeJobsets: get the jobset name from the jobset table
2022-01-15 13:46:32 -05:00
Graham Christensen
b2cdde0901
DeclarativeJobsets: test basic functionality
2022-01-15 13:46:32 -05:00
Graham Christensen
eef633c1cc
tests: create a declarative project spec and the autoconfig machinery
2022-01-15 13:46:32 -05:00