Update flake and fix build with mostly async liblixstore #25
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "ma27/hydra:flake-update"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
cc @raito @pennae a review would be greatly appreciated (and as always, feel free to point out the weird stuff, I don't do too much C++).
The vast majority of changes was pretty straight-forward: we need an
AsyncIoRoot per thread to resolve promises we get from Lix. The
aio
object is created per thread and passed as lvalue ref instead of making
it a field of the State struct since that struct is shared between the
threads.
The biggest change affects the NAR extractor: parseAndCopyDump has been
removed, instead the
nar::parse
andnar::dump
parts are public now.This won't work with the original parse visitor anymore, so instead the
NAR is parsed once and dumped again to copy it to the binary cache. In
between, the generator is walked through to search for relevant files
(such as build products) and after that the generators are recreated for
AsyncGeneratorInputStream
to read again.haven't checked whether aio roots are indeed unique per thread, but kj has runtime checks for that
@ -397,6 +399,7 @@ void State::abortUnsupported()
bool stepFinished = false;
failStep(
aio,
weird indentation
@ -41,1 +38,3 @@
std::unordered_set<Path> filesToKeep {
static Generator<nar::Entry> discoverHydraDataInNAR(
NarMemberDatas &members,
const Path &&prefix,
rule of thumb: don't pass rvalue references into coroutines. in this case you can get away with it because generator semantics are in your favor, but in general it'll cause dangling references and the most undebuggable segfaults of that day. (just remove the
&&
in coroutine parameters and it'll be fine)@ -47,2 +50,2 @@
NarMemberDatas & members;
Path prefix;
nix::overloaded handlers {
[&](nar::File f) -> Generator<nar::Entry> {
none of these have to be generators of their own, they can return a
nar::Entry
alone@ -75,0 +94,4 @@
std::cref(e->first),
*discoverHydraDataInNAR(
members,
std::move(prefix),
that's only valid if directories never have more than one entry, otherwise all but the first item of a directory will see
prefix == ""
.(just don't move anything that doesn't absolutely need moved, it'll make your life a lot easier)
@ -96,0 +127,4 @@
auto items = nar::parse(source);
co_yield nar::dump(*discoverHydraDataInNAR(
members,
std::move(prefix),
this does not do what you except because
prefix
is a const-ref. instead it'll copyprefix
into a temporary and then pass the reference to that temporary todiscoverHydraDataInNAR
@ -96,0 +129,4 @@
members,
std::move(prefix),
"",
std::move(*items.next())
the move is unnecessary, and calling
next
on a nar parser only once is not safe. you must always drain the parser fully@ -96,0 +130,4 @@
std::move(prefix),
"",
std::move(*items.next())
).next());
same here because the wrapped parser is still a parser
93bf5c6175
to5dca3d8cd5
looks like it should be good
@ -49,0 +75,4 @@
members.insert_or_assign(
prefix + pathSuffix,
NarMemberData{.type = FSAccessor::Type::tSymlink});
return std::move(sl);
NRVO can skip the move altogether
5dca3d8cd5
to48168af261