2023-07-13 02:22:44 +00:00
|
|
|
|
#include "built-path.hh"
|
|
|
|
|
#include "derivations.hh"
|
|
|
|
|
#include "store-api.hh"
|
|
|
|
|
|
|
|
|
|
#include <nlohmann/json.hpp>
|
|
|
|
|
|
|
|
|
|
#include <optional>
|
|
|
|
|
|
|
|
|
|
namespace nix {
|
|
|
|
|
|
Make the Derived Path family of types inductive for dynamic derivations
We want to be able to write down `foo.drv^bar.drv^baz`:
`foo.drv^bar.drv` is the dynamic derivation (since it is itself a
derivation output, `bar.drv` from `foo.drv`).
To that end, we create `Single{Derivation,BuiltPath}` types, that are
very similar except instead of having multiple outputs (in a set or
map), they have a single one. This is for everything to the left of the
rightmost `^`.
`NixStringContextElem` has an analogous change, and now can reuse
`SingleDerivedPath` at the top level. In fact, if we ever get rid of
`DrvDeep`, `NixStringContextElem` could be replaced with
`SingleDerivedPath` entirely!
Important note: some JSON formats have changed.
We already can *produce* dynamic derivations, but we can't refer to them
directly. Today, we can merely express building or example at the top
imperatively over time by building `foo.drv^bar.drv`, and then with a
second nix invocation doing `<result-from-first>^baz`, but this is not
declarative. The ethos of Nix of being able to write down the full plan
everything you want to do, and then execute than plan with a single
command, and for that we need the new inductive form of these types.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-01-15 22:39:04 +00:00
|
|
|
|
#define CMP_ONE(CHILD_TYPE, MY_TYPE, FIELD, COMPARATOR) \
|
|
|
|
|
bool MY_TYPE ::operator COMPARATOR (const MY_TYPE & other) const \
|
|
|
|
|
{ \
|
|
|
|
|
const MY_TYPE* me = this; \
|
2024-03-04 06:32:31 +00:00
|
|
|
|
auto fields1 = std::tie(*me->drvPath, me->FIELD); \
|
Make the Derived Path family of types inductive for dynamic derivations
We want to be able to write down `foo.drv^bar.drv^baz`:
`foo.drv^bar.drv` is the dynamic derivation (since it is itself a
derivation output, `bar.drv` from `foo.drv`).
To that end, we create `Single{Derivation,BuiltPath}` types, that are
very similar except instead of having multiple outputs (in a set or
map), they have a single one. This is for everything to the left of the
rightmost `^`.
`NixStringContextElem` has an analogous change, and now can reuse
`SingleDerivedPath` at the top level. In fact, if we ever get rid of
`DrvDeep`, `NixStringContextElem` could be replaced with
`SingleDerivedPath` entirely!
Important note: some JSON formats have changed.
We already can *produce* dynamic derivations, but we can't refer to them
directly. Today, we can merely express building or example at the top
imperatively over time by building `foo.drv^bar.drv`, and then with a
second nix invocation doing `<result-from-first>^baz`, but this is not
declarative. The ethos of Nix of being able to write down the full plan
everything you want to do, and then execute than plan with a single
command, and for that we need the new inductive form of these types.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-01-15 22:39:04 +00:00
|
|
|
|
me = &other; \
|
2024-03-04 06:32:31 +00:00
|
|
|
|
auto fields2 = std::tie(*me->drvPath, me->FIELD); \
|
Make the Derived Path family of types inductive for dynamic derivations
We want to be able to write down `foo.drv^bar.drv^baz`:
`foo.drv^bar.drv` is the dynamic derivation (since it is itself a
derivation output, `bar.drv` from `foo.drv`).
To that end, we create `Single{Derivation,BuiltPath}` types, that are
very similar except instead of having multiple outputs (in a set or
map), they have a single one. This is for everything to the left of the
rightmost `^`.
`NixStringContextElem` has an analogous change, and now can reuse
`SingleDerivedPath` at the top level. In fact, if we ever get rid of
`DrvDeep`, `NixStringContextElem` could be replaced with
`SingleDerivedPath` entirely!
Important note: some JSON formats have changed.
We already can *produce* dynamic derivations, but we can't refer to them
directly. Today, we can merely express building or example at the top
imperatively over time by building `foo.drv^bar.drv`, and then with a
second nix invocation doing `<result-from-first>^baz`, but this is not
declarative. The ethos of Nix of being able to write down the full plan
everything you want to do, and then execute than plan with a single
command, and for that we need the new inductive form of these types.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-01-15 22:39:04 +00:00
|
|
|
|
return fields1 COMPARATOR fields2; \
|
2023-07-13 02:22:44 +00:00
|
|
|
|
}
|
Make the Derived Path family of types inductive for dynamic derivations
We want to be able to write down `foo.drv^bar.drv^baz`:
`foo.drv^bar.drv` is the dynamic derivation (since it is itself a
derivation output, `bar.drv` from `foo.drv`).
To that end, we create `Single{Derivation,BuiltPath}` types, that are
very similar except instead of having multiple outputs (in a set or
map), they have a single one. This is for everything to the left of the
rightmost `^`.
`NixStringContextElem` has an analogous change, and now can reuse
`SingleDerivedPath` at the top level. In fact, if we ever get rid of
`DrvDeep`, `NixStringContextElem` could be replaced with
`SingleDerivedPath` entirely!
Important note: some JSON formats have changed.
We already can *produce* dynamic derivations, but we can't refer to them
directly. Today, we can merely express building or example at the top
imperatively over time by building `foo.drv^bar.drv`, and then with a
second nix invocation doing `<result-from-first>^baz`, but this is not
declarative. The ethos of Nix of being able to write down the full plan
everything you want to do, and then execute than plan with a single
command, and for that we need the new inductive form of these types.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-01-15 22:39:04 +00:00
|
|
|
|
#define CMP(CHILD_TYPE, MY_TYPE, FIELD) \
|
|
|
|
|
CMP_ONE(CHILD_TYPE, MY_TYPE, FIELD, ==) \
|
|
|
|
|
CMP_ONE(CHILD_TYPE, MY_TYPE, FIELD, !=) \
|
|
|
|
|
CMP_ONE(CHILD_TYPE, MY_TYPE, FIELD, <)
|
|
|
|
|
|
|
|
|
|
#define FIELD_TYPE std::pair<std::string, StorePath>
|
|
|
|
|
CMP(SingleBuiltPath, SingleBuiltPathBuilt, output)
|
|
|
|
|
#undef FIELD_TYPE
|
|
|
|
|
|
|
|
|
|
#define FIELD_TYPE std::map<std::string, StorePath>
|
|
|
|
|
CMP(SingleBuiltPath, BuiltPathBuilt, outputs)
|
|
|
|
|
#undef FIELD_TYPE
|
|
|
|
|
|
|
|
|
|
#undef CMP
|
|
|
|
|
#undef CMP_ONE
|
|
|
|
|
|
|
|
|
|
StorePath SingleBuiltPath::outPath() const
|
|
|
|
|
{
|
|
|
|
|
return std::visit(
|
|
|
|
|
overloaded{
|
|
|
|
|
[](const SingleBuiltPath::Opaque & p) { return p.path; },
|
|
|
|
|
[](const SingleBuiltPath::Built & b) { return b.output.second; },
|
|
|
|
|
}, raw()
|
|
|
|
|
);
|
2023-07-13 02:22:44 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
StorePathSet BuiltPath::outPaths() const
|
|
|
|
|
{
|
|
|
|
|
return std::visit(
|
|
|
|
|
overloaded{
|
|
|
|
|
[](const BuiltPath::Opaque & p) { return StorePathSet{p.path}; },
|
|
|
|
|
[](const BuiltPath::Built & b) {
|
|
|
|
|
StorePathSet res;
|
|
|
|
|
for (auto & [_, path] : b.outputs)
|
|
|
|
|
res.insert(path);
|
|
|
|
|
return res;
|
|
|
|
|
},
|
|
|
|
|
}, raw()
|
|
|
|
|
);
|
|
|
|
|
}
|
|
|
|
|
|
Allow dynamic derivation deps in `inputDrvs`
We use the same nested map representation we used for goals, again in
order to save space. We might someday want to combine with `inputDrvs`,
by doing `V = bool` instead of `V = std::set<OutputName>`, but we are
not doing that yet for sake of a smaller diff.
The ATerm format for Derivations also needs to be extended, in addition
to the in-memory format. To accomodate this, we added a new basic
versioning scheme, so old versions of Nix will get nice errors. (And
going forward, if the ATerm format changes again the errors will be even
better.)
`parsedStrings`, an internal function used as part of parsing
derivations in A-Term format, used to consume the final `]` but expect
the initial `[` to already be consumed. This made for what looked like
unbalanced brackets at callsites, which was confusing. Now it consumes
both which is hopefully less confusing.
As part of testing, we also created a unit test for the A-Term format for
regular non-experimental derivations too.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2021-10-01 22:05:53 +00:00
|
|
|
|
SingleDerivedPath::Built SingleBuiltPath::Built::discardOutputPath() const
|
|
|
|
|
{
|
|
|
|
|
return SingleDerivedPath::Built {
|
|
|
|
|
.drvPath = make_ref<SingleDerivedPath>(drvPath->discardOutputPath()),
|
|
|
|
|
.output = output.first,
|
|
|
|
|
};
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
SingleDerivedPath SingleBuiltPath::discardOutputPath() const
|
|
|
|
|
{
|
|
|
|
|
return std::visit(
|
|
|
|
|
overloaded{
|
|
|
|
|
[](const SingleBuiltPath::Opaque & p) -> SingleDerivedPath {
|
|
|
|
|
return p;
|
|
|
|
|
},
|
|
|
|
|
[](const SingleBuiltPath::Built & b) -> SingleDerivedPath {
|
|
|
|
|
return b.discardOutputPath();
|
|
|
|
|
},
|
|
|
|
|
}, raw()
|
|
|
|
|
);
|
|
|
|
|
}
|
|
|
|
|
|
Make the Derived Path family of types inductive for dynamic derivations
We want to be able to write down `foo.drv^bar.drv^baz`:
`foo.drv^bar.drv` is the dynamic derivation (since it is itself a
derivation output, `bar.drv` from `foo.drv`).
To that end, we create `Single{Derivation,BuiltPath}` types, that are
very similar except instead of having multiple outputs (in a set or
map), they have a single one. This is for everything to the left of the
rightmost `^`.
`NixStringContextElem` has an analogous change, and now can reuse
`SingleDerivedPath` at the top level. In fact, if we ever get rid of
`DrvDeep`, `NixStringContextElem` could be replaced with
`SingleDerivedPath` entirely!
Important note: some JSON formats have changed.
We already can *produce* dynamic derivations, but we can't refer to them
directly. Today, we can merely express building or example at the top
imperatively over time by building `foo.drv^bar.drv`, and then with a
second nix invocation doing `<result-from-first>^baz`, but this is not
declarative. The ethos of Nix of being able to write down the full plan
everything you want to do, and then execute than plan with a single
command, and for that we need the new inductive form of these types.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-01-15 22:39:04 +00:00
|
|
|
|
nlohmann::json BuiltPath::Built::toJSON(const Store & store) const
|
|
|
|
|
{
|
|
|
|
|
nlohmann::json res;
|
|
|
|
|
res["drvPath"] = drvPath->toJSON(store);
|
|
|
|
|
for (const auto & [outputName, outputPath] : outputs) {
|
|
|
|
|
res["outputs"][outputName] = store.printStorePath(outputPath);
|
|
|
|
|
}
|
|
|
|
|
return res;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
nlohmann::json SingleBuiltPath::Built::toJSON(const Store & store) const
|
|
|
|
|
{
|
|
|
|
|
nlohmann::json res;
|
|
|
|
|
res["drvPath"] = drvPath->toJSON(store);
|
|
|
|
|
auto & [outputName, outputPath] = output;
|
|
|
|
|
res["output"] = outputName;
|
|
|
|
|
res["outputPath"] = store.printStorePath(outputPath);
|
|
|
|
|
return res;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
nlohmann::json SingleBuiltPath::toJSON(const Store & store) const
|
|
|
|
|
{
|
|
|
|
|
return std::visit([&](const auto & buildable) {
|
|
|
|
|
return buildable.toJSON(store);
|
|
|
|
|
}, raw());
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
nlohmann::json BuiltPath::toJSON(const Store & store) const
|
|
|
|
|
{
|
|
|
|
|
return std::visit([&](const auto & buildable) {
|
|
|
|
|
return buildable.toJSON(store);
|
|
|
|
|
}, raw());
|
|
|
|
|
}
|
|
|
|
|
|
2023-07-13 02:22:44 +00:00
|
|
|
|
RealisedPath::Set BuiltPath::toRealisedPaths(Store & store) const
|
|
|
|
|
{
|
|
|
|
|
RealisedPath::Set res;
|
|
|
|
|
std::visit(
|
|
|
|
|
overloaded{
|
|
|
|
|
[&](const BuiltPath::Opaque & p) { res.insert(p.path); },
|
|
|
|
|
[&](const BuiltPath::Built & p) {
|
|
|
|
|
auto drvHashes =
|
Make the Derived Path family of types inductive for dynamic derivations
We want to be able to write down `foo.drv^bar.drv^baz`:
`foo.drv^bar.drv` is the dynamic derivation (since it is itself a
derivation output, `bar.drv` from `foo.drv`).
To that end, we create `Single{Derivation,BuiltPath}` types, that are
very similar except instead of having multiple outputs (in a set or
map), they have a single one. This is for everything to the left of the
rightmost `^`.
`NixStringContextElem` has an analogous change, and now can reuse
`SingleDerivedPath` at the top level. In fact, if we ever get rid of
`DrvDeep`, `NixStringContextElem` could be replaced with
`SingleDerivedPath` entirely!
Important note: some JSON formats have changed.
We already can *produce* dynamic derivations, but we can't refer to them
directly. Today, we can merely express building or example at the top
imperatively over time by building `foo.drv^bar.drv`, and then with a
second nix invocation doing `<result-from-first>^baz`, but this is not
declarative. The ethos of Nix of being able to write down the full plan
everything you want to do, and then execute than plan with a single
command, and for that we need the new inductive form of these types.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-01-15 22:39:04 +00:00
|
|
|
|
staticOutputHashes(store, store.readDerivation(p.drvPath->outPath()));
|
2023-07-13 02:22:44 +00:00
|
|
|
|
for (auto& [outputName, outputPath] : p.outputs) {
|
|
|
|
|
if (experimentalFeatureSettings.isEnabled(
|
|
|
|
|
Xp::CaDerivations)) {
|
|
|
|
|
auto drvOutput = get(drvHashes, outputName);
|
|
|
|
|
if (!drvOutput)
|
|
|
|
|
throw Error(
|
|
|
|
|
"the derivation '%s' has unrealised output '%s' (derived-path.cc/toRealisedPaths)",
|
Make the Derived Path family of types inductive for dynamic derivations
We want to be able to write down `foo.drv^bar.drv^baz`:
`foo.drv^bar.drv` is the dynamic derivation (since it is itself a
derivation output, `bar.drv` from `foo.drv`).
To that end, we create `Single{Derivation,BuiltPath}` types, that are
very similar except instead of having multiple outputs (in a set or
map), they have a single one. This is for everything to the left of the
rightmost `^`.
`NixStringContextElem` has an analogous change, and now can reuse
`SingleDerivedPath` at the top level. In fact, if we ever get rid of
`DrvDeep`, `NixStringContextElem` could be replaced with
`SingleDerivedPath` entirely!
Important note: some JSON formats have changed.
We already can *produce* dynamic derivations, but we can't refer to them
directly. Today, we can merely express building or example at the top
imperatively over time by building `foo.drv^bar.drv`, and then with a
second nix invocation doing `<result-from-first>^baz`, but this is not
declarative. The ethos of Nix of being able to write down the full plan
everything you want to do, and then execute than plan with a single
command, and for that we need the new inductive form of these types.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-01-15 22:39:04 +00:00
|
|
|
|
store.printStorePath(p.drvPath->outPath()), outputName);
|
2023-07-13 02:22:44 +00:00
|
|
|
|
auto thisRealisation = store.queryRealisation(
|
|
|
|
|
DrvOutput{*drvOutput, outputName});
|
|
|
|
|
assert(thisRealisation); // We’ve built it, so we must
|
|
|
|
|
// have the realisation
|
|
|
|
|
res.insert(*thisRealisation);
|
|
|
|
|
} else {
|
|
|
|
|
res.insert(outputPath);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
},
|
|
|
|
|
},
|
|
|
|
|
raw());
|
|
|
|
|
return res;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
}
|