2020-06-29 12:14:23 +00:00
|
|
|
#include "installables.hh"
|
2023-02-03 16:21:47 +00:00
|
|
|
#include "installable-derived-path.hh"
|
2023-02-05 17:16:17 +00:00
|
|
|
#include "installable-value.hh"
|
2020-06-29 12:14:23 +00:00
|
|
|
#include "store-api.hh"
|
|
|
|
#include "eval-inline.hh"
|
2020-06-29 17:08:50 +00:00
|
|
|
#include "eval-cache.hh"
|
|
|
|
#include "names.hh"
|
2021-05-17 07:48:51 +00:00
|
|
|
#include "command.hh"
|
2022-03-17 22:29:15 +00:00
|
|
|
#include "derivations.hh"
|
2023-05-11 22:01:41 +00:00
|
|
|
#include "downstream-placeholder.hh"
|
2020-06-29 12:14:23 +00:00
|
|
|
|
|
|
|
namespace nix {
|
|
|
|
|
2021-05-17 07:48:51 +00:00
|
|
|
/**
|
|
|
|
* Return the rewrites that are needed to resolve a string whose context is
|
2022-04-19 18:48:13 +00:00
|
|
|
* included in `dependencies`.
|
2021-05-17 07:48:51 +00:00
|
|
|
*/
|
2022-11-21 09:49:01 +00:00
|
|
|
StringPairs resolveRewrites(
|
|
|
|
Store & store,
|
|
|
|
const std::vector<BuiltPathWithResult> & dependencies)
|
2021-05-17 07:48:51 +00:00
|
|
|
{
|
|
|
|
StringPairs res;
|
Allow dynamic derivation deps in `inputDrvs`
We use the same nested map representation we used for goals, again in
order to save space. We might someday want to combine with `inputDrvs`,
by doing `V = bool` instead of `V = std::set<OutputName>`, but we are
not doing that yet for sake of a smaller diff.
The ATerm format for Derivations also needs to be extended, in addition
to the in-memory format. To accomodate this, we added a new basic
versioning scheme, so old versions of Nix will get nice errors. (And
going forward, if the ATerm format changes again the errors will be even
better.)
`parsedStrings`, an internal function used as part of parsing
derivations in A-Term format, used to consume the final `]` but expect
the initial `[` to already be consumed. This made for what looked like
unbalanced brackets at callsites, which was confusing. Now it consumes
both which is hopefully less confusing.
As part of testing, we also created a unit test for the A-Term format for
regular non-experimental derivations too.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2021-10-01 22:05:53 +00:00
|
|
|
if (experimentalFeatureSettings.isEnabled(Xp::CaDerivations)) {
|
|
|
|
for (auto & dep : dependencies) {
|
|
|
|
if (auto drvDep = std::get_if<BuiltPathBuilt>(&dep.path)) {
|
|
|
|
for (auto & [ outputName, outputPath ] : drvDep->outputs) {
|
2023-07-13 03:33:43 +00:00
|
|
|
res.emplace(
|
Allow dynamic derivation deps in `inputDrvs`
We use the same nested map representation we used for goals, again in
order to save space. We might someday want to combine with `inputDrvs`,
by doing `V = bool` instead of `V = std::set<OutputName>`, but we are
not doing that yet for sake of a smaller diff.
The ATerm format for Derivations also needs to be extended, in addition
to the in-memory format. To accomodate this, we added a new basic
versioning scheme, so old versions of Nix will get nice errors. (And
going forward, if the ATerm format changes again the errors will be even
better.)
`parsedStrings`, an internal function used as part of parsing
derivations in A-Term format, used to consume the final `]` but expect
the initial `[` to already be consumed. This made for what looked like
unbalanced brackets at callsites, which was confusing. Now it consumes
both which is hopefully less confusing.
As part of testing, we also created a unit test for the A-Term format for
regular non-experimental derivations too.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2021-10-01 22:05:53 +00:00
|
|
|
DownstreamPlaceholder::fromSingleDerivedPathBuilt(
|
|
|
|
SingleDerivedPath::Built {
|
|
|
|
.drvPath = make_ref<SingleDerivedPath>(drvDep->drvPath->discardOutputPath()),
|
|
|
|
.output = outputName,
|
|
|
|
}).render(),
|
2023-07-13 03:33:43 +00:00
|
|
|
store.printStorePath(outputPath)
|
|
|
|
);
|
Allow dynamic derivation deps in `inputDrvs`
We use the same nested map representation we used for goals, again in
order to save space. We might someday want to combine with `inputDrvs`,
by doing `V = bool` instead of `V = std::set<OutputName>`, but we are
not doing that yet for sake of a smaller diff.
The ATerm format for Derivations also needs to be extended, in addition
to the in-memory format. To accomodate this, we added a new basic
versioning scheme, so old versions of Nix will get nice errors. (And
going forward, if the ATerm format changes again the errors will be even
better.)
`parsedStrings`, an internal function used as part of parsing
derivations in A-Term format, used to consume the final `]` but expect
the initial `[` to already be consumed. This made for what looked like
unbalanced brackets at callsites, which was confusing. Now it consumes
both which is hopefully less confusing.
As part of testing, we also created a unit test for the A-Term format for
regular non-experimental derivations too.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2021-10-01 22:05:53 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
2021-05-17 07:48:51 +00:00
|
|
|
return res;
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
2022-04-19 18:48:13 +00:00
|
|
|
* Resolve the given string assuming the given context.
|
2021-05-17 07:48:51 +00:00
|
|
|
*/
|
2022-11-21 09:49:01 +00:00
|
|
|
std::string resolveString(
|
|
|
|
Store & store,
|
|
|
|
const std::string & toResolve,
|
|
|
|
const std::vector<BuiltPathWithResult> & dependencies)
|
2021-05-17 07:48:51 +00:00
|
|
|
{
|
|
|
|
auto rewrites = resolveRewrites(store, dependencies);
|
|
|
|
return rewriteStrings(toResolve, rewrites);
|
|
|
|
}
|
|
|
|
|
2023-02-05 17:16:17 +00:00
|
|
|
UnresolvedApp InstallableValue::toApp(EvalState & state)
|
2020-06-29 12:14:23 +00:00
|
|
|
{
|
2022-04-14 12:04:19 +00:00
|
|
|
auto cursor = getCursor(state);
|
2022-04-19 10:09:12 +00:00
|
|
|
auto attrPath = cursor->getAttrPath();
|
2020-06-29 12:14:23 +00:00
|
|
|
|
2020-06-29 17:08:50 +00:00
|
|
|
auto type = cursor->getAttr("type")->getString();
|
2020-06-29 12:14:23 +00:00
|
|
|
|
2022-10-26 08:05:27 +00:00
|
|
|
std::string expected = !attrPath.empty() &&
|
|
|
|
(state.symbols[attrPath[0]] == "apps" || state.symbols[attrPath[0]] == "defaultApp")
|
|
|
|
? "app" : "derivation";
|
2022-04-19 10:09:12 +00:00
|
|
|
if (type != expected)
|
|
|
|
throw Error("attribute '%s' should have type '%s'", cursor->getAttrPathStr(), expected);
|
|
|
|
|
2021-05-17 15:49:20 +00:00
|
|
|
if (type == "app") {
|
|
|
|
auto [program, context] = cursor->getAttr("program")->getStringWithContext();
|
2021-02-17 16:54:13 +00:00
|
|
|
|
2023-01-10 16:27:19 +00:00
|
|
|
std::vector<DerivedPath> context2;
|
2023-01-03 16:44:59 +00:00
|
|
|
for (auto & c : context) {
|
|
|
|
context2.emplace_back(std::visit(overloaded {
|
|
|
|
[&](const NixStringContextElem::DrvDeep & d) -> DerivedPath {
|
|
|
|
/* We want all outputs of the drv */
|
|
|
|
return DerivedPath::Built {
|
Make the Derived Path family of types inductive for dynamic derivations
We want to be able to write down `foo.drv^bar.drv^baz`:
`foo.drv^bar.drv` is the dynamic derivation (since it is itself a
derivation output, `bar.drv` from `foo.drv`).
To that end, we create `Single{Derivation,BuiltPath}` types, that are
very similar except instead of having multiple outputs (in a set or
map), they have a single one. This is for everything to the left of the
rightmost `^`.
`NixStringContextElem` has an analogous change, and now can reuse
`SingleDerivedPath` at the top level. In fact, if we ever get rid of
`DrvDeep`, `NixStringContextElem` could be replaced with
`SingleDerivedPath` entirely!
Important note: some JSON formats have changed.
We already can *produce* dynamic derivations, but we can't refer to them
directly. Today, we can merely express building or example at the top
imperatively over time by building `foo.drv^bar.drv`, and then with a
second nix invocation doing `<result-from-first>^baz`, but this is not
declarative. The ethos of Nix of being able to write down the full plan
everything you want to do, and then execute than plan with a single
command, and for that we need the new inductive form of these types.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-01-15 22:39:04 +00:00
|
|
|
.drvPath = makeConstantStorePathRef(d.drvPath),
|
2023-01-11 23:57:18 +00:00
|
|
|
.outputs = OutputsSpec::All {},
|
2023-01-03 16:44:59 +00:00
|
|
|
};
|
|
|
|
},
|
|
|
|
[&](const NixStringContextElem::Built & b) -> DerivedPath {
|
|
|
|
return DerivedPath::Built {
|
|
|
|
.drvPath = b.drvPath,
|
2023-01-11 23:57:18 +00:00
|
|
|
.outputs = OutputsSpec::Names { b.output },
|
2023-01-03 16:44:59 +00:00
|
|
|
};
|
|
|
|
},
|
|
|
|
[&](const NixStringContextElem::Opaque & o) -> DerivedPath {
|
|
|
|
return DerivedPath::Opaque {
|
|
|
|
.path = o.path,
|
|
|
|
};
|
|
|
|
},
|
2023-08-16 16:29:23 +00:00
|
|
|
}, c.raw));
|
2023-01-10 16:27:19 +00:00
|
|
|
}
|
2020-06-29 12:14:23 +00:00
|
|
|
|
2021-05-17 15:49:20 +00:00
|
|
|
return UnresolvedApp{App {
|
|
|
|
.context = std::move(context2),
|
|
|
|
.program = program,
|
|
|
|
}};
|
2020-06-29 17:08:50 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
else if (type == "derivation") {
|
|
|
|
auto drvPath = cursor->forceDerivation();
|
2022-04-26 12:01:21 +00:00
|
|
|
auto outPath = cursor->getAttr(state.sOutPath)->getString();
|
|
|
|
auto outputName = cursor->getAttr(state.sOutputName)->getString();
|
|
|
|
auto name = cursor->getAttr(state.sName)->getString();
|
2021-10-27 18:18:30 +00:00
|
|
|
auto aPname = cursor->maybeGetAttr("pname");
|
2022-04-20 14:39:47 +00:00
|
|
|
auto aMeta = cursor->maybeGetAttr(state.sMeta);
|
2021-02-17 16:54:13 +00:00
|
|
|
auto aMainProgram = aMeta ? aMeta->maybeGetAttr("mainProgram") : nullptr;
|
|
|
|
auto mainProgram =
|
|
|
|
aMainProgram
|
|
|
|
? aMainProgram->getString()
|
2021-10-27 18:18:30 +00:00
|
|
|
: aPname
|
|
|
|
? aPname->getString()
|
2021-02-17 16:54:13 +00:00
|
|
|
: DrvName(name).name;
|
2021-05-17 15:49:20 +00:00
|
|
|
auto program = outPath + "/bin/" + mainProgram;
|
|
|
|
return UnresolvedApp { App {
|
2023-01-10 16:27:19 +00:00
|
|
|
.context = { DerivedPath::Built {
|
Make the Derived Path family of types inductive for dynamic derivations
We want to be able to write down `foo.drv^bar.drv^baz`:
`foo.drv^bar.drv` is the dynamic derivation (since it is itself a
derivation output, `bar.drv` from `foo.drv`).
To that end, we create `Single{Derivation,BuiltPath}` types, that are
very similar except instead of having multiple outputs (in a set or
map), they have a single one. This is for everything to the left of the
rightmost `^`.
`NixStringContextElem` has an analogous change, and now can reuse
`SingleDerivedPath` at the top level. In fact, if we ever get rid of
`DrvDeep`, `NixStringContextElem` could be replaced with
`SingleDerivedPath` entirely!
Important note: some JSON formats have changed.
We already can *produce* dynamic derivations, but we can't refer to them
directly. Today, we can merely express building or example at the top
imperatively over time by building `foo.drv^bar.drv`, and then with a
second nix invocation doing `<result-from-first>^baz`, but this is not
declarative. The ethos of Nix of being able to write down the full plan
everything you want to do, and then execute than plan with a single
command, and for that we need the new inductive form of these types.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-01-15 22:39:04 +00:00
|
|
|
.drvPath = makeConstantStorePathRef(drvPath),
|
2023-01-11 23:57:18 +00:00
|
|
|
.outputs = OutputsSpec::Names { outputName },
|
2023-01-10 16:27:19 +00:00
|
|
|
} },
|
2021-05-17 15:49:20 +00:00
|
|
|
.program = program,
|
|
|
|
}};
|
2020-06-29 17:08:50 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
else
|
2022-04-14 12:04:19 +00:00
|
|
|
throw Error("attribute '%s' has unsupported type '%s'", cursor->getAttrPathStr(), type);
|
2021-05-17 15:49:20 +00:00
|
|
|
}
|
|
|
|
|
2021-07-16 14:04:47 +00:00
|
|
|
// FIXME: move to libcmd
|
|
|
|
App UnresolvedApp::resolve(ref<Store> evalStore, ref<Store> store)
|
2021-05-17 15:49:20 +00:00
|
|
|
{
|
|
|
|
auto res = unresolved;
|
|
|
|
|
Make command infra less stateful and more regular
Already, we had classes like `BuiltPathsCommand` and `StorePathsCommand`
which provided alternative `run` virtual functions providing the
implementation with more arguments. This was a very nice and easy way to
make writing command; just fill in the virtual functions and it is
fairly clear what to do.
However, exception to this pattern were `Installable{,s}Command`. These
two classes instead just had a field where the installables would be
stored, and various side-effecting `prepare` and `load` machinery too
fill them in. Command would wish out those fields.
This isn't so clear to use.
What this commit does is make those command classes like the others,
with richer `run` functions.
Not only does this restore the pattern making commands easier to write,
it has a number of other benefits:
- `prepare` and `load` are gone entirely! One command just hands just
hands off to the next.
- `useDefaultInstallables` because `defaultInstallables`. This takes
over `prepare` for the one case that needs it, and provides enough
flexiblity to handle `nix repl`'s idiosyncratic migration.
- We can use `ref` instead of `std::shared_ptr`. The former must be
initialized (so it is like Rust's `Box` rather than `Option<Box>`,
This expresses the invariant that the installable are in fact
initialized much better.
This is possible because since we just have local variables not
fields, we can stop worrying about the not-yet-initialized case.
- Fewer lines of code! (Finally I have a large refactor that makes the
number go down not up...)
- `nix repl` is now implemented in a clearer way.
The last item deserves further mention. `nix repl` is not like the other
installable commands because instead working from once-loaded
installables, it needs to be able to load them again and again.
To properly support this, we make a new superclass
`RawInstallablesCommand`. This class has the argument parsing and
completion logic, but does *not* hand off parsed installables but
instead just the raw string arguments.
This is exactly what `nix repl` needs, and allows us to instead of
having the logic awkwardly split between `prepare`,
`useDefaultInstallables,` and `load`, have everything right next to each
other. I think this will enable future simplifications of that argument
defaulting logic, but I am saving those for a future PR --- best to keep
code motion and more complicated boolean expression rewriting separate
steps.
The "diagnostic ignored `-Woverloaded-virtual`" pragma helps because C++
doesn't like our many `run` methods. In our case, we don't mind the
shadowing it all --- it is *intentional* that the derived class only
provides a `run` method, and doesn't call any of the overridden `run`
methods.
Helps with https://github.com/NixOS/rfcs/pull/134
2023-02-04 17:03:47 +00:00
|
|
|
Installables installableContext;
|
2021-05-17 15:49:20 +00:00
|
|
|
|
|
|
|
for (auto & ctxElt : unresolved.context)
|
|
|
|
installableContext.push_back(
|
Make command infra less stateful and more regular
Already, we had classes like `BuiltPathsCommand` and `StorePathsCommand`
which provided alternative `run` virtual functions providing the
implementation with more arguments. This was a very nice and easy way to
make writing command; just fill in the virtual functions and it is
fairly clear what to do.
However, exception to this pattern were `Installable{,s}Command`. These
two classes instead just had a field where the installables would be
stored, and various side-effecting `prepare` and `load` machinery too
fill them in. Command would wish out those fields.
This isn't so clear to use.
What this commit does is make those command classes like the others,
with richer `run` functions.
Not only does this restore the pattern making commands easier to write,
it has a number of other benefits:
- `prepare` and `load` are gone entirely! One command just hands just
hands off to the next.
- `useDefaultInstallables` because `defaultInstallables`. This takes
over `prepare` for the one case that needs it, and provides enough
flexiblity to handle `nix repl`'s idiosyncratic migration.
- We can use `ref` instead of `std::shared_ptr`. The former must be
initialized (so it is like Rust's `Box` rather than `Option<Box>`,
This expresses the invariant that the installable are in fact
initialized much better.
This is possible because since we just have local variables not
fields, we can stop worrying about the not-yet-initialized case.
- Fewer lines of code! (Finally I have a large refactor that makes the
number go down not up...)
- `nix repl` is now implemented in a clearer way.
The last item deserves further mention. `nix repl` is not like the other
installable commands because instead working from once-loaded
installables, it needs to be able to load them again and again.
To properly support this, we make a new superclass
`RawInstallablesCommand`. This class has the argument parsing and
completion logic, but does *not* hand off parsed installables but
instead just the raw string arguments.
This is exactly what `nix repl` needs, and allows us to instead of
having the logic awkwardly split between `prepare`,
`useDefaultInstallables,` and `load`, have everything right next to each
other. I think this will enable future simplifications of that argument
defaulting logic, but I am saving those for a future PR --- best to keep
code motion and more complicated boolean expression rewriting separate
steps.
The "diagnostic ignored `-Woverloaded-virtual`" pragma helps because C++
doesn't like our many `run` methods. In our case, we don't mind the
shadowing it all --- it is *intentional* that the derived class only
provides a `run` method, and doesn't call any of the overridden `run`
methods.
Helps with https://github.com/NixOS/rfcs/pull/134
2023-02-04 17:03:47 +00:00
|
|
|
make_ref<InstallableDerivedPath>(store, DerivedPath { ctxElt }));
|
2021-05-17 07:48:51 +00:00
|
|
|
|
2022-03-02 12:54:08 +00:00
|
|
|
auto builtContext = Installable::build(evalStore, store, Realise::Outputs, installableContext);
|
2021-05-17 15:49:20 +00:00
|
|
|
res.program = resolveString(*store, unresolved.program, builtContext);
|
2021-05-18 11:54:05 +00:00
|
|
|
if (!store->isInStore(res.program))
|
2021-05-17 15:49:20 +00:00
|
|
|
throw Error("app program '%s' is not in the Nix store", res.program);
|
|
|
|
|
|
|
|
return res;
|
2020-06-29 12:14:23 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
}
|