lix/src/nix-env/user-env.cc

172 lines
6 KiB
C++
Raw Normal View History

2013-11-22 15:41:48 +00:00
#include "user-env.hh"
#include "util.hh"
#include "derivations.hh"
#include "store-api.hh"
#include "path-with-outputs.hh"
#include "local-fs-store.hh"
#include "globals.hh"
#include "shared.hh"
#include "eval.hh"
2013-11-19 13:09:03 +00:00
#include "eval-inline.hh"
#include "profiles.hh"
Unify and refactor value printing Previously, there were two mostly-identical value printers -- one in `libexpr/eval.cc` (which didn't force values) and one in `libcmd/repl.cc` (which did force values and also printed ANSI color codes). This PR unifies both of these printers into `print.cc` and provides a `PrintOptions` struct for controlling the output, which allows for toggling whether values are forced, whether repeated values are tracked, and whether ANSI color codes are displayed. Additionally, `PrintOptions` allows tuning the maximum number of attributes, list items, and bytes in a string that will be displayed; this makes it ideal for contexts where printing too much output (e.g. all of Nixpkgs) is distracting. (As requested by @roberth in https://github.com/NixOS/nix/pull/9554#issuecomment-1845095735) Please read the tests for example output. Future work: - It would be nice to provide this function as a builtin, perhaps `builtins.toStringDebug` -- a printing function that never fails would be useful when debugging Nix code. - It would be nice to support customizing `PrintOptions` members on the command line, e.g. `--option to-string-max-attrs 1000`. (cherry picked from commit 0fa08b451682fb3311fe58112ff05c4fe5bee3a4, ) === Restore ambiguous value printer for `nix-instantiate` The Nix team has requested that this output format remain unchanged. I've added a warning to the man page explaining that `nix-instantiate --eval` output will not parse correctly in many situations. (cherry picked from commit df84dd4d8dd3fd6381ac2ca3064432ab31a16b79) Change-Id: I7cca6b4b53cd0642f2d49af657d5676a8554c9f8
2024-03-08 02:05:47 +00:00
#include "print-ambiguous.hh"
#include <limits>
namespace nix {
DrvInfos queryInstalled(EvalState & state, const Path & userEnv)
{
DrvInfos elems;
if (pathExists(userEnv + "/manifest.json"))
throw Error("profile '%s' is incompatible with 'nix-env'; please use 'nix profile' instead", userEnv);
auto manifestFile = userEnv + "/manifest.nix";
if (pathExists(manifestFile)) {
Value v;
state.evalFile(state.rootPath(CanonPath(manifestFile)), v);
Bindings & bindings(*state.allocBindings(0));
getDerivations(state, v, "", bindings, elems, false);
}
return elems;
}
bool createUserEnv(EvalState & state, DrvInfos & elems,
const Path & profile, bool keepDerivations,
const std::string & lockToken)
{
/* Build the components in the user environment, if they don't
exist already. */
std::vector<StorePathWithOutputs> drvsToBuild;
2015-07-17 17:24:28 +00:00
for (auto & i : elems)
if (auto drvPath = i.queryDrvPath())
drvsToBuild.push_back({*drvPath});
debug("building user environment dependencies");
state.store->buildPaths(
2021-04-05 13:48:18 +00:00
toDerivedPaths(drvsToBuild),
state.repair ? bmRepair : bmNormal);
/* Construct the whole top level derivation. */
StorePathSet references;
Value manifest;
state.mkList(manifest, elems.size());
size_t n = 0;
2015-07-17 17:24:28 +00:00
for (auto & i : elems) {
/* Create a pseudo-derivation containing the name, system,
output paths, and optionally the derivation path, as well
as the meta attributes. */
std::optional<StorePath> drvPath = keepDerivations ? i.queryDrvPath() : std::nullopt;
DrvInfo::Outputs outputs = i.queryOutputs(true, true);
StringSet metaNames = i.queryMetaNames();
auto attrs = state.buildBindings(7 + outputs.size());
attrs.alloc(state.sType).mkString("derivation");
attrs.alloc(state.sName).mkString(i.queryName());
2017-07-17 17:02:56 +00:00
auto system = i.querySystem();
if (!system.empty())
attrs.alloc(state.sSystem).mkString(system);
attrs.alloc(state.sOutPath).mkString(state.store->printStorePath(i.queryOutPath()));
if (drvPath)
attrs.alloc(state.sDrvPath).mkString(state.store->printStorePath(*drvPath));
// Copy each output meant for installation.
auto & vOutputs = attrs.alloc(state.sOutputs);
state.mkList(vOutputs, outputs.size());
for (const auto & [m, j] : enumerate(outputs)) {
(vOutputs.listElems()[m] = state.allocValue())->mkString(j.first);
auto outputAttrs = state.buildBindings(2);
outputAttrs.alloc(state.sOutPath).mkString(state.store->printStorePath(*j.second));
attrs.alloc(j.first).mkAttrs(outputAttrs);
/* This is only necessary when installing store paths, e.g.,
`nix-env -i /nix/store/abcd...-foo'. */
state.store->addTempRoot(*j.second);
state.store->ensurePath(*j.second);
references.insert(*j.second);
}
// Copy the meta attributes.
auto meta = state.buildBindings(metaNames.size());
2015-07-17 17:24:28 +00:00
for (auto & j : metaNames) {
Value * v = i.queryMeta(j);
if (!v) continue;
meta.insert(state.symbols.create(j), v);
}
attrs.alloc(state.sMeta).mkAttrs(meta);
(manifest.listElems()[n++] = state.allocValue())->mkAttrs(attrs);
2012-12-03 17:19:49 +00:00
if (drvPath) references.insert(*drvPath);
}
/* Also write a copy of the list of user environment elements to
the store; we need it for future modifications of the
environment. */
std::ostringstream str;
Unify and refactor value printing Previously, there were two mostly-identical value printers -- one in `libexpr/eval.cc` (which didn't force values) and one in `libcmd/repl.cc` (which did force values and also printed ANSI color codes). This PR unifies both of these printers into `print.cc` and provides a `PrintOptions` struct for controlling the output, which allows for toggling whether values are forced, whether repeated values are tracked, and whether ANSI color codes are displayed. Additionally, `PrintOptions` allows tuning the maximum number of attributes, list items, and bytes in a string that will be displayed; this makes it ideal for contexts where printing too much output (e.g. all of Nixpkgs) is distracting. (As requested by @roberth in https://github.com/NixOS/nix/pull/9554#issuecomment-1845095735) Please read the tests for example output. Future work: - It would be nice to provide this function as a builtin, perhaps `builtins.toStringDebug` -- a printing function that never fails would be useful when debugging Nix code. - It would be nice to support customizing `PrintOptions` members on the command line, e.g. `--option to-string-max-attrs 1000`. (cherry picked from commit 0fa08b451682fb3311fe58112ff05c4fe5bee3a4, ) === Restore ambiguous value printer for `nix-instantiate` The Nix team has requested that this output format remain unchanged. I've added a warning to the man page explaining that `nix-instantiate --eval` output will not parse correctly in many situations. (cherry picked from commit df84dd4d8dd3fd6381ac2ca3064432ab31a16b79) Change-Id: I7cca6b4b53cd0642f2d49af657d5676a8554c9f8
2024-03-08 02:05:47 +00:00
printAmbiguous(manifest, state.symbols, str, nullptr, std::numeric_limits<int>::max());
auto manifestFile = state.store->addTextToStore("env-manifest.nix",
str.str(), references);
/* Get the environment builder expression. */
Value envBuilder;
2020-03-11 15:41:22 +00:00
state.eval(state.parseExprFromString(
#include "buildenv.nix.gen.hh"
, state.rootPath(CanonPath::root)), envBuilder);
/* Construct a Nix expression that calls the user environment
builder with the manifest as argument. */
auto attrs = state.buildBindings(3);
Use `std::set<StringContextElem>` not `PathSet` for string contexts Motivation `PathSet` is not correct because string contexts have other forms (`Built` and `DrvDeep`) that are not rendered as plain store paths. Instead of wrongly using `PathSet`, or "stringly typed" using `StringSet`, use `std::std<StringContextElem>`. ----- In support of this change, `NixStringContext` is now defined as `std::std<StringContextElem>` not `std:vector<StringContextElem>`. The old definition was just used by a `getContext` method which was only used by the eval cache. It can be deleted altogether since the types are now unified and the preexisting `copyContext` function already suffices. Summarizing the previous paragraph: Old: - `value/context.hh`: `NixStringContext = std::vector<StringContextElem>` - `value.hh`: `NixStringContext Value::getContext(...)` - `value.hh`: `copyContext(...)` New: - `value/context.hh`: `NixStringContext = std::set<StringContextElem>` - `value.hh`: `copyContext(...)` ---- The string representation of string context elements no longer contains the store dir. The diff of `src/libexpr/tests/value/context.cc` should make clear what the new representation is, so we recommend reviewing that file first. This was done for two reasons: Less API churn: `Value::mkString` and friends did not take a `Store` before. But if `NixStringContextElem::{parse, to_string}` *do* take a store (as they did before), then we cannot have the `Value` functions use them (in order to work with the fully-structured `NixStringContext`) without adding that argument. That would have been a lot of churn of threading the store, and this diff is already large enough, so the easier and less invasive thing to do was simply make the element `parse` and `to_string` functions not take the `Store` reference, and the easiest way to do that was to simply drop the store dir. Space usage: Dropping the `/nix/store/` (or similar) from the internal representation will safe space in the heap of the Nix programming being interpreted. If the heap contains many strings with non-trivial contexts, the saving could add up to something significant. ---- The eval cache version is bumped. The eval cache serialization uses `NixStringContextElem::{parse, to_string}`, and since those functions are changed per the above, that means the on-disk representation is also changed. This is simply done by changing the name of the used for the eval cache from `eval-cache-v4` to eval-cache-v5`. ---- To avoid some duplication `EvalCache::mkPathString` is added to abstract over the simple case of turning a store path to a string with just that string in the context. Context This PR picks up where #7543 left off. That one introduced the fully structured `NixStringContextElem` data type, but kept `PathSet context` as an awkward middle ground between internal `char[][]` interpreter heap string contexts and `NixStringContext` fully parsed string contexts. The infelicity of `PathSet context` was specifically called out during Nix team group review, but it was agreeing that fixing it could be left as future work. This is that future work. A possible follow-up step would be to get rid of the `char[][]` evaluator heap representation, too, but it is not yet clear how to do that. To use `NixStringContextElem` there we would need to get the STL containers to GC pointers in the GC build, and I am not sure how to do that. ---- PR #7543 effectively is writing the inverse of a `mkPathString`, `mkOutputString`, and one more such function for the `DrvDeep` case. I would like that PR to have property tests ensuring it is actually the inverse as expected. This PR sets things up nicely so that reworking that PR to be in that more elegant and better tested way is possible. Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2023-01-29 01:31:10 +00:00
state.mkStorePathString(manifestFile, attrs.alloc("manifest"));
attrs.insert(state.symbols.create("derivations"), &manifest);
Value args;
args.mkAttrs(attrs);
Value topLevel;
topLevel.mkApp(&envBuilder, &args);
2012-12-03 17:19:49 +00:00
/* Evaluate it. */
debug("evaluating user environment builder");
state.forceValue(topLevel, topLevel.determinePos(noPos));
Use `std::set<StringContextElem>` not `PathSet` for string contexts Motivation `PathSet` is not correct because string contexts have other forms (`Built` and `DrvDeep`) that are not rendered as plain store paths. Instead of wrongly using `PathSet`, or "stringly typed" using `StringSet`, use `std::std<StringContextElem>`. ----- In support of this change, `NixStringContext` is now defined as `std::std<StringContextElem>` not `std:vector<StringContextElem>`. The old definition was just used by a `getContext` method which was only used by the eval cache. It can be deleted altogether since the types are now unified and the preexisting `copyContext` function already suffices. Summarizing the previous paragraph: Old: - `value/context.hh`: `NixStringContext = std::vector<StringContextElem>` - `value.hh`: `NixStringContext Value::getContext(...)` - `value.hh`: `copyContext(...)` New: - `value/context.hh`: `NixStringContext = std::set<StringContextElem>` - `value.hh`: `copyContext(...)` ---- The string representation of string context elements no longer contains the store dir. The diff of `src/libexpr/tests/value/context.cc` should make clear what the new representation is, so we recommend reviewing that file first. This was done for two reasons: Less API churn: `Value::mkString` and friends did not take a `Store` before. But if `NixStringContextElem::{parse, to_string}` *do* take a store (as they did before), then we cannot have the `Value` functions use them (in order to work with the fully-structured `NixStringContext`) without adding that argument. That would have been a lot of churn of threading the store, and this diff is already large enough, so the easier and less invasive thing to do was simply make the element `parse` and `to_string` functions not take the `Store` reference, and the easiest way to do that was to simply drop the store dir. Space usage: Dropping the `/nix/store/` (or similar) from the internal representation will safe space in the heap of the Nix programming being interpreted. If the heap contains many strings with non-trivial contexts, the saving could add up to something significant. ---- The eval cache version is bumped. The eval cache serialization uses `NixStringContextElem::{parse, to_string}`, and since those functions are changed per the above, that means the on-disk representation is also changed. This is simply done by changing the name of the used for the eval cache from `eval-cache-v4` to eval-cache-v5`. ---- To avoid some duplication `EvalCache::mkPathString` is added to abstract over the simple case of turning a store path to a string with just that string in the context. Context This PR picks up where #7543 left off. That one introduced the fully structured `NixStringContextElem` data type, but kept `PathSet context` as an awkward middle ground between internal `char[][]` interpreter heap string contexts and `NixStringContext` fully parsed string contexts. The infelicity of `PathSet context` was specifically called out during Nix team group review, but it was agreeing that fixing it could be left as future work. This is that future work. A possible follow-up step would be to get rid of the `char[][]` evaluator heap representation, too, but it is not yet clear how to do that. To use `NixStringContextElem` there we would need to get the STL containers to GC pointers in the GC build, and I am not sure how to do that. ---- PR #7543 effectively is writing the inverse of a `mkPathString`, `mkOutputString`, and one more such function for the `DrvDeep` case. I would like that PR to have property tests ensuring it is actually the inverse as expected. This PR sets things up nicely so that reworking that PR to be in that more elegant and better tested way is possible. Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2023-01-29 01:31:10 +00:00
NixStringContext context;
Attr & aDrvPath(*topLevel.attrs->find(state.sDrvPath));
auto topLevelDrv = state.coerceToStorePath(aDrvPath.pos, *aDrvPath.value, context, "");
Attr & aOutPath(*topLevel.attrs->find(state.sOutPath));
auto topLevelOut = state.coerceToStorePath(aOutPath.pos, *aOutPath.value, context, "");
2012-12-03 17:19:49 +00:00
/* Realise the resulting store expression. */
debug("building user environment");
std::vector<StorePathWithOutputs> topLevelDrvs;
topLevelDrvs.push_back({topLevelDrv});
state.store->buildPaths(
2021-04-05 13:48:18 +00:00
toDerivedPaths(topLevelDrvs),
state.repair ? bmRepair : bmNormal);
/* Switch the current user environment to the output path. */
auto store2 = state.store.dynamic_pointer_cast<LocalFSStore>();
if (store2) {
PathLocks lock;
lockProfile(lock, profile);
Path lockTokenCur = optimisticLockProfile(profile);
if (lockToken != lockTokenCur) {
2020-05-13 15:52:36 +00:00
printInfo("profile '%1%' changed while we were busy; restarting", profile);
return false;
}
2012-12-03 17:19:49 +00:00
debug("switching to new user environment");
Path generation = createGeneration(*store2, profile, topLevelOut);
switchLink(profile, generation);
}
return true;
}
}