lix/src/libstore
John Ericson 37fca662b0 Make KeyedBuildResult, BuildResult like before, and fix bug another way
In https://github.com/NixOS/nix/pull/6311#discussion_r834863823, I
realized since derivation goals' wanted outputs can "grow" due to
overlapping dependencies (See `DerivationGoal::addWantedOutputs`, called
by `Worker::makeDerivationGoalCommon`), the previous bug fix had an
unfortunate side effect of causing more pointless rebuilds.

In paticular, we have this situation:

1. Goal made from `DerivedPath::Built { foo, {a} }`.

2. Goal gives on on substituting, starts building.

3. Goal made from `DerivedPath::Built { foo, {b} }`, in fact is just
   modified original goal.

4. Though the goal had gotten as far as building, so all outputs were
   going to be produced, `addWantedOutputs` no longer knows that and so
   the goal is flagged to be restarted.

This might sound far-fetched with input-addressed drvs, where we usually
basically have all our goals "planned out" before we start doing
anything, but with CA derivation goals and especially RFC 92, where *drv
resolution* means goals are created after some building is completed, it
is more likely to happen.

So the first thing to do was restore the clearing of `wantedOutputs` we
used to do, and then filter the outputs in `buildPathsWithResults` to
only get the ones we care about.

But fix also has its own side effect in that the `DerivedPath` in the
`BuildResult` in `DerivationGoal` cannot be trusted; it is merely the
*first* `DerivedPath` for which this goal was originally created.

To remedy this, I made `BuildResult` be like it was before, and instead
made `KeyedBuildResult` be a subclass wit the path. Only
`buildPathsWithResults` returns `KeyedBuildResult`s, everything else
just becomes like it was before, where the "key" is unambiguous from
context.

I think separating the "primary key" field(s) from the other fields is
good practical in general anyways. (I would like to do the same thing
for `ValidPathInfo`.) Among other things, it allows constructions like
`std::map<Key, ThingWithKey>` where doesn't contain duplicate keys and
just precludes the possibility of those duplicate keys being out of
sync.

We might leverage the above someday to overload `buildPathsWithResults`
to take a *set* of return a *map* per the above.

-----

Unfortunately, we need to avoid C++20 strictness on designated
initializers.

(BTW
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2287r1.html
this offers some new syntax for this use-case. Hopefully this will be
adopted and we can eventually use it.)

No having that yet, maybe it would be better to not make
`KeyedBuildResult` a subclass to just avoid this.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-04-15 11:01:31 -04:00
..
build Make KeyedBuildResult, BuildResult like before, and fix bug another way 2023-04-15 11:01:31 -04:00
builtins Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
tests Legacy vs non-legacy to_string/parse for DerivedPath 2023-04-14 20:45:11 -04:00
binary-cache-store.cc Merge pull request #7203 from graham33/feature/cpp20 2023-02-01 15:41:04 +01:00
binary-cache-store.hh Finish converting existing comments for internal API docs (#8146) 2023-04-07 13:55:28 +00:00
build-result.hh Make KeyedBuildResult, BuildResult like before, and fix bug another way 2023-04-15 11:01:31 -04:00
builtins.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
ca-specific-schema.sql
content-address.cc
content-address.hh Create Derivation::fromJSON 2023-04-07 08:34:58 -04:00
crypto.cc
crypto.hh Finish converting existing comments for internal API docs (#8146) 2023-04-07 13:55:28 +00:00
daemon.cc Add Store::isTrustedClient() 2023-04-06 19:59:57 -04:00
daemon.hh Add Store::isTrustedClient() 2023-04-06 19:59:57 -04:00
derivations.cc Liberate checkDerivationOutputs from LocalStore 2023-04-07 08:34:58 -04:00
derivations.hh Finish converting existing comments for internal API docs (#8146) 2023-04-07 13:55:28 +00:00
derived-path.cc Legacy vs non-legacy to_string/parse for DerivedPath 2023-04-14 20:45:11 -04:00
derived-path.hh Legacy vs non-legacy to_string/parse for DerivedPath 2023-04-14 20:45:11 -04:00
dummy-store.cc Push getFSAccessor unsupported(...) down Store class hierarchy 2023-04-13 13:39:44 -04:00
dummy-store.md Support per-store Markdown documentation 2023-03-21 14:03:40 +01:00
export-import.cc Remove FormatOrString and remaining uses of format() 2023-03-02 15:57:54 +01:00
filetransfer.cc Allow open switch-enum in 5 places 2023-04-03 18:45:20 +02:00
filetransfer.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
fs-accessor.hh Finish converting existing comments for internal API docs (#8146) 2023-04-07 13:55:28 +00:00
gc-store.hh Finish converting existing comments for internal API docs (#8146) 2023-04-07 13:55:28 +00:00
gc.cc Make findRuntimeRoots() more resilient to disappearing processes 2023-03-09 16:44:51 +01:00
globals.cc Merge pull request #8062 from edolstra/ssl-cert-file 2023-03-27 15:37:15 +02:00
globals.hh Finish converting existing comments for internal API docs (#8146) 2023-04-07 13:55:28 +00:00
http-binary-cache-store.cc Add Store::isTrustedClient() 2023-04-06 19:59:57 -04:00
http-binary-cache-store.md Support per-store Markdown documentation 2023-03-21 14:03:40 +01:00
legacy-ssh-store.cc Make KeyedBuildResult, BuildResult like before, and fix bug another way 2023-04-15 11:01:31 -04:00
legacy-ssh-store.md Support per-store Markdown documentation 2023-03-21 14:03:40 +01:00
local-binary-cache-store.cc Add Store::isTrustedClient() 2023-04-06 19:59:57 -04:00
local-binary-cache-store.md Support per-store Markdown documentation 2023-03-21 14:03:40 +01:00
local-fs-store.cc getBuildLog: factor out resolving derivations 2022-12-15 15:58:54 -05:00
local-fs-store.hh Finish converting existing comments for internal API docs (#8146) 2023-04-07 13:55:28 +00:00
local-store.cc Liberate checkDerivationOutputs from LocalStore 2023-04-07 08:34:58 -04:00
local-store.hh Finish converting existing comments for internal API docs (#8146) 2023-04-07 13:55:28 +00:00
local-store.md Tweaks 2023-03-23 15:32:59 +01:00
local.mk Include macOS sandbox files in the Nix binary 2023-01-04 04:36:07 -08:00
lock.cc Move enabled experimental feature to libutil struct 2023-03-20 11:05:22 -04:00
lock.hh Finish converting existing comments for internal API docs (#8146) 2023-04-07 13:55:28 +00:00
log-store.cc Move the getBuildLog implementation to its own implementation file 2023-01-13 11:05:44 +01:00
log-store.hh Finish converting existing comments for internal API docs (#8146) 2023-04-07 13:55:28 +00:00
machines.cc
machines.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
make-content-addressed.cc
make-content-addressed.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
misc.cc Move enabled experimental feature to libutil struct 2023-03-20 11:05:22 -04:00
names.cc
names.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
nar-accessor.cc Add explicit case statements where -Wswitch-enum would report them 2023-04-03 18:17:32 +02:00
nar-accessor.hh Finish converting existing comments for internal API docs (#8146) 2023-04-07 13:55:28 +00:00
nar-info-disk-cache.cc NarInfoDiskCache: Keep BinaryCache.id stable and improve test 2023-02-07 23:34:36 +01:00
nar-info-disk-cache.hh Finish converting existing comments for internal API docs (#8146) 2023-04-07 13:55:28 +00:00
nar-info.cc libstore/nar-info: drop unused system field 2022-06-23 14:25:10 -04:00
nar-info.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
nix-store.pc.in Fix building with GCC 9 2023-02-10 18:38:57 +01:00
optimise-store.cc Remove FormatOrString and remaining uses of format() 2023-03-02 15:57:54 +01:00
outputs-spec.cc Test store paths, with property tests 2023-01-23 07:05:50 -05:00
outputs-spec.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
parsed-derivations.cc Merge pull request #3600 from NixOS/auto-uid-allocation 2022-11-29 14:01:42 +01:00
parsed-derivations.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
path-info.cc Move ValidPathInfo defintions to path-info.cc 2023-01-13 15:39:19 -05:00
path-info.hh Finish converting existing comments for internal API docs (#8146) 2023-04-07 13:55:28 +00:00
path-regex.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
path-with-outputs.cc Merge pull request #6815 from obsidiansystems/better-wanted-outputs 2023-01-13 16:03:12 +01:00
path-with-outputs.hh Finish converting existing comments for internal API docs (#8146) 2023-04-07 13:55:28 +00:00
path.cc Test store paths, with property tests 2023-01-23 07:05:50 -05:00
path.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
pathlocks.cc Remove FormatOrString and remaining uses of format() 2023-03-02 15:57:54 +01:00
pathlocks.hh Finish converting existing comments for internal API docs (#8146) 2023-04-07 13:55:28 +00:00
profiles.cc Factor out the generation of the profile/channel directory 2023-03-23 10:22:42 +01:00
profiles.hh Finish converting existing comments for internal API docs (#8146) 2023-04-07 13:55:28 +00:00
realisation.cc
realisation.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
references.cc Remove FormatOrString and remaining uses of format() 2023-03-02 15:57:54 +01:00
references.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
remote-fs-accessor.cc Replace src/libutil/json.cc with nlohmann json generation 2022-11-16 16:50:50 +01:00
remote-fs-accessor.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
remote-store.cc Make KeyedBuildResult, BuildResult like before, and fix bug another way 2023-04-15 11:01:31 -04:00
remote-store.hh Make KeyedBuildResult, BuildResult like before, and fix bug another way 2023-04-15 11:01:31 -04:00
repair-flag.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
s3-binary-cache-store.cc Add Store::isTrustedClient() 2023-04-06 19:59:57 -04:00
s3-binary-cache-store.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
s3-binary-cache-store.md Support per-store Markdown documentation 2023-03-21 14:03:40 +01:00
s3.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
schema.sql schema.sql: add comment about hash being in base16 2022-06-01 14:59:57 +02:00
serve-protocol.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
sqlite.cc Move warning timing logic into handleSQLiteBusy. 2023-04-11 16:14:16 +01:00
sqlite.hh Move warning timing logic into handleSQLiteBusy. 2023-04-11 16:14:16 +01:00
ssh-store-config.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
ssh-store.cc Fix SSHStore 2023-03-23 09:35:35 +01:00
ssh-store.md Support per-store Markdown documentation 2023-03-21 14:03:40 +01:00
ssh.cc SSHMaster: pause logger to show password prompt 2023-03-22 09:45:08 +04:00
ssh.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
store-api.cc Move querySubstitutablePathInfos from LocalStore to Store 2023-04-02 20:32:01 -04:00
store-api.hh Make KeyedBuildResult, BuildResult like before, and fix bug another way 2023-04-15 11:01:31 -04:00
store-cast.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
uds-remote-store.cc Register LocalStore to ensure it's included in the manual 2023-03-23 15:23:13 +01:00
uds-remote-store.hh Ensure all headers have #pragma once and are in API docs 2023-03-31 23:19:44 -04:00
uds-remote-store.md Support per-store Markdown documentation 2023-03-21 14:03:40 +01:00
worker-protocol.hh Make KeyedBuildResult, BuildResult like before, and fix bug another way 2023-04-15 11:01:31 -04:00