We now build the context (so this has the side-effect of making builtins.{path,filterSource} work on derivations outputs, if IFD is enabled) and then check that the path has no references (which is what we really care about).