lix-project/lix
20
55
Fork 31
Code Issues 345 Code Review (Gerrit) Projects 3 Releases Wiki Activity

Speed up fetching submodules

Browse source 
Previously we would completely refetch the submodules from the
network, even though the repo might already have them. Now we copy the
.git/modules directory from the repo as an optimisation. This speeds
up evaluating

  builtins.fetchTree { type = "git"; url = "/path/to/blender"; submodules = true; }

(where /path/to/blender already has the needed submodules) from 121s
to 57s.

This is still pretty inefficient and a hack, but a better solution is
best done on the lazy-trees branch.

This change also help in the case where the repo already has the
submodules but the origin is unfetchable for whatever reason
(e.g. there have been cases where Nix in a GitHub action doesn't have
the right authentication set up).
This commit is contained in:
Eelco Dolstra 2023-02-07 15:50:35 +01:00
parent 2edd5cf618
commit a8fe0dc16c
1 changed files with 8 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
src/libfetchers/git.cc
View file

@ -639,6 +639,14 @@ struct GitInputScheme : InputScheme
} else } else
runProgram("git", true, { "-C", tmpDir, "config", "remote.origin.url", actualUrl }); runProgram("git", true, { "-C", tmpDir, "config", "remote.origin.url", actualUrl });
/* As an optimisation, copy the modules directory of the
source repo if it exists. */
auto modulesPath = repoDir + "/" + gitDir + "/modules";
if (pathExists(modulesPath)) {
Activity act(*logger, lvlTalkative, actUnknown, fmt("copying submodules of '%s'", actualUrl));
runProgram("cp", true, { "-R", "--", modulesPath, tmpGitDir + "/modules" });
}
{ {
Activity act(*logger, lvlTalkative, actUnknown, fmt("fetching submodules of '%s'", tmpDir)); Activity act(*logger, lvlTalkative, actUnknown, fmt("fetching submodules of '%s'", tmpDir));
runProgram("git", true, { "-C", tmpDir, "submodule", "--quiet", "update", "--init", "--recursive" }); runProgram("git", true, { "-C", tmpDir, "submodule", "--quiet", "update", "--init", "--recursive" });