lix/tests/functional/phantom-referrers-gc.sh

63 lines
1.3 KiB
Bash
Raw Normal View History

fix(local-store): invalidate phantom referrers at garbage collection time Sometimes, a path can disappear from the `ValidPaths` table (I have 23 such cases on my 1.4TB Nix store). When this occurs and you try to run a garbage collection, `queryReferrers` will report no referrer because it's performing a *RIGHT JOIN* between `Refs` and `ValidPaths`, finally, when you issue the deletion SQL statement, this will throw an uncaught exception from SQLite side regarding a foreign key violation because `reference` in `Refs` is a foreign key to `ValidPaths` (which we are trying to delete). Why can this happen? Two reasons: * `PRAGMA foreign_keys=off;` will disable deletion on cascade. * Trigger recursion *limits*, a deletion on cascade is a *trigger*, when a delete is issued and it triggers a bunch of deletion on cascade, there's a documented limit by SQLite: https://www.sqlite.org/limits.html#max_trigger_depth > Recursion limit on foreign key actions. The SQLITE_MAX_TRIGGER_DEPTH > and SQLITE_LIMIT_TRIGGER_DEPTH settings determine the maximum > allowable depth of trigger program recursion. For the purposes of > these limits, foreign key actions are considered trigger programs. The > PRAGMA recursive_triggers setting does not affect the operation of > foreign key actions. It is not possible to disable recursive foreign > key actions. As I do not see easy ways to solve the root cause, garbage collection should be self-healing in that regards, so I propose to invalidate phantom referrers as we go. As part of a work improving the consistency of the SQLite database, it would make sense to count how many times this happen and try to find ways to reproduce this issue. Change-Id: I055a8a1d8c0e44d4388a411abe8e5a5e385f7b55 Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-09-14 17:50:08 +00:00
source common.sh
startDaemon
requireDaemonNewerThan "2.92.0"
requireSqliteDatabase
clearStore
depOutPath=$(nix-build --no-out-link -E '
with import ./config.nix;
mkDerivation {
name = "phantom";
outputs = [ "out" ];
buildCommand = "
echo i will become a phantom soon > $out
";
}
')
finalOutPath=$(nix-build --no-out-link -E '
with import ./config.nix;
let dep = mkDerivation {
name = "phantom";
outputs = [ "out" ];
buildCommand = "
echo i will become a phantom soon > $out
";
}; in
mkDerivation {
name = "phantom-gc";
outputs = [ "out" ];
buildCommand = "
echo UNUSED: ${dep} > $out
";
}
')
echo "displaying all valid paths"
sqlite3 "$NIX_SQLITE_DATABASE" <<EOF
select * from validpaths;
EOF
echo "displaying the relevant IDs..."
sqlite3 "$NIX_SQLITE_DATABASE" <<EOF
select r.referrer, r.reference from Refs r join ValidPaths vp on r.referrer = vp.id where path = '$finalOutPath';
EOF
echo "corrupting the SQLite database manually..."
sqlite3 "$NIX_SQLITE_DATABASE" <<EOF
pragma foreign_keys = off;
delete from ValidPaths where path = '$finalOutPath';
select * from Refs;
EOF
restartDaemon
# expect this to work and maybe warn about phantom referrers
expectStderr 0 nix-collect-garbage -vvvv | grepQuiet 'phantom referrers'