raito 2a7a6cb85a fix(local-store): invalidate phantom referrers at garbage collection time ... Sometimes, a path can disappear from the `ValidPaths` table (I have 23 such cases on my 1.4TB Nix store). When this occurs and you try to run a garbage collection, `queryReferrers` will report no referrer because it's performing a *RIGHT JOIN* between `Refs` and `ValidPaths`, finally, when you issue the deletion SQL statement, this will throw an uncaught exception from SQLite side regarding a foreign key violation because `reference` in `Refs` is a foreign key to `ValidPaths` (which we are trying to delete). Why can this happen? Two reasons: * `PRAGMA foreign_keys=off;` will disable deletion on cascade. * Trigger recursion *limits*, a deletion on cascade is a *trigger*, when a delete is issued and it triggers a bunch of deletion on cascade, there's a documented limit by SQLite: https://www.sqlite.org/limits.html#max_trigger_depth > Recursion limit on foreign key actions. The SQLITE_MAX_TRIGGER_DEPTH > and SQLITE_LIMIT_TRIGGER_DEPTH settings determine the maximum > allowable depth of trigger program recursion. For the purposes of > these limits, foreign key actions are considered trigger programs. The > PRAGMA recursive_triggers setting does not affect the operation of > foreign key actions. It is not possible to disable recursive foreign > key actions. As I do not see easy ways to solve the root cause, garbage collection should be self-healing in that regards, so I propose to invalidate phantom referrers as we go. As part of a work improving the consistency of the SQLite database, it would make sense to count how many times this happen and try to find ways to reproduce this issue. Change-Id: I055a8a1d8c0e44d4388a411abe8e5a5e385f7b55 Signed-off-by: Raito Bezarius <raito@lix.systems>		2024-09-14 20:09:17 +02:00
..
ca	chore: rebrand Nix to Lix when it makes sense	2024-06-01 20:31:24 +02:00
common	fix(local-store): invalidate phantom referrers at garbage collection time	2024-09-14 20:09:17 +02:00
config
dyn-drv	remove the autoconf+Make buildsystem	2024-05-07 17:04:30 -06:00
flakes	flake: don't refetch unmodified inputs by recursive follows	2024-08-19 19:57:12 +00:00
lang	libexpr: Soft-deprecate ancient let syntax	2024-08-21 12:59:03 +02:00
nested-sandboxing	chore: rebrand Nix to Lix when it makes sense	2024-06-01 20:31:24 +02:00
plugins	remove the autoconf+Make buildsystem	2024-05-07 17:04:30 -06:00
repl_characterization	tree-wide: add support for asan!	2024-07-31 14:13:39 -07:00
test-libstoreconsumer	tree-wide: add support for asan!	2024-07-31 14:13:39 -07:00
add.sh
bad.tar.xz
bash-profile.sh	Fix failing darwin tests	2024-05-06 18:56:40 +02:00
big-derivation-attr.nix
binary-cache-build-remote.sh
binary-cache.sh	chore: rebrand Nix to Lix when it makes sense	2024-06-01 20:31:24 +02:00
brotli.sh
build-delete.sh
build-dry.sh
build-hook-ca-fixed.nix	Merge pull request #9032 from Ma27/structured-attrs-env-vars	2024-03-07 10:46:47 +01:00
build-hook-ca-floating.nix
build-hook.nix	Merge pull request #9032 from Ma27/structured-attrs-env-vars	2024-03-07 10:46:47 +01:00
build-jobless.sh	libstore: diagnose local build failure in goal	2024-08-25 19:55:47 +02:00
build-remote-content-addressed-fixed.sh
build-remote-content-addressed-floating.sh
build-remote-input-addressed.sh
build-remote-trustless-after.sh
build-remote-trustless-should-fail-0.sh	chore: rebrand Nix to Lix when it makes sense	2024-06-01 20:31:24 +02:00
build-remote-trustless-should-pass-0.sh
build-remote-trustless-should-pass-1.sh
build-remote-trustless-should-pass-2.sh
build-remote-trustless-should-pass-3.sh
build-remote-trustless.sh
build-remote.sh
build.sh	add dedicated test for hash mismatch url reporting	2024-08-25 22:21:32 +00:00
ca-shell.nix
case-hack.sh
case.nar
check-refs.nix
check-refs.sh
check-reqs.nix
check-reqs.sh
check.nix
check.sh	Add build-dir setting, clean up default TMPDIR handling	2024-06-24 11:30:32 +03:00
common.sh
completions.sh	Merge pull request #8817 from iFreilicht/flake-update-lock-overhaul	2024-03-25 17:36:24 -06:00
compression-levels.sh
compute-levels.sh
config.nix.in	Merge pull request #9032 from Ma27/structured-attrs-env-vars	2024-03-07 10:46:47 +01:00
config.sh	Rename nix show-config to nix config show	2024-05-03 16:26:16 +02:00
db-migration.sh	chore: rebrand Nix to Lix when it makes sense	2024-06-01 20:31:24 +02:00
debugger.sh	Merge pull request #9933 from pennae/debugger-fix	2024-03-04 08:57:44 +01:00
dependencies.builder0.sh
dependencies.nix
dependencies.sh
derivation-json.sh
dummy
dump-db.sh
eval-store.sh	Merge pull request #9589 from obsidiansystems/floating-content-addressing-derivations-eval-store	2024-03-04 07:15:08 +01:00
eval.nix
eval.sh	distinguish between throws & errors during throw	2024-07-04 17:43:03 -06:00
experimental-features.sh	libexpr/flake: allow automatic rejection of configuration options from flakes	2024-06-30 19:28:14 +02:00
export-graph.nix
export-graph.sh
export.sh
extra-sandbox-profile.nix	libstore: Fix sandbox=relaxed	2024-05-08 19:31:43 +00:00
extra-sandbox-profile.sh	libstore: Fix sandbox=relaxed	2024-05-08 19:31:43 +00:00
failing.nix	Merge pull request #9032 from Ma27/structured-attrs-env-vars	2024-03-07 10:46:47 +01:00
fetchClosure.sh	Add pre-commit checks	2024-03-29 22:57:40 -07:00
fetchers.sh	Reapply "libfetchers: make attribute / URL query handling consistent"	2024-08-01 15:41:30 -07:00
fetchGit.sh	treewide: Stop using URL literals	2024-08-17 15:48:10 +00:00
fetchGitRefs.sh
fetchGitSubmodules.sh	treewide: Stop using URL literals	2024-08-17 15:48:10 +00:00
fetchMercurial.sh	treewide: Stop using URL literals	2024-08-17 15:48:10 +00:00
fetchPath.sh
fetchTree-file.sh	chore: rebrand Nix to Lix when it makes sense	2024-06-01 20:31:24 +02:00
fetchurl.sh
filter-source.nix
filter-source.sh
fixed.builder1.sh
fixed.builder2.sh
fixed.nix	Merge pull request #9662 from shlevy/flat-fixed-references-assert	2024-03-04 07:50:02 +01:00
fixed.sh	Merge pull request #9662 from shlevy/flat-fixed-references-assert	2024-03-04 07:50:02 +01:00
fmt.sh
fmt.simple.sh	Add pre-commit checks	2024-03-29 22:57:40 -07:00
fod-failing.nix	store: guess the URL of failing fixed-output derivations	2024-06-27 22:44:16 -07:00
function-trace.sh	Add pre-commit checks	2024-03-29 22:57:40 -07:00
gc-auto.sh
gc-concurrent.builder.sh
gc-concurrent.nix
gc-concurrent.sh
gc-concurrent2.builder.sh
gc-non-blocking.sh	gc: refactor the gc server thread out into a class without changing it	2024-07-19 20:55:55 +00:00
gc-runtime.nix	gc: Find roots using libproc on Darwin	2024-04-25 23:24:21 -04:00
gc-runtime.sh	gc: Find roots using libproc on Darwin	2024-04-25 23:24:21 -04:00
gc.sh
hash-check.nix	Add pre-commit checks	2024-03-29 22:57:40 -07:00
hash.sh
hermetic.nix	Merge pull request #9032 from Ma27/structured-attrs-env-vars	2024-03-07 10:46:47 +01:00
ifd.nix
import-derivation.nix
import-derivation.sh
impure-derivations.nix
impure-derivations.sh	chore: rebrand Nix to Lix when it makes sense	2024-06-01 20:31:24 +02:00
impure-eval.sh	Merge pull request #4093 from matthewbauer/eval-system	2024-03-04 07:21:01 +01:00
init.sh	Fix failing darwin tests	2024-05-06 18:56:40 +02:00
install-darwin.sh	change shebangs of all .sh scripts to bash	2024-06-24 14:00:43 -07:00
lang-test-infra.sh
lang.sh	libexpr: Soft-deprecate __overrides	2024-08-21 06:55:52 +00:00
legacy-ssh-store.sh	"but doctor, I AM the untrusted store": nix doctor had wrong trustedness	2024-04-08 15:40:12 -07:00
linux-sandbox-cert-test.nix	Add pre-commit checks	2024-03-29 22:57:40 -07:00
linux-sandbox.sh	libstore/build: copy ca-certificates too	2024-05-31 07:54:18 +00:00
local-store.sh	"but doctor, I AM the untrusted store": nix doctor had wrong trustedness	2024-04-08 15:40:12 -07:00
logging.sh	Merge pull request #9637 from 9999years/fix-logging-test-on-macos	2024-03-04 07:26:34 +01:00
meson.build	fix(local-store): invalidate phantom referrers at garbage collection time	2024-09-14 20:09:17 +02:00
misc.sh	libutil/args: warn on unknown settings after parsing all flags	2024-05-30 03:07:21 +00:00
multiple-outputs.nix
multiple-outputs.sh
nar-access.nix	Add pre-commit checks	2024-03-29 22:57:40 -07:00
nar-access.sh	Merge pull request #9288 from edolstra/fix-nar-access-test	2024-03-04 05:27:18 +01:00
nested-sandboxing.sh
nix-build-examples.nix
nix-build.sh
nix-channel.sh	Add pre-commit checks	2024-03-29 22:57:40 -07:00
nix-collect-garbage-d.sh
nix-collect-garbage-dry-run.sh	Fix dry-run flag for nix-collect-garbage	2024-07-09 13:55:05 +00:00
nix-copy-ssh-common.sh	Merge pull request #9604 from obsidiansystems/test-substitute-on-destination	2024-03-07 03:34:00 +01:00
nix-copy-ssh-ng.sh	Merge pull request #9604 from obsidiansystems/test-substitute-on-destination	2024-03-07 03:34:00 +01:00
nix-copy-ssh.sh	Merge pull request #9604 from obsidiansystems/test-substitute-on-destination	2024-03-07 03:34:00 +01:00
nix-daemon-untrusting.sh	change shebangs of all .sh scripts to bash	2024-06-24 14:00:43 -07:00
nix-profile.sh	libfetchers: log fetches by URL just before they happen	2024-05-24 15:15:42 -06:00
nix-shell.sh	Merge pull request #9930 from rvl/print-dev-env-unbound-variables	2024-03-04 08:51:49 +01:00
nix_path.sh
optimise-store.sh
output-normalization.sh
parallel.builder.sh
parallel.nix
parallel.sh
pass-as-file.sh
path-from-hash-part.sh
path.nix
phantom-referrers-gc.sh	fix(local-store): invalidate phantom referrers at garbage collection time	2024-09-14 20:09:17 +02:00
placeholders.sh
plugins.sh	tests: don't build test plugin shared libs on static builds	2024-05-12 23:04:21 +02:00
post-hook.sh
pure-eval.nix
pure-eval.sh	treewide: Stop using URL literals	2024-08-17 15:48:10 +00:00
push-to-store-old.sh	change shebangs of all .sh scripts to bash	2024-06-24 14:00:43 -07:00
push-to-store.sh	change shebangs of all .sh scripts to bash	2024-06-24 14:00:43 -07:00
read-only-store.sh	Add pre-commit checks	2024-03-29 22:57:40 -07:00
readfile-context.nix
readfile-context.sh
recursive.nix
recursive.sh
referrers.sh
regression-484.sh	fix: good errors for failures caused by allowSubstitutes	2024-08-23 17:49:15 -07:00
remote-store.sh	"but doctor, I AM the untrusted store": nix doctor had wrong trustedness	2024-04-08 15:40:12 -07:00
repair.sh
repl.sh	src/libcmd/repl.cc: allow :log /path/to/store.drv	2024-08-07 15:58:44 +03:00
restricted.nix
restricted.sh	tests/functional/restricted: Don't use a process substitution	2024-08-19 20:37:51 -07:00
search.nix
search.sh	cli: eat terminal codes from stdout also	2024-08-04 20:41:19 -07:00
secure-drv-outputs.nix
secure-drv-outputs.sh
selfref-gc.sh
shell-hello.nix	Merge pull request #9648 from cole-h/nix-shell-ordering	2024-03-04 07:50:55 +01:00
shell.nix
shell.sh	Merge pull request #9648 from cole-h/nix-shell-ordering	2024-03-04 07:50:55 +01:00
shell.shebang.nix	Add pre-commit checks	2024-03-29 22:57:40 -07:00
shell.shebang.rb	Add pre-commit checks	2024-03-29 22:57:40 -07:00
shell.shebang.sh
signing.sh
simple-failing.nix
simple.builder.sh	Add pre-commit checks	2024-03-29 22:57:40 -07:00
simple.nix
simple.sh
ssh-relay.sh
store-ping.sh	Merge pull request #9632 from cole-h/nix-daemon-testing	2024-03-04 08:17:32 +01:00
structured-attrs-shell.nix
structured-attrs.nix
structured-attrs.sh	Merge pull request #9032 from Ma27/structured-attrs-env-vars	2024-03-07 10:46:47 +01:00
substitute-truncated-nar.sh	libutil: remove sinkToSource eof callback	2024-06-23 11:52:49 +00:00
substitute-with-invalid-ca.sh
suggestions.sh
supplementary-groups.sh
symlink-derivation.nix	Merge pull request #10456 from NixOS/fixpermdeniedbind	2024-04-11 15:43:58 +03:00
tarball.sh	treewide: Stop using URL literals	2024-08-17 15:48:10 +00:00
test-infra.sh
test-libstoreconsumer.sh
test-repl-characterization.sh	Implement a parser for a literate testing system for the repl	2024-03-14 14:30:38 -07:00
timeout.nix
timeout.sh	tests: fix functional-timeout	2024-05-28 04:20:00 +00:00
toString-path.sh
undefined-variable.nix
user-envs-migration.sh
user-envs.builder.sh
user-envs.nix
user-envs.sh	Merge pull request #9257 from Artturin/nixenvjsondrvpath	2024-03-04 05:59:15 +01:00
why-depends.sh
zstd.sh