Assertion failure in nix::flake::fetchOrSubstituteTree #529

Open
opened 2024-09-21 11:03:44 +00:00 by yshui · 8 comments

Describe the bug

src/libexpr/flake/flake.cc:72: std::tuple<nix::fetchers::Tree, nix::FlakeRef, nix::FlakeRef> nix::flake::fetchOrSubstituteTree(nix::EvalState&, const nix::FlakeRef&, bool, FlakeCache&): Assertion `!originalRef.input.getNarHash() || tree.storePath == originalRef.input.computeStorePath(*state.store)' failed.

This can be triggered by naming your derivation "source" and later import the built result in a flake. This is because a derivation named "source" will not be copied to store when it's imported:

if (!storePath || storePath->name() != "source" || !store->isValidPath(*storePath)) {

And store paths are computed differently for built outputs.

Why am I doing this?

Because I don't want to create a copy of the input when it's already in the store, it slows down evaluation and take unnecessary disk space.

Proposal

Remove this assertion, and remove the == "source" check as well. Avoid all store to store copies.

## Describe the bug ``` src/libexpr/flake/flake.cc:72: std::tuple<nix::fetchers::Tree, nix::FlakeRef, nix::FlakeRef> nix::flake::fetchOrSubstituteTree(nix::EvalState&, const nix::FlakeRef&, bool, FlakeCache&): Assertion `!originalRef.input.getNarHash() || tree.storePath == originalRef.input.computeStorePath(*state.store)' failed. ``` This can be triggered by naming your derivation "source" and later import the built result in a flake. This is because a derivation named "source" will not be copied to store when it's imported: https://git.lix.systems/lix-project/lix/src/commit/5f298f74c92402a8390b01c736463b17b36277e3/src/libfetchers/path.cc#L131 And store paths are computed differently for built outputs. ## Why am I doing this? Because I don't want to create a copy of the input when it's already in the store, it slows down evaluation and take unnecessary disk space. ## Proposal Remove this assertion, and remove the `== "source"` check as well. Avoid all store to store copies.
yshui added the
bug
label 2024-09-21 11:03:44 +00:00
Author

Note I somehow can't reliably reproduce this, perhaps due to caching or some other oddities.

Here is a stack trace of the assertion failure:

(gdb) bt
#0  0x00007a92cc49b7dc in __pthread_kill_implementation ()
   from /nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib/libc.so.6
#1  0x00007a92cc449516 in raise () from /nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib/libc.so.6
#2  0x00007a92cc431935 in abort () from /nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib/libc.so.6
#3  0x00007a92cc431859 in __assert_fail_base.cold ()
   from /nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib/libc.so.6
#4  0x00007a92cc4419f6 in __assert_fail ()
   from /nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib/libc.so.6
#5  0x00007a92ccdd3ec6 in nix::flake::fetchOrSubstituteTree(nix::EvalState&, nix::FlakeRef const&, bool, std::vector<std::pair<nix::FlakeRef, std::pair<nix::fetchers::Tree, nix::FlakeRef> >, std::allocator<std::pair<nix::FlakeRef, std::pair<nix::fetchers::Tree, nix::FlakeRef> > > >&) ()
   from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixexpr.so
#6  0x00007a92ccdd5950 in nix::flake::getFlake(nix::EvalState&, nix::FlakeRef const&, bool, std::vector<std::pair<nix::FlakeRef, std::pair<nix::fetchers::Tree, nix::FlakeRef> >, std::allocator<std::pair<nix::FlakeRef, std::pair<nix::fetchers::Tree, nix::FlakeRef> > > >&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >) ()
   from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixexpr.so
#7  0x00007a92ccdd6c08 in nix::flake::getFlake(nix::EvalState&, nix::FlakeRef const&, bool, std::vector<std::pair<nix::FlakeRef, std::pair<nix::fetchers::Tree, nix::FlakeRef> >, std::allocator<std::pair<nix::FlakeRef, std::pair<nix::fetchers::Tree, nix::FlakeRef> > > >&) ()
   from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixexpr.so
#8  0x00007a92ccdd95e5 in nix::flake::lockFlake(nix::EvalState&, nix::FlakeRef const&, nix::flake::LockFlags const&)
    () from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixexpr.so
#9  0x00007a92cd6e331b in nix::InstallableFlake::getLockedFlake() const ()
   from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so
#10 0x00007a92cd6e4503 in nix::InstallableFlake::getCursors(nix::EvalState&) ()
   from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so
#11 0x00007a92cd6ea1f8 in nix::InstallableValue::getCursor(nix::EvalState&) ()
   from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so
#12 0x00007a92cd6e3620 in nix::InstallableFlake::toDerivedPaths() ()
   from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so
#13 0x00007a92cd6f77b3 in nix::Installable::build2(nix::ref<nix::Store>, nix::ref<nix::Store>, nix::Realise, std::vector<nix::ref<nix::Installable>, std::allocator<nix::ref<nix::Installable> > > const&, nix::BuildMode) ()
   from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so
#14 0x00007a92cd6f9ec7 in nix::Installable::build(nix::ref<nix::Store>, nix::ref<nix::Store>, nix::Realise, std::vector<nix::ref<nix::Installable>, std::allocator<nix::ref<nix::Installable> > > const&, nix::BuildMode) ()
   from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so
#15 0x00007a92cd6fa13e in nix::Installable::toBuiltPaths(nix::ref<nix::Store>, nix::ref<nix::Store>, nix::Realise, nix::OperateOn, std::vector<nix::ref<nix::Installable>, std::allocator<nix::ref<nix::Installable> > > const&) ()
   from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so
#16 0x00007a92cd6fa60f in nix::Installable::toStorePathSet(nix::ref<nix::Store>, nix::ref<nix::Store>, nix::Realise, nix::OperateOn, std::vector<nix::ref<nix::Installable>, std::allocator<nix::ref<nix::Installable> > > const&) ()
   from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so
#17 0x00000000004f3f5c in CmdDevelop::run(nix::ref<nix::Store>, nix::ref<nix::Installable>) ()
#18 0x00007a92cd6f6b6a in nix::InstallableCommand::run(nix::ref<nix::Store>) ()
   from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so
#19 0x00007a92cd6c4757 in nix::StoreCommand::run() ()
   from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so
#20 0x0000000000558274 in nix::mainWrapped(int, char**) ()
#21 0x00007a92ccfea3fa in nix::handleExceptions(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void ()>) ()
   from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixmain.so
#22 0x000000000048029f in main ()
Note I somehow can't reliably reproduce this, perhaps due to caching or some other oddities. Here is a stack trace of the assertion failure: ``` (gdb) bt #0 0x00007a92cc49b7dc in __pthread_kill_implementation () from /nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib/libc.so.6 #1 0x00007a92cc449516 in raise () from /nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib/libc.so.6 #2 0x00007a92cc431935 in abort () from /nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib/libc.so.6 #3 0x00007a92cc431859 in __assert_fail_base.cold () from /nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib/libc.so.6 #4 0x00007a92cc4419f6 in __assert_fail () from /nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib/libc.so.6 #5 0x00007a92ccdd3ec6 in nix::flake::fetchOrSubstituteTree(nix::EvalState&, nix::FlakeRef const&, bool, std::vector<std::pair<nix::FlakeRef, std::pair<nix::fetchers::Tree, nix::FlakeRef> >, std::allocator<std::pair<nix::FlakeRef, std::pair<nix::fetchers::Tree, nix::FlakeRef> > > >&) () from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixexpr.so #6 0x00007a92ccdd5950 in nix::flake::getFlake(nix::EvalState&, nix::FlakeRef const&, bool, std::vector<std::pair<nix::FlakeRef, std::pair<nix::fetchers::Tree, nix::FlakeRef> >, std::allocator<std::pair<nix::FlakeRef, std::pair<nix::fetchers::Tree, nix::FlakeRef> > > >&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >) () from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixexpr.so #7 0x00007a92ccdd6c08 in nix::flake::getFlake(nix::EvalState&, nix::FlakeRef const&, bool, std::vector<std::pair<nix::FlakeRef, std::pair<nix::fetchers::Tree, nix::FlakeRef> >, std::allocator<std::pair<nix::FlakeRef, std::pair<nix::fetchers::Tree, nix::FlakeRef> > > >&) () from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixexpr.so #8 0x00007a92ccdd95e5 in nix::flake::lockFlake(nix::EvalState&, nix::FlakeRef const&, nix::flake::LockFlags const&) () from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixexpr.so #9 0x00007a92cd6e331b in nix::InstallableFlake::getLockedFlake() const () from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so #10 0x00007a92cd6e4503 in nix::InstallableFlake::getCursors(nix::EvalState&) () from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so #11 0x00007a92cd6ea1f8 in nix::InstallableValue::getCursor(nix::EvalState&) () from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so #12 0x00007a92cd6e3620 in nix::InstallableFlake::toDerivedPaths() () from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so #13 0x00007a92cd6f77b3 in nix::Installable::build2(nix::ref<nix::Store>, nix::ref<nix::Store>, nix::Realise, std::vector<nix::ref<nix::Installable>, std::allocator<nix::ref<nix::Installable> > > const&, nix::BuildMode) () from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so #14 0x00007a92cd6f9ec7 in nix::Installable::build(nix::ref<nix::Store>, nix::ref<nix::Store>, nix::Realise, std::vector<nix::ref<nix::Installable>, std::allocator<nix::ref<nix::Installable> > > const&, nix::BuildMode) () from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so #15 0x00007a92cd6fa13e in nix::Installable::toBuiltPaths(nix::ref<nix::Store>, nix::ref<nix::Store>, nix::Realise, nix::OperateOn, std::vector<nix::ref<nix::Installable>, std::allocator<nix::ref<nix::Installable> > > const&) () from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so #16 0x00007a92cd6fa60f in nix::Installable::toStorePathSet(nix::ref<nix::Store>, nix::ref<nix::Store>, nix::Realise, nix::OperateOn, std::vector<nix::ref<nix::Installable>, std::allocator<nix::ref<nix::Installable> > > const&) () from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so #17 0x00000000004f3f5c in CmdDevelop::run(nix::ref<nix::Store>, nix::ref<nix::Installable>) () #18 0x00007a92cd6f6b6a in nix::InstallableCommand::run(nix::ref<nix::Store>) () from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so #19 0x00007a92cd6c4757 in nix::StoreCommand::run() () from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixcmd.so #20 0x0000000000558274 in nix::mainWrapped(int, char**) () #21 0x00007a92ccfea3fa in nix::handleExceptions(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void ()>) () from /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17/lib/liblixmain.so #22 0x000000000048029f in main () ```
Owner

Do you still have the core dump? Can you grab some info from it?

Here's how you can get debug info:

NIX_DEBUG_INFO_DIRS="$(nix derivation show /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17 | jq -r '.[] | .outputs.debug.path')/lib/debug" coredumpctl debug

then inside the gdb, put the path resulting from nix derivation show /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17 | jq -r '.[] | .env.src' as dir so you have sources for the Lix:

For example, dir /nix/store/fdlsflkgdslglkjg-source

What I would like is to know the output of:

f 5
p tree
p originalRef.input
Do you still have the core dump? Can you grab some info from it? Here's how you can get debug info: ``` NIX_DEBUG_INFO_DIRS="$(nix derivation show /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17 | jq -r '.[] | .outputs.debug.path')/lib/debug" coredumpctl debug ``` then inside the gdb, put the path resulting from `nix derivation show /nix/store/83ac3g127apfhick95hlrjd5s2wgd5zf-lix-2.91.0pre20240820_ed51a17 | jq -r '.[] | .env.src'` as `dir` so you have sources for the Lix: For example, `dir /nix/store/fdlsflkgdslglkjg-source` What I would like is to know the output of: ``` f 5 p tree p originalRef.input ```
Owner

I am not really sure how that assertion gets triggered. It implies that we can have a fetcher grab an input (and do so successfully) with a different narHash attribute than the one in originalRef.input, which I think is the one in the lock file, without first throwing an error that the hash did not match.

That definitely should be a bug, but I don't know what we can do about it without a reproducer.

I did notice a possible soundness bug in the equals operator of Input, which is that if you have two path inputs with different parent but the same attrs, they will compare as equal, but I am not sure if this is possible to trigger. The inference I can draw from this is that maybe the fetcher cache could have mixed up two path inputs? But this is no smoking gun.

I am not really sure how that assertion gets triggered. It implies that we can have a fetcher grab an input (and do so successfully) with a different narHash attribute than the one in `originalRef.input`, which I think is the one in the lock file, without first throwing an error that the hash did not match. That definitely *should* be a bug, but I don't know what we can do about it without a reproducer. I did notice a possible soundness bug in the equals operator of `Input`, which is that if you have two path inputs with different `parent` but the same attrs, they will compare as equal, but I am not sure if this is possible to trigger. The inference I can draw from this is that maybe the fetcher cache could have mixed up two path inputs? But this is no smoking gun.
Author

sorry I gc'd and doesn't have the original executable anymore. maybe if it happens again i can get the info for you.

sorry I gc'd and doesn't have the original executable anymore. maybe if it happens again i can get the info for you.
Author

@jade

It implies that we can have a fetcher grab an input (and do so successfully) with a different narHash attribute than the one in originalRef.input

This is not what this assertion checks. !originalRef.input.getNarHash() is false, because this input has a narHash, so it is actually this part that fails:

tree.storePath == originalRef.input.computeStorePath(*state.store)

This is checking to make sure the input's store path is content addressable, which will not be true if the input is the output of a derivation.

@jade > It implies that we can have a fetcher grab an input (and do so successfully) with a different narHash attribute than the one in originalRef.input This is not what this assertion checks. `!originalRef.input.getNarHash()` is false, because this input has a narHash, so it is actually this part that fails: `tree.storePath == originalRef.input.computeStorePath(*state.store)` This is checking to make sure the input's store path is content addressable, which will not be true if the input is the output of a derivation.
Author

And normally nix copies the store path if you use the output of a derivation as input, so the result of the copy will have a content addressable store path. That is unless you name the derivation "source". Basically what I already explained in my OP.

And normally nix copies the store path if you use the output of a derivation as input, so the result of the copy will have a content addressable store path. That is _unless_ you name the derivation `"source"`. Basically what I already explained in my OP.
jade added the
Area/flakes
Area/fetching
labels 2024-10-20 01:03:19 +00:00
Owner

Hm. It's an input-addressed derivation that is used as a flake input? That's creative. Yeah I can imagine nobody tried that yet.

Hm. It's an input-addressed derivation that is used as a flake input? That's creative. Yeah I can imagine nobody tried that yet.
Author

That's creative.

Thanks XD

For context I have a bunch of patches on top of upstream nixpkgs, which is applied with applyPatches and the result stored in nix store. I use the same patched version of nixpkgs system wide, by putting it in registry.json, as well as NIX_PATH.

This works, but every time I evaluate something, nix copies the patched version of nixpkgs from store to store, which is slow, and feels dumb. Then I discovered, by reading the code, I can sort of avoid that by naming the patched nixpkgs derivation "source". Except doing so sometimes triggers this assertion failure.

> That's creative. Thanks XD For context I have a bunch of patches on top of upstream nixpkgs, which is applied with `applyPatches` and the result stored in nix store. I use the same patched version of nixpkgs system wide, by putting it in `registry.json`, as well as `NIX_PATH`. This works, but every time I evaluate something, nix copies the patched version of nixpkgs from store to store, which is slow, and feels dumb. Then I discovered, by reading the code, I can sort of avoid that by naming the patched nixpkgs derivation "source". Except doing so sometimes triggers this assertion failure.
jade added the
crash 💥
label 2024-11-10 02:18:32 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: lix-project/lix#529
No description provided.