lix-project/lix
22
83
Fork 57
Code Issues 522 Code Review (Gerrit) Projects 8 Releases Packages Wiki Activity

Figure out what circumstances lead to nix~case~hack~1 getting into a NAR #726

New issue
Closed
opened 2025-03-11 22:45:14 +00:00 by jade · 11 comments
jade commented 2025-03-11 22:45:14 +00:00
Owner
Copy link
error: NAR contains invalid file name 'pod~nix~case~hack~1'
error: builder for '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv' failed with exit code 1

I was building a NixOS configuration on macOS at work with this command:

nix build -L --builders @$HOME/.config/nix/machines .#nixosConfigurations.redacted.config.system.build.toplevel

It reproduces with the following command from a Mac once you do have that drv file on your machine (and the package exists on cache.nixos.org):

nb --max-jobs 0 --builders @$HOME/.config/nix/machines '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv^*' --debug

Salient debug output:

building of '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv^*' from .drv file: trying to build
locking path '/nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env'
lock acquired on '/nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env.lock'
removing invalid path '/nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env'
starting build hook '/nix/store/jwlpchd4c701hrkqlinhcbsixwh861nh-lix-2.93.0-dev-pre20250311-a60a362/bin/nix __build-remote'
got 2 remote builders
considering building on remote machine 'ssh-ng://nix-remote-build@thinnix'
considering building on remote machine 'ssh-ng://nix-remote-build@nucury'
connecting to 'ssh-ng://nix-remote-build@nucury'...
Running 'ssh' '-O' 'check' 'nix-remote-build@nucury' '-i' '/Users/jade/.ssh/id_remotebuild' '-oPermitLocalCommand=yes' '-oLocalCo
mmand=echo started'
hook reply is 'accept'
building '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv' on 'ssh-ng://nix-remote-build@nucury'...
waiting for the upload lock to 'ssh-ng://nix-remote-build@nucury'...
copying dependencies to 'ssh-ng://nix-remote-build@nucury'...
copying 0 paths...
copying outputs from 'ssh-ng://nix-remote-build@nucury'...
copying 1 paths...
copying path '/nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env' from 'ssh-ng://nix-remote-build@nucury'...
acquired write lock on '/nix/var/nix/temproots/12302'
case collision between 'Pod' and 'pod'
killing process 12304
error: NAR contains invalid file name 'pod~nix~case~hack~1'
building of '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv^*' from .drv file: build done
killing process 12302
builder process for '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv' finished
lock released on '/nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env.lock'
building of '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv^*' from .drv file: done
building of '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv^*' from .drv file: goal destroyed
error: builder for '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv' failed with exit code 1

I am wondering if use-case-hack got sent to the remote which then generated a garbage NAR???! I have no god damn idea.

Remote is 2.93.0-dev-pre20250228-99bc686, local is 2.93.0-dev-pre20250311-a60a362.

Package contents:

nucury$ tree /nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env
/nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env
├── bin
│   ├── corelist
SNIP
│   └── zipdetails
├── lib
│   └── perl5
│       ├── 5.40.0
│       │   ├── AnyDBM_File.pm -> /nix/store/ygz7vmxcmm3p9md71ck883qdlkfi7c62-perl-5.40.0/lib/perl5/5.40.0/AnyDBM_File.pm
SNIP

HERE

│       │   ├── pod -> /nix/store/ygz7vmxcmm3p9md71ck883qdlkfi7c62-perl-5.40.0/lib/perl5/5.40.0/pod
│       │   ├── Pod -> /nix/store/ygz7vmxcmm3p9md71ck883qdlkfi7c62-perl-5.40.0/lib/perl5/5.40.0/Pod
``` error: NAR contains invalid file name 'pod~nix~case~hack~1' error: builder for '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv' failed with exit code 1 ``` I was building a NixOS configuration on macOS at work with this command: ``` nix build -L --builders @$HOME/.config/nix/machines .#nixosConfigurations.redacted.config.system.build.toplevel ``` It reproduces with the following command from a Mac once you do have that drv file on your machine (and the package exists on cache.nixos.org): ``` nb --max-jobs 0 --builders @$HOME/.config/nix/machines '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv^*' --debug ``` Salient debug output: ``` building of '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv^*' from .drv file: trying to build locking path '/nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env' lock acquired on '/nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env.lock' removing invalid path '/nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env' starting build hook '/nix/store/jwlpchd4c701hrkqlinhcbsixwh861nh-lix-2.93.0-dev-pre20250311-a60a362/bin/nix __build-remote' got 2 remote builders considering building on remote machine 'ssh-ng://nix-remote-build@thinnix' considering building on remote machine 'ssh-ng://nix-remote-build@nucury' connecting to 'ssh-ng://nix-remote-build@nucury'... Running 'ssh' '-O' 'check' 'nix-remote-build@nucury' '-i' '/Users/jade/.ssh/id_remotebuild' '-oPermitLocalCommand=yes' '-oLocalCo mmand=echo started' hook reply is 'accept' building '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv' on 'ssh-ng://nix-remote-build@nucury'... waiting for the upload lock to 'ssh-ng://nix-remote-build@nucury'... copying dependencies to 'ssh-ng://nix-remote-build@nucury'... copying 0 paths... copying outputs from 'ssh-ng://nix-remote-build@nucury'... copying 1 paths... copying path '/nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env' from 'ssh-ng://nix-remote-build@nucury'... acquired write lock on '/nix/var/nix/temproots/12302' case collision between 'Pod' and 'pod' killing process 12304 error: NAR contains invalid file name 'pod~nix~case~hack~1' building of '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv^*' from .drv file: build done killing process 12302 builder process for '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv' finished lock released on '/nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env.lock' building of '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv^*' from .drv file: done building of '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv^*' from .drv file: goal destroyed error: builder for '/nix/store/vwy0kr2da1q0g0f31ry2v14wdrlshyi4-perl-5.40.0-env.drv' failed with exit code 1 ``` I am wondering if `use-case-hack` got sent to the remote which then generated a garbage NAR???! I have no god damn idea. Remote is 2.93.0-dev-pre20250228-99bc686, local is 2.93.0-dev-pre20250311-a60a362. Package contents: ``` nucury$ tree /nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env /nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env ├── bin │   ├── corelist SNIP │   └── zipdetails ├── lib │   └── perl5 │   ├── 5.40.0 │   │   ├── AnyDBM_File.pm -> /nix/store/ygz7vmxcmm3p9md71ck883qdlkfi7c62-perl-5.40.0/lib/perl5/5.40.0/AnyDBM_File.pm SNIP HERE │   │   ├── pod -> /nix/store/ygz7vmxcmm3p9md71ck883qdlkfi7c62-perl-5.40.0/lib/perl5/5.40.0/pod │   │   ├── Pod -> /nix/store/ygz7vmxcmm3p9md71ck883qdlkfi7c62-perl-5.40.0/lib/perl5/5.40.0/Pod ```
jade commented 2025-03-11 22:48:45 +00:00
Author
Owner
Copy link

Okay I have verified on remote:

nix-store --use-case-hack --export /nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env > foo

does NOT contain any case hacks.

Okay I have verified on remote: ``` nix-store --use-case-hack --export /nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env > foo ``` does NOT contain any case hacks.
jade commented 2025-03-11 22:49:30 +00:00
Author
Owner
Copy link

Hypothesis: somehow the case hacked filename ban applies too late?

Hypothesis: somehow the case hacked filename ban applies too late?
jade commented 2025-03-11 22:53:12 +00:00
Author
Owner
Copy link

Shorter reproducer:

nix copy --from ssh-ng://linux-box /nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env --to $PWD/store
Shorter reproducer: ``` nix copy --from ssh-ng://linux-box /nix/store/jw8wr2cdqmnwrgk7kfa46w8mmrn94qyq-perl-5.40.0-env --to $PWD/store ```
jade commented 2025-03-11 23:33:43 +00:00
Author
Owner
Copy link

Okay, the case hack is being applied locally, but why is it getting into a NAR?

Stack trace at the debug("case collision between '%1%' and '%2%'", i->first, name);:

(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 7.1
  * frame #0: 0x0000000100d3c9dc liblixutil.dylib`nix::nar::(anonymous namespace)::Parser::parse(this=<unavailable>, completed=<unavailabl
e>)::'lambda'(bool&)::operator()(bool&) const at archive.cc:459:29 [opt]
    frame #1: 0x0000000100d3e6ec liblixutil.dylib`nix::nar::(anonymous namespace)::SyncParser::readDir(nix::Generator<std::__1::variant<ni
x::nar::(anonymous namespace)::Parser::WantBytes, std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::alloc
ator<char>> const&, nix::Generator<std::__1::variant<nix::nar::(anonymous namespace)::Parser::FileHeader, nix::nar::(anonymous namespace):
:Parser::Symlink, nix::nar::(anonymous namespace)::Parser::Directory, nix::nar::(anonymous namespace)::Parser::WantBytes>, void>>>, void>)
 [inlined] std::__1::coroutine_handle<void>::resume[abi:v160006](this=0x000000012c805c50) const at coroutine_handle.h:79:9 [opt]
    frame #2: 0x0000000100d3e6dc liblixutil.dylib`nix::nar::(anonymous namespace)::SyncParser::readDir(nix::Generator<std::__1::variant<ni
x::nar::(anonymous namespace)::Parser::WantBytes, std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::alloc
ator<char>> const&, nix::Generator<std::__1::variant<nix::nar::(anonymous namespace)::Parser::FileHeader, nix::nar::(anonymous namespace):
:Parser::Symlink, nix::nar::(anonymous namespace)::Parser::Directory, nix::nar::(anonymous namespace)::Parser::WantBytes>, void>>>, void>)
 at generator.hh:151:27 [opt]
    frame #3: 0x0000000100d3e6d4 liblixutil.dylib`nix::nar::(anonymous namespace)::SyncParser::readDir(nix::Generator<std::__1::variant<ni
x::nar::(anonymous namespace)::Parser::WantBytes, std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::alloc
ator<char>> const&, nix::Generator<std::__1::variant<nix::nar::(anonymous namespace)::Parser::FileHeader, nix::nar::(anonymous namespace):
:Parser::Symlink, nix::nar::(anonymous namespace)::Parser::Directory, nix::nar::(anonymous namespace)::Parser::WantBytes>, void>>>, void>)
 [inlined] nix::Generator<std::__1::variant<nix::nar::(anonymous namespace)::Parser::WantBytes, std::__1::pair<std::__1::basic_string<char
, std::__1::char_traits<char>, std::__1::allocator<char>> const&, nix::Generator<std::__1::variant<nix::nar::(anonymous namespace)::Parser
::FileHeader, nix::nar::(anonymous namespace)::Parser::Symlink, nix::nar::(anonymous namespace)::Parser::Directory, nix::nar::(anonymous n
amespace)::Parser::WantBytes>, void>>>, void>::next(this=0x000000012c805c50) at generator.hh:267:21 [opt]
    frame #4: 0x0000000100d3e6d4 liblixutil.dylib`nix::nar::(anonymous namespace)::SyncParser::readDir(this=<unavailable>, stream=nix::nar
::(anonymous namespace)::Parser::Directory::Stream @ 0x000000012c805f10) at archive.cc:541:32 [opt]
    frame #5: 0x0000000100d36cfc liblixutil.dylib`nix::dumpSingle(nix::nar::Directory) [inlined] std::__1::coroutine_handle<void>::resume[
abi:v160006](this=0x000000012c805dd0) const at coroutine_handle.h:79:9 [opt]
    frame #6: 0x0000000100d36cec liblixutil.dylib`nix::dumpSingle(nix::nar::Directory) at generator.hh:151:27 [opt]
    frame #7: 0x0000000100d36ce4 liblixutil.dylib`nix::dumpSingle(nix::nar::Directory) [inlined] nix::Generator<std::__1::pair<std::__1::b
asic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::variant<nix::nar::File, nix::nar::Symlink, nix
::nar::Directory>>, void>::next(this=0x000000012c805dd0) at generator.hh:267:21 [opt]
    frame #8: 0x0000000100d36ce4 liblixutil.dylib`nix::dumpSingle(d=Directory @ 0x0000600002ed8390) at archive.cc:88:32 [opt]
    frame #9: 0x00000001018139cc liblixstore.dylib`nix::GeneratorSource::read(char*, unsigned long) [inlined] std::__1::coroutine_handle<v
oid>::resume[abi:v160006](this=0x0000600001fce8f0) const at coroutine_handle.h:79:9 [opt]
    frame #10: 0x00000001018139bc liblixstore.dylib`nix::GeneratorSource::read(char*, unsigned long) at generator.hh:151:27 [opt]
    frame #11: 0x00000001018139b4 liblixstore.dylib`nix::GeneratorSource::read(char*, unsigned long) [inlined] nix::Generator<std::__1::sp
an<char const, 18446744073709551615ul>, void>::next(this=<unavailable>) at generator.hh:267:21 [opt]
    frame #12: 0x00000001018139b4 liblixstore.dylib`nix::GeneratorSource::read(this=0x0000600001fce8e0, data="", len=8) at serialise.hh:33
7:31 [opt]
    frame #13: 0x0000000100d590c4 liblixutil.dylib`nix::AsyncSourceInputStream::read(this=0x00006000011fecc0, buffer=0x0000600001fd40f0, s
ize=8) at async-io.cc:28:30 [opt]
    frame #14: 0x0000000101a444c0 liblixstore.dylib`nix::(anonymous namespace)::CopyPathStream::read(this=0x00006000032d72f0, data=<unavai
lable>, len=<unavailable>) at store-api.cc:1108:23 [opt]
    frame #15: 0x0000000100d59494 liblixutil.dylib`nix::AsyncTeeInputStream::read(this=0x000000012d00fc98, buffer=0x0000600001fd40f0, size
=<unavailable>) at async-io.cc:50:16 [opt]
    frame #16: 0x0000000100d318cc liblixutil.dylib`nix::nar::(anonymous namespace)::AsyncParser::read(this=<unavailable>, buffer="", n=8) 
at archive.cc:684:24 [opt]
    frame #17: 0x0000000100d40664 liblixutil.dylib`nix::nar::(anonymous namespace)::AsyncParser::parse(nix::Generator<std::__1::variant<ni
x::nar::(anonymous namespace)::Parser::FileHeader, nix::nar::(anonymous namespace)::Parser::Symlink, nix::nar::(anonymous namespace)::Pars
er::Directory, nix::nar::(anonymous namespace)::Parser::WantBytes>, void>&, nix::NARParseVisitor&, std::__1::basic_string<char, std::__1::
char_traits<char>, std::__1::allocator<char>> const&) [inlined] nix::nar::(anonymous namespace)::AsyncParser::feed(this=0x000000012d0101d0
, n=8) at archive.cc:700:16 [opt]
    frame #18: 0x0000000100d40634 liblixutil.dylib`nix::nar::(anonymous namespace)::AsyncParser::parse(this=<unavailable>, stream=<unavail
able>, target=<unavailable>, name=<unavailable>) at archive.cc:726:25 [opt]
Okay, the case hack is being applied locally, but why is it getting *into* a NAR? Stack trace at the `debug("case collision between '%1%' and '%2%'", i->first, name);`: ``` (lldb) bt * thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 7.1 * frame #0: 0x0000000100d3c9dc liblixutil.dylib`nix::nar::(anonymous namespace)::Parser::parse(this=<unavailable>, completed=<unavailabl e>)::'lambda'(bool&)::operator()(bool&) const at archive.cc:459:29 [opt] frame #1: 0x0000000100d3e6ec liblixutil.dylib`nix::nar::(anonymous namespace)::SyncParser::readDir(nix::Generator<std::__1::variant<ni x::nar::(anonymous namespace)::Parser::WantBytes, std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::alloc ator<char>> const&, nix::Generator<std::__1::variant<nix::nar::(anonymous namespace)::Parser::FileHeader, nix::nar::(anonymous namespace): :Parser::Symlink, nix::nar::(anonymous namespace)::Parser::Directory, nix::nar::(anonymous namespace)::Parser::WantBytes>, void>>>, void>) [inlined] std::__1::coroutine_handle<void>::resume[abi:v160006](this=0x000000012c805c50) const at coroutine_handle.h:79:9 [opt] frame #2: 0x0000000100d3e6dc liblixutil.dylib`nix::nar::(anonymous namespace)::SyncParser::readDir(nix::Generator<std::__1::variant<ni x::nar::(anonymous namespace)::Parser::WantBytes, std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::alloc ator<char>> const&, nix::Generator<std::__1::variant<nix::nar::(anonymous namespace)::Parser::FileHeader, nix::nar::(anonymous namespace): :Parser::Symlink, nix::nar::(anonymous namespace)::Parser::Directory, nix::nar::(anonymous namespace)::Parser::WantBytes>, void>>>, void>) at generator.hh:151:27 [opt] frame #3: 0x0000000100d3e6d4 liblixutil.dylib`nix::nar::(anonymous namespace)::SyncParser::readDir(nix::Generator<std::__1::variant<ni x::nar::(anonymous namespace)::Parser::WantBytes, std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::alloc ator<char>> const&, nix::Generator<std::__1::variant<nix::nar::(anonymous namespace)::Parser::FileHeader, nix::nar::(anonymous namespace): :Parser::Symlink, nix::nar::(anonymous namespace)::Parser::Directory, nix::nar::(anonymous namespace)::Parser::WantBytes>, void>>>, void>) [inlined] nix::Generator<std::__1::variant<nix::nar::(anonymous namespace)::Parser::WantBytes, std::__1::pair<std::__1::basic_string<char , std::__1::char_traits<char>, std::__1::allocator<char>> const&, nix::Generator<std::__1::variant<nix::nar::(anonymous namespace)::Parser ::FileHeader, nix::nar::(anonymous namespace)::Parser::Symlink, nix::nar::(anonymous namespace)::Parser::Directory, nix::nar::(anonymous n amespace)::Parser::WantBytes>, void>>>, void>::next(this=0x000000012c805c50) at generator.hh:267:21 [opt] frame #4: 0x0000000100d3e6d4 liblixutil.dylib`nix::nar::(anonymous namespace)::SyncParser::readDir(this=<unavailable>, stream=nix::nar ::(anonymous namespace)::Parser::Directory::Stream @ 0x000000012c805f10) at archive.cc:541:32 [opt] frame #5: 0x0000000100d36cfc liblixutil.dylib`nix::dumpSingle(nix::nar::Directory) [inlined] std::__1::coroutine_handle<void>::resume[ abi:v160006](this=0x000000012c805dd0) const at coroutine_handle.h:79:9 [opt] frame #6: 0x0000000100d36cec liblixutil.dylib`nix::dumpSingle(nix::nar::Directory) at generator.hh:151:27 [opt] frame #7: 0x0000000100d36ce4 liblixutil.dylib`nix::dumpSingle(nix::nar::Directory) [inlined] nix::Generator<std::__1::pair<std::__1::b asic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::variant<nix::nar::File, nix::nar::Symlink, nix ::nar::Directory>>, void>::next(this=0x000000012c805dd0) at generator.hh:267:21 [opt] frame #8: 0x0000000100d36ce4 liblixutil.dylib`nix::dumpSingle(d=Directory @ 0x0000600002ed8390) at archive.cc:88:32 [opt] frame #9: 0x00000001018139cc liblixstore.dylib`nix::GeneratorSource::read(char*, unsigned long) [inlined] std::__1::coroutine_handle<v oid>::resume[abi:v160006](this=0x0000600001fce8f0) const at coroutine_handle.h:79:9 [opt] frame #10: 0x00000001018139bc liblixstore.dylib`nix::GeneratorSource::read(char*, unsigned long) at generator.hh:151:27 [opt] frame #11: 0x00000001018139b4 liblixstore.dylib`nix::GeneratorSource::read(char*, unsigned long) [inlined] nix::Generator<std::__1::sp an<char const, 18446744073709551615ul>, void>::next(this=<unavailable>) at generator.hh:267:21 [opt] frame #12: 0x00000001018139b4 liblixstore.dylib`nix::GeneratorSource::read(this=0x0000600001fce8e0, data="", len=8) at serialise.hh:33 7:31 [opt] frame #13: 0x0000000100d590c4 liblixutil.dylib`nix::AsyncSourceInputStream::read(this=0x00006000011fecc0, buffer=0x0000600001fd40f0, s ize=8) at async-io.cc:28:30 [opt] frame #14: 0x0000000101a444c0 liblixstore.dylib`nix::(anonymous namespace)::CopyPathStream::read(this=0x00006000032d72f0, data=<unavai lable>, len=<unavailable>) at store-api.cc:1108:23 [opt] frame #15: 0x0000000100d59494 liblixutil.dylib`nix::AsyncTeeInputStream::read(this=0x000000012d00fc98, buffer=0x0000600001fd40f0, size =<unavailable>) at async-io.cc:50:16 [opt] frame #16: 0x0000000100d318cc liblixutil.dylib`nix::nar::(anonymous namespace)::AsyncParser::read(this=<unavailable>, buffer="", n=8) at archive.cc:684:24 [opt] frame #17: 0x0000000100d40664 liblixutil.dylib`nix::nar::(anonymous namespace)::AsyncParser::parse(nix::Generator<std::__1::variant<ni x::nar::(anonymous namespace)::Parser::FileHeader, nix::nar::(anonymous namespace)::Parser::Symlink, nix::nar::(anonymous namespace)::Pars er::Directory, nix::nar::(anonymous namespace)::Parser::WantBytes>, void>&, nix::NARParseVisitor&, std::__1::basic_string<char, std::__1:: char_traits<char>, std::__1::allocator<char>> const&) [inlined] nix::nar::(anonymous namespace)::AsyncParser::feed(this=0x000000012d0101d0 , n=8) at archive.cc:700:16 [opt] frame #18: 0x0000000100d40634 liblixutil.dylib`nix::nar::(anonymous namespace)::AsyncParser::parse(this=<unavailable>, stream=<unavail able>, target=<unavailable>, name=<unavailable>) at archive.cc:726:25 [opt] ```
jade commented 2025-03-12 00:04:22 +00:00
Author
Owner
Copy link

CAUGHT IT IN THE ACT:

nix store dump-path --store ssh-ng://linuxbox /nix/store/ygz7vmxcmm3p9md71ck883qdlkfi7c62-perl-5.40.0 > foo.nar
CAUGHT IT IN THE ACT: ``` nix store dump-path --store ssh-ng://linuxbox /nix/store/ygz7vmxcmm3p9md71ck883qdlkfi7c62-perl-5.40.0 > foo.nar ```
jade commented 2025-03-12 00:10:16 +00:00
Author
Owner
Copy link

Bug does not repro with --no-use-case-hack on the store dump-path command above. Need to confirm whether that is making it to the remote daemon or what is the deal with that.

Bug does not repro with `--no-use-case-hack` on the `store dump-path` command above. Need to confirm whether that is making it to the remote daemon or what is the deal with that.
pennae commented 2025-03-12 00:11:01 +00:00
Owner
Copy link

oh, i see what's happening. it's copyNAR, because all copies of nars must parse the nar. and if the thing doing the parsing has case hacking turned on it'll hack the stream

oh, i see what's happening. it's copyNAR, because all copies of nars must parse the nar. and if the thing doing the parsing has case hacking turned on it'll hack the stream
jade commented 2025-03-12 00:13:01 +00:00
Author
Owner
Copy link

Okay. It is not related to settings on remote, the corruption is happening on the local machine, not the remote. I confirmed this by patching out the case-hack adding from the local nix. I don't know what lix is doing that leads to the NAR visitor of parsing one NAR being used to construct a NAR, but it is definitely the pattern I put that filename check in to ensure it doesn't happen as defense in depth.

Okay. It is not related to settings on remote, the corruption is happening on the local machine, not the remote. I confirmed this by patching out the case-hack adding from the local `nix`. I don't know what lix is doing that leads to the NAR visitor of parsing one NAR being used to construct a NAR, but it is definitely the pattern I put that filename check in to ensure it doesn't happen as defense in depth.
pennae commented 2025-03-12 00:14:06 +00:00
Owner
Copy link

Bug does not repro with --no-use-case-hack on the store dump-path command above. Need to confirm whether that is making it to the remote daemon or what is the deal with that.

this is purely local to your machine. the nar lister unhacks files, but the dumper does not, so if the parser hacks a nar stream copyNAR will spit it out all fucked-up-like. check out 45be1e8d66

> Bug does not repro with --no-use-case-hack on the store dump-path command above. Need to confirm whether that is making it to the remote daemon or what is the deal with that. this is purely local to your machine. the nar *lister* unhacks files, but the *dumper* does not, so if the parser hacks a nar stream copyNAR will spit it out all fucked-up-like. check out 45be1e8d668a56c2e2c9b9d6ec47e40d4c97f165
jade commented 2025-03-12 00:17:43 +00:00
Author
Owner
Copy link

The design of the case hack setting is so fucking sketchy and definitely made it much easier for this bug to happen; we should hoist it (or maybe move the entire case hacking into the FS visitor). I can work on a CL to move the case hacking code to the FS side to hopefully make this much less likely in the future.

See also: #633

The design of the case hack setting is so fucking sketchy and definitely made it much easier for this bug to happen; we should hoist it (or maybe move the entire case hacking into the FS visitor). I can work on a CL to move the case hacking code to the FS side to hopefully make this much less likely in the future. See also: https://git.lix.systems/lix-project/lix/issues/633
jade self-assigned this 2025-03-12 17:41:59 +00:00
lix-bot commented 2025-03-12 20:33:43 +00:00
Member
Copy link

This issue was mentioned on Gerrit on the following CLs:

  • commit message in cl/2794 ("libstore: move case hacking to the FS interaction code")
<!-- GERRIT_LINKBOT: {"cls": [{"backlink": "https://gerrit.lix.systems/c/lix/+/2794", "number": 2794, "kind": "commit message"}], "cl_meta": {"2794": {"change_title": "libstore: move case hacking to the FS interaction code"}}} --> This issue was mentioned on Gerrit on the following CLs: * commit message in [cl/2794](https://gerrit.lix.systems/c/lix/+/2794) ("libstore: move case hacking to the FS interaction code")
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
sb/raito/flakes-out-of-core
sb/raito/eval-explorations
release-2.93
release-2.94
ci-config
sb/pennae/ci-config
release-2.91
release-2.92
sb/raito/darwin-kqueue-disablement
sb/raito/hydra-merge
sb/raito/phantom-referrers-gc
repl-overlay-example
reuse-compliance
release-2.90
sb/pennae/io-rewrite
git-series/activity-cleanup
sb/tom-hubrecht/primops.cc-v2
sb/bacchanalia/cache-strlen%topic=substr-perf
sb/qyriad/beta-1
sb/qyriad/2.90-beta.0
sb/rbt/test-branch
sb/rbt/justfile
sb/rbt/repl-overlays
sb/qyriad/maint-meson
sb/qyriad/maint/meson
sb/puck/meson-c-api
sb/jade/burn-it-all
dependabot/github_actions/zeebe-io/backport-action-2.4.1
dependabot/github_actions/cachix/install-nix-action-25
dependabot/github_actions/actions/labeler-5
dependabot/github_actions/cachix/cachix-action-14
2.94.0
2.93.3
2.92.3
2.93.2
2.91.3
2.92.2
2.91.2
2.93.1
2.93.0
2.92.0
2.91.1
2.91.0
2.90.0
2.90.0-rc1
2.90-beta.1
2.90-beta.0
Labels
Clear labels   
Affects/CppNix

Probably affects CppNix. This is a best-effort tag for troubles in old code; it exists to inform the CppNix team about problems they might care about also.

  
Affects/Nightly

Found in Lix nightly.

  
Affects/Only nightly

Tag for regressions in Lix nightly. We intend to fix all of these prior to releases.

  
Affects/Stable

Affects the stable release of Lix. Usually such issues affect nightly too but there is no guarantee.

  
Area/build-packaging

regarding the building or packaging of Lix itself

  
Area/cli

The command line interface of Lix

  
Area/evaluator

libexpr and anything else related to the evaluation of Nix expressions

  
Area/fetching

Fetching sources or other materials from the World Wide Web in Lix

  
Area/flakes

   
Area/language

language design, features, extensions, etc

  
Area/lix ci

Lix's own CI and deployment pipeline

  
Area/nix-eval-jobs

nix-eval-jobs for Lix

  
Area/profiles

profiles: nix-env, nix profile, and installer flavors all alike

  
Area/protocol

The protocol, used for communicating between the client and the daemon

  
Area/releng

Release engineering

  
Area/remote-builds

remote builds

  
Area/repl

nix repl

  
Area/repl/debugger

The --debugger that brings up a Lix repl

  
Area/store

libstore and anything else relating to the Nix store

  
awaiting
author
needs more information from the issue author

  
awaiting
contributors
needs more information from contributors

  
bug

it's a bug

  
Context
contributors
This issue might require some context that isn't written down. It might be possible to handle as a person who already worked on this area.

  
Context
drive-by
This issue contains all the necessary information to do a drive-by contribution: that is, you don't have to talk with anyone at all to successfully solve it!

  
Context
maintainers
This issue requires a lot of know-how. Attention from maintainers is needed to proceed.

  
Context
RFD
This issue needs discussion from multiple maintainers and interested parties: it's a Request For Discussion.

  
crash 💥

Lix crashes (possibly bad error handling)

  
Cross Compilation

   
devx

development experience issues

  
docs

requires or is about documentation changes

  
Downstream Dependents

regarding an external tool's usage with Lix (e.g. nix-eval-jobs)

  
E/easy

bug is relatively simple and could simply be fixed with some minor effort

  
E/hard

this issue should probably be solved with close involvement of an expert

  
E/help wanted

maintainers may not get to this soon, but we want it fixed

  
E/reproducible

This bug is reproducible and obvious.

  
E/requires rearchitecture

this bug cannot be fixed properly without fixing a known architectural flaw

  
Feature/S3

S3 support in Lix

  
imported

Issue imported from upstream nix (please do not remove this label, it will cause dupes if import is rerun)

  
Language/Bash

   
Language/C++

   
Language/NixLang

   
Language/Python

   
Language/Rust

   
Needs Langver

We can't get to this until we implement language versioning

  
OS/Linux

Affects specifically Lix on Linux

  
OS/macOS

Affects specifically Lix on macOS

  
performance

make lix go faster

  
regression

This is a regression since some previous version of Lix

  
release-blocker

Blocks public release

  
stability

Fixing this makes the software more stable

  
Status
blocked
Blocked on something

  
Status
invalid
not our bug, duplicate, similar

  
Status
postponed
closed but should maybe look at later (generally with S-wontfix)

  
Status
wontfix
We chose not to change this (see issue comments for more detail).

  
testing

Tests that should be added

  
testing/flakey

Tests that are unreliable/flakey (not to be confused with flakes that test)

  
Topic/Large Scale Installations

Large scale installations of Lix, e.g. NixOS Hydra, Lix CI, big corporate installations etc.

  
ux

makes the user experience better :)

No labels
Affects/CppNix
Affects/Nightly
Affects/Only nightly
Affects/Stable
Area/build-packaging
Area/cli
Area/evaluator
Area/fetching
Area/flakes
Area/language
Area/lix ci
Area/nix-eval-jobs
Area/profiles
Area/protocol
Area/releng
Area/remote-builds
Area/repl
Area/repl/debugger
Area/store
awaiting
author
awaiting
contributors
bug
Context
contributors
Context
drive-by
Context
maintainers
Context
RFD
crash 💥
Cross Compilation
devx
docs
Downstream Dependents
E/easy
E/hard
E/help wanted
E/reproducible
E/requires rearchitecture
Feature/S3
imported
Language/Bash
Language/C++
Language/NixLang
Language/Python
Language/Rust
Needs Langver
OS/Linux
OS/macOS
performance
regression
release-blocker
stability
Status
blocked
Status
invalid
Status
postponed
Status
wontfix
testing
testing/flakey
Topic/Large Scale Installations
ux
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Case hack removal
Assignees
Clear assignees
No assignees
jade
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lix-project/lix#726
No description provided.