`repl-overlays` can cause suffering by "environment full" #337

New issue

Closed

opened 2024-05-22 22:49:52 +00:00 by jade · 4 comments

jade commented

2024-05-22 22:49:52 +00:00

Owner

cc @rbt

dev/nixpkgs » nix repl --repl-overlays <(echo '{ currentSystem, ...}: final: prev: { }') -f '<nixpkgs>'
Lix 2.90.0-lixpre20240520-992c63f
Type :? for help.
Loading installable ''...
Added 21362 variables.
Loading 'repl-overlays'...
error: environment full; cannot add more variables

cc @rbt ``` dev/nixpkgs » nix repl --repl-overlays <(echo '{ currentSystem, ...}: final: prev: { }') -f '<nixpkgs>' Lix 2.90.0-lixpre20240520-992c63f Type :? for help. Loading installable ''... Added 21362 variables. Loading 'repl-overlays'... error: environment full; cannot add more variables ```

jade added the

bug

label 2024-05-22 22:49:52 +00:00

pennae commented

2024-05-22 22:52:29 +00:00

Owner

easy fix: just increase the static env size to something more reasonable, like 1 << 20 (each entry is 8 bytes, let's just send it?)

easy fix: just increase the [static env size](https://git.lix.systems/lix-project/lix/src/branch/main/src/libcmd/repl.cc#L93) to something more reasonable, like `1 << 20` (each entry is 8 bytes, let's just send it?)

jade commented

2024-05-22 23:15:42 +00:00

Author

Owner

Sure, but I think this is a real bug:

(gdb) bt
#0  nix::NixRepl::addAttrsToScope (this=this@entry=0x7fffdcb8fea0, attrs=...)
    at src/libcmd/repl.cc:938
#1  0x00007ffff7d383b0 in nix::NixRepl::loadReplOverlays (
    this=this@entry=0x7fffdcb8fea0) at src/libcmd/repl.cc:857
#2  0x00007ffff7d38915 in nix::NixRepl::loadFiles (this=this@entry=0x7fffdcb8fea0)
    at src/libcmd/repl.cc:835
#3  0x00007ffff7d3bc30 in nix::NixRepl::mainLoop (this=0x7fffdcb8fea0)
    at src/libcmd/repl.cc:258
#4  0x000000000058b163 in nix::CmdRepl::run(nix::ref<nix::Store>, std::vector<std::__cx
x11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<
std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&&)
    ()
#5  0x00007ffff7d1cc1b in nix::RawInstallablesCommand::run (this=0x770480, store=...)
    at src/libcmd/installables.cc:815
#6  0x00007ffff7cf30b7 in nix::StoreCommand::run (this=0x770730)
    at src/libcmd/command.cc:55
#7  0x000000000054d0b6 in nix::mainWrapped(int, char**) ()
#8  0x00007ffff7faa0ea in std::function<void ()>::operator()() const (
    this=0x7ffffffed7b0)
    at /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/bits/s
td_function.h:591
#9  nix::handleExceptions(std::__cxx11::basic_string<char, std::char_traits<char>, std:
:allocator<char> > const&, std::function<void ()>) (
    programName="/nix/store/mck2gzfldi692qfgygr0wn94ffysdxjv-nix-2.90.0pre20240522_06c1
375/bin/nix", fun=...) at src/libmain/shared.cc:323
#10 0x000000000047e077 in main ()
(gdb) list
933         return info;
934     }
935
936
937     void NixRepl::addAttrsToScope(Value & attrs)
938     {
939         state->forceAttrs(attrs, [&]() { return attrs.determinePos(noPos); }, "whil
e evaluating an attribute set to be merged in the global scope");
940         if (displ + attrs.attrs->size() >= envSize)
941             throw Error("environment full; cannot add more variables");
942
(gdb) p displ
$1 = 21362
(gdb) p attrs.attrs
$2 = (nix::Bindings *) 0x7fffdf3a6000
(gdb) p *attrs.attrs
$3 = {pos = {id = 0}, size_ = 21362, capacity_ = 21362, attrs = 0x7fffdf3a6010}
(gdb) p envSize
$4 = 32768

Sure, but I think this is a real bug: ``` (gdb) bt #0 nix::NixRepl::addAttrsToScope (this=this@entry=0x7fffdcb8fea0, attrs=...) at src/libcmd/repl.cc:938 #1 0x00007ffff7d383b0 in nix::NixRepl::loadReplOverlays ( this=this@entry=0x7fffdcb8fea0) at src/libcmd/repl.cc:857 #2 0x00007ffff7d38915 in nix::NixRepl::loadFiles (this=this@entry=0x7fffdcb8fea0) at src/libcmd/repl.cc:835 #3 0x00007ffff7d3bc30 in nix::NixRepl::mainLoop (this=0x7fffdcb8fea0) at src/libcmd/repl.cc:258 #4 0x000000000058b163 in nix::CmdRepl::run(nix::ref<nix::Store>, std::vector<std::__cx x11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator< std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&&) () #5 0x00007ffff7d1cc1b in nix::RawInstallablesCommand::run (this=0x770480, store=...) at src/libcmd/installables.cc:815 #6 0x00007ffff7cf30b7 in nix::StoreCommand::run (this=0x770730) at src/libcmd/command.cc:55 #7 0x000000000054d0b6 in nix::mainWrapped(int, char**) () #8 0x00007ffff7faa0ea in std::function<void ()>::operator()() const ( this=0x7ffffffed7b0) at /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/bits/s td_function.h:591 #9 nix::handleExceptions(std::__cxx11::basic_string<char, std::char_traits<char>, std: :allocator<char> > const&, std::function<void ()>) ( programName="/nix/store/mck2gzfldi692qfgygr0wn94ffysdxjv-nix-2.90.0pre20240522_06c1 375/bin/nix", fun=...) at src/libmain/shared.cc:323 #10 0x000000000047e077 in main () (gdb) list 933 return info; 934 } 935 936 937 void NixRepl::addAttrsToScope(Value & attrs) 938 { 939 state->forceAttrs(attrs, [&]() { return attrs.determinePos(noPos); }, "whil e evaluating an attribute set to be merged in the global scope"); 940 if (displ + attrs.attrs->size() >= envSize) 941 throw Error("environment full; cannot add more variables"); 942 (gdb) p displ $1 = 21362 (gdb) p attrs.attrs $2 = (nix::Bindings *) 0x7fffdf3a6000 (gdb) p *attrs.attrs $3 = {pos = {id = 0}, size_ = 21362, capacity_ = 21362, attrs = 0x7fffdf3a6010} (gdb) p envSize $4 = 32768 ```

jade commented

2024-05-22 23:32:45 +00:00

Author

Owner

The bug: the environment is not cleared before addAttrsToScope, so we are adding the entirety of nixpkgs to the environment twice. This is obviously busted.

The bug: the environment is not cleared before `addAttrsToScope`, so we are adding the entirety of nixpkgs to the environment twice. This is obviously busted.

pennae commented

2024-05-22 23:44:00 +00:00

Owner

it cannot be cleared because that'll break existing unforced thunks using bindings from the previous environment

jade self-assigned this 2024-05-23 00:01:16 +00:00