Commit graph

163 commits

Author SHA1 Message Date
jade 519957bd59 unnamed threads: Obliterate
Ever read gdb output and you just kinda get a headache because you have
to infer what a thread is by reading the stack trace? It's not hard, but
we could also just never have to do that again, which is also not hard.

Sample:

(gdb) info thr
  Id   Target Id                    Frame
* 1    LWP 3719283 "nix-daemon"     0x00007e558587da0f in accept ()
   from target:/nix/store/c10zhkbp6jmyh0xc5kd123ga8yy2p4hk-glibc-2.39-52/lib/libc.so.6
  2    LWP 3719284 "signal handler" 0x00007e55857b2bea in sigtimedwait ()
   from target:/nix/store/c10zhkbp6jmyh0xc5kd123ga8yy2p4hk-glibc-2.39-52/lib/libc.so.6

The API design for this is forced by the macOS pthread_setname_np only
being able to change the current thread's name, but if we just conform
everything to that, it works everywhere.

Change-Id: I2b1d6ed41e3c94170cb0b4e73ad66f239ebd9c88
2024-11-18 18:53:40 -08:00
jade 103cd44a04 Include rearrangement: rest
Change-Id: If9b5502ff348d358d7007b885e26e98a0d228f39
2024-11-18 12:31:19 -08:00
eldritch horrors 8088927b90 libexpr: initGC -> initLibExpr
we'll put more stuff in there shortly. unfortunately.

Change-Id: I74be957469c3dcd5157f999e2cf23b1c668426d8
2024-11-15 16:15:11 +01:00
eldritch horrors b22f2bc8e9 libstore: use http for exceptionAbortsDownload tests
file:// urls are no longer handled by curl itself, and this is supposed
to test the curl wrapper. use http to force the wrapper to be involved.

Change-Id: Ib13087db07b3b2f1ae44ce8e3ec7a96d935b1bab
2024-11-15 16:15:11 +01:00
Emilia Bopp a378c61948 libutil: fix non-ASCII chars in URL encoding
Due to the cast to (unsigned int), the encoding appended broken
bytes padding. This is fixed here with a bitmask.

Fixes: lix-project/lix#562
Change-Id: I0c93bd2b8c2f82df208d4693b7254544e3121dc3
2024-11-11 14:26:03 +01:00
eldritch horrors 5d02800e57 libstore: do not retry FileTransfer uploads
this only ever worked for empty uploads, and there it worked only by
complete accident: curl was asked to send more data than the wrapper
would provide, which curl would not like and report as an error. the
error would cause a retry with even less data to send, until finally
failing by running into the retry limit. let's just forbid all this.

Change-Id: I229a94b3b8b33e2c6cdb8ea19edd57cd6740e6c6
2024-11-09 20:08:48 +00:00
eldritch horrors 40be91afbf libstore: add filetranfer retry handling tests
we did not have any, despite retry handling being somewhat complex.

Change-Id: I5051a1c0a3861849ff67f512b33f6d3dda12cc95
2024-11-09 20:08:48 +00:00
Lily Ballard 3c8096e5cb feat: Add temp-dir setting
This adds a new temp-dir setting for controlling the temporary directory
without having to change the TMPDIR env var. This can be used to e.g.
use a path on a case-sensitive store on macOS for temporary files
without changing the TMPDIR var used by interactive shells or commands
invoked with `nix run`.

This also stops unsetting `TMPDIR` on darwin when the env var value
starts with `/var/folders/`, preferring instead to just do the check
when reading `TMPDIR`. This way the inherited `TMPDIR` env var is
preserved for child processes (such as interactive shells).

As a side effect this changes the behavior of `nix-build -o ''` to act
like `nix-build --no-out-link` instead of failing with an error caused
by trying to create a symlink at the cwd.

Fixes: lix-project/lix#253
Fixes: lix-project/lix#112
Change-Id: I9ee826323f2deca62854715a77ca7a373a948a29
2024-11-06 18:11:47 -08:00
eldritch horrors 4ae6fb5a8f libstore: pause only stalling transfers
don't pause the entire curl thread. we have multiple consumer threads
after all, not just one, so stalling all of them is likely not great.

note that libcurl advises against using transfer pauses if compressed
encodings are allowed and automatically decoded. this should not lead
to problems in practice because our data is usually not compressed to
such a degree that curl buffering *uncompressed* data matters. should
this cause problems we can reintroduce the whole-thread pause, but we
will probably get away with this until the entire file transfer class
is made kj::Promise-using async (and *then* curl can be hardpaused if
it cannot get rid of its data, solving the problem once and for all).

Change-Id: I218e41bfa5a27c7454eafb0bdb54f2a29a7f6493
2024-10-30 22:52:19 +00:00
eldritch horrors d65838a900 libstore: remove FileTransfer::enqueueDownload
it's no longer needed. `download` can do everything `enqueueDownload`
did, and a lot more. e.g. not block the calling thread, for instance.

Change-Id: I4b36235ed707c92d117b4c33efa3db50d26f9a84
2024-10-28 18:52:49 +00:00
eldritch horrors c68f0cdf00 libstore: return transfer metadata from download
as promised earlier. nothing uses it yet, but just you wait.

Change-Id: I77d185578d96c2134b756d20f2fcf1c02de0da6f
2024-10-28 18:52:49 +00:00
eldritch horrors 982d049d3b libstore: remove FileTransferResult::data
return it as a separate item in a pair instead. this will let us remove
enqueueDownload() in favor of returning metadata from download() itself

Change-Id: I74fad2ca15f920da1eefabc950c2baa2c360f2ba
2024-10-27 21:44:38 +00:00
eldritch horrors 6f18e1ebde libstore: remove FileTransferRequest
it's just a uri and some headers now. those can be function arguments
with no loss of clarity. *actual* additional arguments, for example a
TLS context with additional certificates, could be added on a new and
improved FileTransfer class that carries not just a backend reference
but some real, visible context for its transfers. curl not being very
multi-threading-friendly when using multi handles will make sharing a
bit hard anyway once we drop the single global download worker thread

Change-Id: Id2112c95cbd118c6d920488f38d272d7da926460
2024-10-27 21:44:38 +00:00
eldritch horrors af27d1ecd8 libstore: make baseRetryTimeMs a FileTransfer property
we don't even need this outside of tests. maybe we should not do
automatic retries at this level at all and use retrying wrappers
instead? at some point we may have to do this, but not just yet.

Change-Id: If0088aa55215be81f1770c25b3bb1b5268c65cf8
2024-10-26 21:42:35 +00:00
eldritch horrors ce3e1d1e7a libstore: remove FileTransferRequests::data
use separate upload and download methods instead.

Change-Id: I5baa2177c8ddd70268c75ff074e361b2f17dddbd
2024-10-26 21:42:35 +00:00
eldritch horrors 2d49efaa2e libstore: remove Filetransfer::transfer
just use enqueueFileTransfer().get() insteaad.

Change-Id: I67a43c9d3d5f68ac3f9e8ba7973c243dd78b86a3
2024-10-26 21:42:35 +00:00
alois31 4dbbd721eb
treewide: consistently mark overridden settings as such
Only overridden settings are sent to the daemon, and we're going to do the same
for the build hook to. It needs to be ensured that overridden settings are in
fact consistently marked as such, so that they actually get sent.

Change-Id: I7cd58d925702f86cf2c35ad121eb191ceb62a355
2024-10-15 19:55:50 +02:00
jade 9865ebaaa6 Merge "Remove static initializers for RegisterLegacyCommand" into main 2024-10-09 20:37:58 +00:00
jade 7f7a38f278 Merge changes Ib27cb43d,I03687b8b into main
* changes:
  testsuite: override NIX_CONF_DIR and NIX_USER_CONF_FILES
  Remove some outdated `make test` invocation suggestions
2024-10-09 20:37:16 +00:00
Lulu 299813f324 Merge "Avoid calling memcpy when len == 0 in filetransfer.cc" into main 2024-10-08 01:41:41 +00:00
Lulu d6e1b11d3e Fix gcc warning -Wsign-compare
Add the compile flag '-Wsign-compare' and adapt the code to fix all
cases of this warning.

Change-Id: I26b08fa5a03e4ac294daf697d32cf9140d84350d
2024-10-08 01:32:12 +02:00
Lulu 51a5025913 Avoid calling memcpy when len == 0 in filetransfer.cc
There was a bug report about a potential call to `memcpy` with a null
pointer which is not reproducible:
lix-project/lix#492

This occurred in `src/libstore/filetransfer.cc` in `InnerSource::read`.

To ensure that this doesn't happen, an early return is added before
calling `memcpy` if the length of the data to be copied is 0.

This change also adds a test that ensures that when `InnerSource::read`
is called with an empty file, it throws an `EndOfFile` exception.

Change-Id: Ia18149bee9a3488576c864f28475a3a0c9eadfbb
2024-10-08 01:26:30 +02:00
jade 345e3d068a testsuite: override NIX_CONF_DIR and NIX_USER_CONF_FILES
The test suite can load the global configuration files under certain
circumstances, and, though we would really rather it didn't ever do that
at all, we should at least break the mechanism.

Fixes: lix-project/lix#474
Change-Id: Ib27cb43dd5dfaa70ac491c395b5ba308fd7bd289
2024-10-04 19:17:08 -07:00
Rebecca Turner b63d4a0c62
Remove static initializers for RegisterLegacyCommand
This moves the "legacy"/"nix2" commands under a new `src/legacy/`
directory, instead of being scattered around in a bunch of different
directories.

A new `liblegacy` build target is defined, and the `nix` binary is
linked against it.

Then, `RegisterLegacyCommand` is replaced with `LegacyCommand::add`
calls in functions like `registerNixCollectGarbage()`. These
registration functions are called explicitly in `src/nix/main.cc`.

See: lix-project/lix#359

Change-Id: Id450ffc3f793374907599cfcc121863b792aac1a
2024-10-01 16:08:58 -07:00
eldritch horrors 619a93bd54 Merge "libutil: add async collection mechanism" into main 2024-09-26 17:23:52 +00:00
jade b6038e988d Merge "main: log stack traces for std::terminate" into main 2024-09-26 17:06:01 +00:00
eldritch horrors 531d040e8c libutil: add async collection mechanism
like kj::joinPromisesFailFast this allows waiting for the results of
multiple promises at once, but unlike it not all input promises must
be complete (or any of them failed) for results to become available.

Change-Id: I0e4a37e7bd90651d56b33d0bc5afbadc56cde70c
2024-09-26 16:56:08 +00:00
eldritch horrors ca9256a789 libutil: add an async semaphore implementation
like a normal semaphore, but with awaitable acquire actions. this is
primarily intended as an intermediate concurrency limiting device in
the Worker code, but it may find other uses over time. we do not use
std::counting_semaphore as a base because the counter of that is not
inspectable as will be needed for Worker. we also do not need atomic
operations for cross-thread consistency since we don't have multiple
threads (thanks to kj event loops being confined to a single thread)

Change-Id: Ie2bcb107f3a2c0185138330f7cbba4cec6cbdd95
2024-09-26 16:32:02 +00:00
jade 19e0ce2c03 main: log stack traces for std::terminate
These stack traces kind of suck for the reasons mentioned on the
CppTrace page here (no symbols for inline functions is a major one):
https://github.com/jeremy-rifkin/cpptrace

I would consider using CppTrace if it were packaged, but to be honest, I
think that the more reasonable option is actually to move entirely to
out-of-process crash handling and symbolization.

The reason for this is that if you want to generate anything of
substance on SIGSEGV or really any deadly signal, you are stuck in
async-signal-safe land, which is not a place to be trying to run a
symbolizer. LLVM does it anyway, probably carefully, and chromium *can*
do it on debug builds but in general uses crashpad:
https://source.chromium.org/chromium/chromium/src/+/main:base/debug/stack_trace_posix.cc;l=974;drc=82dff63dbf9db05e9274e11d9128af7b9f51ceaa;bpv=1;bpt=1

However, some stack traces are better than *no* stack traces when we get
mystery exceptions falling out the bottom of the program. I've also
promoted the path for "mystery exceptions falling out the bottom of the
program" to hard crash and generate a core dump because although there's
been some months since the last one of these, these are nonetheless
always *atrociously* diagnosed.

We can't improve the crash handling further until either we use Crashpad
(which involves more C++ deps, no thanks) or we put in the ostensibly
work in progress Rust minidump infrastructure, in which case we need to
finish full support for Rust in libutil first.

Sample report:

Lix crashed. This is a bug. We would appreciate if you report it at https://git.lix.systems/lix-project/lix/issues with the following information included:

Exception: std::runtime_error: lol
Stack trace:
 0# nix::printStackTrace() in /home/jade/lix/lix3/build/src/nix/../libutil/liblixutil.so
 1# 0x000073C9862331F2 in /home/jade/lix/lix3/build/src/nix/../libmain/liblixmain.so
 2# 0x000073C985F2E21A in /nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libstdc++.so.6
 3# 0x000073C985F2E285 in /nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libstdc++.so.6
 4# nix::handleExceptions(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void ()>) in /home/jade/lix/lix3/build/src/nix/../libmain/liblixmain.so
 5# 0x00005CF65B6B048B in /home/jade/lix/lix3/build/src/nix/nix
 6# 0x000073C985C8810E in /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libc.so.6
 7# __libc_start_main in /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libc.so.6
 8# 0x00005CF65B610335 in /home/jade/lix/lix3/build/src/nix/nix

Change-Id: I1a9f6d349b617fd7145a37159b78ecb9382cb4e9
2024-09-25 14:03:45 -07:00
jade 789b19a0cf util: fix brotli decompression of empty input
This caused an infinite loop before since it would just keep asking the
underlying source for more data.

In practice this happened because an HTTP server served a
response to a HEAD request (for which curl will not retrieve any body or
call our write callback function) with Content-Encoding: br, leading to
decompressing nothing at all and going into an infinite loop.

This adds a test to make sure none of our compression methods do that
again, as well as just patching the HTTP client to never feed empty data
into a compression algorithm (since they absolutely have the right to
throw CompressionError on unexpectedly-short streams!).

Reported on Matrix: https://matrix.to/#/!lymvtcwDJ7ZA9Npq:lix.systems/$8BWQR_zKxCQDJ40C5NnDo4bQPId3pZ_aoDj2ANP7Itc?via=lix.systems&via=matrix.org&via=tchncs.de

Change-Id: I027566e280f0f569fdb8df40e5ecbf46c211dad1
2024-09-18 15:37:29 -07:00
jade 4046e019ca tests/compression: rewrite
This test suite was in desperate need of using the parameterization
available with gtest, and was a bunch of useless duplicated code. At
least now it's not duplicated code, though it still probably should be
more full of property tests.

Change-Id: Ia8ccee7ef4f02b2fa40417b79aa8c8f0626ea479
2024-09-17 19:07:48 -07:00
Rebecca Turner 75c0de3e3c
Test including relative paths in configuration
Change-Id: If6c69a5e16d1ccd223fba392890f08f0032fb754
2024-09-01 15:52:48 -07:00
Rebecca Turner 02eb07cfd5 Merge changes I5566a985,I88cf53d3 into main
* changes:
  Support relative and `~/` paths in config settings
  Thread `ApplyConfigOptions` through config parsing
2024-09-01 22:06:36 +00:00
jade 04f8a14833
tree-wide: shuffle headers around for about 30s compile time
This didn't really feel so worth it afterwards, but I did untangle a
bunch of stuff that should not have been tangled.

The general gist of this change is that variant bullshit was causing a
bunch of compile time, and it seems like the only way to deal with
variant induced compile time is to keep variant types out of headers.
Explicit template instantiation seems to do nothing for them.

I also seem to have gotten some back-end time improvement from
explicitly instantiating regex, but I don't know why. There is no
corresponding front-end time improvement from it: regex is still at the
top of the sinners list.

**** Templates that took longest to instantiate:
 15231 ms: std::basic_regex<char>::_M_compile (28 times, avg 543 ms)
 15066 ms: std::__detail::_Compiler<std::regex_traits<char>>::_Compiler (28 times, avg 538 ms)
 12571 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_disjunction (28 times, avg 448 ms)
 12454 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_alternative (28 times, avg 444 ms)
 12225 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_term (28 times, avg 436 ms)
 11363 ms: nlohmann::basic_json<>::parse<const char *> (21 times, avg 541 ms)
 10628 ms: nlohmann::basic_json<>::basic_json (109 times, avg 97 ms)
 10134 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_atom (28 times, avg 361 ms)

Back-end time before messing with the regex:
**** Function sets that took longest to compile / optimize:
  8076 ms: void boost::io::detail::put<$>(boost::io::detail::put_holder<$> cons... (177 times, avg 45 ms)
  4382 ms: std::_Rb_tree<$>::_M_erase(std::_Rb_tree_node<$>*) (1247 times, avg 3 ms)
  3137 ms: boost::stacktrace::detail::to_string_impl_base<boost::stacktrace::de... (137 times, avg 22 ms)
  2896 ms: void boost::io::detail::mk_str<$>(std::__cxx11::basic_string<$>&, ch... (177 times, avg 16 ms)
  2304 ms: std::_Rb_tree<$>::_M_get_insert_hint_unique_pos(std::_Rb_tree_const_... (210 times, avg 10 ms)
  2116 ms: bool std::__detail::_Compiler<$>::_M_expression_term<$>(std::__detai... (112 times, avg 18 ms)
  2051 ms: std::_Rb_tree_iterator<$> std::_Rb_tree<$>::_M_emplace_hint_unique<$... (244 times, avg 8 ms)
  2037 ms: toml::result<$> toml::detail::sequence<$>::invoke<$>(toml::detail::l... (93 times, avg 21 ms)
  1928 ms: std::__detail::_Compiler<$>::_M_quantifier() (28 times, avg 68 ms)
  1859 ms: nlohmann::json_abi_v3_11_3::detail::serializer<$>::dump(nlohmann::js... (41 times, avg 45 ms)
  1824 ms: std::_Function_handler<$>::_M_manager(std::_Any_data&, std::_Any_dat... (973 times, avg 1 ms)
  1810 ms: std::__detail::_BracketMatcher<$>::_BracketMatcher(std::__detail::_B... (112 times, avg 16 ms)
  1793 ms: nix::fetchers::GitInputScheme::fetch(nix::ref<$>, nix::fetchers::Inp... (1 times, avg 1793 ms)
  1759 ms: std::_Rb_tree<$>::_M_get_insert_unique_pos(std::__cxx11::basic_strin... (281 times, avg 6 ms)
  1722 ms: bool nlohmann::json_abi_v3_11_3::detail::parser<$>::sax_parse_intern... (19 times, avg 90 ms)
  1677 ms: boost::io::basic_altstringbuf<$>::overflow(int) (194 times, avg 8 ms)
  1674 ms: std::__cxx11::basic_string<$>::_M_mutate(unsigned long, unsigned lon... (249 times, avg 6 ms)
  1660 ms: std::_Rb_tree_node<$>* std::_Rb_tree<$>::_M_copy<$>(std::_Rb_tree_no... (304 times, avg 5 ms)
  1599 ms: bool nlohmann::json_abi_v3_11_3::detail::parser<$>::sax_parse_intern... (19 times, avg 84 ms)
  1568 ms: void std::__detail::_Compiler<$>::_M_insert_bracket_matcher<$>(bool) (112 times, avg 14 ms)
  1541 ms: std::__shared_ptr<$>::~__shared_ptr() (531 times, avg 2 ms)
  1539 ms: nlohmann::json_abi_v3_11_3::detail::serializer<$>::dump_escaped(std:... (41 times, avg 37 ms)
  1471 ms: void std::__detail::_Compiler<$>::_M_insert_character_class_matcher<... (112 times, avg 13 ms)

After messing with the regex (notice std::__detail::_Compiler vanishes
here, but I don't know why):

**** Function sets that took longest to compile / optimize:
  8054 ms: void boost::io::detail::put<$>(boost::io::detail::put_holder<$> cons... (177 times, avg 45 ms)
  4313 ms: std::_Rb_tree<$>::_M_erase(std::_Rb_tree_node<$>*) (1217 times, avg 3 ms)
  3259 ms: boost::stacktrace::detail::to_string_impl_base<boost::stacktrace::de... (137 times, avg 23 ms)
  3045 ms: void boost::io::detail::mk_str<$>(std::__cxx11::basic_string<$>&, ch... (177 times, avg 17 ms)
  2314 ms: std::_Rb_tree<$>::_M_get_insert_hint_unique_pos(std::_Rb_tree_const_... (207 times, avg 11 ms)
  1923 ms: std::_Rb_tree_iterator<$> std::_Rb_tree<$>::_M_emplace_hint_unique<$... (216 times, avg 8 ms)
  1817 ms: bool nlohmann::json_abi_v3_11_3::detail::parser<$>::sax_parse_intern... (18 times, avg 100 ms)
  1816 ms: toml::result<$> toml::detail::sequence<$>::invoke<$>(toml::detail::l... (93 times, avg 19 ms)
  1788 ms: nlohmann::json_abi_v3_11_3::detail::serializer<$>::dump(nlohmann::js... (40 times, avg 44 ms)
  1749 ms: std::_Rb_tree<$>::_M_get_insert_unique_pos(std::__cxx11::basic_strin... (278 times, avg 6 ms)
  1724 ms: std::__cxx11::basic_string<$>::_M_mutate(unsigned long, unsigned lon... (248 times, avg 6 ms)
  1697 ms: boost::io::basic_altstringbuf<$>::overflow(int) (194 times, avg 8 ms)
  1684 ms: nix::fetchers::GitInputScheme::fetch(nix::ref<$>, nix::fetchers::Inp... (1 times, avg 1684 ms)
  1680 ms: std::_Rb_tree_node<$>* std::_Rb_tree<$>::_M_copy<$>(std::_Rb_tree_no... (303 times, avg 5 ms)
  1589 ms: bool nlohmann::json_abi_v3_11_3::detail::parser<$>::sax_parse_intern... (18 times, avg 88 ms)
  1483 ms: non-virtual thunk to boost::wrapexcept<$>::~wrapexcept() (181 times, avg 8 ms)
  1447 ms: nlohmann::json_abi_v3_11_3::detail::serializer<$>::dump_escaped(std:... (40 times, avg 36 ms)
  1441 ms: std::__shared_ptr<$>::~__shared_ptr() (496 times, avg 2 ms)
  1420 ms: boost::stacktrace::basic_stacktrace<$>::init(unsigned long, unsigned... (137 times, avg 10 ms)
  1396 ms: boost::basic_format<$>::~basic_format() (194 times, avg 7 ms)
  1290 ms: std::__cxx11::basic_string<$>::_M_replace_cold(char*, unsigned long,... (231 times, avg 5 ms)
  1258 ms: std::vector<$>::~vector() (354 times, avg 3 ms)
  1222 ms: std::__cxx11::basic_string<$>::_M_replace(unsigned long, unsigned lo... (231 times, avg 5 ms)
  1194 ms: std::_Rb_tree<$>::_M_get_insert_hint_unique_pos(std::_Rb_tree_const_... (49 times, avg 24 ms)
  1186 ms: bool tao::pegtl::internal::sor<$>::match<$>(std::integer_sequence<$>... (1 times, avg 1186 ms)
  1149 ms: std::__detail::_Executor<$>::_M_dfs(std::__detail::_Executor<$>::_Ma... (70 times, avg 16 ms)
  1123 ms: toml::detail::sequence<$>::invoke(toml::detail::location&) (69 times, avg 16 ms)
  1110 ms: nlohmann::json_abi_v3_11_3::basic_json<$>::json_value::destroy(nlohm... (55 times, avg 20 ms)
  1079 ms: std::_Function_handler<$>::_M_manager(std::_Any_data&, std::_Any_dat... (541 times, avg 1 ms)
  1033 ms: nlohmann::json_abi_v3_11_3::detail::lexer<$>::scan_number() (20 times, avg 51 ms)

Change-Id: I10af282bcd4fc39c2d3caae3453e599e4639c70b
2024-08-28 09:55:05 -07:00
jade 0cc285f87b
treewide: fix a bunch of lints
Fixes:
- Identifiers starting with _ are prohibited
- Some driveby header dependency cleaning which wound up with doing some
  extra fixups.
- Fucking C style casts, man. C++ made these 1000% worse by letting you
  also do memory corruption with them with references.
  - Remove casts to Expr * where ExprBlackHole is an incomplete type by
    introducing an explicitly-cast eBlackHoleAddr as Expr *.
  - An incredibly illegal cast of the text bytes of the StorePath hash
    into a size_t directly. You can't DO THAT.

    Replaced with actually parsing the hash so we get 100% of the bits
    being entropy, then memcpying the start of the hash. If this shows
    up in a profile we should just make the hash parser faster with a
    lookup table or something sensible like that.
  - This horrendous bit of UB which I thankfully slapped a deprecation
    warning on, built, and it didn't trigger anywhere so it was dead
    code and I just deleted it. But holy crap you *cannot* do that.

    inline void mkString(const Symbol & s)
    {
        mkString(((const std::string &) s).c_str());
    }
- Some wrong lints. Lots of wrong macro lints, one wrong
  suspicious-sizeof lint triggered by the template being instantiated
  with only pointers, but the calculation being correct for both
  pointers and not-pointers.
- Exceptions in destructors strike again. I tried to catch the
  exceptions that might actually happen rather than all the exceptions
  imaginable. We can let the runtime hard-kill it on other exceptions
  imo.

Change-Id: I71761620846cba64d66ee7ca231b20c061e69710
2024-08-26 16:13:03 -07:00
Rebecca Turner 690f07272e
Support relative and ~/ paths in config settings
Change-Id: I5566a9858ba255f4ac5051d1368c7dfb24460f0a
2024-08-25 15:54:22 -07:00
Rebecca Turner 5fc6fcb310
Thread ApplyConfigOptions through config parsing
This makes no changes to logic but makes the `ApplyConfigOptions` value
available to consumers.

Change-Id: I88cf53d38faac8472c556aee55c13d0acbd1e5db
2024-08-25 15:54:22 -07:00
alois31 e3c289dbe9
libutil/config: unify path setting types
There have been multiple setting types for paths that are supposed to be
canonicalised, depending on whether zero or one, one, or any number of paths is
to be specified. Naturally, they behaved in slightly different ways in the
code. Simplify things by unifying them and removing special behaviour (mainly
the "multiple paths type can coerce to boolean" thing).

Change-Id: I7c1ce95e9c8e1829a866fb37d679e167811e9705
2024-08-21 17:57:23 +02:00
piegames 278fddc317 libexpr: Deprecate URL literals
Closes #437.

Change-Id: I9f67fc965bb4a7e7fd849e5067ac1cb3bab064cd
2024-08-17 20:31:57 +02:00
piegames 49d61b2e4b libexpr: Introduce Deprecated features
They are like experimental features, but opt-in instead of opt-out. They
will allow us to gracefully remove language features. See #437

Change-Id: I9ca04cc48e6926750c4d622c2b229b25cc142c42
2024-08-17 19:47:51 +02:00
jade 757041c3e7 Merge changes I526cceed,Ia4e2f1fa,I22e66972,I9fbd55a9,Ifca22e44 into main
* changes:
  sqlite: add a Use::fromStrNullable
  util: implement charptr_cast
  tree-wide: fix a pile of lints
  refactor: make HashType and Base enum classes for type safety
  build: integrate clang-tidy into CI
2024-08-08 22:43:10 +00:00
jade e34833c025 tree-wide: fix a pile of lints
This:
- Converts a bunch of C style casts into C++ casts.
- Removes some very silly pointer subtraction code (which is no more or
  less busted on i686 than it began)
- Fixes some "technically UB" that never had to be UB in the first
  place.
- Makes finally follow the noexcept status of the inner function. Maybe
  in the future we should ban the function from not being noexcept, but
  that is not today.
- Makes various locally-used exceptions inherit from std::exception.

Change-Id: I22e66972602604989b5e494fd940b93e0e6e9297
2024-08-08 14:53:17 -07:00
jade 370ac940dd refactor: make HashType and Base enum classes for type safety
Change-Id: I9fbd55a9d50464a56fe11cb42a06a206914150d8
2024-08-08 14:53:17 -07:00
piegames 28ae24f3f7 libexpr: Add experimental pipe operator
The |> operator is a reverse function operator with low binding strength
to replace lib.pipe. Implements RFC 148, see the RFC text for more
details. Closes #438.

Change-Id: I21df66e8014e0d4dd9753dd038560a2b0b7fd805
2024-08-08 11:13:53 +02:00
piegames ec7552ff74 libexpr/parser: Test experimental features
Currently, the parser relies on the global experimental feature flags.
In order to properly test conditional language features, we instead need
to pass it around in the parser::State.

This means that the parser cannot cache the result of isEnabled anymore,
which wouldn't necessarily hurt performance if the function didn't
perform a linear search on the list of enabled features on every single
call. While we could simply evaluate once at the start of parsing and
cache the result in the parser state, the more sustainable solution
would be to fix `isEnabled` such that all callers may profit from the
performance improvement.

Change-Id: Ic9b9c5d882b6270e1114988b63e6064d36c25cf2
2024-08-07 13:07:50 +00:00
jade ca9d3e6e00 tree-wide: fix various lint warnings
Change-Id: I0fc80718eb7e02d84cc4b5d5deec4c0f41116134
2024-08-04 20:55:45 -07:00
jade 19ae87e5ce tree-wide: add support for asan!
What if you could find memory bugs in Lix without really trying very
hard? I've had variously scuffed patches to do this, but this is
blocked on boost coroutines removal at this point tbh.

Change-Id: Id762af076aa06ad51e77a6c17ed10275929ed578
2024-07-31 14:13:39 -07:00
alois31 40c39aa5d2
libexpr/print: do not show elided nested items when there are none
When the configured maximum depth has been reached, attribute sets and lists
are printed with ellipsis to indicate the elision of nested items. Previously,
this happened even in case the structure being printed is empty, so that such
items do not in fact exist. This is confusing, so stop doing it.

Change-Id: I0016970dad3e42625e085dc896e6f476b21226c9
2024-07-18 18:41:34 +02:00
alois31 b5da823138
libexpr/print: never show empty attrsets or derivations as «repeated»
The repeated value detection logic exists so that the occurrence of large
common substructures does not fill up the screen or the computer's memory.
However, empty attribute sets and derivations (when their detection is enabled)
are always cheap to print, and in practice I have observed them to make up a
significant majority of the cases where I was annoyed by the repeated value
detection kicking in. Furthermore, `nix-instantiate --eval` already disables
this logic for empty attribute sets, and empty lists are already exempted
everywhere. For these reasons, always print empty attribute sets and
derivations as what they are.

Change-Id: I5dac8e7739f9d726b76fd0521ec46f38af94463f
2024-07-18 18:41:34 +02:00
eldritch horrors 6b4d46e9e0 libstore: remove WriteConn::sink fields
we no longer need these since we're no longer using sinks to serialize things.

Change-Id: Iffb1a3eab33c83f611c88fa4e8beaa8d5ffa079b
2024-07-16 00:57:42 +00:00