Commit graph

9552 commits

Author SHA1 Message Date
puck 23c92f0815 Merge "primops: change to std::function, allowing the passing of user data" into main 2024-05-17 21:37:41 +00:00
puck 92e1df23b3 Merge "Loosen constness on listElems() result" into main 2024-05-17 21:37:35 +00:00
Qyriad 93b7edfd07 Merge "docs: mention importNative/exec in allow-unsafe-native-code-during-evaluation" into main 2024-05-17 18:17:05 +00:00
julia 005ee33a9a Merge "Allow enabling core dumps from builds for nix & child processes" into main 2024-05-17 01:10:44 +00:00
Qyriad 5ff076d8ad docs: mention importNative/exec in allow-unsafe-native-code-during-evaluation
Both of these still needs their own actual documentation, but they are
at least now mentioned that they exist and what they're enabled by.

Change-Id: I235b9e8e627e04ed06611423c8e67a8eca233120
2024-05-17 00:41:35 +00:00
julia 5b7dcb3005 Allow enabling core dumps from builds for nix & child processes
Fixes lix-project/lix#268

Change-Id: I3f1b0ddf064f891cca8b53229c5c31c74cea3d9f
2024-05-16 17:11:21 -07:00
Qyriad 5756be19f6 Merge "add clarifying doc-comments on {get,create}NixStateDir()" into main 2024-05-17 00:10:38 +00:00
jade 9322a1cbe7 Merge "builtins: fix builtins.langVersion docs to state it's deprecated" into main 2024-05-16 21:19:15 +00:00
Yorick 194654c96f primops: change to std::function, allowing the passing of user data
(cherry picked from commit 48aa57549d514432d6621c1e29f051951eca2d7f)
Change-Id: Ib7d5c6514031ceb6c42ac44588be6b0c1c3c225b
2024-05-16 13:01:40 +00:00
puck c6bb377c91 Loosen constness on listElems() result
Change-Id: I1caff000362c83e5172413a036c22a2e9ed3ede8
2024-05-16 13:01:40 +00:00
jade 6d84d6bdc1 builtins: fix builtins.langVersion docs to state it's deprecated
This builtin was always a problem and nixpkgs uses it in exactly one
place, to give up if the Nix version is absurdly old. It has no other
use cases, and doesn't work in a multi-implementation world anyway.

Change-Id: I03c36e118591029e2ef14b091fe14a311c66a08a
2024-05-15 21:54:12 -07:00
Mel Zuser 9249c89dc6 build: fix build when gc is disabled
Change-Id: I8d3eb8874a4138668011b525c3b400a55a1f4866
2024-05-15 18:48:00 -07:00
Maximilian Bosch aabc902040 Merge "nix3-build: show all FOD errors with --keep-going" into main 2024-05-15 18:46:06 +00:00
alois31 0903a99bad Merge changes I8456c47b,I48253f5f into main
* changes:
  repl: clear the interrupt before reading the next line
  libutil: remove the interrupt-blocking code
2024-05-15 15:46:11 +00:00
Maximilian Bosch e146393183
nix3-build: show all FOD errors with --keep-going
Basically I'd expect the same behavior as with `nix-build`, i.e.
with `--keep-going` the hash-mismatch error of each failing
fixed-output derivation is shown.

The approach is derived from `Store::buildPaths` (`entry-point.cc`):
instead of throwing the first build-result, check if there are any build
errors and if so, display all of them and throw after that.

Unfortunately, the BuildResult struct doesn't have an `ErrorInfo`
(there's a FIXME for that at least), so I have to construct my own here.
This is a rather cheap bugfix and I decided against touching too many
parts of libstore for that (also I don't know if that's in line with the
ongoing refactoring work).

Closes lix-project/lix#302

Change-Id: I378ab984fa271e6808c6897c45e0f070eb4c6fac
2024-05-15 15:35:18 +02:00
alois31 eeb7e71810
repl: clear the interrupt before reading the next line
Otherwise, it will be thrown again during exit when the repl is terminated by
end-of-input after the last command was interrupted.

Change-Id: I8456c47bc36cfb0892efdad5420f318f7e6526d5
2024-05-13 09:04:05 +02:00
alois31 914b0febf7
libutil: remove the interrupt-blocking code
The interrupt-blocking code was originally introduced 20 years ago so that
trying to log an error message does not result in an interrupt exception being
thrown and then going unhandled (c8d3882cdc).
However, the logging code does not check for interrupts any more
(054be50257), so this reasoning is no longer
applicable. Delete this code so that later interrupts are unblocked again, for
example in the next line entered into the repl.

Closes: lix-project/lix#296
Change-Id: I48253f5f4272e75001148c13046e709ef5427fbd
2024-05-13 09:03:53 +02:00
Artemis Tosini 4b35e6a75e Merge "libstore: Fix sandbox=relaxed" into main 2024-05-12 03:51:19 +00:00
Qyriad 88d9b70f67 add clarifying doc-comments on {get,create}NixStateDir()
NixStateDir() != NIX_STATE_DIR. These functions should honestly probably
be renamed.

Change-Id: I00f54b742bba6188bbc7f2410956d956780b99d3
2024-05-11 15:40:00 -06:00
Pierre Bourdon 38d825b21e
filetransfer: fix decompression regression from 121edecf
121edecf65 added a new state field to
carry over content encoding settings from transfer to sink creation, but
never actually set that field.

Change-Id: I714b2efe745561e851b78a4791479b3501db8c72
2024-05-11 14:49:23 +02:00
Qyriad f6dc40cd1c Merge "bump the extra --version info from info to notice, -vv -> -v" into main 2024-05-10 21:56:09 +00:00
raito 4ebbd4362f Merge "feat(libstore): print the first line of stdout of SSH in case of failure" into main 2024-05-10 19:33:36 +00:00
raito 8404a1f66d feat(libstore): print the first line of stdout of SSH in case of failure
In case of failure to connect as can be seen in
https://buildbot.lix.systems/#/builders/39/builds/1386/steps/1/logs/stdio

It is difficult to understand what happened, if we enabled the talkative
verbose level, we could learn about the first line SSH sent us.

In practice, this is not workable, we can just make it warn all the
time.

Change-Id: Iaaf56894060a58f2dfc78254bb60b1c43482f9bb
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-10 20:22:47 +02:00
Qyriad d003dcd7f4 bump the extra --version info from info to notice, -vv -> -v
requiring *two* --verbose to print extra information on --version is a
weird flex

Change-Id: I05d043da1bf583f34e9d1fc206144ea4ca9a859d
2024-05-10 11:55:17 -06:00
eldritch horrors ceccac835c libutil: remove callback.hh
it's no longer used. it really shouldn't have existed this long since it
was just a mashup of both std::promise and std::packaged_task in a shape
that makes composition unnecessarily difficult. all but a single case of
Callback pattern calls were fully synchronous anyway, and even this sole
outlier was by far not important enough to justify the extra complexity.

Change-Id: I208aec4572bf2501cdbd0f331f27d505fca3a62f
2024-05-10 02:21:11 +02:00
eldritch horrors b66451ae7f libstore: de-callback-ify FileTransfer
also add a few more tests for exception propagation behavior. using
packaged_tasks and futures (which only allow a single call to a few
of their methods) introduces error paths that weren't there before.

Change-Id: I42ca5236f156fefec17df972f6e9be45989cf805
2024-05-10 02:21:11 +02:00
eldritch horrors 28a98d152c libstore: de-callback-ify Store::queryRealisation
Change-Id: I8d74745c519518f163f51dfaa39063836f17599e
2024-05-09 23:18:05 +02:00
eldritch horrors 17965bf11c libstore: un-callback-ify Store::queryRealisationUncached
Change-Id: I4a328f46eaac3bb8b19ddc091306de83348be9cf
2024-05-09 23:18:05 +02:00
eldritch horrors 2f4a1dd6e0 libstore: de-callback-ify Store::queryPathInfoUncached
Change-Id: I23a156aaff5328f67ca16ccd85c0ea1711b21e35
2024-05-09 23:18:05 +02:00
eldritch horrors c77bd88259 libstore: de-callback-ify BinaryCacheStore::getFile
Change-Id: I36b3eb9f645aa04058151e7b2353e15e6f29057b
2024-05-09 23:18:05 +02:00
eldritch horrors 1a002d1a11 libstore: de-callback-ify CA realisation substitution
this is the *only* real user of file transfer download completion
callbacks, and a pretty spurious user at that (seeing how nothing
here is even turned on by default and indeed a dependency of path
substitution which *isn't* async, and concurrency-limited). it'll
be a real pain to keep this around, and realistically it would be
a lot better to overhaul substitution in general to be *actually*
async. that requires a proper async framework footing though, and
we don't have anything of the sort, but it's also blocking *that*

Change-Id: I1bf671f217c654a67377087607bf608728cbfc83
2024-05-09 23:18:05 +02:00
Qyriad 1f9b0fba23 add and fix -Wignored-qualifiers
Change-Id: I4bffa766ae04dd80355f9b8c10e59700e4b406da
2024-05-09 07:08:44 -06:00
Qyriad 2bbe3efd16 add and fix -Wdeprecated-copy
*so* many warnings, from only two definitions

Change-Id: If2561cd500c05a1e33cce984faf9f3e42a8a95ac
2024-05-09 13:02:45 +00:00
Qyriad 010bbd1b0e Merge "nix3-eval: don't elide top-level errors" into main 2024-05-09 11:46:13 +00:00
Qyriad 54322f09d3 nix3-eval: don't elide top-level errors
Fixes #276.

Change-Id: I83e71beb5c35d6f3b10a4186caa5e52a2f95b510
2024-05-08 13:37:20 -06:00
Théophane Hufschmitt adea821d87
libstore: Fix sandbox=relaxed
The fix for the Darwin vulnerability in ecdbc3b207
also broke setting `__sandboxProfile` when `sandbox=relaxed` or
`sandbox=false`. This cppnix change fixes `sandbox=relaxed` and
adds a suitable test.

Co-Authored-By: Artemis Tosini <lix@artem.ist>
Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
Change-Id: I40190f44f3e1d61846df1c7b89677c20a1488522
2024-05-08 19:31:43 +00:00
alois31 243c0f18da
repl: show a progress bar while performing builds
In commit 946fc12e4e, the progress bar in the
repl was disabled again because it was observed to erase incremental output
from attrset evaluations from the terminal. Let's try adding the progress bar
again, this time showing up only when a build is initiated, which does not have
incremental output that could be destroyed to begin with. While this does mean
that we won't have a progress bar for eval-time fetching or IFD, it's still
better than nothing.

Change-Id: If4eb1035cd0c876f5b4ff1e2434b9baf99f150ac
2024-05-08 20:54:41 +02:00
Qyriad b9be46fb31 remove the autoconf+Make buildsystem
We're not using it anymore. Any leftover bugs in the Meson buildsystem
are now just bugs.

Closes #249.

Change-Id: I0465a0c37ae819f94d40e7829f5bff046aa63d73
2024-05-07 17:04:30 -06:00
Nikodem Rabuliński e8a603fb2f Merge changes Icf26010a,Ib6161567 into main
* changes:
  Always initialize curl in parent process on darwin
  Fix failing darwin tests
2024-05-07 21:26:24 +00:00
Nikodem Rabuliński f894cce79b
Always initialize curl in parent process on darwin
Because of an objc quirk[1], calling curl_global_init for the first time
after fork() will always result in a crash.
Up until now the solution has been to set
OBJC_DISABLE_INITIALIZE_FORK_SAFETY for every nix process to ignore
that error.
This is less than ideal because we were setting it in package.nix,
which meant that running nix tests locally would fail because
that variable was not set.
Instead of working around that error we address it at the core -
by calling curl_global_init inside initLibStore, which should mean
curl will already have been initialized by the time we try to do so in
a forked process.

[1] 01edf1705f/runtime/objc-initialize.mm (L614-L636)

Change-Id: Icf26010a8be655127cc130efb9c77b603a6660d0
2024-05-07 20:43:17 +02:00
eldritch horrors 964ac8b0e8 libutil: de-callback-ify computeClosure
only two users of this function exist. only one used it in a way that
even bears resemblance to asynchronicity, and even that one didn't do
it right. fully async and parallel computation would have only worked
if any getEdgesAsync never calls the continuation it receives itself,
only from more derived callbacks running on other threads. calling it
directly would cause the decoupling promise to be awaited immediately
*on the original thread*, completely negating all nice async effects.

Change-Id: I0aa640950cf327533a32dee410105efdabb448df
2024-05-07 14:35:20 +00:00
eldritch horrors 230860dbb8 libstore: limit CA realisation info substitution concurrency
this seems to be an oversight, considering that regular substitutions
are concurrency-limited. while not particularly necessary at present,
once we've removed the `Callback` based interfaces it will be needed.

Change-Id: Ide2d08169fcc24752cbd07a1d33fb8482f7034f5
2024-05-07 14:35:20 +00:00
eldritch horrors 29f93e1e0d libutil: throw EndOfFile at sourceToSink end
... how has this never broken anything before

Change-Id: If3789c02028e8f929481514f63d76b0b46bfc182
2024-05-07 14:35:20 +00:00
Qyriad 005b2b61e6 fix fallback chroot store creation after b247ef72d
When /nix/var (or, more precisely, NIX_STATE_DIR) does not exist at all,
Lix falls back to creating an adhoc chroot store in XDG_DATA_HOME.

b247ef72d[1] changed the way Store classes are initialized, and in the
migration, a `params2` was accidentally changed to `params`. This commit
restores the correct behavior, and in lieu of a single *character* fix,
this commit also changes the variable name to something more reasonable.

Fixes #274.

[1]: b247ef72dc

n.b., this code might deserve some more looking at anyway. this fallback
store creation throws away *all* Store params passed to
openFromNonUri() in favor of an entirely new set which only contains
the `root` param, which may or may not be the correct behavior

Change-Id: Ibea559b88a50e6d6e75a1f87d9d7816cabb2a8f3
2024-05-06 20:54:21 -06:00
eldritch horrors f75d0752ce filetransfer: correctly abort empty transfers
returning 0 from the callback for errors signals successful transfer if
the source returned no data even though the exception we've just caught
clearly disagrees. while this is not all that important (since the only
viable cause of such errors will be dataCallback, and the sole instance
of it being used already takes care of exceptions) we can just do this.

Change-Id: I2bb150eff447121d82e8e3aa4e00057c40523ac6
2024-05-06 21:18:23 +02:00
eldritch horrors 121edecf65 filetransfer: extract decompressor creation
this will be necessary if we want download() to return a source instead
of consuming a sink, which will in turn be needed to remove coroutines.

Change-Id: I34ec241e9bbc5d32fbcd243b244e29c3757533aa
2024-05-06 20:19:03 +02:00
jade 748d8310fa Fix the pages in the manual for Lix
This doesn't comprehensively fix everything outdated in the manual, or
make the manual greatly better, but it does note down where at least
jade noticed it was wrong, and it does fix all the instances of
referencing Nix to conform to the style guide to the best of our
ability.

A lot of things have been commented out for being wrong, and there are
three types of FIXME introduced:

- FIXME(Lix): generically Lix needs to fix it
- FIXME(Qyriad): re lix-project/lix#215
- FIXME(meson): docs got outdated by meson changes and need rewriting

I did fix a bunch of it that I could, but there could certainly be
mistakes and this is definitely just an incremental improvement.

Fixes: lix-project/lix#266
Change-Id: I5993c4603d7f026a887089fce77db08394362135
2024-05-05 16:11:01 -07:00
Qyriad 99903f24eb fix integer overflow on i686 with high phys memory
sizeof(long) is 4 bytes on i686 GCC.
With ~32 GiB of memory and a page size of 4096, there are 7988420 pages.
(7988420 * 4096) is bigger than INT32_MAX folks.

This has gone unnoticed for 9 years, and only came up thanks to
94ea517db[1] adding integer overflow sensitization checks, which caused
this broken code to emit an illegal instruction, crashing Lix the
instant the buildsystem ran Lix to generate the docs files.

[1]: 94ea517dbe729765b69638190f4bea3f6a632b40

Change-Id: I50bb9ea072aac11b449d79e5d55525887a6e5a99
2024-05-05 19:47:46 +00:00
Qyriad 4998699e1a Merge "point nix3-upgrade-nix to releases.lix.systems/manifest.nix" into main 2024-05-05 19:47:36 +00:00
jade 4fd32351f3 Merge "Warn on untrusted client settings being ignored" into main 2024-05-05 19:35:35 +00:00