Commit graph

15653 commits

Author SHA1 Message Date
Pierre Bourdon a30c567336
filetransfer: unit test content-encoding handling
Very basic behavior test to ensure that gzip data gets internally
decompressed by the file transfer pipeline.

Change a std::string_view return value in the test harness to
std::string. I wouldn't call myself a C++ beginner and I still managed
to shoot myself in the foot like three times with the lifetime
managements there (e.g. [&] { return an_std_string; } ends up with a
dangling string_view!).

Change-Id: I1360750d4181ce1ca2a3aa4dc0e97e131351c469
2024-05-11 15:58:32 +02:00
Pierre Bourdon 38d825b21e
filetransfer: fix decompression regression from 121edecf
121edecf65 added a new state field to
carry over content encoding settings from transfer to sink creation, but
never actually set that field.

Change-Id: I714b2efe745561e851b78a4791479b3501db8c72
2024-05-11 14:49:23 +02:00
Qyriad f6dc40cd1c Merge "bump the extra --version info from info to notice, -vv -> -v" into main 2024-05-10 21:56:09 +00:00
raito 4ebbd4362f Merge "feat(libstore): print the first line of stdout of SSH in case of failure" into main 2024-05-10 19:33:36 +00:00
raito 8404a1f66d feat(libstore): print the first line of stdout of SSH in case of failure
In case of failure to connect as can be seen in
https://buildbot.lix.systems/#/builders/39/builds/1386/steps/1/logs/stdio

It is difficult to understand what happened, if we enabled the talkative
verbose level, we could learn about the first line SSH sent us.

In practice, this is not workable, we can just make it warn all the
time.

Change-Id: Iaaf56894060a58f2dfc78254bb60b1c43482f9bb
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-10 20:22:47 +02:00
Qyriad d003dcd7f4 bump the extra --version info from info to notice, -vv -> -v
requiring *two* --verbose to print extra information on --version is a
weird flex

Change-Id: I05d043da1bf583f34e9d1fc206144ea4ca9a859d
2024-05-10 11:55:17 -06:00
eldritch horrors ceccac835c libutil: remove callback.hh
it's no longer used. it really shouldn't have existed this long since it
was just a mashup of both std::promise and std::packaged_task in a shape
that makes composition unnecessarily difficult. all but a single case of
Callback pattern calls were fully synchronous anyway, and even this sole
outlier was by far not important enough to justify the extra complexity.

Change-Id: I208aec4572bf2501cdbd0f331f27d505fca3a62f
2024-05-10 02:21:11 +02:00
eldritch horrors b66451ae7f libstore: de-callback-ify FileTransfer
also add a few more tests for exception propagation behavior. using
packaged_tasks and futures (which only allow a single call to a few
of their methods) introduces error paths that weren't there before.

Change-Id: I42ca5236f156fefec17df972f6e9be45989cf805
2024-05-10 02:21:11 +02:00
eldritch horrors 28a98d152c libstore: de-callback-ify Store::queryRealisation
Change-Id: I8d74745c519518f163f51dfaa39063836f17599e
2024-05-09 23:18:05 +02:00
eldritch horrors 17965bf11c libstore: un-callback-ify Store::queryRealisationUncached
Change-Id: I4a328f46eaac3bb8b19ddc091306de83348be9cf
2024-05-09 23:18:05 +02:00
eldritch horrors 2f4a1dd6e0 libstore: de-callback-ify Store::queryPathInfoUncached
Change-Id: I23a156aaff5328f67ca16ccd85c0ea1711b21e35
2024-05-09 23:18:05 +02:00
eldritch horrors c77bd88259 libstore: de-callback-ify BinaryCacheStore::getFile
Change-Id: I36b3eb9f645aa04058151e7b2353e15e6f29057b
2024-05-09 23:18:05 +02:00
eldritch horrors 1a002d1a11 libstore: de-callback-ify CA realisation substitution
this is the *only* real user of file transfer download completion
callbacks, and a pretty spurious user at that (seeing how nothing
here is even turned on by default and indeed a dependency of path
substitution which *isn't* async, and concurrency-limited). it'll
be a real pain to keep this around, and realistically it would be
a lot better to overhaul substitution in general to be *actually*
async. that requires a proper async framework footing though, and
we don't have anything of the sort, but it's also blocking *that*

Change-Id: I1bf671f217c654a67377087607bf608728cbfc83
2024-05-09 23:18:05 +02:00
Qyriad 9ae90612a7 Merge changes I4bffa766,If2561cd5 into main
* changes:
  add and fix -Wignored-qualifiers
  add and fix -Wdeprecated-copy
2024-05-09 17:52:11 +00:00
jade d5804d0c6d Merge "fix: eval error of .#devShells.x86_64-linux.x86_64-freebsd13" into main 2024-05-09 15:22:03 +00:00
Maximilian Bosch b962a266c4 Merge "flake: update nixpkgs input to latest nixos-23.11" into main 2024-05-09 13:17:21 +00:00
Qyriad 1f9b0fba23 add and fix -Wignored-qualifiers
Change-Id: I4bffa766ae04dd80355f9b8c10e59700e4b406da
2024-05-09 07:08:44 -06:00
jade d1dacad708 fix: eval error of .#devShells.x86_64-linux.x86_64-freebsd13
This is broken and our resident nixbsd maintainers say it should
probably just be temporarily removed till we switch to 24.05 instead of
diagnosing it.

Originally introduced in: https://github.com/nixos/nix/pull/8887

Fixes: lix-project/lix#277
Change-Id: I1e7db8859620024a7b37dbd0cc1c5ec139b9e5cb
2024-05-09 13:07:30 +00:00
Qyriad 2bbe3efd16 add and fix -Wdeprecated-copy
*so* many warnings, from only two definitions

Change-Id: If2561cd500c05a1e33cce984faf9f3e42a8a95ac
2024-05-09 13:02:45 +00:00
Qyriad 010bbd1b0e Merge "nix3-eval: don't elide top-level errors" into main 2024-05-09 11:46:13 +00:00
alois31 da0e1f5716 Merge "repl: show a progress bar while performing builds" into main 2024-05-09 04:32:55 +00:00
Maximilian Bosch a4c943403f
flake: update nixpkgs input to latest nixos-23.11
This includes the update to libseccomp 2.5.5[1], so we don't need to
override it on our own.

[1] https://nixpk.gs/pr-tracker.html?pr=306070

Change-Id: I1fa9c7fcc23e501d75f774745107c6bb086ced70
2024-05-08 23:03:28 +02:00
Patrick Jackson 8552519bb8 Merge "Improve the justfile" into main 2024-05-08 20:54:48 +00:00
Qyriad 54322f09d3 nix3-eval: don't elide top-level errors
Fixes #276.

Change-Id: I83e71beb5c35d6f3b10a4186caa5e52a2f95b510
2024-05-08 13:37:20 -06:00
Théophane Hufschmitt adea821d87
libstore: Fix sandbox=relaxed
The fix for the Darwin vulnerability in ecdbc3b207
also broke setting `__sandboxProfile` when `sandbox=relaxed` or
`sandbox=false`. This cppnix change fixes `sandbox=relaxed` and
adds a suitable test.

Co-Authored-By: Artemis Tosini <lix@artem.ist>
Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
Change-Id: I40190f44f3e1d61846df1c7b89677c20a1488522
2024-05-08 19:31:43 +00:00
alois31 243c0f18da
repl: show a progress bar while performing builds
In commit 946fc12e4e, the progress bar in the
repl was disabled again because it was observed to erase incremental output
from attrset evaluations from the terminal. Let's try adding the progress bar
again, this time showing up only when a build is initiated, which does not have
incremental output that could be destroyed to begin with. While this does mean
that we won't have a progress bar for eval-time fetching or IFD, it's still
better than nothing.

Change-Id: If4eb1035cd0c876f5b4ff1e2434b9baf99f150ac
2024-05-08 20:54:41 +02:00
Patrick Jackson 85f51fc10b Improve the justfile
Adds descriptions and a 'list' function that runs as the default

Change-Id: Ifee2c8ccd2694af0ca8bd94744f8be99f91b254a
2024-05-08 08:19:37 -07:00
Qyriad f782c8a60a flake: refactor devShell creation
Now instead of a derivation overridden from Lix, we use a mkShell
derivation parameterized on an already called package.nix. This also
lets callPackage take care of the buildPackages distinction for the
devShell.

Change-Id: I5ddfec40d83fa6136032da7606fe6d3d5014ef42
2024-05-07 18:09:51 -06:00
Qyriad 8822fd7dd5 package: default the build-release-notes arg like we do with lix-doc
Change-Id: I0e2df55efc1cd6ea0a3252b9f26676e84612fdb6
2024-05-07 17:07:53 -06:00
Qyriad b9be46fb31 remove the autoconf+Make buildsystem
We're not using it anymore. Any leftover bugs in the Meson buildsystem
are now just bugs.

Closes #249.

Change-Id: I0465a0c37ae819f94d40e7829f5bff046aa63d73
2024-05-07 17:04:30 -06:00
Patrick Jackson d184981af0 Merge "feat: setup gerrit commit-msg hook with nix develop" into main 2024-05-07 22:51:53 +00:00
Qyriad 8715a0ac4e Merge changes If1077a7b,I62da3161,Iebb4667b into main
* changes:
  flake: fix devShell on i686-linux by disabling ClangBuildAnalyzer on it
  flake: fix eval of checks & devshell on i686-linux
  flake: move the pre-commit definition to its own file
2024-05-07 22:14:22 +00:00
Qyriad 7e940cc170 flake: fix devShell on i686-linux by disabling ClangBuildAnalyzer on it
ClangBuildAnalyzer doesn't build on i686-linux due to
`long long int`/`size_t` conversion errors, so let's just exclude it
from the devshell on that platform

Change-Id: If1077a7b3860db4381999c8e304f6d4b2bc96a05
2024-05-07 15:31:25 -06:00
Nikodem Rabuliński e8a603fb2f Merge changes Icf26010a,Ib6161567 into main
* changes:
  Always initialize curl in parent process on darwin
  Fix failing darwin tests
2024-05-07 21:26:24 +00:00
Patrick Jackson 9af8694367 feat: setup gerrit commit-msg hook with nix develop
Closes #273

Change-Id: Id883d2cda06adbcae53b8c360ad015330f0af81b
2024-05-07 14:20:09 -07:00
Nikodem Rabuliński f894cce79b
Always initialize curl in parent process on darwin
Because of an objc quirk[1], calling curl_global_init for the first time
after fork() will always result in a crash.
Up until now the solution has been to set
OBJC_DISABLE_INITIALIZE_FORK_SAFETY for every nix process to ignore
that error.
This is less than ideal because we were setting it in package.nix,
which meant that running nix tests locally would fail because
that variable was not set.
Instead of working around that error we address it at the core -
by calling curl_global_init inside initLibStore, which should mean
curl will already have been initialized by the time we try to do so in
a forked process.

[1] 01edf1705f/runtime/objc-initialize.mm (L614-L636)

Change-Id: Icf26010a8be655127cc130efb9c77b603a6660d0
2024-05-07 20:43:17 +02:00
Qyriad aac32327d5 flake: fix eval of checks & devshell on i686-linux
Change-Id: I62da3161327051005e3f48f83974140efef4417e
2024-05-07 12:38:01 -06:00
Qyriad 4f98d21b71 flake: move the pre-commit definition to its own file
It's a good hundred LOC, and wasn't coupled to the actual flake logic at
all.

Change-Id: Iebb4667b3197dbd8cb2b019014e99fa651848832
2024-05-07 12:38:01 -06:00
eldritch horrors 964ac8b0e8 libutil: de-callback-ify computeClosure
only two users of this function exist. only one used it in a way that
even bears resemblance to asynchronicity, and even that one didn't do
it right. fully async and parallel computation would have only worked
if any getEdgesAsync never calls the continuation it receives itself,
only from more derived callbacks running on other threads. calling it
directly would cause the decoupling promise to be awaited immediately
*on the original thread*, completely negating all nice async effects.

Change-Id: I0aa640950cf327533a32dee410105efdabb448df
2024-05-07 14:35:20 +00:00
eldritch horrors 230860dbb8 libstore: limit CA realisation info substitution concurrency
this seems to be an oversight, considering that regular substitutions
are concurrency-limited. while not particularly necessary at present,
once we've removed the `Callback` based interfaces it will be needed.

Change-Id: Ide2d08169fcc24752cbd07a1d33fb8482f7034f5
2024-05-07 14:35:20 +00:00
eldritch horrors 29f93e1e0d libutil: throw EndOfFile at sourceToSink end
... how has this never broken anything before

Change-Id: If3789c02028e8f929481514f63d76b0b46bfc182
2024-05-07 14:35:20 +00:00
Qyriad 005b2b61e6 fix fallback chroot store creation after b247ef72d
When /nix/var (or, more precisely, NIX_STATE_DIR) does not exist at all,
Lix falls back to creating an adhoc chroot store in XDG_DATA_HOME.

b247ef72d[1] changed the way Store classes are initialized, and in the
migration, a `params2` was accidentally changed to `params`. This commit
restores the correct behavior, and in lieu of a single *character* fix,
this commit also changes the variable name to something more reasonable.

Fixes #274.

[1]: b247ef72dc

n.b., this code might deserve some more looking at anyway. this fallback
store creation throws away *all* Store params passed to
openFromNonUri() in favor of an entirely new set which only contains
the `root` param, which may or may not be the correct behavior

Change-Id: Ibea559b88a50e6d6e75a1f87d9d7816cabb2a8f3
2024-05-06 20:54:21 -06:00
jade 241b9992fd Merge "fix: readme typo" into main 2024-05-07 01:49:34 +00:00
jade f5ea4cd045 Merge "Remove README.md from build dependencies" into main 2024-05-07 01:49:21 +00:00
jade 4e327b8c60 Merge "chore: re-work the contribution guide" into main 2024-05-07 01:49:04 +00:00
raito 36d69864f3 chore: re-work the contribution guide
As per our bootstrap governance discussions, here's a very simple
proposal which links as much as possible to our wiki.

Change-Id: I88b1c43f933ff7e529151b1e933fad40283383c4
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-07 01:28:35 +00:00
jade bf1a22aa8e Remove README.md from build dependencies
Now we will simply hit cache every time anyone changes readme, yay!

Change-Id: I5906f589d319ff6d43cbd2b467887e08f7474283
2024-05-06 18:23:57 -07:00
Abhiram 864ffff2cc fix: readme typo
The text on the matrix link was wrong.

Change-Id: I5d15b27eef0d7d03505920ebf1c03fecbb7faabe
2024-05-06 18:14:48 -07:00
eldritch horrors f75d0752ce filetransfer: correctly abort empty transfers
returning 0 from the callback for errors signals successful transfer if
the source returned no data even though the exception we've just caught
clearly disagrees. while this is not all that important (since the only
viable cause of such errors will be dataCallback, and the sole instance
of it being used already takes care of exceptions) we can just do this.

Change-Id: I2bb150eff447121d82e8e3aa4e00057c40523ac6
2024-05-06 21:18:23 +02:00
eldritch horrors 121edecf65 filetransfer: extract decompressor creation
this will be necessary if we want download() to return a source instead
of consuming a sink, which will in turn be needed to remove coroutines.

Change-Id: I34ec241e9bbc5d32fbcd243b244e29c3757533aa
2024-05-06 20:19:03 +02:00