Commit graph

2037 commits

Author SHA1 Message Date
Ilya K 7d52d74bbe BrotliDecompressionSource: don't bail out too early
If we've consumed the entire input, that doesn't actually mean we're
done decompressing - there might be more output left. This worked (?)
in most cases because the input and output sizes are pretty comparable,
but sometimes they're not and then things get very funny.

Change-Id: I73435a654a911b8ce25119f713b80706c5783c1b
2024-06-20 09:21:13 +03:00
eldritch horrors c55dcc6c13 filetransfer: return a Source from download()
without this we will not be able to get rid of makeDecompressionSink,
which in turn will be necessary to get rid of sourceToSink (since the
libarchive archive wrapper *must* be a Source due to api limitations)

Change-Id: Iccd3d333ba2cbcab49cb5a1d3125624de16bce27
2024-06-19 10:50:12 +00:00
eldritch horrors 67f778670c libutil: add makeDecompressionSource
Change-Id: Iac7f24d79e24417436b9b5cbefd6af051aeea0a6
2024-06-19 10:50:12 +00:00
alois31 fed34594d8 Merge "libfetchers: represent unfetched submodules consistently" into main 2024-06-19 07:08:19 +00:00
eldritch horrors 0b9a72524a filetransfer: {up,down}load -> transfer
even the transfer function is not all that necessary since there aren't
that many users, but we'll keep it for now. we could've kept both names
but we also kind of want to use `download` for something else very soon

Change-Id: I005e403ee59de433e139e37aa2045c26a523ccbf
2024-06-18 23:58:25 +00:00
alois31 aa00a5a8c9 libfetchers: represent unfetched submodules consistently
Unfetched submodules are included as empty directories in archives, so they end
up as such in the store when fetched in clean mode. Make sure the same happens
in dirty mode too. Fortunately, they are already correctly represented in the
ls-files output, so we just need to make sure to include the empty directory in
our filter.

Fixes: https://github.com/NixOS/nix/issues/6247
Change-Id: I60d06ff360cfa305d081b920838c893c06da801c
2024-06-18 00:54:51 +00:00
jade ce2b48aa41 Merge changes from topic "protocol" into main
* changes:
  libstore client: remove remaining dead code
  libstore: refuse to serialise ancient protocols
  libstore client: remove support for <2.3 clients
  libstore daemon: remove very old protocol support (<2.3)
  Delete old ValidPathInfo test, fix UnkeyedValidPathInfo
  Set up minimum protocol version
2024-06-17 22:08:48 +00:00
eldritch horrors bcb774688f libexpr: add expr memory management
with the prepatory work done this mostly means turning plain pointers
into unique_ptrs, with all the associated churn that necessitates. we
might want to change some of these to box_ptrs at some point as well,
but that would be a semantic change that isn't fully appropriate yet.

Change-Id: I0c238c118617420650432f4ed45569baa3e3f413
2024-06-17 19:46:44 +00:00
eldritch horrors ad5366c2ad libexpr: pass Exprs as references, not pointers
almost all places where Exprs are passed as pointers expect the pointers
to be non-null. pass them as references to encode this constraint in the
type system as well (and also communicate that Exprs must not be freed).

Change-Id: Ia98f166fec3c23151f906e13acb4a0954a5980a2
2024-06-17 19:46:44 +00:00
jade c22a7f50cb libstore: refuse to serialise ancient protocols
We don't want to deal with these at all, let's stop doing so.

(marking this one as the fix commit since its immediate predecessors
aren't the complete fix)
Fixes: lix-project/lix#325

Change-Id: Ieea1b0b8ac0f903d1e24e5b3e63cfe12eeec119d
2024-06-16 19:15:08 -07:00
jade 24255748b4 Delete old ValidPathInfo test, fix UnkeyedValidPathInfo
The UnkeyedValidPathInfo test was testing an ancient version but not the
current version. Doesn't make much sense to me.

Change-Id: Ib476a4297d9075f2dcd31a073b3e7b149b2189af
2024-06-16 19:13:51 -07:00
Robert Hensing b588a761fe Merge pull request #10799 from hercules-ci/safer-tab-completion
Add repl completion test

(cherry picked from commit 1e2b26734b4da101247678aec405c9dcfdc33f98)

Change-Id: Ic3de39e71960a05a8676190b1ec9a7f0bb6057f5
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
2024-06-16 04:03:32 +00:00
alois31 1d6fd94cf9 Merge "tests/libcmd: set HOME to a temporary directory" into main 2024-06-13 05:27:55 +00:00
jade 73898cad0e tests/flake-registry: Fix occasional deadlocks
This seems to have been caused by having the wrong PID. I don't know why
it worked before in the sandbox, but the code was definitely wrong
before, so let's just fix it.

Change-Id: I556580bdf614c716566310e975a36daa6d6c9a91
2024-06-12 15:34:23 -07:00
jade d9345d8836 tests: verify that NIX_DAEMON_PACKAGE's version looks like a version
Followup to https://gerrit.lix.systems/c/lix/+/1417 to ensure that this
parser will never take something that doesn't look like a version.

It turns out this problem is less alarming than initially thought
because it only applies to the testsuite in a non-default mode.

Change-Id: I26aba24aaf0215f2b782966314b94784db766266
2024-06-12 15:34:23 -07:00
Pierre Bourdon f7b6552699 [resubmit] flake: update nixpkgs pin 23.11->24.05 (+ boehmgc compat changes)
-- message from cl/1418 --

The boehmgc changes are bundled into this commit because doing otherwise
would require an annoying dance of "adding compatibility for < 8.2.6 and
>= 8.2.6" then updating the pin then removing the (now unneeded)
compatibility. It doesn't seem worth the trouble to me given the low
complexity of said changes.

Rebased coroutine-sp-fallback.diff patch taken from https://github.com/NixOS/nixpkgs/pull/317227

-- jade resubmit changes --

This is a resubmission of https://gerrit.lix.systems/c/lix/+/1418, which
was reverted in https://gerrit.lix.systems/c/lix/+/1432 for breaking CI
evaluation without being detected.

I have run `nix flake check -Lv` on this one before submission and it
passes on my machine and crucially without eval errors, so the CI result
should be accurate.

It seems like someone renamed forbiddenDependenciesRegex to
forbiddenDependenciesRegexes in nixpkgs and also changed the type
incompatibly. That's pretty silly, but at least it's just an eval error.

Also, `xonsh` regressed the availability of `xonsh-unwrapped`, but it
was fixed by us in https://github.com/NixOS/nixpkgs/pull/317636, which
is now in our channel, so we update nixpkgs compared to the original
iteration of this to simply get that.

We originally had a regression related to some reorganization of the
nixpkgs lib test suite in which there was broken parameter passing.
This, too, we got quickfixed in nixpkgs, so we don't need any changes
for it: https://github.com/NixOS/nixpkgs/pull/317772

Related: https://gerrit.lix.systems/c/lix/+/1428
Fixes: lix-project/lix#385

Change-Id: I26d41ea826fec900ebcad0f82a727feb6bcd28f3
2024-06-12 15:34:22 -07:00
alois31 3c0434999e tests/libcmd: set HOME to a temporary directory
The libcmd unit test creates files (more specifically, the fetcher cache) in
its home directory. In the single-user sandbox, this leads to the creation of
/homeless-shelter, since this is the default HOME and the root is writable.
Unfortunately, this conflicts with the assumption of the functional tests that
this directory does not exist. Use a different home directory to prevent these
test failures, and thus restore the ability to build inside the single-user
sandbox.

Fixes: lix-project/lix#365
Change-Id: I4df8c53d043234b95a7c0ac45fc5ee89e8d46aff
2024-06-12 22:13:55 +00:00
jade f432e464dd Merge "tests: fix daemon version in isDaemonNewer function" into main 2024-06-10 23:22:05 +00:00
Pierre Bourdon 9281a12532
tests/nixos/nix-copy: fix NixOS >= 24.05 compatibility
4b128008c5d9fde881ce1b0a25e60ae0415a14d5 in nixpkgs introduced a default
hashedPasswordFile for root in NixOS tests, which takes precedence over
the password option set in the nix-copy test.

Change-Id: Iffaebec5992e50614b854033f0d14312c8d275b5
2024-06-08 17:59:08 +02:00
Mario Rodas a05de58ebd tests: fix daemon version in isDaemonNewer function
Since ad8a4b380e, the version printer returns "nix (Lix, like Nix) 2.x",
hence the `daemonVersion` was being set to the string "like".

Using `compareVersions` with a letter compares them lexicographically:

   builtins.compareVersions "like" "2.12pre20230103"  // => -1
   builtins.compareVersions "like" "2.16.0"           // => -1

This caused that `isDaemonNewer` always returned 1, falsy in Bash terms.
Therefore, the test suite skipped those tests where they use it.

Fixes lix-project/lix#324

Change-Id: If6682515bf0bf8b8add641af9a4e98b50a9acb51
2024-06-08 04:20:00 +00:00
Qyriad 06e65e537b build: expose option to enable or disable precompiled std headers
They are enabled by default, and Meson will also prints whether or not
they're enabled at the bottom at the end of configuration.

Change-Id: I48db238510bf9e74340b86f243f4bbe360794281
2024-06-06 12:46:26 -06:00
Pierre Bourdon 6e59b4b407 Merge "tests/nixos: make the tarball-flakes test better reflect real use cases" into main 2024-06-04 15:14:02 +00:00
Pierre Bourdon a3256a9375
tests/nixos: make the tarball-flakes test better reflect real use cases
In most real world cases, the Link header is set on the redirect, not on
the final file. This regressed in Lix earlier and while new unit tests
were added to cover it, this integration test should probably have also
caught it.

Change-Id: I2a9d8d952fff36f2c22cfd751451c2b523f7045c
2024-06-04 08:12:59 +02:00
Olmo Kramer 72d85acba4 nix flake update: add test for multiple inputs from nix#10073
Upstream-PR: https://github.com/NixOS/nix/pull/10073
Change-Id: I53fcb43b387e55439e062e208877afeb88493bb4
2024-06-03 21:50:33 +00:00
Qyriad d374a9908f Merge "build: fix static linking with a hack" into main 2024-06-01 19:17:13 +00:00
raito b8cb7abcf0 chore: rebrand Nix to Lix when it makes sense
Here's my guide so far:

$ rg '((?!(recursive).*) Nix
(?!(daemon|store|expression|Rocks!|Packages|language|derivation|archive|account|user|sandbox|flake).*))'
-g '!doc/' --pcre2

All items from this query have been tackled. For the documentation side:
that's for lix-project/lix#162.

Additionally, all remaining references to github.com/NixOS/nix which
were not relevant were also replaced.

Fixes: lix-project/lix#148.
Fixes: lix-project/lix#162.
Change-Id: Ib3451fae5cb8ab8cd9ac9e4e4551284ee6794545
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-06-01 20:31:24 +02:00
Qyriad e54d4c9381 build: fix static linking with a hack
This causes libstore, libexpr, libfetchers, and libutil to be linked
with -Wl,--whole-archive to executables, when building statically.

libstore for the store backends, libexpr for the primops, libfetchers
for the fetcher backends I assume(?), and libutil for the nix::logger
initializer (which notably shows in pre-main constructors when HOME is
not owned by the user. cursed.).

This workaround should be removed when #359 is fixed.

Fixes #306.

Change-Id: Ie9ef0154e09a6ed97920ee8ab23810ca5e2de84c
2024-05-31 21:47:16 -06:00
jade adedac70fa Merge changes Ifcb0d310,I664366b8,Ibe7cf546 into main
* changes:
  gitignore: delete 90% of it
  build-time: remove 20% more by PCH'ing C++ stdlib
  shellHook: make it actually run
2024-05-31 19:19:29 +00:00
alois31 cf756fdf3c libstore/build: copy ca-certificates too
In b469c6509b, the ca-certificates file was
missed. It should be copied too so that we don't end up bind-mounting a broken
symlink.

Change-Id: Ic9b292d602eb94b0e78f77f2a27a19d24665783c
2024-05-31 07:54:18 +00:00
jade 0f99ed43f1 build-time: remove 20% more by PCH'ing C++ stdlib
It seems like someone implemented precompiled headers a long time ago
and then it never got ported to meson or maybe didn't work at all.

This is, however, blessedly easy to simply implement. I went looking for
`#define` that could affect the result of precompiling the headers, and
as far as I can tell we aren't doing any of that, so this should truly
just be free build time savings.

Previous state:
Compilation (551 times):
  Parsing (frontend):         1302.1 s
  Codegen & opts (backend):    956.3 s

New state:
**** Time summary:
Compilation (567 times):
  Parsing (frontend):         1123.0 s
  Codegen & opts (backend):   1078.1 s

I wonder if the "regression" in codegen time is just doing the PCH
operation a few times, because meson does it per-target.

Change-Id: I664366b8069bab4851308b3a7571bea97ac64022
2024-05-30 21:54:21 +00:00
jade 2f104bbe3b Merge "Revert "tests/filetransfer: reënable on Darwin"" into main 2024-05-30 21:53:38 +00:00
jade 533d469875 Revert "tests/filetransfer: reënable on Darwin"
This reverts commit 285bc67318.

Reason for revert: lix-project/lix#364

For some reason this broke `main` even though the change we are reverting passed CI! Mysterious, haunted, etc. Needs more debugging, let's turn it off for now.

Change-Id: Ica4819d61cd35b83eb52985bfcb657e858f025a9
2024-05-30 21:38:32 +00:00
jade c161687b5f Merge "tests/filetransfer: reënable on Darwin" into main 2024-05-30 14:58:23 +00:00
jade 285bc67318 tests/filetransfer: reënable on Darwin
Since we put __darwinAllowLocalNetworking in our derivation in
I752b81c85ebeaab4e582ac01c239d69d65580f37, this stuff will just work
fine. I checked our derivation works on the darwin community builder.

Change-Id: I40e3a801d6bb38efede79af4aded65c1e1f57cec
2024-05-29 20:41:22 -07:00
Cole Helbling 031d924116 libutil/args: warn on unknown settings after parsing all flags
Upstream change: https://github.com/NixOS/nix/pull/10701

Change-Id: Icf271df57ec529dd8c64667d1ef9f6dbf02d33d3
2024-05-30 03:07:21 +00:00
jade 562ff516ab Merge changes from topic "libutil-split" into main
* changes:
  util.hh: Delete remaining file and clean up headers
  util.hh: Move nativeSystem to local-derivation-goal.cc
  util.hh: Move stuff to types.hh
  util.cc: Delete remaining file
  util.{hh,cc}: Move ignoreException to error.{hh,cc}
  util.{hh,cc}: Split out namespaces.{hh,cc}
  util.{hh,cc}: Split out users.{hh,cc}
  util.{hh,cc}: Split out strings.{hh,cc}
  util.{hh,cc}: Split out unix-domain-socket.{hh,cc}
  util.{hh,cc}: Split out child.{hh,cc}
  util.{hh,cc}: Split out current-process.{hh,cc}
  util.{hh,cc}: Split out processes.{hh,cc}
  util.{hh,cc}: Split out file-descriptor.{hh,cc}
  util.{hh,cc}: Split out file-system.{hh,cc}
  util.{hh,cc}: Split out terminal.{hh,cc}
  util.{hh,cc}: Split out environment-variables.{hh,cc}
2024-05-30 02:33:05 +00:00
Mario Rodas c71f21da3a Merge "tests: fix functional-timeout" into main 2024-05-29 23:57:49 +00:00
eldritch horrors dd4a2c1759 libstore: fix http abuses no longer working
while refactoring the curl wrapper we inadvertently broken the immutable
flake protocol, because the immutable flake protocol accumulates headers
across the entire redirect chain instead of using only the headers given
in the final response of the chain. this is a problem because Some Known
Providers Of Flake Infrastructure set rel=immutable link headers only in
the penultimate entry of the redirect chain, and curl does not regard it
as worth returning to us via its response header enumeration mechanisms.

fixes lix-project/lix#358

Change-Id: I645c3932b465cde848bd6a3565925a1e3cbcdda0
2024-05-29 22:35:29 +00:00
Tom Hubrecht a39ba22ff7 util.hh: Delete remaining file and clean up headers
Change-Id: Ic1f68e6af658e94ef7922841dd3ad4c69551ef56
2024-05-29 12:38:51 +02:00
Tom Hubrecht b910551120 util.{hh,cc}: Split out strings.{hh,cc}
Change-Id: I4f642d1046d56b5db26f1b0296ee16a0e02d444a
2024-05-29 11:01:34 +02:00
Tom Hubrecht 9a52e4688c util.{hh,cc}: Split out processes.{hh,cc}
Change-Id: I39280dc40ca3f7f9007bc6c898ffcf760e2238b7
2024-05-29 11:01:34 +02:00
Tom Hubrecht 8cd9aa24a8 util.{hh,cc}: Split out file-descriptor.{hh,cc}
Change-Id: I0dd0f9a9c2003fb887e076127e7f825fd3289c76
2024-05-29 09:54:47 +02:00
Tom Hubrecht 6b5078c815 util.{hh,cc}: Split out file-system.{hh,cc}
Change-Id: Ifa89a529e7e34e7291eca87d802d2f569cf2493e
2024-05-29 09:54:47 +02:00
Tom Hubrecht 81bdf8d2d6 util.{hh,cc}: Split out terminal.{hh,cc}
Change-Id: I9de2296b4012d50f540124001d54d6ca3be4c6da
2024-05-29 09:54:47 +02:00
Tom Hubrecht 6fd6795bc4 util.{hh,cc}: Split out environment-variables.{hh,cc}
Change-Id: Icff0aa33fda5147bd5dbe256a0b9d6a6c8a2c3f6
2024-05-28 11:29:29 +02:00
Mario Rodas ec5f025ec2 tests: fix functional-timeout
grepQuietInvert is a typo introduced by c11836126b.

The test functional-timeout was failing silently because Bash considered
the command-not-found error as truthy.

Change-Id: Ic13829d02ec55d6ecd63a0f4d34ec0d32379609f
2024-05-28 04:20:00 +00:00
alois31 ddfe379a6b Merge "libstore/build: always enable seccomp filtering and no-new-privileges" into main 2024-05-25 04:21:53 +00:00
alois31 f047e4357b libstore/build: always enable seccomp filtering and no-new-privileges
Seccomp filtering and the no-new-privileges functionality improve the security
of the sandbox, and have been enabled by default for a long time. In
lix-project/lix#265 it was decided that they
should be enabled unconditionally. Accordingly, remove the allow-new-privileges
(which had weird behavior anyway) and filter-syscall settings, and force the
security features on. Syscall filtering can still be enabled at build time to
support building on architectures libseccomp doesn't support.

Change-Id: Iedbfa18d720ae557dee07a24f69b2520f30119cb
2024-05-24 21:19:29 +00:00
Qyriad 8c06b7b431 libfetchers: log fetches by URL just before they happen
Addresses but does not close #305, as we still need an indicator for
frozen fetches.

Change-Id: Iba34ad42dc1c8772f7da249b90fe794b041bbf73
2024-05-24 15:15:42 -06:00
Pierre Bourdon 6ade981476 Merge "nixos/tests: enable remoteBuilds tests against Nix 2.18" into main 2024-05-24 00:55:25 +00:00