Commit graph

9570 commits

Author SHA1 Message Date
eldritch horrors f281727e70 Merge "libstore: remove unused copyPath function" into main 2024-05-23 02:31:50 +00:00
jade d05e0b9f1f fix: enlarge envSize by several times to not be close to nixpkgs size
nixpkgs has 23000 attributes, and our previous limit would be hit if you
have more than one nixpkgs in the environment, for example, because
`repl-overlays` will load the new stuff from the environment on top of
the existing environment.

This is not really testable since if we did write such a test, it would
just be testing this constant tbh...

Fixes: lix-project/lix#337
Change-Id: I49197bfb4db55b082f914f0d70e84f5f5f110954
2024-05-22 17:08:37 -07:00
Qyriad 06c1375e52 Merge "libfetchers: fallback to memory SQLite if fs IO fails" into main 2024-05-22 21:17:46 +00:00
jade dcc7ea5498 release notes: add a bunch of them
Also fix typos introduced by the commits I read.

I have run the addDrvOutputDependencies release note past Ericson since
I was confused by what the heck it was doing, and he was saying it was
reasonable.

Change-Id: Id015353b00938682f7faae7de43df7f991a5237e
2024-05-22 21:13:56 +02:00
eldritch horrors 47523944c5 libstore: remove unused copyPath function
Change-Id: Ibfb0848d8b337bd38947a745c240838865cea401
2024-05-22 18:26:01 +02:00
Qyriad 6881476232 libfetchers: fallback to memory SQLite if fs IO fails
nix::fetchers::CacheImpl uses $XDG_CACHE_HOME, or its default based on
$HOME, to store its SQLite database. If the current process can't write
to that directory for whatever reason, though, any eval-time fetching
would fail just initializing the cache.

With this change, IO errors initializing the fetcher cache are logged
but ignored, and nix::fetchers::CacheImpl falls back to an in-memory¹
database instead.

Notably, this will fix any uses eval fetching while Lix itself is being
run in a derivation builder (such as during tests), as the derivation
builder does not set $XDG_CACHE_HOME, and sets $HOME to the non-existent
directory /homeless-shelter.

Before:

$ env -u XDG_CACHE_HOME HOME=/homeless-shelter nix -Lv eval --impure -E 'fetchTarball "https://git.lix.systems/lix-project/lix/archive/main.tar.gz"'
error:
       … while calling the 'fetchTarball' builtin
         at «string»:1:1:
            1| fetchTarball "https://git.lix.systems/lix-project/lix/archive/main.tar.gz"
             | ^

       error: creating directory '/homeless-shelter': Permission denied

After:

$ env -u XDG_CACHE_HOME HOME=/homeless-shelter nix -Lv eval --impure -E 'fetchTarball "https://git.lix.systems/lix-project/lix/archive/main.tar.gz"'
warning: ignoring error initializing Lix fetcher cache: error: creating directory '/homeless-shelter': Permission denied
"/nix/store/s9lxdnn0awp37n560bg4fgr497ah4hvw-source"

¹: https://www.sqlite.org/inmemorydb.html

Change-Id: I15c38c9baaf215fc6e192b8a4c70b9692a69bc22
2024-05-21 09:30:25 -06:00
Qyriad 20981461d4 print type and value in "flake attr is not a derivation" errors
This turns errors like:

error: flake output attribute 'hydraJobs' is not a derivation or path

into errors like:

error: expected flake output attribute 'hydraJobs' to be a derivation or
path but found a set: { binaryTarball = «thunk»; build = «thunk»; etc> }

This change affects all InstallableFlake commands.

Change-Id: I899757af418b6f98201006ec6ee13a448c07077c
2024-05-21 05:55:13 -06:00
Artemis Tosini 3de77e6dbd Merge "libutil: Create chmodPath function" into main 2024-05-20 15:13:53 +00:00
Qyriad 589953e832 Merge "fix -Wdeprecated-copy on clang (BaseError copy assignment)" into main 2024-05-20 00:11:12 +00:00
puck bfb91db4f6 repl-interacter: save history after entering every line
Fixes: lix-project/lix#328
Change-Id: Iedd79ff5f72e84766ebd234c63856170afc624f0
2024-05-19 22:47:45 +00:00
Artemis Tosini 5411fbf204
libutil: Create chmodPath function
Move the identical static `chmod_` functions in libstore to
libutil. the function is called `chmodPath` instead of `chmod`
as otherwise it will shadow the standard library chmod in the nix
namespace, which is somewhat confusing.

Change-Id: I7b5ce379c6c602e3d3a1bbc49dbb70b1ae8f7bad
2024-05-19 22:07:58 +00:00
Qyriad 4eb6779ea8 fix -Wdeprecated-copy on clang (BaseError copy assignment)
2bbe3efd1¹ added the -Wdeprecated-copy warning, and fixed the instances
of it which GCC warned about, in HintFmt and ref<T>. However, when
building with Clang, there is an additional deprecated-copy warning in
BaseError. This commit explicitly defaults the copy assignment operator
for BaseError and silences this warning.

1: 2bbe3efd16
Change-Id: I50aa4a7ab1a7aae5d7b31f765994abd3db06379d
2024-05-19 12:32:13 -06:00
eldritch horrors 774c56094f libstore: fix old RemoteStore::addToStore serializer
having the serializer write into `*conn` is not legal because we are
in a sinkToSource that will be drained by the remote we're connected
to. writing into `*conn` directly can break the framing protocol. it
is unlikely this code was ever run: to protocol it caters to is from
2016(!) and thoroughly untested in-tree, and since it's been present
since nix 2.17 and the 1.18 protocol broken here is nix 2.0 we might
safely assume that daemons older than nix 2.1 are no longer used now

see also #325 (though that wants <2.3 gone, this is sadly only <2.1)

Change-Id: I9d674c18f6d802f61c5d85dfd9608587b73e70a5
2024-05-19 11:57:55 +00:00
Alyssa Ross 139d31f876 Improve nix-store --delete failure message
On several occasions I've found myself confused when trying to delete
a store path, because I am told it's still alive, but
nix-store --query --roots doesn't show anything.  Let's save future
users this confusion by mentioning that a path might be alive due to
having referrers, not just roots.

(cherry picked from commit 979a019014569eee7d0071605f6ff500b544f6ac)

Upstream-PR: https://github.com/NixOS/nix/pull/10733
Change-Id: I54ae839a85f3de3393493fba27fd40d7d3af0516
2024-05-18 14:49:40 -06:00
puck 1fe58bd8a7 nix cat/dump-path/key: stop progress bar before writeFull
These commands outputs data that may not end with a newline. This
causes problems when the progress bar redraws, as that completely
wipes the last line of output. As nix key generate-secret outputs
a single line of text with no output, it shows up entirely blank,
making it look like nothing happened.

Fixes: lix-project/lix#320
Change-Id: I5ac706d71d839b6dfa760b60a351414cd96297cf
2024-05-18 17:51:16 +00:00
Pierre Bourdon d1c8fd3b09 Merge "derived-path: refuse built derived path with a non-derivation base" into main 2024-05-18 07:26:26 +00:00
julia 7a3745b076
Deprecate the online flake registries and vendor the default registry
Fixes #183, #110, #116.

The default flake-registry option becomes 'vendored', and refers
to a vendored flake-registry.json file in the install path.

Vendored copy of the flake-registry is from github:NixOS/flake-registry
at commit 9c69f7bd2363e71fe5cd7f608113290c7614dcdd.

Change-Id: I752b81c85ebeaab4e582ac01c239d69d65580f37
2024-05-18 12:27:23 +10:00
puck 23c92f0815 Merge "primops: change to std::function, allowing the passing of user data" into main 2024-05-17 21:37:41 +00:00
puck 92e1df23b3 Merge "Loosen constness on listElems() result" into main 2024-05-17 21:37:35 +00:00
Qyriad 93b7edfd07 Merge "docs: mention importNative/exec in allow-unsafe-native-code-during-evaluation" into main 2024-05-17 18:17:05 +00:00
julia 005ee33a9a Merge "Allow enabling core dumps from builds for nix & child processes" into main 2024-05-17 01:10:44 +00:00
Qyriad 5ff076d8ad docs: mention importNative/exec in allow-unsafe-native-code-during-evaluation
Both of these still needs their own actual documentation, but they are
at least now mentioned that they exist and what they're enabled by.

Change-Id: I235b9e8e627e04ed06611423c8e67a8eca233120
2024-05-17 00:41:35 +00:00
Pierre Bourdon 5a1824ebe1
derived-path: refuse built derived path with a non-derivation base
Example: /nix/store/dr53sp25hyfsnzjpm8mh3r3y36vrw3ng-neovim-0.9.5^out

This is nonsensical since selecting outputs can only be done for a
buildable derivation, not for a realised store path. The build worker
side of things ends up crashing with an assertion when trying to handle
such malformed paths.

Change-Id: Ia3587c71fe3da5bea45d4e506e1be4dd62291ddf
2024-05-17 02:16:15 +02:00
julia 5b7dcb3005 Allow enabling core dumps from builds for nix & child processes
Fixes lix-project/lix#268

Change-Id: I3f1b0ddf064f891cca8b53229c5c31c74cea3d9f
2024-05-16 17:11:21 -07:00
Qyriad 5756be19f6 Merge "add clarifying doc-comments on {get,create}NixStateDir()" into main 2024-05-17 00:10:38 +00:00
jade 9322a1cbe7 Merge "builtins: fix builtins.langVersion docs to state it's deprecated" into main 2024-05-16 21:19:15 +00:00
Yorick 194654c96f primops: change to std::function, allowing the passing of user data
(cherry picked from commit 48aa57549d514432d6621c1e29f051951eca2d7f)
Change-Id: Ib7d5c6514031ceb6c42ac44588be6b0c1c3c225b
2024-05-16 13:01:40 +00:00
puck c6bb377c91 Loosen constness on listElems() result
Change-Id: I1caff000362c83e5172413a036c22a2e9ed3ede8
2024-05-16 13:01:40 +00:00
jade 6d84d6bdc1 builtins: fix builtins.langVersion docs to state it's deprecated
This builtin was always a problem and nixpkgs uses it in exactly one
place, to give up if the Nix version is absurdly old. It has no other
use cases, and doesn't work in a multi-implementation world anyway.

Change-Id: I03c36e118591029e2ef14b091fe14a311c66a08a
2024-05-15 21:54:12 -07:00
Mel Zuser 9249c89dc6 build: fix build when gc is disabled
Change-Id: I8d3eb8874a4138668011b525c3b400a55a1f4866
2024-05-15 18:48:00 -07:00
Maximilian Bosch aabc902040 Merge "nix3-build: show all FOD errors with --keep-going" into main 2024-05-15 18:46:06 +00:00
alois31 0903a99bad Merge changes I8456c47b,I48253f5f into main
* changes:
  repl: clear the interrupt before reading the next line
  libutil: remove the interrupt-blocking code
2024-05-15 15:46:11 +00:00
Maximilian Bosch e146393183
nix3-build: show all FOD errors with --keep-going
Basically I'd expect the same behavior as with `nix-build`, i.e.
with `--keep-going` the hash-mismatch error of each failing
fixed-output derivation is shown.

The approach is derived from `Store::buildPaths` (`entry-point.cc`):
instead of throwing the first build-result, check if there are any build
errors and if so, display all of them and throw after that.

Unfortunately, the BuildResult struct doesn't have an `ErrorInfo`
(there's a FIXME for that at least), so I have to construct my own here.
This is a rather cheap bugfix and I decided against touching too many
parts of libstore for that (also I don't know if that's in line with the
ongoing refactoring work).

Closes lix-project/lix#302

Change-Id: I378ab984fa271e6808c6897c45e0f070eb4c6fac
2024-05-15 15:35:18 +02:00
alois31 eeb7e71810
repl: clear the interrupt before reading the next line
Otherwise, it will be thrown again during exit when the repl is terminated by
end-of-input after the last command was interrupted.

Change-Id: I8456c47bc36cfb0892efdad5420f318f7e6526d5
2024-05-13 09:04:05 +02:00
alois31 914b0febf7
libutil: remove the interrupt-blocking code
The interrupt-blocking code was originally introduced 20 years ago so that
trying to log an error message does not result in an interrupt exception being
thrown and then going unhandled (c8d3882cdc).
However, the logging code does not check for interrupts any more
(054be50257), so this reasoning is no longer
applicable. Delete this code so that later interrupts are unblocked again, for
example in the next line entered into the repl.

Closes: lix-project/lix#296
Change-Id: I48253f5f4272e75001148c13046e709ef5427fbd
2024-05-13 09:03:53 +02:00
Artemis Tosini 4b35e6a75e Merge "libstore: Fix sandbox=relaxed" into main 2024-05-12 03:51:19 +00:00
Qyriad 88d9b70f67 add clarifying doc-comments on {get,create}NixStateDir()
NixStateDir() != NIX_STATE_DIR. These functions should honestly probably
be renamed.

Change-Id: I00f54b742bba6188bbc7f2410956d956780b99d3
2024-05-11 15:40:00 -06:00
Pierre Bourdon 38d825b21e
filetransfer: fix decompression regression from 121edecf
121edecf65 added a new state field to
carry over content encoding settings from transfer to sink creation, but
never actually set that field.

Change-Id: I714b2efe745561e851b78a4791479b3501db8c72
2024-05-11 14:49:23 +02:00
Qyriad f6dc40cd1c Merge "bump the extra --version info from info to notice, -vv -> -v" into main 2024-05-10 21:56:09 +00:00
raito 4ebbd4362f Merge "feat(libstore): print the first line of stdout of SSH in case of failure" into main 2024-05-10 19:33:36 +00:00
raito 8404a1f66d feat(libstore): print the first line of stdout of SSH in case of failure
In case of failure to connect as can be seen in
https://buildbot.lix.systems/#/builders/39/builds/1386/steps/1/logs/stdio

It is difficult to understand what happened, if we enabled the talkative
verbose level, we could learn about the first line SSH sent us.

In practice, this is not workable, we can just make it warn all the
time.

Change-Id: Iaaf56894060a58f2dfc78254bb60b1c43482f9bb
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-10 20:22:47 +02:00
Qyriad d003dcd7f4 bump the extra --version info from info to notice, -vv -> -v
requiring *two* --verbose to print extra information on --version is a
weird flex

Change-Id: I05d043da1bf583f34e9d1fc206144ea4ca9a859d
2024-05-10 11:55:17 -06:00
eldritch horrors ceccac835c libutil: remove callback.hh
it's no longer used. it really shouldn't have existed this long since it
was just a mashup of both std::promise and std::packaged_task in a shape
that makes composition unnecessarily difficult. all but a single case of
Callback pattern calls were fully synchronous anyway, and even this sole
outlier was by far not important enough to justify the extra complexity.

Change-Id: I208aec4572bf2501cdbd0f331f27d505fca3a62f
2024-05-10 02:21:11 +02:00
eldritch horrors b66451ae7f libstore: de-callback-ify FileTransfer
also add a few more tests for exception propagation behavior. using
packaged_tasks and futures (which only allow a single call to a few
of their methods) introduces error paths that weren't there before.

Change-Id: I42ca5236f156fefec17df972f6e9be45989cf805
2024-05-10 02:21:11 +02:00
eldritch horrors 28a98d152c libstore: de-callback-ify Store::queryRealisation
Change-Id: I8d74745c519518f163f51dfaa39063836f17599e
2024-05-09 23:18:05 +02:00
eldritch horrors 17965bf11c libstore: un-callback-ify Store::queryRealisationUncached
Change-Id: I4a328f46eaac3bb8b19ddc091306de83348be9cf
2024-05-09 23:18:05 +02:00
eldritch horrors 2f4a1dd6e0 libstore: de-callback-ify Store::queryPathInfoUncached
Change-Id: I23a156aaff5328f67ca16ccd85c0ea1711b21e35
2024-05-09 23:18:05 +02:00
eldritch horrors c77bd88259 libstore: de-callback-ify BinaryCacheStore::getFile
Change-Id: I36b3eb9f645aa04058151e7b2353e15e6f29057b
2024-05-09 23:18:05 +02:00
eldritch horrors 1a002d1a11 libstore: de-callback-ify CA realisation substitution
this is the *only* real user of file transfer download completion
callbacks, and a pretty spurious user at that (seeing how nothing
here is even turned on by default and indeed a dependency of path
substitution which *isn't* async, and concurrency-limited). it'll
be a real pain to keep this around, and realistically it would be
a lot better to overhaul substitution in general to be *actually*
async. that requires a proper async framework footing though, and
we don't have anything of the sort, but it's also blocking *that*

Change-Id: I1bf671f217c654a67377087607bf608728cbfc83
2024-05-09 23:18:05 +02:00
Qyriad 1f9b0fba23 add and fix -Wignored-qualifiers
Change-Id: I4bffa766ae04dd80355f9b8c10e59700e4b406da
2024-05-09 07:08:44 -06:00