Commit graph

15245 commits

Author SHA1 Message Date
eldritch horrors bb253a95cb libutil: convert readFile to generator
Change-Id: I5f92b15fd367d46eb047d74ab6e317b4f51a46d3
2024-03-26 02:23:48 +01:00
eldritch horrors 98d9dcb221 libutil: convert drainFD to source
Change-Id: I54eb2e59a4009014e324797f16b80b962759c7d3
2024-03-26 02:23:48 +01:00
eldritch horrors 6f30245e0a libstore: have path dump return a generator
Change-Id: Ic4cf5562504aa29130304469936f958c0426e5ef
2024-03-26 02:23:48 +01:00
eldritch horrors 05c04089ee libutil: prototype serialization mechanism on coroutines
Change-Id: I361d89ff556354f6930d9204f55117565f2f7f20
2024-03-26 02:23:48 +01:00
eldritch horrors 1aa630bdf5 libutil: add some serialize.hh serializer tests
Change-Id: I0116265a18bc44bba16c07bf419af70d5195f07d
2024-03-26 02:23:48 +01:00
eldritch horrors f19e7f6974 libutil: basic generator type with mapping
Change-Id: I2cebcefa0148b631fb30df4c8cfa92167a407e34
2024-03-26 02:23:48 +01:00
eldritch horrors a81ec42ecf libutil: move overloaded into own header
util.hh is huge and an upcoming new header wants just overloaded, not
the entire rest. also step towards reducing compile times.

Change-Id: I5ebe8c4692b4e41bc794b8f2d6d138073134dd9e
2024-03-26 02:23:48 +01:00
eldritch horrors e94033c4af libutil: make ChainSource *more*
- make it chain more than two sources together
- make it own its parts
- make it properly moveable
- test it

this is the first step towards eliminating a bunch of produce-to-sink
functions we have right now (such as readFile, dumpPath, etc).

Change-Id: I518eb7a7f9e1e1a9199a410074c83b77b38c8a06
2024-03-26 02:23:47 +01:00
eldritch horrors b627b0b432 libutil: drop Fs{Source,Sink}::good
setting this only on exceptions caused by actual fd access is not
sufficient to diagnose all errors (such as SerialisationError) in
some cases. this usually does not have any negative effects since
those errors will end up killing the process in another way. this
is not a reliable assumption though and we should be using proper
error handling (and closing connections more often, preferring to
close over keeping something open that might be in a weird state)

Change-Id: I1b792cd7ad8ba9ff0f6bd174945ab2575ff2208e
2024-03-26 02:23:47 +01:00
eldritch horrors 85fce67b8a libutil: allow graceful dropping of Pool::Handle
not needed yet, but returning a resource from the exception handling
path that has ownership of a handle is currently not well-supported.
we could also add a default constructor to Handle, but then we would
also need to change the pool reference to a pointer. eventually that
should be done since now resources can be swapped between pools with
clever moves, but since that's not a problem yet we won't do it now.

Change-Id: I26eb06581f7be34569e9e67a33da736128d167af
2024-03-26 02:23:47 +01:00
eldritch horrors faac17405e libstore: using throwing finally in withFramedSink
the duplication of exception handling was added without justification,
so we can only assume that it was done like this because Finally could
not throw exceptions safely. since this has now been rectified we will
deduplicate this handler code again.

Change-Id: I40721f3378c0fd9f34e2914a16d383f6e2713b40
2024-03-26 02:23:47 +01:00
eldritch horrors a5e40bf4fe libutil: make ~Finally noexcept(false)
this is supposed to act like a finally block does in other languages. a
finally block should be able to throw exceptions of its own rather than
just crashing the entire program when it throws it own exceptions. even
in the rare case of a finally throwing an unexpected exception it might
be better to report the exception from Finally instead of the original,
at least that can keep our program running instead of letting it crash.

Change-Id: Id42011e46b1df369152b4564938c0e93fa1acf32
2024-03-26 02:23:47 +01:00
eldritch horrors 0b77ba6980 libstore: remove one Resource::good flag
usage of this flag previously kept connections open much longer than
necessary, and at the same time obscured that a connection was being
dropped when it *was* set. new variable names clarify this somewhat.

Change-Id: I11f6f08f37a5e4dc04ea6c6036ea589154b121c6
2024-03-26 02:23:47 +01:00
eldritch horrors 7f94f986d4 libutil: remove Pool::Handle::bad
it was used incorrectly (not swapped on handle move), only used in one
place (that is now handled with exception handling detection in Handle
itself), and if ever reintroduced should be replaced with a different,
more understandable mechanism (like an explicit dropAsInvalid method).

Change-Id: Ie3e5d5cfa81d335429cb2ee5c3ad85c74a9df17b
2024-03-26 02:23:47 +01:00
eldritch horrors 4556892c17 libutil: remove Pool::flushBad
this was never actually used, and bad design in the first place—why
should a bad resource be put back into the idle pool? just drop it.

Change-Id: Idab8774bee19dadae0209d404c4fb86dd4aeba1e
2024-03-26 02:23:47 +01:00
eldritch horrors fe7e924026 libutil: drop Pool resources on exceptional free
if a scope owning a resource does not gracefully drop that resource
while handling exceptions from deeper down the call stack we should
assume the resource is invalid state and drop it. currently it *is*
true that such cases do not cause resources to be freed, but thanks
to validator misuses this has so far not caused any larger problem.

Change-Id: Ie4f91bcd60a64d05c5ff9d22cc97954816d13b97
2024-03-26 02:23:47 +01:00
Lunaphied 73624f7d9c Merge "Merge pull request #8817 from iFreilicht/flake-update-lock-overhaul" into main 2024-03-26 00:45:49 +00:00
Théophane Hufschmitt 86881226b0 Merge pull request #8817 from iFreilicht/flake-update-lock-overhaul
Overhaul `nix flake update` and `nix flake lock` UX

(cherry picked from commit 12a0ae73dbb37becefa5a442eb4532ff0de9ce65)
Change-Id: Iff3b4f4235ebb1948ec612036b39ab29e4ca22b2
2024-03-25 17:36:24 -06:00
eldritch horrors c3a5f937d3 flake.nix: linearize meson builds
parallel meson builds need too much ram. linearize them for now, and
hopefully we can remove the make build system and this hack soonish.

Change-Id: I42c092db8b0c63680e77da2263cdfe9e7f6575be
2024-03-25 21:48:55 +00:00
Qyriad 787c4397f1 Merge "issue template: use nix --version instead of nix-env --version" into main 2024-03-25 21:47:53 +00:00
Qyriad 1da1aa5045 issue template: use nix --version instead of nix-env --version
`nix --version` doesn't require `nix-command` experimental feature to
run and we could all do with less nix-env

Change-Id: I90748d591c574d96eda46591e9f9ce828311da29
2024-03-25 12:59:30 -06:00
Eelco Dolstra aa7653608d Minor cleanup in libexpr/flake/flake.cc
(cherry picked from commit 05316d401fa509557c71140e17bb19814412fcb8)
Change-Id: I6ba0b55709f5fe21beb4e9f3bf72ee28715d15f3
2024-03-25 15:30:36 +00:00
Eelco Dolstra b525d0f20c Input: Replace markFileChanged() by putFile()
Committing a lock file using markFileChanged() required the input to
be writable by the caller in the local filesystem (using the path
returned by getSourcePath()). putFile() abstracts over this.

(cherry picked from commit 95d657c8b3ae4282e24628ba7426edb90c8f3942)
Change-Id: Ie081c5d9eb4e923b229191c5e23ece85145557ff
2024-03-25 15:30:36 +00:00
John Ericson 3d065192c0 Overhaul completions, redo #6693 (#8131)
As I complained in
https://github.com/NixOS/nix/pull/6784#issuecomment-1421777030 (a
comment on the wrong PR, sorry again!), #6693 introduced a second
completions mechanism to fix a bug. Having two completion mechanisms
isn't so nice.

As @thufschmitt also pointed out, it was a bummer to go from `FlakeRef`
to `std::string` when collecting flake refs. Now it is `FlakeRefs`
again.

The underlying issue that sought to work around was that completion of
arguments not at the end can still benefit from the information from
latter arguments.

To fix this better, we rip out that change and simply defer all
completion processing until after all the (regular, already-complete)
arguments have been passed.

In addition, I noticed the original completion logic used some global
variables. I do not like global variables, because even if they save
lines of code, they also obfuscate the architecture of the code.

I got rid of them  moved them to a new `RootArgs` class, which now has
`parseCmdline` instead of `Args`. The idea is that we have many argument
parsers from subcommands and what-not, but only one root args that owns
the other per actual parsing invocation. The state that was global is
now part of the root args instead.

This did, admittedly, add a bunch of new code. And I do feel bad about
that. So I went and added a lot of API docs to try to at least make the
current state of things clear to the next person.

--

This is needed for RFC 134 (tracking issue #7868). It was very hard to
modularize `Installable` parsing when there were two completion
arguments. I wouldn't go as far as to say it is *easy* now, but at least
it is less hard (and the completions test finally passed).

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Change-Id: If18cd5be78da4a70635e3fdcac6326dbfeea71a5
(cherry picked from commit 67eb37c1d0de28160cd25376e51d1ec1b1c8305b)
2024-03-25 15:30:36 +00:00
Tom Bereknyei 4494f9097f feat: notation to refer to no attribute search prefix
An attrPath prefix of "." indicates no need to try default attrPath prefixes. For example `nixpkgs#legacyPackages.x86_64-linux.ERROR` searches through

```
trying flake output attribute 'packages.x86_64-linux.legacyPackages.x86_64-linux.ERROR'
using cached attrset attribute ''
trying flake output attribute 'legacyPackages.x86_64-linux.legacyPackages.x86_64-linux.ERROR'
using cached attrset attribute 'legacyPackages.x86_64-linux'
trying flake output attribute 'legacyPackages.x86_64-linux.ERROR'
using cached attrset attribute 'legacyPackages.x86_64-linux'
```

And there is no way to specify that one does not want the automatic
search behavior. Now one can specify
`nixpkgs#.legacyPackages.x86_64-linux.ERROR` to only refer to the rooted
attribute path without any default injection of attribute search path or
system.

Change-Id: Iac1334e1470137b7ce11dcf845513810230638ec
(cherry picked from commit d4aed18883b361133607296fb6cd789c47427a38)
2024-03-25 15:30:36 +00:00
Lunaphied d3d7489571 Merge "Improve new CLI UX by supporting short -E flag for --expr" into main 2024-03-25 14:13:44 +00:00
jade 5a1c35f907 Merge "Restore system-install profile files from the previous installer" into main 2024-03-25 04:17:36 +00:00
jade dee6b75702 Restore system-install profile files from the previous installer
These files are required to get Nix in PATH in existing multi-user installs using
the legacy installer. We really could use some tests.

Cc: #33

This partially reverts commit 93cc063344.

Fixes: #173

Change-Id: Iafb55280596732670a432f604b897f48562868e4
2024-03-25 04:17:14 +00:00
Lunaphied 185ecf1f45 Improve new CLI UX by supporting short -E flag for --expr
Change-Id: I55881c846da8416a92a14deedfa5bbbf09a122fb
2024-03-24 21:17:51 -06:00
eldritch horrors c856b82c2e libstore: despecialcase protocol version check
protocol versions are sent as u64. on the peer we read them as uint64,
check that the upper half is 0, and throw an exception if not. we then
read an arbitrary amount of data from the peer and dump it to the user
terminal. this is a little bit ridiculous, can never happen in correct
implementation, and is severly untested. let us just drop it entirely.

Change-Id: Ibd2f53a765341ed6439d40d9d1eac11e79c6b5e3
2024-03-24 18:45:22 +00:00
eldritch horrors 3e428f2289 libstore: un-inline copyNAR expansions
these are copies of copyNAR with only some variables renamed.

Change-Id: I98ddd7a98250fa5d304e18e1debf417e9f7768dd
2024-03-24 15:24:02 +01:00
jade 946fc12e4e Revert "Merge pull request #9476 from alois31/restore-progress-bar"
Observed to regress nix repl attrset printing with narrow windows.

This reverts commit a2d5e803cf.

Fixes: #168

Change-Id: I8e0031475b4ec26d6a71014357d973578b70815c
2024-03-23 18:04:29 -07:00
eldritch horrors 652f52f071 libutil: don't memset 64k in drainFD
this is not needed and introduces a bunch of memset calls, making up for
3% of valgrind cycle estimation *alone*. real-world impact is a lot
lower on our test machine, but we suspect that less powerful machines
would see an impact from dropping this.

Change-Id: Iad10e9d556e64fdeb0bee0059a4e52520058d11e
2024-03-23 22:17:46 +00:00
Qyriad b4d07656ff build: optionally build and install with meson
This commit adds several meson.build, which successfully build and
install Lix executables, libraries, and headers. Meson does not yet
build docs, Perl bindings, or run tests, which will be added in
following commits. As such, this commit does not remove the existing
build system, or make it the default, and also as such, this commit has
several FIXMEs and TODOs as notes for what should be done before the
existing autoconf + make buildsystem can be removed and Meson made the
default. This commit does not modify any source files.

A Meson-enabled build is also added as a Hydra job, and to
`nix flake check`.

Change-Id: I667c8685b13b7bab91e281053f807a11616ae3d4
2024-03-22 08:36:50 -06:00
jade a7161b6c0f Merge "clang-tidy check infrastructure" into main 2024-03-21 12:28:13 -06:00
Qyriad fab55aff0e flake: fix arm32 Linux cross devShell on macOS (fix nix flake check)
Change-Id: Iacac97de0b3d5f2df52c7bc985148624a351f45d
2024-03-21 08:22:38 -06:00
eldritch horrors 22e3f0e987 libexpr: unbreak PosTable performance
this was mostly an inconvenience for error reporting, but fully broke
the debugger (because the debugger does *a lot* of eager position
resolution). copying the line offsets into a local and filling that
local when empty without also storing the calculated offsets back does
kind of ... not cache anything.

fixes #165

Change-Id: Iccb0ba193ce2f15c832978daecf7b9bebbbe8585
2024-03-20 13:45:36 +01:00
jade 5a28d70d1e Merge "un-ups your start" into main 2024-03-19 14:29:57 -06:00
eldritch horrors d9a83886f9 libutil: remove exception handling workingness check
within lix itself this problem is caught by the test suite. outside of
lix itself three cases can be had: either the problem is fully inside
lix libs, fully inside user code, or it exists at the boundary. the
first is caught by the test suite, the second isn't caught at all, and
the third is something lix should not be responsible for.

Change-Id: I95aa35d8cb6f0ef5816a2941c467bc0c15916063
2024-03-19 06:09:42 -06:00
jade 4050245faa Merge changes I72c945ca,I2138bb4d,Ib96749f3 into main
* changes:
  Release notes for builtins.nixVersion change
  un-nixes ur lix, a little
  issue importer: list issues that are *not* closed when finding existing issues
2024-03-18 20:19:53 -06:00
jade 985bd5eb9f un-ups your start
We do not need upstart, it is so thoroughly obsolete that we should not
care about supporting it.

Change-Id: Ie0ca084740845555fddffacc899cd129c9a4c1fe
2024-03-18 18:28:08 -07:00
jade 20b4a97af3 Release notes for builtins.nixVersion change
Change-Id: I72c945cab464d26d73f5594ef0a4bb2184545da4
2024-03-18 18:26:52 -07:00
jade 30233d87f9 un-nixes ur lix, a little
I didn't really go attack the docs because we need to pull a bunch of
PRs. I went looking for strings in the code that called lix nix.

Change-Id: I2138bb4dd239096bc530946b281db7f875195b39
2024-03-18 18:20:24 -07:00
jade 81be5eb7c6 issue importer: list issues that are *not* closed when finding existing issues
Turns out also, you cannot set the queue to 0 with any success. So we
really should just like, prevent notifications in forgejo itself.

Filed a bug for that:
lix-project/web-services#38

Change-Id: Ib96749f3159659182904963cab7b2ef88fc64442
2024-03-18 18:14:31 -07:00
jade 6b0020749d clang-tidy check infrastructure
This brings in infrastructure for developing new custom clang-tidy lints
and refactors for Lix.

Change-Id: I3df5f5855712ab4f97d4e84d771e5e818f81f881
2024-03-18 16:10:29 -07:00
eldritch horrors f38ae92a38 libutil: make AutoCloseFD a better resource
add a reset() method to close the wrapped fd instead of assigning magic
constants. also make the from-fd constructor explicit so you can't
accidentally assign the *wrong* magic constant, or even an unrelated
integer that also just happens to be an fd by pure chance.

Change-Id: I51311b0f6e040240886b5103d39d1794a6acc325
2024-03-18 15:42:52 -06:00
eldritch horrors 0f518f44e2 Merge "libexpr: associate let exprs with the correct StaticEnv" into main 2024-03-18 15:20:21 -06:00
eldritch horrors afb839a0c9 libexpr: associate let exprs with the correct StaticEnv
static env association is from expr to its enclosing scope, but let
exprs set their association to their *inner* scope. this skips one level
of envs and will cause segfaults if the parent is a with expr.

fixes #145

Change-Id: I1d22146110f071ede21b4eed7ed34b5850ef2ef3
2024-03-18 14:15:22 -07:00
jade 37c4b10c44 Add clang format configuration
Contemplate the configuration with: https://clang-format-configurator.site/

(cherry picked from commit 53fdcbca509b6c5dacaea3d3c465d86e49b0dd74)
Change-Id: I5446fd45de2bf644e34112f719afb3318a440b30
2024-03-18 13:31:39 -07:00
eldritch horrors b3599166ad libexpr: sort binding name in debugger
not doing this exposes the binding name order to the annoying
interference of parse order on symbol order, which wouldn't be so bad if
it didn't make the tests less reliable and, importantly, dependent on
linker behavior (due to primop initialization being done in static
initializer, and the order of static initializers being defined only
within a single translation unit).

fixes #143

Change-Id: I3cf417893fbcf19e9ad3ff8986deb7cbcf3ca511
2024-03-18 20:03:31 +01:00