Ever read gdb output and you just kinda get a headache because you have
to infer what a thread is by reading the stack trace? It's not hard, but
we could also just never have to do that again, which is also not hard.
Sample:
(gdb) info thr
Id Target Id Frame
* 1 LWP 3719283 "nix-daemon" 0x00007e558587da0f in accept ()
from target:/nix/store/c10zhkbp6jmyh0xc5kd123ga8yy2p4hk-glibc-2.39-52/lib/libc.so.6
2 LWP 3719284 "signal handler" 0x00007e55857b2bea in sigtimedwait ()
from target:/nix/store/c10zhkbp6jmyh0xc5kd123ga8yy2p4hk-glibc-2.39-52/lib/libc.so.6
The API design for this is forced by the macOS pthread_setname_np only
being able to change the current thread's name, but if we just conform
everything to that, it works everywhere.
Change-Id: I2b1d6ed41e3c94170cb0b4e73ad66f239ebd9c88
This allows <lix/*> includes to work from external projects not taking
Lix as a buildInput with Nix, but just using pkg-config.
Change-Id: I5b24fd0981344b013beaf406c16988dad17f0edb
Context: we have include paths that are "types.hh" and similarly common
names. We currently have these compatibly available as
"lix/libutil/types.hh" externally but *not yet internally*. This is
because we don't have any way for the src directory to appear as
`"lix/"` from inside of Lix: the lix/ include directory is created by
the install process.
The goal of this whole thing is to make it clearer which component of
Lix that files are a part of, which should hopefully help at least a
little bit to new developers. One disadvantage of un-mixing these is
that it will cause some API changes if we ever move a file between
libraries, but that is not very common, and we don't care that much
about external API users.
This was planned for a while and is why we have a FixIncludes check to
begin with.
Personally I don't see a great benefit in rearranging our source code,
and in fact, it would probably be counterproductive:
- Moving the includes into a separate `include/` directory would just
make developers have to deal with more directories, when we can
already generate the desired layout through the build process.
- This would also decouple the .cc and .hh files which currently
conventionally have each others' definitions and declarations
respectively, right next to each other, making it easier for them to
feel decoupled and diverge.
Content: Add ../include as an include directory so that lix/ in include
paths will resolve to src/ within Lix itself, just as it does externally
today. This prepares for a further series of commits applying the actual
change to each library one-by-one by accepting both include versions at
once.
This could have been done with ../ and a symlink called lix, but we
would like to not accept libexpr/foo.hh internally for it would be
broken externally, so we need an otherwise empty directory for the
include.
Change-Id: Ideac17faadae2bcea2dffbab34eb27c582ede399
transferComplete has outlived its usefulness, and the other two phases
can be a boolean flag telling us whether we're still receiving headers
Change-Id: Ia943f95dcf0ce8ee2cc6d0f44ddf13151aef4346
things can be much simpler if the read loop knows whether the transfer
is done or not. this information is available, let's pass it back down
Change-Id: Idb99afeb06a0d7a0999c3f4a1c6ce5adeab5f054
file:// urls are no longer handled by curl itself, and this is supposed
to test the curl wrapper. use http to force the wrapper to be involved.
Change-Id: Ib13087db07b3b2f1ae44ce8e3ec7a96d935b1bab
they don't need these any more if we consider unpausing and cancelling
of transfers a cooperative affair between transfers and the curl_multi
handler. we can also remove an enable_shared_from_this base class now.
Change-Id: Iefa380df60c0cada53719cdaa500bab953892319
wrap all curl pointers in unique pointers for move correctness. the
destructor of TransferItem can go away completely since it does not
do anything special any more: memory bookkeeping is done using raii
wrappers, and its effects on the transfer state are not observable.
only TransferSource inspects the result, and even it does not check
for transfer results in its own destructor (only during transfers).
Change-Id: Ibb58484703854d5d5425e7c63cf2985d93920e97
keep the download buffer in TransferItem. keep only *one* download
buffer (not one for success returns and one for failures). we also
remove the asynchronous callbacks since they are no longer needed.
Change-Id: I9b1c1e4c59d5333602aea1c36e477c481eaaf98e
when no custom READFUNCTION is set curl will fall back to using its
READDATA as a FILE pointer and calling fread on it to retrieve data
to send to the peer. we can wrap our upload data in a FILE pointer.
doing so also allows for upload retries, but we don't do that here.
Change-Id: I0509f936229dd978e89135862dacc8a3937cd4de
The progress bar is not supposed to show very short activities (<10ms) in order
to prevent excessive flickering. This behaviour was inadvertedly disabled in
commit da4e46dd1f due to not actually skipping
the short activity after calculating that it should have been. Fix this
oversight.
Fixes: lix-project/lix#561
Change-Id: Ibee256a7018cdc431662cd8ea9f6c14ef1706972
* changes:
libexpr: generate builtins from data
treewide: generate global settings from data
libutil: generate experimental and deprecated features from data
this function is called only once per object lifetime, and it's no
longer written such that it must be called by the curl thread. all
curl options set here are independent of the multi handle, there's
no good reason any more to split the constructor in two like that.
Change-Id: Ibfa9075bc1cfdbab979f995c5a942b47add9789d
this simplifies the immediate curl wrappers significantly and clarifies
control flow for retries. we can almost see the promise land from here!
Change-Id: Idc66b744631eec9c1ad5a2be388beb942a04f1f9
future changes will need to add template functions to this class, which
cannot be done for classes declared locally in unctions. otherwise this
is mostly code motion to clean up enqueueFileTransfer a little further.
Change-Id: If9a9d9eb47ceadfa75a4eebd54e2db39f2305643
it's used only once, and the transfer is used for something else
afterwards as well. returning it hides that the same transfer is
being returned unchanged, and pointlessly extends the signature.
Change-Id: Idbd0af00f9aebd3606f6b537b9b7cc9263a45cf7
we only wait for the worker thread to remove the handle from its multi
and then signal us back. after that's done the transfer can be cleaned
up independently of the multi handle and no transfer progress is made.
we also no longer remove transfer items from multi handles in the item
destructor; only the worker thread can add a transfer and as such only
the worker thread should be responsible for removing them again later.
Change-Id: I6ff57bb5e7c1e91faf8657b257e73d6a72aa928e
this only ever worked for empty uploads, and there it worked only by
complete accident: curl was asked to send more data than the wrapper
would provide, which curl would not like and report as an error. the
error would cause a retry with even less data to send, until finally
failing by running into the retry limit. let's just forbid all this.
Change-Id: I229a94b3b8b33e2c6cdb8ea19edd57cd6740e6c6
we'll need this shared code for kjified transfers that don't use
sources. it's a while out, but we can clean this up now already.
Change-Id: Ife8c160e6ab379761362d6c54aba05093deee99e
since 4ae6fb5a8f dropping a source of a
download might not properly cancel the associated curl transfer after
the transfer was paused. we have also not unpaused the transfer often
enough, only if the transfer buffer had been drained in its entirety.
Change-Id: Ic9298d9df71daa0f3d1c3fd718ed441edae9e863
Most builtins are now generated from data too, with two exceptions:
* Undocumented builtins, since supporting them would add complexity to the
generator, the harms of the current implementation mostly don't apply, and
the proper fix is to document them.
* `derivation` is somewhat magic (it is a function, but defined in the code as
a constant), so the current treatment of having it separately documented is
kept (for now, at least).
Since it is slightly easier to do and probably a good idea anyway, the builtin
function registrations generated this way are now processed directly in code
and don't go through global variables any more.
Unfortunately, a slight breaking change is introduced because the order of the
builtins' names in the symbol table changes. Hopefully, this will turn out to
not matter in practice.
Change-Id: I7b4379a93ae380b6524e41a916a21c5c6f70555e
Currently, a bunch of documentation is generated by embedding parts of it in
the nix executable, getting it out again by running it, and then postprocessing
the output. This is bad, since it creates a pointless dependency of the
documentation on the executable, and also makes documentation generation
impossible when cross-compiling.
Instead, both the code and the documentation should be generated from data, see
lix-project/lix#292 . Here we start applying
this approach to the experimental and deprecated features, which are done in
one go since the technical implementation is very similar.
Of course, the actual benefits are not realised yet, since the offending
pattern is used in several more places. These will be fixed later.
Change-Id: I4c802052cc7e865c61119a34b8f1063c4decc9cb
Commit 4dbbd721eb intended to mark all settings
as overridden when they are. Unfortunately, due to an oversight, this marking
was accidentally performed in the implementation details of non-appendable
options. Move it to the common codepath so that it works for appendable options
too.
Fixes: lix-project/lix#573
Change-Id: Idc3402bac48b19d832acd9b553e16e5791470c26
So I recently saw it the first time in the wild, I liked that you get
interactively asked about the nix.conf settings from the flake, but
there were a few minor things that I'd like to see changed:
* The `(y/N)` was somewhere in the middle of the line. Moved it to
the end. At first I assumed it was a bug because another thread into
my terminal while I was answering the question.
* I had to say no four times for a single flake with two options. So if
you already know you don't want any of the config for _this_ flake, I
found a `No to all` switch that ignores the rest of the nix.conf
settings a little more ergonomic than having to stop the invocation,
looking up the exact wording of `--no-accept-flake-config` and
restarting it. Hence, I added it.
* Added a note where the choices which settings to trust are persisted.
My initial assumption was that this went into `nix.conf` which is not
writable on NixOS, so I said no there as well.
Change-Id: I0a0d9c403f0662df4707697a77f08e6cd003ec6f
This adds a new temp-dir setting for controlling the temporary directory
without having to change the TMPDIR env var. This can be used to e.g.
use a path on a case-sensitive store on macOS for temporary files
without changing the TMPDIR var used by interactive shells or commands
invoked with `nix run`.
This also stops unsetting `TMPDIR` on darwin when the env var value
starts with `/var/folders/`, preferring instead to just do the check
when reading `TMPDIR`. This way the inherited `TMPDIR` env var is
preserved for child processes (such as interactive shells).
As a side effect this changes the behavior of `nix-build -o ''` to act
like `nix-build --no-out-link` instead of failing with an error caused
by trying to create a symlink at the cwd.
Fixes: lix-project/lix#253
Fixes: lix-project/lix#112
Change-Id: I9ee826323f2deca62854715a77ca7a373a948a29
libcurl may call callbacks during unpause. if libcurl calls these
callbacks often enough they may find that they've exhausted their
allotted buffer space, at which point they will call dataCallback
as provided in enqueueFileTransfer. download() provides callbacks
that have their own state locks and may call unpause on transfers
with *their* state locks they share with curlFileTransfer. it was
possible to cause a deadlock between these two if the curl worker
thread tried to provide some data to a callback with the transfer
state lock held and the user transfer simultaneously unpaused its
associated curl transfer, with its own state lock held. obviously
not a great situation, but avoidable by not holding any lock when
we unpause transfers and execute their callbacks, as is otherwise
the case when a transfer runs normally and is never paused at all
Change-Id: I58556d292adaf7dfb14001d3a6c5c38fa71994da
jade
eldritch horrors
Emilia Bopp
alois31
Lily Ballard
Maximilian Bosch
V.
Kiara Grouwstra
kloenk