Commit graph

95 commits

Author SHA1 Message Date
Tom Hubrecht a39ba22ff7 util.hh: Delete remaining file and clean up headers
Change-Id: Ic1f68e6af658e94ef7922841dd3ad4c69551ef56
2024-05-29 12:38:51 +02:00
Tom Hubrecht b910551120 util.{hh,cc}: Split out strings.{hh,cc}
Change-Id: I4f642d1046d56b5db26f1b0296ee16a0e02d444a
2024-05-29 11:01:34 +02:00
Tom Hubrecht 8cd9aa24a8 util.{hh,cc}: Split out file-descriptor.{hh,cc}
Change-Id: I0dd0f9a9c2003fb887e076127e7f825fd3289c76
2024-05-29 09:54:47 +02:00
eldritch horrors 821ad98beb Revert "libutil: drop Fs{Source,Sink}::good"
This reverts commit 1340807e30dba4b3972c31f02861bbaeaeb60e61.

Change-Id: I34d2a80eb3c3e9d79cb02b92cd1189da32d18cb6
2024-04-05 20:13:02 +00:00
eldritch horrors 45623f077f libutil: drop Fs{Source,Sink}::good
setting this only on exceptions caused by actual fd access is not
sufficient to diagnose all errors (such as SerialisationError) in
some cases. this usually does not have any negative effects since
those errors will end up killing the process in another way. this
is not a reliable assumption though and we should be using proper
error handling (and closing connections more often, preferring to
close over keeping something open that might be in a weird state)

Change-Id: I1b792cd7ad8ba9ff0f6bd174945ab2575ff2208e
2024-03-31 16:42:40 +00:00
eldritch horrors 68f148ed45 Merge pull request #9798 from edolstra/remote-store-eof
Print a more helpful message if the daemon crashes

(cherry picked from commit 32706b14a7531c2c21b9f96da083a540a0031ec4)
Change-Id: Ief7c465bca7666e2b7e7c9d1dd0c01c5f9014146
2024-03-04 08:12:15 +01:00
Eelco Dolstra 33fc09c2a5
Merge pull request #8176 from tweag/rename-confusing-write-method
Rename and protect `BufferedSink::write`
2023-04-14 10:44:36 +02:00
Yorick 58d24a4cb6 Always disable GC in a coroutine unless the patch is applied 2023-04-07 14:54:38 +02:00
Yorick 00bc34430b DisableGC: replace by CoroutineContext, std::shared_ptr<void> 2023-04-07 14:53:40 +02:00
Yorick 2c53ef1bfe Disable GC inside coroutines on mac OS 2023-04-07 14:52:59 +02:00
Théophane Hufschmitt 214f1d6791 Rename and protect BufferedSink::write
The `write` name is ambiguous and could lead to some funny bugs like
https://github.com/NixOS/nix/pull/8173#issuecomment-1500009480. So
rename it to the more explicit `writeUnbuffered`.
Besides, this method shouldn't be (and isn't) used outside of the class
implementation, so mark it `protected`.

This makes it more symetrical to `BufferedSource` which uses a
`protected readUnbuffered` method.
2023-04-07 09:21:50 +02:00
John Ericson f4ab297b31 Ensure all headers have #pragma once and are in API docs
`///@file` makes them show up in the internal API dos. A tiny few were
missing `#pragma once`.
2023-03-31 23:19:44 -04:00
John Ericson abd5e7dec0 Extend internal API docs, part 2
Picking up from #8111.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2023-03-31 23:01:40 -04:00
Théophane Hufschmitt 39700c5cbe
Revert "Disable GC during coroutine execution + test" 2023-03-08 20:47:52 +01:00
Yorick 176005749c
Always disable GC in a coroutine unless the patch is applied 2023-03-01 15:07:00 +01:00
Yorick 4c73eab923
DisableGC: replace by CoroutineContext, std::shared_ptr<void> 2023-03-01 13:55:41 +01:00
Yorick eaeb994d8b
Disable GC inside coroutines on mac OS 2023-03-01 13:55:41 +01:00
Eelco Dolstra 703d863a48 Trivial changes from the lazy-trees branch 2022-12-07 14:06:34 +01:00
Eelco Dolstra 53e7b7e8ac Remove warnLargeDump()
This message was unhelpful (#1184) and probably misleading since
memory is O(1) in most cases now.
2022-08-17 11:32:01 +02:00
Eelco Dolstra df552ff53e Remove std::string alias (for real this time)
Also use std::string_view in a few more places.
2022-02-25 16:13:02 +01:00
Eelco Dolstra d62a9390fc Get rid of std::shared_ptr<std::string> and ref<std::string>
These were needed back in the pre-C++11 era because we didn't have
move semantics. But now we do.
2022-01-18 11:12:30 +01:00
Eelco Dolstra 52ee7ec002 StringSource: Use std::string_view 2022-01-17 22:20:23 +01:00
Eelco Dolstra 776eb97a43 serialise.hh: Use std::string_view 2022-01-17 22:20:23 +01:00
Yorick 8a0c00b856
Use libarchive for all compression 2021-03-10 22:34:29 +01:00
Eelco Dolstra 1b79b5b983 read(): Use char * instead of unsigned char *
This gets rid of some pointless casts.
2020-12-02 14:17:27 +01:00
Eelco Dolstra faa31f4084 Sink: Use std::string_view 2020-12-02 14:17:27 +01:00
Robert Hensing c4d903ddb0 Fix memory corruption caused by GC-invisible coroutine stacks
Crucially this introduces BoehmGCStackAllocator, but it also
adds a bunch of wiring to avoid making libutil depend on bdw-gc.

Part of the solutions for #4178, #4200
2020-10-30 21:21:59 +01:00
Eelco Dolstra be149acfda Serialize exceptions from the sandbox process to the parent
Fixes #4118.
2020-10-07 16:34:03 +02:00
Eelco Dolstra 92ac8df0ec Merge branch 'add-ca-to-store' of https://github.com/hercules-ci/nix 2020-09-22 11:31:33 +02:00
regnat c1e79f870c Silence a compiler warning in serialise.hh
Explicitely cast to `uint64_t` in `readNum` to avoid a "comparison
between signed and unsigned" warning
2020-09-22 10:39:29 +02:00
Robert Hensing 8279178b07 Move FramedSink next to FramedSource 2020-09-21 07:55:47 +02:00
Robert Hensing 14b30b3f3d Move FramedSource and FramedSink, extract withFramedSink 2020-09-17 20:21:04 +02:00
Robert Hensing 29c82ccc77 Add Source.drainInto(Sink) 2020-09-17 20:21:04 +02:00
John Ericson 8017fe7487 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-08-28 19:59:14 +00:00
Eelco Dolstra e915fd6d2a Typo 2020-08-27 14:51:50 +02:00
Eelco Dolstra a0f19d9f3a RemoteStore::addToStore(): Fix race between stderrThread and NAR writer
As pointed out by @B4dM4n, the call to to.flush() on stderrThread is
unsafe because the NAR writer thread is also writing to 'to'.

Fixes #3943.
2020-08-27 14:50:51 +02:00
John Ericson 3c8b5b6219 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-08-14 17:00:13 +00:00
John Ericson 85aacbee64 Use TeeSink and TeeSouce in a few more places 2020-08-13 14:51:17 +00:00
John Ericson e913a2989f Squashed get CA derivations building 2020-08-07 19:51:55 +00:00
Eelco Dolstra 3f6e88a552 unsigned long long -> uint64_t 2020-07-30 13:34:04 +02:00
John Ericson 3dcca18c30 Fix bug in TeeSource
We use this to simplify `LocalStore::addToStoreFromDump`.

Also, hope I fixed build error with old clang (used in Darwin CI).
2020-07-16 13:39:27 +00:00
John Ericson 5602637d9e Revert "LocalStore::addToStoreFromDump copy in chunks"
This reverts commit 592851fb67. We don't
need this extra feature anymore
2020-07-15 23:37:49 +00:00
John Ericson bc109648c4 Get rid of LocalStore::addToStoreCommon
I got it to just become `LocalStore::addToStoreFromDump`, cleanly taking
a store and then doing nothing too fancy with it.

`LocalStore::addToStore(...Path...)` is now just a simple wrapper with a
bare-bones sinkToSource of the right dump command.
2020-07-15 23:14:30 +00:00
John Ericson 592851fb67 LocalStore::addToStoreFromDump copy in chunks
Rather than copying byte-by-byte, we let the coroutine know how much
data we would like it to send back to us.
2020-07-14 13:56:36 +00:00
Eelco Dolstra 7c2fef0a81 Make 'nix copy' to s3:// binary caches run in constant memory 2020-07-13 20:07:19 +02:00
Eelco Dolstra 0a9da00a10 NarAccessor: Run in constant memory 2020-07-13 17:30:42 +02:00
Eelco Dolstra fc84c358d9 Make 'nix copy' to file:// binary caches run in constant memory 2020-07-13 16:28:45 +02:00
Guillaume Bouchard 7afcb5af98 Remove the drain argument from readFile
Now it is always `drain` (see previous commit).
2020-04-29 18:43:45 +02:00
Eelco Dolstra 8918bae098 Drop remaining uses of external "tar"
Also, fetchGit now runs in O(1) memory since we pipe the output of
'git archive' directly into unpackTarball() (rather than first reading
it all into memory).
2019-11-26 22:07:28 +01:00
John Ericson 96e6e680c1 Fix extra ; warnings involving MakeError 2019-11-10 11:24:47 -05:00