Commit graph

690 commits

Author SHA1 Message Date
eldritch horrors ceccac835c libutil: remove callback.hh
it's no longer used. it really shouldn't have existed this long since it
was just a mashup of both std::promise and std::packaged_task in a shape
that makes composition unnecessarily difficult. all but a single case of
Callback pattern calls were fully synchronous anyway, and even this sole
outlier was by far not important enough to justify the extra complexity.

Change-Id: I208aec4572bf2501cdbd0f331f27d505fca3a62f
2024-05-10 02:21:11 +02:00
eldritch horrors 17965bf11c libstore: un-callback-ify Store::queryRealisationUncached
Change-Id: I4a328f46eaac3bb8b19ddc091306de83348be9cf
2024-05-09 23:18:05 +02:00
eldritch horrors 2f4a1dd6e0 libstore: de-callback-ify Store::queryPathInfoUncached
Change-Id: I23a156aaff5328f67ca16ccd85c0ea1711b21e35
2024-05-09 23:18:05 +02:00
Artemis Tosini b247ef72dc libstore: Create platform LocalStore subclasses
This creates new subclasses of LocalStore for each OS to include
platform-specific functionality. Currently this just includes garbage
collector roots but it could be extended to sandboxing as well.

In order to make sure that the generic LocalStore is not accidentally
constructed, its constructor is protected. A Fallback is provided which
implements no functionality except constructors.

Change-Id: I836a28e90b68309873f75afb83e0f1b2e2c89fb3
2024-04-23 16:17:05 +00:00
jade 1fa6a3e335 Fix various clang-tidy lints
* some things that can throw are marked noexcept
  yet the linter seems to think not. Maybe they can't throw in practice.
  I would rather not have the UB possibility in pretty obvious cold
  paths.
* various default-case-missing complaints
* a fair pile of casts from integer to character, which are in fact
  deliberate.
* an instance of <https://clang.llvm.org/extra/clang-tidy/checks/bugprone/move-forwarding-reference.html>
* bugprone-not-null-terminated-result on handing a string to curl in
  chunks of bytes. our usage is fine.
* reassigning a unique_ptr by CRIMES instead of using release(), then
  using release() and ignoring the result. wild. let's use release() for
  its intended purpose.

Change-Id: Ic3e7affef12383576213a8a7c8145c27e662513d
2024-03-29 20:26:38 -07:00
raito 8044540c42 feat: unprivileged read-only open of SQLite DB
If the state SQLite database is configured to use a write-ahead-log, it
creates WAL files in the state directory.

When the state SQLite database is closed by the `nix-daemon` after
builds, those files are removed.

When an unprivileged user would like to open _in read only_ that
database, they cannot do so because they would need to create those WAL
files and they do not have the permission to do so.

For this, SQLite offers a "persistent WAL" feature [1] to leave the WAL
files around, even after closing the database.

This CL enable the persistent WAL mode.

Fixes: https://github.com/NixOS/nix/issues/10300
[1]: https://www.sqlite.org/wal.html

Change-Id: Id8ae534d7d2290457af28782e5215222ae051fe5
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-23 15:07:48 +01:00
eldritch horrors f38ae92a38 libutil: make AutoCloseFD a better resource
add a reset() method to close the wrapped fd instead of assigning magic
constants. also make the from-fd constructor explicit so you can't
accidentally assign the *wrong* magic constant, or even an unrelated
integer that also just happens to be an fd by pure chance.

Change-Id: I51311b0f6e040240886b5103d39d1794a6acc325
2024-03-18 15:42:52 -06:00
jade 8be7030299 util.hh: split out signals stuff
Copies part of the changes of ac89bb064aeea85a62b82a6daf0ecca7190a28b7

Change-Id: I9ce601875cd6d4db5eb1132d7835c5bab9f126d8
2024-03-11 00:52:09 -07:00
eldritch horrors 298db5d1cd Merge pull request #9804 from edolstra/missing-nar-crash
Fix crash when NAR is missing from binary cache

(cherry picked from commit 3b20cca9625a1701a10a883735e7315185629563)
Change-Id: I50ff18f4a6de69c323473b4a8e3e098d1f365145
2024-03-04 08:16:28 +01:00
eldritch horrors dd180911d8 Merge pull request #9582 from pennae/misc-opts
a packet of small optimizations

(cherry picked from commit ee439734e924eb337a869ff2e48aff8b989198bc)
Change-Id: I125d870710750a32a0dece48f39a3e9132b0d023
2024-03-04 07:32:31 +01:00
Simon Rainerson 31a6e10fe5 Fix misread of source if path is already valid
When receiving a stream of NARs through the ssh-ng protocol, an already
existing path would cause the NAR archive to not be read in the stream,
resulting in trying to parse the NAR as a ValidPathInfo. This results in
the error message:
    error: not an absolute path: 'nix-archive-1'

Fixes #6253

Usually this problem is avoided by running QueryValidPaths before
AddMultipleToStore, but can arise when two parallel nix processes gets
the same response from QueryValidPaths. This makes the problem more
prominent when running builds in parallel.
2023-08-07 10:27:40 +02:00
John Ericson 66550878df Add comment explaining the use of readDirectory(realStoreDir) 2023-08-02 12:46:07 -04:00
John Ericson 770d50e49c local-store verifying: Rename store to something more clear
It is not a `Store` but a `StorePathSet`.
2023-08-02 12:40:04 -04:00
John Ericson 6525265f46 LocalStore::verifyPath: Try to clarify data flow with more scopes
It was initially unclear to me which of these are temporary state for
the verify paths computation, and which of these are the results of that
computation to be used in the rest of the function. Now, it is clear,
and enforced.
2023-07-31 12:44:27 -04:00
John Ericson 2a5f5fbb17 LocalStore::verifyPath: Use StorePathSet for store local var
We don't care about non-store-paths in there (things like `.links`, are,
in fact, allowed). So let's just skip them up front and be more strongly
typed.
2023-07-31 12:44:18 -04:00
Ben Radford c9a87ce7ca Refactor verifyPath to take StorePath instead of Path.
This way we avoid having to convert from Path to StorePath and vice versa in
the body of verifyPath.
2023-07-31 11:17:55 -04:00
John Ericson fe1fbdb5a1
Merge pull request #8724 from obsidiansystems/queryPartialDerivationOutputMap-evalStore
Give `queryPartialDerivationOutputMap` an `evalStore` parameter
2023-07-21 08:53:18 -04:00
John Ericson 6bc98c7fba Give queryPartialDerivationOutputMap an evalStore parameter
This makes it more useful. In general, the derivation will be in one
store, and the realisation info is in another.

This also helps us avoid duplication. See how `resolveDerivedPath` is
now simpler because it uses `queryPartialDerivationOutputMap`. In #8369
we get more flavors of derived path, and need more code to resolve them
all, and this problem only gets worse.

The fact that we need a new method to deal with the multiple dispatch is
unfortunate, but this generally relates to the fact that `Store` is a
sub-par interface, too bulky/unwieldy and conflating separate concerns.
Solving that is out of scope of this PR.

This is part of the RFC 92 work. See tracking issue #6316
2023-07-20 15:59:52 -04:00
John Ericson f62543fe1c Remove unneeded copy
It appeared in 8eb73a8724 (by me!) without
justification.
2023-07-20 15:42:06 -04:00
John Ericson 903700c5e1 Simplify ContentAddress
Whereas `ContentAddressWithReferences` is a sum type complex because different
varieties support different notions of reference, and
`ContentAddressMethod` is a nested enum to support that,
`ContentAddress` can be a simple pair of a method and hash.

`ContentAddress` does not need to be a sum type on the outside because
the choice of method doesn't effect what type of hashes we can use.

Co-Authored-By: Cale Gibbard <cgibbard@gmail.com>
2023-07-07 07:30:01 -04:00
Ben Radford 6ae35534b7
Support opening local store with database on read-only filesystem (#8356)
Previously it was not possible to open a local store when its database is on a read-only filesystem. Obviously a store on a read-only filesystem cannot be modified, but it would still be useful to be able to query it.

This change adds a new read-only setting to LocalStore. When set to true, Nix will skip operations that fail when the database is on a read-only filesystem (acquiring big-lock, schema migration, etc), and the store database will be opened in immutable mode.

Co-authored-by: Ben Radford <benradf@users.noreply.github.com>
Co-authored-by: cidkidnix <cidkidnix@protonmail.com>
Co-authored-by: Dylan Green <67574902+cidkidnix@users.noreply.github.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-20 11:34:09 +02:00
John Ericson 537e8719f2
Explain various .self = false,
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-04-17 09:15:11 -04:00
John Ericson fd21f9d76e Merge remote-tracking branch 'upstream/master' into path-info 2023-04-07 20:39:04 -04:00
John Ericson 4e9f32f993 Liberate checkDerivationOutputs from LocalStore
Make it instead a method on `Derivation` that can work with any store.
We will need this for a CLI command to create a derivation.
2023-04-07 08:34:58 -04:00
matthewcroughan 9207f94582 Add Store::isTrustedClient()
This function returns true or false depending on whether the Nix client
is trusted or not. Mostly relevant when speaking to a remote store with
a daemon.

We include this information in `nix ping store` and `nix doctor`

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
2023-04-06 19:59:57 -04:00
John Ericson 9383520b75 Move querySubstitutablePathInfos from LocalStore to Store
The code is not local-store-specific, so we should share it with all
stores. More uniform behavior is better, and a less store-specific
functionality is more maintainable.

This fixes a FIXME added in f73d911628 by @edolstra himself.
2023-04-02 20:32:01 -04:00
John Ericson c51d554c93 Use "raw pattern" for content address types
We weren't because this ancient PR predated it!

This is actually a new version of the pattern which addresses some
issues identified in #7479.
2023-03-30 17:12:49 -04:00
John Ericson aa99005004 Merge remote-tracking branch 'upstream/master' into path-info
Also improve content-address.hh API docs.
2023-03-30 16:28:53 -04:00
Eelco Dolstra 237587bc0a
Merge pull request #8084 from edolstra/store-docs
Auto-generate store documentation
2023-03-27 15:46:18 +02:00
Eelco Dolstra b79df9dedc Register LocalStore to ensure it's included in the manual 2023-03-23 15:23:13 +01:00
Eelco Dolstra 9eb53bbf17 Support per-store Markdown documentation 2023-03-21 14:03:40 +01:00
John Ericson 296831f641 Move enabled experimental feature to libutil struct
This is needed in subsequent commits to allow the settings and CLI args
infrastructure itself to read this setting.
2023-03-20 11:05:22 -04:00
Eelco Dolstra 29abc8e764 Remove FormatOrString and remaining uses of format() 2023-03-02 15:57:54 +01:00
John Ericson d381248ec0 No inheritance for TextInfo and FixedOutputInfo 2023-02-28 12:14:11 -05:00
John Ericson 85bb865d20 Revert "Remove some designated initializers"
This reverts commit ee9eb83a84.
2023-02-28 11:57:20 -05:00
John Ericson ee9eb83a84 Remove some designated initializers
With the switch to C++20, the rules became more strict, and we can no
longer initialize base classes. Make them comments instead.

(BTW
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2287r1.html
this offers some new syntax for this use-case. Hopefully this will be
adopted and we can eventually use it.)
2023-02-01 11:25:56 -05:00
John Ericson adb3608034 Merge branch 'small-storePath-cleanups' into path-info 2023-01-30 09:46:43 -05:00
John Ericson 4540e7b940 Don't add StorePathDescriptor for now
We don't need it yet, we can add it back later.
2023-01-23 12:58:27 -05:00
Théophane Hufschmitt a5919f4754 Move the default profiles to the user’s home
Rather than using `/nix/var/nix/{profiles,gcroots}/per-user/`, put the user
profiles and gcroots under `$XDG_DATA_DIR/nix/{profiles,gcroots}`.

This means that the daemon no longer needs to manage these paths itself
(they are fully handled client-side). In particular, it doesn’t have to
`chown` them anymore (removing one need for root).

This does change the layout of the gc-roots created by nix-env, and is
likely to break some stuff, so I’m not sure how to properly handle that.
2023-01-17 14:17:28 +01:00
John Ericson b3d91239ae Make ValidPathInfo have plain StorePathSet references like before
This change can wait for another PR.
2023-01-14 16:42:03 -05:00
John Ericson 46e942ff9e Do big rename to clean up code
- `PathReferences` -> `References`

- `PathReferences<StorePath>` -> `StoreReference`

- `references` -> `others`

- `hasSelfReference` -> `self`

And get rid of silly subclassing
2023-01-06 15:36:05 -05:00
John Ericson 6a168254ce Use named field initialization for references 2023-01-06 12:24:20 -05:00
John Ericson e9fc1e4fdb Merge remote-tracking branch 'upstream/master' into path-info 2023-01-06 10:35:20 -05:00
Eelco Dolstra 224b56f10e Move creation of the temp roots file into its own function
This also moves the file handle into its own Sync object so we're not
holding the _state while acquiring the file lock. There was no real
deadlock risk here since locking a newly created file cannot block,
but it's still a bit nicer.
2023-01-03 14:51:23 +01:00
Naïm Favier 81c3f99b36
Release shared lock before acquiring exclusive lock
In principle, this should avoid deadlocks where two instances of Nix are
holding a shared lock on big-lock and are both waiting to get an
exclusive lock.

However, it seems like `flock(2)` is supposed to do this automatically,
so it's not clear whether this is actually where the problem comes from.
2022-12-27 15:58:14 +01:00
Eelco Dolstra ec45f4b82e Fix indentation 2022-11-21 11:12:45 +01:00
Eelco Dolstra b95faccf03 Merge remote-tracking branch 'origin/master' into auto-uid-allocation 2022-11-03 17:43:40 +01:00
John Ericson a2a8cb10ac Dodge "trusted" vs "trustworthy" by being explicit
Hopefully this is best!
2022-09-22 14:37:52 -04:00
John Ericson 752f967c0f "valid signature" -> "trustworthy signature"
I just had a colleague get confused by the previous phrase for good
reason. "valid" sounds like an *objective* criterion, e.g. and *invalid
signature* would be one that would be trusted by no one, e.g. because it
misformatted or something.

What is actually going is that there might be a signature which is
perfectly valid to *someone else*, but not to the user, because they
don't trust the corresponding public key. This is a *subjective*
criterion, because it depends on the arbitrary and personal choice of
which public keys to trust.

I therefore think "trustworthy" is a better adjective to use. Whether
something is worthy of trust is clearly subjective, and then "trust"
within that word nicely evokes `trusted-public-keys` and friends.
2022-09-22 10:49:31 -04:00
squalus 1b595026e1 Improve durability of schema version file writes
- call close explicitly in writeFile to prevent the close exception
  from being ignored
- fsync after writing schema file to flush data to disk
- fsync schema file parent to flush metadata to disk

https://github.com/NixOS/nix/issues/7064
2022-09-19 20:13:30 -07:00