Commit graph

9773 commits

Author SHA1 Message Date
John Ericson b2066c3574
Merge pull request #9521 from fricklerhandwerk/conf-reword-max-jobs
reword description of the `max-jobs` setting
2024-01-22 08:29:12 -05:00
Théophane Hufschmitt c8d33bca8d
Merge pull request #9822 from obsidiansystems/algo-vs-hash-algo
Start standardizing hash algo flags
2024-01-22 11:08:24 +01:00
Théophane Hufschmitt f51409cf98
Merge pull request #9805 from yshui/prefetch-unpack
Add --unpack to nix store prefetch-file
2024-01-22 10:51:50 +01:00
John Ericson 202c5e2afc Start standardizing hash algo flags
Do this if we want to do `--hash-algo` everywhere, and not `--algo` for
hash commands.

The new `nix hash convert` is updated. Deprecated new CLI commands are
left as-is (`nix hash path` needs to be redone and is also left as-is).
2024-01-20 17:29:35 -05:00
John Ericson a93b204c27
Merge pull request #9348 from obsidiansystems/json-formats
Document JSON formats
2024-01-20 17:22:39 -05:00
John Ericson edf3ecc497 Document JSON formats
Good to document these formats separately from commands that happen to
use them.

Eventually I would like this and `builtins.derivation` to refer to a
store section on derivations that is authoritative, but that doesn't yet
exist, and will take some time to make. So I think we're just best off
merging this now as is.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-01-20 17:03:47 -05:00
John Ericson 356352c370 Add missing --hash-algo flag to nix store add 2024-01-19 23:11:18 -05:00
Robert Hensing 382fa51ff0
Merge pull request #9794 from hercules-ci/queryPathInfoFromClientCache
refactor: Extract `Store::queryPathInfoFromClientCache`
2024-01-19 17:33:52 +01:00
Robert Hensing d19627e8b4 refactor: Remove throw from queryPathInfoFromClientCache
Return a value instead of throwing.

Rather than the more trivial refactor of wrapping the return value in
another std::optional, we retain the meaning of the outer optional:
"we know at least something."

So we have changed:
return nullopt    -> return nullopt
throw InvalidPath -> return make_optional(nullptr)
return vpi        -> return make_optional(vpi)
2024-01-19 17:01:19 +01:00
Robert Hensing 8983ee8b2e refactor: Un-callback transform Store::queryPathInfoFromClientCache
This part of the code was not necessarily callback based.
Removing CPS is always nice; particularly if there's no loss of
functionality, like here.
2024-01-19 17:01:19 +01:00
Yuxuan Shui 75a6e6dd0e Add --unpack to nix store prefetch-file 2024-01-19 15:53:45 +00:00
Théophane Hufschmitt 10165c7593
Merge pull request #9811 from DavHau/fix-git-auth
fetchTree/fetchGit: re-enable shallow fetching
2024-01-19 16:14:41 +01:00
John Ericson 57d6284eb7
Merge pull request #9812 from NixOS/remove-n-shorthand-for---mode
Remove a nonsensical shorthand flag in `nix store add`
2024-01-19 08:34:43 -05:00
DavHau bc00fa4647 fetchTree/fetchGit: re-enable shallow fetching
Add several tests for git fetching:
- shallow-cache-separation: can fetch the same repo shallowly and non-shallowly
- shallow-ignore-ref: ensure that ref gets ignored when shallow=true is set
- ssh-shallow: can fetch a git repo via ssh using shallow=1
2024-01-19 20:30:47 +07:00
Robert Hensing d762caff46
Merge pull request #9806 from hercules-ci/fix-git-auth
Fix git auth
2024-01-19 13:47:52 +01:00
Théophane Hufschmitt 28d7db249a
Remove a nonsensical shorthand flag in nix store add
`-n` was an alias for `--mode`, but that seems to just be a copy-paste error as it doesn't make sense.
`--mode` probably doesn't need a shorthand flag at all, so remove it.

Noticed in https://github.com/NixOS/nix/pull/9809#issuecomment-1899890555
2024-01-19 10:10:00 +01:00
Robert Hensing 8d422c2fef Revert libgit2 fetching
libgit2 is not capable of using git-credentials helpers yet.
This prevents private repositories from being used.

Based on code that was replaced in https://github.com/NixOS/nix/pull/9240
(Introduce libgit2); hence:

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-01-18 22:29:26 +01:00
Robert Hensing 76a50b3a69 doc: GitRepoImpl::path 2024-01-18 22:29:26 +01:00
Eelco Dolstra dca0a80240 copyStorePath(): Bail out early if the store path already exists
In rare cases (e.g. when using allowSubstitutes = false), it's
possible that we simultaneously have a DerivationGoal *and* a
SubstitutionGoal building the same path. So if a DerivationGoal
already built the path while the SubstitutionGoal was waiting for a
download slot, it saves us a superfluous download to exit early.
2024-01-18 17:19:15 +01:00
Eelco Dolstra a18d8d688a LocalStore::addToStore(): Ignore exceptions from parseDump()
In the "discard" case (i.e. when the store path already exists
locally), when we call parseDump() from a Finally and it throws an
exception (e.g. if the download of the NAR fails), Nix crashes:

   terminate called after throwing an instance of 'nix::SubstituteGone'
     what():  error: file 'nar/06br3254rx4gz4cvjzxlv028jrx80zg5i4jr62vjmn416dqihgr7.nar.xz' does not exist in binary cache 'http://localhost'
   Aborted (core dumped)
2024-01-18 17:19:15 +01:00
Eelco Dolstra ab786e22f1 Show what goal is waiting for a build slot 2024-01-18 17:19:15 +01:00
Théophane Hufschmitt e652322a61
Merge pull request #9799 from obsidiansystems/push-add-to-store-from-dump-unsupported-down
Push `addToStoreFromDump` `unsupported(...)` down `Store` class hierarchy
2024-01-18 16:54:44 +01:00
Eelco Dolstra 32706b14a7
Merge pull request #9798 from edolstra/remote-store-eof
Print a more helpful message if the daemon crashes
2024-01-18 15:47:59 +01:00
Eelco Dolstra 1fe8f54bd3 Use BackedStringView 2024-01-18 15:27:57 +01:00
Théophane Hufschmitt 28674247ec
Merge pull request #9787 from obsidiansystems/bind-proc-syserror
`bind`: give same treatment as `connect` in #8544, dedup
2024-01-18 09:34:15 +01:00
John Ericson 574db83504 Push addToStoreFromDump unsupported(...) down Store class hierarchy
Instead of having it be the default method in `Store` itself, have it be
the implementation in `DummyStore` and `LegacySSHStore`. Then just the
implementations which fail to provide the method pay the "penalty" of
dealing with the icky `unimplemented` function for non-compliance.

Picks up where #8217. Getting close to no `unsupported` in the `Store`
interface itself!

More progress on issue #5729.
2024-01-17 23:46:03 -05:00
John Ericson 3016e67c21 bind: give same treatment as connect in #8544, dedup
It is good to propagate the underlying error so whether or not we use a
process to deal with path length issues is not observable.

Also, as these wrapper functions got more and more complex, the code
duplication got worse and worse. The new `bindConnectProcHelper`
function deduplicates them.
2024-01-17 23:35:08 -05:00
Eelco Dolstra a3cf27ca47 Print a more helpful message if the daemon crashes
Instead of

   error: unexpected end-of-file

you now get

   error: Nix daemon disconnected unexpectedly (maybe it crashed?)
2024-01-17 22:54:02 +01:00
Eelco Dolstra 39ab50f9ee Store::buildPaths(): Fix display of store paths
This was broken in 7ac39ff05c.
2024-01-17 21:41:37 +01:00
Robert Hensing d0a284284b refactor: Extract simply, awkwardly Store::queryPathInfoFromClientCache
This is useful for determining quickly which substituters to query.

An alternative would be for users to invoke the narinfo cache db directly,
so why do we need this change?

 - It is easier to use. I believe Nix itself should also use it.

 - This way, the narinfo cache db remains an implementation detail.

 - Callers get to use the in-memory cache as well.
2024-01-17 17:54:19 +01:00
Robert Hensing 2a3c5e6b8b
Merge pull request #9480 from NixOS/libfetchers-git-exportIgnore
libfetchers/git: Support export-ignore
2024-01-16 23:03:46 +01:00
Robert Hensing 01271f2996
Merge pull request #9786 from hercules-ci/package
DerivationInfo -> PackageInfo
2024-01-16 22:43:14 +01:00
Jacek Galowicz 8ae3aeec94 Don't use std::make_unique right before release 2024-01-16 17:00:46 +00:00
Jacek Galowicz 9d9f42cc38 Remove C-style casts 2024-01-16 17:00:46 +00:00
Jacek Galowicz d11d7849f7 Use ChildWrapperFunction type and make casts more explicit 2024-01-16 17:00:46 +00:00
Jacek Galowicz c924147c9d Drop parentheses from thunks 2024-01-16 17:00:46 +00:00
Jacek Galowicz 1885d579db Improve String Handling 2024-01-16 17:00:46 +00:00
Jacek Galowicz 0bc66e529f Use npos member variables instead of full type 2024-01-16 15:17:57 +00:00
Robert Hensing ea6aa5ffd8 Package{,Info}: comments 2024-01-16 15:44:02 +01:00
Robert Hensing 65255edc9b DerivationInfo -> PackageInfo
This does not yet resolve the coupling between packages and
derivations, but it makes the code more consistent with the
terminology, and it accentuates places where the coupling is
obvious, such as

         auto drvPath = packageInfo.queryDrvPath();
         if (!drvPath)
             throw Error("'%s' is not a derivation", what());

... which isn't wrong, and in my opinion, doesn't even look
wrong, because it just reflects the current logic.
However, I do like that we can now start to see in the code that
this coupling is perhaps a bit arbitrary.
After this rename, we can bring the DerivingPath concept into type
and start to lift this limitation.
2024-01-16 15:28:03 +01:00
Eelco Dolstra d005bade7f connect(): Propagate errno from the child process
This is necessary on macOS since addTempRoot() relies on errno.
2024-01-16 15:26:17 +01:00
Eelco Dolstra 0b1d93d2ba Sleep a bit between attempts to connect to the root server 2024-01-16 15:26:17 +01:00
Eelco Dolstra 302625e83b Merge remote-tracking branch 'origin/master' into handle-missing-gc-socket 2024-01-16 13:18:58 +01:00
Robert Hensing b2deff1947
Merge pull request #9747 from awakesecurity/mz/fix-quadratic-splitString
Fix performance of builtins.substring for empty substrings
2024-01-16 12:18:59 +01:00
John Ericson beed00c04e absPath: just take a std::string_view
1. Slightly more efficient

2. Easier to call

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2024-01-15 08:21:54 -05:00
Robert Hensing 268c49264a
Merge pull request #9763 from L-as/avoid-unnecessary-copy
Avoid unnecessary copy of goal log
2024-01-15 08:32:03 +01:00
Las Safin f61d951909
Avoid unnecessary copy of goal log
The data was (accidentally?) copied into a std::string,
even though the string is immediately converted into a std::string_view.
The code has been changed to construct a std::string_view directly,
such that one copy less happens.
2024-01-13 19:32:37 +00:00
John Ericson e739a5002d Avoid Windows macros in the parser and lexer
`FLOAT`, `INT`, and `IN` are identifers taken by macros.

The name `IN_KW` is chosen to match `OR_KW`, which is presumably named
that way for the same reason of dodging macros.
2024-01-12 19:51:36 -05:00
Robert Hensing 15f7bdaf27 CanonPath: Add rel_c_str()
Defensively because isRoot() is also defensive.
2024-01-12 22:55:37 +01:00
John Ericson c58da62a06
Merge pull request #9737 from obsidiansystems/sys-error-split
Separate `SystemError` from `SysError`
2024-01-12 12:41:36 -05:00