Eelco Dolstra
636455c471
Remove 'using namespace fetchers'
2020-10-06 11:16:32 +02:00
Eelco Dolstra
6691256e79
Factor out common showBytes()
2020-10-06 10:40:49 +02:00
Eelco Dolstra
d0bb544128
Add missing #pragma once
2020-10-06 10:40:07 +02:00
Eelco Dolstra
88a667e49e
Fix s3:// store
...
Fixes https://github.com/NixOS/nixos-org-configurations/issues/123 .
2020-10-05 17:53:30 +02:00
Eelco Dolstra
51c299213b
Merge pull request #3935 from obsidiansystems/binary-cache-addToStoreFromDump
...
Get rid of Hash::dummy from BinaryCacheStore
2020-10-05 14:41:28 +02:00
Eelco Dolstra
f3aba88737
Merge pull request #3895 from obsidiansystems/templated-daemon-protocol
...
More templated STL support for the daemon protocol
2020-10-05 14:40:27 +02:00
Fabian Möller
d5d196b0a1
Fix profile update in nix command
2020-10-02 12:10:31 +02:00
DavHau
f3280004e2
add more examples to --help of nix run
2020-10-01 11:34:13 +07:00
Eelco Dolstra
20a1e20d91
Style
2020-09-30 13:35:56 +02:00
Eelco Dolstra
274357eb6a
Simplify
2020-09-30 12:09:18 +02:00
Eelco Dolstra
002ce8449d
Merge branch 'access-tokens' of https://github.com/kquick/nix
2020-09-30 11:35:15 +02:00
John Ericson
69afaeace3
Merge remote-tracking branch 'upstream/master' into templated-daemon-protocol
2020-09-30 00:42:28 +00:00
John Ericson
b759701652
nix::worker_proto -> worker_proto
2020-09-30 00:41:18 +00:00
John Ericson
45a0ed82f0
Revert "Use template structs instead of phantoms"
...
This reverts commit 9ab07e99f5
.
2020-09-30 00:39:06 +00:00
Kevin Quick
5e7838512e
Remove github-access-token in favor of access-token.
2020-09-29 16:26:34 -07:00
Eelco Dolstra
76f000712e
Merge branch 'kwq/path-dir-query' of https://github.com/kquick/nix
2020-09-29 23:42:43 +02:00
Eelco Dolstra
64e9b3c83b
nix registry list: Show 'dir' attribute
...
Issue #4050 .
2020-09-29 23:33:16 +02:00
Kevin Quick
66c3959e8c
Merge branch 'master' into access-tokens
2020-09-29 08:32:06 -07:00
Eelco Dolstra
5999978a05
Make Headers an optional argument
2020-09-29 13:05:19 +02:00
Eelco Dolstra
de86abbf3f
Cleanup
2020-09-29 12:55:06 +02:00
Eelco Dolstra
cebd2fc35d
Merge branch 'github-api-token' of https://github.com/imalsogreg/nix
2020-09-29 12:17:00 +02:00
John Ericson
00135e13f4
Clarify comment a bit
2020-09-28 18:19:10 +00:00
Kevin Quick
887be7b6f2
Switch comment format from '// ...' to '/* ... */' for consistency.
2020-09-28 09:37:26 -07:00
Kevin Quick
128c98ab09
Clarification in the description of the FlakeInput.
2020-09-28 09:34:23 -07:00
Kevin Quick
5ae164b7cf
Update description of FlakeRef, incorporating suggestion.
2020-09-28 09:23:05 -07:00
Kevin Quick
bcb3da3b6b
Fix spelling error.
2020-09-28 08:58:14 -07:00
John Ericson
80e335bb58
Use drvPath2
and give it a better name
2020-09-28 15:43:56 +00:00
John Ericson
10202bbf29
Merge remote-tracking branch 'upstream/master' into ca-floating-upstream
2020-09-28 15:39:11 +00:00
John Ericson
6c31297d80
Update src/libstore/binary-cache-store.cc
...
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2020-09-28 11:32:58 -04:00
Eelco Dolstra
649c465873
Merge pull request #4064 from serokell/balsoft/fix-max-jobs
...
Fix max-jobs option
2020-09-28 10:42:02 +02:00
Eelco Dolstra
5df687c316
Merge pull request #4077 from Ma27/left-whitespace-log
...
libmain/progress-bar: don't trim whitespace on the left
2020-09-28 10:40:00 +02:00
Kevin Quick
5885b0cfd8
Miscellaneous spelling fixes in comments. ( #4071 )
2020-09-27 20:04:06 +00:00
Kevin Quick
bd5328814f
Add some internal documentation for flake support objects.
2020-09-26 14:32:58 -07:00
Maximilian Bosch
a76fb07314
libmain/progress-bar: don't trim whitespace on the left
...
When running `nix build -L` it can be fairly hard to read the output if
the build program intentionally renders whitespace on the left. A
typical example is `g++` displaying compilation errors.
With this patch, the whitespace on the left is retained to make the log
more readable:
```
foo> no configure script, doing nothing
foo> building
foo> foobar.cc: In function 'int main()':
foo> foobar.cc:5:5: error: 'wrong_func' was not declared in this scope
foo> 5 | wrong_func(1);
foo> | ^~~~~~~~~~
error: --- Error ------------------------------------------------------------------------------------- nix
error: --- Error --- nix-daemon
builder for '/nix/store/i1q76cw6cyh91raaqg5p5isd1l2x6rx2-foo-1.0.drv' failed with exit code 1
```
2020-09-26 17:38:11 +02:00
John Ericson
25fffdda86
Remove redundant nar hash and size setting
...
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2020-09-26 10:17:30 -04:00
John Ericson
1832436526
Fix up BinaryCacheStore::addToStore taking a path
2020-09-26 04:56:29 +00:00
John Ericson
5db83dd771
BinaryCacheStore::addTextToStore include CA field
2020-09-26 03:21:36 +00:00
John Ericson
c40c832f19
Merge remote-tracking branch 'obsidian/master' into binary-cache-addToStoreFromDump
2020-09-26 00:21:45 +00:00
Kevin Quick
cb186f1e75
Use "?dir=..." portion of "registry add" local path specification.
...
The registry targets generally follow a URL formatting schema with
support for a query parameter of "?dir=subpath" to specify a sub-path
location below the URL root.
Alternatively, an absolute path can be specified. This specification
mode accepts the query parameter but ignores/drops it. It would
probably be better to either (a) disallow the query parameter for the
path form, or (b) recognize the query parameter and add to the path.
This patch implements (b) for consistency, and to make it easier for
tooling that might switch between a remote git reference and a local
path reference.
See also issue #4050 .
2020-09-25 09:36:18 -07:00
Gregory Hale
faa5607f54
Merge remote-tracking branch 'origin/master' into github-api-token
2020-09-25 12:10:58 -04:00
John Ericson
cfe791a638
stdout_ -> cout
...
Better to get creative than just sprinkle arbitrary underscores.
2020-09-25 11:30:04 -04:00
Eelco Dolstra
7d81582488
Merge pull request #3953 from obsidiansystems/basic-derivation-goal-outputs
...
Deduplicate basic derivation goals too
2020-09-25 17:21:19 +02:00
Kevin Quick
5a35cc29bf
Re-add support for github-access-token, but mark as deprecated.
2020-09-25 08:09:56 -07:00
Kevin Quick
ef2a14be19
Fix reference to older name for access-tokens config value.
2020-09-25 08:08:27 -07:00
Eelco Dolstra
cbb9862cd9
Merge pull request #3626 from W95Psp/master
...
Make `functionArgs` primitive accept primops (fix #3624 )
2020-09-25 15:14:18 +02:00
Eelco Dolstra
7b2ae472ff
expectArg(): Respect the 'optional' flag
2020-09-25 10:27:40 +02:00
Kevin Quick
8fba2a8b54
Update to use access-tokens configuration for github/gitlab access.
...
This change provides support for using access tokens with other
instances of GitHub and GitLab beyond just github.com and
gitlab.com (especially company-specific or foundation-specific
instances).
This change also provides the ability to specify the type of access
token being used, where different types may have different handling,
based on the forge type.
2020-09-24 22:49:44 -07:00
Kevin Quick
c2f48cfcee
Complete conversion of "url" to "host" with associated variable renaming.
...
Completes the change begun in commit 56f1e0d
to consistently use the
"host" attribute for "github" and "gitlab" inputs instead of a "url"
attribute.
2020-09-24 22:46:03 -07:00
Kevin Quick
a439e9488d
Support StringMap configuration settings.
...
Allows Configuration values that are space-separated key=value pairs.
2020-09-24 22:42:59 -07:00
Paul Opiyo
4d863a9fcb
Remove redundant value checks
...
std::optional had redundant checks for whether it had a value.
An object is emplaced either way so it can be dereferenced
without repeating a value check
2020-09-24 18:32:03 -05:00
Alexander Bantyev
ed218e1d6c
Fix max-jobs option
...
After 0ed946aa61
, max-jobs setting (-j/--max-jobs)
stopped working.
The reason was that nrLocalBuilds (which compared to maxBuildJobs to figure
out whether the limit is reached or not) is not incremented yet when tryBuild
is started; So, the solution is to move the check to tryLocalBuild.
Closes https://github.com/nixos/nix/issues/3763
2020-09-25 00:07:42 +03:00
Kevin Quick
bd5f3dbe11
Fixes fall-through to report correct description of hash-file command.
2020-09-24 12:30:03 -07:00
Eelco Dolstra
8d9402f411
Merge pull request #4054 from edolstra/fix-4021
...
registerOutputs(): Don't call canonicalisePathMetaData() twice
2020-09-23 21:57:53 +02:00
Eelco Dolstra
4ce8a3ed45
Hopefully fix EPERM on macOS
2020-09-23 21:29:10 +02:00
Eelco Dolstra
9a24ece122
Fix exception
2020-09-23 20:21:08 +02:00
Eelco Dolstra
236d9ee7f7
lstat() cleanup
2020-09-23 19:17:28 +02:00
Eelco Dolstra
688bd4fb50
After rewriting a path, make it read-only
2020-09-23 19:10:16 +02:00
Dominique Martinet
2548347bba
libutil/archive: add preallocate-contents option
...
Make archive preallocation (fallocate) optional because some filesystems
like btrfs do not behave as expected with fallocate.
See #3550 .
2020-09-23 18:49:11 +02:00
Eelco Dolstra
cec9473871
DerivationGoal::registerOutputs(): Don't canonicalize twice
...
Fixes #4021 .
2020-09-23 18:21:22 +02:00
Eelco Dolstra
d4f8163d10
canonicalisePathMetaData_(): Change assertion to error message
2020-09-23 18:21:22 +02:00
Eelco Dolstra
e8f0b1e996
DerivationGoal::registerOutputs(): Fix bad format string
2020-09-23 18:21:22 +02:00
John Ericson
3f226f71c1
Return more info from BinaryCacheStore::addToStoreCommon
...
We don't need it yet, but we could/should in the future, and it's a
cost-free change since we already have the reference. I like it.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2020-09-23 14:40:41 +00:00
John Ericson
412b3a54fb
Clarify FIXME in BinaryCacheStore::addToStoreCommon
...
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2020-09-23 10:36:55 -04:00
Eelco Dolstra
8a2e10827f
Remove unused Flake::vOutputs field
2020-09-23 14:08:52 +02:00
Domen Kožar
dac8a1a5e1
Merge pull request #4052 from ujjwaljainnn/fix-typo
...
fixed typo
2020-09-23 09:58:30 +02:00
John Ericson
9fbc31a65b
Get rid of Hash::dummy from BinaryCacheStore
2020-09-23 04:56:04 +00:00
ujjwal
a2f5c921d4
fixed typo
2020-09-22 23:37:06 +05:30
John Ericson
993229cdaf
Deduplicate basic derivation goals too
...
See comments for security concerns.
Also optimize goal creation by not traversing map twice.
2020-09-22 17:13:59 +00:00
John Ericson
e9fc2031f0
Merge remote-tracking branch 'upstream/master' into templated-daemon-protocol
2020-09-22 14:18:31 +00:00
regnat
67cc356bb3
Merge remote-tracking branch 'origin/master' into non-ca-depending-on-ca
2020-09-22 16:11:25 +02:00
Eelco Dolstra
980edd1f3a
RemoteStore::addCAToStore(): Don't hold connection while calling queryPathInfo()
...
This leads to a deadlock if we're at the connection limit.
2020-09-22 15:28:20 +02:00
Eelco Dolstra
5b107f2c5f
Merge pull request #4038 from maljub01/master
...
Add a nix.conf option for allowing a symlinked store
2020-09-22 13:19:22 +02:00
Eelco Dolstra
35a0ac1838
Style fixes
2020-09-22 11:40:19 +02:00
Eelco Dolstra
92ac8df0ec
Merge branch 'add-ca-to-store' of https://github.com/hercules-ci/nix
2020-09-22 11:31:33 +02:00
Eelco Dolstra
e7f1109f06
Merge pull request #4040 from OmnipotentEntity/master
...
Fix compatibility with nlohmann-json 3.9.1
2020-09-22 11:09:25 +02:00
Eelco Dolstra
7dd8baafe1
Merge pull request #4041 from cole-h/enum-stringify
...
Serialize SandboxMode enum to string for JSON
2020-09-22 11:08:56 +02:00
regnat
c1e79f870c
Silence a compiler warning in serialise.hh
...
Explicitely cast to `uint64_t` in `readNum` to avoid a "comparison
between signed and unsigned" warning
2020-09-22 10:39:29 +02:00
John Ericson
3786a801c3
Merge remote-tracking branch 'upstream/master' into ca-floating-upstream
2020-09-22 04:15:55 +00:00
John Ericson
b92d3b2edd
Merge remote-tracking branch 'upstream/master' into templated-daemon-protocol
2020-09-22 00:45:55 +00:00
Michael Reilly
d860295e11
Bump nlohmann-json version to 3.9.1
2020-09-21 14:24:43 -04:00
Cole Helbling
ba37299a03
Serialize SandboxMode enum to string for JSON
...
Rather than showing an integer as the default, instead show the boolean
referenced in the description.
The nix.conf.5 manpage used to show "default: 0", which is unnecessarily
opaque and confusing (doesn't 0 mean false, even though the default is
true?); now it properly shows that the default is true.
2020-09-21 10:36:45 -07:00
Eelco Dolstra
ecc8672aa0
fmt.hh: Don't include boost/algorithm/string/replace.hpp
...
This cuts compilation time by ~49s.
Issue #4045 .
2020-09-21 19:07:55 +02:00
Eelco Dolstra
557d2427ee
Random header cleanup
2020-09-21 18:59:02 +02:00
Eelco Dolstra
0716adaa8b
abstractsettingtojson.hh -> abstract-setting-to-json.hh
2020-09-21 18:49:43 +02:00
Eelco Dolstra
340ca382c4
Don't include nlohmann/json.hpp in globals.hh
...
This reduces compilation time by 207s.
Issue #4045 .
2020-09-21 18:47:18 +02:00
Eelco Dolstra
d51ba43047
Move Callback into its own header
...
This gets rid of the inclusion of <future> in util.hh, cutting
compilation time by ~20s (CPU time).
Issue #4045 .
2020-09-21 18:42:21 +02:00
Marwan Aljubeh
f80ffeb8c9
Update the variable name accordingly
2020-09-21 17:29:08 +01:00
Eelco Dolstra
e8e1d420f3
Don't include <regex> in header files
...
This reduces compilation time by ~15 seconds (CPU time).
Issue #4045 .
2020-09-21 18:22:45 +02:00
Marwan Aljubeh
4e1a04733d
Use a better name for the config option
2020-09-21 16:32:22 +01:00
Eelco Dolstra
cbe0bb29f4
Merge pull request #4035 from Ma27/url-attr
...
libfetchers/github: allow `url` attribute
2020-09-21 17:26:36 +02:00
Maximilian Bosch
56f1e0df05
libfetchers/github: rename url
to host
2020-09-21 16:29:08 +02:00
Robert Hensing
ca30abb3fb
Document addCAToStore/addToStoreFromDump source drainage
...
Also checked that all usages satisfy the requirement and
removed dead code.
2020-09-21 07:55:47 +02:00
Robert Hensing
fa08db5c4c
wopAddToStore: return ValidPathInfo
...
A ValidPathInfo is created anyway. By returning it we can save a
roundtrip and we have a nicer interface.
2020-09-21 07:55:47 +02:00
Robert Hensing
7c68264085
wopAddToStore: add RepairFlag
2020-09-21 07:55:47 +02:00
Robert Hensing
fbf509c113
parseContentAddressMethodPrefix: use string_view
...
Co-authored-by: John Ericson <git@JohnEricson.me>
2020-09-21 07:55:47 +02:00
Robert Hensing
8279178b07
Move FramedSink next to FramedSource
2020-09-21 07:55:47 +02:00
Robert Hensing
ecc8088cb7
wopAddToStore: Throw to clarify unused refs
...
Co-authored-by: John Ericson <git@JohnEricson.me>
2020-09-21 07:55:47 +02:00
Robert Hensing
c602ebfb34
Refactor wopAddToStore to make wopAddTextToStore obsolete
2020-09-21 07:55:45 +02:00
Robert Hensing
e34fe47d0c
Overhaul wopAddToStore
2020-09-21 07:54:05 +02:00
Marwan Aljubeh
e40772cd35
Lint issue: replacing tabs with spaces
2020-09-18 17:18:45 +01:00
Marwan Aljubeh
c00e078343
Add a nix.conf option for allowing a symlinked store
2020-09-18 17:10:39 +01:00
Bryan Richter
5fe375a8f1
nix-prefetch-url: Add --executable flag
...
pkgs.fetchurl supports an executable argument, which is especially nice
when downloading a large executable. This patch adds the same option to
nix-prefetch-url.
I have tested this to work on the simple case of prefetching a little
executable:
1. nix-prefetch-url --executable https://my/little/script
2. Paste the hash into a pkgs.fetchurl-based package, script-pkg.nix
3. Delete the output from the store to avoid any misidentified artifacts
4. Realise the package script-pkg.nix
5. Run the executable
I repeated the above while using --name, as well.
I suspect --executable would have no meaningful effect if combined with
--unpack, but I have not tried it.
2020-09-18 19:09:45 +03:00
Maximilian Bosch
2bcf8cbe7a
libfetchers/github: allow url
attribute
...
Since 108debef6f
we allow a
`url`-attribute for the `github`-fetcher to fetch tarballs from
self-hosted `gitlab`/`github` instances.
However it's not used when defining e.g. a flake-input
foobar = {
type = "github";
url = "gitlab.myserver";
/* ... */
}
and breaks with an evaluation-error:
error: --- Error --------------------------------------nix
unsupported input attribute 'url'
(use '--show-trace' to show detailed location information)
This patch allows flake-inputs to be fetched from self-hosted instances
as well.
2020-09-18 14:10:45 +02:00
Eelco Dolstra
958bf57123
nix build: find() -> get()
...
find() returns an iterator so "!attr" doesn't work.
2020-09-18 13:10:42 +02:00
Robert Hensing
14b30b3f3d
Move FramedSource and FramedSink, extract withFramedSink
2020-09-17 20:21:04 +02:00
Robert Hensing
dfa547c6a8
Add ContentAddressMethod and parse/render it
2020-09-17 20:21:04 +02:00
Robert Hensing
29c82ccc77
Add Source.drainInto(Sink)
2020-09-17 20:21:04 +02:00
Robert Hensing
9ee3122ec7
Remove redundant import
2020-09-17 20:21:04 +02:00
Robert Hensing
3f93bc0d39
Typo
2020-09-17 20:21:04 +02:00
John Ericson
b7df353f27
Merge remote-tracking branch 'upstream/master' into ca-floating-upstream
2020-09-17 16:33:10 +00:00
Eelco Dolstra
649d3aaf24
Merge pull request #3829 from obsidiansystems/remove-storetype-delegate-regStore
...
Remove storetype delegate reg store -- contains #3736
2020-09-17 13:55:01 +02:00
Eelco Dolstra
b94a35ef40
Merge pull request #4027 from tweag/fix-gc-of-ca-derivations
...
Fix garbage collection of CA derivations
2020-09-17 13:46:26 +02:00
regnat
520895b1da
Fix garbage collection of CA derivations
...
Fix #4026
2020-09-17 13:36:58 +02:00
Eelco Dolstra
787469c7b6
Remove corepkgs/unpack-channel.nix
2020-09-17 09:41:02 +02:00
Eelco Dolstra
10d1865f5f
Remove corepkgs/derivation.nix
2020-09-17 09:41:02 +02:00
John Ericson
f60b380a7f
Merge remote-tracking branch 'upstream/master' into remove-storetype-delegate-regStore
2020-09-16 22:35:24 +00:00
John Ericson
2741fffa35
Ensure resolved CA derivations are written
...
so we can link outputs to deriver and thus properly cache.
2020-09-16 17:57:32 +00:00
John Ericson
7fdbb377ba
Start to fix floating CA + remote building
2020-09-16 17:50:48 +00:00
John Ericson
c5ccebae00
Merge remote-tracking branch 'upstream/master' into ca-floating-upstream
2020-09-16 17:50:40 +00:00
Greg Hale
a303c0b6dc
Fetch commits from github/gitlab using Auth header
...
`nix flake info` calls the github 'commits' API, which requires
authorization when the repository is private. Currently this request
fails with a 404.
This commit adds an authorization header when calling the 'commits' API.
It also changes the way that the 'tarball' API authenticates, moving the
user's token from a query parameter into the Authorization header.
The query parameter method is recently deprecated and will be disallowed
in November 2020. Using them today triggers a warning email.
2020-09-16 13:46:48 -04:00
Eelco Dolstra
5080d4e7b2
Merge branch 'document-store-options' of https://github.com/tweag/nix
2020-09-16 17:02:30 +02:00
Eelco Dolstra
0066ef6c59
Fix doc generation
2020-09-16 16:56:28 +02:00
Eelco Dolstra
2eacc1bc00
builtins.toFile: Fix indentation
2020-09-16 14:18:46 +02:00
Théophane Hufschmitt
77a0e2c5be
Remove useless exception copy
...
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2020-09-16 14:00:21 +02:00
regnat
d72927aa7a
Fix the s3 store
...
Add some necessary casts in the initialisation of the store's config
2020-09-16 13:53:09 +02:00
regnat
e0817cbcdc
Don't include nlohmann/json.hpp in config.hh
...
Instead make a separate header with the template implementation of
`BaseSetting<T>::toJSONObj` that can be included where needed
2020-09-16 13:53:09 +02:00
regnat
93c0e14a30
Include the full nlohmann/json header in config.hh
...
It is apparently required for using `toJSONObject()`, which we do inside
the header file (because it's in a template).
This was accidentally working when building Nix itself (presumably because
`config.hh` was always included after `nlohman/json.hpp`) but caused a
(pretty dirty) build failure in the perl bindings package.
2020-09-16 13:53:09 +02:00
regnat
fc2d31c423
Add (StoreConfig*)
casts to work around a GCC bug
...
Work around https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80431 that was
already there in the code but was accidentally removed in the last
commits
2020-09-16 13:53:09 +02:00
regnat
a1e82ba450
fixup! Add a default value for the settings
2020-09-16 13:53:09 +02:00
regnat
888f7afe9f
Fix build issues with gcc
2020-09-16 13:53:09 +02:00
regnat
634cb2a5ae
Add a markdown output to nix describe-stores
2020-09-16 13:53:09 +02:00
regnat
b73adacc1e
Add a name to the stores
...
So that it can be printed by `nix describe-stores`
2020-09-16 13:53:09 +02:00
regnat
f24f0888f9
Document the new store hierarchy
2020-09-16 13:53:09 +02:00
regnat
d65962db4d
Make uri schemes grammar more RFC-compliant
...
Allow `-` and `.` in the RFC schemes as stated by
[RFC3986](https://tools.ietf.org/html/rfc3986#section-3.1 ).
Practically, this is needed so that `ssh-ng` is a valid URI scheme
2020-09-16 13:53:09 +02:00
regnat
29a632386e
fixup! Make the store plugins more introspectable
2020-09-16 13:53:09 +02:00
regnat
1129913c4e
fixup! Correctly call all the parent contructors of the stores
2020-09-16 13:53:09 +02:00
regnat
7f103dcddd
Properly filter the stores according to their declared uriSchemes
...
When opening a store, only try the stores whose `uriSchemes()` include
the current one
2020-09-16 13:53:09 +02:00
regnat
5895184df4
Correctly call all the parent contructors of the stores
...
Using virtual inheritance means that only the default constructors of
the parent classes will be called, which isn't what we want
2020-09-16 13:53:09 +02:00
regnat
dae39f0a7a
Make nix describe-stores
functional
...
Using the `*Config` class hierarchy
2020-09-16 13:53:08 +02:00
regnat
22afa8fb4d
Separate store configs from the implems
...
Rework the `Store` hierarchy so that there's now one hierarchy for the
store configs and one for the implementations (where each implementation
extends the corresponding config). So a class hierarchy like
```
StoreConfig-------->Store
| |
v v
SubStoreConfig----->SubStore
| |
v v
SubSubStoreConfig-->SubSubStore
```
(with virtual inheritance to prevent DDD).
The advantage of this architecture is that we can now introspect the configuration of a store without having to instantiate the store itself
2020-09-16 13:53:08 +02:00
regnat
aa4eac3788
fixup! Separate the instantiation and initialisation of the stores
2020-09-16 13:53:08 +02:00
regnat
35042c9623
Add a default value for the settings
...
The default value is initialized when creating the setting and unchanged
after that
2020-09-16 13:53:08 +02:00
regnat
3c525d1590
Complete the toJSON
instance for Setting<T>
...
Don't let it just contain the value, but also the other fields of the
setting (description, aliases, etc..)
2020-09-16 13:53:08 +02:00
regnat
3b57181f8e
Separate the instantiation and initialisation of the stores
...
Add a new `init()` method to the `Store` class that is supposed to
handle all the effectful initialisation needed to set-up the store.
The constructor should remain side-effect free and just initialize the
c++ data structure.
The goal behind that is that we can create “dummy” instances of each
store to query static properties about it (the parameters it accepts for
example)
2020-09-16 13:53:08 +02:00
regnat
fa32560169
Fix the registration of stores
2020-09-16 13:53:08 +02:00
regnat
7d5bdf8b56
Make the store plugins more introspectable
...
Directly register the store classes rather than a function to build an
instance of them.
This gives the possibility to introspect static members of the class or
choose different ways of instantiating them.
2020-09-16 13:53:08 +02:00
John Ericson
3a5cdd737c
Rename Derivation::pathOpt
to Derivation::path
...
We no longer need the `*Opt` to disambiguate.
2020-09-15 15:21:39 +00:00
John Ericson
6387550d58
Get rid of confusing std::optional<bool>
for validity
2020-09-15 15:19:45 +00:00
John Ericson
c4bf219b55
Don't link deriver until after any delayed exception is thrown
...
Otherwise, we will associate fixed-output derivations with outputs that
they did indeed produce, but which had the wrong hash. That's no good.
2020-09-15 14:28:06 +00:00
John Ericson
3ba552b245
Merge remote-tracking branch 'upstream/master' into single-ca-drv-build
2020-09-15 14:17:06 +00:00
John Ericson
c08c9f08c7
Merge remote-tracking branch 'upstream/master' into remove-storetype-delegate-regStore
2020-09-15 14:08:35 +00:00
Eelco Dolstra
2a80170920
Merge pull request #4014 from tweag/fix-queryDrvOutputMaps-for-old-daemons
...
gracefully handle old daemon versions
2020-09-15 11:25:19 +02:00
regnat
057c6203b5
gracefully handle old daemon versions
...
Add a fallback path in `queryPartialDerivationOutputMap` for daemons
that don't support it.
Also upstreams a couple methods from `SSHStore` to `RemoteStore` as this
is needed to handle the fallback path.
2020-09-15 09:58:01 +02:00
regnat
250f8a4bba
Escape ${
in strings when printing Nix expressions
...
Otherwise the result of the printing can't be parsed back correctly by
Nix (because the unescaped `${` will be parsed as the begining of an
anti-quotation).
Fix #3989
2020-09-14 17:19:25 +02:00
Eelco Dolstra
a59e77d9e5
nix-daemon: Lower verbosity of restricted setting warning
...
Fixes #3992 .
2020-09-14 13:48:51 +02:00
Eelco Dolstra
35ba092019
Merge pull request #4005 from Infinisil/fix-autoArgs
...
Fix auto argument passing for more auto arguments than formals
2020-09-14 13:31:23 +02:00
aszlig
525b38eee8
Fix unspecified behaviour in readStorePathCAMap
...
When deploying a Hydra instance with current Nix master, most builds
would not run because of errors like this:
queue monitor: error: --- Error --- hydra-queue-runner
error: --- UsageError --- nix-daemon
not a content address because it is not in the form '<prefix>:<rest>': /nix/store/...-somedrv
The last error message is from parseContentAddress, which expects a
colon-separated string, however what we got here is a store path.
Looking at the worker protocol, the following message sent to the Nix
daemon caused the error above:
0x1E -> wopQuerySubstitutablePathInfos
0x01 -> Number of paths
0x16 -> Length of string
"/nix/store/...-somedrv"
0x00 -> Length of string
""
Looking at writeStorePathCAMap, the store path is indeed the first field
that's transmitted. However, readStorePathCAMap expects it to be the
*second* field *on my machine*, since expression evaluation order is a
classic form of unspecified behaviour[1] in C++.
This has been introduced in https://github.com/NixOS/nix/pull/3689 ,
specifically in commit 66a62b3189
.
[1]: https://en.wikipedia.org/wiki/Unspecified_behavior#Order_of_evaluation_of_subexpressions
Signed-off-by: aszlig <aszlig@nix.build>
2020-09-13 02:40:51 +02:00
Silvan Mosberger
8dbd57a6a5
Fix auto argument passing for more auto arguments than formals
...
The change in 626200713b
didn't account
for when the number of auto arguments is bigger than the number of
formal arguments. This causes the following:
$ nix-instantiate --eval -E '{ ... }@args: args.foo' --argstr foo foo
nix-instantiate: src/libexpr/attr-set.hh:55: void nix::Bindings::push_back(const nix::Attr&): Assertion `size_ < capacity_' failed.
Aborted (core dumped)
2020-09-11 20:41:51 +02:00
Gabriel Gonzalez
ee5906243a
Add nix-shell
support for preserving PS1
...
Fixes https://github.com/NixOS/nix/issues/1268
`nix-shell` will now preserve `PS1` if the `NIX_SHELL_PRESERVE_PROMPT`
environment variable is set.
2020-09-04 20:05:43 -07:00
John Ericson
98dfd7531d
Fix querying outputs for CA derivations some more
...
If we resolve using the known path of a derivation whose output we
didn't have, we previously blew up. Now we just fail gracefully,
returning the map of all outputs unknown.
2020-09-04 18:33:58 +00:00
John Ericson
075d399e3f
Merge remote-tracking branch 'obsidian/single-ca-drv-build' into ca-floating-upstream
2020-09-04 16:04:35 +00:00
John Ericson
5aed6f9b25
Document mkOutputString
2020-09-04 15:58:42 +00:00
John Ericson
e9fad3006b
Fix some of the issues raised by @edolstra
...
- More and better comments
- The easier renames
2020-09-04 15:18:25 +00:00
John Ericson
e86dd59dcc
Apply suggestions from code review
...
Thanks!
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2020-09-04 10:48:50 -04:00
John Ericson
c9f1ed912c
Don't chmod symlink before moving outputs around
...
Co-authored-by: Théophane Hufschmitt <regnat@users.noreply.github.com>
2020-09-04 14:41:53 +00:00
John Ericson
25f7ff16fa
Merge remote-tracking branch 'upstream/master' into fix-and-ci-static-builds
2020-09-04 02:40:36 +00:00
John Ericson
aad4abcc9c
Fix floating CA tests
...
We will sometimes try to query the outputs of derivations we can't
resolve. That's fine; it just means we don't know what those outputs are
yet.
2020-09-04 01:17:38 +00:00
John Ericson
975a47f7fe
Merge remote-tracking branch 'obsidian/single-ca-drv-build' into ca-floating-upstream
2020-09-03 22:09:04 +00:00
John Ericson
e7d93e7ece
Merge remote-tracking branch 'upstream/master' into single-ca-drv-build
2020-09-03 15:43:17 +00:00
Eelco Dolstra
8a945d6ddb
Remove gc-check-reachability
2020-09-03 11:30:15 +02:00
Eelco Dolstra
82b77a7726
addPermRoot(): Remove indirect flag
2020-09-03 11:29:34 +02:00
Eelco Dolstra
00d25e8457
Remove the --indirect flag
...
All GC roots are now indirect.
2020-09-03 11:22:00 +02:00
Eelco Dolstra
b07167be5a
createGeneration(): Always create an indirect root
...
This means profiles outside of /nix/var/nix/profiles don't get
garbage-collected. It also means we don't need to scan
/nix/var/nix/profiles for GC roots anymore, except for compatibility
with previously existing generations.
2020-09-03 11:13:53 +02:00
Eelco Dolstra
b74f5cdd23
createGeneration(): Take a StorePath
2020-09-03 11:06:56 +02:00
Eelco Dolstra
94a043ff3b
EvalCache: Fix caching of strings
...
This was broken in 50f13b06fb
. Once
again it turns out that putting a bool in a std::variant is a bad
idea, since pointers get silently cast to them...
2020-09-02 14:16:44 +02:00
Domen Kožar
dd4b56c87f
Allow HTTP binary cache to request absolute uris
2020-09-01 21:35:48 +02:00
John Ericson
ef278d00f9
Merge remote-tracking branch 'upstream/master' into single-ca-drv-build
2020-09-01 18:01:48 +00:00
Eelco Dolstra
6d7f7efb89
github: Use access token when calling .../commits API
2020-09-01 15:29:04 +02:00
Eelco Dolstra
84f5cabbea
Merge remote-tracking branch 'origin/master' into markdown
2020-08-31 14:24:26 +02:00
John Ericson
02e0001fc0
Merge branch 'ca-floating-upstream' of github.com:obsidiansystems/nix into ca-floating-upstream
2020-08-28 21:23:09 +00:00
John Ericson
421ed527c7
Update src/libstore/build.cc
...
Thanks for catching, @regnat.
2020-08-28 17:22:57 -04:00
John Ericson
48e8828050
Merge remote-tracking branch 'obsidian/single-ca-drv-build' into ca-floating-upstream
2020-08-28 20:38:25 +00:00
John Ericson
8017fe7487
Merge remote-tracking branch 'upstream/master' into single-ca-drv-build
2020-08-28 19:59:14 +00:00
Eelco Dolstra
f15651303f
nix develop: Add convenience flags for running specific phases
...
For example, for building the Nix flake, you would do:
$ nix develop --configure
$ nix develop --install
$ nix develop --installcheck
2020-08-28 19:24:29 +02:00
Eelco Dolstra
3156560d41
nix develop: Set output paths to writable locations
...
Currently, they're set to $(pwd)/outputs/$outputName. This allows
commands like 'make install' to work.
2020-08-28 18:16:03 +02:00
Eelco Dolstra
691a1bd717
Merge branch 'minimal-logger' of https://github.com/Ma27/nix
2020-08-28 10:50:52 +02:00
Griffin Smith
626200713b
Pass all args when auto-calling a function with an ellipsis
...
The command line options --arg and --argstr that are used by a bunch of
CLI commands to pass arguments to top-level functions in files go
through the same code-path as auto-calling top-level functions with
their default arguments - this, however, was only passing the arguments
that were *explicitly* mentioned in the formals of the function - in the
case of an as-pattern with an ellipsis (eg args @ { ... }) extra passed
arguments would get omitted. This fixes that to instead pass *all*
specified auto args in the case that our function has an ellipsis.
Fixes #598
2020-08-27 12:38:25 -04:00
Eelco Dolstra
eb75282b8d
Merge pull request #3434 from Ericson2314/derivation-header-include-order
...
Revise division of labor in deserialization of derivations
2020-08-27 16:39:28 +02:00
Eelco Dolstra
e915fd6d2a
Typo
2020-08-27 14:51:50 +02:00
Eelco Dolstra
a0f19d9f3a
RemoteStore::addToStore(): Fix race between stderrThread and NAR writer
...
As pointed out by @B4dM4n, the call to to.flush() on stderrThread is
unsafe because the NAR writer thread is also writing to 'to'.
Fixes #3943 .
2020-08-27 14:50:51 +02:00
Eelco Dolstra
c3efef9275
Remove obsolete comment
2020-08-26 09:28:10 +02:00
Eelco Dolstra
4bf5faf416
Merge remote-tracking branch 'origin/master' into markdown
2020-08-25 19:47:34 +02:00
Eelco Dolstra
7a02865b94
Move import docs
2020-08-25 14:06:01 +02:00
Eelco Dolstra
f53b5f1058
Add getDoc() function
2020-08-25 13:31:11 +02:00
Eelco Dolstra
2a2121d264
Use RegisterPrimOp for some undocumented primops
2020-08-25 11:25:01 +02:00
Eelco Dolstra
b8416779e3
Document some primops
2020-08-25 11:16:45 +02:00
Eelco Dolstra
24b1c2c66b
Fix tests
2020-08-25 10:51:14 +02:00
Eelco Dolstra
b42789f013
Fix clang build
2020-08-24 21:13:39 +02:00
John Ericson
8eb73a8724
CA derivations that depend on other CA derivations
...
Co-authored-by: Théophane Hufschmitt <regnat@users.noreply.github.com>
2020-08-24 19:01:15 +00:00
John Ericson
e0b0e18905
Add constructor for BasicDerivation -> Derivation
2020-08-24 19:01:07 +00:00
John Ericson
59979e7053
Fix bad debug format string
2020-08-24 19:01:07 +00:00
Eelco Dolstra
d9a8619762
Don't barf if corepkgs is in the store but not a valid path
...
This can happen when using a dummy store (or indeed any non-local store).
2020-08-24 19:15:17 +02:00
Eelco Dolstra
6a67e57019
Add DummyStore (dummy://)
...
DummyStore does not allow building or adding paths. This is useful for
evaluation tests when you don't want to initialize a "proper" store.
2020-08-24 18:54:16 +02:00
Eelco Dolstra
d0690bc311
nix repl ':doc': Render using lowdown
2020-08-24 18:10:33 +02:00
Eelco Dolstra
0f314f3c25
Generate builtins section of the manual
2020-08-24 14:49:30 +02:00
Eelco Dolstra
a990f063ff
Move primop docs inline
...
This makes them available to 'nix repl'.
2020-08-24 14:31:10 +02:00
Eelco Dolstra
33b1679d75
Allow primops to have Markdown documentation
2020-08-24 13:16:02 +02:00
Eelco Dolstra
88d5c9ec58
Fix tests
2020-08-24 10:37:10 +02:00
Eelco Dolstra
2530cc6827
Merge pull request #3950 from obsidiansystems/tabs-spaces
...
tabs -> spaces
2020-08-24 10:20:28 +02:00
John Ericson
b0b59fd05a
Merge remote-tracking branch 'obsidian/write-derivation-borrow' into HEAD
2020-08-23 15:11:10 +00:00
John Ericson
35e6288be1
writeDerivation
just needs a plain store reference
2020-08-23 15:01:11 +00:00
John Ericson
3a7b330b64
"Downstream placeholders" should not be store paths
...
Insead they should be opaque `/<hash>` like the placeholders we already
have.
2020-08-21 19:35:35 +00:00
John Ericson
422affe102
tabs -> spaces
...
Sorry I let the tab sneak in there in the first place.
2020-08-21 19:19:24 +00:00
John Ericson
c265e0ea82
Merge remote-tracking branch 'upstream/master' into templated-daemon-protocol
2020-08-20 20:24:22 +00:00
John Ericson
27a3f82c0b
Merge remote-tracking branch 'upstream/master' into single-ca-drv-build
2020-08-20 18:28:17 +00:00
John Ericson
45a2f1baab
Rename drv output querying functions, like master
...
- `queryDerivationOutputMapAssumeTotal` -> `queryPartialDerivationOutputMap`
- `queryDerivationOutputMapAssumeTotal` -> `queryDerivationOutputMap
2020-08-20 18:14:12 +00:00
Eelco Dolstra
4d77513d97
Merge pull request #3859 from obsidiansystems/drv-outputs-map-allow-missing
...
`queryDerivationOutputMap` no longer assumes all outputs have a mapping
2020-08-20 16:49:23 +02:00
John Ericson
9a9d834dc7
Rename drv output querying functions
...
- `queryDerivationOutputMapAssumeTotal` -> `queryPartialDerivationOutputMap`
- `queryDerivationOutputMapAssumeTotal` -> `queryDerivationOutputMap`
2020-08-20 14:12:51 +00:00
Eelco Dolstra
25ecfffdc3
Remove PrimOp constructor
2020-08-20 12:34:04 +02:00
Eelco Dolstra
dc2f278c95
Allow 'nix' subcommands to provide docs in Markdown format
2020-08-20 12:21:46 +02:00
Eelco Dolstra
3c4f8c9175
List deprecated option aliases in the docs
2020-08-20 11:13:17 +02:00
Eelco Dolstra
acb99f03f9
Config: Use nlohmann/json
2020-08-20 11:02:16 +02:00
John Ericson
3df78858f2
Fix max fd calc and add test
2020-08-20 05:08:50 +00:00
Ryan Mulligan
f36793c7b9
fix spelling
2020-08-19 20:31:01 -07:00
John Ericson
a83694c7a1
Use RemoteStore
to open connection for proxying daemon
...
Removes duplicate websocket opening code, and also means we should be
able to to ssh-ssh-... daemon relays, not just uds-uds-... ones.
2020-08-19 19:34:47 +00:00
Eelco Dolstra
c8fa39324a
Generate the nix.conf docs from the source code
...
This means we don't have two (divergent) sets of option descriptions
anymore.
2020-08-19 18:28:04 +02:00
John Ericson
d5af5763cf
Merge branch 'master' of github.com:NixOS/nix into remove-storetype-delegate-regStore
2020-08-19 15:54:50 +00:00
Eelco Dolstra
34b22e0123
Change option descriptions to Markdown
2020-08-19 14:21:27 +02:00
John Ericson
be0d429b95
Merge branch 'master' of github.com:NixOS/nix into templated-daemon-protocol
2020-08-19 03:17:41 +00:00
John Ericson
950ddfdb82
Merge remote-tracking branch 'upstream/master' into derivation-header-include-order
2020-08-18 14:36:44 +00:00
Eelco Dolstra
1c8b550e34
Merge pull request #3917 from obsidiansystems/output-env-var-unconditional
...
Simplify code as output env vars are unconditional
2020-08-18 16:21:17 +02:00
Eelco Dolstra
dfeb76dbf9
Merge pull request #3930 from obsidiansystems/legacy-ssh-build-paths
...
Define `LegacySSHStore::buildPaths` using `cmdBuildPaths`
2020-08-18 16:07:40 +02:00
Eelco Dolstra
069340179e
Improve nix.1 manpage generator
2020-08-18 15:15:35 +02:00
Eelco Dolstra
6f19c776db
Start generation of the nix.1 manpage
2020-08-17 19:33:18 +02:00
Eelco Dolstra
a72a20d68f
Add 'nix dump-args' to dump all commands/flags for manpage generation
2020-08-17 17:44:52 +02:00
Eelco Dolstra
7cdc739ece
Merge remote-tracking branch 'origin/master' into markdown
2020-08-17 13:43:39 +02:00
Eelco Dolstra
e849b19872
Merge pull request #3932 from chkno/no-show-signature
...
Don't try to parse signature check as commit timestamp
2020-08-17 11:00:24 +02:00
Eelco Dolstra
847a5392f4
Merge branch 'ca-no-need-trust' of https://github.com/obsidiansystems/nix
2020-08-17 10:57:02 +02:00
Eelco Dolstra
3c619f6290
Merge branch 'test-RemoteStore-buildDerivation' of https://github.com/obsidiansystems/nix
2020-08-17 10:53:18 +02:00
John Ericson
6f7ac5e865
Remove extra closing paren
2020-08-14 21:59:31 +00:00
John Ericson
f899a7c6d7
Work around clang bug
2020-08-14 18:51:31 +00:00
John Ericson
3c8b5b6219
Merge remote-tracking branch 'upstream/master' into single-ca-drv-build
2020-08-14 17:00:13 +00:00
John Ericson
4b571ea321
Update src/libstore/daemon.cc
...
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2020-08-14 11:52:37 -04:00
Eelco Dolstra
13e49be660
Merge pull request #3875 from obsidiansystems/new-interface-for-path-pathOpt
...
Offer a safer interface for path and pathOpt
2020-08-14 17:19:19 +02:00
Eelco Dolstra
7714d9a943
Merge pull request #3924 from obsidiansystems/features-per-store
...
Make `system-features` a store setting
2020-08-14 17:13:07 +02:00
Eelco Dolstra
9b9d5297ba
Merge pull request #3909 from matthewbauer/readd-hashed-mirrors
...
Add hashed-mirrors back
2020-08-14 17:05:47 +02:00
Eelco Dolstra
d81f13f7cb
Merge pull request #3899 from obsidiansystems/make-narHash-not-optional
...
Make narHash in ValidPathInfo not optional
2020-08-14 17:00:18 +02:00
Chuck
ed026f7206
Don't try to parse signature check as commit timestamp
...
When the log.showSignature git setting is enabled, the output of
"git log" contains signature verification information in addition to the
timestamp GitInputScheme::fetch wants:
$ git log -1 --format=%ct
gpg: Signature made Sat 07 Sep 2019 02:02:03 PM PDT
gpg: using RSA key 0123456789ABCDEF0123456789ABCDEF01234567
gpg: issuer "user@example.com"
gpg: Good signature from "User <user@example.com>" [ultimate] 1567890123
1567890123
For folks that had log.showSignature set, this caused all nix operations
on flakes to fail:
$ nix build
error: stoull
2020-08-13 17:44:42 -07:00
John Ericson
e1308b1211
Define LegacySSHStore::buildPaths
using cmdBuildPaths
...
Evidentally this was never implemented because Nix switched to using
`buildDerivation` exclusively before `build-remote.pl` was rewritten.
The `nix-copy-ssh` test (already) tests this.
2020-08-13 21:27:55 +00:00
John Ericson
5ccd94501d
Allow trustless building of CA derivations
...
Include a long comment explaining the policy. Perhaps this can be moved
to the manual at some point in the future.
Also bump the daemon protocol minor version, so clients can tell whether
`wopBuildDerivation` supports trustless CA derivation building. I hope
to take advantage of this in a follow-up PR to support trustless remote
building with the minimal sending of derivation closures.
2020-08-13 18:15:57 +00:00