lix-project/lix
20
58
Fork 31
Code Issues 352 Code Review (Gerrit) Projects 3 Releases Wiki Activity
9463 commits 29 branches 6 tags 89 MiB
Commit graph

5932 commits

Author SHA1 Message Date
John Ericson c40c832f19 Merge remote-tracking branch 'obsidian/master' into binary-cache-addToStoreFromDump 2020-09-26 00:21:45 +00:00
Kevin Quick cb186f1e75
Use "?dir=..." portion of "registry add" local path specification. 
The registry targets generally follow a URL formatting schema with
support for a query parameter of "?dir=subpath" to specify a sub-path
location below the URL root.

Alternatively, an absolute path can be specified.  This specification
mode accepts the query parameter but ignores/drops it.  It would
probably be better to either (a) disallow the query parameter for the
path form, or (b) recognize the query parameter and add to the path.

This patch implements (b) for consistency, and to make it easier for
tooling that might switch between a remote git reference and a local
path reference.

See also issue #4050.
 2020-09-25 09:36:18 -07:00
Gregory Hale faa5607f54 Merge remote-tracking branch 'origin/master' into github-api-token 2020-09-25 12:10:58 -04:00
Eelco Dolstra 7d81582488
Merge pull request #3953 from obsidiansystems/basic-derivation-goal-outputs 
Deduplicate basic derivation goals too
 2020-09-25 17:21:19 +02:00
Kevin Quick 5a35cc29bf
Re-add support for github-access-token, but mark as deprecated. 2020-09-25 08:09:56 -07:00
Kevin Quick ef2a14be19
Fix reference to older name for access-tokens config value. 2020-09-25 08:08:27 -07:00
Eelco Dolstra cbb9862cd9
Merge pull request #3626 from W95Psp/master 
Make `functionArgs` primitive accept primops (fix #3624)
 2020-09-25 15:14:18 +02:00
Eelco Dolstra 7b2ae472ff
expectArg(): Respect the 'optional' flag 2020-09-25 10:27:40 +02:00
Kevin Quick 8fba2a8b54
Update to use access-tokens configuration for github/gitlab access. 
This change provides support for using access tokens with other
instances of GitHub and GitLab beyond just github.com and
gitlab.com (especially company-specific or foundation-specific
instances).

This change also provides the ability to specify the type of access
token being used, where different types may have different handling,
based on the forge type.
 2020-09-24 22:49:44 -07:00
Kevin Quick c2f48cfcee
Complete conversion of "url" to "host" with associated variable renaming. 
Completes the change begun in commit 56f1e0d to consistently use the
"host" attribute for "github" and "gitlab" inputs instead of a "url"
attribute.
 2020-09-24 22:46:03 -07:00
Kevin Quick a439e9488d
Support StringMap configuration settings. 
Allows Configuration values that are space-separated key=value pairs.
 2020-09-24 22:42:59 -07:00
Paul Opiyo 4d863a9fcb Remove redundant value checks 
std::optional had redundant checks for whether it had a value.
An object is emplaced either way so it can be dereferenced
without repeating a value check
 2020-09-24 18:32:03 -05:00
Alexander Bantyev ed218e1d6c
Fix max-jobs option 
After 0ed946aa61, max-jobs setting (-j/--max-jobs)
stopped working.

The reason was that nrLocalBuilds (which compared to maxBuildJobs to figure
out whether the limit is reached or not) is not incremented yet when tryBuild
is started; So, the solution is to move the check to tryLocalBuild.

Closes https://github.com/nixos/nix/issues/3763
 2020-09-25 00:07:42 +03:00
Kevin Quick bd5f3dbe11
Fixes fall-through to report correct description of hash-file command. 2020-09-24 12:30:03 -07:00
Eelco Dolstra 8d9402f411
Merge pull request #4054 from edolstra/fix-4021 
registerOutputs(): Don't call canonicalisePathMetaData() twice
 2020-09-23 21:57:53 +02:00
Eelco Dolstra 4ce8a3ed45 Hopefully fix EPERM on macOS 2020-09-23 21:29:10 +02:00
Eelco Dolstra 9a24ece122 Fix exception 2020-09-23 20:21:08 +02:00
Eelco Dolstra 236d9ee7f7 lstat() cleanup 2020-09-23 19:17:28 +02:00
Eelco Dolstra 688bd4fb50 After rewriting a path, make it read-only 2020-09-23 19:10:16 +02:00
Dominique Martinet 2548347bba libutil/archive: add preallocate-contents option 
Make archive preallocation (fallocate) optional because some filesystems
like btrfs do not behave as expected with fallocate.

See #3550.
 2020-09-23 18:49:11 +02:00
Eelco Dolstra cec9473871 DerivationGoal::registerOutputs(): Don't canonicalize twice 
Fixes #4021.
 2020-09-23 18:21:22 +02:00
Eelco Dolstra d4f8163d10 canonicalisePathMetaData_(): Change assertion to error message 2020-09-23 18:21:22 +02:00
Eelco Dolstra e8f0b1e996 DerivationGoal::registerOutputs(): Fix bad format string 2020-09-23 18:21:22 +02:00
John Ericson 3f226f71c1 Return more info from BinaryCacheStore::addToStoreCommon 
We don't need it yet, but we could/should in the future, and it's a
cost-free change since we already have the reference. I like it.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
 2020-09-23 14:40:41 +00:00
John Ericson 412b3a54fb
Clarify FIXME in BinaryCacheStore::addToStoreCommon 
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
 2020-09-23 10:36:55 -04:00
Eelco Dolstra 8a2e10827f Remove unused Flake::vOutputs field 2020-09-23 14:08:52 +02:00
Domen Kožar dac8a1a5e1
Merge pull request #4052 from ujjwaljainnn/fix-typo 
fixed typo
 2020-09-23 09:58:30 +02:00
John Ericson 9fbc31a65b Get rid of Hash::dummy from BinaryCacheStore 2020-09-23 04:56:04 +00:00
ujjwal a2f5c921d4 fixed typo 2020-09-22 23:37:06 +05:30
John Ericson 993229cdaf Deduplicate basic derivation goals too 
See comments for security concerns.

Also optimize goal creation by not traversing map twice.
 2020-09-22 17:13:59 +00:00
John Ericson e9fc2031f0 Merge remote-tracking branch 'upstream/master' into templated-daemon-protocol 2020-09-22 14:18:31 +00:00
regnat 67cc356bb3 Merge remote-tracking branch 'origin/master' into non-ca-depending-on-ca 2020-09-22 16:11:25 +02:00
Eelco Dolstra 980edd1f3a RemoteStore::addCAToStore(): Don't hold connection while calling queryPathInfo() 
This leads to a deadlock if we're at the connection limit.
 2020-09-22 15:28:20 +02:00
Eelco Dolstra 5b107f2c5f
Merge pull request #4038 from maljub01/master 
Add a nix.conf option for allowing a symlinked store
 2020-09-22 13:19:22 +02:00
Eelco Dolstra 35a0ac1838 Style fixes 2020-09-22 11:40:19 +02:00
Eelco Dolstra 92ac8df0ec Merge branch 'add-ca-to-store' of https://github.com/hercules-ci/nix 2020-09-22 11:31:33 +02:00
Eelco Dolstra e7f1109f06
Merge pull request #4040 from OmnipotentEntity/master 
Fix compatibility with nlohmann-json 3.9.1
 2020-09-22 11:09:25 +02:00
Eelco Dolstra 7dd8baafe1
Merge pull request #4041 from cole-h/enum-stringify 
Serialize SandboxMode enum to string for JSON
 2020-09-22 11:08:56 +02:00
regnat c1e79f870c Silence a compiler warning in serialise.hh 
Explicitely cast to `uint64_t` in `readNum` to avoid a "comparison
between signed and unsigned" warning
 2020-09-22 10:39:29 +02:00
John Ericson 3786a801c3 Merge remote-tracking branch 'upstream/master' into ca-floating-upstream 2020-09-22 04:15:55 +00:00
John Ericson b92d3b2edd Merge remote-tracking branch 'upstream/master' into templated-daemon-protocol 2020-09-22 00:45:55 +00:00
Michael Reilly d860295e11 Bump nlohmann-json version to 3.9.1 2020-09-21 14:24:43 -04:00
Cole Helbling ba37299a03
Serialize SandboxMode enum to string for JSON 
Rather than showing an integer as the default, instead show the boolean
referenced in the description.

The nix.conf.5 manpage used to show "default: 0", which is unnecessarily
opaque and confusing (doesn't 0 mean false, even though the default is
true?); now it properly shows that the default is true.
 2020-09-21 10:36:45 -07:00
Eelco Dolstra ecc8672aa0 fmt.hh: Don't include boost/algorithm/string/replace.hpp 
This cuts compilation time by ~49s.

Issue #4045.
 2020-09-21 19:07:55 +02:00
Eelco Dolstra 557d2427ee Random header cleanup 2020-09-21 18:59:02 +02:00
Eelco Dolstra 0716adaa8b abstractsettingtojson.hh -> abstract-setting-to-json.hh 2020-09-21 18:49:43 +02:00
Eelco Dolstra 340ca382c4 Don't include nlohmann/json.hpp in globals.hh 
This reduces compilation time by 207s.

Issue #4045.
 2020-09-21 18:47:18 +02:00
Eelco Dolstra d51ba43047 Move Callback into its own header 
This gets rid of the inclusion of <future> in util.hh, cutting
compilation time by ~20s (CPU time).

Issue #4045.
 2020-09-21 18:42:21 +02:00
Marwan Aljubeh f80ffeb8c9 Update the variable name accordingly 2020-09-21 17:29:08 +01:00
Eelco Dolstra e8e1d420f3 Don't include <regex> in header files 
This reduces compilation time by ~15 seconds (CPU time).

Issue #4045.
 2020-09-21 18:22:45 +02:00
Marwan Aljubeh 4e1a04733d Use a better name for the config option 2020-09-21 16:32:22 +01:00
Eelco Dolstra cbe0bb29f4
Merge pull request #4035 from Ma27/url-attr 
libfetchers/github: allow `url` attribute
 2020-09-21 17:26:36 +02:00
Maximilian Bosch 56f1e0df05
libfetchers/github: rename url to host 2020-09-21 16:29:08 +02:00
Robert Hensing ca30abb3fb Document addCAToStore/addToStoreFromDump source drainage 
Also checked that all usages satisfy the requirement and
removed dead code.
 2020-09-21 07:55:47 +02:00
Robert Hensing fa08db5c4c wopAddToStore: return ValidPathInfo 
A ValidPathInfo is created anyway. By returning it we can save a
roundtrip and we have a nicer interface.
 2020-09-21 07:55:47 +02:00
Robert Hensing 7c68264085 wopAddToStore: add RepairFlag 2020-09-21 07:55:47 +02:00
Robert Hensing fbf509c113 parseContentAddressMethodPrefix: use string_view 
Co-authored-by: John Ericson <git@JohnEricson.me>
 2020-09-21 07:55:47 +02:00
Robert Hensing 8279178b07 Move FramedSink next to FramedSource 2020-09-21 07:55:47 +02:00
Robert Hensing ecc8088cb7 wopAddToStore: Throw to clarify unused refs 
Co-authored-by: John Ericson <git@JohnEricson.me>
 2020-09-21 07:55:47 +02:00
Robert Hensing c602ebfb34 Refactor wopAddToStore to make wopAddTextToStore obsolete 2020-09-21 07:55:45 +02:00
Robert Hensing e34fe47d0c Overhaul wopAddToStore 2020-09-21 07:54:05 +02:00
Marwan Aljubeh e40772cd35 Lint issue: replacing tabs with spaces 2020-09-18 17:18:45 +01:00
Marwan Aljubeh c00e078343 Add a nix.conf option for allowing a symlinked store 2020-09-18 17:10:39 +01:00
Bryan Richter 5fe375a8f1
nix-prefetch-url: Add --executable flag 
pkgs.fetchurl supports an executable argument, which is especially nice
when downloading a large executable. This patch adds the same option to
nix-prefetch-url.

I have tested this to work on the simple case of prefetching a little
executable:

1. nix-prefetch-url --executable https://my/little/script
2. Paste the hash into a pkgs.fetchurl-based package, script-pkg.nix
3. Delete the output from the store to avoid any misidentified artifacts
4. Realise the package script-pkg.nix
5. Run the executable

I repeated the above while using --name, as well.

I suspect --executable would have no meaningful effect if combined with
--unpack, but I have not tried it.
 2020-09-18 19:09:45 +03:00
Maximilian Bosch 2bcf8cbe7a
libfetchers/github: allow url attribute 
Since 108debef6f we allow a
`url`-attribute for the `github`-fetcher to fetch tarballs from
self-hosted `gitlab`/`github` instances.

However it's not used when defining e.g. a flake-input

    foobar = {
        type = "github";
        url = "gitlab.myserver";
        /* ... */
    }

and breaks with an evaluation-error:

    error: --- Error --------------------------------------nix
    unsupported input attribute 'url'
    (use '--show-trace' to show detailed location information)

This patch allows flake-inputs to be fetched from self-hosted instances
as well.
 2020-09-18 14:10:45 +02:00
Eelco Dolstra 958bf57123 nix build: find() -> get() 
find() returns an iterator so "!attr" doesn't work.
 2020-09-18 13:10:42 +02:00
Robert Hensing 14b30b3f3d Move FramedSource and FramedSink, extract withFramedSink 2020-09-17 20:21:04 +02:00
Robert Hensing dfa547c6a8 Add ContentAddressMethod and parse/render it 2020-09-17 20:21:04 +02:00
Robert Hensing 29c82ccc77 Add Source.drainInto(Sink) 2020-09-17 20:21:04 +02:00
Robert Hensing 9ee3122ec7 Remove redundant import 2020-09-17 20:21:04 +02:00
Robert Hensing 3f93bc0d39 Typo 2020-09-17 20:21:04 +02:00
John Ericson b7df353f27 Merge remote-tracking branch 'upstream/master' into ca-floating-upstream 2020-09-17 16:33:10 +00:00
Eelco Dolstra 649d3aaf24
Merge pull request #3829 from obsidiansystems/remove-storetype-delegate-regStore 
Remove storetype delegate reg store -- contains #3736
 2020-09-17 13:55:01 +02:00
Eelco Dolstra b94a35ef40
Merge pull request #4027 from tweag/fix-gc-of-ca-derivations 
Fix garbage collection of CA derivations
 2020-09-17 13:46:26 +02:00
regnat 520895b1da Fix garbage collection of CA derivations 
Fix #4026
 2020-09-17 13:36:58 +02:00
Eelco Dolstra 787469c7b6 Remove corepkgs/unpack-channel.nix 2020-09-17 09:41:02 +02:00
Eelco Dolstra 10d1865f5f Remove corepkgs/derivation.nix 2020-09-17 09:41:02 +02:00
John Ericson f60b380a7f Merge remote-tracking branch 'upstream/master' into remove-storetype-delegate-regStore 2020-09-16 22:35:24 +00:00
John Ericson 2741fffa35 Ensure resolved CA derivations are written 
so we can link outputs to deriver and thus properly cache.
 2020-09-16 17:57:32 +00:00
John Ericson 7fdbb377ba Start to fix floating CA + remote building 2020-09-16 17:50:48 +00:00
John Ericson c5ccebae00 Merge remote-tracking branch 'upstream/master' into ca-floating-upstream 2020-09-16 17:50:40 +00:00
Greg Hale a303c0b6dc Fetch commits from github/gitlab using Auth header 
`nix flake info` calls the github 'commits' API, which requires
authorization when the repository is private. Currently this request
fails with a 404.

This commit adds an authorization header when calling the 'commits' API.
It also changes the way that the 'tarball' API authenticates, moving the
user's token from a query parameter into the Authorization header.

The query parameter method is recently deprecated and will be disallowed
in November 2020. Using them today triggers a warning email.
 2020-09-16 13:46:48 -04:00
Eelco Dolstra 5080d4e7b2 Merge branch 'document-store-options' of https://github.com/tweag/nix 2020-09-16 17:02:30 +02:00
Eelco Dolstra 0066ef6c59 Fix doc generation 2020-09-16 16:56:28 +02:00
Eelco Dolstra 2eacc1bc00 builtins.toFile: Fix indentation 2020-09-16 14:18:46 +02:00
Théophane Hufschmitt 77a0e2c5be
Remove useless exception copy 
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
 2020-09-16 14:00:21 +02:00
regnat d72927aa7a Fix the s3 store 
Add some necessary casts in the initialisation of the store's config
 2020-09-16 13:53:09 +02:00
regnat e0817cbcdc Don't include nlohmann/json.hpp in config.hh 
Instead make a separate header with the template implementation of
`BaseSetting<T>::toJSONObj` that can be included where needed
 2020-09-16 13:53:09 +02:00
regnat 93c0e14a30 Include the full nlohmann/json header in config.hh 
It is apparently required for using `toJSONObject()`, which we do inside
the header file (because it's in a template).

This was accidentally working when building Nix itself (presumably because
`config.hh` was always included after `nlohman/json.hpp`) but caused a
(pretty dirty) build failure in the perl bindings package.
 2020-09-16 13:53:09 +02:00
regnat fc2d31c423 Add (StoreConfig*) casts to work around a GCC bug 
Work around https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80431 that was
already there in the code but was accidentally removed in the last
commits
 2020-09-16 13:53:09 +02:00
regnat a1e82ba450 fixup! Add a default value for the settings 2020-09-16 13:53:09 +02:00
regnat 888f7afe9f Fix build issues with gcc 2020-09-16 13:53:09 +02:00
regnat 634cb2a5ae Add a markdown output to nix describe-stores 2020-09-16 13:53:09 +02:00
regnat b73adacc1e Add a name to the stores 
So that it can be printed by `nix describe-stores`
 2020-09-16 13:53:09 +02:00
regnat f24f0888f9 Document the new store hierarchy 2020-09-16 13:53:09 +02:00
regnat d65962db4d Make uri schemes grammar more RFC-compliant 
Allow `-` and `.` in the RFC schemes as stated by
[RFC3986](https://tools.ietf.org/html/rfc3986#section-3.1).

Practically, this is needed so that `ssh-ng` is a valid URI scheme
 2020-09-16 13:53:09 +02:00
regnat 29a632386e fixup! Make the store plugins more introspectable 2020-09-16 13:53:09 +02:00
regnat 1129913c4e fixup! Correctly call all the parent contructors of the stores 2020-09-16 13:53:09 +02:00
regnat 7f103dcddd Properly filter the stores according to their declared uriSchemes 
When opening a store, only try the stores whose `uriSchemes()` include
the current one
 2020-09-16 13:53:09 +02:00
regnat 5895184df4 Correctly call all the parent contructors of the stores 
Using virtual inheritance means that only the default constructors of
the parent classes will be called, which isn't what we want
 2020-09-16 13:53:09 +02:00