Commit graph

40 commits

Author SHA1 Message Date
dramforever fc6bfb261d libfetchers/tarball: Lock on effectiveUrl
Basically, if a tarball URL is used as a flake input, and the URL leads
to a redirect, the final redirect destination would be recorded as the
locked URL.

This allows tarballs under https://nixos.org/channels to be used as
flake inputs. If we, as before, lock on to the original URL it would
break every time the channel updates.
2021-03-02 21:56:50 +08:00
Eelco Dolstra 1bbc66f865 Merge branch 'slashes-in-github-branches' of https://github.com/Ma27/nix 2021-01-18 12:55:06 +01:00
Matthew Bauer d4870462f8 Cast variants fully for libc++10
libc++10 seems to be stricter on what it allows in variant conversion.
I'm not sure what the rules are here, but this is the minimal change
needed to get through the compilation errors.
2020-12-23 23:41:58 -06:00
Maximilian Bosch 59f2dd8e8d
libfetchers/github: allow slashes in refs
Refs #4061
2020-10-06 20:08:51 +02:00
Eelco Dolstra ad143c5b3b Shut up some clang warnings 2020-10-06 14:52:30 +02:00
Eelco Dolstra 85c8be6286 Remove static variable name clashes
This was useful for an experiment with building Nix as a single
compilation unit. It's not very useful otherwise but also doesn't
hurt...
2020-10-06 13:49:20 +02:00
Eelco Dolstra 20a1e20d91 Style 2020-09-30 13:35:56 +02:00
Eelco Dolstra 274357eb6a Simplify 2020-09-30 12:09:18 +02:00
Kevin Quick 5e7838512e
Remove github-access-token in favor of access-token. 2020-09-29 16:26:34 -07:00
Kevin Quick 66c3959e8c
Merge branch 'master' into access-tokens 2020-09-29 08:32:06 -07:00
Eelco Dolstra 5999978a05 Make Headers an optional argument 2020-09-29 13:05:19 +02:00
Eelco Dolstra de86abbf3f Cleanup 2020-09-29 12:55:06 +02:00
Gregory Hale faa5607f54 Merge remote-tracking branch 'origin/master' into github-api-token 2020-09-25 12:10:58 -04:00
Kevin Quick 5a35cc29bf
Re-add support for github-access-token, but mark as deprecated. 2020-09-25 08:09:56 -07:00
Kevin Quick 8fba2a8b54
Update to use access-tokens configuration for github/gitlab access.
This change provides support for using access tokens with other
instances of GitHub and GitLab beyond just github.com and
gitlab.com (especially company-specific or foundation-specific
instances).

This change also provides the ability to specify the type of access
token being used, where different types may have different handling,
based on the forge type.
2020-09-24 22:49:44 -07:00
Kevin Quick c2f48cfcee
Complete conversion of "url" to "host" with associated variable renaming.
Completes the change begun in commit 56f1e0d to consistently use the
"host" attribute for "github" and "gitlab" inputs instead of a "url"
attribute.
2020-09-24 22:46:03 -07:00
Eelco Dolstra e8e1d420f3 Don't include <regex> in header files
This reduces compilation time by ~15 seconds (CPU time).

Issue #4045.
2020-09-21 18:22:45 +02:00
Maximilian Bosch 56f1e0df05
libfetchers/github: rename url to host 2020-09-21 16:29:08 +02:00
Maximilian Bosch 2bcf8cbe7a
libfetchers/github: allow url attribute
Since 108debef6f we allow a
`url`-attribute for the `github`-fetcher to fetch tarballs from
self-hosted `gitlab`/`github` instances.

However it's not used when defining e.g. a flake-input

    foobar = {
        type = "github";
        url = "gitlab.myserver";
        /* ... */
    }

and breaks with an evaluation-error:

    error: --- Error --------------------------------------nix
    unsupported input attribute 'url'
    (use '--show-trace' to show detailed location information)

This patch allows flake-inputs to be fetched from self-hosted instances
as well.
2020-09-18 14:10:45 +02:00
Greg Hale a303c0b6dc Fetch commits from github/gitlab using Auth header
`nix flake info` calls the github 'commits' API, which requires
authorization when the repository is private. Currently this request
fails with a 404.

This commit adds an authorization header when calling the 'commits' API.
It also changes the way that the 'tarball' API authenticates, moving the
user's token from a query parameter into the Authorization header.

The query parameter method is recently deprecated and will be disallowed
in November 2020. Using them today triggers a warning email.
2020-09-16 13:46:48 -04:00
Eelco Dolstra 6d7f7efb89
github: Use access token when calling .../commits API 2020-09-01 15:29:04 +02:00
John Ericson cc0d77f8c9 Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors 2020-07-16 17:28:52 +00:00
Maximilian Bosch cf9f33995b
Fix gitlab-fetcher to obtain tags and branches
Until now, the `gitlab`-fetcher determined the source's rev by checking
the latest commit of the given `ref` using the
`/repository/branches`-API.

This breaks however when trying to fetch a gitlab-repo by its tag:

```
$ nix repl
nix-repl> builtins.fetchTree gitlab:Ma27/nvim.nix/0.2.0
error: --- Error ------------------------------------------------------------------------------------- nix
unable to download 'https://gitlab.com/api/v4/projects/Ma27%2Fnvim.nix/repository/branches/0.2.0': HTTP error 404 ('')
```

When using the `/commits?ref_name`-endpoint[1] you can pass any kind of
valid ref to the `gitlab`-fetcher.

Please note that this fetches the only first 20 commits on a ref,
unfortunately there's currently no endpoint which only retrieves the
latest commit of any kind of `ref`.

[1] https://docs.gitlab.com/ee/api/commits.html#list-repository-commits
2020-07-13 19:22:59 +02:00
Carlo Nucera 263ccdd489 Rename two hash constructors to proper functions 2020-07-01 18:34:18 -04:00
Eelco Dolstra 3d492199bb github: Respect default branch 2020-06-18 13:25:08 +02:00
Eelco Dolstra 1524752c17 Merge remote-tracking branch 'origin/master' into flakes 2020-06-17 10:26:52 +02:00
John Ericson f6f01416b7 Use std::string_view in a few more places 2020-06-12 21:32:30 +00:00
Eelco Dolstra 6cfc2db494 Fix applyOverride() for github 2020-06-09 13:45:07 +02:00
Eelco Dolstra ab54031e04 getDefaultUrl() -> value_or() 2020-06-04 20:24:28 +02:00
Finn Behrens 108debef6f
add support for selfhosted gitlab/github 2020-06-04 14:42:39 +02:00
Matthew Kenigsberg c254254a80 use Tree ctor 2020-06-01 12:32:17 -06:00
Eelco Dolstra 0e7f77a59a Check revCount / lastModified input attributes if specified 2020-05-30 00:59:13 +02:00
Eelco Dolstra 950b46821f Remove TreeInfo
The attributes previously stored in TreeInfo (narHash, revCount,
lastModified) are now stored in Input. This makes it less arbitrary
what attributes are stored where.

As a result, the lock file format has changed. An entry like

    "info": {
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },
    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github"
    },

is now stored as

    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github",
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },

The 'Input' class is now a dumb set of attributes. All the fetcher
implementations subclass InputScheme, not Input. This simplifies the
API.

Also, fix substitution of flake inputs. This was broken since lazy
flake fetching started using fetchTree internally.
2020-05-30 00:44:11 +02:00
Eelco Dolstra 5633c0975b Factor out GitHub / GitLab commonality 2020-05-29 14:23:32 +02:00
Eelco Dolstra d103c79144
Merge remote-tracking branch 'origin/master' into flakes 2020-04-09 15:54:21 +02:00
Nikola Knezevic 7867685dcd after flake rebase 2020-04-08 22:26:57 +02:00
Eelco Dolstra f58a9b0e62 Respect the narHash attribute in more input types
(cherry picked from commit a6ff66b658)
2020-04-07 09:09:18 +02:00
Eelco Dolstra 462421d345 Backport libfetchers from the flakes branch
This provides a pluggable mechanism for defining new fetchers. It adds
a builtin function 'fetchTree' that generalizes existing fetchers like
'fetchGit', 'fetchMercurial' and 'fetchTarball'. 'fetchTree' takes a
set of attributes, e.g.

  fetchTree {
    type = "git";
    url = "https://example.org/repo.git";
    ref = "some-branch";
    rev = "abcdef...";
  }

The existing fetchers are just wrappers around this. Note that the
input attributes to fetchTree are the same as flake input
specifications and flake lock file entries.

All fetchers share a common cache stored in
~/.cache/nix/fetcher-cache-v1.sqlite. This replaces the ad hoc caching
mechanisms in fetchGit and download.cc (e.g. ~/.cache/nix/{tarballs,git-revs*}).

This also adds support for Git worktrees (c169ea5904).
2020-04-07 09:03:14 +02:00
Eelco Dolstra a6ff66b658 Respect the narHash attribute in more input types
call-flake.nix now passes node.info.narHash to fetchTree. This ensures
that dirty Git trees work even in pure mode.
2020-04-02 19:04:33 +02:00
Eelco Dolstra e0a0ae0467 Move fetchers from libstore to libfetchers 2020-03-30 14:04:53 +02:00
Renamed from src/libstore/fetchers/github.cc (Browse further)