Commit graph

24 commits

Author SHA1 Message Date
Maximilian Bosch 2857b1baaf
Add explicit allRefs = true; argument to fetchGit
Sometimes it's necessary to fetch a git repository at a revision and
it's unknown which ref contains the revision in question. An example
would be a Cargo.lock which only provides the URL and the revision when
using a git repository as build input.

However it's considered a bad practice to perform a full checkout of a
repository since this may take a lot of time and can eat up a lot of
disk space. This patch makes a full checkout explicit by adding an
`allRefs` argument to `builtins.fetchGit` which fetches all refs if
explicitly set to true.

Closes #2409
2020-12-22 10:46:00 +01:00
Maximilian Bosch 629af83b2d
Provide a more meaningful error-message for builtins.fetchGit if a revision can't be checked out
A common pitfall when using e.g. `builtins.fetchGit` is the `fatal: not
a tree object`-error when trying to fetch a revision of a git-repository
that isn't on the `master` branch and no `ref` is specified.

In order to make clear what's the problem, I added a simple check
whether the revision in question exists and if it doesn't a more
meaningful error-message is displayed:

```
nix-repl> builtins.fetchGit { url = "https://github.com/owner/myrepo"; rev = "<commit not on master>"; }
moderror: --- Error -------------------------------------------------------------------- nix
Cannot find Git revision 'bf1cc5c648e6aed7360448a3745bb2fe4fbbf0e9' in ref 'master' of repository 'https://gitlab.com/Ma27/nvim.nix'! Please make sure that the rev exists on the ref you've specified or add allRefs = true; to fetchGit.
```

Closes #2431
2020-12-22 10:46:00 +01:00
Eelco Dolstra 4badb6943f Fix use of dirty Git/Mercurial inputs with chroot stores
Fixes:

  $ nix build --store /tmp/nix /home/eelco/Dev/patchelf#hydraJobs.build.x86_64-linux
  warning: Git tree '/home/eelco/Dev/patchelf' is dirty
  error: --- RestrictedPathError ------------------------------------------------------------------------------------------- nix
  access to path '/tmp/nix/nix/store/xmkvfmffk7xfnazykb5kx999aika8an4-source/flake.nix' is forbidden in restricted mode
  (use '--show-trace' to show detailed location information)
2020-11-10 23:22:45 +01:00
Eelco Dolstra 85c8be6286 Remove static variable name clashes
This was useful for an experiment with building Nix as a single
compilation unit. It's not very useful otherwise but also doesn't
hurt...
2020-10-06 13:49:20 +02:00
Eelco Dolstra e8e1d420f3 Don't include <regex> in header files
This reduces compilation time by ~15 seconds (CPU time).

Issue #4045.
2020-09-21 18:22:45 +02:00
Chuck ed026f7206 Don't try to parse signature check as commit timestamp
When the log.showSignature git setting is enabled, the output of
"git log" contains signature verification information in addition to the
timestamp GitInputScheme::fetch wants:

  $ git log -1 --format=%ct
  gpg: Signature made Sat 07 Sep 2019 02:02:03 PM PDT
  gpg:                using RSA key 0123456789ABCDEF0123456789ABCDEF01234567
  gpg:                issuer "user@example.com"
  gpg: Good signature from "User <user@example.com>" [ultimate] 1567890123
  1567890123

For folks that had log.showSignature set, this caused all nix operations
on flakes to fail:

  $ nix build
  error: stoull
2020-08-13 17:44:42 -07:00
John Ericson 839f0fe095 Merge remote-tracking branch 'upstream/master' into misc-ca 2020-08-05 14:40:01 +00:00
Carlo Nucera 362ae93851 Add UnimplementedError to ease grepping for these 2020-07-20 14:13:37 -04:00
John Ericson cc0d77f8c9 Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors 2020-07-16 17:28:52 +00:00
Carlo Nucera 263ccdd489 Rename two hash constructors to proper functions 2020-07-01 18:34:18 -04:00
Eelco Dolstra 7d554f295c Support building flakes from a shallow Git repo
Fixes #3756.
2020-07-01 14:57:59 +02:00
Eelco Dolstra c20591ddc3 Merge remote-tracking branch 'origin/master' into flakes 2020-06-03 16:15:22 +02:00
Matthew Kenigsberg c254254a80 use Tree ctor 2020-06-01 12:32:17 -06:00
Nikola Knezevic fb38459d6e Ensure we restrict refspec interpretation while fetching
As `git fetch` may chose to interpret refspec to it's liking, ensure that we
only pass refs that begin with `refs/` as is, otherwise, prepend them with
`refs/heads`. Otherwise, branches named `heads/foo` (I know it's bad, but it's
allowed), would be fetched as `foo`, instead of `heads/foo`.
2020-05-30 12:33:38 +02:00
Nikola Knezevic 77007d4eab Improve ref validity checking in fetchGit
The previous regex was too strict and did not match what git was allowing. It
could lead to `fetchGit` not accepting valid branch names, even though they
exist in a repository (for example, branch names containing `/`, which are
pretty standard, like `release/1.0` branches).

The new regex defines what a branch name should **NOT** contain. It takes the
definitions from `refs.c` in https://github.com/git/git and `git help
check-ref-format` pages.

This change also introduces a test for ref name validity checking, which
compares the result from Nix with the result of `git check-ref-format --branch`.
2020-05-30 12:29:35 +02:00
Eelco Dolstra 950b46821f Remove TreeInfo
The attributes previously stored in TreeInfo (narHash, revCount,
lastModified) are now stored in Input. This makes it less arbitrary
what attributes are stored where.

As a result, the lock file format has changed. An entry like

    "info": {
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },
    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github"
    },

is now stored as

    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github",
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },

The 'Input' class is now a dumb set of attributes. All the fetcher
implementations subclass InputScheme, not Input. This simplifies the
API.

Also, fix substitution of flake inputs. This was broken since lazy
flake fetching started using fetchTree internally.
2020-05-30 00:44:11 +02:00
Carlo Nucera b90241ceb1 Change remaining bools with FileIngestionMethod 2020-05-26 11:32:41 -04:00
Eelco Dolstra 849d3968db
Update src/libfetchers/git.cc
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
2020-05-13 10:41:21 +02:00
Eelco Dolstra 54955867a6 Merge remote-tracking branch 'origin/master' into flakes 2020-04-07 14:29:45 +02:00
Eelco Dolstra 55cefd41d6 Merge branch 'fetchgit-recursive' of https://github.com/blitz/nix 2020-04-07 13:45:17 +02:00
Eelco Dolstra f58a9b0e62 Respect the narHash attribute in more input types
(cherry picked from commit a6ff66b658)
2020-04-07 09:09:18 +02:00
Eelco Dolstra 462421d345 Backport libfetchers from the flakes branch
This provides a pluggable mechanism for defining new fetchers. It adds
a builtin function 'fetchTree' that generalizes existing fetchers like
'fetchGit', 'fetchMercurial' and 'fetchTarball'. 'fetchTree' takes a
set of attributes, e.g.

  fetchTree {
    type = "git";
    url = "https://example.org/repo.git";
    ref = "some-branch";
    rev = "abcdef...";
  }

The existing fetchers are just wrappers around this. Note that the
input attributes to fetchTree are the same as flake input
specifications and flake lock file entries.

All fetchers share a common cache stored in
~/.cache/nix/fetcher-cache-v1.sqlite. This replaces the ad hoc caching
mechanisms in fetchGit and download.cc (e.g. ~/.cache/nix/{tarballs,git-revs*}).

This also adds support for Git worktrees (c169ea5904).
2020-04-07 09:03:14 +02:00
Eelco Dolstra a6ff66b658 Respect the narHash attribute in more input types
call-flake.nix now passes node.info.narHash to fetchTree. This ensures
that dirty Git trees work even in pure mode.
2020-04-02 19:04:33 +02:00
Eelco Dolstra e0a0ae0467 Move fetchers from libstore to libfetchers 2020-03-30 14:04:53 +02:00
Renamed from src/libstore/fetchers/git.cc (Browse further)