This provides a pluggable mechanism for defining new fetchers. It adds
a builtin function 'fetchTree' that generalizes existing fetchers like
'fetchGit', 'fetchMercurial' and 'fetchTarball'. 'fetchTree' takes a
set of attributes, e.g.
fetchTree {
type = "git";
url = "https://example.org/repo.git";
ref = "some-branch";
rev = "abcdef...";
}
The existing fetchers are just wrappers around this. Note that the
input attributes to fetchTree are the same as flake input
specifications and flake lock file entries.
All fetchers share a common cache stored in
~/.cache/nix/fetcher-cache-v1.sqlite. This replaces the ad hoc caching
mechanisms in fetchGit and download.cc (e.g. ~/.cache/nix/{tarballs,git-revs*}).
This also adds support for Git worktrees (c169ea5904).
When encountering an unsupported protocol, there's no need to retry.
Chances are, it won't suddenly be supported between retry attempts;
error instead. Otherwise, you see something like the following:
$ nix-env -i -f git://git@github.com/foo/bar
warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 335 ms
warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 604 ms
warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 1340 ms
warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 2685 ms
With this change, you now see:
$ nix-env -i -f git://git@github.com/foo/bar
error: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1)
This allows querying the location of function arguments. E.g.
builtins.unsafeGetAttrPos "x" (builtins.functionArgs ({ x }: null))
=> { column = 57; file = "/home/infinisil/src/nix/inst/test.nix"; line = 1; }
Using std::filesystem means also having to link with -lstdc++fs on
some platforms and it's hard to discover for what platforms this is
needed. As all the functionality is already implemented as utilities,
use those instead.
Due to fetchGit not checking if rev is an ancestor of ref (there is even
a FIXME comment about it in the code), the cache repo might not have the
ref even though it has the rev. This doesn't matter when submodule =
false, but the submodule = true code blows up because it tries to fetch
the (missing) ref from the cache repo.
Fix this in the simplest way possible: fetch all refs from the local
cache repo when submodules = true.
TODO: Add tests.
The .link file is used as a lock, so I think we should put the
"submodule" attribute in there since turning on submodules creates a new
.link file path.
The install-multi-user script uses blue, green, and red colors, as
well as bold and underline, to add helpful formatting that helps
structure its rather voluminous output.
Unfortunately, the terminal escape sequences it uses are not quite
well-formed. The relevant information is all there, just obscured
by some extra noise, a leading parameter `38`. Empirically, the
result is:
* On macOS, in both Terminal.app and iTerm2, the spurious `38` is
ignored, the rest of the escape sequence is applied, and the colors
show up as intended.
* On Linux, in at least gnome-terminal and xterm, the spurious `38`
and the next parameter after it are ignored, and what's left is
applied. So in the sequence `38;4;32`, the 4 (underline) is
ignored but the 32 (green) takes effect; in a more typical sequence
like `38;34`, the 34 (blue) is ignored and nothing happens.
These codes are all unchanged since this script's origins as a
Darwin-only script -- so the fact that they work fine in common macOS
terminals goes some way to explain how the bug arose.
Happily, we can make the colors work as intended by just deleting the
extra `38;`. Tested in all four terminals mentioned above; the new
codes work correctly on all of them, and on the two macOS terminals
they work exactly the same as before.
---
In a bit more technical detail -- perhaps more than anyone, me
included, ever wanted to know, but now that I've gone and learned it
I'll write it down anyway :) -- here's what's happening in these codes:
An ECMA-48 "control sequence" begins with `\033[` aka "CSI", contains
any number of parameters as semicolon-separated decimal numbers (plus
sometimes other wrinkles), and ends with a byte from 0x40..0x7e. In
our case, with `m` aka "SGR", "Select Graphic Rendition".
An SGR control sequence `\033[...m` sets colors, fonts, text styles,
etc. In particular a parameter `31` means red, `32` green, `34` blue,
`4` underline, and `0` means reset to normal. Those are all we use.
There is also a `38`. This is used for setting colors too... but it
needs arguments. `38;5;nn` is color nn from a 256-color palette, and
`38;2;rr;gg;bb` has the given RGB values.
There is no meaning defined for `38;1` or `38;34` etc. On seeing a
parameter `38` followed by an unrecognized argument for it, apparently
some implementations (as seen on macOS) discard only the `38` and
others (as seen on Linux) discard the argument too before resuming.
This line has been this way since it was written, in 9e08f5efe
in 2006.
I think it was just a small mistake then; Eelco's thesis earlier
that year says the permission on each file is set to 0444 or 0555
in a derivation's output as part of the build process. In any
case I'm pretty sure that's the behavior now.
Julian Stecklina