Commit graph

8700 commits

Author SHA1 Message Date
Carlo Nucera fd2eb41e64 Move file-hash to content-address 2020-06-02 15:44:58 -04:00
Carlo Nucera 343c20a404 WIP Completed implementation 2020-06-02 15:23:21 -04:00
John Ericson c664e68b87 Fix to-base --type handler to correctly set std::optional flag
Now that we have a separate flag function, also describe why it is
optional.
2020-06-02 18:25:32 +00:00
Carlo Nucera 390bf64858 WIP 2020-06-02 14:15:58 -04:00
John Ericson c502119fd3 to-base supports parsing SRI hashes, so make type flag optional 2020-06-02 18:05:26 +00:00
John Ericson a33270ce1d Clean up ValidPathInfo::isContentAddressed with std::visit 2020-06-02 17:04:21 +00:00
John Ericson 25e61812f3
Apply suggestions from code review
Co-authored-by: Matthew Bauer <mjbauer95@gmail.com>
2020-06-02 12:47:18 -04:00
John Ericson d73dbc8e4c Remove hashingWithUnknownAlgoExits
A valid hash type must be provided now. The hash itself can still be
invalid, but that doesn't cause an `abort()`.
2020-06-02 16:28:54 +00:00
John Ericson 64cffb804a Merge remote-tracking branch 'upstream/master' into no-hash-type-unknown 2020-06-02 16:07:25 +00:00
John Ericson 450dcf2c1b Remove HashType::Unknown
Instead, `Hash` uses `std::optional<HashType>`. In the future, we may
also make `Hash` itself require a known hash type, encoraging people to
use `std::optional<Hash>` instead.
2020-06-02 15:52:13 +00:00
Carlo Nucera a5d820a0a3 Change parseCa(Opt) to parseContentAddress(Opt) 2020-06-02 11:00:10 -04:00
Ben Burdette 156d4f8bc8 remove extra space in SysErrors 2020-06-02 08:45:37 -06:00
John Ericson 1b6461f671 Merge remote-tracking branch 'upstream/master' into validPathInfo-ca-proper-datatype 2020-06-02 14:31:18 +00:00
Ben Burdette d82d230b40 elide the 'ErrorInfo' in logError and logWarning calls 2020-06-02 08:22:24 -06:00
Eelco Dolstra bfa1acd85c
Merge pull request #3639 from obsidiansystems/do-fixme-store-removes
Remove `addToStore` variant as requested by `FIXME`
2020-06-02 15:39:07 +02:00
Eelco Dolstra c16fdda3a6 Merge branch 'lru-tests' of https://github.com/gilligan/nix 2020-06-02 12:07:48 +02:00
Eelco Dolstra e9fee8e6a7
src/libutil/tests/lru-cache.cc: Check erase()
Co-authored-by: James Lee <jbit@jbit.net>
2020-06-02 12:06:59 +02:00
Eelco Dolstra 0748a72a20
Merge pull request #3642 from knl/improve-ref-validity-checking-in-fetchgit
Improve ref validity checking in fetchgit
2020-06-02 12:00:24 +02:00
Eelco Dolstra 7dbba0a94e
Merge pull request #3645 from mkenigs/fetchOrSubstituteTree-improvements
Cache tree in fetchOrSubstituteTree
2020-06-02 11:58:20 +02:00
John Ericson efcd30da89 WIP 2020-06-02 00:37:43 +00:00
John Ericson 754c910953 WIP more progress 2020-06-01 19:26:40 -04:00
Carlo Nucera da39092a39 WIP 2020-06-01 18:53:31 -04:00
Carlo Nucera 0e9438b6d3 Create new file-hash files 2020-06-01 17:32:40 -04:00
Carlo Nucera 0cb67ecbd3 Merge branch 'derivation-header-include-order' of github.com:Ericson2314/nix into validPathInfo-ca-proper-datatype 2020-06-01 17:13:11 -04:00
Carlo Nucera f4b89e11a4 Merge branch 'no-stringly-typed-derivation-output' of github.com:Ericson2314/nix into validPathInfo-ca-proper-datatype 2020-06-01 17:12:50 -04:00
Matthew Kenigsberg c254254a80 use Tree ctor 2020-06-01 12:32:17 -06:00
Matthew Kenigsberg 7680993506 Tree ctors 2020-06-01 09:01:37 -06:00
Matthew Kenigsberg ff1320b850 fetchOrSubstituteTree improvements
Caches tree in addition to lockedRef, and explicitly writes out the logic for different combinations of cached/uncached flakes and indirect/resolved/locked flakes. This eliminates uneccessary calls to lookupInFlakeCache, fetchTree, maybeLookupFlake, and flakeCache.push_back
2020-06-01 02:57:22 -06:00
Tobias Pflug eca1ff7a9f Add tests for lru-cache.hh 2020-05-31 01:05:05 +02:00
Nikola Knezevic fb38459d6e Ensure we restrict refspec interpretation while fetching
As `git fetch` may chose to interpret refspec to it's liking, ensure that we
only pass refs that begin with `refs/` as is, otherwise, prepend them with
`refs/heads`. Otherwise, branches named `heads/foo` (I know it's bad, but it's
allowed), would be fetched as `foo`, instead of `heads/foo`.
2020-05-30 12:33:38 +02:00
Nikola Knezevic 77007d4eab Improve ref validity checking in fetchGit
The previous regex was too strict and did not match what git was allowing. It
could lead to `fetchGit` not accepting valid branch names, even though they
exist in a repository (for example, branch names containing `/`, which are
pretty standard, like `release/1.0` branches).

The new regex defines what a branch name should **NOT** contain. It takes the
definitions from `refs.c` in https://github.com/git/git and `git help
check-ref-format` pages.

This change also introduces a test for ref name validity checking, which
compares the result from Nix with the result of `git check-ref-format --branch`.
2020-05-30 12:29:35 +02:00
Eelco Dolstra 89e0b3e2d6 Move substitution into Input::fetch()
Closes #3520.
2020-05-30 01:16:53 +02:00
Eelco Dolstra 0e7f77a59a Check revCount / lastModified input attributes if specified 2020-05-30 00:59:13 +02:00
Eelco Dolstra 950b46821f Remove TreeInfo
The attributes previously stored in TreeInfo (narHash, revCount,
lastModified) are now stored in Input. This makes it less arbitrary
what attributes are stored where.

As a result, the lock file format has changed. An entry like

    "info": {
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },
    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github"
    },

is now stored as

    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github",
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },

The 'Input' class is now a dumb set of attributes. All the fetcher
implementations subclass InputScheme, not Input. This simplifies the
API.

Also, fix substitution of flake inputs. This was broken since lazy
flake fetching started using fetchTree internally.
2020-05-30 00:44:11 +02:00
John Ericson fac0c2d54a Remove addToStore variant as requested by FIXME
The idea is it's always more flexible to consumer a `Source` than a
plain string, and it might even reduce memory consumption.

I also looked at `addToStoreFromDump` with its `// FIXME: remove?`, but
the worked needed for that is far more up for interpretation, so I
punted for now.
2020-05-29 17:02:32 -04:00
Ben Burdette 734283d636 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-05-29 09:51:37 -06:00
Eelco Dolstra 5633c0975b Factor out GitHub / GitLab commonality 2020-05-29 14:23:32 +02:00
Eelco Dolstra 56f9abffbe Merge branch 'gitlab' of https://github.com/Kloenk/nixos-nix into flakes 2020-05-29 13:55:21 +02:00
Finn Behrens 5256bc77ca
add gitlab libfetcher 2020-05-28 23:00:08 +02:00
Carlo Nucera 6dd471ebf6 Fixing the result of merge 2020-05-28 12:14:36 -04:00
Carlo Nucera 4f597fb901 Merge branch 'master' of github.com:NixOS/nix into enum-class 2020-05-28 10:58:22 -04:00
John Ericson 5b4cd84bc2 Merge remote-tracking branch 'me/more-rust-ffi' into no-stringly-typed-derivation-output 2020-05-28 10:35:53 -04:00
John Ericson ef71caba29 Merge remote-tracking branch 'upstream/master' into more-rust-ffi 2020-05-28 10:31:46 -04:00
Eelco Dolstra f60ce4fa20
Merge pull request #3631 from andir/libutil-config-tests
Add unit tests for config.cc
2020-05-28 13:51:37 +02:00
Eelco Dolstra de141fcb79
Merge pull request #3455 from Ericson2314/enum-FileIngestionMethod
Replace some `bool recursive` with a new `FileIngestionMethod` enum
2020-05-28 13:50:06 +02:00
Eelco Dolstra 17ca997fc6 Merge remote-tracking branch 'origin/master' into flakes 2020-05-28 12:55:24 +02:00
Eelco Dolstra c3eff22f46 Merge branch 'store-visited' of https://github.com/mkenigs/nix into flakes 2020-05-28 12:15:39 +02:00
Eelco Dolstra 6286272371 nixpkgsFlakeRef(): Use locked nixpkgs 2020-05-28 12:13:13 +02:00
Eelco Dolstra 04fb4e8a0f Merge branch 'nixpkgs#bashInteractive' of https://github.com/mkenigs/nix into flakes 2020-05-28 11:59:54 +02:00
Eelco Dolstra d2a537568a
Merge pull request #3632 from LnL7/darwin-xz
installer: don't require xz on darwin
2020-05-28 11:19:17 +02:00