Commit graph

8715 commits

Author SHA1 Message Date
Ben Burdette 4335ba999b Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-06-03 17:00:00 -06:00
Ben Burdette f97576c5d9 newline-as-prefix; no final newline in output. 2020-06-03 14:47:00 -06:00
Carlo Nucera 132d6f2c24 Clarify the description of StorePath construction 2020-06-03 16:08:32 -04:00
Eelco Dolstra 81cafda306 Fix GitHub test 2020-06-03 16:29:04 +02:00
Eelco Dolstra c20591ddc3 Merge remote-tracking branch 'origin/master' into flakes 2020-06-03 16:15:22 +02:00
Jonas Chevalier 6ee03b8444
libutils/hash: remove default encoding
This will make it easier to reason about the hash encoding and switch to
SRI everywhere where possible.
2020-06-03 13:49:51 +02:00
John Ericson 01572c2198
Missing #include <cassert> in lru-cache.hh (#3654)
This was a latent bug that just appeared because of the tests that were
added. Remember to wait for CI! :)
2020-06-03 10:15:22 +00:00
John Ericson 3c78ac348c Merge remote-tracking branch 'obsidian/no-hash-type-unknown' into validPathInfo-ca-proper-datatype 2020-06-03 04:44:24 +00:00
John Ericson fecff16a6e Merge remote-tracking branch 'obsidian/missing-include-0' into validPathInfo-ca-proper-datatype 2020-06-02 23:23:30 +00:00
John Ericson 39ba87be9b Missing #include <cassert> in lru-cache.hh
This was a latent bug that just appeared because of the tests that were
added. Remember to wait for CI! :)
2020-06-02 21:36:53 +00:00
John Ericson 406dbb7fce outputHashAlgo can be blank so parse accordingly
It is blank for SRI hashes.
2020-06-02 21:09:15 +00:00
John Ericson 1fcd3afc38 Fix hashes 2020-06-02 20:35:17 +00:00
Carlo Nucera 75d2581390 Typo 2020-06-02 16:21:18 -04:00
Carlo Nucera 78f137e931 Validate text version instead, throw Errors 2020-06-02 16:20:22 -04:00
Carlo Nucera a5cdf1867e Add assertions for SHA256 in fixed case 2020-06-02 16:13:08 -04:00
Carlo Nucera fd2eb41e64 Move file-hash to content-address 2020-06-02 15:44:58 -04:00
Carlo Nucera 343c20a404 WIP Completed implementation 2020-06-02 15:23:21 -04:00
John Ericson c664e68b87 Fix to-base --type handler to correctly set std::optional flag
Now that we have a separate flag function, also describe why it is
optional.
2020-06-02 18:25:32 +00:00
Carlo Nucera 390bf64858 WIP 2020-06-02 14:15:58 -04:00
John Ericson c502119fd3 to-base supports parsing SRI hashes, so make type flag optional 2020-06-02 18:05:26 +00:00
John Ericson a33270ce1d Clean up ValidPathInfo::isContentAddressed with std::visit 2020-06-02 17:04:21 +00:00
John Ericson 25e61812f3
Apply suggestions from code review
Co-authored-by: Matthew Bauer <mjbauer95@gmail.com>
2020-06-02 12:47:18 -04:00
John Ericson d73dbc8e4c Remove hashingWithUnknownAlgoExits
A valid hash type must be provided now. The hash itself can still be
invalid, but that doesn't cause an `abort()`.
2020-06-02 16:28:54 +00:00
John Ericson 64cffb804a Merge remote-tracking branch 'upstream/master' into no-hash-type-unknown 2020-06-02 16:07:25 +00:00
John Ericson 450dcf2c1b Remove HashType::Unknown
Instead, `Hash` uses `std::optional<HashType>`. In the future, we may
also make `Hash` itself require a known hash type, encoraging people to
use `std::optional<Hash>` instead.
2020-06-02 15:52:13 +00:00
Carlo Nucera a5d820a0a3 Change parseCa(Opt) to parseContentAddress(Opt) 2020-06-02 11:00:10 -04:00
Ben Burdette 156d4f8bc8 remove extra space in SysErrors 2020-06-02 08:45:37 -06:00
John Ericson 1b6461f671 Merge remote-tracking branch 'upstream/master' into validPathInfo-ca-proper-datatype 2020-06-02 14:31:18 +00:00
Ben Burdette d82d230b40 elide the 'ErrorInfo' in logError and logWarning calls 2020-06-02 08:22:24 -06:00
Eelco Dolstra bfa1acd85c
Merge pull request #3639 from obsidiansystems/do-fixme-store-removes
Remove `addToStore` variant as requested by `FIXME`
2020-06-02 15:39:07 +02:00
Eelco Dolstra c16fdda3a6 Merge branch 'lru-tests' of https://github.com/gilligan/nix 2020-06-02 12:07:48 +02:00
Eelco Dolstra e9fee8e6a7
src/libutil/tests/lru-cache.cc: Check erase()
Co-authored-by: James Lee <jbit@jbit.net>
2020-06-02 12:06:59 +02:00
Eelco Dolstra 0748a72a20
Merge pull request #3642 from knl/improve-ref-validity-checking-in-fetchgit
Improve ref validity checking in fetchgit
2020-06-02 12:00:24 +02:00
Eelco Dolstra 7dbba0a94e
Merge pull request #3645 from mkenigs/fetchOrSubstituteTree-improvements
Cache tree in fetchOrSubstituteTree
2020-06-02 11:58:20 +02:00
John Ericson efcd30da89 WIP 2020-06-02 00:37:43 +00:00
John Ericson 754c910953 WIP more progress 2020-06-01 19:26:40 -04:00
Carlo Nucera da39092a39 WIP 2020-06-01 18:53:31 -04:00
Carlo Nucera 0e9438b6d3 Create new file-hash files 2020-06-01 17:32:40 -04:00
Carlo Nucera 0cb67ecbd3 Merge branch 'derivation-header-include-order' of github.com:Ericson2314/nix into validPathInfo-ca-proper-datatype 2020-06-01 17:13:11 -04:00
Carlo Nucera f4b89e11a4 Merge branch 'no-stringly-typed-derivation-output' of github.com:Ericson2314/nix into validPathInfo-ca-proper-datatype 2020-06-01 17:12:50 -04:00
Matthew Kenigsberg c254254a80 use Tree ctor 2020-06-01 12:32:17 -06:00
Matthew Kenigsberg 7680993506 Tree ctors 2020-06-01 09:01:37 -06:00
Matthew Kenigsberg ff1320b850 fetchOrSubstituteTree improvements
Caches tree in addition to lockedRef, and explicitly writes out the logic for different combinations of cached/uncached flakes and indirect/resolved/locked flakes. This eliminates uneccessary calls to lookupInFlakeCache, fetchTree, maybeLookupFlake, and flakeCache.push_back
2020-06-01 02:57:22 -06:00
Tobias Pflug eca1ff7a9f Add tests for lru-cache.hh 2020-05-31 01:05:05 +02:00
Nikola Knezevic fb38459d6e Ensure we restrict refspec interpretation while fetching
As `git fetch` may chose to interpret refspec to it's liking, ensure that we
only pass refs that begin with `refs/` as is, otherwise, prepend them with
`refs/heads`. Otherwise, branches named `heads/foo` (I know it's bad, but it's
allowed), would be fetched as `foo`, instead of `heads/foo`.
2020-05-30 12:33:38 +02:00
Nikola Knezevic 77007d4eab Improve ref validity checking in fetchGit
The previous regex was too strict and did not match what git was allowing. It
could lead to `fetchGit` not accepting valid branch names, even though they
exist in a repository (for example, branch names containing `/`, which are
pretty standard, like `release/1.0` branches).

The new regex defines what a branch name should **NOT** contain. It takes the
definitions from `refs.c` in https://github.com/git/git and `git help
check-ref-format` pages.

This change also introduces a test for ref name validity checking, which
compares the result from Nix with the result of `git check-ref-format --branch`.
2020-05-30 12:29:35 +02:00
Eelco Dolstra 89e0b3e2d6 Move substitution into Input::fetch()
Closes #3520.
2020-05-30 01:16:53 +02:00
Eelco Dolstra 0e7f77a59a Check revCount / lastModified input attributes if specified 2020-05-30 00:59:13 +02:00
Eelco Dolstra 950b46821f Remove TreeInfo
The attributes previously stored in TreeInfo (narHash, revCount,
lastModified) are now stored in Input. This makes it less arbitrary
what attributes are stored where.

As a result, the lock file format has changed. An entry like

    "info": {
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },
    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github"
    },

is now stored as

    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github",
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },

The 'Input' class is now a dumb set of attributes. All the fetcher
implementations subclass InputScheme, not Input. This simplifies the
API.

Also, fix substitution of flake inputs. This was broken since lazy
flake fetching started using fetchTree internally.
2020-05-30 00:44:11 +02:00
John Ericson fac0c2d54a Remove addToStore variant as requested by FIXME
The idea is it's always more flexible to consumer a `Source` than a
plain string, and it might even reduce memory consumption.

I also looked at `addToStoreFromDump` with its `// FIXME: remove?`, but
the worked needed for that is far more up for interpretation, so I
punted for now.
2020-05-29 17:02:32 -04:00