Commit graph

241 commits

Author SHA1 Message Date
Eelco Dolstra 3f6e88a552 unsigned long long -> uint64_t 2020-07-30 13:34:04 +02:00
John Ericson 699fc89b39 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-07-27 14:22:45 +00:00
Carlo Nucera d090562348 Merge branch 'master' of github.com:NixOS/nix into hash-always-has-type 2020-07-15 17:21:01 -04:00
John Ericson c86fc3a965 Crudely make addToStoreFromDump take Source not string
I just as little beyond the type as possible, so the implementation
changes this enables can be reviewed separately.
2020-07-14 13:56:36 +00:00
Eelco Dolstra 545bb2ed03 Remove 'accessor' from addToStore()
This is only used by hydra-queue-runner and it's better to implement
it there.
2020-07-13 18:31:19 +02:00
Eelco Dolstra c0dd05131e toStorePath(): Return a StorePath and the suffix 2020-07-13 16:25:48 +02:00
Eelco Dolstra 1d01ae816b Fix 'nix verify --all' on a binary cache and add a test 2020-07-13 14:35:01 +02:00
John Ericson 4415765385 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-07-13 03:01:44 +00:00
Eelco Dolstra 5dff49f661 Factor out commonality between nix-prefetch-url and nix-store --add-fixed 2020-07-10 13:21:37 +02:00
John Ericson dbffd309fe Merge branch 'master' of github.com:NixOS/nix into hash-always-has-type 2020-07-03 14:11:38 +00:00
regnat d38f860c3e Add a way to get all the outputs of a derivation with their label
Generalize `queryDerivationOutputNames` and `queryDerivationOutputs` by
adding a `queryDerivationOutputMap` that returns the map
`outputName=>outputPath`

(not that this is not equivalent to merging the results of
`queryDerivationOutputs` and `queryDerivationOutputNames` as sets don't
preserve the order, so we would end up with an incorrect mapping).

squash! Add a way to get all the outputs of a derivation with their label

Rename StorePathMap to OutputPathMap
2020-06-24 20:38:40 +02:00
John Ericson 98e5d1af03 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-06-23 17:03:37 +00:00
John Ericson 20799a5151 WIP: Make Hash always store a valid hash type 2020-06-19 19:32:29 +00:00
John Ericson 3f8dcfe3fd Merge branch 'validPathInfo-temp' into validPathInfo-ca-proper-datatype 2020-06-18 23:01:58 +00:00
John Ericson 517f5980e2 Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output 2020-06-17 04:58:43 +00:00
Eelco Dolstra 29542865ce Remove StorePath::clone() and related functions 2020-06-16 22:20:18 +02:00
Eelco Dolstra 759947bf72 StorePath: Rewrite in C++
On nix-env -qa -f '<nixpkgs>', this reduces maximum RSS by 20970 KiB
and runtime by 0.8%. This is mostly because we're not parsing the hash
part as a hash anymore (just validating that it consists of base-32
characters).

Also, replace storePathToHash() by StorePath::hashPart().
2020-06-16 14:28:41 +02:00
Eelco Dolstra 9f736dd89d Add Store::readDerivation() convenience function 2020-06-12 13:04:52 +02:00
Eelco Dolstra 045b07200c Remove Store::queryDerivationOutputNames()
This function was used in only one place, where it could easily be
replaced by readDerivation() since it's not
performance-critical. (This function appears to have been modelled
after queryDerivationOutputs(), which exists only to make the garbage
collector faster.)
2020-06-12 12:46:33 +02:00
John Ericson e5cc1ebc5d Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output 2020-06-04 21:04:35 +00:00
Carlo Nucera fd2eb41e64 Move file-hash to content-address 2020-06-02 15:44:58 -04:00
John Ericson 1b6461f671 Merge remote-tracking branch 'upstream/master' into validPathInfo-ca-proper-datatype 2020-06-02 14:31:18 +00:00
John Ericson 754c910953 WIP more progress 2020-06-01 19:26:40 -04:00
Carlo Nucera da39092a39 WIP 2020-06-01 18:53:31 -04:00
Carlo Nucera 0e9438b6d3 Create new file-hash files 2020-06-01 17:32:40 -04:00
Carlo Nucera 0cb67ecbd3 Merge branch 'derivation-header-include-order' of github.com:Ericson2314/nix into validPathInfo-ca-proper-datatype 2020-06-01 17:13:11 -04:00
Carlo Nucera f4b89e11a4 Merge branch 'no-stringly-typed-derivation-output' of github.com:Ericson2314/nix into validPathInfo-ca-proper-datatype 2020-06-01 17:12:50 -04:00
John Ericson fac0c2d54a Remove addToStore variant as requested by FIXME
The idea is it's always more flexible to consumer a `Source` than a
plain string, and it might even reduce memory consumption.

I also looked at `addToStoreFromDump` with its `// FIXME: remove?`, but
the worked needed for that is far more up for interpretation, so I
punted for now.
2020-05-29 17:02:32 -04:00
Carlo Nucera 4f597fb901 Merge branch 'master' of github.com:NixOS/nix into enum-class 2020-05-28 10:58:22 -04:00
Carlo Nucera d49e65ba9d Merge remote-tracking branch 'john-ericson/enum-FileIngestionMethod' into no-stringly-typed-derivation-output 2020-05-26 12:30:48 -04:00
Carlo Nucera 6d73c10041 Merge remote-tracking branch 'origin/master' into enum-FileIngestionMethod 2020-05-26 11:14:08 -04:00
Eelco Dolstra 462421d345 Backport libfetchers from the flakes branch
This provides a pluggable mechanism for defining new fetchers. It adds
a builtin function 'fetchTree' that generalizes existing fetchers like
'fetchGit', 'fetchMercurial' and 'fetchTarball'. 'fetchTree' takes a
set of attributes, e.g.

  fetchTree {
    type = "git";
    url = "https://example.org/repo.git";
    ref = "some-branch";
    rev = "abcdef...";
  }

The existing fetchers are just wrappers around this. Note that the
input attributes to fetchTree are the same as flake input
specifications and flake lock file entries.

All fetchers share a common cache stored in
~/.cache/nix/fetcher-cache-v1.sqlite. This replaces the ad hoc caching
mechanisms in fetchGit and download.cc (e.g. ~/.cache/nix/{tarballs,git-revs*}).

This also adds support for Git worktrees (c169ea5904).
2020-04-07 09:03:14 +02:00
John Ericson 832bd534dc Store parsed hashes in DerivationOutput
It's best to detect invalid data as soon as possible, with data types
that make storing it impossible.
2020-03-30 11:33:35 -04:00
John Ericson f5494d9442 Merge remote-tracking branch 'me/enum-FileIngestionMethod' into HEAD 2020-03-30 11:08:13 -04:00
John Ericson 225e62a56a Replace some bool recursive with a new FileIngestionMethod enum 2020-03-29 15:16:20 -04:00
John Ericson 87b32bab05 Use enum struct and drop prefixes
This does a few enums; the rest will be gotten in subsequent commits.
2020-03-29 11:23:15 -04:00
John Ericson e433d4af4c Extend Rust FFI
Do idiomatic C++ copy and move constructors for a few things, so
wrapping structs' defaults can work.
2020-03-25 16:12:14 -04:00
John Ericson bcde5456cc Flip dependency so store-api.hh includes derivations.hh
I think it makes more sense to define the data model (derivations),
before the operations (store api).
2020-03-24 20:39:45 +00:00
Eelco Dolstra 0a10854f85 Misc changes from the flakes branch 2020-03-24 14:34:47 +01:00
Eelco Dolstra 777e21e596 nix path-info --json: Print hash in SRI format
(cherry picked from commit 442e665d6d)
2020-03-24 14:26:23 +01:00
Robert Hensing 3f55f8a8fb pathInfoCache: Respect disk cache TTLs #3398 2020-03-12 10:30:28 +01:00
Eelco Dolstra d37dc71e3c
nix-build: Fix !<output> handling
This was broken by 22a754c091.

https://hydra.nixos.org/eval/1573669
2020-03-04 13:56:17 +01:00
Eelco Dolstra 22a754c091
Fix GC failures on bad store path names
It failed on names like '/nix/store/9ip48nkc9rfy0a4yaw98lp6gipqlib1a-'.
2020-02-28 18:07:10 +01:00
Eelco Dolstra d82b78bf51
Fix segfault in gcc on i686-linux
src/libstore/ssh-store.cc: In constructor 'nix::SSHStore::SSHStore(const string&, const Params&)':
  src/libstore/ssh-store.cc:31:21: internal compiler error: Segmentation fault
               compress)
                       ^
  Please submit a full bug report,
  with preprocessed source if appropriate.

https://hydra.nixos.org/build/111545609
2020-02-07 13:01:48 +01:00
Eelco Dolstra f8abbdd456 Add priority setting to stores
This allows overriding the priority of substituters, e.g.

  $ nix-store --store ~/my-nix/ -r /nix/store/df3m4da96d84ljzxx4mygfshm1p0r2n3-geeqie-1.4 \
    --substituters 'http://cache.nixos.org?priority=100 daemon?priority=10'

Fixes #3264.
2019-12-17 17:17:53 +01:00
Eelco Dolstra 54bf5ba422 nix-store -r: Handle symlinks to store paths
Fixes #3270.
2019-12-16 19:11:47 +01:00
Eelco Dolstra bbe97dff8b Make the Store API more type-safe
Most functions now take a StorePath argument rather than a Path (which
is just an alias for std::string). The StorePath constructor ensures
that the path is syntactically correct (i.e. it looks like
<store-dir>/<base32-hash>-<name>). Similarly, functions like
buildPaths() now take a StorePathWithOutputs, rather than abusing Path
by adding a '!<outputs>' suffix.

Note that the StorePath type is implemented in Rust. This involves
some hackery to allow Rust values to be used directly in C++, via a
helper type whose destructor calls the Rust type's drop()
function. The main issue is the dynamic nature of C++ move semantics:
after we have moved a Rust value, we should not call the drop function
on the original value. So when we move a value, we set the original
value to bitwise zero, and the destructor only calls drop() if the
value is not bitwise zero. This should be sufficient for most types.

Also lots of minor cleanups to the C++ API to make it more modern
(e.g. using std::optional and std::string_view in some places).
2019-12-10 22:06:05 +01:00
Brian Wignall 8737980e75 Fix typos 2019-11-30 19:04:14 -05:00
Eelco Dolstra 5ee23c35b9
Merge pull request #3219 from Ericson2314/semicolons
Fix extra semicolons warnings
2019-11-11 12:13:51 +01:00
John Ericson 8669db1dcc Clean up semicolon and comma
Thanks @bhipple for catching!
2019-11-10 16:21:59 -05:00