Commit graph

8859 commits

Author SHA1 Message Date
John Ericson 8173e7bfef Fix localhost::addToStore(...Path...)
We were calculating the nar hash wrong when the file ingestion method
was flat. I don't think there's anything we can do in that case but dump
the file again, so that's what I do.

As an optomization, we again could reuse the original dump for just the
recursive and non-sha256 case, but I rather do that after this fix, and
after my other PRs which deduplicate this code.
2020-07-14 21:12:11 +00:00
John Ericson 592851fb67 LocalStore::addToStoreFromDump copy in chunks
Rather than copying byte-by-byte, we let the coroutine know how much
data we would like it to send back to us.
2020-07-14 13:56:36 +00:00
John Ericson 9de96ef7d4 Dedup LocalStore::addToStore*
The downsides is that the coroutine has byte-by-byte loop transfer. Will
fix that next.
2020-07-14 13:56:36 +00:00
John Ericson c86fc3a965 Crudely make addToStoreFromDump take Source not string
I just as little beyond the type as possible, so the implementation
changes this enables can be reviewed separately.
2020-07-14 13:56:36 +00:00
John Ericson 9ec10046e0 Narrow scope of temporary value 2020-07-14 13:54:29 +00:00
Eelco Dolstra 298ff6af8f
Merge pull request #3809 from Ma27/gitlab-refs
Fix gitlab-fetcher to obtain tags and branches
2020-07-14 15:22:52 +02:00
Eelco Dolstra da3aea291d EvalCache: Ignore SQLite errors
Fixes #3794.
2020-07-14 15:17:38 +02:00
Eelco Dolstra 832e111494 Merge remote-tracking branch 'origin/master' into flakes 2020-07-14 13:56:18 +02:00
Eelco Dolstra 926c3a6664 Doh 2020-07-14 11:55:54 +02:00
Eelco Dolstra 43b8e96d30 Fix 'nix verify --all' on a binary cache (cached case) 2020-07-13 20:17:00 +02:00
Eelco Dolstra 9502c0e2eb nix verify: Show correct path when using --all on a binary cache 2020-07-13 20:12:44 +02:00
Eelco Dolstra 7c2fef0a81 Make 'nix copy' to s3:// binary caches run in constant memory 2020-07-13 20:07:19 +02:00
Maximilian Bosch cf9f33995b
Fix gitlab-fetcher to obtain tags and branches
Until now, the `gitlab`-fetcher determined the source's rev by checking
the latest commit of the given `ref` using the
`/repository/branches`-API.

This breaks however when trying to fetch a gitlab-repo by its tag:

```
$ nix repl
nix-repl> builtins.fetchTree gitlab:Ma27/nvim.nix/0.2.0
error: --- Error ------------------------------------------------------------------------------------- nix
unable to download 'https://gitlab.com/api/v4/projects/Ma27%2Fnvim.nix/repository/branches/0.2.0': HTTP error 404 ('')
```

When using the `/commits?ref_name`-endpoint[1] you can pass any kind of
valid ref to the `gitlab`-fetcher.

Please note that this fetches the only first 20 commits on a ref,
unfortunately there's currently no endpoint which only retrieves the
latest commit of any kind of `ref`.

[1] https://docs.gitlab.com/ee/api/commits.html#list-repository-commits
2020-07-13 19:22:59 +02:00
Eelco Dolstra 493961b689 Remove istringstream_nocopy 2020-07-13 18:31:19 +02:00
Eelco Dolstra 545bb2ed03 Remove 'accessor' from addToStore()
This is only used by hydra-queue-runner and it's better to implement
it there.
2020-07-13 18:31:19 +02:00
Eelco Dolstra 0a9da00a10 NarAccessor: Run in constant memory 2020-07-13 17:30:42 +02:00
Eelco Dolstra fc84c358d9 Make 'nix copy' to file:// binary caches run in constant memory 2020-07-13 16:28:45 +02:00
Eelco Dolstra 400f1a9b59 Store::pathInfoToJSON(): Use consistent format for downloadHash 2020-07-13 16:26:09 +02:00
Eelco Dolstra c0dd05131e toStorePath(): Return a StorePath and the suffix 2020-07-13 16:25:48 +02:00
Eelco Dolstra 143a5f32ed Add a test for local NAR caching 2020-07-13 16:25:48 +02:00
Eelco Dolstra 1d01ae816b Fix 'nix verify --all' on a binary cache and add a test 2020-07-13 14:35:01 +02:00
Eelco Dolstra 2900a441f5 Add a test for DWARF debug info index generation 2020-07-13 13:38:01 +02:00
Eelco Dolstra 41bdf429ec Add a test for NAR listing generation 2020-07-13 13:32:33 +02:00
Eelco Dolstra d7026cc571
Merge pull request #3805 from Ma27/ansi-color-fix
Fix ANSI color constants
2020-07-13 10:10:22 +02:00
John Ericson 4415765385 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-07-13 03:01:44 +00:00
John Ericson 230c9b4329 Change types to prepare the way for CA derivations
We've added the variant to `DerivationOutput` to support them, but made
`DerivationOutput::path` partial to avoid actually implementing them.

With this chage, we can all collaborate on "just" removing
`DerivationOutput::path` calls to implement CA derivations.
2020-07-12 23:53:33 +00:00
John Ericson fedfc913ad Use more std::visit to prepare for new variant
N.B. not using `std::visit` for fetchurl because there is no attempt to
handle all the cases (e.g. no `else`) and lambda complicates early
return.
2020-07-12 23:51:40 +00:00
John Ericson ffc18583b1 Move C++17 "pattern matching" boilerplat to utils.hh 2020-07-12 22:15:14 +00:00
John Ericson 886c91dfcc Try to fix perl bindings 2020-07-12 18:26:10 +00:00
John Ericson 503b425690 DerivationOutputExtensional -> DerivationOutputInputAddressed
Thanks @regnat for the great name.
2020-07-12 15:56:20 +00:00
John Ericson a8d4707107 Undo erroneous indentation change 2020-07-12 15:54:12 +00:00
John Ericson 18152406ce String .drv suffix to create derivation name 2020-07-12 15:40:14 +00:00
John Ericson 5d0b75e5b6 std::string_view for new derivation name parameters 2020-07-12 15:02:36 +00:00
Maximilian Bosch 64f03635d7
Fix ANSI color constants
The `m` acts as termination-symbol when declaring graphics. Because
of this, the `;1m` doesn't have any effect and is directly printed to
the console:

```
$ nix repl
> builtins.fetchGit { /* ... */ }
{ outPath = "/nix/store/s0f0iz4a41cxx2h055lmh6p2d5k5bc6r-source"; rev = "e73e45b723a9a6eecb98bd5f3df395d9ab3633b6"; revCount = ;1m428; shortRev = "e73e45b"; submodules = ;1mfalse; }
```

Introduced by 6403508f5a.
2020-07-12 16:52:20 +02:00
John Ericson 13ec627e0a Set derivation name in dervationStrict 2020-07-12 03:03:12 +00:00
John Ericson 1c9bec226f Don't improperly assume path is store path 2020-07-12 02:38:03 +00:00
John Ericson abea26a968 BasicDerivation::findOutput cannot return reference anymore 2020-07-12 01:57:06 +00:00
John Ericson 442d43178b Merge remote-tracking branch 'upstream/master' into optional-derivation-output-storepath 2020-07-11 02:27:45 +00:00
Eelco Dolstra 8efa23bb99 Avoid a redundant hash 2020-07-10 15:56:24 +02:00
Eelco Dolstra 5dff49f661 Factor out commonality between nix-prefetch-url and nix-store --add-fixed 2020-07-10 13:21:37 +02:00
Eelco Dolstra 7f1a86d57c nix-store --add-fixed: Run in constant memory 2020-07-10 12:51:56 +02:00
Eelco Dolstra 06e3dd9005 nix-prefetch-url: Run in constant memory when using RemoteStore
Fixes #3684.
2020-07-10 11:22:48 +02:00
Eelco Dolstra 062a584f12 .dir-locals.el: Set c-block-comment-prefix 2020-07-10 11:21:06 +02:00
Matthew Bauer a7884970c5 Fix DerivationOutputExtensional name 2020-07-09 11:37:18 -04:00
Eelco Dolstra a2c27022e9 LocalStore::addToStore(srcPath): Handle the flat case
This helps nix-prefetch-url when using a local store.
2020-07-09 15:54:32 +02:00
Eelco Dolstra 2dd8443e30
Merge pull request #3797 from nix-macos-perf-test/macos-perf-test
add temp CI job to test syspolicy impact
2020-07-09 11:59:22 +02:00
Travis A. Everett cfe6ea746c
add temp CI job to test syspolicy impact
Starting in Catalina, macOS runs a syspolicyd "assessment" that hits the network for each binary/script executable. It does cache these results, but Nix tends to introduce many "new" executables per build. (You can read more about this at https://github.com/NixOS/nix/issues/3789).

This PR adds a temporary, redundant macOS job with these assessments disabled. I'm hoping you can adopt it for a few weeks to help me collect more data on how this affects real projects.
2020-07-08 20:10:22 -05:00
Matthew Bauer 06a4e15478 Fix build.cc on linux 2020-07-08 19:27:51 -04:00
Matthew Bauer 8e0d0689be Only store hash of fixed derivation output
we don’t need a full storepath for a fixedoutput derivation. So just
putting the ingestion method + the hash is sufficient.
2020-07-08 19:11:39 -04:00
Eelco Dolstra b981e5aacf Cleanup 2020-07-08 22:07:21 +02:00