Commit graph

8861 commits

Author SHA1 Message Date
Matthew Bauer 05ac4db39a Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-07-30 12:38:24 -05:00
Eelco Dolstra a785b3eddf Fix build 2020-07-30 15:27:28 +02:00
Eelco Dolstra 3f6e88a552 unsigned long long -> uint64_t 2020-07-30 13:34:04 +02:00
Eelco Dolstra ebee2b7852 receiveContents(): unsigned int -> size_t 2020-07-30 13:00:30 +02:00
Eelco Dolstra 3f3740d676
Merge pull request #3879 from illiusdope/nix-build-shell-flags
Prevent nix-build from accepting nix-shell flags
2020-07-30 09:19:28 +02:00
Mat Marini d9e23bfee2 Prevent nix-build from accepting --run/--command 2020-07-29 20:53:02 -04:00
Mat Marini e4f6f8da77 Prevent nix-build from accepting --packages 2020-07-29 20:50:06 -04:00
Eelco Dolstra f63839bfa4 Cleanup 2020-07-29 18:04:51 +02:00
Eelco Dolstra 39311e7009
Merge pull request #3549 from Ma27/fetchgit-hash
Merge legacy `fetchGit`-builtin with the generic `fetchTree`-function
2020-07-29 11:33:27 +02:00
Eelco Dolstra 7c097275c4
Merge pull request #3877 from matthewbauer/develop-continuous-regex
Set regex_constants::match_continuous for quicker search in nix develop
2020-07-29 10:07:52 +02:00
Eelco Dolstra 428e716193
Merge pull request #3876 from NixOS/nix-copy-latency
Fix RemoteStore::addToStore() latency
2020-07-29 10:04:26 +02:00
Matthew Bauer fa8515d7ec Set regex_constants::match_continuous for quicker search
match_continuous limits the search to the current start position,
instead of searching the entire file.

On libc++, this improves performance dramatically:

$ time /nix/store/70ai68dfm6xbzwn26j5n4li9di52ylia-nix-3.0pre20200728_c159f48/bin/nix print-dev-env >/dev/null
/nix/store/70ai68dfm6xbzwn26j5n4li9di52ylia-nix-3.0pre20200728_c159f48/bin/ni  2.39s user 0.19s system 64% cpu 4.032 total
$ time /nix/store/cwjfxxlp83zln4mfyy1d2dbsx7f6s962-nix-3.0pre20200728_dirty/bin/nix print-dev-env >/dev/null
/nix/store/cwjfxxlp83zln4mfyy1d2dbsx7f6s962-nix-3.0pre20200728_dirty/bin/nix   0.09s user 0.05s system 65% cpu 0.204 total

Fixes #3874
2020-07-29 00:57:20 -05:00
Eelco Dolstra 4c0077a07d Fix RemoteStore::addToStore() latency
Since 6185d25e52, this was very
latency-bound since it required a round-trip for every 32 KiB. So for
example copying a 514 MiB closure over a virtual ethernet device with
a articial delay of just 1 ms took 343s. Now it takes 2.7s.

Fixes #3372.
2020-07-29 00:48:39 +02:00
Eelco Dolstra c159f48a39 Cleanup 2020-07-29 00:24:55 +02:00
Maximilian Bosch e4940e90f3
Restore backwards-compat for current builtins.fetchGit
If a repo is dirty, it used to return a `rev` object with an "empty"
sha1 (0000000000000000000000000000000000000000). Please note that this
only applies for `builtins.fetchGit` and *not* for `builtins.fetchTree{
type = "git"; }`.
2020-07-28 22:46:39 +02:00
Eelco Dolstra b8eea7e81a Remove putBytes
istream->tellg() returns -1 so we can't get the number of bytes
written.

Fixes 'uploaded 's3://nix-cache/nar/00819r9lp5kajr6baxfw5dhhc0cx8ndxaz43qmd2f0gn1hk1ynlp.nar.xz' (-1 bytes) in 11620 ms' messages.
2020-07-28 13:27:56 +02:00
Eelco Dolstra 189e6f5e1d Bump version to 3.0
Since there are some incompatible changes, it's better to bump the
major version number.
2020-07-28 13:18:27 +02:00
Maximilian Bosch f742438465
Merge legacy fetchGit-builtin with the generic fetchTree-function
The original idea was to implement a git-fetcher in Nix's core that
supports content hashes[1]. In #3549[2] it has been suggested to
actually use `fetchTree` for this since it's a fairly generic wrapper
over the new fetcher-API[3] and already supports content-hashes.

This patch implements a new git-fetcher based on `fetchTree` by
incorporating the following changes:

* Removed the original `fetchGit`-implementation and replaced it with an
  alias on the `fetchTree` implementation.

* Ensured that the `git`-fetcher from `libfetchers` always computes a
  content-hash and returns an "empty" revision on dirty trees (the
  latter one is needed to retain backwards-compatibility).

* The hash-mismatch error in the fetcher-API exits with code 102 as it
  usually happens whenever a hash-mismatch is detected by Nix.

* Removed the `flakes`-feature-flag: I didn't see a reason why this API
  is so tightly coupled to the flakes-API and at least `fetchGit` should
  remain usable without any feature-flags.

* It's only possible to specify a `narHash` for a `git`-tree if either a
  `ref` or a `rev` is given[4].

* It's now possible to specify an URL without a protocol. If it's missing,
  `file://` is automatically added as it was the case in the original
  `fetchGit`-implementation.

[1] https://github.com/NixOS/nix/pull/3216
[2] https://github.com/NixOS/nix/pull/3549#issuecomment-625194383
[3] https://github.com/NixOS/nix/pull/3459
[4] https://github.com/NixOS/nix/pull/3216#issuecomment-553956703
2020-07-28 00:44:39 +02:00
Eelco Dolstra 86805a2c0a
Merge pull request #3738 from obsidiansystems/hash-always-has-type
Hash always has a valid type
2020-07-27 18:40:57 +02:00
John Ericson 43f2bd8dc5 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-07-27 16:13:57 +00:00
Eelco Dolstra a5f7d310dd
Merge pull request #3795 from obsidiansystems/optional-derivation-output-storepath
Only store hash in DerivationOutput for fixed output derivations
2020-07-27 16:48:41 +02:00
John Ericson d5bb67cfa4 Merge remote-tracking branch 'upstream/master' into optional-derivation-output-storepath 2020-07-27 14:29:32 +00:00
John Ericson 699fc89b39 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-07-27 14:22:45 +00:00
Eelco Dolstra d7c0f094cb
Merge pull request #3860 from matthewbauer/precompile-headers-cross
Allow PRECOMPILE_HEADERS in cross-compilation
2020-07-26 13:09:36 +02:00
Matthew Bauer 72f8771094 Allow PRECOMPILE_HEADERS in cross-compilation
In cross, CXX will look like aarch64-unknown-linux-gnu-g++. We could
run some command to check what kind of compiler it is, but for now we
can just check if g++ is anywhere in the string. I couldn’t find any
"ends with" for makefile, so it can be anywhere in CXX.
2020-07-25 18:02:42 -05:00
Eelco Dolstra 2292814049
createUnixDomainSocket(): Fix off-by-one error in copying the socket path
Reported by Kane York.
2020-07-24 11:19:17 +02:00
Eelco Dolstra 26fcab53e0
Remove unused file 2020-07-23 18:25:09 +02:00
Eelco Dolstra a7b8f79938
Merge pull request #3855 from obsidiansystems/delete-find-output
Get rid of `basicDerivation::findOutput`
2020-07-23 16:56:58 +02:00
John Ericson 2274f63453 Get rid of basicDerivation::findOutput
It's a tiny function which is:

 - hardly worth abstrating over, and also only used once.

 - doesn't work once we get CA drvs

I rewrote the one callsite to be forwards compatable with CA
derivations, and also potentially more performant: instead of reading in
the derivation it can ust consult the SQLite DB in the common case.
2020-07-23 14:39:02 +00:00
Eelco Dolstra 4bfba1305e
Merge pull request #3854 from B4dM4n/registry-pin-write
Save changes made by "nix registry pin" to user registry
2020-07-23 16:13:05 +02:00
Fabian Möller b9ead08ca8
Save changes made by "nix registry pin" to user registry 2020-07-23 14:21:27 +02:00
Eelco Dolstra d58ab591d4
Merge pull request #3850 from obsidiansystems/references-scan-cleanup
Separate concerns in `scanForReferences` with TeeSink
2020-07-23 12:51:01 +02:00
John Ericson c56356bacc Separate concerns in scanForReferences with TeeSink
This also will make it easier to use a `HashModuloSink` instead for CA
derivations.
2020-07-22 22:13:44 +00:00
Eelco Dolstra 090960b725
Merge pull request #3848 from knedlsepp/patch-1
README: Fix link to hacking guide
2020-07-22 14:31:40 +02:00
Josef Kemetmüller ae9e9753ce
README: Fix link to hacking guide
The link was previously interpreted as if it were relative to the current file.
2020-07-22 13:45:15 +02:00
Eelco Dolstra ff314f186e
Merge pull request #3736 from obsidiansystems/allow-relative-paths-in-store-option
Allow relative paths in --store option
2020-07-21 19:43:07 +02:00
Eelco Dolstra 0951330680
Merge pull request #3714 from obsidiansystems/add-body-to-network-errors
Add response body to network errors
2020-07-21 18:09:42 +02:00
John Ericson 6cce32c8e8 Change logic for deciding what is a relative path for the local store
The was Eelco's prefered logic, and it looks good to me!
2020-07-21 15:39:47 +00:00
Eelco Dolstra 1346da4669
Merge pull request #3844 from obsidiansystems/update-chunksize-to-suggested-value
Update chunkSize to the suggested value
2020-07-21 16:39:31 +02:00
Carlo Nucera 922a845ffc Update chunkSize to the suggested value
This was a suggested course of action in a review in one of our earlier
commits, https://github.com/NixOS/nix/pull/3801#discussion_r457557079
2020-07-21 10:24:19 -04:00
John Ericson 54e507a7aa Merge remote-tracking branch 'upstream/master' into add-body-to-network-errors 2020-07-21 14:17:59 +00:00
Eelco Dolstra 0835447eaa
Merge pull request #3801 from obsidiansystems/from-dump-stream
Constant space `addToStoreFromDump` and deduplicate code
2020-07-21 15:11:52 +02:00
Eelco Dolstra 7e91cdd9ee
Merge pull request #3840 from Mic92/docs
README: improve development docs
2020-07-21 13:20:13 +02:00
Eelco Dolstra 51ee506693
Merge pull request #3842 from obsidiansystems/fix-and-document-addToStoreSlow
Correct bug, thoroughly document addToStoreSlow
2020-07-21 10:01:34 +02:00
John Ericson 5a37413796 Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into from-dump-stream 2020-07-21 01:20:14 +00:00
Carlo Nucera 9aae179f34 Correct bug, thoroughly document addToStoreSlow 2020-07-20 20:18:12 -04:00
Jörg Thalheim 6633605341
Update doc/manual/hacking.xml
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2020-07-20 22:30:39 +01:00
Carlo Nucera 0ca9744694 Use heuristics to decide when to show the response
Due to https://github.com/NixOS/nix/issues/3841 we don't know how print
different messages for different verbosity levels.
2020-07-20 15:57:58 -04:00
John Ericson eab945f591 Merge remote-tracking branch 'upstream/master' into from-dump-stream 2020-07-20 17:29:23 +00:00
Eelco Dolstra 1c5f8bbfb5
Merge pull request #3822 from obsidiansystems/dump-thrice-fixme
Optimize `addToStoreSlow` and remove `TeeParseSink`
2020-07-20 18:55:05 +02:00