Commit graph

8821 commits

Author SHA1 Message Date
Eelco Dolstra 39311e7009
Merge pull request #3549 from Ma27/fetchgit-hash
Merge legacy `fetchGit`-builtin with the generic `fetchTree`-function
2020-07-29 11:33:27 +02:00
Eelco Dolstra 7c097275c4
Merge pull request #3877 from matthewbauer/develop-continuous-regex
Set regex_constants::match_continuous for quicker search in nix develop
2020-07-29 10:07:52 +02:00
Eelco Dolstra 428e716193
Merge pull request #3876 from NixOS/nix-copy-latency
Fix RemoteStore::addToStore() latency
2020-07-29 10:04:26 +02:00
Matthew Bauer fa8515d7ec Set regex_constants::match_continuous for quicker search
match_continuous limits the search to the current start position,
instead of searching the entire file.

On libc++, this improves performance dramatically:

$ time /nix/store/70ai68dfm6xbzwn26j5n4li9di52ylia-nix-3.0pre20200728_c159f48/bin/nix print-dev-env >/dev/null
/nix/store/70ai68dfm6xbzwn26j5n4li9di52ylia-nix-3.0pre20200728_c159f48/bin/ni  2.39s user 0.19s system 64% cpu 4.032 total
$ time /nix/store/cwjfxxlp83zln4mfyy1d2dbsx7f6s962-nix-3.0pre20200728_dirty/bin/nix print-dev-env >/dev/null
/nix/store/cwjfxxlp83zln4mfyy1d2dbsx7f6s962-nix-3.0pre20200728_dirty/bin/nix   0.09s user 0.05s system 65% cpu 0.204 total

Fixes #3874
2020-07-29 00:57:20 -05:00
Eelco Dolstra 4c0077a07d Fix RemoteStore::addToStore() latency
Since 6185d25e52, this was very
latency-bound since it required a round-trip for every 32 KiB. So for
example copying a 514 MiB closure over a virtual ethernet device with
a articial delay of just 1 ms took 343s. Now it takes 2.7s.

Fixes #3372.
2020-07-29 00:48:39 +02:00
Eelco Dolstra c159f48a39 Cleanup 2020-07-29 00:24:55 +02:00
Maximilian Bosch e4940e90f3
Restore backwards-compat for current builtins.fetchGit
If a repo is dirty, it used to return a `rev` object with an "empty"
sha1 (0000000000000000000000000000000000000000). Please note that this
only applies for `builtins.fetchGit` and *not* for `builtins.fetchTree{
type = "git"; }`.
2020-07-28 22:46:39 +02:00
Eelco Dolstra b8eea7e81a Remove putBytes
istream->tellg() returns -1 so we can't get the number of bytes
written.

Fixes 'uploaded 's3://nix-cache/nar/00819r9lp5kajr6baxfw5dhhc0cx8ndxaz43qmd2f0gn1hk1ynlp.nar.xz' (-1 bytes) in 11620 ms' messages.
2020-07-28 13:27:56 +02:00
Eelco Dolstra 189e6f5e1d Bump version to 3.0
Since there are some incompatible changes, it's better to bump the
major version number.
2020-07-28 13:18:27 +02:00
Maximilian Bosch f742438465
Merge legacy fetchGit-builtin with the generic fetchTree-function
The original idea was to implement a git-fetcher in Nix's core that
supports content hashes[1]. In #3549[2] it has been suggested to
actually use `fetchTree` for this since it's a fairly generic wrapper
over the new fetcher-API[3] and already supports content-hashes.

This patch implements a new git-fetcher based on `fetchTree` by
incorporating the following changes:

* Removed the original `fetchGit`-implementation and replaced it with an
  alias on the `fetchTree` implementation.

* Ensured that the `git`-fetcher from `libfetchers` always computes a
  content-hash and returns an "empty" revision on dirty trees (the
  latter one is needed to retain backwards-compatibility).

* The hash-mismatch error in the fetcher-API exits with code 102 as it
  usually happens whenever a hash-mismatch is detected by Nix.

* Removed the `flakes`-feature-flag: I didn't see a reason why this API
  is so tightly coupled to the flakes-API and at least `fetchGit` should
  remain usable without any feature-flags.

* It's only possible to specify a `narHash` for a `git`-tree if either a
  `ref` or a `rev` is given[4].

* It's now possible to specify an URL without a protocol. If it's missing,
  `file://` is automatically added as it was the case in the original
  `fetchGit`-implementation.

[1] https://github.com/NixOS/nix/pull/3216
[2] https://github.com/NixOS/nix/pull/3549#issuecomment-625194383
[3] https://github.com/NixOS/nix/pull/3459
[4] https://github.com/NixOS/nix/pull/3216#issuecomment-553956703
2020-07-28 00:44:39 +02:00
Eelco Dolstra 86805a2c0a
Merge pull request #3738 from obsidiansystems/hash-always-has-type
Hash always has a valid type
2020-07-27 18:40:57 +02:00
John Ericson 43f2bd8dc5 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-07-27 16:13:57 +00:00
Eelco Dolstra a5f7d310dd
Merge pull request #3795 from obsidiansystems/optional-derivation-output-storepath
Only store hash in DerivationOutput for fixed output derivations
2020-07-27 16:48:41 +02:00
John Ericson d5bb67cfa4 Merge remote-tracking branch 'upstream/master' into optional-derivation-output-storepath 2020-07-27 14:29:32 +00:00
John Ericson 699fc89b39 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-07-27 14:22:45 +00:00
Eelco Dolstra d7c0f094cb
Merge pull request #3860 from matthewbauer/precompile-headers-cross
Allow PRECOMPILE_HEADERS in cross-compilation
2020-07-26 13:09:36 +02:00
Matthew Bauer 72f8771094 Allow PRECOMPILE_HEADERS in cross-compilation
In cross, CXX will look like aarch64-unknown-linux-gnu-g++. We could
run some command to check what kind of compiler it is, but for now we
can just check if g++ is anywhere in the string. I couldn’t find any
"ends with" for makefile, so it can be anywhere in CXX.
2020-07-25 18:02:42 -05:00
Eelco Dolstra 2292814049
createUnixDomainSocket(): Fix off-by-one error in copying the socket path
Reported by Kane York.
2020-07-24 11:19:17 +02:00
Eelco Dolstra 26fcab53e0
Remove unused file 2020-07-23 18:25:09 +02:00
Eelco Dolstra a7b8f79938
Merge pull request #3855 from obsidiansystems/delete-find-output
Get rid of `basicDerivation::findOutput`
2020-07-23 16:56:58 +02:00
John Ericson 2274f63453 Get rid of basicDerivation::findOutput
It's a tiny function which is:

 - hardly worth abstrating over, and also only used once.

 - doesn't work once we get CA drvs

I rewrote the one callsite to be forwards compatable with CA
derivations, and also potentially more performant: instead of reading in
the derivation it can ust consult the SQLite DB in the common case.
2020-07-23 14:39:02 +00:00
Eelco Dolstra 4bfba1305e
Merge pull request #3854 from B4dM4n/registry-pin-write
Save changes made by "nix registry pin" to user registry
2020-07-23 16:13:05 +02:00
Fabian Möller b9ead08ca8
Save changes made by "nix registry pin" to user registry 2020-07-23 14:21:27 +02:00
Eelco Dolstra d58ab591d4
Merge pull request #3850 from obsidiansystems/references-scan-cleanup
Separate concerns in `scanForReferences` with TeeSink
2020-07-23 12:51:01 +02:00
John Ericson c56356bacc Separate concerns in scanForReferences with TeeSink
This also will make it easier to use a `HashModuloSink` instead for CA
derivations.
2020-07-22 22:13:44 +00:00
Eelco Dolstra 090960b725
Merge pull request #3848 from knedlsepp/patch-1
README: Fix link to hacking guide
2020-07-22 14:31:40 +02:00
Josef Kemetmüller ae9e9753ce
README: Fix link to hacking guide
The link was previously interpreted as if it were relative to the current file.
2020-07-22 13:45:15 +02:00
Eelco Dolstra ff314f186e
Merge pull request #3736 from obsidiansystems/allow-relative-paths-in-store-option
Allow relative paths in --store option
2020-07-21 19:43:07 +02:00
Eelco Dolstra 0951330680
Merge pull request #3714 from obsidiansystems/add-body-to-network-errors
Add response body to network errors
2020-07-21 18:09:42 +02:00
John Ericson 6cce32c8e8 Change logic for deciding what is a relative path for the local store
The was Eelco's prefered logic, and it looks good to me!
2020-07-21 15:39:47 +00:00
Eelco Dolstra 1346da4669
Merge pull request #3844 from obsidiansystems/update-chunksize-to-suggested-value
Update chunkSize to the suggested value
2020-07-21 16:39:31 +02:00
Carlo Nucera 922a845ffc Update chunkSize to the suggested value
This was a suggested course of action in a review in one of our earlier
commits, https://github.com/NixOS/nix/pull/3801#discussion_r457557079
2020-07-21 10:24:19 -04:00
John Ericson 54e507a7aa Merge remote-tracking branch 'upstream/master' into add-body-to-network-errors 2020-07-21 14:17:59 +00:00
Eelco Dolstra 0835447eaa
Merge pull request #3801 from obsidiansystems/from-dump-stream
Constant space `addToStoreFromDump` and deduplicate code
2020-07-21 15:11:52 +02:00
Eelco Dolstra 7e91cdd9ee
Merge pull request #3840 from Mic92/docs
README: improve development docs
2020-07-21 13:20:13 +02:00
Eelco Dolstra 51ee506693
Merge pull request #3842 from obsidiansystems/fix-and-document-addToStoreSlow
Correct bug, thoroughly document addToStoreSlow
2020-07-21 10:01:34 +02:00
John Ericson 5a37413796 Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into from-dump-stream 2020-07-21 01:20:14 +00:00
Carlo Nucera 9aae179f34 Correct bug, thoroughly document addToStoreSlow 2020-07-20 20:18:12 -04:00
Jörg Thalheim 6633605341
Update doc/manual/hacking.xml
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2020-07-20 22:30:39 +01:00
Carlo Nucera 0ca9744694 Use heuristics to decide when to show the response
Due to https://github.com/NixOS/nix/issues/3841 we don't know how print
different messages for different verbosity levels.
2020-07-20 15:57:58 -04:00
John Ericson eab945f591 Merge remote-tracking branch 'upstream/master' into from-dump-stream 2020-07-20 17:29:23 +00:00
Eelco Dolstra 1c5f8bbfb5
Merge pull request #3822 from obsidiansystems/dump-thrice-fixme
Optimize `addToStoreSlow` and remove `TeeParseSink`
2020-07-20 18:55:05 +02:00
John Ericson ac2fc7ba1f
Apply suggestions from code review
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2020-07-20 11:29:46 -04:00
Jörg Thalheim f0100f5590
README: improve development docs 2020-07-20 15:46:28 +01:00
Eelco Dolstra a79b6ddaa5
Merge pull request #3834 from ajkovar/source-line-3393
Add newline to profile sourcing line #3393
2020-07-19 14:01:56 +02:00
Alex Kovar 3294b0a4b0 Add newline to profile sourcing line #3393 2020-07-18 10:24:22 -05:00
Eelco Dolstra 31826c60e1
Merge pull request #3832 from Mic92/shell-impurity
fix make's impurity on /bin/sh
2020-07-18 09:39:43 +02:00
Eelco Dolstra 0e9fb4a869
Merge pull request #3828 from BurNiinTRee/restore-signals-before-edit
nix edit: call restoreSignals() before `execvp`-ing the $EDITOR
2020-07-18 09:39:11 +02:00
Jörg Thalheim 5526683ad3
fix make's impurity on /bin/sh
This is important when using tooling like BEAR to generate
compilation database since the used glibc version needs to match
for LD_PRELOAD to work. It might be also beneficial when building
on systems other than NixOS with nix develop since /bin/sh might
be not bash (which is what all nix devs use for testing).
This fix is not perfect because Makefile.config.in itself is
also build with make but strictly better than the status quo.
2020-07-18 08:05:36 +01:00
Carlo Nucera 4178f36a1d Test relative store paths 2020-07-17 15:50:53 -04:00