Commit graph

5027 commits

Author SHA1 Message Date
Eelco Dolstra e5ea01c1a8
Remove flake 'edition' field
Future editions of flakes or the Nix language can be supported by
renaming flake.nix (e.g. flake-v2.nix). This avoids a bootstrap
problem where we don't know which grammar to use to parse
flake*.nix. It also allows a project to support multiple flake
editions, in theory.
2020-04-10 10:24:09 +02:00
Domen Kožar db25a6d7bb
Merge pull request #2689 from tollb/fix/delete_tmp_dir_when_build_check_ok
Delete temporary directory on successful build
2020-04-10 09:47:57 +02:00
Bruce Toll 16a4864759 Delete temporary directory on successful build
With --check and the --keep-failed (-K) flag, the temporary directory
was being retained regardless of whether the build was successful and
reproducible.  This removes the temporary directory, as expected, on
a reproducible check build.

Added tests to verify that temporary build directories are not
retained unnecessarily, particularly when using --check with
--keep-failed.
2020-04-09 16:37:41 -04:00
Philipp Middendorf 04bedda0b6 gc.cc: Ignore hidden files in temproots 2020-04-09 17:05:29 +02:00
Eelco Dolstra 3aaceeb7e2
Fix build 2020-04-09 16:00:43 +02:00
Eelco Dolstra d103c79144
Merge remote-tracking branch 'origin/master' into flakes 2020-04-09 15:54:21 +02:00
Domen Kožar a693a9fa4b
Attach pos to if expression errors 2020-04-09 09:45:15 +02:00
Nikola Knezevic 7867685dcd after flake rebase 2020-04-08 22:26:57 +02:00
Nikola Knezevic f5095594e7 datatransfer.{cc,hh} -> filetransfer.{cc,hh} 2020-04-08 22:26:57 +02:00
Nikola Knezevic c330109bfa DataTransfer -> FileTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic 7848372b0f Add upload method 2020-04-08 22:26:57 +02:00
Nikola Knezevic a0c5931208 actDownload -> actDataTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic c4c1ae0a00 DownloadError -> DataTransferError 2020-04-08 22:26:57 +02:00
Nikola Knezevic 213d124277 DownloadItem -> TransferItem 2020-04-08 22:26:57 +02:00
Nikola Knezevic cd391206e6 {get,make,new}Downloader -> DataTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic 2df2741ec6 enqueueDownload -> enqueueDataTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic 142ed7fe45 DownloadResult -> DataTransferResult 2020-04-08 22:26:57 +02:00
Nikola Knezevic e5cc53beec DownloadSettings -> DataTransferSettings 2020-04-08 22:26:57 +02:00
Nikola Knezevic 741e9012d3 Rename src/lib/download.* to src/lib/datatransfer.* 2020-04-08 22:26:57 +02:00
Nikola Knezevic 65ef57e0cb DownloadRequest -> DataTransferRequest 2020-04-08 22:26:57 +02:00
Eelco Dolstra 5449ff7d8a
Merge pull request #3477 from Ninlives/nix-run-using-env
`nix run` using $SHELL as default command
2020-04-08 19:35:39 +02:00
Ben Burdette 805ffe1bc9 indention 2020-04-08 11:33:46 -06:00
Ben Burdette 8c2bf15c4f format -> fmt 2020-04-08 11:17:02 -06:00
Domen Kožar bf81b31559
build.cc: improve message if home directory exists 2020-04-08 18:27:10 +02:00
Ben Burdette 555baa8fb0 comments 2020-04-08 09:56:10 -06:00
Ben Burdette 54f91923c8 return of NixCode 2020-04-08 09:48:21 -06:00
Eelco Dolstra 96f3c36709
Merge pull request #3478 from edolstra/ignore-failed-data
Downloader: Only write data to the sink on a 200 response
2020-04-08 17:15:53 +02:00
Ben Burdette 47ed067d45 initializer style 2020-04-08 09:07:58 -06:00
Eelco Dolstra 9ed097db7b
Merge pull request #3468 from Infinisil/functionArgsPositions
Make function arguments retain position info
2020-04-08 15:29:39 +02:00
Eelco Dolstra 1ab8d6ac18
Downloader: Only write data to the sink on a 200 response
Hopefully fixes #3278.
2020-04-08 15:27:09 +02:00
Ben Burdette 00c507cc52 columnRange -> column 2020-04-07 14:36:32 -06:00
Ben Burdette 20c0984a46 remove columnrange; switch to fmt in error.cc 2020-04-07 10:14:15 -06:00
Eelco Dolstra 03a4a3c95c Merge branch 'flakes' of github.com:NixOS/nix into flakes 2020-04-07 14:30:25 +02:00
Eelco Dolstra 54955867a6 Merge remote-tracking branch 'origin/master' into flakes 2020-04-07 14:29:45 +02:00
mlatus 9d04b5da17 nix run using $SHELL as default command 2020-04-07 20:29:40 +08:00
Eelco Dolstra 55cefd41d6 Merge branch 'fetchgit-recursive' of https://github.com/blitz/nix 2020-04-07 13:45:17 +02:00
Eelco Dolstra cd39709003 Cleanup 2020-04-07 09:27:17 +02:00
Eelco Dolstra 26aeeb7653 Add FIXME
(cherry picked from commit 2f494531b7)
2020-04-07 09:09:43 +02:00
Eelco Dolstra a6dfa3cb85 PathInput: Add some methods
(cherry picked from commit 78ad5b3d91)
2020-04-07 09:09:43 +02:00
Eelco Dolstra f58a9b0e62 Respect the narHash attribute in more input types
(cherry picked from commit a6ff66b658)
2020-04-07 09:09:18 +02:00
Eelco Dolstra 670feb000a Add 'path' fetcher
This fetchers copies a plain directory (i.e. not a Git/Mercurial
repository) to the store (or does nothing if the path is already a
store path).

One use case is to pin the 'nixpkgs' flake used to build the current
NixOS system, and prevent it from being garbage-collected, via a
system registry entry like this:

  {
      "from": {
          "id": "nixpkgs",
          "type": "indirect"
      },
      "to": {
          "type": "path",
          "path": "/nix/store/rralhl3wj4rdwzjn16g7d93mibvlr521-source",
          "lastModified": 1585388205,
          "rev": "b0c285807d6a9f1b7562ec417c24fa1a30ecc31a"
      },
      "exact": true
  }

Note the fake "lastModified" and "rev" attributes that ensure that the
flake gives the same evaluation results as the corresponding
Git/GitHub inputs.

(cherry picked from commit 12f9379123)
2020-04-07 09:08:51 +02:00
Eelco Dolstra 462421d345 Backport libfetchers from the flakes branch
This provides a pluggable mechanism for defining new fetchers. It adds
a builtin function 'fetchTree' that generalizes existing fetchers like
'fetchGit', 'fetchMercurial' and 'fetchTarball'. 'fetchTree' takes a
set of attributes, e.g.

  fetchTree {
    type = "git";
    url = "https://example.org/repo.git";
    ref = "some-branch";
    rev = "abcdef...";
  }

The existing fetchers are just wrappers around this. Note that the
input attributes to fetchTree are the same as flake input
specifications and flake lock file entries.

All fetchers share a common cache stored in
~/.cache/nix/fetcher-cache-v1.sqlite. This replaces the ad hoc caching
mechanisms in fetchGit and download.cc (e.g. ~/.cache/nix/{tarballs,git-revs*}).

This also adds support for Git worktrees (c169ea5904).
2020-04-07 09:03:14 +02:00
Ben Burdette 55c96b64e4 comment cleanup 2020-04-06 20:14:48 -06:00
Ben Burdette ec449c8450 constructor style basically working 2020-04-06 19:43:22 -06:00
Ben Burdette 85f14c4582 add libutil, libexpr include dirs 2020-04-06 11:15:01 -06:00
Ben Burdette 216263c36f Merge branch 'master' into error-format 2020-04-06 10:00:00 -06:00
Eelco Dolstra 68b43e01dd
nix flake info: Show resolved URL
This is useful for finding out what a registry lookup resolves to, e.g

  $ nix flake info patchelf
  Resolved URL:  github:NixOS/patchelf
  Locked URL:    github:NixOS/patchelf/cd7955af31698c571c30b7a0f78e59fd624d0229
2020-04-06 14:56:13 +02:00
Eelco Dolstra ce3173edc1
nix flake info --json: Don't evaluate
This makes its behaviour consistent with the non-json
variant. Querying the outputs should be done by another command
(e.g. 'nix search')
2020-04-06 14:39:47 +02:00
Eelco Dolstra 2f494531b7
Add FIXME 2020-04-06 14:28:37 +02:00
Cole Helbling c976cb0b8a
Don't retry on "unsupported protocol" error
When encountering an unsupported protocol, there's no need to retry.
Chances are, it won't suddenly be supported between retry attempts;
error instead. Otherwise, you see something like the following:

    $ nix-env -i -f git://git@github.com/foo/bar
    warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 335 ms
    warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 604 ms
    warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 1340 ms
    warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 2685 ms

With this change, you now see:

    $ nix-env -i -f git://git@github.com/foo/bar
    error: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1)
2020-04-05 09:00:34 -07:00
Ben Burdette 1221ae3dd0 libexpr 2020-04-05 07:12:16 -06:00
Ben Burdette 9a8b3e9747 move out of tests/ 2020-04-03 14:55:26 -06:00
Ben Burdette 9bb528d392 handle Pos instead of individual file/line/columnrange args 2020-04-03 13:15:59 -06:00
Eelco Dolstra 63fa92605b nix-env: Refuse to operate on a new-style profile
This prevents users from accidentally nuking their profile via
nix-env.

(cherry picked from commit 021634e3e3)
2020-04-03 20:14:34 +02:00
Ben Burdette 7b7801d3f0 variadic args for hint format 2020-04-03 08:48:20 -06:00
Eelco Dolstra 6e7f252ea6 Make --override-input sticky
When we do something like 'nix flake update --override-input nixpkgs
...', the override is now kept on subsequent calls. (If you don't want
this behaviour, you can use --no-write-lock-file.)
2020-04-03 14:06:29 +02:00
Eelco Dolstra 9c78f7f196 getFlake: In pure mode, check that the argument is an immutable flakeref 2020-04-03 13:07:05 +02:00
Ben Burdette c6b3fcddb0 formatted with astyle 2020-04-02 16:02:40 -06:00
Ben Burdette 1c329ca433 indenting 2020-04-02 14:25:43 -06:00
Eelco Dolstra 485a87f22f Don't barf on registry parse errors 2020-04-02 19:09:17 +02:00
Eelco Dolstra a6ff66b658 Respect the narHash attribute in more input types
call-flake.nix now passes node.info.narHash to fetchTree. This ensures
that dirty Git trees work even in pure mode.
2020-04-02 19:04:33 +02:00
Eelco Dolstra 78ad5b3d91 PathInput: Add some methods 2020-04-02 19:04:27 +02:00
Eelco Dolstra e1fc9f6690 Improve error message 2020-04-02 19:03:32 +02:00
Eelco Dolstra ab47868639 Change lastModified to the number of seconds in the epoch
'lastModifiedDate' is now a string representing the equivalent
date/time.
2020-04-02 18:39:41 +02:00
Eelco Dolstra 6d6467d376 Move parseTreeInfo() 2020-04-02 18:26:39 +02:00
Andreas Rammhold 4fc4eb6c93 libexpr: remove unused attrError
The attrError variable is no longer used but still allocated on every
call to the findAlongAttrPath function.
2020-04-02 17:04:00 +02:00
Eelco Dolstra 6cf91d6fbd fetchTree: Support integer attributes 2020-04-02 16:51:56 +02:00
Eelco Dolstra 12f9379123 Add 'path' fetcher
This fetchers copies a plain directory (i.e. not a Git/Mercurial
repository) to the store (or does nothing if the path is already a
store path).

One use case is to pin the 'nixpkgs' flake used to build the current
NixOS system, and prevent it from being garbage-collected, via a
system registry entry like this:

  {
      "from": {
          "id": "nixpkgs",
          "type": "indirect"
      },
      "to": {
          "type": "path",
          "path": "/nix/store/rralhl3wj4rdwzjn16g7d93mibvlr521-source",
          "lastModified": 1585388205,
          "rev": "b0c285807d6a9f1b7562ec417c24fa1a30ecc31a"
      },
      "exact": true
  }

Note the fake "lastModified" and "rev" attributes that ensure that the
flake gives the same evaluation results as the corresponding
Git/GitHub inputs.
2020-04-02 14:56:20 +02:00
Eelco Dolstra 00e1400eb7 Doh 2020-04-02 11:55:41 +02:00
Eelco Dolstra 021634e3e3 nix-env: Refuse to operate on a new-style profile
This prevents users from accidentally nuking their profile via
nix-env.
2020-04-02 11:54:48 +02:00
Eelco Dolstra ed13457dbf nix flake info --json: Show TreeInfo 2020-04-02 11:51:34 +02:00
Silvan Mosberger c34e96f7e0
Make function arguments retain position info
This allows querying the location of function arguments. E.g.

  builtins.unsafeGetAttrPos "x" (builtins.functionArgs ({ x }: null))

  => { column = 57; file = "/home/infinisil/src/nix/inst/test.nix"; line = 1; }
2020-04-02 05:52:52 +02:00
Ben Burdette e697884f65 using std:: everywhere; fix a formatting error; add exception flags 2020-04-01 21:30:19 -06:00
Ben Burdette dd7b8183a5 indenting 2020-04-01 16:20:20 -06:00
Ben Burdette 8713aeac5e remove using std::*, switch to include guard 2020-04-01 15:51:14 -06:00
Eelco Dolstra 74024515a3 Support registry entries that must match exactly
An example use is for pinning the "nixpkgs" entry the system-wide
registry to a particular store path. Inexact matches
(e.g. "nixpkgs/master") should still use the global registry.
2020-04-01 23:12:45 +02:00
Eelco Dolstra bd10a07d17 Registry: Use a struct instead of a tuple for entries 2020-04-01 23:03:27 +02:00
Eelco Dolstra 77ffaea4fa Add a system-wide flake registry /etc/nix/registry.json
One application for this is pinning the 'nixpkgs' flake to the exact
revision used to build the NixOS system, e.g.

  {
      "flakes": [
          {
              "from": {
                  "id": "nixpkgs",
                  "type": "indirect"
              },
              "to": {
                  "owner": "NixOS",
                  "repo": "nixpkgs",
                  "type": "github",
                  "rev": "b0c285807d6a9f1b7562ec417c24fa1a30ecc31a"
              }
          }
      ],
      "version": 2
  }
2020-04-01 22:56:50 +02:00
Eelco Dolstra 4fdec5f61d Merge remote-tracking branch 'origin/master' into flakes 2020-04-01 00:03:50 +02:00
Eelco Dolstra 03b56e96bf Typo 2020-03-31 23:55:07 +02:00
Ben Burdette 5b3aefff85 add some explanatory comments 2020-03-31 12:42:41 -06:00
Ben Burdette 9e7b89bf10 rename errors/warnings 2020-03-31 11:56:37 -06:00
Ben Burdette 09652f597c enum style 2020-03-31 09:36:20 -06:00
Eelco Dolstra a7540294cf
Merge pull request #3460 from NixOS/dev-shell
Backport 'nix dev-shell' from the flakes branch
2020-03-31 14:46:15 +02:00
Eelco Dolstra 3166b97174 nix shell -> nix dev-shell 2020-03-31 13:45:28 +02:00
mlatus 12556e5709 fix placeholder not substituted in passAsFile 2020-03-31 19:40:16 +08:00
John Ericson 7e9a2718f0 s/outputHashRecursive/ingestionMethod/c 2020-03-30 22:36:15 +00:00
John Ericson 51afea3af2 Never cast FileIngestionMethod to or from boolean 2020-03-30 22:31:51 +00:00
John Ericson c251b011cd Merge remote-tracking branch 'upstream/master' into enum-FileIngestionMethod 2020-03-30 18:16:44 -04:00
John Ericson bbbb7c1bc7 Use auto with some FileIngestionMethod local variables 2020-03-30 18:15:55 -04:00
Eelco Dolstra e1a94ad852 Backport 'nix dev-shell' from the flakes branch
This also adds a '--profile' option to 'nix build' (replacing 'nix-env
--set').
2020-03-30 19:16:45 +02:00
Ben Burdette 35c7bab09a build with make 2020-03-30 09:14:29 -06:00
Eelco Dolstra 367577d9a6 Fix macOS build 2020-03-30 17:00:40 +02:00
Eelco Dolstra d15d91cad1 Makefile cleanup 2020-03-30 16:50:13 +02:00
Eelco Dolstra e322a16523 Remove global -I flags
(cherry picked from commit 2c692a3b14)
2020-03-30 15:30:19 +02:00
Eelco Dolstra 2c692a3b14 Remove global -I flags 2020-03-30 14:39:33 +02:00
Eelco Dolstra e0a0ae0467 Move fetchers from libstore to libfetchers 2020-03-30 14:04:53 +02:00
Eelco Dolstra 2287cc6486 Fix segfault 2020-03-30 13:31:55 +02:00
Julian Stecklina 40c023ecfe fetchGit: don't use std::filesystem to filter git repos
Using std::filesystem means also having to link with -lstdc++fs on
some platforms and it's hard to discover for what platforms this is
needed. As all the functionality is already implemented as utilities,
use those instead.
2020-03-30 00:32:42 +02:00
Bjørn Forsman f686efeed4 fetchGit: fix submodule corner case by fetching all refs from cacheDir
Due to fetchGit not checking if rev is an ancestor of ref (there is even
a FIXME comment about it in the code), the cache repo might not have the
ref even though it has the rev. This doesn't matter when submodule =
false, but the submodule = true code blows up because it tries to fetch
the (missing) ref from the cache repo.

Fix this in the simplest way possible: fetch all refs from the local
cache repo when submodules = true.

TODO: Add tests.
2020-03-29 22:29:58 +02:00
Bjørn Forsman cc522d0d23 fetchGit: fix submodules = true for dirty trees 2020-03-29 22:29:58 +02:00
Bjørn Forsman b306b7039e fetchGit: checkout rev instead of latest ref
Major bugfix for the submodules = true code path.

TODO: Add tests.
2020-03-29 22:29:58 +02:00
Bjørn Forsman 369fffd6f1 fetchGit: add submodules attribute to the .link file
The .link file is used as a lock, so I think we should put the
"submodule" attribute in there since turning on submodules creates a new
.link file path.
2020-03-29 22:29:58 +02:00
Bjørn Forsman 6864ad7cf5 fetchGit: fix submodule output attribute
Before this change it would be false for all evaluations but the first.
Now it follows the input argument (as it should).
2020-03-29 22:29:58 +02:00
Bjørn Forsman 002a3a95dc fetchGit: fix "fatal: couldn't find remote ref refs/heads/master" issue with submodules 2020-03-29 22:29:58 +02:00
Julian Stecklina cc4fe977e5 Link to stdc++fs
Some platforms seem to still require linking with stdc++fs to enable
STL std::filesystem support.
2020-03-29 22:29:58 +02:00
Julian Stecklina 435366ed3c Rename fetchGit fetchSubmodules to just submodules 2020-03-29 22:29:58 +02:00
Julian Stecklina c8d33de777 Add git submodule fixes from @bjornfor
This fixes fetching repositories with no submodules and also cleans up
.git files in checkouts.
2020-03-29 22:29:57 +02:00
Julian Stecklina f58604ac32 Add fetchSubmodules to builtins.fetchGit
There are some downsides to this features:

 - Submodules are not cached (unlike the root repo),
 - Full checkouts are created in a temporary directory.
2020-03-29 22:29:57 +02:00
John Ericson 225e62a56a Replace some bool recursive with a new FileIngestionMethod enum 2020-03-29 15:16:20 -04:00
Eelco Dolstra 4989c04dd2 nix flake info --json: Dump attr-style flakeref 2020-03-28 22:59:38 +01:00
Eelco Dolstra 2fccef0c59 Warn about --override-input / --update-input flags that don't match an input 2020-03-28 19:09:36 +01:00
Eelco Dolstra 2287e2f279 nix flake info: Show flake subdirectory 2020-03-28 18:05:50 +01:00
Eelco Dolstra 015f8f1c13 Improve lock file generation
This is now done in a single pass. Also fixes some issues when
updating flakes with circular dependencies. Finally, when using
'--recreate-lock-file --commit-lock-file', the commit message now
correctly shows the differences.
2020-03-27 21:08:41 +01:00
Ben Burdette 759f39800b remove util.hh from deps 2020-03-27 10:55:09 -06:00
Ben Burdette 00eb3fcb7a more cleanup 2020-03-27 10:13:46 -06:00
Ben Burdette a3ef00be6c camelcase; optional hint 2020-03-27 10:03:02 -06:00
Eelco Dolstra 3fa1e7dace Fix diffLockFiles() 2020-03-27 16:15:50 +01:00
Ben Burdette d44c9c5581 some colors 2020-03-25 11:20:44 -06:00
Ben Burdette 3582dc3c88 programName as static member var 2020-03-25 10:52:03 -06:00
Ben Burdette fc310eda3a switch to one level of builder function, not subobject functions 2020-03-24 14:24:57 -06:00
Ben Burdette 0166e7ab6d MkNixCode, MkErrLine approach 2020-03-24 11:21:35 -06:00
Ben Burdette 4171ab4bbd renaming 2020-03-24 09:18:23 -06:00
Eelco Dolstra 0a10854f85 Misc changes from the flakes branch 2020-03-24 14:34:47 +01:00
Eelco Dolstra c85097da7c Fix --refresh with --no-net
https://hydra.nixos.org/build/110879699
(cherry picked from commit 5bbe793abf)
2020-03-24 14:26:23 +01:00
Eelco Dolstra 6b824c78f1 nix: Add --refresh as an alias for --tarball-ttl 0
(cherry picked from commit e721f99817)
2020-03-24 14:26:23 +01:00
Eelco Dolstra 777e21e596 nix path-info --json: Print hash in SRI format
(cherry picked from commit 442e665d6d)
2020-03-24 14:26:23 +01:00
Eelco Dolstra 7a8de57d3e Pretty-print 'nix why-depends' / 'nix-store -q --tree' output
Extracted from 678301072f.
2020-03-24 14:26:23 +01:00
Eelco Dolstra 4260a22a55 absPath(): Use std::optional
(cherry picked from commit 1bf9eb21b7)
2020-03-24 14:25:28 +01:00
Eelco Dolstra f9611c7ae4 buildenv: Eliminate global variables, other cleanup
(cherry picked from commit b82f75464d)
2020-03-24 14:06:47 +01:00
Eelco Dolstra 76e7d958ed Fix coverage build
https://hydra.nixos.org/build/110757285
(cherry picked from commit b430a81a1f)
2020-03-24 14:06:47 +01:00
Eelco Dolstra 231a8aa2c2 nix edit: Support non-derivation attributes
E.g.

  $ nix edit .#nixosConfigurations.bla

now works.

(cherry picked from commit d2032edb2f)
2020-03-24 14:06:47 +01:00
Eelco Dolstra c1ca4f0acc findAlongAttrPath(): Return position
(cherry picked from commit 0b013a54dc)
2020-03-24 14:06:47 +01:00
Eelco Dolstra 1eb952d27a findAlongAttrPath(): Throw AttrPathNotFound
(cherry picked from commit 6b0ca8e803)
2020-03-24 14:06:47 +01:00
Eelco Dolstra edc34cc1a2 Add function for quoting strings
(cherry picked from commit 7dcf5b011a)
2020-03-24 13:44:04 +01:00
Eelco Dolstra 5a7e7fc35f Use std::string_view
(cherry picked from commit 6529490cc1)
2020-03-24 13:26:37 +01:00
Eelco Dolstra c34a20e1f6 EvalState::allocAttr(): Add convenience method
(cherry picked from commit c02da99757)
2020-03-24 13:26:37 +01:00
Ben Burdette aadd59d005 error test 2020-03-23 15:29:49 -06:00
Eelco Dolstra 7abe3bde8a Merge remote-tracking branch 'origin/master' into flakes 2020-03-23 13:27:54 +01:00
Eelco Dolstra 5885e20404
Merge pull request #3429 from LnL7/darwin-sandbox
darwin sandbox
2020-03-23 09:13:36 +01:00
Ben Burdette f694f43d7d straightforward port of rust mockup code 2020-03-22 12:25:47 -06:00
Daiderd Jordan 2e9bc1245c
sandbox: fix /bin/sh on catalina
Sadly 10.15 changed /bin/sh to a shim which executes bash, this means it
can't be used anymore without also opening up the sandbox to allow bash.

    Failed to exec /bin/bash as variant for /bin/sh (1: Operation not permitted).
2020-03-20 22:12:30 +01:00
Daiderd Jordan f6c122aaeb
sandbox: allow pty devices
Nix now runs builds with a pseudo-terminal to enable colored build
output.
2020-03-20 21:58:45 +01:00
Daiderd Jordan 7f2df903d9
libstore: relax default sandbox-paths on darwin 2020-03-20 21:31:20 +01:00
Daiderd Jordan afb78ebd34
libstore: disable resolve-system-dependencies hook
This is used to determine the dependency tree of impure libraries so nix
knows what paths to open in the sandbox.  With the less restrictive
defaults it isn't needed anymore.
2020-03-20 21:21:56 +01:00
Eelco Dolstra 1537e270fb Merge remote-tracking branch 'origin/master' into flakes 2020-03-20 13:15:11 +01:00
jakobrs c5a488afc0 Remove the --delete option for --gc
Running `nix-store --gc --delete` will, as of Nix 2.3.3, simply fail
because the --delete option conflicts with the --delete operation.

  $ nix-store --gc --delete
  error: only one operation may be specified
  Try 'nix-store --help' for more information.

Furthermore, it has been broken since at least Nix 0.16 (which was
released sometime in 2010), which means that any scripts which depend
on it should have been broken at least nine years ago. This commit
simply formally removes the option. There should be no actual difference
in behaviour as far as the user is concerned: it errors with the exact
same error message. The manual has been edited to remove any references
to the (now gone) --delete option.

Other information:
* Path for Nix 0.16 used:
  /nix/store/rp3sgmskn0p0pj1ia2qwd5al6f6pinz4-nix-0.16
2020-03-19 19:41:23 +01:00
Eelco Dolstra ef74fafc03
nix repl: Put EvalState on the heap
See 0629601da1.
2020-03-19 13:52:28 +01:00
Eelco Dolstra b244e65cdb
nix repl: Scan NixRepl for GC roots
Fixes #3175.
2020-03-19 13:50:01 +01:00
Eelco Dolstra 8aa354fdfd
Register flake-registry.json as a GC root again 2020-03-19 11:45:34 +01:00
Eelco Dolstra 1c127e6a82
downloadFile(): Use expired file if the download fails 2020-03-19 11:42:50 +01:00
Eelco Dolstra f6ddf48882
Get rid of downloadCached()
Everything uses the generic caching system now.
2020-03-18 17:24:19 +01:00
Eelco Dolstra c5ec95e2c7
tarball.cc: Use ETags 2020-03-18 15:14:23 +01:00
Eelco Dolstra 1b49479836
Remove flake closure caching
This is not compatible with lazy flake input fetching.
2020-03-18 14:11:58 +01:00
Eelco Dolstra 1e7ce1d6da
tarball / github fetchers: Use generic caching system 2020-03-18 14:08:25 +01:00
Eelco Dolstra 38e360154d
Git: Use unified caching system 2020-03-17 22:35:29 +01:00
Eelco Dolstra d1165d8791
Require shallow clones to be requested explicitly
If you do a fetchTree on a Git repository, whether the result contains
a revCount attribute should not depend on whether that repository
happens to be a shallow clone or not. That would complicate caching a
lot and would be semantically messy. So applying fetchTree/fetchGit to
a shallow repository is now an error unless you pass the attribute
'shallow = true'. If 'shallow = true', we don't return revCount, even
if the repository is not actually shallow.

Note that Nix itself is not doing shallow clones at the moment. But it
could do so as an optimisation if the user specifies 'shallow = true'.

Issue #2988.
2020-03-17 22:35:29 +01:00
Eelco Dolstra 2a4e4f6a6e
Unified fetcher caching system 2020-03-17 22:35:29 +01:00
Eelco Dolstra fbcb897e21
Add a test for shallow Git clones
Also, don't return a revCount anymore for shallow or dirty Git trees,
since it's incorrect.

Closes #2988.
2020-03-16 13:20:32 +01:00
John Ericson 68fe0d9809 Add missing #include <regex> 2020-03-13 21:24:35 -04:00
Eelco Dolstra 779ef8f5ef
Merge pull request #3380 from contrun/no-attr-path-for-installed
display attr-path only when queried available
2020-03-13 19:26:20 +01:00
YI b6d794fb8d display attr-path only when queried available 2020-03-14 00:36:26 +08:00
Eelco Dolstra 0c2088d438 Merge remote-tracking branch 'origin/master' into flakes 2020-03-13 17:03:30 +01:00
Eelco Dolstra b816515f61
Fix ca-references feature check
Fixes #3406.
2020-03-13 13:15:51 +01:00
Eelco Dolstra ae9119167e Change the lock file to a graph
This enables support for cycles between flakes.
2020-03-12 22:06:57 +01:00
Eelco Dolstra d048577909
Merge pull request #3403 from hercules-ci/issue-3398-path-info-cache-ttls
pathInfoCache: Respect disk cache TTLs #3398
2020-03-12 11:43:31 +01:00
Robert Hensing 3f55f8a8fb pathInfoCache: Respect disk cache TTLs #3398 2020-03-12 10:30:28 +01:00
Will Dietz 15edd2349e local.mk: fix user-env.cc dep on buildenv.nix.gen.hh, resolve occasional build failure 2020-03-12 00:51:56 +01:00
Eelco Dolstra e188fe7c6d Move call-flake.nix into libexpr 2020-03-11 17:04:51 +01:00
Eelco Dolstra 35f6651735 Merge remote-tracking branch 'origin/master' into flakes 2020-03-11 17:03:38 +01:00
Eelco Dolstra 9950cdec35 Move some corepkgs into the nix binary 2020-03-11 16:57:48 +01:00
Eelco Dolstra e02481ded2 parseExprFromString(): Use std::string_view 2020-03-11 16:56:29 +01:00
Eelco Dolstra 97d1c7f932 Merge remote-tracking branch 'origin/master' into flakes 2020-03-11 15:17:21 +01:00
Eelco Dolstra cfc38257cf Fix flake subdirectory handling 2020-03-10 19:21:47 +01:00
Eelco Dolstra 8a1d8701f6
nix-store -q --graph: Fix edges
Fixes #3389.
2020-03-10 11:11:46 +01:00
Eelco Dolstra 983fab7ea9
dotgraph.cc: Remove dead code 2020-03-10 11:06:55 +01:00
Eelco Dolstra 73769b28e3 Move calling flakes into a Nix helper function (call-flake.nix) 2020-03-09 15:28:41 +01:00
Eelco Dolstra 5a1514adb8 Add 'flake' attribute to lock files to denote non-flakes 2020-03-09 15:27:49 +01:00
Eelco Dolstra 73b6d87e17
Merge remote-tracking branch 'origin/master' into flakes 2020-03-04 13:58:42 +01:00
Eelco Dolstra d37dc71e3c
nix-build: Fix !<output> handling
This was broken by 22a754c091.

https://hydra.nixos.org/eval/1573669
2020-03-04 13:56:17 +01:00
Eelco Dolstra 75db069f92
Optimise Derivation::unparse()
In

  nix-instantiate --dry-run '<nixpkgs/nixos/release-combined.nix>' -A nixos.tests.simple.x86_64-linux

this reduces time spent in unparse() from 9.15% to 4.31%. The main
culprit was appending characters one at a time to the destination
string. Even though the string has enough capacity, push_back() still
needs to check this on every call.
2020-03-04 11:44:45 +01:00
Eelco Dolstra 401b5bc541
builtins.cache: Cache regular expressions
The evaluator was spending about 1% of its time compiling a small
number of regexes over and over again.
2020-03-04 11:44:33 +01:00
Eelco Dolstra 22a754c091
Fix GC failures on bad store path names
It failed on names like '/nix/store/9ip48nkc9rfy0a4yaw98lp6gipqlib1a-'.
2020-02-28 18:07:10 +01:00
Eelco Dolstra f443d5ca19 Merge remote-tracking branch 'origin/master' into flakes 2020-02-27 15:19:58 +01:00
Eelco Dolstra 2672a28bb4 nix dev-shell: Add --command option
Note: like 'nix run', and unlike 'nix-shell', this takes an argv
vector rather than a shell command. So

  nix dev-shell -c 'echo $PATH'

doesn't work. Instead you need to do

  nix dev-shell -c bash -c 'echo $PATH'
2020-02-27 15:17:37 +01:00
Eelco Dolstra 4a4521f462 Fix nlohmann::json exception 2020-02-20 23:47:02 +01:00
Eelco Dolstra 73c9840569 Restore subdir -> dir
Got this mixed up somewhere.
2020-02-20 23:44:06 +01:00
Eelco Dolstra d068f9ffff Restore subdir support in registries
Hacky...
2020-02-20 22:14:44 +01:00
Eelco Dolstra 890df325c7
fetchTree: Use a feature flag 2020-02-20 13:36:16 +01:00
Eelco Dolstra 2e953b567e
Merge pull request #3325 from xzfc/clean-tmpdir
nix-shell: clean up the tmpDir and escape variables
2020-02-19 21:29:18 +01:00
Eelco Dolstra 4ad5826a18 nix eval-hydra-jobs: Remove
On second thought, let's move this back to Hydra.
2020-02-19 16:12:49 +01:00
Albert Safin f2a03acf3f nix-shell: clean up the tmpDir and escape variables
The problem fixed: each nix-shell invocation creates a new temporary
directory (`/tmp/nix-shell-*`) and never cleans up.

And while I'm here, shellescape all variables inlined into the rcfile.
See what might happen without escaping:

    $ export TZ="';echo pwned'"
    $ nix-shell -p hello --run hello
    pwned
    Hello, world!
2020-02-19 14:28:49 +00:00
Eelco Dolstra 1d99c4ab25
Merge pull request #3229 from Ma27/flakes-fetchgit-worktree-support
builtins.fetchGit: Fix build when fetching a git worktree
2020-02-19 14:11:18 +01:00
Maximilian Bosch c169ea5904
builtins.fetchGit: Fix build when fetching a git worktree
Worktrees[1] are a feature of git which allow you to check out a ref in
a different directory.

While playing around with flakes I realized that git repositories in a
worktree checkout break when trying to build a flake:

```
$ git worktree add ../nixpkgs-flakes nixpkgs-flakes
$ cd ../nixpkgs-flakes
$ nix build .#hello
error: opening directory '/home/ma27/Projects/nixpkgs-flakes/.git/refs/heads': Not a directory
```

This issue has been fixed by determining with `git rev-parse --git-common-dir`
where the actual `.git` directory is.

Please note that this issue only exists on the `flakes` branch, fetching
worktree checkouts with Nix master seems to work fine.

[1] https://git-scm.com/docs/git-worktree
2020-02-19 14:00:36 +01:00
Eelco Dolstra 2a14c28669
Merge pull request #3357 from carlosdagos/pure-nix-shell-proxy-env
Pass through http proxy env vars in pure shell
2020-02-19 13:02:02 +01:00
Eelco Dolstra f3505a7899 Merge remote-tracking branch 'origin/master' into flakes 2020-02-19 12:57:45 +01:00
Eelco Dolstra 30c8297ded nix eval-hydra-jobs: Add feature 2020-02-19 12:35:03 +01:00
Eelco Dolstra c4d3674de6
Merge pull request #3353 from tbsmoest/priv_tobias_pr_set_deathsig-1.4
Fix PR_SET_PDEATHSIG results in Broken pipe (#2395)
2020-02-19 12:29:12 +01:00
Eelco Dolstra 95468e3c1e Fix nixpkgs.<attr> warning 2020-02-19 00:09:42 +01:00
Eelco Dolstra edee6169bf nix eval-hydra-jobs: Fix aggregate derivation name 2020-02-18 22:05:49 +01:00
Eelco Dolstra 1351101c28 nix eval-hydra-jobs: Check aggregate jobs in --dry-run mode 2020-02-18 19:26:11 +01:00
Eelco Dolstra 553e584f92
LocalStore::checkDerivationOutputs(): Improve error message 2020-02-18 17:51:48 +01:00
Eelco Dolstra d8fd31f50f
Disable the progress bar if $TERM == dumb or unset
Fixes #3363.
2020-02-18 17:51:18 +01:00
Eelco Dolstra 8f9dcfc671
Disable the progress bar if $TERM == dumb or unset
Fixes #3363.
2020-02-18 17:47:53 +01:00
Eelco Dolstra b5e3c04c03
Fix URL parser
Fixes #3361.
2020-02-18 12:51:26 +01:00
Eelco Dolstra 6529490cc1 nix eval-hydra-jobs: Support job names as aggregate constituents
Fixes https://github.com/NixOS/hydra/issues/715.
2020-02-17 15:53:59 +01:00
Eelco Dolstra 8a78bcf6a2 LocalStore::checkDerivationOutputs(): Improve error message 2020-02-17 15:46:07 +01:00
Eelco Dolstra b0336e7cf7 nix eval-hydra-job: Progress indicator 2020-02-17 14:33:10 +01:00
Eelco Dolstra c6e63065f3 nix eval-hydra-jobs: Improve error handling 2020-02-17 14:31:29 +01:00
Eelco Dolstra eb19ff3b82 nix eval-hydra-jobs: Support flakes 2020-02-14 23:23:41 +01:00
Eelco Dolstra 4c24263967 nix eval-hydra-jobs: Support parallel evaluation
Example usage:

  $ nix eval-hydra-jobs -f '<nixpkgs/pkgs/top-level/release.nix>' '' \
      --max-memory-size 2048 --workers 8
2020-02-14 23:05:49 +01:00
Eelco Dolstra e375da6899 Add 'nix eval-hydra-jobs' command 2020-02-14 23:00:42 +01:00
Eelco Dolstra 46a284263f Fix build 2020-02-14 22:45:33 +01:00
Eelco Dolstra 6208d24c38 Merge remote-tracking branch 'origin/master' into flakes 2020-02-14 22:42:35 +01:00
Tobias Möst 3e347220c8 Fix PR_SET_PDEATHSIG results in Broken pipe (#2395)
The ssh client is lazily started by the first worker thread, that
requires a ssh connection. To avoid the ssh client to be killed, when
the worker process is stopped, do not set PR_SET_PDEATHSIG.
2020-02-14 07:51:44 +01:00
Carlos D d78141a886 Pass through http proxy env vars in pure shell 2020-02-14 16:11:22 +11:00
Eelco Dolstra 9af10b753c Bindings::get(): std::optional<Attr *> -> Attr *
Returning a nullable type in an optional is silly.
2020-02-13 17:15:05 +01:00
Eelco Dolstra d8972317fc Prevent uninitialized StorePath creation 2020-02-13 16:12:16 +01:00
Eelco Dolstra 26dacc0983 Add fetchTree builtin function
This allows all supported fetchers to be used, e.g.

  builtins.fetchTree {
    type = "github";
    owner = "NixOS";
    repo = "nix";
    rev = "d4df99a3349cf2228a8ee78dea320afef86eb3ba";
  }
2020-02-11 23:53:46 +01:00
Eelco Dolstra d4df99a334 Parse narHash attribute for all input types 2020-02-11 23:53:24 +01:00
Eelco Dolstra 442e665d6d nix path-info --json: Print hash in SRI format 2020-02-11 23:50:16 +01:00
Eelco Dolstra d2032edb2f nix edit: Support non-derivation attributes
E.g.

  $ nix edit .#nixosConfigurations.bla

now works.
2020-02-07 14:22:01 +01:00
Eelco Dolstra 0b013a54dc findAlongAttrPath(): Return position 2020-02-07 14:08:24 +01:00
Eelco Dolstra 84a3a5c3cd Merge remote-tracking branch 'origin/master' into flakes 2020-02-07 13:57:31 +01:00
Eelco Dolstra d82b78bf51
Fix segfault in gcc on i686-linux
src/libstore/ssh-store.cc: In constructor 'nix::SSHStore::SSHStore(const string&, const Params&)':
  src/libstore/ssh-store.cc:31:21: internal compiler error: Segmentation fault
               compress)
                       ^
  Please submit a full bug report,
  with preprocessed source if appropriate.

https://hydra.nixos.org/build/111545609
2020-02-07 13:01:48 +01:00
Eelco Dolstra 379852a152 Registry: Use attr notation instead of URLs 2020-02-06 14:27:31 +01:00
Eelco Dolstra be2580be01 Warn when we commit a new revision 2020-02-05 15:47:11 +01:00
Eelco Dolstra 022287060b Merge remote-tracking branch 'origin/master' into flakes 2020-02-05 15:41:51 +01:00
Eelco Dolstra 750c993f00 Improve commit subject line 2020-02-05 15:31:16 +01:00
Eelco Dolstra 9d7fb62db6 Add option --commit-lock-file 2020-02-05 14:48:49 +01:00
Eelco Dolstra e2213d77a2 Set 'ref' properly for local trees 2020-02-04 21:55:57 +01:00
Eelco Dolstra db88cb401b
Merge pull request #3344 from LnL7/ssh-ng-remote-params
ssh-store: add remote-store and remote-program query params
2020-02-04 10:10:08 +01:00
Daiderd Jordan 8745c63d3c
ssh-store: add remote-store and remote-program query params
Brings the functionality of ssh-ng:// in sync with the legacy ssh://
implementation.  Specifying the remote store uri enables various useful
things. eg.

    $ nix copy --to ssh-ng://cache?remote-store=file://mnt/cache --all
2020-02-03 23:22:28 +01:00
Eelco Dolstra 0a4e911cf4 Install headers in the correct location 2020-02-03 18:04:09 +01:00
Eelco Dolstra a2628b43bb Fix URL parser
Fixes #3062.
2020-02-03 15:27:26 +01:00
Eelco Dolstra d070e1c532 Fix parsing of '#nixosConfigurations."hostname".config...'
This is not strictly speaking valid but who cares.
2020-02-03 14:29:34 +01:00
Eelco Dolstra 44d6421160 Emit narHash attribute 2020-02-03 14:15:39 +01:00
Eelco Dolstra d5334c466b
Automatically do git/hg add on flake.lock 2020-02-02 16:33:22 +01:00
Eelco Dolstra f83acbbfe3
Don't store fragment in FlakeRef 2020-02-02 16:33:19 +01:00
Eelco Dolstra 86748d3571
Re-read flake after updating flake.lock
Otherwise we'll evaluate the flake with the wrong metadata (rev, ...).
2020-02-02 15:46:30 +01:00
Eelco Dolstra af35b318f3
Detect circular flake imports
Fixes #2997.
2020-02-02 13:14:34 +01:00
Eelco Dolstra a9ebc3ea5d
Remove the git+ and hg+ prefixes from structured input refs 2020-02-02 13:06:00 +01:00
Eelco Dolstra 94a94da075
Substitute flake inputs
This improves reproducibility and may be faster than fetching from the
original source (especially for git/hg inputs, but probably also for
github inputs - our binary cache is probably faster than GitHub's
dynamically generated tarballs).

Unfortunately this doesn't work for the top-level flake since even if
we know the NAR hash of the tree, we don't know the other tree
attributes such as revCount and lastModified.

Fixes #3253.
2020-02-02 12:41:23 +01:00
Eelco Dolstra fad9faf354
Add TreeInfo::computeStorePath() 2020-02-02 12:29:53 +01:00
Eelco Dolstra 7bcc9f2aaf
Add convenience overload for toRealPath() 2020-02-02 12:28:56 +01:00
Eelco Dolstra 958ec5de56
Cleanup 2020-02-02 11:31:58 +01:00
Eelco Dolstra b270869466
Renamed ref / resolvedRef -> lockedRef 2020-02-02 00:06:59 +01:00
Eelco Dolstra 887730aab3
Remove superfluous TreeInfo::rev field 2020-02-01 23:54:20 +01:00
Eelco Dolstra b9d64f9318
Record TreeInfo in the lock file
Necessary for #3253.
2020-02-01 23:33:44 +01:00
Eelco Dolstra 8451298b35
Factor out TreeInfo 2020-02-01 22:06:55 +01:00