Commit graph

5420 commits

Author SHA1 Message Date
John Ericson 574d5460f0 Make sure info.ca tag bit is set in nix add-to-store 2020-06-04 20:33:28 +00:00
Eelco Dolstra ab54031e04 getDefaultUrl() -> value_or() 2020-06-04 20:24:28 +02:00
Eelco Dolstra 810b2c6a48 nix flake init: Add a '--template' flag
The initial contents of the flake is specified by the
'templates.<name>' or 'defaultTemplate' output of another flake. E.g.

  outputs = { self }: {

    templates = {

      nixos-container = {
        path = ./nixos-container;
        description = "An example of a NixOS container";
      };

    };

  };

allows

  $ nix flake init -t templates#nixos-container

Also add a command 'nix flake new', which is identical to 'nix flake
init' except that it initializes a specified directory rather than the
current directory.
2020-06-04 20:22:25 +02:00
Ben Burdette 94427ffee3 add some comments 2020-06-04 11:53:19 -06:00
Matthew Bauer 2299ef705c Add error message when FileIngestionMethod is out of bounds
bool coerces anything >0 to true, but in the future we may have other
file ingestion methods. This shows a better error message when the
“recursive” byte isn’t 1.
2020-06-04 11:32:39 -05:00
Eelco Dolstra dc305500c3
Merge pull request #3660 from Kloenk/selfhost-gitlab
add support for selfhosted gitlab/github
2020-06-04 14:59:33 +02:00
Eelco Dolstra d746ef4a81 Disable eval cache with --impure
Fixes

  $ nix build nixpkgs#zoom-us
  error: Package ‘zoom-us-5.0.399860.0429’ in /nix/store/m79v7h75b69fkk8d2qcwm555l3wq6fmv-source/pkgs/applications/networking/instant-messengers/zoom-us/default.nix:126 has an unfree license (‘unfree’), refusing to evaluate.

  $ nix build nixpkgs#zoom-us --impure
  error: cached failure of attribute 'legacyPackages.x86_64-linux.zoom-us.drvPath'
2020-06-04 14:53:51 +02:00
Finn Behrens 108debef6f
add support for selfhosted gitlab/github 2020-06-04 14:42:39 +02:00
Eelco Dolstra f85606c431 Merge remote-tracking branch 'origin/master' into flakes 2020-06-04 13:16:28 +02:00
Eelco Dolstra 0f44b60e6d Make 'nix dev-shell' a deprecated alias for 'nix develop' 2020-06-04 11:14:19 +02:00
Eelco Dolstra 61e3d598b6 Rename 'nix dev-shell' to 'nix develop'
Fixes #3648.
2020-06-04 10:57:40 +02:00
John Ericson 53bc8ff152 No C++ designated initializers yet with Clang 7 2020-06-03 20:45:14 -04:00
Ben Burdette 4335ba999b Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-06-03 17:00:00 -06:00
Ben Burdette f97576c5d9 newline-as-prefix; no final newline in output. 2020-06-03 14:47:00 -06:00
Carlo Nucera 132d6f2c24 Clarify the description of StorePath construction 2020-06-03 16:08:32 -04:00
Eelco Dolstra c20591ddc3 Merge remote-tracking branch 'origin/master' into flakes 2020-06-03 16:15:22 +02:00
Jonas Chevalier 6ee03b8444
libutils/hash: remove default encoding
This will make it easier to reason about the hash encoding and switch to
SRI everywhere where possible.
2020-06-03 13:49:51 +02:00
John Ericson 01572c2198
Missing #include <cassert> in lru-cache.hh (#3654)
This was a latent bug that just appeared because of the tests that were
added. Remember to wait for CI! :)
2020-06-03 10:15:22 +00:00
John Ericson 3c78ac348c Merge remote-tracking branch 'obsidian/no-hash-type-unknown' into validPathInfo-ca-proper-datatype 2020-06-03 04:44:24 +00:00
John Ericson fecff16a6e Merge remote-tracking branch 'obsidian/missing-include-0' into validPathInfo-ca-proper-datatype 2020-06-02 23:23:30 +00:00
John Ericson 39ba87be9b Missing #include <cassert> in lru-cache.hh
This was a latent bug that just appeared because of the tests that were
added. Remember to wait for CI! :)
2020-06-02 21:36:53 +00:00
John Ericson 406dbb7fce outputHashAlgo can be blank so parse accordingly
It is blank for SRI hashes.
2020-06-02 21:09:15 +00:00
John Ericson 1fcd3afc38 Fix hashes 2020-06-02 20:35:17 +00:00
Carlo Nucera 75d2581390 Typo 2020-06-02 16:21:18 -04:00
Carlo Nucera 78f137e931 Validate text version instead, throw Errors 2020-06-02 16:20:22 -04:00
Carlo Nucera a5cdf1867e Add assertions for SHA256 in fixed case 2020-06-02 16:13:08 -04:00
Carlo Nucera fd2eb41e64 Move file-hash to content-address 2020-06-02 15:44:58 -04:00
Carlo Nucera 343c20a404 WIP Completed implementation 2020-06-02 15:23:21 -04:00
John Ericson c664e68b87 Fix to-base --type handler to correctly set std::optional flag
Now that we have a separate flag function, also describe why it is
optional.
2020-06-02 18:25:32 +00:00
Carlo Nucera 390bf64858 WIP 2020-06-02 14:15:58 -04:00
John Ericson c502119fd3 to-base supports parsing SRI hashes, so make type flag optional 2020-06-02 18:05:26 +00:00
John Ericson a33270ce1d Clean up ValidPathInfo::isContentAddressed with std::visit 2020-06-02 17:04:21 +00:00
John Ericson 25e61812f3
Apply suggestions from code review
Co-authored-by: Matthew Bauer <mjbauer95@gmail.com>
2020-06-02 12:47:18 -04:00
John Ericson d73dbc8e4c Remove hashingWithUnknownAlgoExits
A valid hash type must be provided now. The hash itself can still be
invalid, but that doesn't cause an `abort()`.
2020-06-02 16:28:54 +00:00
John Ericson 64cffb804a Merge remote-tracking branch 'upstream/master' into no-hash-type-unknown 2020-06-02 16:07:25 +00:00
John Ericson 450dcf2c1b Remove HashType::Unknown
Instead, `Hash` uses `std::optional<HashType>`. In the future, we may
also make `Hash` itself require a known hash type, encoraging people to
use `std::optional<Hash>` instead.
2020-06-02 15:52:13 +00:00
Carlo Nucera a5d820a0a3 Change parseCa(Opt) to parseContentAddress(Opt) 2020-06-02 11:00:10 -04:00
Ben Burdette 156d4f8bc8 remove extra space in SysErrors 2020-06-02 08:45:37 -06:00
John Ericson 1b6461f671 Merge remote-tracking branch 'upstream/master' into validPathInfo-ca-proper-datatype 2020-06-02 14:31:18 +00:00
Ben Burdette d82d230b40 elide the 'ErrorInfo' in logError and logWarning calls 2020-06-02 08:22:24 -06:00
Eelco Dolstra bfa1acd85c
Merge pull request #3639 from obsidiansystems/do-fixme-store-removes
Remove `addToStore` variant as requested by `FIXME`
2020-06-02 15:39:07 +02:00
Eelco Dolstra c16fdda3a6 Merge branch 'lru-tests' of https://github.com/gilligan/nix 2020-06-02 12:07:48 +02:00
Eelco Dolstra e9fee8e6a7
src/libutil/tests/lru-cache.cc: Check erase()
Co-authored-by: James Lee <jbit@jbit.net>
2020-06-02 12:06:59 +02:00
John Ericson efcd30da89 WIP 2020-06-02 00:37:43 +00:00
John Ericson 754c910953 WIP more progress 2020-06-01 19:26:40 -04:00
Carlo Nucera da39092a39 WIP 2020-06-01 18:53:31 -04:00
Carlo Nucera 0e9438b6d3 Create new file-hash files 2020-06-01 17:32:40 -04:00
Carlo Nucera 0cb67ecbd3 Merge branch 'derivation-header-include-order' of github.com:Ericson2314/nix into validPathInfo-ca-proper-datatype 2020-06-01 17:13:11 -04:00
Carlo Nucera f4b89e11a4 Merge branch 'no-stringly-typed-derivation-output' of github.com:Ericson2314/nix into validPathInfo-ca-proper-datatype 2020-06-01 17:12:50 -04:00
Matthew Kenigsberg c254254a80 use Tree ctor 2020-06-01 12:32:17 -06:00
Matthew Kenigsberg 7680993506 Tree ctors 2020-06-01 09:01:37 -06:00
Matthew Kenigsberg ff1320b850 fetchOrSubstituteTree improvements
Caches tree in addition to lockedRef, and explicitly writes out the logic for different combinations of cached/uncached flakes and indirect/resolved/locked flakes. This eliminates uneccessary calls to lookupInFlakeCache, fetchTree, maybeLookupFlake, and flakeCache.push_back
2020-06-01 02:57:22 -06:00
Tobias Pflug eca1ff7a9f Add tests for lru-cache.hh 2020-05-31 01:05:05 +02:00
Nikola Knezevic fb38459d6e Ensure we restrict refspec interpretation while fetching
As `git fetch` may chose to interpret refspec to it's liking, ensure that we
only pass refs that begin with `refs/` as is, otherwise, prepend them with
`refs/heads`. Otherwise, branches named `heads/foo` (I know it's bad, but it's
allowed), would be fetched as `foo`, instead of `heads/foo`.
2020-05-30 12:33:38 +02:00
Nikola Knezevic 77007d4eab Improve ref validity checking in fetchGit
The previous regex was too strict and did not match what git was allowing. It
could lead to `fetchGit` not accepting valid branch names, even though they
exist in a repository (for example, branch names containing `/`, which are
pretty standard, like `release/1.0` branches).

The new regex defines what a branch name should **NOT** contain. It takes the
definitions from `refs.c` in https://github.com/git/git and `git help
check-ref-format` pages.

This change also introduces a test for ref name validity checking, which
compares the result from Nix with the result of `git check-ref-format --branch`.
2020-05-30 12:29:35 +02:00
Eelco Dolstra 89e0b3e2d6 Move substitution into Input::fetch()
Closes #3520.
2020-05-30 01:16:53 +02:00
Eelco Dolstra 0e7f77a59a Check revCount / lastModified input attributes if specified 2020-05-30 00:59:13 +02:00
Eelco Dolstra 950b46821f Remove TreeInfo
The attributes previously stored in TreeInfo (narHash, revCount,
lastModified) are now stored in Input. This makes it less arbitrary
what attributes are stored where.

As a result, the lock file format has changed. An entry like

    "info": {
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },
    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github"
    },

is now stored as

    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github",
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },

The 'Input' class is now a dumb set of attributes. All the fetcher
implementations subclass InputScheme, not Input. This simplifies the
API.

Also, fix substitution of flake inputs. This was broken since lazy
flake fetching started using fetchTree internally.
2020-05-30 00:44:11 +02:00
John Ericson fac0c2d54a Remove addToStore variant as requested by FIXME
The idea is it's always more flexible to consumer a `Source` than a
plain string, and it might even reduce memory consumption.

I also looked at `addToStoreFromDump` with its `// FIXME: remove?`, but
the worked needed for that is far more up for interpretation, so I
punted for now.
2020-05-29 17:02:32 -04:00
Ben Burdette 734283d636 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-05-29 09:51:37 -06:00
Eelco Dolstra 5633c0975b Factor out GitHub / GitLab commonality 2020-05-29 14:23:32 +02:00
Finn Behrens 5256bc77ca
add gitlab libfetcher 2020-05-28 23:00:08 +02:00
Carlo Nucera 6dd471ebf6 Fixing the result of merge 2020-05-28 12:14:36 -04:00
Carlo Nucera 4f597fb901 Merge branch 'master' of github.com:NixOS/nix into enum-class 2020-05-28 10:58:22 -04:00
John Ericson 5b4cd84bc2 Merge remote-tracking branch 'me/more-rust-ffi' into no-stringly-typed-derivation-output 2020-05-28 10:35:53 -04:00
John Ericson ef71caba29 Merge remote-tracking branch 'upstream/master' into more-rust-ffi 2020-05-28 10:31:46 -04:00
Eelco Dolstra f60ce4fa20
Merge pull request #3631 from andir/libutil-config-tests
Add unit tests for config.cc
2020-05-28 13:51:37 +02:00
Eelco Dolstra de141fcb79
Merge pull request #3455 from Ericson2314/enum-FileIngestionMethod
Replace some `bool recursive` with a new `FileIngestionMethod` enum
2020-05-28 13:50:06 +02:00
Eelco Dolstra 17ca997fc6 Merge remote-tracking branch 'origin/master' into flakes 2020-05-28 12:55:24 +02:00
Eelco Dolstra c3eff22f46 Merge branch 'store-visited' of https://github.com/mkenigs/nix into flakes 2020-05-28 12:15:39 +02:00
Eelco Dolstra 6286272371 nixpkgsFlakeRef(): Use locked nixpkgs 2020-05-28 12:13:13 +02:00
John Ericson 0f96f45061 Use FileIngestionMethod for nix hash
There was an enum there that matched in perfectly.
2020-05-27 23:50:11 -04:00
Matthew Bauer c66441a646 Rename some variables named “recursive” to “method”
This is much less confusing since recursive is no longer a boolean.
2020-05-27 13:21:26 -05:00
Matthew Bauer 7873fd175d Don’t use FileIngestionMethod for StorePathsCommand
This is a different recursive than used in makeFixedOutputPath.
2020-05-27 13:21:11 -05:00
Andreas Rammhold fc137d2f00
config.hh: Add documentation
Provides some general overview on the mechanics of Config/Setting and
comments for the public methods of Config.
2020-05-27 17:47:18 +02:00
Andreas Rammhold 9df3d8ccd7
tests/config.cc: add tests for Config::applyConfig 2020-05-27 17:47:18 +02:00
Andreas Rammhold e1b8c64c04
config.cc: extract parts of applyConfigFile into applyConfig
This moves the actual parsing of configuration contents into applyConfig
which applyConfigFile is then going to call. By changing this we can now
test the configuration file parsing without actually create a file on
disk.
2020-05-27 17:47:18 +02:00
Andreas Rammhold 93129cf1dd
Add unit tests for config.cc 2020-05-27 17:47:17 +02:00
Eelco Dolstra 66d3ac94c9 Merge pull request #3621 from gilligan/add-json-tests
Add unit tests for "json.hh"
2020-05-27 11:08:12 +02:00
Eelco Dolstra dae6a267a8
Merge pull request #3625 from gilligan/xml-writer-tests
Add unit tests for xml-writer
2020-05-27 11:07:53 +02:00
Carlo Nucera f3f520c14c Change syntax for CI 2020-05-26 12:51:28 -04:00
Carlo Nucera 89a5ac9d3b Merge remote-tracking branch 'john-ericson/more-rust-ffi' into no-stringly-typed-derivation-output 2020-05-26 12:31:26 -04:00
Carlo Nucera d49e65ba9d Merge remote-tracking branch 'john-ericson/enum-FileIngestionMethod' into no-stringly-typed-derivation-output 2020-05-26 12:30:48 -04:00
Carlo Nucera 0f3f901071 Merge remote-tracking branch 'origin/master' into more-rust-ffi 2020-05-26 11:46:42 -04:00
Carlo Nucera c2f33edd1f
Update src/libutil/rust-ffi.hh
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2020-05-26 11:43:18 -04:00
Carlo Nucera b90241ceb1 Change remaining bools with FileIngestionMethod 2020-05-26 11:32:41 -04:00
Carlo Nucera 6d73c10041 Merge remote-tracking branch 'origin/master' into enum-FileIngestionMethod 2020-05-26 11:14:08 -04:00
Tobias Pflug 4b388e8431 Add unit tests for xml-writer 2020-05-25 18:34:55 +02:00
Tobias Pflug c284700867 Add unit tests for "json.hh" 2020-05-25 11:57:45 +02:00
Tobias Pflug ecc5c90dfc Add unit tests for hashing functions 2020-05-25 11:50:41 +02:00
Matthew Kenigsberg 8d67794da1 handle circular flake dependencies in list-inputs 2020-05-21 17:06:11 -06:00
Ben Burdette b7057fa627 remove error-demo from make; clean up comment 2020-05-21 16:04:18 -06:00
Ben Burdette 0e49de6a2b position for stdin, string; (string) for trace; fix tests 2020-05-21 14:28:45 -06:00
Ben Burdette 6a420d672c print LOC for stdin, string args 2020-05-20 22:18:26 -06:00
Ben Burdette 85ce455b85 get code lines from the nix file 2020-05-20 17:25:02 -06:00
Tobias Pflug a73a820a5d Add unit testes for url.cc
This adds tests for

- parseURL
- percentDecode
- decodeQuery
2020-05-20 16:37:35 +02:00
Eelco Dolstra 5ef64f05e6 Cleanup 2020-05-18 15:50:29 +02:00
Eelco Dolstra 0ed946aa61 Merge branch 'wait-for-builders' of https://github.com/serokell/nix 2020-05-18 13:48:45 +02:00
Matthew Kenigsberg c4beded32e rm includes 2020-05-16 11:19:41 -06:00
Matthew Kenigsberg ba7d7ed2e3 Create bashInteractive InstallableFlake 2020-05-16 11:03:06 -06:00
Matthew Kenigsberg 0858793604 Call lockFlake once and store in _lockedFlake 2020-05-16 11:03:06 -06:00
Matthew Kenigsberg 04821bc171 use flake's nixpkgs to find bashInteractive 2020-05-16 11:03:06 -06:00
Matthew Kenigsberg 8fbc8540d3 use nixpkgs#bashInteractive for dev-shell 2020-05-16 11:03:06 -06:00
Ben Burdette 92123c6c79 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-05-15 07:00:36 -06:00
Eelco Dolstra 5f64655ff4 Move registry-related commands from 'nix flake' to 'nix registry'
This makes 'nix flake' less cluttered and more consistent (it's only
subcommands that operator on a flake). Also, the registry is not
inherently flake-related (e.g. fetchTree could also use it to remap
inputs).
2020-05-15 14:38:10 +02:00
Ben Burdette 19694aa213 fix compile errors 2020-05-14 12:28:18 -06:00
Ben Burdette 4daccb279c formatting 2020-05-14 10:28:17 -06:00
Alexander Bantyev 183dd28266
Don't lock a user while doing remote builds 2020-05-14 17:00:54 +03:00
Ben Burdette ef9dd9f9bc formatting and a few minor changes 2020-05-13 15:56:39 -06:00
Ben Burdette c79d4addab consistent capitalization 2020-05-13 10:02:18 -06:00
Ben Burdette bfca5fc395 change status messages to info level 2020-05-13 09:52:36 -06:00
Eelco Dolstra 849d3968db
Update src/libfetchers/git.cc
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
2020-05-13 10:41:21 +02:00
Ben Burdette ecbb8e9c0a no blank line if no LOC 2020-05-12 14:41:30 -06:00
Ben Burdette 960d4362ed hint only 2020-05-12 13:54:18 -06:00
Ben Burdette 72ecccee57 convert to logWarning format 2020-05-12 12:19:34 -06:00
Ben Burdette 19cffc29c9 remove unused extra json fields 2020-05-12 12:09:12 -06:00
Ben Burdette 2a19bf8619 move pos to the first arg, to indicate its not used in a fmt template 2020-05-12 11:27:37 -06:00
Ben Burdette ec870b9c85 new pos format for more errors 2020-05-12 10:52:26 -06:00
Eelco Dolstra 215f09d765
Merge pull request #3587 from NixOS/bash-completion
Generic shell completion support for the 'nix' command
2020-05-12 18:26:13 +02:00
Eelco Dolstra ebc024df22
Show hint how to enable experimental features 2020-05-12 15:47:09 +02:00
Eelco Dolstra 268ecf5b3f
nix: Don't require --experimental-features=nix-command for some subcommands 2020-05-12 15:47:09 +02:00
Eelco Dolstra b8b2dbf272 Fix InstallableCommand 2020-05-12 11:53:32 +02:00
Eelco Dolstra 437614b479 Fix macOS build
macOS doesn't have GLOB_ONLYDIR.
2020-05-12 11:09:09 +02:00
Ben Burdette 7c3138844c more pos reporting 2020-05-11 17:34:57 -06:00
Ben Burdette 631642c5b4 new format for pos 2020-05-11 16:58:08 -06:00
Ben Burdette b93c1bf3d6 fixes to merged code 2020-05-11 15:52:15 -06:00
Ben Burdette 59b1f5c701 Merge branch 'master' into errors-phase-2 2020-05-11 14:35:30 -06:00
Eelco Dolstra 649c2db308 nix flake: Add completion support 2020-05-11 22:10:33 +02:00
Eelco Dolstra 27d34ef770 When completing flakerefs, only return directories 2020-05-11 22:04:13 +02:00
Ben Burdette 536bbf53e1 comments and cleanup 2020-05-11 13:58:38 -06:00
Eelco Dolstra 259ff74bde Add completion for installables
This completes flakerefs using the registry (e.g. 'nix<TAB>' => 'nix
nixpkgs') and flake output attributes by evaluating the flake
(e.g. 'dwarffs#nix<TAB>' => 'dwarffs#nixosModules').
2020-05-11 21:49:02 +02:00
Eelco Dolstra e917332d63 Shut up warnings while running completers 2020-05-11 21:38:17 +02:00
Ben Burdette 958e81987b switch from printError warnings to logWarnings 2020-05-11 13:02:16 -06:00
Domen Kožar 5bdb67c843
Merge pull request #3568 from kolloch/outputHashModeError
libstore/build.cc: more explicit error about form of output
2020-05-11 18:14:32 +02:00
Domen Kožar 1d8144e36b
Update src/libstore/build.cc 2020-05-11 18:14:23 +02:00
Eelco Dolstra 4c3c638a05 Cleanup 2020-05-11 15:57:45 +02:00
Eelco Dolstra 0884f180f5 Simplify 2020-05-10 21:50:32 +02:00
Eelco Dolstra e0c19ee620 Add completion for paths 2020-05-10 21:35:07 +02:00
Eelco Dolstra 91ddee6bf0 nix: Implement basic bash completion 2020-05-10 20:32:21 +02:00
Matthew Kenigsberg 73ee1afffe Reorder to build
This reverts commit 883948d7a0add742ccae58e9845d769a8064371c.
2020-05-09 14:42:32 -06:00
Matthew Kenigsberg 9f4cfbb2e7 Refactor installables
InstallableValue has children InstallableFlake and InstallableAttrPath, but InstallableFlake was overriding toDerivations, and usage was changed so that InstallableFlake didn't need cmd. So these changes were made:
InstallableValue::toDerivations() -> InstalllableAttrPath::toDerivations()
InstallableValue::cmd -> InstallableAttrPath::cmd

InstallableValue uses state instead of cmd

toBuildables() and toDerivations() were made abstract
2020-05-09 14:42:32 -06:00
Matthew Kenigsberg bf81dd40e9 InstallableExpr unused 2020-05-09 10:16:00 -06:00
Benjamin Hipple 146f9c114f doc: consistently refer to 'fixed-output' with a dash
General cleanup that makes it easier to search for the term.
2020-05-09 10:58:43 -04:00
Ben Burdette 55eb717148 add pos to errorinfo, remove from hints 2020-05-08 18:18:28 -06:00
Eelco Dolstra d3d8186c9c
Merge pull request #3571 from gilligan/nix-unit-testing
Add unit tests
2020-05-08 17:02:25 +02:00
Tobias Pflug 181a47d884 Enable toLower umlauts test
Update comment and enable the test
2020-05-08 15:13:55 +02:00
Tobias Pflug 2191141274 Enable baseNameOf test
Add note about removal of trailing slashes in the doc comment of
baseNameOf and enabled the test.
2020-05-08 15:07:40 +02:00
Tobias Pflug e3df9c2a6e Enable dirOf test
Adjusted the doc comment for `dirOf` to reflect the implementation
behavior.
2020-05-08 15:03:44 +02:00
Eelco Dolstra 5b8883faac
configure: Look for gtest 2020-05-08 12:09:37 +02:00
Eelco Dolstra ca657525b8
Don't install unit tests 2020-05-08 12:03:27 +02:00
Eelco Dolstra 7898cdb75a
make check: Run unit tests 2020-05-08 11:49:40 +02:00
Eelco Dolstra 72b9d971bc
Fix warning 2020-05-08 11:35:57 +02:00
Eelco Dolstra 7cc7cef950
Move unit tests to sr/libutil/tests, use mk make rules 2020-05-08 11:34:09 +02:00
Alexander Bantyev 772e5db828
Mention build users in the 'waiting for' message 2020-05-08 12:29:00 +03:00
Alexander Bantyev 14073fb76b
Don't block while waiting for build users 2020-05-08 12:22:39 +03:00
Ben Burdette 1b801cec40 pretending to be const 2020-05-07 16:43:36 -06:00
Tobias Pflug 1f3602a2c9 Remove replaceInSet
The function isn't being used anywhere so it seems safe to remove
2020-05-07 18:15:13 +02:00
Eelco Dolstra 14a3a62bfc
Update src/nix/search.cc
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
2020-05-07 12:13:16 +02:00
Ben Burdette e3901638b5 todo removal 2020-05-06 15:01:13 -06:00
Ben Burdette e76ad2e48a implement SysError errno handling 2020-05-06 14:07:20 -06:00
Eelco Dolstra 1ad71bc62c Remove support for old lockfiles and the epoch/uri attributes 2020-05-06 17:48:18 +02:00
Eelco Dolstra b0e9b07e80 Remove obsolete FIXME 2020-05-06 17:40:01 +02:00
Eelco Dolstra ff394ff206 Remove the nixpkgs.<attr> compatibility hack
Since we've changed a lot of things in the 'nix' command (e.g. rename
'nix run') there is not much point in keeping this around.
2020-05-06 17:20:23 +02:00
Domen Kožar 672985531c
Merge pull request #3570 from Mic92/nix-search
nix/search: no error for empty search results if json is enabled
2020-05-06 16:14:58 +02:00
Jörg Thalheim 85c1932c94
nix/search: no error for empty search results if json is enabled
- result list will be always empty if --json is passed
- for scripts an empty search result is not really an error,
  we rather want to distinguish between evaluation errors and empty results
2020-05-06 14:43:54 +01:00
Eelco Dolstra 2f8ee4578f
Merge remote-tracking branch 'origin/master' into flakes 2020-05-06 12:01:40 +02:00
Eelco Dolstra 74a1bfdcab
Merge pull request #3546 from guibou/nix_readfile_on_0_sized_files
builtins.readFile: do not truncate content
2020-05-06 11:33:55 +02:00
Eelco Dolstra 272c4ba36d
Merge pull request #3557 from Ma27/nix-repl-help
Improve help-message for nix-repl
2020-05-06 11:25:47 +02:00
Peter Kolloch 9be46859a9 libstore/build.cc: more explicit about form of output
Be more explicit about why we expect a regular file as output
when outputHashMode=flat for a fixed output derivation.
2020-05-06 11:21:12 +02:00
Eelco Dolstra 02c5914ea4
Merge pull request #3562 from pikajude/master
Use fragment size for autoGC capacity calculation
2020-05-06 11:19:30 +02:00
Eelco Dolstra fd4911269f
Revert "Merge pull request #3558 from LnL7/ssh-ng-stderr"
This reverts commit 3ebfbecdd1, reversing
changes made to c089c52d5f.

https://github.com/NixOS/nix/pull/3558
2020-05-06 10:54:18 +02:00
Eelco Dolstra 6f3244ce45 Merge remote-tracking branch 'origin/master' into flakes 2020-05-05 18:59:33 +02:00
Eelco Dolstra 909b4a8820 nix doctor: Consistency 2020-05-05 15:27:47 +02:00
Eelco Dolstra f132d82a79 nix --help: Group commands 2020-05-05 15:18:23 +02:00
Alexander Bantyev 04967dee9d
Wait for build users when none are available 2020-05-05 13:04:36 +03:00
Ben Burdette 7ffb5efdbc appending to hints; remove _printError 2020-05-04 16:19:57 -06:00
Ben Burdette f30de61578 add normaltxt, yellowify->yellowtxt 2020-05-04 16:19:20 -06:00
Jude Taylor e2fc575c61 nix auto-gc: use fragment size 2020-05-04 14:42:06 -07:00
Ben Burdette 8c8f2b74ec log as warning 2020-05-04 14:44:42 -06:00
Ben Burdette afaa541013 affinity operator<< 2020-05-04 14:44:00 -06:00
Eelco Dolstra a721a0b114 Flag: Use designated initializers 2020-05-04 22:40:19 +02:00
Ben Burdette 9c5ece44a7 separate msgs instead of appending to what() 2020-05-04 13:46:15 -06:00
Eelco Dolstra 3ebfbecdd1
Merge pull request #3558 from LnL7/ssh-ng-stderr
remote-store: don't log raw stderr by default
2020-05-04 13:02:33 +02:00
Ben Burdette ab6f0b9641 convert some printError calls to logError 2020-05-03 08:01:25 -06:00
Daiderd Jordan 4769eea5e2
logging: handle build log lines in simple logger
The raw stderr output isn't logged anymore so the build logs need to be
printed by the default logger in order for the old commands like
nix-build to still show build output.
2020-05-02 23:40:53 +02:00
Daiderd Jordan f16e24f95e
remote-store: don't log raw stderr by default
For remote stores the log messages are already forwarded as structured
STDERR_RESULT messages so the old format is duplicate information.  But
still included with -vvv since it could be useful for debugging
problems.

    $ nix build -L /nix/store/nl71b2niws857ffiaggyrkjwgx9jjzc0-foo.drv --store ssh-ng://localhost
    Hello World!
    foo> Hello World!
    [1/0/1 built] building foo

Fixes #3556
2020-05-02 23:40:50 +02:00
Maximilian Bosch 2aeb874e83
Improve help-message for nix-repl
* Remove obsolete `printHelp` function
* Add an example to demonstrate how to list all available commands
  within the REPL
2020-05-01 23:32:01 +02:00
Ben Burdette 4b99c09f5c convert some errors 2020-05-01 14:32:06 -06:00
Eelco Dolstra c089c52d5f Fix build 2020-05-01 12:42:39 +02:00
Matthew Kenigsberg d6b4047c2f rename nix run to nix shell and nix app to nix run
(cherry picked from commit 5d8504b978)
2020-05-01 12:17:36 +02:00
Eelco Dolstra 941f95284a Merge remote-tracking branch 'origin/master' into flakes 2020-05-01 11:59:56 +02:00
Ben Burdette a3030e3c31 fix error calls 2020-04-30 17:56:26 -06:00
Ben Burdette f5d3215c87 logError 2020-04-30 16:31:47 -06:00
Ben Burdette 171b4ce85c typo 2020-04-30 09:57:01 -06:00
Eelco Dolstra 0135fd6ec4
nix dev-shell: Unset shellHook
This avoids inheriting the caller's shellHook, which can happen when
running a dev-shell inside a dev-shell.
2020-04-30 14:47:56 +02:00
Eelco Dolstra efe6c186ea
nix dev-shell: Support structured attrs
Tested against https://github.com/NixOS/nixpkgs/pull/72074.

Fixes #3540.
2020-04-30 14:47:47 +02:00
Eelco Dolstra 2fcfc6c2c6
nix dev-shell: Refactor script for getting the environment 2020-04-30 13:05:29 +02:00
Domen Kožar df8e9d691c
Merge pull request #3548 from Ma27/fetchtarball-pos
Fix displaying error-position in `builtins.fetch{Tree,Tarball}`
2020-04-30 10:33:12 +02:00
Ben Burdette 39ff80d031 errorinfo constructor test 2020-04-29 18:57:05 -06:00
Matthew Kenigsberg 5d8504b978 rename nix run to nix shell and nix app to nix run 2020-04-29 15:45:10 -06:00
Maximilian Bosch d1229859c2
Fix displaying error-position in builtins.fetch{Tree,Tarball}
Without dereferencing this pointer, you'd get an error like this:

```
error: unsupported argument 'abc' to 'fetchTarball', at 0x13627e8
```
2020-04-29 22:53:39 +02:00
Guillaume Bouchard 2e5be2a749 StringSink pre allocate
When used with `readFile`, we have a pretty good heuristic of the file
size, so `reserve` this in the `string`. This will save some allocation
/ copy when the string is growing.
2020-04-29 18:44:01 +02:00
Guillaume Bouchard 7afcb5af98 Remove the drain argument from readFile
Now it is always `drain` (see previous commit).
2020-04-29 18:43:45 +02:00
Ben Burdette e2f61263eb uncrustify formatting 2020-04-29 10:14:32 -06:00
Eelco Dolstra 9570036146 nix copy: Build derivations
Fixes

  $ nix copy .#hydraJobs.vendoredCrates --to /tmp/nix
  error: path '/nix/store/...' is not valid
2020-04-29 15:51:45 +02:00
Eelco Dolstra 9c4e05766b nix copy: Move --from / --to check
This means you now get an error message *before* stuff gets built:

  $ nix copy .#hydraJobs.vendoredCrates
  error: you must pass '--from' and/or '--to'
  Try 'nix --help' for more information.
2020-04-29 15:50:59 +02:00
Antoine Eiche ca93b26db6 Only call grantpt on MacOS systems
The commit 3cc1125595 adds a `grantpt`
call on the builder pseudo terminal fd. This call is actually only
required for MacOS, but it however requires a RW access to /dev/pts
which is only RO bindmounted in the Bazel Linux sandbox. So, Nix can
not be actually run in the Bazel Linux sandbox for unneeded reasons.
2020-04-29 15:43:20 +02:00
Eelco Dolstra 70bcd6a55c Evaluation cache: Don't barf in read-only mode
Fixes

  $ nix copy
  warning: Git tree '/home/eelco/Dev/nix-flake' is dirty
  nix: src/nix/installables.cc:348: std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, nix::FlakeRef, nix::InstallableValue::DerivationInfo> nix::InstallableFlake::toDerivation(): Assertion `state->store->isValidPath(drvPath)' failed.
  Aborted (core dumped)
2020-04-29 15:42:53 +02:00
Guillaume Bouchard 5a34a473dd builtins.readFile: do not truncate content
This closes #3026 by allowing `builtins.readFile` to read a file with a
wrongly reported file size, for example, files in `/proc` may report a
file size of 0. Reading file in `/proc` is not a good enough motivation,
however I do think it just makes nix more robust by allowing more file
to be read.  Especially, I do considerer the previous behavior to be
dangerous because nix was previously reading truncated files. Examples
of file system which incorrectly report file size may be network file
system or dynamic file system (for performance reason, a dynamic file
system such as FUSE may generate the content of the file on demand).

```
nix-repl> builtins.readFile "/proc/version"
""
```

With this commit:

```
nix-repl> builtins.readFile "/proc/version"
"Linux version 5.6.7 (nixbld@localhost) (gcc version 9.3.0 (GCC)) #1-NixOS SMP Thu Apr 23 08:38:27 UTC 2020\n"
```

Here is a summary of the behavior changes:

- If the reported size is smaller, previous implementation
was silently returning a truncated file content. The new implementation
is returning the correct file content.

- If a file had a bigger reported file size, previous implementation was
failing with an exception, but the new implementation is returning the
correct file content. This change of behavior is coherent with this pull
request.

Open questions

- The behavior is unchanged for correctly reported file size, however
performances may vary because it uses the more complex sink interface.
Considering that sink is used a lot, I don't think this impacts the
performance a lot.
- `builtins.readFile` on an infinite file, such as `/dev/random` may
fill the memory.
- it does not support adding file to store, such as `${/proc/version}`.
2020-04-29 14:50:52 +02:00
Eelco Dolstra 5ada0831cf Merge remote-tracking branch 'origin/master' into flakes 2020-04-29 13:32:27 +02:00
Ben Burdette 22e6490311 Error classname as name 2020-04-28 21:06:08 -06:00
Eelco Dolstra 06849c3090
Merge pull request #3542 from mkenigs/gcroots
Set GCROOT to store path to prevent garbage collection
2020-04-28 21:04:06 +02:00
Matthew Kenigsberg 6d40fe573c rename to NIX_GCROOT 2020-04-28 11:18:54 -06:00
Eelco Dolstra 52a3ca823d Tweak warning message 2020-04-28 17:56:01 +02:00
Eelco Dolstra 6a8cba83bb Merge branch 'nix-env-warn-unmatched' of https://github.com/lheckemann/nix 2020-04-28 17:45:25 +02:00
Ben Burdette e51a757720 astyle format 2020-04-27 15:15:08 -06:00
Eelco Dolstra 6521c92ce8 Improve path:// handling
In particular, doing 'nix build /path/to/dir' now works if
/path/to/dir is not a Git tree (it only has to contain a flake.nix
file).

Also, 'nix flake init' no longer requires a Git tree (but it will do a
'git add flake.nix' if it's a Git tree)
2020-04-27 22:53:11 +02:00
Eelco Dolstra 829dcb35d5 flake-template.nix: Add defaultPackage 2020-04-27 22:52:49 +02:00
Alyssa Ross c05e20daa1
Fix long paths permanently breaking GC
Suppose I have a path /nix/store/[hash]-[name]/a/a/a/a/a/[...]/a,
long enough that everything after "/nix/store/" is longer than 4096
(MAX_PATH) bytes.

Nix will happily allow such a path to be inserted into the store,
because it doesn't look at all the nested structure.  It just cares
about the /nix/store/[hash]-[name] part.  But, when the path is deleted,
we encounter a problem.  Nix will move the path to /nix/store/trash, but
then when it's trying to recursively delete the trash directory, it will
at some point try to unlink
/nix/store/trash/[hash]-[name]/a/a/a/a/a/[...]/a.  This will fail,
because the path is too long.  After this has failed, any store deletion
operation will never work again, because Nix needs to delete the trash
directory before recreating it to move new things to it.  (I assume this
is because otherwise a path being deleted could already exist in the
trash, and then moving it would fail.)

This means that if I can trick somebody into just fetching a tarball
containing a path of the right length, they won't be able to delete
store paths or garbage collect ever again, until the offending path is
manually removed from /nix/store/trash.  (And even fixing this manually
is quite difficult if you don't understand the issue, because the
absolute path that Nix says it failed to remove is also too long for
rm(1).)

This patch fixes the issue by making Nix's recursive delete operation
use unlinkat(2).  This function takes a relative path and a directory
file descriptor.  We ensure that the relative path is always just the
name of the directory entry, and therefore its length will never exceed
255 bytes.  This means that it will never even come close to AX_PATH,
and Nix will therefore be able to handle removing arbitrarily deep
directory hierachies.

Since the directory file descriptor is used for recursion after being
used in readDirectory, I made a variant of readDirectory that takes an
already open directory stream, to avoid the directory being opened
multiple times.  As we have seen from this issue, the less we have to
interact with paths, the better, and so it's good to reuse file
descriptors where possible.

I left _deletePath as succeeding even if the parent directory doesn't
exist, even though that feels wrong to me, because without that early
return, the linux-sandbox test failed.

Reported-by: Alyssa Ross <hi@alyssa.is>
Thanks-to: Puck Meerburg <puck@puckipedia.com>
Tested-by: Puck Meerburg <puck@puckipedia.com>
Reviewed-by: Puck Meerburg <puck@puckipedia.com>
2020-04-27 20:50:17 +00:00
Matthew Kenigsberg 9e95b95a5d comment 2020-04-27 13:18:26 -06:00
Matthew Kenigsberg a3bc695e7d Set GCROOT to store path to prevent garbage collection 2020-04-27 11:22:20 -06:00
Eelco Dolstra b51dff431c Improve error message when an argument is not a flake 2020-04-27 18:55:20 +02:00
Eelco Dolstra b4e23dcd9e nix search: Search legacyPackages recursively 2020-04-27 16:29:26 +02:00
Ben Burdette 1ff42722ce error.hh 2020-04-26 14:47:41 -06:00
Ben Burdette d4fd7b543e print dashes instead of empty name string 2020-04-25 12:05:26 -06:00
Linus Heckemann f59404e1a6 nix-env: refactor uninstallDerivations
Reduces the number of store queries it performs. Also prints a warning
if any of the selectors did not match any installed derivations.

UX Caveats:
- Will print a warning that nothing matched if a previous selector
  already removed the path
- Will not do anything if no selectors were provided (no change from
  before).

Fixes #3531
2020-04-25 16:30:42 +02:00
Ben Burdette cdac083dc5 don't print blank lines for blank description 2020-04-24 21:40:13 -06:00
Ben Burdette d8d4844b88 all things error to error.hh 2020-04-24 14:57:51 -06:00
Ben Burdette d9632765a8 add has_value check; remove obslete friend class 2020-04-24 12:44:23 -06:00
Eelco Dolstra 0469795978 nix search: Show version 2020-04-24 14:42:17 +02:00
Ben Burdette 833501f6f1 'what' string 2020-04-23 15:55:34 -06:00
Eelco Dolstra ef4d3fc111 Merge remote-tracking branch 'origin/master' into flakes 2020-04-23 15:16:18 +02:00
Ben Burdette 3bc9155dfc a few more 'format's rremoved 2020-04-22 15:00:11 -06:00
Eelco Dolstra c9d0cf7e02
Don't include error.hh in util.hh to prevent header bloat 2020-04-22 15:29:27 +02:00
Eelco Dolstra 16e3bf4537
Merge branch 'error-format' of https://github.com/bburdette/nix 2020-04-22 15:29:10 +02:00
Eelco Dolstra 2ea4d45449
Path fetcher: Fix store path name
(cherry picked from commit c7af247bea)
2020-04-22 15:27:06 +02:00
Eelco Dolstra c7af247bea Path fetcher: Fix store path name 2020-04-22 10:15:32 +02:00
Dustin DeWeese c0d940978a Replace select() with poll() to allow waiting on more than FD_SETSIZE fds 2020-04-21 16:21:28 -07:00
Ben Burdette e4fb9a3849 remove 'format' from Error constructor calls 2020-04-21 17:07:07 -06:00
Ben Burdette d3052197fe add ErrorInfo to BaseError 2020-04-21 13:25:41 -06:00
Eelco Dolstra 8c75621da6 Fix typo 2020-04-20 15:28:56 +02:00
Eelco Dolstra b69323f8c9 Revive 'nix search'
It uses the evaluation cache now rather than the ad hoc JSON cache.
2020-04-20 15:27:09 +02:00
Eelco Dolstra 42a12f9232 Move eval-cache.{cc,hh} 2020-04-20 13:14:59 +02:00
Eelco Dolstra 539a9c1c5f Get rid of the old eval cache 2020-04-20 13:13:52 +02:00
Ben Burdette 15e9564fd1 logEI for tunnelLogger and progressbar 2020-04-19 17:16:51 -06:00
Eelco Dolstra 0725ab2fd7 Store more stuff in the evaluation cache
In particular, we store whether an attribute failed to evaluate (threw
an exception) or was an unsupported type. This is to ensure that a
repeated 'nix flake show' never has to evaluate anything, so it can
execute without fetching the flake.

With this, 'nix flake show nixpkgs/nixos-20.03 --legacy' executes in
0.6s (was 3.4s).
2020-04-19 23:07:06 +02:00
Eelco Dolstra 3738bcb05e Eval cache: Don't replace real attributes with placeholders 2020-04-18 15:12:31 +02:00
Domen Kožar 25ed842725
Merge pull request #3502 from NixOS/more-pos
pass Pos to forceValue to improve infinite recursion error
2020-04-18 14:05:21 +02:00
Ben Burdette 4697552948 demoing other error levels than warn/error; rename line and file fields in errPos 2020-04-17 15:50:46 -06:00
Eelco Dolstra 69cb9f7eee Wrap eval cache creation in a giant transaction
This speeds up the creation of the cache for the nixpkgs flake from
21.2s to 10.2s. Oddly, it also speeds up querying the cache
(i.e. running 'nix flake show nixpkgs/nixos-20.03 --legacy') from 4.2s
to 3.4s.

(For comparison, running with --no-eval-cache takes 9.5s, so the
overhead of building the SQLite cache is only 0.7s.)
2020-04-17 23:17:21 +02:00
Ben Burdette 3d5b1032a1 logError, logWarning; Logger functions; switch to Verbosity enum 2020-04-17 15:07:44 -06:00
Eelco Dolstra aaa109565e Use a more space/time-efficient representation for the eval cache 2020-04-17 23:04:21 +02:00
Eelco Dolstra bdb3226607 Add flag to disable the eval cache 2020-04-17 14:30:04 +02:00
Eelco Dolstra aa34c0ef51 nix flake show: Speed up eval cache bigly
In the fully cached case for the 'nixpkgs' flake, it went from 101s to
4.6s. Populating the cache went from 132s to 17.4s (which could
probably be improved further by combining INSERTs).
2020-04-17 13:57:02 +02:00
Eelco Dolstra 9ea4f93f88 nix flake show: Support apps 2020-04-17 01:21:24 +02:00
Eelco Dolstra a6c4fd044c Hide progress bar on exit 2020-04-17 01:13:13 +02:00
Eelco Dolstra 12b7eefbc5 nix flake show: Use evaluation cache 2020-04-17 01:02:29 +02:00
Eelco Dolstra 7a9687ba30 SQLiteStmt: Use std::string_view 2020-04-17 01:00:56 +02:00
Eelco Dolstra 3b489e8843 Add 'nix flake show' command 2020-04-16 19:52:39 +02:00
Eelco Dolstra 29043e7e9e Fix 2020-04-16 19:01:49 +02:00
Ben Burdette 12814806ef iomanip no longer needed 2020-04-16 10:48:15 -06:00
Eelco Dolstra c277231b7d Use RootValue 2020-04-16 18:33:34 +02:00
Eelco Dolstra f89349f07e Merge remote-tracking branch 'origin/master' into flakes 2020-04-16 18:33:10 +02:00
Eelco Dolstra 0858738355 Merge remote-tracking branch 'origin/master' into flakes 2020-04-16 18:27:37 +02:00
Eelco Dolstra efaffaa9d1 Use Logger::stdout()
(cherry picked from commit 8f41847394)
2020-04-16 18:14:01 +02:00
Eelco Dolstra 67a5941472 Logger: Add method for writing to stdout
Usually this just writes to stdout, but for ProgressBar, we need to
clear the current line, write the line to stdout, and then redraw the
progress bar.

(cherry picked from commit 696c026006)
2020-04-16 18:03:38 +02:00
Eelco Dolstra fcd048a526 Use RootValue 2020-04-16 18:02:59 +02:00
Ben Burdette 96262e744e switch to structs, which don't need public: 2020-04-16 09:55:38 -06:00
Eelco Dolstra 9f46f54de4 JSONSax: Use a RootValue
More #3377.
2020-04-16 17:30:18 +02:00
Eelco Dolstra 10e17eaa58 ValueMap, VectorVector: Use traceable_allocator
We want to *trace* the 'Value *' arrays, not garbage-collect them!
Otherwise the vectors/maps can end up pointing to nowhere.

Fixes #3377. Closes #3384.
2020-04-16 17:30:13 +02:00
Eelco Dolstra b3e5eea4a9 Add function to allocate a Value in traceable memory 2020-04-16 17:30:05 +02:00
Eelco Dolstra 1290411c2d fetchMercurial: Use inputFromAttrs() 2020-04-16 17:29:30 +02:00
Eelco Dolstra 8f41847394 Use Logger::stdout() 2020-04-16 13:47:59 +02:00
Eelco Dolstra 696c026006 Logger: Add method for writing to stdout
Usually this just writes to stdout, but for ProgressBar, we need to
clear the current line, write the line to stdout, and then redraw the
progress bar.
2020-04-16 13:47:59 +02:00
Domen Kožar b865b5b40c
pass Pos to forceValue to improve infinite recursion error 2020-04-16 12:32:07 +02:00
Ben Burdette 057e5b6b2e move implementation to cc 2020-04-15 10:09:43 -06:00
Ben Burdette adf03b0b8e Merge branch 'initializer-style' into error-format 2020-04-15 10:06:20 -06:00
Eelco Dolstra 2f9789c2e6
Merge pull request #3492 from andir/nix-build-gc-free
SourceExprCommand: allocate the vSourceExpr via uncollectable memory
2020-04-15 13:01:04 +02:00
Eelco Dolstra a118293bd0
Merge pull request #3458 from zimbatm/nix-user-conf-dir
NIX_USER_CONF_FILES
2020-04-15 13:00:28 +02:00
Jonas Chevalier 895516cadf
add NIX_USER_CONF_FILES
Motivation: maintain project-level configuration files.

Document the whole situation a bit better so that it corresponds to the
implementation, and add NIX_USER_CONF_FILES that allows overriding
which user files Nix will load during startup.
2020-04-14 18:45:06 +02:00
Eelco Dolstra 3729df34da Make Registry::read() more robust 2020-04-14 17:25:39 +02:00
Eelco Dolstra c0c2cb871d Merge remote-tracking branch 'origin/master' into flakes 2020-04-14 13:02:55 +02:00
Andreas Rammhold d2c371927e SourceExprCommand: allocate the vSourceExpr via uncollectable memory
Previously the memory would occasionally be collected during eval since
the GC doesn't consider the member variable as alive / doesn't scan the
region of memory where the pointer lives.

By using the traceable_allocator<T> allocator provided by Boehm GC we
can ensure the memory isn't collected. It should be properly freed when
SourceExprCommand goes out of scope.
2020-04-13 21:23:54 +02:00
Daiderd Jordan 4d9db420ff
never use /var/folders for TMPDIR on darwin
This doesn't just cause problems for nix-store --serve but also results
in certain build failures. Builds that use unix domain sockets in their
tests often fail because the /var/folders prefix already consumes more
than half of the maximum length of socket paths.

    struct sockaddr_un {
       sa_family_t sun_family;               /* AF_UNIX */
       char        sun_path[108];            /* Pathname */
    };
2020-04-12 09:57:22 +02:00
Domen Kožar ea2148f47c
Merge pull request #2688 from tollb/fix/build_check_keep_failed_sandbox_perms
Fix nix-build --check -K in sandbox w/o root
2020-04-11 20:03:17 +02:00
Domen Kožar fc144242d5
Merge pull request #3447 from DavHau/improve-tofile-error-msg
improve toFile error message when containing potential drv path
2020-04-11 13:12:11 +02:00
DavHau fc14213d2d improve toFile error message when containing potential drv path 2020-04-11 10:54:26 +00:00
Bruce Toll 8132d0a12e Fix nix-build --check -K in sandbox w/o root
Temporarily add user-write permission to build directory so that it
can be moved out of the sandbox to the store with a .check suffix.

This is necessary because the build directory has already had its
permissions set read-only, but write permission is required
to update the directory's parent link to move it out of the sandbox.

Updated the related --check "derivation may not be deterministic"
messages to consistently use the real store paths.

Added test for non-root sandbox nix-build --check -K to demonstrate
issue and help prevent regressions.
2020-04-10 16:23:10 -04:00
Eelco Dolstra e5ea01c1a8
Remove flake 'edition' field
Future editions of flakes or the Nix language can be supported by
renaming flake.nix (e.g. flake-v2.nix). This avoids a bootstrap
problem where we don't know which grammar to use to parse
flake*.nix. It also allows a project to support multiple flake
editions, in theory.
2020-04-10 10:24:09 +02:00
Domen Kožar db25a6d7bb
Merge pull request #2689 from tollb/fix/delete_tmp_dir_when_build_check_ok
Delete temporary directory on successful build
2020-04-10 09:47:57 +02:00
Bruce Toll 16a4864759 Delete temporary directory on successful build
With --check and the --keep-failed (-K) flag, the temporary directory
was being retained regardless of whether the build was successful and
reproducible.  This removes the temporary directory, as expected, on
a reproducible check build.

Added tests to verify that temporary build directories are not
retained unnecessarily, particularly when using --check with
--keep-failed.
2020-04-09 16:37:41 -04:00
Philipp Middendorf 04bedda0b6 gc.cc: Ignore hidden files in temproots 2020-04-09 17:05:29 +02:00
Eelco Dolstra 3aaceeb7e2
Fix build 2020-04-09 16:00:43 +02:00
Eelco Dolstra d103c79144
Merge remote-tracking branch 'origin/master' into flakes 2020-04-09 15:54:21 +02:00
Domen Kožar a693a9fa4b
Attach pos to if expression errors 2020-04-09 09:45:15 +02:00
Nikola Knezevic 7867685dcd after flake rebase 2020-04-08 22:26:57 +02:00
Nikola Knezevic f5095594e7 datatransfer.{cc,hh} -> filetransfer.{cc,hh} 2020-04-08 22:26:57 +02:00
Nikola Knezevic c330109bfa DataTransfer -> FileTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic 7848372b0f Add upload method 2020-04-08 22:26:57 +02:00
Nikola Knezevic a0c5931208 actDownload -> actDataTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic c4c1ae0a00 DownloadError -> DataTransferError 2020-04-08 22:26:57 +02:00
Nikola Knezevic 213d124277 DownloadItem -> TransferItem 2020-04-08 22:26:57 +02:00
Nikola Knezevic cd391206e6 {get,make,new}Downloader -> DataTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic 2df2741ec6 enqueueDownload -> enqueueDataTransfer 2020-04-08 22:26:57 +02:00
Nikola Knezevic 142ed7fe45 DownloadResult -> DataTransferResult 2020-04-08 22:26:57 +02:00
Nikola Knezevic e5cc53beec DownloadSettings -> DataTransferSettings 2020-04-08 22:26:57 +02:00
Nikola Knezevic 741e9012d3 Rename src/lib/download.* to src/lib/datatransfer.* 2020-04-08 22:26:57 +02:00
Nikola Knezevic 65ef57e0cb DownloadRequest -> DataTransferRequest 2020-04-08 22:26:57 +02:00
Eelco Dolstra 5449ff7d8a
Merge pull request #3477 from Ninlives/nix-run-using-env
`nix run` using $SHELL as default command
2020-04-08 19:35:39 +02:00
Ben Burdette 805ffe1bc9 indention 2020-04-08 11:33:46 -06:00
Ben Burdette 8c2bf15c4f format -> fmt 2020-04-08 11:17:02 -06:00
Domen Kožar bf81b31559
build.cc: improve message if home directory exists 2020-04-08 18:27:10 +02:00
Ben Burdette 555baa8fb0 comments 2020-04-08 09:56:10 -06:00
Ben Burdette 54f91923c8 return of NixCode 2020-04-08 09:48:21 -06:00
Eelco Dolstra 96f3c36709
Merge pull request #3478 from edolstra/ignore-failed-data
Downloader: Only write data to the sink on a 200 response
2020-04-08 17:15:53 +02:00
Ben Burdette 47ed067d45 initializer style 2020-04-08 09:07:58 -06:00
Eelco Dolstra 9ed097db7b
Merge pull request #3468 from Infinisil/functionArgsPositions
Make function arguments retain position info
2020-04-08 15:29:39 +02:00
Eelco Dolstra 1ab8d6ac18
Downloader: Only write data to the sink on a 200 response
Hopefully fixes #3278.
2020-04-08 15:27:09 +02:00
Ben Burdette 00c507cc52 columnRange -> column 2020-04-07 14:36:32 -06:00
Ben Burdette 20c0984a46 remove columnrange; switch to fmt in error.cc 2020-04-07 10:14:15 -06:00
Eelco Dolstra 03a4a3c95c Merge branch 'flakes' of github.com:NixOS/nix into flakes 2020-04-07 14:30:25 +02:00
Eelco Dolstra 54955867a6 Merge remote-tracking branch 'origin/master' into flakes 2020-04-07 14:29:45 +02:00
mlatus 9d04b5da17 nix run using $SHELL as default command 2020-04-07 20:29:40 +08:00
Eelco Dolstra 55cefd41d6 Merge branch 'fetchgit-recursive' of https://github.com/blitz/nix 2020-04-07 13:45:17 +02:00
Eelco Dolstra cd39709003 Cleanup 2020-04-07 09:27:17 +02:00
Eelco Dolstra 26aeeb7653 Add FIXME
(cherry picked from commit 2f494531b7)
2020-04-07 09:09:43 +02:00
Eelco Dolstra a6dfa3cb85 PathInput: Add some methods
(cherry picked from commit 78ad5b3d91)
2020-04-07 09:09:43 +02:00
Eelco Dolstra f58a9b0e62 Respect the narHash attribute in more input types
(cherry picked from commit a6ff66b658)
2020-04-07 09:09:18 +02:00
Eelco Dolstra 670feb000a Add 'path' fetcher
This fetchers copies a plain directory (i.e. not a Git/Mercurial
repository) to the store (or does nothing if the path is already a
store path).

One use case is to pin the 'nixpkgs' flake used to build the current
NixOS system, and prevent it from being garbage-collected, via a
system registry entry like this:

  {
      "from": {
          "id": "nixpkgs",
          "type": "indirect"
      },
      "to": {
          "type": "path",
          "path": "/nix/store/rralhl3wj4rdwzjn16g7d93mibvlr521-source",
          "lastModified": 1585388205,
          "rev": "b0c285807d6a9f1b7562ec417c24fa1a30ecc31a"
      },
      "exact": true
  }

Note the fake "lastModified" and "rev" attributes that ensure that the
flake gives the same evaluation results as the corresponding
Git/GitHub inputs.

(cherry picked from commit 12f9379123)
2020-04-07 09:08:51 +02:00
Eelco Dolstra 462421d345 Backport libfetchers from the flakes branch
This provides a pluggable mechanism for defining new fetchers. It adds
a builtin function 'fetchTree' that generalizes existing fetchers like
'fetchGit', 'fetchMercurial' and 'fetchTarball'. 'fetchTree' takes a
set of attributes, e.g.

  fetchTree {
    type = "git";
    url = "https://example.org/repo.git";
    ref = "some-branch";
    rev = "abcdef...";
  }

The existing fetchers are just wrappers around this. Note that the
input attributes to fetchTree are the same as flake input
specifications and flake lock file entries.

All fetchers share a common cache stored in
~/.cache/nix/fetcher-cache-v1.sqlite. This replaces the ad hoc caching
mechanisms in fetchGit and download.cc (e.g. ~/.cache/nix/{tarballs,git-revs*}).

This also adds support for Git worktrees (c169ea5904).
2020-04-07 09:03:14 +02:00
Ben Burdette 55c96b64e4 comment cleanup 2020-04-06 20:14:48 -06:00
Ben Burdette ec449c8450 constructor style basically working 2020-04-06 19:43:22 -06:00
Ben Burdette 85f14c4582 add libutil, libexpr include dirs 2020-04-06 11:15:01 -06:00
Ben Burdette 216263c36f Merge branch 'master' into error-format 2020-04-06 10:00:00 -06:00
Eelco Dolstra 68b43e01dd
nix flake info: Show resolved URL
This is useful for finding out what a registry lookup resolves to, e.g

  $ nix flake info patchelf
  Resolved URL:  github:NixOS/patchelf
  Locked URL:    github:NixOS/patchelf/cd7955af31698c571c30b7a0f78e59fd624d0229
2020-04-06 14:56:13 +02:00
Eelco Dolstra ce3173edc1
nix flake info --json: Don't evaluate
This makes its behaviour consistent with the non-json
variant. Querying the outputs should be done by another command
(e.g. 'nix search')
2020-04-06 14:39:47 +02:00
Eelco Dolstra 2f494531b7
Add FIXME 2020-04-06 14:28:37 +02:00
Cole Helbling c976cb0b8a
Don't retry on "unsupported protocol" error
When encountering an unsupported protocol, there's no need to retry.
Chances are, it won't suddenly be supported between retry attempts;
error instead. Otherwise, you see something like the following:

    $ nix-env -i -f git://git@github.com/foo/bar
    warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 335 ms
    warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 604 ms
    warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 1340 ms
    warning: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1); retrying in 2685 ms

With this change, you now see:

    $ nix-env -i -f git://git@github.com/foo/bar
    error: unable to download 'git://git@github.com/foo/bar': Unsupported protocol (1)
2020-04-05 09:00:34 -07:00
Ben Burdette 1221ae3dd0 libexpr 2020-04-05 07:12:16 -06:00
Ben Burdette 9a8b3e9747 move out of tests/ 2020-04-03 14:55:26 -06:00
Ben Burdette 9bb528d392 handle Pos instead of individual file/line/columnrange args 2020-04-03 13:15:59 -06:00
Eelco Dolstra 63fa92605b nix-env: Refuse to operate on a new-style profile
This prevents users from accidentally nuking their profile via
nix-env.

(cherry picked from commit 021634e3e3)
2020-04-03 20:14:34 +02:00
Ben Burdette 7b7801d3f0 variadic args for hint format 2020-04-03 08:48:20 -06:00
Eelco Dolstra 6e7f252ea6 Make --override-input sticky
When we do something like 'nix flake update --override-input nixpkgs
...', the override is now kept on subsequent calls. (If you don't want
this behaviour, you can use --no-write-lock-file.)
2020-04-03 14:06:29 +02:00
Eelco Dolstra 9c78f7f196 getFlake: In pure mode, check that the argument is an immutable flakeref 2020-04-03 13:07:05 +02:00
Ben Burdette c6b3fcddb0 formatted with astyle 2020-04-02 16:02:40 -06:00
Ben Burdette 1c329ca433 indenting 2020-04-02 14:25:43 -06:00
Eelco Dolstra 485a87f22f Don't barf on registry parse errors 2020-04-02 19:09:17 +02:00
Eelco Dolstra a6ff66b658 Respect the narHash attribute in more input types
call-flake.nix now passes node.info.narHash to fetchTree. This ensures
that dirty Git trees work even in pure mode.
2020-04-02 19:04:33 +02:00
Eelco Dolstra 78ad5b3d91 PathInput: Add some methods 2020-04-02 19:04:27 +02:00