Commit graph

4785 commits

Author SHA1 Message Date
Eelco Dolstra bfaa4db7bd Merge branch 'assert-show-expression' of https://github.com/LnL7/nix 2020-01-13 21:49:55 +01:00
John Ericson d64ab5131c unbreak build without pch 2020-01-13 21:45:33 +01:00
Eelco Dolstra 835e541144 Fix build
https://hydra.nixos.org/eval/1564374
2020-01-13 21:34:54 +01:00
Daiderd Jordan 307bcb9a8e
libexpr: show expression in assertion errors
Includes the expression of the condition in the assertion message if
the assertion failed, making assertions much easier to debug. eg.

    error: assertion (withPython -> (python2Packages != null)) failed at pkgs/tools/security/nmap/default.nix:11:1
2020-01-11 15:45:41 +01:00
Yorick a350d0beb0
json-to-value: use unique_ptr instead of raw pointers 2020-01-09 22:46:41 +01:00
Yorick f1fac0b5c3
builtins.fromJSON: use nlohmann/json parser instead of custom parser 2020-01-09 17:38:27 +01:00
Nikola Knezevic 52a8f9295b Add support for \u escape in fromJSON
As fromTOML supports \u and \U escapes, bring fromJSON on par. As JSON defaults
to UTF-8 encoding (every JSON parser must support UTF-8), this change parses the
`\u hex hex hex hex` sequence (\u followed by 4 hexadecimal digits) into an
UTF-8 representation.

Add a test to verify correct parsing, using all escape sequences from json.org.
2020-01-07 00:09:58 +01:00
Nikola Knezevic cb2d348d48 Remove redundant check in parseJSONString 2020-01-07 00:09:58 +01:00
Eelco Dolstra bc22a7ee6a Fix use of uninitialized store path
Fixes 'building of '/nix/store/00000000000000000000000000000000-': ...'.
2020-01-06 22:20:10 +01:00
Eelco Dolstra e2988f48a1
Merge pull request #3303 from LnL7/darwin-sandbox
build: fix sandboxing on darwin
2020-01-06 20:56:35 +01:00
Eelco Dolstra 1dc29df1d3 Merge remote-tracking branch 'origin/master' into flakes 2020-01-06 12:43:07 +01:00
Daiderd Jordan 66fccd5832
build: fix sandboxing on darwin
Starting ba87b08f85 getEnv now returns an
std::optional which means these getEnv() != "" conditions no longer happen
if the variables are not defined.
2020-01-05 20:23:52 +01:00
Eelco Dolstra 0486e87791
Merge pull request #3302 from LnL7/darwin-repair-with-sandbox
build: fix path repairing with the darwin sandbox
2020-01-05 16:26:17 +01:00
Eelco Dolstra cb90e382b5 Hide FunctionCallTrace constructor/destructor
This prevents them from being inlined. On gcc 9, this reduces the
stack size needed for

  nix-instantiate '<nixpkgs>' -A texlive.combined.scheme-full --dry-run

from 12.9 MiB to 4.8 MiB.
2020-01-05 16:21:34 +01:00
Daiderd Jordan 7d448bc966
build: fix path repairing when hash rewriting is required
Handle store path repairing on darwin when sandboxing is enabled. Unlike
on linux sandboxing on darwin still requires hash rewriting.
2020-01-04 20:25:25 +01:00
Daiderd Jordan b33fefcb92
build: recover store path when replacing fails
This shouldn't happen in normal circumstances, but just in case
attempt to move the temporary path back if possible.
2020-01-04 20:24:27 +01:00
edef c65a6fa86a passAsFile: leave out the hash prefix
Having a colon in the path may cause issues, and having the hash
function indicated isn't actually necessary. We now verify the path 
format in the tests to prevent regressions.
2020-01-02 23:56:06 +00:00
Puck Meerburg 515c0a263e passAsFile: hash the attribute name instead of numbering sequentially
This makes the paths consistent without relying on ordering.

Co-authored-by: edef <edef@edef.eu>
2020-01-02 22:56:03 +00:00
Graham Christensen c502831a1d
exportReferencesGraph: support working
Before, we would get:

    [deploy@bastion:~]$ nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv
    these derivations will be built:
      /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv
      /nix/store/ssxwmll7v21did1c8j027q0m8w6pg41i-unit-prometheus-alertmanager-irc-notifier.service.drv
      /nix/store/mvyvkj46ay7pp7b1znqbkck2mq98k0qd-unit-script-network-local-commands-start.drv
      /nix/store/vsl1y9mz38qfk6pyirjwnfzfggz5akg6-unit-network-local-commands.service.drv
      /nix/store/wi5ighfwwb83fdmav6z6n2fw6npm9ffl-unit-prometheus-hydra-exporter.service.drv
      /nix/store/x0qkv535n75pbl3xn6nn1w7qkrg9wwyg-unit-prometheus-packet-sd.service.drv
      /nix/store/lv491znsjxdf51xnfxh9ld7r1zg14d52-unit-script-packet-sd-env-key-pre-start.drv
      /nix/store/nw4nzlca49agsajvpibx7zg5b873gk9f-unit-script-packet-sd-env-key-start.drv
      /nix/store/x674wwabdwjrkhnykair4c8mpxa9532w-unit-packet-sd-env-key.service.drv
      /nix/store/ywivz64ilb1ywlv652pkixw3vxzfvgv8-unit-wireguard-wg0.service.drv
      /nix/store/v3b648293g3zl8pnn0m1345nvmyd8dwb-unit-script-acme-selfsigned-status.nixos.org-start.drv
      /nix/store/zci5d3zvr6fgdicz6k7jjka6lmx0v3g4-unit-acme-selfsigned-status.nixos.org.service.drv
      /nix/store/f6pwvnm63d0kw5df0v7sipd1rkhqxk5g-system-units.drv
      /nix/store/iax8071knxk9c7krpm9jqg0lcrawf4lc-etc.drv
      /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv
    error: invalid file name 'closure-init-0' in 'exportReferencesGraph'

This was tough to debug, I didn't figure out which one was broken until I did:

    nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv 2>&1 | grep  nix/store | xargs -n1 nix-store -r

and then looking at the remaining build graph:

    $ nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv
    these derivations will be built:
      /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv
      /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv
    error: invalid file name 'closure-init-0' in 'exportReferencesGraph'

and knowing the initrd build is before the system, then:

    $ nix show-derivation /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv
    {
      "/nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv": {
        [...]
        "exportReferencesGraph": "closure-init-0 /nix/store/...-stage-1-init.sh closure-mdadm.conf-1 /nix/store/...-mdadm.conf closure-ubuntu.conf-2 ...",
        [...]
      }
    }

I then searched the repo for "in 'exportReferencesGraph'", found this
recently updated regex, and realized it was missing a "-".
2020-01-01 20:50:40 -05:00
Brian McKenna d25923263e Disable use-sqlite-wal under WSL
Before:

    $ nix-channel --update
    unpacking channels...
    warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)
    warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)
    warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)
    warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)
    warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)

After:

    $ inst/bin/nix-channel --update
    unpacking channels...
    created 1 symlinks in user environment

I've seen complaints that "sandbox" caused problems under WSL but I'm
having no problems. I think recent changes could have fixed the issue.
2019-12-21 08:14:19 +11:00
Eelco Dolstra c7866733d7 Merge remote-tracking branch 'origin/master' into flakes 2019-12-20 12:45:58 +01:00
Eelco Dolstra c84c843e33
tarfile.cc: Restore timestamps
This is needed to get the lastModified attribute of GitHub flakes.
2019-12-19 15:09:54 +01:00
Eelco Dolstra 2550c11373
tarfile.cc: Don't change the cwd
Nix is multithreaded so it's not safe to change the cwd.
2019-12-19 15:08:16 +01:00
Eelco Dolstra be32da0ed0
tarfile.cc: Style fixes 2019-12-19 15:01:58 +01:00
Eelco Dolstra ee235e764c
Merge branch 'libarchive' of https://github.com/yorickvP/nix 2019-12-19 14:47:18 +01:00
Eelco Dolstra 4511f09b49 nix make-content-addressable: Add --json flag
Fixes #3274.
2019-12-18 17:39:02 +01:00
Eelco Dolstra 4da1cd59ba Fix getting the timestamp of GitHub flakes 2019-12-18 17:19:30 +01:00
Eelco Dolstra 87873d0d65 Merge remote-tracking branch 'origin/master' into flakes 2019-12-18 14:25:25 +01:00
Eelco Dolstra f8abbdd456 Add priority setting to stores
This allows overriding the priority of substituters, e.g.

  $ nix-store --store ~/my-nix/ -r /nix/store/df3m4da96d84ljzxx4mygfshm1p0r2n3-geeqie-1.4 \
    --substituters 'http://cache.nixos.org?priority=100 daemon?priority=10'

Fixes #3264.
2019-12-17 17:17:53 +01:00
Eelco Dolstra ad6b738ed8 Merge remote-tracking branch 'origin/master' into flakes 2019-12-16 20:17:21 +01:00
Eelco Dolstra 54bf5ba422 nix-store -r: Handle symlinks to store paths
Fixes #3270.
2019-12-16 19:11:47 +01:00
Eelco Dolstra acb71aa5c6 Tweak error message 2019-12-15 10:44:53 +01:00
Eelco Dolstra 2b0365753a Merge branch 'limit_depth_resolveExprPath' of https://github.com/d-goldin/nix 2019-12-15 00:22:35 +01:00
Eelco Dolstra 8656a2de56
Merge pull request #3269 from xzfc/nix-shell
nix-shell: don't check for "nix-shell" in shebang script name
2019-12-14 23:24:32 +01:00
Eelco Dolstra ba6d2093c7 Fix progress bar 2019-12-14 23:19:04 +01:00
Eelco Dolstra 4947e0a91a Fix InstallableCommand 2019-12-14 23:15:36 +01:00
Eelco Dolstra b8a38fa521 Fix 'nix profile' 2019-12-14 23:09:57 +01:00
Albert Safin a70706b025 nix-shell: don't check for "nix-shell" in shebang script name 2019-12-14 15:37:20 +00:00
Eelco Dolstra ac9cc2ec08 Move some code 2019-12-13 19:10:39 +01:00
Eelco Dolstra b4edc3ca61 Don't leak exceptions 2019-12-13 19:05:26 +01:00
Eelco Dolstra e6bd88878e Improve gzip error message 2019-12-13 19:05:26 +01:00
Eelco Dolstra ca87707c90 Get rid of CBox 2019-12-13 19:05:26 +01:00
Dima d89d9958a7 bugfix: Adding depth limit to resolveExprPath
There is no termination condition for evaluation of cyclical
expression paths which can lead to infinite loops. This addresses
one spot in the parser in a similar fashion as utils.cc/canonPath
does.

This issue can be reproduced by something like:

```
ln -s a b
ln -s b a

nix-instantiate -E 'import ./a'
```
2019-12-13 14:51:30 +01:00
Eelco Dolstra e8aa2290ed Only install *.sb files on macOS 2019-12-13 14:42:55 +01:00
Eelco Dolstra 3e787423c2 Remove FIXME 2019-12-13 12:55:52 +01:00
Eelco Dolstra d1b238ec3c Simplify 2019-12-13 12:53:20 +01:00
Eelco Dolstra 2da4c61049 Merge branch 'libstore-ssh-better-exec-error-message' of https://github.com/Profpatsch/nix 2019-12-13 12:51:36 +01:00
Tom Bereknyei c6295a3afd Initial gzip support
Closes #3256
2019-12-13 03:34:15 -05:00
Profpatsch 38b29fb72c libstore/ssh: Improve error message on failing execvp
If the `throw` is reached, this means that execvp into `ssh` wasn’t
successful. We can hint at a usual problem, which is a missing `ssh`
executable.

Test with:

```
env PATH= ./result/bin/nix-copy-closure --builders '' unusedhost
```

and the bash version with

```
env PATH= ./result/bin/nix-copy-closure --builders '' localhost
```
2019-12-12 15:32:17 +01:00
Eelco Dolstra a045f93396 Temporarily revert to using 'tar' until we have gzip support 2019-12-12 15:07:31 +01:00
Eelco Dolstra ecb3a1afa2 Merge remote-tracking branch 'origin/master' into flakes 2019-12-11 14:53:30 +01:00
Eelco Dolstra bbe97dff8b Make the Store API more type-safe
Most functions now take a StorePath argument rather than a Path (which
is just an alias for std::string). The StorePath constructor ensures
that the path is syntactically correct (i.e. it looks like
<store-dir>/<base32-hash>-<name>). Similarly, functions like
buildPaths() now take a StorePathWithOutputs, rather than abusing Path
by adding a '!<outputs>' suffix.

Note that the StorePath type is implemented in Rust. This involves
some hackery to allow Rust values to be used directly in C++, via a
helper type whose destructor calls the Rust type's drop()
function. The main issue is the dynamic nature of C++ move semantics:
after we have moved a Rust value, we should not call the drop function
on the original value. So when we move a value, we set the original
value to bitwise zero, and the destructor only calls drop() if the
value is not bitwise zero. This should be sufficient for most types.

Also lots of minor cleanups to the C++ API to make it more modern
(e.g. using std::optional and std::string_view in some places).
2019-12-10 22:06:05 +01:00
Eelco Dolstra 98ef11677c EvalState::callFunction(): Make FunctionCallTrace use less stack space
The FunctionCallTrace object consumes a few hundred bytes of stack
space, even when tracing is disabled. This was causing stack overflows:

  $ nix-instantiate '<nixpkgs> -A texlive.combined.scheme-full --dry-run
  error: stack overflow (possible infinite recursion)

This is with the default stack size of 8 MiB.

Putting the object on the heap reduces stack usage to < 5 MiB.
2019-12-10 13:32:30 +01:00
Eelco Dolstra 61cc9f34d2 Remove UserLock self-lock check
This is no longer needed since we're not using POSIX locks anymore.
2019-12-09 23:57:33 +01:00
Yorick eba82b7c88
further clean up libarchive code 2019-12-09 17:21:46 +07:00
puck 28ee687adf Clean up libarchive support 2019-12-07 18:12:21 +00:00
Yorick fe7ec70e6b
remove rust unpack_tarfile ffi 2019-12-07 23:28:31 +07:00
Yorick 1355554d12
code 'cleanup' 2019-12-07 23:23:11 +07:00
Yorick f54c168031
add wrapper function around libarchive to c++ errors 2019-12-07 23:10:27 +07:00
Yorick 232b390766
fixup! libarchive proof of concept 2019-12-07 23:00:37 +07:00
Yorick 9ff5f6492f
libarchive proof of concept 2019-12-07 22:35:14 +07:00
Eelco Dolstra ab88f4bbd4 Merge remote-tracking branch 'origin/master' into flakes 2019-12-05 20:53:32 +01:00
Eelco Dolstra 3b9c9d34e5 Shut up clang warning
(cherry picked from commit 3392f1b778)
2019-12-05 20:41:44 +01:00
Eelco Dolstra 80ab95315d nix doctor: Fix typo
(cherry picked from commit 96c6b08ed7)
2019-12-05 20:40:52 +01:00
Eelco Dolstra 048ef27326 Typo 2019-12-05 20:34:34 +01:00
Eelco Dolstra 47a937d512 Show hash mismatch warnings in SRI format
(cherry picked from commit 63c5c91cc0)
2019-12-05 20:32:25 +01:00
Eelco Dolstra 0678e4d56a Move #include
(cherry picked from commit 8beedd4486)
2019-12-05 20:30:29 +01:00
Eelco Dolstra 79142cbbe1 Bindings: Add convenience method for requiring an attribute
(cherry picked from commit fb692e5f7b)
2019-12-05 20:29:15 +01:00
Eelco Dolstra 0d118ef0c9 Bindings::get(): Add convenience method
This allows writing attribute lookups as

    if (auto name = value.attrs->get(state.sName))
      ...

(cherry picked from commit f216c76c56)
2019-12-05 20:29:00 +01:00
Eelco Dolstra 5e449b43ed Initialize Command::_name
(cherry picked from commit d0a769cb06)
2019-12-05 20:21:22 +01:00
Eelco Dolstra ac67685606 Make subcommand construction in MultiCommand lazy
(cherry picked from commit a0de58f471)
2019-12-05 20:19:26 +01:00
Eelco Dolstra f964f428fe Move Command and MultiCommand to libutil
(cherry picked from commit f70434b1fb)
2019-12-05 20:13:47 +01:00
Eelco Dolstra f1b5c76c1a MultiCommand: Simplify construction
(cherry picked from commit 15a16e5c05)
2019-12-05 20:10:35 +01:00
Eelco Dolstra 092af3c826 Eliminate more pass-by-value in variadic calls 2019-12-05 19:58:52 +01:00
Eelco Dolstra 603b2f583c Revert "Make fmt() non-recursive"
This reverts commit 2b761d5f50.

Also *really* make fmt() take arguments by reference.
2019-12-05 19:58:49 +01:00
Eelco Dolstra 334b8f8af1 fmt(): Pass arguments by reference rather than by value 2019-12-05 17:40:46 +01:00
Eelco Dolstra 1789c56f43 Fix macOS build
https://hydra.nixos.org/build/107716759
2019-12-04 00:36:04 +01:00
Eelco Dolstra c3c23a52ee Merge remote-tracking branch 'origin/master' into flakes 2019-12-04 00:31:09 +01:00
Eelco Dolstra e721f99817 nix: Add --refresh as an alias for --tarball-ttl 0 2019-12-02 15:56:37 +01:00
Eelco Dolstra 0456a4ec65 Merge branch 'run-environment' of https://github.com/mkenigs/nix into flakes 2019-12-02 13:01:43 +01:00
Eelco Dolstra ac2bc721d8 Merge remote-tracking branch 'origin/recursive-nix' 2019-12-02 12:34:46 +01:00
matthew 062012eee1 typo 2019-12-01 18:34:59 -07:00
Brian Wignall 8737980e75 Fix typos 2019-11-30 19:04:14 -05:00
Eelco Dolstra f102d793f1
Merge pull request #2748 from edolstra/rust
Make nix/unpack-channel.nix a builtin builder
2019-11-29 19:33:31 +01:00
Eelco Dolstra 2d6f1ddbb5
Remove builtins.valueSize
Fixes #3246.
2019-11-28 13:52:42 +01:00
Eelco Dolstra 949dc84894 Fix segfault on i686-linux
https://hydra.nixos.org/build/107467517

Seems that on i686-linux, gcc and rustc disagree on how to return
1-word structs: gcc has the caller pass a pointer to the result, while
rustc has the callee return the result in a register. Work around this
by using a bare pointer.
2019-11-27 14:17:15 +01:00
Eelco Dolstra ca8caaec5e nix: Add --expr flag
This replaces the '(...)' installable syntax, which is not very
discoverable. The downside is that you can't have multiple expressions
or mix expressions and other installables.
2019-11-27 00:05:30 +01:00
Eelco Dolstra 8918bae098 Drop remaining uses of external "tar"
Also, fetchGit now runs in O(1) memory since we pipe the output of
'git archive' directly into unpackTarball() (rather than first reading
it all into memory).
2019-11-26 22:07:28 +01:00
Eelco Dolstra d33dd6e6c0 Move code around 2019-11-26 22:07:28 +01:00
Eelco Dolstra b7fba16613 Move code around 2019-11-26 22:07:28 +01:00
Eelco Dolstra f738cd4d97 More Rust FFI adventures
We can now convert Rust Errors to C++ exceptions. At the Rust->C++ FFI
boundary, Result<T, Error> will cause Error to be converted to and
thrown as a C++ exception.
2019-11-26 22:07:28 +01:00
Eelco Dolstra 8110b4ebb2 Rust cleanup 2019-11-26 22:07:28 +01:00
Eelco Dolstra 045708db43 Make <nix/unpack-channel.nix> a builtin builder
This was the last function using a shell script, so this allows us to
get rid of tar, coreutils, bash etc.
2019-11-26 22:07:28 +01:00
Eelco Dolstra e60f6bd4ce Enable Rust code to call C++ Source objects 2019-11-26 22:07:28 +01:00
Eelco Dolstra 11da5b2816 Add some Rust code 2019-11-26 22:07:28 +01:00
Eelco Dolstra abb8ef619b
Fix macOS build
https://hydra.nixos.org/build/107457009
2019-11-26 21:08:56 +01:00
Eelco Dolstra 313106d549
Fix clang warnings 2019-11-26 21:07:44 +01:00
Eelco Dolstra 425991883a
Merge pull request #3141 from xbreak/nocafile
Downloader: Log configured CA file
2019-11-26 20:52:25 +01:00
Eelco Dolstra 7c8d7c17f8
Merge pull request #3144 from matthewbauer/fix-sandbox-fallback
Fix sandbox fallback settings
2019-11-26 20:51:52 +01:00
Eelco Dolstra ec5e7b44ff
Simplify 2019-11-26 20:26:22 +01:00
Eelco Dolstra 96e1c39bb7
Merge branch 'repair-bad-links' of https://github.com/chkno/nix 2019-11-26 20:21:48 +01:00
Eelco Dolstra 872740cf60
Merge pull request #3238 from puckipedia/attrset-overrides-dynamic
Ensure enough space in attrset bindings
2019-11-26 20:14:55 +01:00
Eelco Dolstra c13193017f
Disallow empty store path names
Fixes #3239.
2019-11-26 20:12:15 +01:00
Eelco Dolstra 89db9353d7
Doh 2019-11-26 20:08:25 +01:00
Eelco Dolstra 1ec6e6e11e
Add feature to disable URL literals
E.g.

  $ nix-build '<nixpkgs>' -A hello --experimental-features no-url-literals
  error: URL literals are disabled, at /nix/store/vsjamkzh15r3c779q2711az826hqgvzr-nixpkgs-20.03pre194957.bef773ed53f/nixpkgs/pkgs/top-level/all-packages.nix:1236:11

Helps with implementing https://github.com/NixOS/rfcs/pull/45.
2019-11-26 19:48:34 +01:00
puck cd55f91ad2 Ensure enough space in attrset bindings when using both __overrides and dynamic attributes 2019-11-25 12:37:14 +00:00
Eelco Dolstra d12d69ea1a
Turn NIX_PATH into a config setting
This allows it to be set in nix.conf.
2019-11-22 23:07:35 +01:00
Eelco Dolstra ec9dd9a5ae
Provide a default value for NIX_PATH 2019-11-22 22:08:51 +01:00
Eelco Dolstra ba87b08f85
getEnv(): Return std::optional
This allows distinguishing between an empty value and no value.
2019-11-22 16:18:13 +01:00
Eelco Dolstra fd8ee94ab2 Remove #include 2019-11-20 13:04:39 +01:00
Eelco Dolstra 8beedd4486 Move #include 2019-11-20 13:04:39 +01:00
Chuck 3e2c77d001 Check for and repair bad .links entries
A corrupt entry in .links prevents adding a fixed version of that file
to the store in any path.  The user experience is that corruption
present in the store 'spreads' to new paths added to the store:

(With store optimisation enabled)

1. A file in the store gets corrupted somehow (eg: filesystem bug).
2. The user tries to add a thing to the store which contains a good copy
   of the corrupted file.
3. The file being added to the store is hashed, found to match the bad
   .links entry, and is replaced by a link to the bad .links entry.
   (The .links entry's hash is not verified during add -- this would
   impose a substantial performance burden.)
4. The user observes that the thing in the store that is supposed to be
   a copy of what they were trying to add is not a correct copy -- some
   files have different contents!  Running "nix-store --verify
   --check-contents --repair" does not fix the problem.

This change makes "nix-store --verify --check-contents --repair" fix
this problem.  Bad .links entries are simply removed, allowing future
attempts to insert a good copy of the file to succeed.
2019-11-15 11:55:36 -08:00
Maximilian Bosch b8bddb63e6
Fix attr path to nixpkgs flake in flake template
This doesn't work anymore since `packages` was removed from the
`nixpkgs`-fork with flake support[1], now it's only possible to refer to
pkgs via `legacyPackages`.

[1] 49c9b71e4c
2019-11-14 23:13:37 +01:00
Eelco Dolstra 0352c1a4f8
Typo 2019-11-13 17:18:17 +01:00
Eelco Dolstra 8d2eb1ff56
nix dev-shell: Improve bash output parsing
Fixes handling of '=' in unquoted strings and escaped characters in
$'...' strings.
2019-11-12 12:45:48 +01:00
Eelco Dolstra 5ee23c35b9
Merge pull request #3219 from Ericson2314/semicolons
Fix extra semicolons warnings
2019-11-11 12:13:51 +01:00
John Ericson 8669db1dcc Clean up semicolon and comma
Thanks @bhipple for catching!
2019-11-10 16:21:59 -05:00
John Ericson 4c34054673 Remove unneeded semicolons 2019-11-10 11:24:47 -05:00
John Ericson 96e6e680c1 Fix extra ; warnings involving MakeError 2019-11-10 11:24:47 -05:00
Peter Kolloch 2ba9f22715
De-duplicate struct PrimOp forward declaration 2019-11-10 10:02:22 +01:00
Eelco Dolstra 2c1e05ae93
Merge remote-tracking branch 'origin/master' into flakes 2019-11-08 15:22:54 +01:00
Eelco Dolstra 0d6774468c
Move editorFor srom libutil to nix
libutil should not depend on libexpr.
2019-11-08 15:13:32 +01:00
matthew 6419f5028b use MixEnvironment in run and shell 2019-11-07 17:22:16 -06:00
matthew 75c897cf3d Factor out code to handle environment in run into MixEnvironment 2019-11-07 17:22:16 -06:00
matthew 693e8b1286 changes 2019-11-07 17:22:16 -06:00
matthew d2438f86d5 environment fixes in run
Move environment related code to a separate function. Create a new char** if ignoreEnvironment is set rather than calling clearEnv
2019-11-07 17:22:16 -06:00
Eelco Dolstra 4145cd2da0
Use upstream nlohmann_json 2019-11-07 18:23:17 +01:00
Eelco Dolstra f730841db4
Merge remote-tracking branch 'origin/master' into flakes 2019-11-07 11:44:02 +01:00
Eelco Dolstra 04bf9acd22
Remove #include 2019-11-07 10:12:35 +01:00
Eelco Dolstra 39a2e166dd
Cleanup 2019-11-06 16:53:02 +01:00
Eelco Dolstra e491efe9fb
Use more stable registry URL 2019-11-06 14:20:16 +01:00
Eelco Dolstra d5f1cc3e94
Use revcount/last-modified for computing the flake fingerprint
The store path is not enough. For example, when we build a dirty tree,
commit, and build the clean tree, a re-evaluation is necessary because
the flake may depend on the lastModified or revCount attributes.
2019-11-06 12:01:37 +01:00
Eelco Dolstra 88c452d160
Merge remote-tracking branch 'origin/master' into flakes 2019-11-06 10:56:33 +01:00
Eelco Dolstra 69326f3637
Recursive Nix: Handle concurrent client connections 2019-11-06 00:55:03 +01:00
Eelco Dolstra c119ab9db0
Enable recursive Nix using a feature
Derivations that want to use recursion should now set

  requiredSystemFeatures = [ "recursive-nix" ];

to make the daemon socket appear.

Also, Nix should be configured with "experimental-features =
recursive-nix".
2019-11-06 00:55:03 +01:00
Eelco Dolstra c921074c19
RestrictedStore: Implement addToStore() 2019-11-06 00:55:03 +01:00
Eelco Dolstra c4d7c76b64
Recursive Nix support
This allows Nix builders to call Nix to build derivations, with some
limitations.

Example:

  let nixpkgs = fetchTarball channel:nixos-18.03; in

  with import <nixpkgs> {};

  runCommand "foo"
    {
      buildInputs = [ nix jq ];
      NIX_PATH = "nixpkgs=${nixpkgs}";
    }
    ''
      hello=$(nix-build -E '(import <nixpkgs> {}).hello.overrideDerivation (args: { name = "hello-3.5"; })')

      $hello/bin/hello

      mkdir -p $out/bin
      ln -s $hello/bin/hello $out/bin/hello

      nix path-info -r --json $hello | jq .
    ''

This derivation makes a recursive Nix call to build GNU Hello and
symlinks it from its $out, i.e.

  # ll ./result/bin/
  lrwxrwxrwx 1 root root 63 Jan  1  1970 hello -> /nix/store/s0awxrs71gickhaqdwxl506hzccb30y5-hello-3.5/bin/hello

  # nix-store -qR ./result
  /nix/store/hwwqshlmazzjzj7yhrkyjydxamvvkfd3-glibc-2.26-131
  /nix/store/s0awxrs71gickhaqdwxl506hzccb30y5-hello-3.5
  /nix/store/sgmvvyw8vhfqdqb619bxkcpfn9lvd8ss-foo

This is implemented as follows:

* Before running the outer builder, Nix creates a Unix domain socket
  '.nix-socket' in the builder's temporary directory and sets
  $NIX_REMOTE to point to it. It starts a thread to process
  connections to this socket. (Thus you don't need to have nix-daemon
  running.)

* The daemon thread uses a wrapper store (RestrictedStore) to keep
  track of paths added through recursive Nix calls, to implement some
  restrictions (see below), and to do some censorship (e.g. for
  purity, queryPathInfo() won't return impure information such as
  signatures and timestamps).

* After the build finishes, the output paths are scanned for
  references to the paths added through recursive Nix calls (in
  addition to the inputs closure). Thus, in the example above, $out
  has a reference to $hello.

The main restriction on recursive Nix calls is that they cannot do
arbitrary substitutions. For example, doing

  nix-store -r /nix/store/kmwd1hq55akdb9sc7l3finr175dajlby-hello-2.10

is forbidden unless /nix/store/kmwd... is in the inputs closure or
previously built by a recursive Nix call. This is to prevent
irreproducible derivations that have hidden dependencies on
substituters or the current store contents. Building a derivation is
fine, however, and Nix will use substitutes if available. In other
words, the builder has to present proof that it knows how to build a
desired store path from scratch by constructing a derivation graph for
that path.

Probably we should also disallow instantiating/building fixed-output
derivations (specifically, those that access the network, but
currently we have no way to mark fixed-output derivations that don't
access the network). Otherwise sandboxed derivations can bypass
sandbox restrictions and access the network.

When sandboxing is enabled, we make paths appear in the sandbox of the
builder by entering the mount namespace of the builder and
bind-mounting each path. This is tricky because we do a pivot_root()
in the builder to change the root directory of its mount namespace,
and thus the host /nix/store is not visible in the mount namespace of
the builder. To get around this, just before doing pivot_root(), we
branch a second mount namespace that shares its /nix/store mountpoint
with the parent.

Recursive Nix currently doesn't work on macOS in sandboxed mode
(because we can't change the sandbox policy of a running build) and in
non-root mode (because setns() barfs).
2019-11-06 00:52:38 +01:00
Eelco Dolstra d823381c0a
Merge branch 'fix/nix-doctor-output' of https://github.com/bhipple/nix 2019-11-05 16:04:40 +01:00
Eelco Dolstra 852554bb16
Merge branch 'nix-repl-e' of https://github.com/zimbatm/nix 2019-11-05 11:20:53 +01:00
Eelco Dolstra 9ff4060d26
Merge pull request #3190 from mkenigs/examples
change deprecated attribute syntax in run examples
2019-11-05 11:17:54 +01:00
Eelco Dolstra 72748b4088
Merge pull request #3173 from mkenigs/typo
fix typo
2019-11-05 11:17:26 +01:00
Eelco Dolstra 1b600ecd14
Don't use SOCK_CLOEXEC on macOS
https://hydra.nixos.org/build/105428308
2019-11-05 10:25:09 +01:00
Eelco Dolstra b81d9d26f5
Merge remote-tracking branch 'origin/master' into flakes 2019-11-04 22:29:31 +01:00
Eelco Dolstra f5a46ef0b1
Merge pull request #3202 from kraem/master
Update nix eval --help msg to not include deprecated command
2019-11-04 09:34:30 +01:00
Harald van Dijk c935ad3f02
Fix progress bar when nix-prefetch-url is piped.
The intent of the code was that if the window size cannot be determined,
it would be treated as having the maximum possible size. Because of a
missing assignment, it was actually treated as having a width of 0.

The reason the width could not be determined was because it was obtained
from stdout, not stderr, even though the printing was done to stderr.

This commit addresses both issues.
2019-11-03 21:46:59 +00:00
kraem dcd7a26063
Update nix eval --help msg to not include deprecated command 2019-11-03 18:47:28 +01:00
ng0 b811bd2172 include netinet/in.h in src/nix/main.cc
Fixes #3186
2019-11-01 14:09:42 +00:00
matthew d865085c7e change deprecated attribute syntax in run examples 2019-10-31 23:13:08 -05:00
Eelco Dolstra 6bff1aa46d
Merge pull request #3182 from bhipple/fixup/comments
Minor updates to inline comments
2019-10-31 14:14:35 +01:00
Benjamin Hipple 80d5ec6ff4 Minor updates to inline comments
Add missing docstring on InstallableCommand. Also, some of these were wrapped
when they're right next to a line longer than the unwrapped line, so we can just
unwrap them to save vertical space.
2019-10-31 05:56:37 -04:00
Will Dietz 0e9b72e097 minor: fix mismatch of struct/class forward decl of 'Source'
Fixes the following warning and the indicate potential issue:

src/libstore/worker-protocol.hh:66:1: warning: class 'Source' was previously declared as a struct; this is valid, but may result in linker errors
under the Microsoft C++ ABI [-Wmismatched-tags]

(cherry picked from commit 6e1bb04870b1b723282d32182af286646f13bf3c)
2019-10-30 14:39:01 -05:00
Janne Heß 2d1d1e3083 Replace git show with git log
git show seems to print the entire tag message when being called on a tag
instead of a commit. git log instead always prints the correct timestamp
in my tests.
The error nix prints is: `error: stoull`.
2019-10-30 17:15:38 +01:00
Eelco Dolstra e5319a87ce
queryPathInfoUncached(): Return const ValidPathInfo 2019-10-29 13:53:04 +01:00
Eelco Dolstra 992a2ad475
Move addToStoreFromDump to Store 2019-10-29 13:38:24 +01:00
Eelco Dolstra 05819d013f
Don't create a Store in processConnection() 2019-10-29 13:36:19 +01:00
Eelco Dolstra 63b99af85a
Move Unix domain socket creation to libutil
Also drop multithread-unfriendly hacks like doing a temporary
chmod/umask.
2019-10-29 13:30:51 +01:00
Eelco Dolstra 2d37e88319
Move most of the daemon implementation to libstore 2019-10-29 13:25:33 +01:00
Eelco Dolstra 95c727caef
Remove the check against concurrent builds in the same process 2019-10-29 12:43:20 +01:00
matthew e0bcacf79f fix typo 2019-10-28 18:02:47 -05:00
Jonas Chevalier 9a25059656
findDerivationFilename: add FIXME 2019-10-28 21:40:02 +01:00
Jonas Chevalier d407f4d15f
nix repl: also handle lambda edit 2019-10-28 21:37:22 +01:00
Jonas Chevalier 3774fe55fd
editorFor: take a pos object instead 2019-10-28 21:36:34 +01:00
Jonas Chevalier ec448f8bb6
libexpr: findDerivationFilename return Pos instead of tuple 2019-10-28 21:29:54 +01:00
Eelco Dolstra f7ce80f90a
Factor out linkOrCopy() 2019-10-27 18:19:13 +01:00
Eelco Dolstra f1c0b2c0e1
Add O(1)-memory copyPath() function 2019-10-27 18:18:58 +01:00
Eelco Dolstra 3913afdd69
Simplification 2019-10-27 18:00:09 +01:00
Eelco Dolstra 0e459d79a6
Merge branch 'issue-3147-inNixShell-arg' of https://github.com/hercules-ci/nix 2019-10-27 17:10:19 +01:00
Robert Hensing 9d612c393a Add inNixShell = true to nix-shell auto-call
This is an alternative to the IN_NIX_SHELL environment variable,
allowing the expression to adapt itself to nix-shell without
triggering those adaptations when used as a dependency of another
shell.

Closes #3147
2019-10-27 13:16:02 +01:00
Eelco Dolstra e012384fe9
Merge branch 'tojson-tostring-fix' of https://github.com/mayflower/nix 2019-10-27 12:18:35 +01:00
Robin Gloster e583df5280
builtins.toJSON: fix __toString usage 2019-10-27 10:15:51 +01:00
Jonas Chevalier 59c7249769
libexpr: add findDerivationFilename
extract the derivation to filename:lineno heuristic
2019-10-23 17:21:16 +02:00
Jonas Chevalier 207a537343
libutil: add editorFor heuristic 2019-10-23 16:48:28 +02:00
Jonas Chevalier 73ff84f6a8
nix repl: add :edit command
This allows to have a repl-centric workflow to working on nixpkgs.

Usage:

    :edit <package> - heuristic that find the package file path

    :edit <path> - just open the editor on the file path

Once invoked, `nix repl` will open $EDITOR on that file path. Once the
editor exits, `nix repl` will automatically reload itself.
2019-10-23 16:09:42 +02:00
Eelco Dolstra 9cac895406 Use upstream nlohmann_json 2019-10-22 20:12:23 +02:00
Eelco Dolstra ac9b427541 Convert old-style profile manifest 2019-10-22 15:16:57 +02:00
Eelco Dolstra af786432c5 Add "nix profile upgrade" command 2019-10-22 14:44:51 +02:00
Eelco Dolstra e30a0155d4 Add "nix profile remove" command 2019-10-22 13:06:32 +02:00
Eelco Dolstra 555ca59f2b nix profile info: Index elements 2019-10-22 00:28:16 +02:00
Eelco Dolstra ce27920936 Add start of 'nix profile' command 2019-10-22 00:22:15 +02:00
Eelco Dolstra 91a88f3acb Fix "nixpkgs." compatibility 2019-10-21 23:38:07 +02:00
Eelco Dolstra 1e23b82a53 exportGitHub(): Don't rely on the ETag from GitHub
We relied on it being the Git revision, but that stopped being the
case.
2019-10-21 23:14:29 +02:00
Eelco Dolstra 45b740c18b Use upstream json_fwd.hpp to speed up compilation 2019-10-21 22:11:21 +02:00
Eelco Dolstra cb1a79a96a Fix build 2019-10-21 18:58:38 +02:00
Eelco Dolstra 9a18f544ac Merge remote-tracking branch 'origin/master' into flakes 2019-10-21 18:48:21 +02:00
Eelco Dolstra 629b9b0049 Mark content-addressable paths with references as experimental 2019-10-21 18:05:31 +02:00
Eelco Dolstra e68736936a nix make-content-addressable: Add examples 2019-10-21 17:58:17 +02:00
Eelco Dolstra d77970fde7 Fix build 2019-10-21 17:49:16 +02:00
Eelco Dolstra 0abb3ad537 Allow content-addressable paths to have references
This adds a command 'nix make-content-addressable' that rewrites the
specified store paths into content-addressable paths. The advantage of
such paths is that 1) they can be imported without signatures; 2) they
can enable deduplication in cases where derivation changes do not
cause output changes (apart from store path hashes).

For example,

  $ nix make-content-addressable -r nixpkgs.cowsay
  rewrote '/nix/store/g1g31ah55xdia1jdqabv1imf6mcw0nb1-glibc-2.25-49' to '/nix/store/48jfj7bg78a8n4f2nhg269rgw1936vj4-glibc-2.25-49'
  ...
  rewrote '/nix/store/qbi6rzpk0bxjw8lw6azn2mc7ynnn455q-cowsay-3.03+dfsg1-16' to '/nix/store/iq6g2x4q62xp7y7493bibx0qn5w7xz67-cowsay-3.03+dfsg1-16'

We can then copy the resulting closure to another store without
signatures:

  $ nix copy --trusted-public-keys '' ---to ~/my-nix /nix/store/iq6g2x4q62xp7y7493bibx0qn5w7xz67-cowsay-3.03+dfsg1-16

In order to support self-references in content-addressable paths,
these paths are hashed "modulo" self-references, meaning that
self-references are zeroed out during hashing. Somewhat annoyingly,
this means that the NAR hash stored in the Nix database is no longer
necessarily equal to the output of "nix hash-path"; for
content-addressable paths, you need to pass the --modulo flag:

  $ nix path-info --json /nix/store/iq6g2x4q62xp7y7493bibx0qn5w7xz67-cowsay-3.03+dfsg1-16  | jq -r .[].narHash
  sha256:0ri611gdilz2c9rsibqhsipbfs9vwcqvs811a52i2bnkhv7w9mgw

  $ nix hash-path --type sha256 --base32 /nix/store/iq6g2x4q62xp7y7493bibx0qn5w7xz67-cowsay-3.03+dfsg1-16
  1ggznh07khq0hz6id09pqws3a8q9pn03ya3c03nwck1kwq8rclzs

  $ nix hash-path --type sha256 --base32 /nix/store/iq6g2x4q62xp7y7493bibx0qn5w7xz67-cowsay-3.03+dfsg1-16 --modulo iq6g2x4q62xp7y7493bibx0qn5w7xz67
  0ri611gdilz2c9rsibqhsipbfs9vwcqvs811a52i2bnkhv7w9mgw
2019-10-21 17:47:24 +02:00
Eelco Dolstra b82f75464d buildenv: Eliminate global variables, other cleanup 2019-10-21 17:40:40 +02:00
Eelco Dolstra a7aabd7cc7 Add getDefaultProfile() function 2019-10-21 16:07:19 +02:00
Eelco Dolstra a07da2fd7a Don't ignore revs/refs of local flakerefs
Fixes

  error: the content hash of flake '/home/eelco/Dev/nixpkgs-flake?ref=HEAD&rev=0000000000000000000000000000000000000000' doesn't match the hash recorded in the referring lockfile
2019-10-21 14:57:01 +02:00
Eelco Dolstra 4a1cd10495 Merge remote-tracking branch 'origin/master' into flakes 2019-10-21 13:52:55 +02:00
Eelco Dolstra aabf5c86c9
Add experimental-features setting
Experimental features are now opt-in. There is currently one
experimental feature: "nix-command" (which enables the "nix"
command. This will allow us to merge experimental features more
quickly, without committing to supporting them indefinitely.

Typical usage:

$ nix build --experimental-features 'nix-command flakes' nixpkgs#hello

(cherry picked from commit 8e478c2341,
without the "flakes" feature)
2019-10-21 13:34:44 +02:00
Eelco Dolstra 389a2cebed
SourceExprCommand::getSourceExpr(): Allocate more space
Fixes #3140.
2019-10-21 13:14:39 +02:00
Eelco Dolstra 8426d99b0e Fix InstallableFlake::what() 2019-10-20 16:43:00 +02:00
xbreak 7c568d4c6e Downloader: Warn if no trusted CA file has been configured 2019-10-18 19:08:33 +00:00
Eelco Dolstra 8e478c2341
Add experimental-features setting
Experimental features are now opt-in. There are currently two
experimental features: "nix-command" (which enables the "nix"
command), and "flakes" (which enables support for flakes). This will
allow us to merge experimental features more quickly, without
committing to supporting them indefinitely.

Typical usage:

$ nix build --experimental-features 'nix-command flakes' nixpkgs#hello
2019-10-16 17:49:01 +02:00
Eelco Dolstra 0ab64729e9 Improve GitHub caching
In particular, when building a flake lock file, inputs like 'nixpkgs'
are now downloaded only once. Previously, it would fetch
https://api.github.com/repos/<owner>/<repo>/tarball/<ref> and then
later https://api.github.com/repos/<owner>/<repo>/tarball/<rev>, even
though they produce the same result.

Git and GitHub now also share a cache that maps revs to a store path
and other info.
2019-10-16 00:20:51 +02:00
Eelco Dolstra 14a89aa8cd Fix 'nix flake init' 2019-10-15 19:53:29 +02:00
Eelco Dolstra 7d38060a0d Support non-x86_64-linux system types in flakes
A command like

  $ nix run nixpkgs#hello

will now build the attribute 'packages.${system}.hello' rather than
'packages.hello'. Note that this does mean that the flake needs to
export an attribute for every system type it supports, and you can't
build on unsupported systems. So 'packages' typically looks like this:

  packages = nixpkgs.lib.genAttrs ["x86_64-linux" "i686-linux"] (system: {
    hello = ...;
  });

The 'checks', 'defaultPackage', 'devShell', 'apps' and 'defaultApp'
outputs similarly are now attrsets that map system types to
derivations/apps. 'nix flake check' checks that the derivations for
all platforms evaluate correctly, but only builds the derivations in
'checks.${system}'.

Fixes #2861. (That issue also talks about access to ~/.config/nixpkgs
and --arg, but I think it's reasonable to say that flakes shouldn't
support those.)

The alternative to attribute selection is to pass the system type as
an argument to the flake's 'outputs' function, e.g. 'outputs = { self,
nixpkgs, system }: ...'. However, that approach would be at odds with
hermetic evaluation and make it impossible to enumerate the packages
provided by a flake.
2019-10-15 18:16:29 +02:00
Eelco Dolstra 0bc8f1669d Move code around 2019-10-14 14:40:16 +02:00
Matthew Bauer 96c84937c4 Move tmpDirInSandbox to initTmpDir 2019-10-13 16:41:49 -04:00
Matthew Bauer 499b038875 Fix sandbox fallback settings
The tmpDirInSandbox is different when in sandboxed vs. non-sandboxed.
Since we don’t know ahead of time here whether sandboxing is enabled,
we need to reset all of the env vars we’ve set previously. This fixes
the issue encountered in https://github.com/NixOS/nixpkgs/issues/70856.
2019-10-12 19:22:13 -04:00
Eelco Dolstra 906d56a96b
ssh-ng: Don't set CPU affinity on the remote
Fixes #3138.
2019-10-11 18:49:46 +02:00
Eelco Dolstra 5446eae949
Merge remote-tracking branch 'origin/master' into flakes 2019-10-10 15:07:59 +02:00
Eelco Dolstra 90df25ef7e
Fix build 2019-10-10 15:07:50 +02:00
Eelco Dolstra 95cf23ee7c
nix verify: Fix uninitialized variable 2019-10-10 15:03:01 +02:00
Eelco Dolstra e99bb91217
Merge remote-tracking branch 'origin/master' into flakes 2019-10-10 12:54:37 +02:00
Eelco Dolstra c3aaf3b8da
nix-env: Ignore failures creating ~/.nix-profile and ~/.nix-defexpr
https://hydra.nixos.org/build/102803093
2019-10-10 09:14:05 +02:00
Eelco Dolstra bda64a2b0f
Doh
https://hydra.nixos.org/build/102803044
2019-10-10 00:12:30 +02:00
Eelco Dolstra 20eec802ff
Force per-user group to a known value 2019-10-09 23:35:02 +02:00
Eelco Dolstra d7bae5680f
Go back to 755 permission on per-user directories
700 is pointless since the store is world-readable anyway. And
per-user/root/channels must be world-readable.
2019-10-09 23:35:02 +02:00
Eelco Dolstra c9159f86cc
nix-env: Create ~/.nix-defexpr automatically 2019-10-09 23:35:02 +02:00
Eelco Dolstra 9348f9291e
nix-env: Create ~/.nix-profile automatically 2019-10-09 23:35:01 +02:00
Eelco Dolstra 5a303093dc
Remove world-writability from per-user directories
'nix-daemon' now creates subdirectories for users when they first
connect.

Fixes #509 (CVE-2019-17365).
Should also fix #3127.
2019-10-09 23:34:48 +02:00
Eelco Dolstra 4331eeb13d
Filter ANSI escape sequences in -L output
Otherwise, builds like NixOS VM tests may leave the terminal in a
weird state and do resets.
2019-10-09 23:25:06 +02:00
Eelco Dolstra 55bba8e4f5
Make std::uncaught_exception warning less noisy 2019-10-09 23:04:11 +02:00
Eelco Dolstra 926d3e5bb0
Fix Bison 2.4 warning 2019-10-09 22:57:37 +02:00
Eelco Dolstra 99b73fb507
OCD performance fix: {find,count}+insert => insert 2019-10-09 16:06:29 +02:00
Eelco Dolstra e6e61f0a54
getSourceExpr(): Handle channels
Fixes #1892.
Fixes #1865.
Fixes #3119.
2019-10-09 15:36:51 +02:00
Eelco Dolstra 08ad9714e1
Merge pull request #3132 from matthewbauer/handle-sandbox-shell
Handle empty sandbox_shell
2019-10-09 14:52:51 +02:00
Eelco Dolstra 7c74f075f4
nix search: Don't quietly ignore errors 2019-10-09 14:46:58 +02:00
Eelco Dolstra 64d8872900
nix-build: Fix compilation 2019-10-09 14:46:44 +02:00
Matthew Bauer 199e888785 Handle empty sandbox_shell
Previously, SANDBOX_SHELL was set to empty when unavailable. This
caused issues when actually generating the sandbox. Instead, just set
SANDBOX_SHELL when --with-sandbox-shell= is non-empty. Alternative
implementation to https://github.com/NixOS/nix/pull/3038.
2019-10-08 23:12:54 -04:00
Matthew Bauer 65f6d5db6f Don’t source bashrc in pure mode
Pure mode should not try to source the user’s bashrc file. These may
have many impurities that the user does not expect to get into their
shell.

Fixes #3090
2019-10-08 22:41:59 -04:00
Eelco Dolstra a0bd088d84
Move addRegistrOverrides 2019-10-08 17:00:55 +02:00
Eelco Dolstra 21304c11f9
uri -> url for consistency 2019-10-08 17:00:55 +02:00
Emilio Karakey d24bfe29a1 deleted comment 2019-10-07 18:28:10 -05:00
Eelco Dolstra a15f9b37eb
fetchGit: Support Git trees without any commits
Fixes

  $ nix build
  fatal: bad revision 'HEAD'
  error: program 'git' failed with exit code 128

on a new flake. It is now detected as a dirty tree with revCount = 0.
2019-10-07 15:44:32 +02:00
Sam Doshi 6f6cb5e388 nix search: remove verbose example 2019-10-07 11:40:42 +01:00
Benjamin Hipple c5bd564c69 nix doctor: add more logging output to checks
When running nix doctor on a healthy system, it just prints the store URI and
nothing else. This makes it unclear whether the system is in a good state and
what check(s) it actually ran, since some of the checks are optional depending
on the store type.

This commit updates nix doctor to print an colored log message for every check
that it does, and explicitly state whether that check was a PASS or FAIL to make
it clear to the user whether the system passed its checkup with the doctor.

Fixes #3084
2019-10-06 16:57:57 -04:00
Eelco Dolstra a323b7826c
Merge remote-tracking branch 'origin/master' into flakes 2019-10-04 17:26:32 +02:00
Eelco Dolstra 93b1ce1ac5
Revert "std::uncaught_exception() -> std::uncaught_exceptions()"
This reverts commit 6b83174fff because
it doesn't work on macOS yet.

https://hydra.nixos.org/build/102617587
2019-10-04 16:34:59 +02:00
Eelco Dolstra 90d6018509
Fix aborts when using builtins.getFlake
In that case, 'self' could refer to a value on the stack, so accessing
'self.rev' would abort.
2019-10-02 22:08:19 +02:00
Eelco Dolstra 15e70c662e
Fix indentation 2019-10-02 16:26:15 +02:00
Eelco Dolstra 780c1a8f27
nix dev-shell: Ignore $NIX_LOG_FD 2019-10-02 10:52:56 +02:00
Eelco Dolstra 168a887916
Fix fetchTarball with chroot stores
Fixes #2405.
2019-10-01 07:51:06 +00:00
Domen Kožar 2d2769f68c
Merge pull request #2338 from bobvanderlinden/pr-cannot-delete-alive-why
mention `nix-store --query --roots` when a path cannot be deleted
2019-09-30 14:06:52 +02:00
Eelco Dolstra 9b9de3a5e3
nix dev-shell: Improve environment handling
Only variables that were marked as exported are exported in the dev
shell. Also, we no longer try to parse the function section of the env
file, fixing

  $ nix dev-shell
  error: shell environment '/nix/store/h7ama3kahb8lypf4nvjx34z06g9ncw4h-nixops-1.7pre20190926.4c7acbb-env' has unexpected line '/^[a-z]?"""/ {'
2019-09-27 17:01:25 +02:00
Eelco Dolstra 15b888c9a5
cmatch -> smatch 2019-09-27 15:31:09 +02:00
Eelco Dolstra 454e3a541a
Fix sorting of non-flake input attributes 2019-09-26 17:51:51 +02:00
Eelco Dolstra c32bba7489
Shut up some warnings 2019-09-24 17:28:18 +02:00
Eelco Dolstra 5038e1bec4
Merge pull request #3103 from bhipple/fix/spelling
Fix spelling in comment
2019-09-23 10:46:40 +02:00
Eelco Dolstra 02b4632e77
Merge pull request #3104 from zimbatm/no-show-trace-forwarding
libstore: don't forward --show-trace
2019-09-23 10:44:42 +02:00
Eelco Dolstra 3a022d4599 Shut up some warnings
(cherry picked from commit 99e8e58f2d)
2019-09-22 21:57:05 +02:00
Eelco Dolstra bd79c1f6f6 Don't catch exceptions by value
(cherry picked from commit 893be6f5e3)
2019-09-22 21:56:56 +02:00
Eelco Dolstra 382aa05ff7 nix flake info --json: Get rid of duplicate getFlake() call
Also fix some gcc warnings.
2019-09-22 21:53:01 +02:00
Eelco Dolstra 893be6f5e3 Don't catch exceptions by value 2019-09-22 21:29:33 +02:00
Jonas Chevalier e63c9e73e3
libstore: don't forward --show-trace 2019-09-22 10:57:20 +00:00
Benjamin Hipple c6a542f22a Fix spelling in comment 2019-09-21 18:53:15 -04:00
Eelco Dolstra 14d3f45009 Simplify 2019-09-20 16:06:49 +02:00
Eelco Dolstra 5a0e98d1e5 Use '#' instead of ':' to separate flakeref and attrpath
This is less ambiguous.
2019-09-20 16:01:40 +02:00
Eelco Dolstra 5961c94097 Flake alias -> id 2019-09-20 14:46:37 +02:00
Eelco Dolstra 68e0f23edc Add flags to disallow dirty Git trees and to turn off warnings 2019-09-20 14:29:49 +02:00
Eelco Dolstra 99e8e58f2d Shut up some warnings 2019-09-20 13:48:53 +02:00
Eelco Dolstra 5573365dff
nix flake check: Validate nixosConfigurations outputs 2019-09-19 20:15:42 +02:00
Eelco Dolstra aeb7148afd
Some effort to minimize flake dependencies
For example, if the top-level flake depends on
"nixpkgs/release-19.03", and one of its dependencies depends on
"nixpkgs", then the latter will be mapped to "nixpkgs/release-19.03",
rather than whatever the default branch of "nixpkgs" is. Thus you get
only one "nixpkgs" dependency rather than two.

This currently only works in a breadth-first way, so the other way
around (i.e. if the top-level flake depends on "nixpkgs", and a
dependency depends on "nixpkgs/release-19.03") still results in two
"nixpkgs" dependencies.
2019-09-18 23:59:45 +02:00
Jonas Chevalier 619cc4af85
function-trace: always show the trace
If the user invokes nix with --trace-function-calls it means that they
want to see the trace.
2019-09-18 23:23:21 +02:00
Eelco Dolstra c67407172d
Record original flakerefs in the lock file again
If 'input.<name>.uri' changes, then the entry in the lockfile for
input <name> should be considered stale.

Also print some messages when lock file entries are added/updated.
2019-09-18 21:57:57 +02:00
Eelco Dolstra 092ee24627
Merge remote-tracking branch 'origin/master' into flakes 2019-09-18 12:02:50 +02:00
Eelco Dolstra 6b83174fff std::uncaught_exception() -> std::uncaught_exceptions()
The former is deprecated in C++17. Fixes a clang warning.
2019-09-13 20:05:44 +02:00
Eelco Dolstra a25c022af3 Merge remote-tracking branch 'origin/master' into flakes 2019-09-13 19:52:03 +02:00
Eelco Dolstra 55e55b34e6
nix flake check: Check hydraJobs 2019-09-10 17:39:55 +02:00
Eelco Dolstra 4b9dee6bcc
nix flake check: Do some basic checks on NixOS modules
Also show more position info.
2019-09-10 15:25:32 +02:00
Eelco Dolstra dc3f52a144
nix flake check: Check overlays 2019-09-10 14:52:22 +02:00
Eelco Dolstra f97d3753a1
Require flake.nix to be an attrset (not a non-trivial thunk) 2019-09-09 17:34:38 +02:00
Eelco Dolstra c87840ae14
Don't allow arbitrary computations in flake attributes
E.g. you can write 'edition = 201909' but not 'edition = 201909 + 0'.

Fixes #3075.
2019-09-09 16:34:44 +02:00
Eelco Dolstra 2fa7f2a56a
Use git+ prefix in flake URI schemes
Fixes #3045.
2019-09-05 17:15:09 +02:00
Eelco Dolstra a56b51a0ba
Disable OpenSSL lock callback on OpenSSL >= 1.1.1 2019-09-04 21:45:01 +02:00
Eelco Dolstra 5dafde28db
BinaryCacheStore: Add index-debug-info option
This integrates the functionality of the index-debuginfo program in
nixos-channel-scripts to maintain an index of DWARF debuginfo files in
a format usable by dwarffs. Thus the debug info index is updated by
Hydra rather than by the channel mirroring script.

Example usage:

  $ nix copy --to 'file:///tmp/binary-cache?index-debug-info=true' /nix/store/vr9mhcch3fljzzkjld3kvkggvpq38cva-nix-2.2.2-debug

  $ cat /tmp/binary-cache/debuginfo/036b210b03bad75ab2d8fc80b7a146f98e7f1ecf.debug
  {"archive":"../nar/0313h2kdhk4v73xna9ysiksp2v8xrsk5xsw79mmwr3rg7byb4ka8.nar.xz","member":"lib/debug/.build-id/03/6b210b03bad75ab2d8fc80b7a146f98e7f1ecf.debug"}

Fixes #3083.
2019-09-04 19:28:26 +02:00
Eelco Dolstra 6f88fed819
Disable OpenSSL lock callback on OpenSSL >= 1.1.1 2019-09-04 14:14:03 +02:00
Eelco Dolstra 4caeefaf00
Revert "Remove obsolete OpenSSL locking code"
This reverts commit aeb695c007.
2019-09-04 14:06:52 +02:00
Eelco Dolstra e302ba0e65
Merge remote-tracking branch 'origin/master' into flakes 2019-09-04 13:30:11 +02:00
Eelco Dolstra e07ec8d27e
Support allowSubstitutes attribute in structured attribute derivations
Hopefully fixes #3081 (didn't test).
2019-09-03 16:03:49 +02:00
Eelco Dolstra f186000367
Add some noexcepts
This is to assert that callback functions should never throw (since
the context in which they're called may not be able to handle the
exception).
2019-09-03 13:45:35 +02:00
Eelco Dolstra 7348653ff4
Ensure that Callback is called only once
Also, make Callback movable but uncopyable.
2019-09-03 13:45:35 +02:00
Eelco Dolstra 8c4ea7a451
Downloader: Remove a possible double call to Callback 2019-09-03 13:45:32 +02:00
Eelco Dolstra 2dbd69dbf4 nix repl: Run in impure mode 2019-09-02 23:04:27 +02:00
Eelco Dolstra aeb695c007
Remove obsolete OpenSSL locking code
OpenSSL 1.1.1 no longer needs this (2e52e7df51).

This shuts up a clang warning about opensslLockCallback being unused.
2019-09-02 17:50:44 +02:00
Eelco Dolstra c693f80b81
Shut up some clang warnings 2019-09-02 17:43:27 +02:00
Eelco Dolstra 61fdb16aac
Improve error message when a directory is not a flake
So you now get

  $ nix build
  error: path '.' is not a flake (because it does not reference a Git repository)

rather than

  $ nix build
  error: unsupported argument '.'
2019-09-02 17:35:35 +02:00
Eelco Dolstra 5ec2a1ed82
nix dev-shell --profile: Support relative path 2019-09-02 15:59:19 +02:00
Eelco Dolstra a49b6761a5 Fix sourceInfo 2019-08-30 17:27:51 +02:00
Eelco Dolstra 80c36d4562 Remove 'name' attribute from flakes
This is no longer needed since flakes are given an identity in the
'inputs' attribute.
2019-08-30 16:38:27 +02:00
Eelco Dolstra 30ccf4e52d Turn flake inputs into an attrset
Instead of a list, inputs are now an attrset like

  inputs = {
    nixpkgs.uri = github:NixOS/nixpkgs;
  };

If 'uri' is omitted, than the flake is a lookup in the flake registry, e.g.

  inputs = {
    nixpkgs = {};
  };

but in that case, you can also just omit the input altogether and
specify it as an argument to the 'outputs' function, as in

  outputs = { self, nixpkgs }: ...

This also gets rid of 'nonFlakeInputs', which are now just a special
kind of input that have a 'flake = false' attribute, e.g.

  inputs = {
    someRepo = {
      uri = github:example/repo;
      flake = false;
    };
  };
2019-08-30 16:27:51 +02:00
Eelco Dolstra 2341f30ec6 Clean up the 'outputs' interface 2019-08-30 13:06:23 +02:00
Eelco Dolstra 89468410d5 Extract flake dependencies from the 'outputs' arguments
That is, instead of

  inputs = [ "nixpkgs" ];

  outputs = inputs: ... inputs.nixpkgs ...;

you can write

  outputs = { nixpkgs }: ... inputs.nixpkgs ...;
2019-08-30 11:22:34 +02:00
Eelco Dolstra ebc4dae517 Merge remote-tracking branch 'origin/master' into flakes 2019-08-29 16:11:38 +02:00
Eelco Dolstra 84de821004
Merge pull request #3069 from matthewbauer/max-name
Set maximum name length in Nix
2019-08-29 15:22:36 +02:00
Eelco Dolstra a2c4fcd5e9 Don't rely on st_blocks
It doesn't seem very reliable on ZFS.
2019-08-29 14:49:58 +02:00
Eelco Dolstra f27e53f77e Cleanup 2019-08-29 12:09:58 +02:00
Eelco Dolstra 7ef2645f45
Merge pull request #2921 from matthewbauer/handle-sigwinch
Handle SIGWINCH in main thread
2019-08-28 21:48:14 +02:00
Matthew Bauer 693e68e09c Set maximum name length in Nix
Previously we allowed any length of name for Nix derivations. This is
bad because different file systems have different max lengths. To make
things predictable, I have picked a max. This was done by trying to
build this derivation:

  derivation {
    name = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
    builder = "/no-such-path";
    system = "x86_64-linux";
  }

Take off one a and it will not lead to file name too long. That ends
up being 212 a’s. An even smaller max could be picked if we want to
support more file systems.

Working backwards, this is why:

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-${name}.drv.chroot

> 255 - 32 - 1 - 4 - 7 = 211
2019-08-28 12:32:54 -04:00
Eelco Dolstra 7298a38a07
Don't send certain setting overrides to the daemon
These are already handled separately. This fixes warnings like

  warning: ignoring the user-specified setting 'max-jobs', because it is a restricted setting and you are not a trusted user

when using the -j flag.
2019-08-28 16:29:44 +02:00
Eelco Dolstra e5b397b2c7 Merge branch 'test-sandboxing' of https://github.com/matthewbauer/nix 2019-08-27 20:58:47 +02:00
Matthew Bauer 5c06a8d328 Reset tmpDirInSandbox for unsandboxed 2019-08-23 20:24:39 -04:00
Jonas Chevalier b226b5cd97
nix-store: fix out of sync protocol
If a NAR is already in the store, addToStore doesn't read the source
which makes the protocol go out of sync. This happens for example when
two client try to nix-copy-closure the same derivation at the same time.
2019-08-16 15:05:45 +02:00
Jonas Chevalier 91b00b145f
libutil: add SizedSource
Introduce the SizeSource which allows to bound how much data is being
read from a source. It also contains a drainAll() function to discard
the rest of the source, useful to keep the nix protocol in sync.
2019-08-16 15:05:40 +02:00
Eelco Dolstra 477f82e5a7
Merge pull request #2782 from grahamc/flames
Track function start and end
2019-08-15 14:20:42 +02:00
Graham Christensen ee9c988a1b
Track function start and ends for flame graphs
With this patch, and this file I called `log.py`:

    #!/usr/bin/env nix-shell
    #!nix-shell -i python3 -p python3 --pure

    import sys
    from pprint import pprint

    stack = []
    timestack = []

    for line in open(sys.argv[1]):
        components = line.strip().split(" ", 2)
        if components[0] != "function-trace":
            continue

        direction = components[1]
        components = components[2].rsplit(" ", 2)

        loc = components[0]
        _at = components[1]
        time = int(components[2])

        if direction == "entered":
            stack.append(loc)
            timestack.append(time)
        elif direction == "exited":
            dur = time - timestack.pop()
            vst = ";".join(stack)
            print(f"{vst} {dur}")
            stack.pop()

and:

    nix-instantiate --trace-function-calls -vvvv ../nixpkgs/pkgs/top-level/release.nix -A unstable > log.matthewbauer 2>&1
    ./log.py ./log.matthewbauer > log.matthewbauer.folded
    flamegraph.pl --title matthewbauer-post-pr log.matthewbauer.folded > log.matthewbauer.folded.svg

I can make flame graphs like: http://gsc.io/log.matthewbauer.folded.svg

---

Includes test cases around function call failures and tryEval. Uses
RAII so the finish is always called at the end of the function.
2019-08-14 16:09:35 -04:00