Commit graph

7271 commits

Author SHA1 Message Date
Eelco Dolstra 04bbfa692f
Merge pull request #3305 from knl/interpret-u-escapes-in-JSON-strings
Add support for unicode escape sequences in fromJSON
2020-01-07 01:04:16 +01:00
Nikola Knezevic 52a8f9295b Add support for \u escape in fromJSON
As fromTOML supports \u and \U escapes, bring fromJSON on par. As JSON defaults
to UTF-8 encoding (every JSON parser must support UTF-8), this change parses the
`\u hex hex hex hex` sequence (\u followed by 4 hexadecimal digits) into an
UTF-8 representation.

Add a test to verify correct parsing, using all escape sequences from json.org.
2020-01-07 00:09:58 +01:00
Nikola Knezevic cb2d348d48 Remove redundant check in parseJSONString 2020-01-07 00:09:58 +01:00
Eelco Dolstra bc22a7ee6a Fix use of uninitialized store path
Fixes 'building of '/nix/store/00000000000000000000000000000000-': ...'.
2020-01-06 22:20:10 +01:00
Eelco Dolstra e2988f48a1
Merge pull request #3303 from LnL7/darwin-sandbox
build: fix sandboxing on darwin
2020-01-06 20:56:35 +01:00
Daiderd Jordan 66fccd5832
build: fix sandboxing on darwin
Starting ba87b08f85 getEnv now returns an
std::optional which means these getEnv() != "" conditions no longer happen
if the variables are not defined.
2020-01-05 20:23:52 +01:00
Eelco Dolstra 0486e87791
Merge pull request #3302 from LnL7/darwin-repair-with-sandbox
build: fix path repairing with the darwin sandbox
2020-01-05 16:26:17 +01:00
Eelco Dolstra cb90e382b5 Hide FunctionCallTrace constructor/destructor
This prevents them from being inlined. On gcc 9, this reduces the
stack size needed for

  nix-instantiate '<nixpkgs>' -A texlive.combined.scheme-full --dry-run

from 12.9 MiB to 4.8 MiB.
2020-01-05 16:21:34 +01:00
Daiderd Jordan 7d448bc966
build: fix path repairing when hash rewriting is required
Handle store path repairing on darwin when sandboxing is enabled. Unlike
on linux sandboxing on darwin still requires hash rewriting.
2020-01-04 20:25:25 +01:00
Daiderd Jordan b33fefcb92
build: recover store path when replacing fails
This shouldn't happen in normal circumstances, but just in case
attempt to move the temporary path back if possible.
2020-01-04 20:24:27 +01:00
Eelco Dolstra 0de33cc81b
Merge pull request #3298 from edef1c/passasfile-noprefix
passAsFile: leave out the hash prefix
2020-01-03 12:43:06 +01:00
edef c65a6fa86a passAsFile: leave out the hash prefix
Having a colon in the path may cause issues, and having the hash
function indicated isn't actually necessary. We now verify the path 
format in the tests to prevent regressions.
2020-01-02 23:56:06 +00:00
Eelco Dolstra 3ad4a332eb
Merge pull request #3297 from edef1c/passasfile-hash
passAsFile: hash the attribute name instead of numbering sequentially
2020-01-03 00:08:23 +01:00
Puck Meerburg 515c0a263e passAsFile: hash the attribute name instead of numbering sequentially
This makes the paths consistent without relying on ordering.

Co-authored-by: edef <edef@edef.eu>
2020-01-02 22:56:03 +00:00
Eelco Dolstra 3469062e76
Merge pull request #3296 from grahamc/export-reference-graph
exportReferencesGraph: support working
2020-01-02 11:04:03 +01:00
Graham Christensen c502831a1d
exportReferencesGraph: support working
Before, we would get:

    [deploy@bastion:~]$ nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv
    these derivations will be built:
      /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv
      /nix/store/ssxwmll7v21did1c8j027q0m8w6pg41i-unit-prometheus-alertmanager-irc-notifier.service.drv
      /nix/store/mvyvkj46ay7pp7b1znqbkck2mq98k0qd-unit-script-network-local-commands-start.drv
      /nix/store/vsl1y9mz38qfk6pyirjwnfzfggz5akg6-unit-network-local-commands.service.drv
      /nix/store/wi5ighfwwb83fdmav6z6n2fw6npm9ffl-unit-prometheus-hydra-exporter.service.drv
      /nix/store/x0qkv535n75pbl3xn6nn1w7qkrg9wwyg-unit-prometheus-packet-sd.service.drv
      /nix/store/lv491znsjxdf51xnfxh9ld7r1zg14d52-unit-script-packet-sd-env-key-pre-start.drv
      /nix/store/nw4nzlca49agsajvpibx7zg5b873gk9f-unit-script-packet-sd-env-key-start.drv
      /nix/store/x674wwabdwjrkhnykair4c8mpxa9532w-unit-packet-sd-env-key.service.drv
      /nix/store/ywivz64ilb1ywlv652pkixw3vxzfvgv8-unit-wireguard-wg0.service.drv
      /nix/store/v3b648293g3zl8pnn0m1345nvmyd8dwb-unit-script-acme-selfsigned-status.nixos.org-start.drv
      /nix/store/zci5d3zvr6fgdicz6k7jjka6lmx0v3g4-unit-acme-selfsigned-status.nixos.org.service.drv
      /nix/store/f6pwvnm63d0kw5df0v7sipd1rkhqxk5g-system-units.drv
      /nix/store/iax8071knxk9c7krpm9jqg0lcrawf4lc-etc.drv
      /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv
    error: invalid file name 'closure-init-0' in 'exportReferencesGraph'

This was tough to debug, I didn't figure out which one was broken until I did:

    nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv 2>&1 | grep  nix/store | xargs -n1 nix-store -r

and then looking at the remaining build graph:

    $ nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv
    these derivations will be built:
      /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv
      /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv
    error: invalid file name 'closure-init-0' in 'exportReferencesGraph'

and knowing the initrd build is before the system, then:

    $ nix show-derivation /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv
    {
      "/nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv": {
        [...]
        "exportReferencesGraph": "closure-init-0 /nix/store/...-stage-1-init.sh closure-mdadm.conf-1 /nix/store/...-mdadm.conf closure-ubuntu.conf-2 ...",
        [...]
      }
    }

I then searched the repo for "in 'exportReferencesGraph'", found this
recently updated regex, and realized it was missing a "-".
2020-01-01 20:50:40 -05:00
Eelco Dolstra b0cadf547b
Merge pull request #3289 from michaelforney/tar-J
Pass -J to tar for xz decompression
2019-12-25 13:07:58 +01:00
Michael Forney 43eb7b6756 Pass -J to tar for xz decompression
Some tar implementations can't auto-detect compression formats, so
they must be specified explicitly.
2019-12-22 17:17:14 -08:00
Eelco Dolstra aaf57c983d
Merge pull request #3284 from puffnfresh/wsl
Disable use-sqlite-wal under WSL
2019-12-23 00:44:29 +01:00
Eelco Dolstra 7dcfa8042e
Merge pull request #3287 from michaelforney/cp-flag
Pass -P to cp to preserve symlinks
2019-12-23 00:43:38 +01:00
Michael Forney 10414d467b Pass -P to cp to preserve symlinks
This is commonly the default behavior with -R, but POSIX leaves the
default unspecified.
2019-12-21 21:30:38 -08:00
Brian McKenna d25923263e Disable use-sqlite-wal under WSL
Before:

    $ nix-channel --update
    unpacking channels...
    warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)
    warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)
    warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)
    warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)
    warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)

After:

    $ inst/bin/nix-channel --update
    unpacking channels...
    created 1 symlinks in user environment

I've seen complaints that "sandbox" caused problems under WSL but I'm
having no problems. I think recent changes could have fixed the issue.
2019-12-21 08:14:19 +11:00
Eelco Dolstra c84c843e33
tarfile.cc: Restore timestamps
This is needed to get the lastModified attribute of GitHub flakes.
2019-12-19 15:09:54 +01:00
Eelco Dolstra 2550c11373
tarfile.cc: Don't change the cwd
Nix is multithreaded so it's not safe to change the cwd.
2019-12-19 15:08:16 +01:00
Eelco Dolstra be32da0ed0
tarfile.cc: Style fixes 2019-12-19 15:01:58 +01:00
Eelco Dolstra ee235e764c
Merge branch 'libarchive' of https://github.com/yorickvP/nix 2019-12-19 14:47:18 +01:00
Eelco Dolstra 9f7b4d068c
Cleanup: Remove unused makeDeb/makeRPM functions 2019-12-19 13:17:48 +01:00
Eelco Dolstra 4511f09b49 nix make-content-addressable: Add --json flag
Fixes #3274.
2019-12-18 17:39:02 +01:00
Eelco Dolstra f8abbdd456 Add priority setting to stores
This allows overriding the priority of substituters, e.g.

  $ nix-store --store ~/my-nix/ -r /nix/store/df3m4da96d84ljzxx4mygfshm1p0r2n3-geeqie-1.4 \
    --substituters 'http://cache.nixos.org?priority=100 daemon?priority=10'

Fixes #3264.
2019-12-17 17:17:53 +01:00
Eelco Dolstra 54bf5ba422 nix-store -r: Handle symlinks to store paths
Fixes #3270.
2019-12-16 19:11:47 +01:00
Eelco Dolstra 14d82baba4 StorePath::new(): Check store directory 2019-12-16 17:41:56 +01:00
Eelco Dolstra 410acd29c0 Fix cargo test 2019-12-15 10:47:59 +01:00
Eelco Dolstra acb71aa5c6 Tweak error message 2019-12-15 10:44:53 +01:00
Eelco Dolstra 2b0365753a Merge branch 'limit_depth_resolveExprPath' of https://github.com/d-goldin/nix 2019-12-15 00:22:35 +01:00
Eelco Dolstra 8656a2de56
Merge pull request #3269 from xzfc/nix-shell
nix-shell: don't check for "nix-shell" in shebang script name
2019-12-14 23:24:32 +01:00
Eelco Dolstra ba6d2093c7 Fix progress bar 2019-12-14 23:19:04 +01:00
Albert Safin a70706b025 nix-shell: don't check for "nix-shell" in shebang script name 2019-12-14 15:37:20 +00:00
Eelco Dolstra ac9cc2ec08 Move some code 2019-12-13 19:10:39 +01:00
Eelco Dolstra b4edc3ca61 Don't leak exceptions 2019-12-13 19:05:26 +01:00
Eelco Dolstra e6bd88878e Improve gzip error message 2019-12-13 19:05:26 +01:00
Eelco Dolstra ca87707c90 Get rid of CBox 2019-12-13 19:05:26 +01:00
Eelco Dolstra 5a6d6da7ae Validate tarball components 2019-12-13 19:05:26 +01:00
Eelco Dolstra 4581159e3f Simplify tarball test 2019-12-13 17:26:58 +01:00
Dima d89d9958a7 bugfix: Adding depth limit to resolveExprPath
There is no termination condition for evaluation of cyclical
expression paths which can lead to infinite loops. This addresses
one spot in the parser in a similar fashion as utils.cc/canonPath
does.

This issue can be reproduced by something like:

```
ln -s a b
ln -s b a

nix-instantiate -E 'import ./a'
```
2019-12-13 14:51:30 +01:00
Eelco Dolstra e8aa2290ed Only install *.sb files on macOS 2019-12-13 14:42:55 +01:00
Eelco Dolstra 3e787423c2 Remove FIXME 2019-12-13 12:55:52 +01:00
Eelco Dolstra d1b238ec3c Simplify 2019-12-13 12:53:20 +01:00
Eelco Dolstra 2da4c61049 Merge branch 'libstore-ssh-better-exec-error-message' of https://github.com/Profpatsch/nix 2019-12-13 12:51:36 +01:00
Tom Bereknyei c6295a3afd Initial gzip support
Closes #3256
2019-12-13 03:34:15 -05:00
Profpatsch 38b29fb72c libstore/ssh: Improve error message on failing execvp
If the `throw` is reached, this means that execvp into `ssh` wasn’t
successful. We can hint at a usual problem, which is a missing `ssh`
executable.

Test with:

```
env PATH= ./result/bin/nix-copy-closure --builders '' unusedhost
```

and the bash version with

```
env PATH= ./result/bin/nix-copy-closure --builders '' localhost
```
2019-12-12 15:32:17 +01:00