Commit graph

4179 commits

Author SHA1 Message Date
Daiderd Jordan 7d448bc966
build: fix path repairing when hash rewriting is required
Handle store path repairing on darwin when sandboxing is enabled. Unlike
on linux sandboxing on darwin still requires hash rewriting.
2020-01-04 20:25:25 +01:00
Daiderd Jordan b33fefcb92
build: recover store path when replacing fails
This shouldn't happen in normal circumstances, but just in case
attempt to move the temporary path back if possible.
2020-01-04 20:24:27 +01:00
edef c65a6fa86a passAsFile: leave out the hash prefix
Having a colon in the path may cause issues, and having the hash
function indicated isn't actually necessary. We now verify the path 
format in the tests to prevent regressions.
2020-01-02 23:56:06 +00:00
Puck Meerburg 515c0a263e passAsFile: hash the attribute name instead of numbering sequentially
This makes the paths consistent without relying on ordering.

Co-authored-by: edef <edef@edef.eu>
2020-01-02 22:56:03 +00:00
Graham Christensen c502831a1d
exportReferencesGraph: support working
Before, we would get:

    [deploy@bastion:~]$ nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv
    these derivations will be built:
      /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv
      /nix/store/ssxwmll7v21did1c8j027q0m8w6pg41i-unit-prometheus-alertmanager-irc-notifier.service.drv
      /nix/store/mvyvkj46ay7pp7b1znqbkck2mq98k0qd-unit-script-network-local-commands-start.drv
      /nix/store/vsl1y9mz38qfk6pyirjwnfzfggz5akg6-unit-network-local-commands.service.drv
      /nix/store/wi5ighfwwb83fdmav6z6n2fw6npm9ffl-unit-prometheus-hydra-exporter.service.drv
      /nix/store/x0qkv535n75pbl3xn6nn1w7qkrg9wwyg-unit-prometheus-packet-sd.service.drv
      /nix/store/lv491znsjxdf51xnfxh9ld7r1zg14d52-unit-script-packet-sd-env-key-pre-start.drv
      /nix/store/nw4nzlca49agsajvpibx7zg5b873gk9f-unit-script-packet-sd-env-key-start.drv
      /nix/store/x674wwabdwjrkhnykair4c8mpxa9532w-unit-packet-sd-env-key.service.drv
      /nix/store/ywivz64ilb1ywlv652pkixw3vxzfvgv8-unit-wireguard-wg0.service.drv
      /nix/store/v3b648293g3zl8pnn0m1345nvmyd8dwb-unit-script-acme-selfsigned-status.nixos.org-start.drv
      /nix/store/zci5d3zvr6fgdicz6k7jjka6lmx0v3g4-unit-acme-selfsigned-status.nixos.org.service.drv
      /nix/store/f6pwvnm63d0kw5df0v7sipd1rkhqxk5g-system-units.drv
      /nix/store/iax8071knxk9c7krpm9jqg0lcrawf4lc-etc.drv
      /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv
    error: invalid file name 'closure-init-0' in 'exportReferencesGraph'

This was tough to debug, I didn't figure out which one was broken until I did:

    nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv 2>&1 | grep  nix/store | xargs -n1 nix-store -r

and then looking at the remaining build graph:

    $ nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv
    these derivations will be built:
      /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv
      /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv
    error: invalid file name 'closure-init-0' in 'exportReferencesGraph'

and knowing the initrd build is before the system, then:

    $ nix show-derivation /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv
    {
      "/nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv": {
        [...]
        "exportReferencesGraph": "closure-init-0 /nix/store/...-stage-1-init.sh closure-mdadm.conf-1 /nix/store/...-mdadm.conf closure-ubuntu.conf-2 ...",
        [...]
      }
    }

I then searched the repo for "in 'exportReferencesGraph'", found this
recently updated regex, and realized it was missing a "-".
2020-01-01 20:50:40 -05:00
Brian McKenna d25923263e Disable use-sqlite-wal under WSL
Before:

    $ nix-channel --update
    unpacking channels...
    warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)
    warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)
    warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)
    warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)
    warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)

After:

    $ inst/bin/nix-channel --update
    unpacking channels...
    created 1 symlinks in user environment

I've seen complaints that "sandbox" caused problems under WSL but I'm
having no problems. I think recent changes could have fixed the issue.
2019-12-21 08:14:19 +11:00
Eelco Dolstra c84c843e33
tarfile.cc: Restore timestamps
This is needed to get the lastModified attribute of GitHub flakes.
2019-12-19 15:09:54 +01:00
Eelco Dolstra 2550c11373
tarfile.cc: Don't change the cwd
Nix is multithreaded so it's not safe to change the cwd.
2019-12-19 15:08:16 +01:00
Eelco Dolstra be32da0ed0
tarfile.cc: Style fixes 2019-12-19 15:01:58 +01:00
Eelco Dolstra ee235e764c
Merge branch 'libarchive' of https://github.com/yorickvP/nix 2019-12-19 14:47:18 +01:00
Eelco Dolstra 4511f09b49 nix make-content-addressable: Add --json flag
Fixes #3274.
2019-12-18 17:39:02 +01:00
Eelco Dolstra f8abbdd456 Add priority setting to stores
This allows overriding the priority of substituters, e.g.

  $ nix-store --store ~/my-nix/ -r /nix/store/df3m4da96d84ljzxx4mygfshm1p0r2n3-geeqie-1.4 \
    --substituters 'http://cache.nixos.org?priority=100 daemon?priority=10'

Fixes #3264.
2019-12-17 17:17:53 +01:00
Eelco Dolstra 54bf5ba422 nix-store -r: Handle symlinks to store paths
Fixes #3270.
2019-12-16 19:11:47 +01:00
Eelco Dolstra acb71aa5c6 Tweak error message 2019-12-15 10:44:53 +01:00
Eelco Dolstra 2b0365753a Merge branch 'limit_depth_resolveExprPath' of https://github.com/d-goldin/nix 2019-12-15 00:22:35 +01:00
Eelco Dolstra 8656a2de56
Merge pull request #3269 from xzfc/nix-shell
nix-shell: don't check for "nix-shell" in shebang script name
2019-12-14 23:24:32 +01:00
Eelco Dolstra ba6d2093c7 Fix progress bar 2019-12-14 23:19:04 +01:00
Albert Safin a70706b025 nix-shell: don't check for "nix-shell" in shebang script name 2019-12-14 15:37:20 +00:00
Eelco Dolstra ac9cc2ec08 Move some code 2019-12-13 19:10:39 +01:00
Eelco Dolstra b4edc3ca61 Don't leak exceptions 2019-12-13 19:05:26 +01:00
Eelco Dolstra e6bd88878e Improve gzip error message 2019-12-13 19:05:26 +01:00
Eelco Dolstra ca87707c90 Get rid of CBox 2019-12-13 19:05:26 +01:00
Dima d89d9958a7 bugfix: Adding depth limit to resolveExprPath
There is no termination condition for evaluation of cyclical
expression paths which can lead to infinite loops. This addresses
one spot in the parser in a similar fashion as utils.cc/canonPath
does.

This issue can be reproduced by something like:

```
ln -s a b
ln -s b a

nix-instantiate -E 'import ./a'
```
2019-12-13 14:51:30 +01:00
Eelco Dolstra e8aa2290ed Only install *.sb files on macOS 2019-12-13 14:42:55 +01:00
Eelco Dolstra 3e787423c2 Remove FIXME 2019-12-13 12:55:52 +01:00
Eelco Dolstra d1b238ec3c Simplify 2019-12-13 12:53:20 +01:00
Eelco Dolstra 2da4c61049 Merge branch 'libstore-ssh-better-exec-error-message' of https://github.com/Profpatsch/nix 2019-12-13 12:51:36 +01:00
Tom Bereknyei c6295a3afd Initial gzip support
Closes #3256
2019-12-13 03:34:15 -05:00
Profpatsch 38b29fb72c libstore/ssh: Improve error message on failing execvp
If the `throw` is reached, this means that execvp into `ssh` wasn’t
successful. We can hint at a usual problem, which is a missing `ssh`
executable.

Test with:

```
env PATH= ./result/bin/nix-copy-closure --builders '' unusedhost
```

and the bash version with

```
env PATH= ./result/bin/nix-copy-closure --builders '' localhost
```
2019-12-12 15:32:17 +01:00
Eelco Dolstra bbe97dff8b Make the Store API more type-safe
Most functions now take a StorePath argument rather than a Path (which
is just an alias for std::string). The StorePath constructor ensures
that the path is syntactically correct (i.e. it looks like
<store-dir>/<base32-hash>-<name>). Similarly, functions like
buildPaths() now take a StorePathWithOutputs, rather than abusing Path
by adding a '!<outputs>' suffix.

Note that the StorePath type is implemented in Rust. This involves
some hackery to allow Rust values to be used directly in C++, via a
helper type whose destructor calls the Rust type's drop()
function. The main issue is the dynamic nature of C++ move semantics:
after we have moved a Rust value, we should not call the drop function
on the original value. So when we move a value, we set the original
value to bitwise zero, and the destructor only calls drop() if the
value is not bitwise zero. This should be sufficient for most types.

Also lots of minor cleanups to the C++ API to make it more modern
(e.g. using std::optional and std::string_view in some places).
2019-12-10 22:06:05 +01:00
Eelco Dolstra 98ef11677c EvalState::callFunction(): Make FunctionCallTrace use less stack space
The FunctionCallTrace object consumes a few hundred bytes of stack
space, even when tracing is disabled. This was causing stack overflows:

  $ nix-instantiate '<nixpkgs> -A texlive.combined.scheme-full --dry-run
  error: stack overflow (possible infinite recursion)

This is with the default stack size of 8 MiB.

Putting the object on the heap reduces stack usage to < 5 MiB.
2019-12-10 13:32:30 +01:00
Eelco Dolstra 61cc9f34d2 Remove UserLock self-lock check
This is no longer needed since we're not using POSIX locks anymore.
2019-12-09 23:57:33 +01:00
Yorick eba82b7c88
further clean up libarchive code 2019-12-09 17:21:46 +07:00
puck 28ee687adf Clean up libarchive support 2019-12-07 18:12:21 +00:00
Yorick fe7ec70e6b
remove rust unpack_tarfile ffi 2019-12-07 23:28:31 +07:00
Yorick 1355554d12
code 'cleanup' 2019-12-07 23:23:11 +07:00
Yorick f54c168031
add wrapper function around libarchive to c++ errors 2019-12-07 23:10:27 +07:00
Yorick 232b390766
fixup! libarchive proof of concept 2019-12-07 23:00:37 +07:00
Yorick 9ff5f6492f
libarchive proof of concept 2019-12-07 22:35:14 +07:00
Eelco Dolstra 3b9c9d34e5 Shut up clang warning
(cherry picked from commit 3392f1b778)
2019-12-05 20:41:44 +01:00
Eelco Dolstra 80ab95315d nix doctor: Fix typo
(cherry picked from commit 96c6b08ed7)
2019-12-05 20:40:52 +01:00
Eelco Dolstra 47a937d512 Show hash mismatch warnings in SRI format
(cherry picked from commit 63c5c91cc0)
2019-12-05 20:32:25 +01:00
Eelco Dolstra 0678e4d56a Move #include
(cherry picked from commit 8beedd4486)
2019-12-05 20:30:29 +01:00
Eelco Dolstra 79142cbbe1 Bindings: Add convenience method for requiring an attribute
(cherry picked from commit fb692e5f7b)
2019-12-05 20:29:15 +01:00
Eelco Dolstra 0d118ef0c9 Bindings::get(): Add convenience method
This allows writing attribute lookups as

    if (auto name = value.attrs->get(state.sName))
      ...

(cherry picked from commit f216c76c56)
2019-12-05 20:29:00 +01:00
Eelco Dolstra 5e449b43ed Initialize Command::_name
(cherry picked from commit d0a769cb06)
2019-12-05 20:21:22 +01:00
Eelco Dolstra ac67685606 Make subcommand construction in MultiCommand lazy
(cherry picked from commit a0de58f471)
2019-12-05 20:19:26 +01:00
Eelco Dolstra f964f428fe Move Command and MultiCommand to libutil
(cherry picked from commit f70434b1fb)
2019-12-05 20:13:47 +01:00
Eelco Dolstra f1b5c76c1a MultiCommand: Simplify construction
(cherry picked from commit 15a16e5c05)
2019-12-05 20:10:35 +01:00
Eelco Dolstra 092af3c826 Eliminate more pass-by-value in variadic calls 2019-12-05 19:58:52 +01:00