Commit graph

7391 commits

Author SHA1 Message Date
Eelco Dolstra b6120d26a8
gc-auto.sh: Increase verbosity 2019-08-28 22:19:31 +02:00
Eelco Dolstra c128031492
Fix macOS build
https://hydra.nixos.org/build/99500938
2019-08-28 22:04:45 +02:00
Eelco Dolstra 7ef2645f45
Merge pull request #2921 from matthewbauer/handle-sigwinch
Handle SIGWINCH in main thread
2019-08-28 21:48:14 +02:00
Matthew Bauer 693e68e09c Set maximum name length in Nix
Previously we allowed any length of name for Nix derivations. This is
bad because different file systems have different max lengths. To make
things predictable, I have picked a max. This was done by trying to
build this derivation:

  derivation {
    name = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
    builder = "/no-such-path";
    system = "x86_64-linux";
  }

Take off one a and it will not lead to file name too long. That ends
up being 212 a’s. An even smaller max could be picked if we want to
support more file systems.

Working backwards, this is why:

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-${name}.drv.chroot

> 255 - 32 - 1 - 4 - 7 = 211
2019-08-28 12:32:54 -04:00
Eelco Dolstra 7298a38a07
Don't send certain setting overrides to the daemon
These are already handled separately. This fixes warnings like

  warning: ignoring the user-specified setting 'max-jobs', because it is a restricted setting and you are not a trusted user

when using the -j flag.
2019-08-28 16:29:44 +02:00
Eelco Dolstra ad03159e25
Merge pull request #2745 from samueldr/install/detect-systemd-separately
install-multi-user: Detect and fail lack of systemd separately
2019-08-28 11:34:23 +02:00
Eelco Dolstra bd285849ed
Merge pull request #3054 from matthewbauer/nix-dir-macos
Allow empty /nix directory in multi-user installer
2019-08-28 11:29:43 +02:00
Eelco Dolstra 5fa8b3f965 Update the libboost hack
This cuts about 46 MiB from the closure.
2019-08-27 22:38:48 +02:00
Eelco Dolstra ceefddafe8 Compress binary tarballs using xz
Fixes https://github.com/NixOS/nix/issues/240.

Apparently 'tar -xf' can decompress xz files on macOS nowadays.
2019-08-27 22:18:34 +02:00
Eelco Dolstra 787015fec0 Merge branch 'master' of github.com:NixOS/nix 2019-08-27 21:18:20 +02:00
Eelco Dolstra fdff96501f Update release notes 2019-08-27 21:18:00 +02:00
Eelco Dolstra e5b397b2c7 Merge branch 'test-sandboxing' of https://github.com/matthewbauer/nix 2019-08-27 20:58:47 +02:00
Eelco Dolstra 177e5742fa
Merge pull request #3056 from grahamc/operators
operators: document exact precedence, split up similar operators
2019-08-27 17:12:36 +02:00
Eelco Dolstra 73728874ab Hopefully fix post-hook test on macOS
https://hydra.nixos.org/build/99262744
2019-08-27 17:01:54 +02:00
Eelco Dolstra 45b3dc325a Add 2.3 release notes 2019-08-27 17:00:04 +02:00
Matthew Bauer 800fba1037 Use wait4path on org.nixos.nix-daemon.plist
When using a volume, the nix-daemon path may not exist. To avoid this
issue, we must use the wait4path tool. This should solve one of the
issues in multi-user on macOS Catalina.
2019-08-27 10:58:48 -04:00
Graham Christensen 171d784404
docs: operators: Make OR and AND capitalized 2019-08-27 06:55:22 -04:00
Eelco Dolstra ee07ce7554
Merge pull request #3064 from pszubiak/systemd-unit-service-fix
nix-daemon.service: add install section.
2019-08-27 10:50:07 +02:00
Piotr Szubiakowski d459224724 nix-daemon.service: add install section.
Signed-off-by: Piotr Szubiakowski <pszubiak@eso.org>
2019-08-27 10:35:35 +02:00
Graham Christensen 15ee2bc2fe
Merge pull request #2946 from vmandela/proxy
installer: handle network proxy in systemd multi-user install
2019-08-24 14:55:19 -04:00
Graham Christensen 057af1dbd8
docs: document the installer's use of proxy env vars 2019-08-24 09:08:44 -04:00
Venkateswara Rao Mandela 6dab42a551
installer: handle network proxy in systemd install
If a network proxy configuration is detected, setup an override
systemd unit file for nix-daemon service with the non-empty
proxy variables.

Proxy detection is performed by looking for http/https/ftp proxy and no
proxy variables in user environment
2019-08-24 09:08:41 -04:00
Matthew Bauer 5c06a8d328 Reset tmpDirInSandbox for unsandboxed 2019-08-23 20:24:39 -04:00
Graham Christensen 92ddce4f46
operators: document exact precedenc, split up similar operators 2019-08-23 15:50:54 -04:00
Matthew Bauer 0463d5e36f Allow empty /nix directory in multi-user installer
With macOS catalina, we can no longer modify the root system
volume (#2925). macOS provides a system configuration file in
synthetic.conf(5) to create empty root directories. This can be used
to mount /nix to a separate volume. As a result, this directory will
need to already exist prior to installation. Instead, check for
/nix/store and /nix/var for a live Nix installation.
2019-08-22 23:38:52 -04:00
Toon Nolten 1dbaf11948 Fix nix-env documentation for --delete-generations
The documentation for `--delete-generations` had an erroneous fullstop
and as it turns out inaccurate information on the `+No.` syntax.
2019-08-17 16:33:35 +02:00
Eelco Dolstra f435634a29
Merge pull request #3041 from zimbatm/nix-store-error-13
Fix for `unknown serve command 13`
2019-08-16 16:15:05 +02:00
Jonas Chevalier b226b5cd97
nix-store: fix out of sync protocol
If a NAR is already in the store, addToStore doesn't read the source
which makes the protocol go out of sync. This happens for example when
two client try to nix-copy-closure the same derivation at the same time.
2019-08-16 15:05:45 +02:00
Jonas Chevalier 91b00b145f
libutil: add SizedSource
Introduce the SizeSource which allows to bound how much data is being
read from a source. It also contains a drainAll() function to discard
the rest of the source, useful to keep the nix protocol in sync.
2019-08-16 15:05:40 +02:00
Eelco Dolstra b7ea98bf34
Merge pull request #2945 from danidiaz/doc001
Expanded documentation for .nix-defexpr
2019-08-15 15:03:21 +02:00
Eelco Dolstra 477f82e5a7
Merge pull request #2782 from grahamc/flames
Track function start and end
2019-08-15 14:20:42 +02:00
Daniel Diaz 653c407784
Expanded documentation for .nix-defexpr 2019-08-15 08:05:22 -04:00
Graham Christensen ee9c988a1b
Track function start and ends for flame graphs
With this patch, and this file I called `log.py`:

    #!/usr/bin/env nix-shell
    #!nix-shell -i python3 -p python3 --pure

    import sys
    from pprint import pprint

    stack = []
    timestack = []

    for line in open(sys.argv[1]):
        components = line.strip().split(" ", 2)
        if components[0] != "function-trace":
            continue

        direction = components[1]
        components = components[2].rsplit(" ", 2)

        loc = components[0]
        _at = components[1]
        time = int(components[2])

        if direction == "entered":
            stack.append(loc)
            timestack.append(time)
        elif direction == "exited":
            dur = time - timestack.pop()
            vst = ";".join(stack)
            print(f"{vst} {dur}")
            stack.pop()

and:

    nix-instantiate --trace-function-calls -vvvv ../nixpkgs/pkgs/top-level/release.nix -A unstable > log.matthewbauer 2>&1
    ./log.py ./log.matthewbauer > log.matthewbauer.folded
    flamegraph.pl --title matthewbauer-post-pr log.matthewbauer.folded > log.matthewbauer.folded.svg

I can make flame graphs like: http://gsc.io/log.matthewbauer.folded.svg

---

Includes test cases around function call failures and tryEval. Uses
RAII so the finish is always called at the end of the function.
2019-08-14 16:09:35 -04:00
Eelco Dolstra 35ebae198f
Merge pull request #3031 from grahamc/low-speed-limit
conf: stalled-download-timeout: make tunable
2019-08-08 22:06:26 +02:00
Graham Christensen a02457db71
conf: stalled-download-timeout: make tunable
Make curl's low speed limit configurable via stalled-download-timeout.
Before, this limit was five minutes without receiving a single byte.
This is much too long as if the remote end may not have even
acknowledged the HTTP request.
2019-08-08 10:22:13 -04:00
Eelco Dolstra 05a10dd835
tests/post-hook.sh: Don't put result link in cwd 2019-08-08 15:47:56 +02:00
Eelco Dolstra 2053ac7747
Rename file for consistency 2019-08-08 12:18:46 +02:00
Eelco Dolstra f9021c4c6c
Merge pull request #3030 from dtzWill/fix/missing-include-ocloexec
pathlocks: add include to fcntl.h for O_CLOEXEC
2019-08-07 22:03:09 +02:00
Graham Christensen 1eeaf99cf8
fixup: docs for post-build-hook 2019-08-07 14:53:11 -04:00
Eelco Dolstra 56df30cd3f
Merge pull request #2995 from tweag/post-build-hook
Add a post build hook
2019-08-07 15:02:29 +02:00
Will Dietz c3fefd1a6e
pathlocks: add include to fcntl.h for O_CLOEXEC 2019-08-07 07:41:22 -05:00
Graham Christensen 363a2f6826
post-build-hook: docs fixup 2019-08-06 14:26:43 -04:00
Eelco Dolstra 399b6f3c46
nix-store --verify: Don't repair while holding the GC lock 2019-08-02 18:48:26 +02:00
Eelco Dolstra a2597d5f27
Simplify
With BSD locks we don't have to guard against reading our own
temproots.
2019-08-02 18:39:16 +02:00
Eelco Dolstra e349f2c0a3
Use BSD instead of POSIX file locks
POSIX file locks are essentially incompatible with multithreading. BSD
locks have much saner semantics. We need this now that there can be
multiple concurrent LocalStore::buildPaths() invocations.
2019-08-02 18:39:16 +02:00
Eelco Dolstra ec415d7166
Add a test for auto-GC
This currently fails because we're using POSIX file locks. So when the
garbage collector opens and closes its own temproots file, it causes
the lock to be released and then deleted by another GC instance.
2019-08-02 18:39:16 +02:00
regnat 7c5596734f
Add a post-build-hook
Passing `--post-build-hook /foo/bar` to a nix-* command will cause
`/foo/bar` to be executed after each build with the following
environment variables set:

    DRV_PATH=/nix/store/drv-that-has-been-built.drv
    OUT_PATHS=/nix/store/...build /nix/store/...build-bin /nix/store/...build-dev

This can be useful in particular to upload all the builded artifacts to
the cache (including the ones that don't appear in the runtime closure
of the final derivation or are built because of IFD).

This new feature prints the stderr/stdout output to the `nix-build`
and `nix build` client, and the output is printed in a Nix 2
compatible format:

    [nix]$ ./inst/bin/nix-build ./test.nix
    these derivations will be built:
      /nix/store/ishzj9ni17xq4hgrjvlyjkfvm00b0ch9-my-example-derivation.drv
    building '/nix/store/ishzj9ni17xq4hgrjvlyjkfvm00b0ch9-my-example-derivation.drv'...
    hello!
    bye!
    running post-build-hook '/home/grahamc/projects/github.com/NixOS/nix/post-hook.sh'...
    post-build-hook: + sleep 1
    post-build-hook: + echo 'Signing paths' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: Signing paths /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: + sleep 1
    post-build-hook: + echo 'Uploading paths' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: Uploading paths /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: + sleep 1
    post-build-hook: + printf 'very important stuff'
    /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation

    [nix-shell:~/projects/github.com/NixOS/nix]$ ./inst/bin/nix build -L -f ./test.nix
    my-example-derivation> hello!
    my-example-derivation> bye!
    my-example-derivation (post)> + sleep 1
    my-example-derivation (post)> + echo 'Signing paths' /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> Signing paths /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> + sleep 1
    my-example-derivation (post)> + echo 'Uploading paths' /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> Uploading paths /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> + sleep 1
    my-example-derivation (post)> + printf 'very important stuff'
    [1 built, 0.0 MiB DL]

Co-authored-by: Graham Christensen <graham@grahamc.com>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2019-08-02 10:48:15 -04:00
Eelco Dolstra 320126aeeb
Tweak min-free/max-free descriptions 2019-08-02 14:04:09 +02:00
Matthew Bauer 9a0855bbb6 Don’t rely on EPERM
startProcess does not appear to send the exit code to the helper
correctly. Not sure why this is, but it is probably safe to just
fallback on all sandbox errors.
2019-07-30 17:53:37 -04:00
Eelco Dolstra 41d010fff6
Merge pull request #3009 from codedownio/add-pname-and-version-to-json
Add pname and version to nix-env -q --json
2019-07-30 11:43:45 +02:00