Commit graph

933 commits

Author SHA1 Message Date
Eelco Dolstra 477f82e5a7
Merge pull request #2782 from grahamc/flames
Track function start and end
2019-08-15 14:20:42 +02:00
Graham Christensen ee9c988a1b
Track function start and ends for flame graphs
With this patch, and this file I called `log.py`:

    #!/usr/bin/env nix-shell
    #!nix-shell -i python3 -p python3 --pure

    import sys
    from pprint import pprint

    stack = []
    timestack = []

    for line in open(sys.argv[1]):
        components = line.strip().split(" ", 2)
        if components[0] != "function-trace":
            continue

        direction = components[1]
        components = components[2].rsplit(" ", 2)

        loc = components[0]
        _at = components[1]
        time = int(components[2])

        if direction == "entered":
            stack.append(loc)
            timestack.append(time)
        elif direction == "exited":
            dur = time - timestack.pop()
            vst = ";".join(stack)
            print(f"{vst} {dur}")
            stack.pop()

and:

    nix-instantiate --trace-function-calls -vvvv ../nixpkgs/pkgs/top-level/release.nix -A unstable > log.matthewbauer 2>&1
    ./log.py ./log.matthewbauer > log.matthewbauer.folded
    flamegraph.pl --title matthewbauer-post-pr log.matthewbauer.folded > log.matthewbauer.folded.svg

I can make flame graphs like: http://gsc.io/log.matthewbauer.folded.svg

---

Includes test cases around function call failures and tryEval. Uses
RAII so the finish is always called at the end of the function.
2019-08-14 16:09:35 -04:00
Graham Christensen a02457db71
conf: stalled-download-timeout: make tunable
Make curl's low speed limit configurable via stalled-download-timeout.
Before, this limit was five minutes without receiving a single byte.
This is much too long as if the remote end may not have even
acknowledged the HTTP request.
2019-08-08 10:22:13 -04:00
Graham Christensen 1eeaf99cf8
fixup: docs for post-build-hook 2019-08-07 14:53:11 -04:00
Eelco Dolstra 56df30cd3f
Merge pull request #2995 from tweag/post-build-hook
Add a post build hook
2019-08-07 15:02:29 +02:00
Graham Christensen 363a2f6826
post-build-hook: docs fixup 2019-08-06 14:26:43 -04:00
regnat 7c5596734f
Add a post-build-hook
Passing `--post-build-hook /foo/bar` to a nix-* command will cause
`/foo/bar` to be executed after each build with the following
environment variables set:

    DRV_PATH=/nix/store/drv-that-has-been-built.drv
    OUT_PATHS=/nix/store/...build /nix/store/...build-bin /nix/store/...build-dev

This can be useful in particular to upload all the builded artifacts to
the cache (including the ones that don't appear in the runtime closure
of the final derivation or are built because of IFD).

This new feature prints the stderr/stdout output to the `nix-build`
and `nix build` client, and the output is printed in a Nix 2
compatible format:

    [nix]$ ./inst/bin/nix-build ./test.nix
    these derivations will be built:
      /nix/store/ishzj9ni17xq4hgrjvlyjkfvm00b0ch9-my-example-derivation.drv
    building '/nix/store/ishzj9ni17xq4hgrjvlyjkfvm00b0ch9-my-example-derivation.drv'...
    hello!
    bye!
    running post-build-hook '/home/grahamc/projects/github.com/NixOS/nix/post-hook.sh'...
    post-build-hook: + sleep 1
    post-build-hook: + echo 'Signing paths' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: Signing paths /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: + sleep 1
    post-build-hook: + echo 'Uploading paths' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: Uploading paths /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: + sleep 1
    post-build-hook: + printf 'very important stuff'
    /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation

    [nix-shell:~/projects/github.com/NixOS/nix]$ ./inst/bin/nix build -L -f ./test.nix
    my-example-derivation> hello!
    my-example-derivation> bye!
    my-example-derivation (post)> + sleep 1
    my-example-derivation (post)> + echo 'Signing paths' /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> Signing paths /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> + sleep 1
    my-example-derivation (post)> + echo 'Uploading paths' /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> Uploading paths /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> + sleep 1
    my-example-derivation (post)> + printf 'very important stuff'
    [1 built, 0.0 MiB DL]

Co-authored-by: Graham Christensen <graham@grahamc.com>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2019-08-02 10:48:15 -04:00
Eelco Dolstra 320126aeeb
Tweak min-free/max-free descriptions 2019-08-02 14:04:09 +02:00
Graham Christensen cf6172f05e
docs: document balancing cores and max-jobs 2019-07-19 08:28:44 -04:00
Eelco Dolstra b5ae85f088
Merge pull request #2882 from grahamc/docs/1115-tarball-ttl
tarball-ttl: document
2019-07-06 00:15:27 +02:00
Graham Christensen 648bdf153d
tarball-ttl: document
Incorporates text from Niklas Hambüchen in #2978

Closes #1115
2019-07-05 15:55:28 -04:00
Eelco Dolstra 7d6ba1dc90
Merge branch 'autoconf-ubuntu-16.04-fixes' of https://github.com/nh2/nix 2019-07-03 08:02:45 +02:00
Niklas Hambüchen 57daa860e8 autoconf: Fix C++17 detection not working on Ubuntu 16.04.
And probably many other distributions.

Until now, ./configure would fail silently printing a warning

    ./configure: line 4621: AX_CXX_COMPILE_STDCXX_17: command not found

and then continuing, later failing with a C++ #error saying that some C++11
feature isn't supported (it didn't even get to the C++17 features).

This is because older distributions don't come with the
`AX_CXX_COMPILE_STDCXX_17` m4 macro.

This commit vendors that macro accordingly.

Now ./configure complains correctly:

    configure: error: *** A compiler with support for C++17 language features is required.

On Ubuntu 16.04, ./configure completes if a newer compiler is used, e.g. with
gcc-7 from https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test
using:

    ./bootstrap.sh
    ./configure CXX=g++-7 --disable-doc-gen --with-boost=$(nix-build --no-link '<nixpkgs>' -A boost.dev)
2019-07-03 04:32:25 +02:00
Niklas Hambüchen 1f97b16b1d autoconf: Work around editline not being found on Ubuntu 16.04.
And probably other Linux distributions with long-term support releases.

Also update manual stating what version is needed;
I checked that 1.14 is the oldest version with which current nix compiles,
and added autoconf feature checks for some functions added in that release
that nix uses.
2019-07-03 04:32:25 +02:00
Eelco Dolstra 7e1c85c5fb
Merge pull request #2779 from LnL7/build-exit-codes
build: add exit code for hash and check mismatches
2019-07-02 17:37:49 +02:00
Eelco Dolstra 7c0b0dbec8
Merge pull request #2724 from LnL7/manpage-add-fixed
nix-store: document --add-fixed
2019-07-02 15:47:40 +02:00
Eelco Dolstra 33db1d35ae
Merge pull request #2582 from LnL7/fetchgit-refs
fetchGit: allow fetching explicit refs
2019-07-02 15:44:31 +02:00
Daiderd Jordan a3c77c1536
nix-store: document --add-fixed 2019-07-02 09:12:02 -04:00
Graham Christensen c8205a3413
builtins.fetchGit: document absolute ref support 2019-07-02 09:05:56 -04:00
Daiderd Jordan a52c331edb
build: replace 100 offset for build exit codes 2019-07-02 00:12:38 +02:00
Daiderd Jordan 1ac399dd11
nix-store: document exit codes 2019-07-02 00:12:38 +02:00
Eelco Dolstra 88571219d9
nix-channel: Don't fetch binary-cache-url
This has been ignored since the Perl->C++ rewrite.
2019-06-25 13:27:16 +02:00
Bruno Bieth 74a65d313f Nix uses the CPP SDK, not Java 2019-06-20 14:47:45 +02:00
JorisE 4b0d613383
Minor typo 2019-06-04 14:12:03 +02:00
Maximilian Bosch b502b6682b
doc: clarify that optional attrs in a function argument will be ignored unless specified
In `args@{ a ? 1 }: /* ... */` the value `a` won't be a part of `args`
unless it's specified when calling the function, the default value will
be ignored in this case.

My personal point of view is that this behavior is a matter of taste, at
least I was pretty sure that unmatched arguments will be a part of
`args@` while debugging some Nix code last week.

I decided to add a warning to the docs which hopefully reduces the
confusion of further Nix developers who thought the same about `args@`.
2019-05-21 18:08:26 +02:00
Vladimír Čunát c0559a1d60
docs: describe $IN_NIX_SHELL values (#2796)
See commit 1bffd83e1a
2019-05-17 09:50:42 +02:00
Graham Christensen a5efe61786
Clarify where output from the diff hook goes. 2019-05-12 13:23:30 -04:00
Graham Christensen 6df61db060
diff hook: execute as the build user, and pass the temp dir 2019-05-12 13:17:26 -04:00
Graham Christensen c78686e411
build: run diff-hook under --check and document diff-hook 2019-05-12 13:17:24 -04:00
Eelco Dolstra 71eb76a0d4
Merge pull request #2765 from nh2/manual-nixpkgs-word
manual: "Nix Package collection" -> "Nixpkgs package collection".
2019-05-08 10:19:59 +02:00
Daniel Schaefer 3f192ac80c Add builtins.hashFile
For text files it is possible to do it like so:
`builtins.hashString "sha256" (builtins.readFile /tmp/a)`
but that doesn't work for binary files.

With builtins.hashFile any kind of file can be conveniently hashed.
2019-05-03 17:23:36 +02:00
Niklas Hambüchen 288f93cec0 manual: "Nix Package collection" -> "Nixpkgs package collection".
Makes difference between Nix and Nixpkgs clearer to avoid
some common confusion this sentence on IRC.

Also disambiguate an "it" reference.
2019-04-14 03:20:57 +02:00
Niklas Hambüchen caa76c369a docs: Mention --max-jobs 0 to build remotely only 2019-03-31 03:57:45 +02:00
Dmitry Kalinkin 75ec68f93a
manual: mention the "channel:" shorthand for NIX_PATH
Bumped to 15.09 because older channels, when downloaded from
nixos.org, require firefox to be accessed via `pkgs.firefox`
2019-03-25 20:55:28 -04:00
Jonas Chevalier 514b3c7f83
Add isPath primop
this is added for completeness' sake since all the other possible
`builtins.typeOf` results have a corresponding `builtins.is<Type>`
2019-03-24 11:36:49 +01:00
Linus Heckemann 6f093073b6 manual: include builtins.* for globally available builtins
This improves searchability.
2019-03-21 09:37:22 +01:00
Joachim Breitner 684c7fff80 Link to builders-use-substitutes in chapter on distributed builds
fixes #2730.
2019-03-16 19:03:40 +01:00
Andreas Rammhold 34fade478a
docs: fix typo in AllowDirectReads
It was just missing an `l` but should be fixed anyway.
2019-03-14 23:51:18 +01:00
Vladimír Čunát 4cfc131ec4
manual nitpick: document sha512 support in hashString 2019-03-01 14:30:30 +01:00
Spencer Baugh 5f1891b795 nix-store: make --dump-db take a list of paths to dump
Inside a derivation, exportReferencesGraph already provides a way to
dump the Nix database for a specific closure. On the command line,
--dump-db gave us the same information, but only for the entire Nix
database at once.

With this change, one can now pass a list of paths to --dump-db to get
the Nix database dumped for just those paths. (The user is responsible
for ensuring this is a closure, like for --export).

Among other things, this is useful for deploying a closure to a new
host without using --import/--export; one can use tar to transfer the
store paths, and --dump-db/--load-db to transfer the validity
information. This is useful if the new host doesn't actually have Nix
yet, and the closure that is being deployed itself contains Nix.
2019-01-23 01:24:34 +00:00
Domen Kožar 92d08c02c8
Merge pull request #2635 from veprbl/patch-7
manual: "sandbox" option default is "true" on Linux
2019-01-20 09:43:26 +07:00
Dmitry Kalinkin 15efd54373
manual: "sandbox" option default is "true" on Linux
Fixes: 812e39313c ('Enable sandboxing by default')
2019-01-19 15:04:43 -05:00
Samuel Evans-Powell 2ae5624b2f Update instructions for Haskell script using nix-shell interpreter
- The instructions for using nix-shell as an interpreter has a Haskell script
  example that doesn't work on more recent versions of Nix. Update the
  instructions with a working command
2019-01-18 10:50:54 +10:00
Eelco Dolstra bbaf865655 Add 2.2 release notes 2019-01-10 21:02:09 +01:00
John Ericson 2733287046 Don't look for a "pv" program, or mention it in the manual source
The manual reference was commented out, and the sole reference to this
program other than the configure script. Removed both.
2019-01-07 16:40:19 -05:00
Patrick Hilhorst 567941fb59
Clarify nix-instantiate --read-write-mode 2018-12-14 09:36:40 +01:00
Eelco Dolstra c37e6d77ea
Merge pull request #2569 from veprbl/pr/update_prerequisites
docs: add missing prerequisites: brotli, boost, libseccomp
2018-12-13 10:42:02 +01:00
Dmitry Kalinkin 0cc4728f71
docs: raise minimal boost version to 1.66 2018-12-13 01:50:39 -05:00
Dmitry Kalinkin 419949bf61
docs: add missing prerequisites: brotli, boost, libseccomp 2018-12-10 12:37:15 -05:00
Daiderd Jordan 77cc632186
s3: document scheme query parameter 2018-12-08 00:15:00 +01:00