Commit graph

983 commits

Author SHA1 Message Date
Graham Christensen ee9c988a1b
Track function start and ends for flame graphs
With this patch, and this file I called `log.py`:

    #!/usr/bin/env nix-shell
    #!nix-shell -i python3 -p python3 --pure

    import sys
    from pprint import pprint

    stack = []
    timestack = []

    for line in open(sys.argv[1]):
        components = line.strip().split(" ", 2)
        if components[0] != "function-trace":
            continue

        direction = components[1]
        components = components[2].rsplit(" ", 2)

        loc = components[0]
        _at = components[1]
        time = int(components[2])

        if direction == "entered":
            stack.append(loc)
            timestack.append(time)
        elif direction == "exited":
            dur = time - timestack.pop()
            vst = ";".join(stack)
            print(f"{vst} {dur}")
            stack.pop()

and:

    nix-instantiate --trace-function-calls -vvvv ../nixpkgs/pkgs/top-level/release.nix -A unstable > log.matthewbauer 2>&1
    ./log.py ./log.matthewbauer > log.matthewbauer.folded
    flamegraph.pl --title matthewbauer-post-pr log.matthewbauer.folded > log.matthewbauer.folded.svg

I can make flame graphs like: http://gsc.io/log.matthewbauer.folded.svg

---

Includes test cases around function call failures and tryEval. Uses
RAII so the finish is always called at the end of the function.
2019-08-14 16:09:35 -04:00
Graham Christensen a02457db71
conf: stalled-download-timeout: make tunable
Make curl's low speed limit configurable via stalled-download-timeout.
Before, this limit was five minutes without receiving a single byte.
This is much too long as if the remote end may not have even
acknowledged the HTTP request.
2019-08-08 10:22:13 -04:00
Graham Christensen 1eeaf99cf8
fixup: docs for post-build-hook 2019-08-07 14:53:11 -04:00
Eelco Dolstra 56df30cd3f
Merge pull request #2995 from tweag/post-build-hook
Add a post build hook
2019-08-07 15:02:29 +02:00
Graham Christensen 363a2f6826
post-build-hook: docs fixup 2019-08-06 14:26:43 -04:00
regnat 7c5596734f
Add a post-build-hook
Passing `--post-build-hook /foo/bar` to a nix-* command will cause
`/foo/bar` to be executed after each build with the following
environment variables set:

    DRV_PATH=/nix/store/drv-that-has-been-built.drv
    OUT_PATHS=/nix/store/...build /nix/store/...build-bin /nix/store/...build-dev

This can be useful in particular to upload all the builded artifacts to
the cache (including the ones that don't appear in the runtime closure
of the final derivation or are built because of IFD).

This new feature prints the stderr/stdout output to the `nix-build`
and `nix build` client, and the output is printed in a Nix 2
compatible format:

    [nix]$ ./inst/bin/nix-build ./test.nix
    these derivations will be built:
      /nix/store/ishzj9ni17xq4hgrjvlyjkfvm00b0ch9-my-example-derivation.drv
    building '/nix/store/ishzj9ni17xq4hgrjvlyjkfvm00b0ch9-my-example-derivation.drv'...
    hello!
    bye!
    running post-build-hook '/home/grahamc/projects/github.com/NixOS/nix/post-hook.sh'...
    post-build-hook: + sleep 1
    post-build-hook: + echo 'Signing paths' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: Signing paths /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: + sleep 1
    post-build-hook: + echo 'Uploading paths' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: Uploading paths /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: + sleep 1
    post-build-hook: + printf 'very important stuff'
    /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation

    [nix-shell:~/projects/github.com/NixOS/nix]$ ./inst/bin/nix build -L -f ./test.nix
    my-example-derivation> hello!
    my-example-derivation> bye!
    my-example-derivation (post)> + sleep 1
    my-example-derivation (post)> + echo 'Signing paths' /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> Signing paths /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> + sleep 1
    my-example-derivation (post)> + echo 'Uploading paths' /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> Uploading paths /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> + sleep 1
    my-example-derivation (post)> + printf 'very important stuff'
    [1 built, 0.0 MiB DL]

Co-authored-by: Graham Christensen <graham@grahamc.com>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2019-08-02 10:48:15 -04:00
Eelco Dolstra 320126aeeb
Tweak min-free/max-free descriptions 2019-08-02 14:04:09 +02:00
Graham Christensen cf6172f05e
docs: document balancing cores and max-jobs 2019-07-19 08:28:44 -04:00
Eelco Dolstra b5ae85f088
Merge pull request #2882 from grahamc/docs/1115-tarball-ttl
tarball-ttl: document
2019-07-06 00:15:27 +02:00
Graham Christensen 648bdf153d
tarball-ttl: document
Incorporates text from Niklas Hambüchen in #2978

Closes #1115
2019-07-05 15:55:28 -04:00
Eelco Dolstra 7d6ba1dc90
Merge branch 'autoconf-ubuntu-16.04-fixes' of https://github.com/nh2/nix 2019-07-03 08:02:45 +02:00
Niklas Hambüchen 57daa860e8 autoconf: Fix C++17 detection not working on Ubuntu 16.04.
And probably many other distributions.

Until now, ./configure would fail silently printing a warning

    ./configure: line 4621: AX_CXX_COMPILE_STDCXX_17: command not found

and then continuing, later failing with a C++ #error saying that some C++11
feature isn't supported (it didn't even get to the C++17 features).

This is because older distributions don't come with the
`AX_CXX_COMPILE_STDCXX_17` m4 macro.

This commit vendors that macro accordingly.

Now ./configure complains correctly:

    configure: error: *** A compiler with support for C++17 language features is required.

On Ubuntu 16.04, ./configure completes if a newer compiler is used, e.g. with
gcc-7 from https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test
using:

    ./bootstrap.sh
    ./configure CXX=g++-7 --disable-doc-gen --with-boost=$(nix-build --no-link '<nixpkgs>' -A boost.dev)
2019-07-03 04:32:25 +02:00
Niklas Hambüchen 1f97b16b1d autoconf: Work around editline not being found on Ubuntu 16.04.
And probably other Linux distributions with long-term support releases.

Also update manual stating what version is needed;
I checked that 1.14 is the oldest version with which current nix compiles,
and added autoconf feature checks for some functions added in that release
that nix uses.
2019-07-03 04:32:25 +02:00
Eelco Dolstra 7e1c85c5fb
Merge pull request #2779 from LnL7/build-exit-codes
build: add exit code for hash and check mismatches
2019-07-02 17:37:49 +02:00
Eelco Dolstra 7c0b0dbec8
Merge pull request #2724 from LnL7/manpage-add-fixed
nix-store: document --add-fixed
2019-07-02 15:47:40 +02:00
Eelco Dolstra 33db1d35ae
Merge pull request #2582 from LnL7/fetchgit-refs
fetchGit: allow fetching explicit refs
2019-07-02 15:44:31 +02:00
Daiderd Jordan a3c77c1536
nix-store: document --add-fixed 2019-07-02 09:12:02 -04:00
Graham Christensen c8205a3413
builtins.fetchGit: document absolute ref support 2019-07-02 09:05:56 -04:00
Daiderd Jordan a52c331edb
build: replace 100 offset for build exit codes 2019-07-02 00:12:38 +02:00
Daiderd Jordan 1ac399dd11
nix-store: document exit codes 2019-07-02 00:12:38 +02:00
Eelco Dolstra 88571219d9
nix-channel: Don't fetch binary-cache-url
This has been ignored since the Perl->C++ rewrite.
2019-06-25 13:27:16 +02:00
Bruno Bieth 74a65d313f Nix uses the CPP SDK, not Java 2019-06-20 14:47:45 +02:00
JorisE 4b0d613383
Minor typo 2019-06-04 14:12:03 +02:00
Maximilian Bosch b502b6682b
doc: clarify that optional attrs in a function argument will be ignored unless specified
In `args@{ a ? 1 }: /* ... */` the value `a` won't be a part of `args`
unless it's specified when calling the function, the default value will
be ignored in this case.

My personal point of view is that this behavior is a matter of taste, at
least I was pretty sure that unmatched arguments will be a part of
`args@` while debugging some Nix code last week.

I decided to add a warning to the docs which hopefully reduces the
confusion of further Nix developers who thought the same about `args@`.
2019-05-21 18:08:26 +02:00
Vladimír Čunát c0559a1d60
docs: describe $IN_NIX_SHELL values (#2796)
See commit 1bffd83e1a
2019-05-17 09:50:42 +02:00
Graham Christensen a5efe61786
Clarify where output from the diff hook goes. 2019-05-12 13:23:30 -04:00
Graham Christensen 6df61db060
diff hook: execute as the build user, and pass the temp dir 2019-05-12 13:17:26 -04:00
Graham Christensen c78686e411
build: run diff-hook under --check and document diff-hook 2019-05-12 13:17:24 -04:00
Eelco Dolstra 71eb76a0d4
Merge pull request #2765 from nh2/manual-nixpkgs-word
manual: "Nix Package collection" -> "Nixpkgs package collection".
2019-05-08 10:19:59 +02:00
Daniel Schaefer 3f192ac80c Add builtins.hashFile
For text files it is possible to do it like so:
`builtins.hashString "sha256" (builtins.readFile /tmp/a)`
but that doesn't work for binary files.

With builtins.hashFile any kind of file can be conveniently hashed.
2019-05-03 17:23:36 +02:00
Niklas Hambüchen 288f93cec0 manual: "Nix Package collection" -> "Nixpkgs package collection".
Makes difference between Nix and Nixpkgs clearer to avoid
some common confusion this sentence on IRC.

Also disambiguate an "it" reference.
2019-04-14 03:20:57 +02:00
Niklas Hambüchen caa76c369a docs: Mention --max-jobs 0 to build remotely only 2019-03-31 03:57:45 +02:00
Dmitry Kalinkin 75ec68f93a
manual: mention the "channel:" shorthand for NIX_PATH
Bumped to 15.09 because older channels, when downloaded from
nixos.org, require firefox to be accessed via `pkgs.firefox`
2019-03-25 20:55:28 -04:00
Jonas Chevalier 514b3c7f83
Add isPath primop
this is added for completeness' sake since all the other possible
`builtins.typeOf` results have a corresponding `builtins.is<Type>`
2019-03-24 11:36:49 +01:00
Linus Heckemann 6f093073b6 manual: include builtins.* for globally available builtins
This improves searchability.
2019-03-21 09:37:22 +01:00
Joachim Breitner 684c7fff80 Link to builders-use-substitutes in chapter on distributed builds
fixes #2730.
2019-03-16 19:03:40 +01:00
Andreas Rammhold 34fade478a
docs: fix typo in AllowDirectReads
It was just missing an `l` but should be fixed anyway.
2019-03-14 23:51:18 +01:00
Vladimír Čunát 4cfc131ec4
manual nitpick: document sha512 support in hashString 2019-03-01 14:30:30 +01:00
Spencer Baugh 5f1891b795 nix-store: make --dump-db take a list of paths to dump
Inside a derivation, exportReferencesGraph already provides a way to
dump the Nix database for a specific closure. On the command line,
--dump-db gave us the same information, but only for the entire Nix
database at once.

With this change, one can now pass a list of paths to --dump-db to get
the Nix database dumped for just those paths. (The user is responsible
for ensuring this is a closure, like for --export).

Among other things, this is useful for deploying a closure to a new
host without using --import/--export; one can use tar to transfer the
store paths, and --dump-db/--load-db to transfer the validity
information. This is useful if the new host doesn't actually have Nix
yet, and the closure that is being deployed itself contains Nix.
2019-01-23 01:24:34 +00:00
Domen Kožar 92d08c02c8
Merge pull request #2635 from veprbl/patch-7
manual: "sandbox" option default is "true" on Linux
2019-01-20 09:43:26 +07:00
Dmitry Kalinkin 15efd54373
manual: "sandbox" option default is "true" on Linux
Fixes: 812e39313c ('Enable sandboxing by default')
2019-01-19 15:04:43 -05:00
Samuel Evans-Powell 2ae5624b2f Update instructions for Haskell script using nix-shell interpreter
- The instructions for using nix-shell as an interpreter has a Haskell script
  example that doesn't work on more recent versions of Nix. Update the
  instructions with a working command
2019-01-18 10:50:54 +10:00
Eelco Dolstra bbaf865655 Add 2.2 release notes 2019-01-10 21:02:09 +01:00
John Ericson 2733287046 Don't look for a "pv" program, or mention it in the manual source
The manual reference was commented out, and the sole reference to this
program other than the configure script. Removed both.
2019-01-07 16:40:19 -05:00
Patrick Hilhorst 567941fb59
Clarify nix-instantiate --read-write-mode 2018-12-14 09:36:40 +01:00
Eelco Dolstra c37e6d77ea
Merge pull request #2569 from veprbl/pr/update_prerequisites
docs: add missing prerequisites: brotli, boost, libseccomp
2018-12-13 10:42:02 +01:00
Dmitry Kalinkin 0cc4728f71
docs: raise minimal boost version to 1.66 2018-12-13 01:50:39 -05:00
Dmitry Kalinkin 419949bf61
docs: add missing prerequisites: brotli, boost, libseccomp 2018-12-10 12:37:15 -05:00
Daiderd Jordan 77cc632186
s3: document scheme query parameter 2018-12-08 00:15:00 +01:00
Eelco Dolstra 6c6bbeb439 Merge branch 'feature/editline-pr' of https://github.com/dtzWill/nix 2018-11-13 20:42:13 +01:00
Felix C. Stegerman fb2c21f71c
manual: quote $servlets 2018-11-13 01:18:26 +01:00
Felix C. Stegerman 18215be59d
fix typo (s/gift/git/) 2018-11-12 17:50:39 +01:00
Eelco Dolstra b05d6dac7b
Manual: build locally -> build 2018-11-09 16:08:36 +01:00
Eelco Dolstra 4ea4d0b1a3
Urgh 2018-11-09 10:29:16 +01:00
Linus Heckemann 34d2948f21 Fix manual build
This was broken by some missing closing tags in 0bea4a50e0
2018-11-08 14:07:19 +01:00
Eelco Dolstra 750400e0fa
Merge pull request #2524 from domenkozar/deprecate-toPath
Deprecate builtins.toPath
2018-11-08 13:22:21 +01:00
Domen Kožar 0bea4a50e0
Deprecate builtins.toPath 2018-11-08 12:03:50 +00:00
Eelco Dolstra 812e39313c
Enable sandboxing by default
Closes #179.
2018-11-07 17:08:28 +01:00
Eelco Dolstra 5a3f140856
Document allowSubstitutes 2018-11-07 11:42:22 +01:00
Eelco Dolstra 1109193ea3
Fix preferLocalBuild description
preferLocalBuild does not in fact prevent substitution.
2018-11-07 11:37:43 +01:00
Will Dietz 3d974d31fa editline: wip 2018-10-29 08:44:58 -05:00
Antoine Eiche 73c2ae43f0 Add --graphml option to the nix-store --query command
This prints the references graph of the store paths in the graphML
format [1]. The graphML format is supported by several graph tools
such as the Python Networkx library or the Apache Thinkerpop project.

[1] http://graphml.graphdrawing.org
2018-10-20 09:48:34 +02:00
Benjamin Hipple 65453e2d77 Docs: update fixed output hashes
`fetchurl` will now throw if given an `md5`, and the hashes have generally
been upgraded to avoid it and use `sha256` as a default. This updates the
documentation examples in the manual accordingly.
2018-10-04 02:37:15 +00:00
Profpatsch 05cfc71cab manual: add XML IDs to all builtin list entries 2018-10-02 15:38:16 +02:00
Eelco Dolstra aa5e47b2f4
Merge pull request #2456 from grahamc/s3-substituters
Update docs to describe how s3:// URLS does in fact support endpoint, region, and profile for upload
2018-10-01 22:47:00 +02:00
Eelco Dolstra 5b7cfa487e
Add missing file 2018-10-01 11:58:54 +02:00
Eelco Dolstra 1e7b8deea7
Check requiredSystemFeatures for local builds
For example, this prevents a "kvm" build on machines that don't have
KVM.

Fixes #2012.
2018-09-28 16:10:27 +02:00
Graham Christensen 51cbeec49a
Update docs to describe how s3:// URLS does in fact support endpoint, region, and profile for upload 2018-09-27 17:01:39 -04:00
Eelco Dolstra a48c2e3411
Merge pull request #2434 from grahamc/graham-improve-upgrade
Upgrade docs: improve the upgrade command
2018-09-19 21:53:50 +02:00
Anders Riutta 1915862767
Upgrade docs: improve the upgrade command
and make it more copy-pastable.
2018-09-19 14:17:06 -04:00
Graham Christensen be9e356e61
nix-shell: document double quotes being necessary for a simple Nix expression in the shebang 2018-09-19 13:01:27 -04:00
Graham Christensen 10a7f19937
nix-shell: explain the merging property close to the example 2018-09-19 12:48:08 -04:00
Graham Christensen 0cd863197b
docs: document deprecated aliases 2018-09-18 12:23:32 -04:00
Graham Christensen 0aca1ffb6e Prefer 'remote builder' over 'slave' 2018-09-10 18:57:46 +02:00
Vladimír Čunát 5f3b72cfc2
docs: change expired bzip2.org to archive.org
Fixes #2396.
2018-09-03 17:57:13 +02:00
Eelco Dolstra 4dd09210d7
Release notes tweaks 2018-09-02 21:47:10 +02:00
Eelco Dolstra 1f49926601
Merge pull request #2388 from grahamc/graham/document-multi-user
Document multi-user installation, and add release notes about it not being the default on 2.1
2018-09-01 22:43:48 +02:00
Graham Christensen c42eaaf684
Create upgrade notes 2018-09-01 15:35:46 -04:00
Eelco Dolstra c5ab07ec2b
Merge pull request #2168 from grahamc/explain-why-netrc-is-busted
manual: document why ~/.netrc doesn't work
2018-09-01 21:25:04 +02:00
Eelco Dolstra 51003f892d
Merge pull request #2309 from symphorien/disallowed-man
add manual entries for disallowedRequisites and disallowedReferences
2018-09-01 21:01:41 +02:00
Eelco Dolstra 291f67aecf
Merge pull request #2237 from nh2/patch-1
manual: distributed-builds: Mention - as default
2018-09-01 20:41:49 +02:00
Graham Christensen 4be7652dd3
release notes: note that the multi-user installer is available but not selected for Linux with systemd, and the bug about selinux 2018-09-01 10:45:57 -04:00
Graham Christensen c3e508d924
Document the multi-user installer some
Use sh <(...) syntax for installation to preserve stdin and prompting

also update installation docs to account for changes in multi-user selection
2018-09-01 10:45:50 -04:00
Graham Christensen f66fa7cd20
We support aarch64 now 2018-09-01 10:44:33 -04:00
Eelco Dolstra 6ed4a6bd0e
Merge pull request #2384 from graham-at-target/fetch-git-examples
docs: Add some examples to fetchGit
2018-09-01 14:08:32 +02:00
Graham Christensen 0b7568fb73
Drop ssh://... as a required formatting for builtins.fetchGit 2018-08-31 19:49:56 -04:00
Eelco Dolstra 4095cd6438
Add contributors 2018-09-01 00:01:05 +02:00
Graham Christensen 149d10c308 docs: Add IDs to important sections 2018-08-31 10:06:33 -04:00
Graham Christensen 2df21b78b9 docs: Add some examples to fetchGit 2018-08-31 10:00:32 -04:00
Eelco Dolstra 308689f94b
Merge pull request #2365 from erikarvstedt/fix-docs
Docs: Fix install prefix
2018-08-30 16:24:52 +02:00
Eelco Dolstra 64d7d1a884
Update release notes 2018-08-30 15:28:18 +02:00
Domen Kožar d16ff76c69
nix.conf: clarify -j0 doesn't affect preferLocalBuild 2018-08-29 14:33:14 +01:00
Domen Kožar 54df4bb0b5
nix.conf: mention -j0 is useful 2018-08-29 11:51:51 +01:00
Erik Arvstedt 8ad2defdf0 Docs: Fix install prefix
shell.nix defines the install prefix as $(pwd)/inst
2018-08-21 21:46:43 +02:00
Graham Christensen 06080e4abc
2.1 release notes: Add note about s3-compatible stores 2018-08-03 11:09:31 -04:00
Eelco Dolstra 87356cc8a4
Add Nix 2.1 release notes 2018-08-03 17:05:17 +02:00
Eelco Dolstra 122e1a61f8
Merge pull request #2323 from samueldr/feature/selective-impurity
Allows selectively adding environment variables to pure shells.
2018-08-03 16:58:18 +02:00
Samuel Dionne-Riel 438e02529d Allows selectively adding environment variables to pure shells.
Includes documentation and test.
2018-08-02 08:21:30 -04:00
Graham Christensen 49a53c1d3f s3 binary cache: support specifying an endpoint
Works for uploading and not downloading.
2018-08-02 08:19:50 -04:00
Graham Christensen b27431b7cb Document s3 substitutions 2018-08-01 11:07:41 -04:00