Commit graph

1460 commits

Author SHA1 Message Date
Eelco Dolstra 0629601da1
Move EvalState from the stack to the heap
EvalState contains a few counters (e.g. nrValues) that increase
quickly enough that they end up being interpreted as pointers by the
garbage collector. Moving it to the heap makes them invisible to the
garbage collector.

This reduces the max RSS doing 100 evaluations of
nixos.tests.firefox.x86_64-linux.drvPath from 455 MiB to 292 MiB.

Note: ideally, allocations would be much further up in the 64-bit
address space to reduce the odds of an integer being misinterpreted as
a pointer. Maybe we can use some linker magic to move the .bss segment
to a higher address.
2018-06-12 17:49:55 +02:00
Eelco Dolstra 1672bcd230
Move evaluator-specific settings out of libstore 2018-05-30 13:29:50 +02:00
Eelco Dolstra 737ed88f35
Modularize config settings
Allow global config settings to be defined in multiple Config
classes. For example, this means that libutil can have settings and
evaluator settings can be moved out of libstore. The Config classes
are registered in a new GlobalConfig class to which config files
etc. are applied.

Relevant to https://github.com/NixOS/nix/issues/2009 in that it
removes the need for ad hoc handling of useCaseHack, which was the
underlying cause of that issue.
2018-05-30 13:28:01 +02:00
Daniel Poelzleithner b712d4674b Allow multiple search experssions in nix search
The common use case is to search for packages containing multiple words
like a "git" "frontend". Having only one expressions makes this simple regular
use case very complicated. Instead, search accepts multiple regular epressions
which all need to match.

nix search git 'gui|frontend'

returns a list of all git uis for example
2018-04-18 21:08:35 +02:00
Eelco Dolstra a4aac7f88c
Handle arguments in $EDITOR
Fixes #2079.
2018-04-17 12:16:04 +02:00
Félix Baylac-Jacqué a91fb422fe
nix repl: Fix multiline SIGINT handling.
Fixes #2076
2018-04-16 16:09:30 +02:00
Eelco Dolstra 1839a5542a
Fix #2057 2018-04-11 11:42:17 +02:00
Jonas Chevalier 865ca2402f nix copy: add an example with S3
I couldn't find a good example how to use it with non-us-east-1 buckets.
2018-03-30 02:31:14 +01:00
Eelco Dolstra 446bb88f13
Remove reference to non-existent manpages 2018-03-29 12:29:10 +02:00
Eelco Dolstra 4868721506
Filter ANSI colors when not writing to a terminal
Fixes https://github.com/NixOS/nixpkgs/issues/37114.
2018-03-15 16:08:07 +01:00
Shea Levy 56f2ed0081
Merge pull request #1906 from dtzWill/fix/nix-search
nix search: tests and fix #1893 and part of #1892
2018-03-13 18:43:47 -06:00
Shea Levy 3748a0ca1e
Merge branch 'improve-search-algorithm' of git://github.com/Ma27/nix 2018-03-02 11:41:08 -05:00
Shea Levy aa8bbbf69d
Merge branch 'write-failure-fixes' of git://github.com/lheckemann/nix 2018-03-02 10:59:59 -05:00
Eelco Dolstra 24ec750003
nix run: Fix segfault on macOS
Note that clearenv() is not available on macOS.

Fixes #1907.
2018-02-26 18:29:40 +01:00
Will Dietz 009cf9cd23 nix search: explicitly handle empty search string, fixes #1893
This is important since this is given as an example.
Other patterns containing "empty search string" will still
be handled differently on different platforms ("asdf|")
but that's less of an issue.
2018-02-25 16:40:56 -06:00
Will Dietz 3cac8734ac nix search: fix bug where we wrote to cache when shouldn't, breaking
This is exposed by the tests added previously,
and resolves the error reported in #1892:
"expected JSON value".
2018-02-25 16:40:05 -06:00
Eelco Dolstra de4c03d201
Merge branch 'fix/dry-run-partially' of https://github.com/dtzWill/nix 2018-02-22 12:18:20 +01:00
Eelco Dolstra 0d54671b7b
Manual: Update chapter on remote builds
Alos add a command "nix ping-store" to make it easier to see if Nix
can connect to a remote builder (e.g. 'nix ping-store --store
ssh://mac').
2018-02-21 16:24:26 +01:00
Jörg Thalheim fa7fd76c5e nix-copy: fix examples
maybe a left-over from nix-store -r ?
2018-02-21 01:13:46 +00:00
Eelco Dolstra 70eb64147e
Update release notes
Also add some examples to nix --help.
2018-02-19 20:38:53 +01:00
Linus Heckemann 78ac3eb4eb Fix #1762
nix-store --export, nix-store --dump, and nix dump-path would previously
fail silently if writing the data out failed, because
 a) FdSink::write ignored exceptions, and
 b) the commands relied on FdSink's destructor, which ignores
    exceptions, to flush the data out.

This could cause rather opaque issues with installing nixos, because
nix-store --export would happily proceed even if it couldn't write its
data out (e.g. if nix-store --import on the other side of the pipe
failed).

This commit adds tests that expose these issues in the nix-store
commands, and fixes them for all three.
2018-02-13 14:26:19 +00:00
Eelco Dolstra 1c10a74c73
Merge branch 'plugins' of https://github.com/shlevy/nix 2018-02-13 12:15:27 +01:00
Eelco Dolstra 35fd31770c
toBuildables -> build 2018-02-12 17:06:06 +01:00
Shea Levy 88cd2d41ac
Add plugins to make Nix more extensible.
All plugins in plugin-files will be dlopened, allowing them to
statically construct instances of the various Register* types Nix
supports.
2018-02-08 12:44:37 -05:00
Will Dietz 98031b6050 nix build: Don't create output links with --dry-run.
Fixes #1849.
2018-02-07 15:20:54 -06:00
Eelco Dolstra 84989d3af2
Improve filtering of ANSI escape sequences in build logs
All ANSI sequences except color setting are now filtered out. In
particular, terminal resets (such as from NixOS VM tests) are filtered
out.

Also, fix the completely broken tab character handling.
2018-02-07 15:23:57 +01:00
Eelco Dolstra 6f6bfc8205 Update the progress bar at most 20 times per second
Fixes #1834.
2018-02-06 20:51:37 +01:00
Eelco Dolstra 55012ec0b9
Tweak progress bar message
Say "copying" instead of "fetching" when copying from another local
store. Nice for nixos-install.
2018-02-05 18:32:23 +01:00
Eelco Dolstra 6fa690291a
Add 'nix upgrade-nix' command
This command upgrades Nix to the latest stable version by installing a
store path obtained from

  https://github.com/NixOS/nixpkgs/raw/master/nixos/modules/installer/tools/nix-fallback-paths.nix

which is the same store path that the installer at
https://nixos.org/nix/install.sh uses.

The upgrade fails if Nix is not installed in a profile (e.g. on NixOS,
or when installed outside of the Nix store).
2018-01-31 16:24:43 +01:00
Eelco Dolstra 27b510af5c
nix eval: Stop progress bar before printing the result 2018-01-17 12:04:44 +01:00
Eelco Dolstra 16e0287556
nix eval: Take only one argument
Thus --json no longer produces a list.
2018-01-17 12:03:06 +01:00
Eelco Dolstra 23fa7e3606
parseExprFromFile -> evalFile
parseExprFromFile() should be avoided since it doesn't cache anything.
2018-01-16 17:11:58 +01:00
Will Dietz 6454ca393a nix log: use pager 2018-01-12 15:05:26 -06:00
Will Dietz bd17ccf1d8 nix repl: use linenoiseKeyType to differentiate ^C and ^D
Fixes #1757.
2017-12-26 19:25:50 -06:00
Ben Gamari 626a94d70e edit: Catch stoi exceptions from line number parsing 2017-12-14 19:11:56 -05:00
Maximilian Bosch 0413aeb35d
search.cc: sort attribute names with std::map 2017-12-12 15:13:36 +01:00
Will Dietz c800f47395 Fix forward-decl of Bindings as "class", match definition.
(appease clang -Wmismatched-tags warning)
2017-12-11 11:51:55 -06:00
Shea Levy 5ba5ca7888
Merge branch 'feature/linenoise-ng' of git://github.com/dtzWill/nix 2017-12-11 07:52:20 -05:00
Eelco Dolstra f1efb97075
Fix #1635. 2017-12-04 13:19:16 +01:00
Will Dietz 964349e44d no "linenoiseFree" in linenoise-ng 2017-11-27 18:30:05 -06:00
Will Dietz 21f515724c replace vendored linenoise with linenoise-ng 2017-11-27 18:27:28 -06:00
Eelco Dolstra f5f29dc2b7
nix eval --raw: Use coerceToString()
Thus,

  $ nix eval --raw '("foo")'
  foo

  $ nix eval --raw nixpkgs.hello
  /nix/store/1y6ckg6khrdsvll54s5spcmf3w6ka9k4-hello-2.10

  $ nix eval --raw '(/etc/resolv.conf)'
  /nix/store/vml92ama92i8mz013nny461mlvg8mvap-resolv.conf
2017-11-20 18:27:29 +01:00
Eelco Dolstra 1ff01187e2
nix run: Fix "flag '--command' requires 2 argument(s)" 2017-11-20 18:07:58 +01:00
Eelco Dolstra 4eb9e20028
nix run: Fix accidental removal of /nix/store existence check
Parenthetical to #1686, we don't need to create a new root if we can
just bind-mount on top of the existing /nix/store.
2017-11-20 17:59:32 +01:00
Eelco Dolstra a3aa850f0f
nix run: Ignore non-directories while setting up the chroot
Fixes #1686.
2017-11-20 17:58:47 +01:00
Eelco Dolstra 4db0a9555e
nix ls-{nar,store} --json: Respect -R 2017-11-14 14:31:38 +01:00
Eelco Dolstra c8155e9f5f
Don't indent JSON output 2017-11-14 14:28:03 +01:00
Eelco Dolstra bac8055652
nix ls-{store,nar}: Add --json flag 2017-11-14 14:23:53 +01:00
Eelco Dolstra 5ef5d70b5e
Make config options available to legacy commands 2017-11-14 14:04:09 +01:00
Eelco Dolstra 3460e4cf00
More progress indicator improvements
In particular, don't show superfluous "fetching path" and "building
path(s)" messages, and show the current round (with --repeat).
2017-10-24 15:32:38 +02:00
Eelco Dolstra be220702a7
Progress indicator: Show on what machine we're building
E.g.

  $ nix build nixpkgs.hello --builders 'root@wendy'
  [1/0/1 built] building hello-2.10 on ssh://root@wendy: checking for minix/config.h... no
2017-10-24 14:24:57 +02:00
Eelco Dolstra 0d59f1ca49
nix: Respect -I, --arg, --argstr
Also, random cleanup to argument handling.
2017-10-24 12:58:34 +02:00
Eelco Dolstra f32cdc4fab
Turn $NIX_REMOTE into a configuration option 2017-10-23 20:50:28 +02:00
Eelco Dolstra f3e0d46821
nix run: Restore signals
Otherwise Ctrl-C doesn't work.
2017-09-28 17:58:59 +02:00
Eelco Dolstra 5421ad243e
nix copy: make recursive by default 2017-09-27 18:28:54 +02:00
Eelco Dolstra b8875213dc
Tweak message 2017-09-27 13:21:25 +02:00
Eelco Dolstra bfef3ec855 nix build: Fix realising a store path 2017-09-27 13:14:45 +02:00
Eelco Dolstra d1c6e0fe8c
nix why-depends: Write to stdout 2017-09-25 16:59:16 +02:00
Eelco Dolstra e8d6ee7c1b
Add "nix show-derivation"
This debug command prints a store derivation in JSON format. For
example:

  $ nix show-derivation nixpkgs.hello
  {
    "/nix/store/ayjwpwwiyy04nh9z71rsdgd3q7bra7ch-hello-2.10.drv": {
      "outputs": {
        "out": {
          "path": "/nix/store/w5w4v29ql0qwqhczkdxs94ix2lh7ibgs-hello-2.10"
        }
      },
      "inputSrcs": [
        "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"
      ],
      "inputDrvs": {
        "/nix/store/13839aqdf6x4k3b785rw5f2l7857l6y3-bash-4.4-p12.drv": [
          "out"
        ],
        "/nix/store/vgdx7fdc7d4iirmnwj2py1nrvr5qwzj7-hello-2.10.tar.gz.drv": [
          "out"
        ],
        "/nix/store/x3kkd0vsqfflbvwf1055l9mr39bg0ms0-stdenv.drv": [
          "out"
        ]
      },
      "platform": "x86_64-linux",
      "builder": "/nix/store/qp5fw57d38bd1n07ss4zxh88zg67c3vg-bash-4.4-p12/bin/bash",
      "args": [
        "-e",
        "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"
      ],
      "env": {
        "buildInputs": "",
        "builder": "/nix/store/qp5fw57d38bd1n07ss4zxh88zg67c3vg-bash-4.4-p12/bin/bash",
        "configureFlags": "",
        "doCheck": "1",
        "name": "hello-2.10",
        "nativeBuildInputs": "",
        "out": "/nix/store/w5w4v29ql0qwqhczkdxs94ix2lh7ibgs-hello-2.10",
        "propagatedBuildInputs": "",
        "propagatedNativeBuildInputs": "",
        "src": "/nix/store/3x7dwzq014bblazs7kq20p9hyzz0qh8g-hello-2.10.tar.gz",
        "stdenv": "/nix/store/6zngq1rdh0ans9qyckqimqibgnlvlfrm-stdenv",
        "system": "x86_64-linux"
      }
    }
  }

This removes the need for pp-aterm.
2017-09-25 13:43:35 +02:00
Eelco Dolstra 970366266b
Add "nix add-to-store" command 2017-09-14 13:29:33 +02:00
Eelco Dolstra 1c58ad2ffa
nix why-depends: Fix showing self-references 2017-09-14 11:17:32 +02:00
Eelco Dolstra fc0ded3408
nix why-depends: Add option to show all edges causing a dependency
For example, without --all:

  $ nix why-depends nixpkgs.nixUnstable nixpkgs.libssh2
  /nix/store/s9n5gvj2l49b4n19nz6xl832654nf7n7-nix-1.12pre5511_c94f3d55
  └───lib/libnixstore.so: …/lib:/nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0/lib…
      => /nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0
      └───lib/libcurl.la: …ib -L/nix/store/4mbayl1y5hpjbjzkx8ndyhkv98kqw1wi-libssh2-1.8.0/l…
          => /nix/store/4mbayl1y5hpjbjzkx8ndyhkv98kqw1wi-libssh2-1.8.0

but with --all:

  $ nix why-depends -a nixpkgs.nixUnstable nixpkgs.libssh2
  /nix/store/s9n5gvj2l49b4n19nz6xl832654nf7n7-nix-1.12pre5511_c94f3d55
  ├───lib/libnixstore.so: …/lib:/nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0/lib…
  │   => /nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0
  │   └───lib/libcurl.la: …ib -L/nix/store/4mbayl1y5hpjbjzkx8ndyhkv98kqw1wi-libssh2-1.8.0/l…
  │       lib/libcurl.so.4.4.0: …/lib:/nix/store/4mbayl1y5hpjbjzkx8ndyhkv98kqw1wi-libssh2-1.8.0/l…
  │       => /nix/store/4mbayl1y5hpjbjzkx8ndyhkv98kqw1wi-libssh2-1.8.0
  └───lib/libnixstore.so: …/lib:/nix/store/bx2i9vi76lps6w9rr73fxf6my31s4dg5-aws-sdk-cpp-1.0…
      => /nix/store/bx2i9vi76lps6w9rr73fxf6my31s4dg5-aws-sdk-cpp-1.0.153
      └───lib/libaws-cpp-sdk-core.so: …e.so./nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0/lib…
          lib/libaws-cpp-sdk-s3.so: …/lib:/nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0/lib…
          => /nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0
2017-09-11 16:21:27 +02:00
Eelco Dolstra d41c5eb13f Add command "nix why-depends"
This command shows why a package has another package in its runtime
closure. For example, to see why VLC has libdrm.dev in its closure:

  $ nix why-depends nixpkgs.vlc nixpkgs.libdrm.dev
  /nix/store/g901z9pcj0n5yy5n6ykxk3qm4ina1d6z-vlc-2.2.5.1:
    lib/libvlccore.so.8.0.0: …nfig:/nix/store/405lmx6jl8lp0ad1vrr6j498chrqhz8g-libdrm-2.4.75-d…

  /nix/store/s3nm7kd8hlcg0facn2q1ff2n7wrwdi2l-mesa-noglu-17.0.7-dev:
    nix-support/propagated-native-build-inputs: …-dev /nix/store/405lmx6jl8lp0ad1vrr6j498chrqhz8g-libdrm-2.4.75-d…

Thus, VLC's lib/libvlccore.so.8.0.0 as well as mesa-noglu's
nix-support/propagated-native-build-inputs cause the dependency.
2017-09-10 22:40:33 +02:00
Eelco Dolstra 8af704eef1 Typo 2017-09-10 22:40:33 +02:00
Eelco Dolstra ad228d84e5 nix build: Only download the requested derivation outputs
Also some refactoring.
2017-09-10 22:40:33 +02:00
Eelco Dolstra e02edb1483
nix copy: Add --substitute flag 2017-09-08 15:32:07 +02:00
Eelco Dolstra 3cf1705583
nix copy: Don't open the --from store twice 2017-09-08 14:46:55 +02:00
Eelco Dolstra a2740c9ca2 nix copy: Add examples 2017-09-08 11:33:46 +02:00
Eelco Dolstra 24e23a1a73 nix search: Add examples 2017-09-07 20:42:11 +02:00
Eelco Dolstra 4967f0509a nix eval: Add examples 2017-09-07 20:37:46 +02:00
Eelco Dolstra 2ebeffcfd4 nix log: Add examples 2017-09-07 20:18:29 +02:00
Eelco Dolstra 17afc42895 nix build: Add examples 2017-09-07 20:14:04 +02:00
Eelco Dolstra 2e9b7c4cb2 nix run: Add some examples 2017-09-07 20:09:04 +02:00
Eelco Dolstra dff440aab3
nix build: Add --out-link and --no-link options 2017-09-06 16:20:34 +02:00
Eelco Dolstra df4342bc17
nix build: Create result symlinks 2017-09-06 16:03:22 +02:00
Eelco Dolstra 7a108d904e
Fix verbosity level for nix build --dry-run 2017-09-01 12:37:09 +02:00
Eelco Dolstra fd73c1e20a
Add an activity for binary cache queries 2017-08-31 15:25:58 +02:00
Eelco Dolstra 7d4a7136db
More macOS build fixes 2017-08-31 12:52:07 +02:00
Eelco Dolstra fabde432dc
Fix build failure on non-Linux
https://hydra.nixos.org/build/59649086
2017-08-31 11:05:18 +02:00
Eelco Dolstra e9c07a3b26
nix edit / log: Operate on a single Installable 2017-08-29 16:18:00 +02:00
Eelco Dolstra 9b82ecbae0
nix search: Warn about cached results 2017-08-29 15:22:05 +02:00
Eelco Dolstra c8235c5313
nix run: Flush the progress bar before starting the command 2017-08-29 15:13:30 +02:00
Eelco Dolstra 05d68a6e23
nix run: Add some flags for clearing/keeping the environment
This is useful for testing commands in isolation.

For example,

  $ nix run nixpkgs.geeqie -i -k DISPLAY -k XAUTHORITY -c geeqie

runs geeqie in an empty environment, except for $DISPLAY and
$XAUTHORITY.
2017-08-29 15:00:08 +02:00
Eelco Dolstra 5cc8609e30
nix run: Allow passing a command to execute
E.g.

  nix run nixpkgs.hello -c hello --greeting Hallo

Note that unlike "nix-shell --command", no quoting of arguments is
necessary.

"-c" (short for "--command") cannot be combined with "--" because they
both consume all remaining arguments. But since installables shouldn't
start with a dash, this is unlikely to cause problems.
2017-08-29 14:42:48 +02:00
Eelco Dolstra 93a5ef0516
nix run: Fix chroot execution
Running "nix run" with a diverted store, e.g.

  $ nix run --store local?root=/tmp/nix nixpkgs.hello

stopped working when Nix became multithreaded, because
unshare(CLONE_NEWUSER) doesn't work in multithreaded processes. The
obvious solution is to terminate all other threads first, but 1) there
is no way to terminate Boehm GC marker threads; and 2) it appears that
the kernel has a race where unshare(CLONE_NEWUSER) will still fail for
some indeterminate amount of time after joining other threads.

So instead, "nix run" will now exec() a single-threaded helper ("nix
__run_in_chroot") that performs the actual unshare()/chroot()/exec().
2017-08-29 13:21:07 +02:00
Eelco Dolstra 1c58e13bee
Hide commands that don't have a description
These are assumed to be internal.
2017-08-29 11:52:55 +02:00
Eelco Dolstra 2cc345b95f
Give activities a verbosity level again
And print them (separately from the progress bar) given sufficient -v
flags.
2017-08-28 19:13:24 +02:00
Eelco Dolstra e681b1f064
Simplify 2017-08-28 14:30:35 +02:00
Eelco Dolstra 94a0548dc4
Simplify 2017-08-25 21:26:37 +02:00
Eelco Dolstra 9b845e6936
Doh 2017-08-25 20:52:34 +02:00
Eelco Dolstra 0ac35b67b8
Allow derivations to update the build phase
So the progress bar can show

  [1/0/1 built, 0.0 MiB DL] building hello-2.10 (configuring): checking whether pread is declared without a macro... yes
2017-08-25 18:04:05 +02:00
Eelco Dolstra c137c0a5eb
Allow activities to be nested
In particular, this allows more relevant activities ("substituting X")
to supersede inferior ones ("downloading X").
2017-08-25 17:49:40 +02:00
Eelco Dolstra f194629f96
Fix Debian build
https://hydra.nixos.org/build/59390148
2017-08-25 16:11:18 +02:00
Eelco Dolstra db1d45037c
Handle SIGWINCH 2017-08-25 15:59:03 +02:00
Eelco Dolstra ec9e0c03c3
When truncating the progress bar, take ANSI escape sequences into account 2017-08-25 15:59:03 +02:00
Eelco Dolstra 0e9ddcc306
Restore activity metadata
This allows the progress bar to display "building perl-5.22.3" instead
of "building /nix/store/<hash>-perl-5.22.3.drv".
2017-08-25 15:58:35 +02:00
Eelco Dolstra 4c6a26539c
Remove debug line 2017-08-21 12:18:46 +02:00
Eelco Dolstra c2cab20732
nix verify: Restore the progress indicator 2017-08-16 20:56:03 +02:00
Eelco Dolstra b4ed97e3a3
nix optimise-store: Show how much space has been freed 2017-08-16 20:56:03 +02:00
Eelco Dolstra 23b8b7e096
nix optimise-store: Add
This replaces "nix-store --optimise". Main difference is that it has a
progress indicator.
2017-08-16 20:56:03 +02:00
Eelco Dolstra 40bffe0a43
Progress indicator: Cleanup 2017-08-16 20:56:03 +02:00
Eelco Dolstra dff12b38f9
Progress indicator: More improvements 2017-08-16 20:56:03 +02:00
Eelco Dolstra bf1f123b09
Progress indicator: Show number of active items 2017-08-16 20:56:03 +02:00
Eelco Dolstra 0e0dcf2c7e
Progress indicator: Unify "copying" and "substituting"
They're the same thing after all.

Example:

  $ nix build --store local?root=/tmp/nix nixpkgs.firefox-unwrapped
  [0/1 built, 49/98 copied, 16.3/92.8 MiB DL, 55.8/309.2 MiB copied] downloading 'https://cache.nixos.org/nar/0pl9li1jigcj2dany47hpmn0r3r48wc4nz48v5mqhh426lgz3bz6.nar.xz'
2017-08-16 20:56:03 +02:00
Eelco Dolstra c36467ad2e
Improve substitution progress indicator
E.g.

  $ nix build --store local?root=/tmp/nix nixpkgs.firefox-unwrapped
  [0/1 built, 1/97/98 fetched, 65.8/92.8 MiB DL, 203.2/309.2 MiB copied] downloading 'https://cache.nixos.org/nar/1czm9fk0svacy4h6a3fzkpafi4f7a9gml36kk8cq1igaghbspg3k.nar.xz'
2017-08-16 20:56:02 +02:00
Eelco Dolstra b29b6feaba
nix copy: Improve progress indicator
It now shows the amount of data copied:

  [8/1038 copied, 160.4/1590.9 MiB copied] copying path '...'
2017-08-16 20:56:02 +02:00
Eelco Dolstra c5e4404580
nix copy: Revive progress bar 2017-08-16 20:56:02 +02:00
Eelco Dolstra dffc3fe43b
nix copy: Add --no-check-sigs flag 2017-08-16 20:56:02 +02:00
Eelco Dolstra c6184dec6c
nix repl: Support printing floating-point numbers 2017-08-09 15:17:29 +02:00
Jörg Thalheim 2fd8f8bb99 Replace Unicode quotes in user-facing strings by ASCII
Relevant RFC: NixOS/rfcs#4

$ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-07-30 12:32:45 +01:00
Eelco Dolstra 57b9505731
nix search: Add a cache
The package list is now cached in
~/.cache/nix/package-search.json. This gives a substantial speedup to
"nix search" queries. For example (on an SSD):

First run: (no package search cache, cold page cache)

  $ time nix search blender
  Attribute name: nixpkgs.blender
  Package name: blender
  Version: 2.78c
  Description: 3D Creation/Animation/Publishing System

  real    0m6.516s

Second run: (package search cache populated)

  $ time nix search blender
  Attribute name: nixpkgs.blender
  Package name: blender
  Version: 2.78c
  Description: 3D Creation/Animation/Publishing System

  real    0m0.143s
2017-07-26 17:29:10 +02:00
Eelco Dolstra c94f3d5575
nix-shell: Use bashInteractive from <nixpkgs>
This adds about 0.1s to nix-shell runtime in the case where
bashInteractive already exists.

See discussion at https://github.com/NixOS/nixpkgs/issues/27493.
2017-07-20 13:50:25 +02:00
Eelco Dolstra 57a30e101b
nix search: Ignore top-level eval errors
$NIX_PATH may contain elements that don't evaluate to an attrset (like
"nixos-config"), so ignore those.
2017-07-20 13:33:13 +02:00
Eelco Dolstra b144c4d617
nix search: Add --json flag 2017-07-20 13:33:13 +02:00
Eelco Dolstra 90825dea51
Add "nix search" command 2017-07-20 13:33:13 +02:00
Eelco Dolstra 766ad5db3b
nix path-info: Show download sizes for binary cache stores
E.g.

  $ nix path-info --json --store https://cache.nixos.org nixpkgs.thunderbird -S
  ...
      "downloadHash": "sha256:1jlixpzi225wwa0f4xdrwrqgi47ip1qpj9p06fyxxg07sfmyi4q0",
      "downloadSize": 43047620,
      "closureDownloadSize": 84745960
    }
  ]
2017-07-14 18:29:10 +02:00
Eelco Dolstra fdc9da034f
Avoid a call to derivationFromPath()
This doesn't work in read-only mode, ensuring that operations like

  nix path-info --store https://cache.nixos.org -S nixpkgs.hello

(asking for the closure size of nixpkgs.hello in cache.nixos.org) work
when nixpkgs.hello doesn't exist in the local store.
2017-07-14 18:29:10 +02:00
Eelco Dolstra 3908d3929c
nix path-info: Don't barf on invalid paths
Now you get

  [
    {
      "path": "/nix/store/fzvliz4j5xzvnd0w5zgw2l0ksqh578yk-bla",
      "valid": false
    }
  ]
2017-07-14 18:29:10 +02:00
Eelco Dolstra 6438ba22af
StorePathsCommand: Don't build installables
On second though this was annoying. E.g. "nix log nixpkgs.hello" would
build/download Hello first, even though the log can be fetched
directly from the binary cache.

May need to revisit this.
2017-07-14 18:29:07 +02:00
Eelco Dolstra 112ff7833d
nix: Show help when no arguments are given
Fixes #1464.
2017-07-14 13:44:45 +02:00
Eelco Dolstra ad8b96f1f2
Fix handling of expression installables with a / in them 2017-07-04 15:38:23 +02:00
Eelco Dolstra c0015e87af
Support base-64 hashes
Also simplify the Hash API.

Fixes #1437.
2017-07-04 15:07:41 +02:00
Eelco Dolstra 186571965d
Don't show flags from config settings in "nix --help" 2017-06-07 18:41:20 +02:00
Eelco Dolstra aa952d5f0b
nix: Add --help-config flag 2017-06-07 16:49:54 +02:00
Eelco Dolstra b8283773bd
nix: Make all options available as flags
Thus, instead of ‘--option <name> <value>’, you can write ‘--<name>
<value>’. So

  --option http-connections 100

becomes

  --http-connections 100

Apart from brevity, the difference is that it's not an error to set a
non-existent option via --option, but unrecognized arguments are
fatal.

Boolean options have special treatment: they're mapped to the
argument-less flags ‘--<name>’ and ‘--no-<name>’. E.g.

  --option auto-optimise-store false

becomes

  --no-auto-optimise-store
2017-06-07 16:17:17 +02:00
Eelco Dolstra 588dad4084
Fix build failure on Debian/Ubuntu
http://hydra.nixos.org/build/53537463
2017-05-29 15:59:18 +02:00
Eelco Dolstra b01d62285c
Improve progress indicator 2017-05-16 16:09:57 +02:00
Benno Fünfstück 06880d7ed8 nix ls: support '/' for the root directory 2017-05-15 10:25:55 +02:00
Eelco Dolstra c5f23f10a8
Replace readline by linenoise
Using linenoise avoids a license compatibility issue (#1356), is a lot
smaller and doesn't pull in ncurses.
2017-05-10 18:37:42 +02:00
Eelco Dolstra 03ae5e6459
Add "nix edit" command
This is a little convenience command that opens the Nix expression of
the specified package. For example,

  nix edit nixpkgs.perlPackages.Moose

opens <nixpkgs/pkgs/top-level/perl-packages.nix> in $EDITOR (at the
right line number for some editors).

This requires the package to have a meta.position attribute.
2017-05-08 18:42:30 +02:00
Eelco Dolstra 2da6a42448
nix dump-path: Add
This is primarily useful for extracting NARs from other stores (like
binary caches), which "nix-store --dump" cannot do.
2017-05-04 14:21:22 +02:00
Eelco Dolstra 782c0bff45
nix eval: Add a --raw flag
Similar to "jq -r", this prints the evaluation result (which must be a
string value) unquoted.
2017-05-03 14:08:18 +02:00
Eelco Dolstra cef8c169b1
Fix "nix ... --all"
When "--all" is used, we should not fill in a default installable.
2017-05-02 15:46:10 +02:00
Eelco Dolstra 73bba12d8b
Check for libreadline 2017-04-28 16:53:56 +02:00
Eelco Dolstra 6734c18c99
nix repl: Fix Ctrl-C 2017-04-25 19:19:48 +02:00
Eelco Dolstra 23aa1619da
Minor cleanup 2017-04-25 19:10:47 +02:00
Eelco Dolstra 536f061765
"using namespace std" considered harmful 2017-04-25 18:58:02 +02:00
Eelco Dolstra 5bd8795e1f
nix repl: Use $XDG_DATA_HOME for the readline history 2017-04-25 18:56:29 +02:00
Eelco Dolstra 921a2aeb05
Make "nix repl" build 2017-04-25 18:48:40 +02:00
Eelco Dolstra c31000bc93
Merge nix-repl repository 2017-04-25 18:14:13 +02:00
Eelco Dolstra 40daf0d800
Cleanup in preparation of merging nix-repl repo into nix repo 2017-04-25 18:13:23 +02:00
Eelco Dolstra c30330df6f
StorePathCommands: Build installables
So for instance "nix copy --to ... nixpkgs.hello" will build
nixpkgs.hello first. It's debatable whether this is a good idea. It
seems desirable for commands like "nix copy" but maybe not for
commands like "nix path-info".
2017-04-25 16:19:22 +02:00
Eelco Dolstra d48c973ece
Set default installable
Thus

  $ nix build -f foo.nix

will build foo.nix.

And

  $ nix build

will build default.nix. However, this may not be a good idea because
it's kind of inconsistent, given that "nix build foo" will build the
"foo" attribute from the default installation source (i.e. the
synthesis of $NIX_PATH), rather than ./default.nix. So I may revert
this.
2017-04-25 15:18:05 +02:00
Eelco Dolstra 0b6220fbd6
Interpret any installable containing a slash as a path
So "nix path-info ./result" now works.
2017-04-25 14:09:01 +02:00
Eelco Dolstra 7ee81f3887
Make StorePathsCommand a subclass of InstallablesCommand
This allows commands like 'nix path-info', 'nix copy', 'nix verify'
etc. to work on arbitrary installables. E.g. to copy geeqie to a
binary cache:

  $ nix copy -r --to file:///tmp/binary-cache nixpkgs.geeqie

Or to get the closure size of thunderbird:

  $ nix path-info -S nixpkgs.thunderbird
2017-04-25 13:20:26 +02:00
Eelco Dolstra c769841bc4
Move code around 2017-04-25 12:07:31 +02:00
Eelco Dolstra 6267d74889
Add "nix eval" command
This replaces "nix-instantiate --eval". The result is evaluated
strictly since this seems more useful.
2017-04-25 11:23:47 +02:00
Eelco Dolstra bcecc99007
Restructure installables handling in the "nix" command 2017-04-25 11:20:37 +02:00
Eelco Dolstra 66577a1c64
Factor out --json 2017-04-24 14:21:36 +02:00
Eelco Dolstra efa4bdbfcd
Improve nix show-config --json
In particular, show descriptions. This could be used for manpage
generation etc.
2017-04-20 17:34:47 +02:00
Eelco Dolstra 872ba75d8b
Add "nix show-config" command
This dumps the entire Nix configuration, including all options that
have default values.
2017-04-13 20:59:38 +02:00
Eelco Dolstra ba9ad29fdb
Convert Settings to the new config system
This makes all config options self-documenting.

Unknown or unparseable config settings and --option flags now cause a
warning.
2017-04-13 20:53:23 +02:00
Will Dietz 30f89e0d65 Process nix.conf options in "new" nix commands, add test.
Without this (minor) change, the options set using "--option"
or read from nix.conf were parsed but not used.
2017-04-08 12:59:42 -05:00
Eelco Dolstra 558eda0115
nix copy: Make -r option use the "from" store
Previously, we tried to compute the closure in the local store, which
obviously doesn't work.
2017-03-16 14:25:54 +01:00
Eelco Dolstra c5b83d8913
copyPaths(): Use queryValidPaths() to reduce SSH latency 2017-03-16 13:50:01 +01:00
Eelco Dolstra 5b86451f02
Add a "nix log" command
This replaces "nix-store --read-log". It checks the local store and
any configured substituters for the requested logs.
2017-03-15 16:48:29 +01:00
Eelco Dolstra c2b0d8749f
exportReferencesGraph: Export more complete info in JSON format
This writes info about every path in the closure in the same format as
‘nix path-info --json’. Thus it also includes NAR hashes and sizes.

Example:

  [
    {
      "path": "/nix/store/10h6li26i7g6z3mdpvra09yyf10mmzdr-hello-2.10",
      "narHash": "sha256:0ckdc4z20kkmpqdilx0wl6cricxv90lh85xpv2qljppcmz6vzcxl",
      "narSize": 197648,
      "references": [
        "/nix/store/10h6li26i7g6z3mdpvra09yyf10mmzdr-hello-2.10",
        "/nix/store/27binbdy296qvjycdgr1535v8872vz3z-glibc-2.24"
      ],
      "closureSize": 20939776
    },
    {
      "path": "/nix/store/27binbdy296qvjycdgr1535v8872vz3z-glibc-2.24",
      "narHash": "sha256:1nfn3m3p98y1c0kd0brp80dn9n5mycwgrk183j17rajya0h7gax3",
      "narSize": 20742128,
      "references": [
        "/nix/store/27binbdy296qvjycdgr1535v8872vz3z-glibc-2.24"
      ],
      "closureSize": 20742128
    }
  ]

Fixes #1134.
2017-01-26 20:41:08 +01:00
Eelco Dolstra 8af062f372 Merge pull request #981 from shlevy/build-remote-c++
build-remote: Implement in C++
2017-01-19 18:21:55 +01:00
Eelco Dolstra 215b70f51e
Revert "Get rid of unicode quotes (#1140)"
This reverts commit f78126bfd6. There
really is no need for such a massive change...
2016-11-26 00:38:01 +01:00
Guillaume Maudoux f78126bfd6 Get rid of unicode quotes (#1140) 2016-11-25 15:48:27 +01:00
Eelco Dolstra dd77f7d593 Store::computeFSClosure(): Support a set of paths
This way, callers can exploits the parallelism of computeFSClosure()
when they have multiple paths that they need the (combined) closure of.
2016-11-10 17:45:04 +01:00
Shea Levy 167d12b02c build-remote: Implement in C++ 2016-11-10 11:09:15 -05:00
Eelco Dolstra 4b8f1b0ec0 Merge branch 'ssh-store' of https://github.com/shlevy/nix 2016-11-09 18:57:22 +01:00
Eelco Dolstra c55bf085eb printMsg(lvlError, ...) -> printError(...) etc. 2016-09-21 16:54:53 +02:00
Eelco Dolstra 2e1493037b nix path-info: Add some more examples 2016-09-14 18:20:11 +02:00
Shea Levy a91954f0c6 Merge openStore and openStoreAt with default arguments 2016-09-02 06:35:48 -04:00
Eelco Dolstra d74c8a3f4e Fix 32-bit build 2016-08-30 17:38:09 +02:00
Eelco Dolstra c0a7b84748 nix path-info: Add --json flag
Also, factor out JSON generation from value-to-json.{cc,hh}, and
support producing indented JSON.
2016-08-29 17:29:24 +02:00
Eelco Dolstra d74236d1f2 nix build: Use Nix search path
That is, unless --file is specified, the Nix search path is
synthesized into an attribute set. Thus you can say

  $ nix build nixpkgs.hello

assuming $NIX_PATH contains an entry of the form "nixpkgs=...". This
is more verbose than

  $ nix build hello

but is less ambiguous.
2016-08-23 17:11:19 +02:00
Eelco Dolstra d961c29c9c Mark content-addressed paths in the Nix database and in .narinfo
This allows such paths to be imported without signatures.
2016-08-10 18:05:35 +02:00
Eelco Dolstra f8a8b4d8f8 nix run: Set a reasonable uid/gid 2016-06-02 19:04:09 +02:00
Eelco Dolstra eda2aaae92 nix run: Handle the case where the /nix/store mount point doesn't exist 2016-06-02 18:24:51 +02:00
Eelco Dolstra a24f2c9b84 nix run: Mount the Nix store in a private namespace
This is a convenience command to allow users who are not privileged to
create /nix/store to use Nix with regular binary caches. For example,

  $ NIX_REMOTE="local?state=$HOME/nix/var&real=/$HOME/nix/store" nix run firefox bashInteractive

will download Firefox and bash from cache.nixos.org, then start a
shell in which $HOME/nix/store is mounted on /nix/store.
2016-06-02 16:51:43 +02:00
Eelco Dolstra 6f2d51287c Add basic "nix run" command 2016-06-02 16:29:49 +02:00
Eelco Dolstra 7850d3d279 Make the store directory a member variable of Store 2016-06-01 16:24:17 +02:00
Eelco Dolstra a9fa5e050a Shut up some clang warnings 2016-05-31 13:31:04 +02:00
Eelco Dolstra 538a64e8c3 Add a Store::addToStore() variant that accepts a NAR
As a side effect, this ensures that signatures are propagated when
copying paths between stores.

Also refactored import/export to make use of this.
2016-05-04 16:15:54 +02:00
Eelco Dolstra dfebfc835f Add a copyStorePath() utility function 2016-05-04 11:01:48 +02:00
Eelco Dolstra ca9de88a51 nix: Disable verbose builds 2016-04-25 19:18:45 +02:00
Eelco Dolstra 41633f9f73 Improved logging abstraction
This also gets rid of --log-type, since the nested log type isn't
useful in a multi-threaded situation, and nobody cares about the
"pretty" log type.
2016-04-25 19:18:45 +02:00
Eelco Dolstra c879a20850 Factor out parallel processing of work items that have dependencies 2016-04-22 20:50:06 +02:00
Eelco Dolstra 91539d305f nix copy: Parallelise 2016-04-22 18:19:48 +02:00
Eelco Dolstra 456179018a Fold "nix query-path-sigs" into "nix path-info" 2016-04-22 14:39:37 +02:00
Eelco Dolstra 95abf9c402 Add "nix copy" command
This replaces nix-push. For example,

  $ nix copy --to file:///tmp/cache -r $(type -p firefox)

copies the closure of firefox to the specified binary cache. And

  $ nix copy --from file:///tmp/cache --to s3://my-cache /nix/store/abcd...

copies between two binary caches.

It will also replace nix-copy-closure, once we have an SSHStore class,
e.g.

  $ nix copy --from ssh://alice@machine /nix/store/abcd...
2016-04-22 14:33:01 +02:00
Eelco Dolstra 7d14f5c331 Implement S3BinaryCacheStore::queryAllValidPaths()
This allows commands like "nix verify --all" or "nix path-info --all"
to work on S3 caches.

Unfortunately, this requires some ugly hackery: when querying the
contents of the bucket, we don't want to have to read every .narinfo
file. But the S3 bucket keys only include the hash part of each store
path, not the name part. So as a special exception
queryAllValidPaths() can now return store paths *without* the name
part, and queryPathInfo() accepts such store paths (returning a
ValidPathInfo object containing the full name).
2016-04-21 17:53:47 +02:00
Eelco Dolstra 1a71495273 nix path-info: Add
Forgot to commit this earlier...
2016-04-21 15:00:45 +02:00
Eelco Dolstra 69e3ffb076 nix --help: Show usage examples 2016-04-21 14:58:32 +02:00
Eelco Dolstra e0204f8d46 Move path info caching from BinaryCacheStore to Store
Caching path info is generally useful. For instance, it speeds up "nix
path-info -rS /run/current-system" (i.e. showing the closure sizes of
all paths in the closure of the current system) from 5.6s to 0.15s.

This also eliminates some APIs like Store::queryDeriver() and
Store::queryReferences().
2016-04-19 18:52:53 +02:00
Eelco Dolstra 99851c6f06 Unify "nix verify-paths" and "nix verify-store"
"verify-store" is now simply an "--all" flag to "nix verify". This
flag can be used for any other store path command as well (e.g. "nix
path-info", "nix copy-sigs", ...).
2016-04-15 15:39:48 +02:00
Eelco Dolstra 05fbc606fc nix verify-paths: Add ‘--sigs-needed <N>’ flag
This specifies the number of distinct signatures required to consider
each path "trusted".

Also renamed ‘--no-sigs’ to ‘--no-trust’ for the flag that disables
verifying whether a path is trusted (since a path can also be trusted
if it has no signatures, but was built locally).
2016-04-07 15:16:57 +02:00
Eelco Dolstra b654381eb3 Add "nix sign-paths" command
E.g.

  $ nix sign-paths -k ./secret -r $(type -p geeqie)

signs geeqie and all its dependencies using the key in ./secret.
2016-04-05 16:39:29 +02:00
Eelco Dolstra d0f5719c2a Add "nix copy-sigs" command
This imports signatures from one store into another. E.g.

  $ nix copy-sigs -r /run/current-system -s https://cache.nixos.org/
  imported 595 signatures
2016-04-05 16:39:23 +02:00
Eelco Dolstra 39a6abc0bc nix verify: Support checking against signatures in other stores
Typical usage is to check local paths using the signatures from a
binary cache:

  $ nix verify-paths -r /run/current-system -s https://cache.nixos.org
  path ‘/nix/store/c1k4zqfb74wba5sn4yflb044gvap0x6k-nixos-system-mandark-16.03.git.fc2d7a5M’ is untrusted
  ...
  checked 844 paths, 119 untrusted
2016-03-30 11:39:34 +02:00
Eelco Dolstra 4f34c40398 Add "nix verify-store" command
Like "nix-store --verify --check-contents", but with the same
advantages as "nix verify-paths".
2016-03-29 16:37:16 +02:00
Eelco Dolstra 784ee35c80 Add "nix verify-paths" command
Unlike "nix-store --verify-path", this command verifies signatures in
addition to store path contents, is multi-threaded (especially useful
when verifying binary caches), and has a progress indicator.

Example use:

$ nix verify-paths --store https://cache.nixos.org -r $(type -p thunderbird)
...
[17/132 checked] checking ‘/nix/store/rawakphadqrqxr6zri2rmnxh03gqkrl3-autogen-5.18.6’
2016-03-29 16:37:16 +02:00
Eelco Dolstra cebc150b7c nix: Add --store flag
This is a bit user-friendlier than using $NIX_REMOTE.
2016-03-21 18:03:36 +01:00
Eelco Dolstra 02654f782f Fix Darwin build
http://hydra.nixos.org/build/33279996
2016-03-15 12:11:27 +01:00
Eelco Dolstra dc4a71aae5 Fix build on clang due to -Wmismatched-tags
http://hydra.nixos.org/build/33073389
2016-03-14 12:37:30 +01:00
Eelco Dolstra ce113c32d2 Add warning about "nix" being experimental 2016-03-04 15:54:41 +01:00
Eelco Dolstra 00b2c05749 nix: Add commands to query contents of NARs / binary caches
For example,

  $ NIX_REMOTE=file:///my-cache nix ls-store -lR /nix/store/f4kbgl8shhyy76rkk3nbxr0lz8d2ip7q-binutils-2.23.1
  dr-xr-xr-x                    0 ./bin
  -r-xr-xr-x                30748 ./bin/addr2line
  -r-xr-xr-x                66973 ./bin/ar
  ...

Similarly, "nix ls-nar" lists the contents of a NAR file, "nix
cat-nar" extracts a file from a NAR file, and "nix cat-store" extract
a file from a Nix store.
2016-02-25 17:57:00 +01:00
Eelco Dolstra 7873cfb18d Fix build 2016-02-25 11:55:05 +01:00
Tristan Hume 9b05d5848c Fix to-base16 description 2016-02-16 10:15:58 +01:00
Eelco Dolstra 206bbb5dc9 Add basic "nix build" command
Currently only builds by attribute from <nixpkgs> or the specified
file, e.g. "nix build hello".
2016-02-09 21:34:24 +01:00
Eelco Dolstra cd2196b089 Start of new Nix command-line interface 2016-02-09 21:28:29 +01:00
Eelco Dolstra b1117ef29d * nix -> nix-store, fix -> nix-instantiate. 2003-11-18 11:38:25 +00:00
Eelco Dolstra ce92d1bf14 * "Nix expression" -> "store expression".
* More refactoring.
2003-11-18 11:22:29 +00:00
Eelco Dolstra 9f0f020929 * libnix -> libstore. 2003-11-18 10:55:27 +00:00
Eelco Dolstra 8798fae304 * Source tree refactoring. 2003-11-18 10:47:59 +00:00
Eelco Dolstra c62433751d * Finished refactoring the tree. 2003-10-20 10:05:01 +00:00
Eelco Dolstra 53e376d836 * Refactored the source tree. 2003-10-20 09:20:11 +00:00