Commit graph

640 commits

Author SHA1 Message Date
Shea Levy 7459388448
Merge branch 'fix/ruby-shebang' of git://github.com/samueldr/nix 2018-04-09 10:54:39 -04:00
Samuel Dionne-Riel a4c9b2595e Hardcodes nix-shell instead of /usr/bin/env nix-shell...
... in the ruby shebang test.
2018-04-09 09:36:54 -04:00
Samuel Dionne-Riel 9478f88681 Adds nix-shell test for special-cased ruby interpreter.
The test fakes the interpreter only to verify the arguments it would be
given.
2018-04-08 00:45:28 -04:00
Doug Beardsley 72902ec399 Add missing eval-okay-regex-split.exp test file 2018-04-06 12:18:59 -06:00
Linus Heckemann 12913ccf45 nix-shell: allow symlinks to .drvs
This makes persistent shell environments easier to use.
2018-03-16 22:15:24 +00:00
Eelco Dolstra 64441f0551
Merge pull request #1939 from dezgeg/lexer-fix
libexpr: Recognize newline in more places in lexer
2018-03-16 12:14:58 +01:00
Eelco Dolstra ca14b14200
Use boost::format from the boost package
Note that this only requires headers from boost so it doesn't add a
runtime dependency.

Also, use Nixpkgs 18.03.
2018-03-14 19:24:04 +01:00
Shea Levy 56f2ed0081
Merge pull request #1906 from dtzWill/fix/nix-search
nix search: tests and fix #1893 and part of #1892
2018-03-13 18:43:47 -06:00
Shea Levy aa8bbbf69d
Merge branch 'write-failure-fixes' of git://github.com/lheckemann/nix 2018-03-02 10:59:59 -05:00
Tuomas Tynkkynen a0e38c16bc libexpr: Recognize newline in more places in lexer
Flex's regexes have an annoying feature: the dot matches everything
except a newline. This causes problems for expressions like:

"${0}\
"

where the backslash-newline combination matches this rule instead of the
intended one mentioned in the comment:

    <STRING>\$|\\|\$\\ {
                    /* This can only occur when we reach EOF, otherwise the above
                    (...|\$[^\{\"\\]|\\.|\$\\.)+ would have triggered.
                    This is technically invalid, but we leave the problem to the
                    parser who fails with exact location. */
                    return STR;
                }
However, the parser actually accepts the resulting token sequence
('"' DOLLAR_CURLY 0 '}' STR '"'), which is a problem because the lexer
rule didn't assign anything to yylval. Ultimately this leads to a crash
when dereferencing a NULL pointer in ExprConcatStrings::bindVars().

The fix does change the syntax of the language in some corner cases
but I think it's only turning previously invalid (or crashing) syntax
to valid syntax. E.g.

"a\
b"

and

''a''\
b''

were previously syntax errors but now both result in "a\nb".

Found by afl-fuzz.
2018-03-02 17:30:48 +02:00
Will Dietz c577186f59 tests/search.sh: simplify, don't rely on consistent ordering 2018-03-01 13:25:06 -06:00
Tuomas Tynkkynen 77e9e1ed91 libexpr: Fix prim_replaceStrings() to work on an empty source string
Otherwise, running e.g.

nix-instantiate --eval -E --strict 'builtins.replaceStrings [""] ["X"] "abc"'

would just hang in an infinite loop.

Found by afl-fuzz.

First attempt of this was reverted in e2d71bd186 because it caused
another infinite loop, which is fixed now and a test added.
2018-02-26 19:49:13 +02:00
Will Dietz 8282c60d74 tests: test nix search behavior 2018-02-25 16:40:05 -06:00
Eelco Dolstra de4c03d201
Merge branch 'fix/dry-run-partially' of https://github.com/dtzWill/nix 2018-02-22 12:18:20 +01:00
Eelco Dolstra e2d71bd186
Revert "libexpr: Fix prim_replaceStrings() to work on an empty source string"
This reverts commit 4ea9707591.

It causes an infinite loop in Nixpkgs evaluation,
e.g. "nix-instantiate -A hello" hung.

PR #1886.
2018-02-21 15:35:28 +01:00
Tuomas Tynkkynen 4ea9707591 libexpr: Fix prim_replaceStrings() to work on an empty source string
Otherwise, running e.g.

nix-instantiate --eval -E --strict 'builtins.replaceStrings [""] ["X"] "abc"'

would just hang in an infinite loop.

Found by afl-fuzz.
2018-02-19 23:20:26 +02:00
Eelco Dolstra 75a1d96cfd
Merge branch 'register-settings' of https://github.com/shlevy/nix 2018-02-19 13:58:34 +01:00
Shea Levy b095c06139
Add splitVersion primop.
Fixes #1868.
2018-02-14 09:55:43 -05:00
Shea Levy de4934ab3b
Allow plugins to define new settings. 2018-02-13 14:43:32 -05:00
Shea Levy 3fe9767dd3
Fix plugin tests on darwin 2018-02-13 12:49:14 -05:00
Linus Heckemann 78ac3eb4eb Fix #1762
nix-store --export, nix-store --dump, and nix dump-path would previously
fail silently if writing the data out failed, because
 a) FdSink::write ignored exceptions, and
 b) the commands relied on FdSink's destructor, which ignores
    exceptions, to flush the data out.

This could cause rather opaque issues with installing nixos, because
nix-store --export would happily proceed even if it couldn't write its
data out (e.g. if nix-store --import on the other side of the pipe
failed).

This commit adds tests that expose these issues in the nix-store
commands, and fixes them for all three.
2018-02-13 14:26:19 +00:00
Shea Levy 6eb1040e90
Allow includes from nix.conf 2018-02-13 08:16:32 -05:00
Eelco Dolstra 7828dca9e8
Merge branch 'register-constant' of https://github.com/shlevy/nix 2018-02-13 12:24:48 +01:00
Eelco Dolstra 1c10a74c73
Merge branch 'plugins' of https://github.com/shlevy/nix 2018-02-13 12:15:27 +01:00
Shea Levy 081f14a169
Allow using RegisterPrimop to define constants.
This enables plugins to add new constants, as well as new primops.
2018-02-08 14:35:50 -05:00
Shea Levy 88cd2d41ac
Add plugins to make Nix more extensible.
All plugins in plugin-files will be dlopened, allowing them to
statically construct instances of the various Register* types Nix
supports.
2018-02-08 12:44:37 -05:00
Eelco Dolstra ad97a21834
nix-env: Fix parsing of --system
https://hydra.nixos.org/build/68827814
2018-02-08 15:25:03 +01:00
Eelco Dolstra 444bae44ef
dsa -> ed25519
DSS is disabled by default in NixOS 18.03.

https://hydra.nixos.org/build/68788560
2018-02-08 13:46:23 +01:00
Will Dietz c7e0be1bfc build-dry: disable failing portion of test until issue is fixed 2018-02-07 15:20:54 -06:00
Will Dietz 3780435a0e tests: Add (failing) tests for reported --dry-run issues. 2018-02-07 15:20:54 -06:00
Shea Levy 48c192ca2d
builtins.path test: Don't rely on shlevy's XDG_RUNTIME_DIR 2018-02-07 10:26:53 -05:00
Eelco Dolstra 84989d3af2
Improve filtering of ANSI escape sequences in build logs
All ANSI sequences except color setting are now filtered out. In
particular, terminal resets (such as from NixOS VM tests) are filtered
out.

Also, fix the completely broken tab character handling.
2018-02-07 15:23:57 +01:00
Eelco Dolstra abe6be578b
Merge pull request #1816 from shlevy/add-path
Add path primop.
2018-02-07 13:32:35 +01:00
Shea Levy 69d82e5c58
Add path primop.
builtins.path allows specifying the name of a path (which makes paths
with store-illegal names now addable), allows adding paths with flat
instead of recursive hashes, allows specifying a filter (so is a
generalization of filterSource), and allows specifying an expected
hash (enabling safe path adding in pure mode).
2018-02-06 16:48:08 -05:00
Eelco Dolstra 43f8ef73c6
realiseContext(): Add derivation outputs to the allowed paths
This makes import-from-derivation work in restricted mode again.
2018-02-06 15:38:45 +01:00
Eelco Dolstra f539085e65
Fix evaluation 2018-02-05 21:48:09 +01:00
Eelco Dolstra 84722d67d2
Remove nix-build --hash
Instead, if a fixed-output derivation produces has an incorrect output
hash, we now unconditionally move the outputs to the path
corresponding with the actual hash and register it as valid. Thus,
after correcting the hash in the Nix expression (e.g. in a fetchurl
call), the fixed-output derivation doesn't have to be built again.

It would still be good to have a command for reporting the actual hash
of a fixed-output derivation (instead of throwing an error), but
"nix-build --hash" didn't do that.
2018-02-03 10:08:05 +01:00
Eelco Dolstra 855699855f
Remove obsolete references to manifests
Closes #323.
2018-02-01 10:39:16 +01:00
Eelco Dolstra 98f3c75a0e
Merge pull request #1797 from dezgeg/userns-tests-fix
Fix tests using user namespaces on kernels that don't have it
2018-01-24 17:17:48 +01:00
Eelco Dolstra c382866cd2
Fix test
https://hydra.nixos.org/build/67806811
2018-01-22 17:04:08 +01:00
Eelco Dolstra 9304fde8de
Don't access tarballs.nixos.org in a test
https://hydra.nixos.org/build/67806811
2018-01-22 16:56:28 +01:00
Eelco Dolstra 3c4c30eadd
Rewrite builtin derivation environment
Also add a test.

Fixes #1803.
Closes #1805.
2018-01-19 15:00:38 +01:00
Eelco Dolstra 87e3d142cc
Add a test for --check / --repeat 2018-01-19 13:58:28 +01:00
Eelco Dolstra d4dcffd643
Add pure evaluation mode
In this mode, the following restrictions apply:

* The builtins currentTime, currentSystem and storePath throw an
  error.

* $NIX_PATH and -I are ignored.

* fetchGit and fetchMercurial require a revision hash.

* fetchurl and fetchTarball require a sha256 attribute.

* No file system access is allowed outside of the paths returned by
  fetch{Git,Mercurial,url,Tarball}. Thus 'nix build -f ./foo.nix' is
  not allowed.

Thus, the evaluation result is completely reproducible from the
command line arguments. E.g.

  nix build --pure-eval '(
    let
      nix = fetchGit { url = https://github.com/NixOS/nixpkgs.git; rev = "9c927de4b179a6dd210dd88d34bda8af4b575680"; };
      nixpkgs = fetchGit { url = https://github.com/NixOS/nixpkgs.git; ref = "release-17.09"; rev = "66b4de79e3841530e6d9c6baf98702aa1f7124e4"; };
    in (import (nix + "/release.nix") { inherit nix nixpkgs; }).build.x86_64-linux
  )'

The goal is to enable completely reproducible and traceable
evaluation. For example, a NixOS configuration could be fully
described by a single Git commit hash. 'nixos-rebuild' would do
something like

  nix build --pure-eval '(
    (import (fetchGit { url = file:///my-nixos-config; rev = "..."; })).system
  ')

where the Git repository /my-nixos-config would use further fetchGit
calls or Git externals to fetch Nixpkgs and whatever other
dependencies it has. Either way, the commit hash would uniquely
identify the NixOS configuration and allow it to reproduced.
2018-01-16 19:23:18 +01:00
Tuomas Tynkkynen 59086e459c Fix tests using user namespaces on kernels that don't have it
Disable various tests if the kernel doesn't support unprivileged user
namespaces (e.g. Arch Linux disables them) or disable them via a sysctl
(Debian, Ubuntu).

Fixes #1521
Fixes #1625
2018-01-13 19:08:41 +02:00
Will Dietz 84d9e213d2 fetchGit.sh: Test we don't "corrupt" cache if invoke w/o git avail 2018-01-09 08:58:19 -06:00
Eelco Dolstra 44272d8719
Rename "use-substitutes" to "substitute"
Commit c2154d4c84 renamed
"build-use-substitutes" to "use-substitutes", but that broke
"nix-copy-closure --use-substitutes".
2018-01-04 16:58:39 +01:00
Eelco Dolstra f68c2b5a78
Merge pull request #1770 from dtzWill/fix/run-test-sandbox-ubuntu
run.sh: include lib64 in sandbox-paths to fix on ubuntu 16.XX
2018-01-02 11:32:18 +01:00
Will Dietz 5afee18726 run.sh: include lib64 in sandbox-paths to fix on ubuntu 16.XX
(cc #1769)
2017-12-30 22:41:49 -06:00
Will Dietz 9dd2b8ac7b use libbrotli directly when available
* Look for both 'brotli' and 'bro' as external command,
  since upstream has renamed it in newer versions.
  If neither are found, current runtime behavior
  is preserved: try to find 'bro' on PATH.
* Limit amount handed to BrotliEncoderCompressStream
  to ensure interrupts are processed in a timely manner.
  Testing shows negligible performance impact.
  (Other compression sinks don't seem to require this)
2017-12-30 20:26:33 -06:00