Commit graph

339 commits

Author SHA1 Message Date
Eelco Dolstra fa738e50bc
Revert "builtins.readFile: Put the references of the file, not those needed to realize the file, into the context"
Reverting commit 451c223dee for now
because it breaks http://hydra.nixos.org/build/46805136, not clear
why.
2017-01-24 15:29:46 +01:00
Shea Levy 451c223dee builtins.readFile: Put the references of the file, not those needed to realize the file, into the context 2017-01-10 12:22:22 -05:00
Eelco Dolstra 215b70f51e
Revert "Get rid of unicode quotes (#1140)"
This reverts commit f78126bfd6. There
really is no need for such a massive change...
2016-11-26 00:38:01 +01:00
Guillaume Maudoux f78126bfd6 Get rid of unicode quotes (#1140) 2016-11-25 15:48:27 +01:00
Tuomas Tynkkynen f0fc3dd88b Fix SIGFPE from integer overflow during division
On some architectures (like x86_64 or i686, but not ARM for example)
overflow during integer division causes a crash due to SIGFPE.
Reproduces on a 64-bit system with:

    nix-instantiate --eval -E '(-9223372036854775807 - 1) / -1'

The only way this can happen is when the smallest possible integer is
divided by -1, so just special-case that.
2016-10-26 18:41:59 +03:00
Alexander Ried c935e8eeaf fixup! replace own regex class with std::regex 2016-10-19 16:03:53 +02:00
Alexander Ried b05b98df75 replace own regex class with std::regex 2016-10-18 20:22:25 +02:00
Eelco Dolstra c55bf085eb printMsg(lvlError, ...) -> printError(...) etc. 2016-09-21 16:54:53 +02:00
Eelco Dolstra 90ad02bf62 Enable HTTP/2 support
The binary cache store can now use HTTP/2 to do lookups. This is much
more efficient than HTTP/1.1 due to multiplexing: we can issue many
requests in parallel over a single TCP connection. Thus it's no longer
necessary to use a bunch of concurrent TCP connections (25 by
default).

For example, downloading 802 .narinfo files from
https://cache.nixos.org/, using a single TCP connection, takes 11.8s
with HTTP/1.1, but only 0.61s with HTTP/2.

This did require a fairly substantial rewrite of the Downloader class
to use the curl multi interface, because otherwise curl wouldn't be
able to do multiplexing for us. As a bonus, we get connection reuse
even with HTTP/1.1. All downloads are now handled by a single worker
thread. Clients call Downloader::enqueueDownload() to tell the worker
thread to start the download, getting a std::future to the result.
2016-09-14 16:36:02 +02:00
Eelco Dolstra c42d1acfeb forceBool(): Show position info 2016-08-29 19:37:19 +02:00
Eelco Dolstra 26d92017d3 Add builtin function "partition"
The implementation of "partition" in Nixpkgs is O(n^2) (because of the
use of ++), and for some reason was causing stack overflows in
multi-threaded evaluation (not sure why).

This reduces "nix-env -qa --drv-path" runtime by 0.197s and memory
usage by 298 MiB (in non-Boehm mode).
2016-08-29 19:36:54 +02:00
Eelco Dolstra 22d6e31fc6 Add a mechanism for derivation attributes to reference the derivation's outputs
For example, you can now say:

  configureFlags = "--prefix=${placeholder "out"} --includedir=${placeholder "dev"}";

The strings returned by the ‘placeholder’ builtin are replaced at
build time by the actual store paths corresponding to the specified
outputs.

Previously, you had to work around the inability to self-reference by doing stuff like:

  preConfigure = ''
    configureFlags+=" --prefix $out --includedir=$dev"
  '';

or rely on ad-hoc variable interpolation semantics in Autoconf or Make
(e.g. --prefix=\$(out)), which doesn't always work.
2016-08-17 17:19:32 +02:00
Eelco Dolstra ac841a4679 Merge pull request #1031 from abbradar/replacestrings-context
Allow contexted strings in replaceStrings
2016-08-16 12:47:20 +02:00
Nikolay Amiantov f1b8dd43be Allow contexted strings in replaceStrings 2016-08-16 13:20:56 +03:00
Shea Levy d52d391164 builtins.fetch{url,tarball}: Allow name attribute 2016-08-15 07:37:11 -04:00
Eelco Dolstra ee22a91ab8 makeFixedOutputPath(): Drop superfluous HashType argument 2016-07-26 21:25:52 +02:00
Eelco Dolstra 06bbfb6004 builtins.{fetchurl,fetchTarball}: Support a sha256 attribute
Also, allow builtins.{fetchurl,fetchTarball} in restricted mode if a
hash is specified.
2016-07-26 21:16:52 +02:00
Eelco Dolstra 7850d3d279 Make the store directory a member variable of Store 2016-06-01 16:24:17 +02:00
Eelco Dolstra 6c75cf69c3 Cleanup: Remove singleton() 2016-05-04 16:16:53 +02:00
Eelco Dolstra 41633f9f73 Improved logging abstraction
This also gets rid of --log-type, since the nested log type isn't
useful in a multi-threaded situation, and nobody cares about the
"pretty" log type.
2016-04-25 19:18:45 +02:00
Eelco Dolstra 363f37d084 Make the search path lazier with non-fatal errors
Thus, -I / $NIX_PATH entries are now downloaded only when they are
needed for evaluation. An error to download an entry is a non-fatal
warning (just like non-existant paths).

This does change the semantics of builtins.nixPath, which now returns
the original, rather than resulting path. E.g., before we had

  [ { path = "/nix/store/hgm3yxf1lrrwa3z14zpqaj5p9vs0qklk-nixexprs.tar.xz"; prefix = "nixpkgs"; } ... ]

but now

  [ { path = "https://nixos.org/channels/nixos-16.03/nixexprs.tar.xz"; prefix = "nixpkgs"; } ... ]

Fixes #792.
2016-04-14 15:32:24 +02:00
Eelco Dolstra 12b257f045 Make primop registration pluggable
This way we don't have to put all primops in one giant file.
2016-04-14 12:50:01 +02:00
Nikolay Amiantov f7f0116dd7 Propagate path context via builtins.readFile 2016-03-04 14:04:51 +03:00
Eelco Dolstra e9c50064b5 Add an HTTP binary cache store
Allowing stuff like

  NIX_REMOTE=https://cache.nixos.org nix-store -qR /nix/store/x1p1gl3a4kkz5ci0nfbayjqlqmczp1kq-geeqie-1.1

or

  NIX_REMOTE=https://cache.nixos.org nix-store --export /nix/store/x1p1gl3a4kkz5ci0nfbayjqlqmczp1kq-geeqie-1.1 | nix-store --import
2016-02-29 18:15:20 +01:00
Eelco Dolstra b3e8d72770 Merge pull request #762 from ctheune/ctheune-floats
Implement floats
2016-02-12 12:49:59 +01:00
Eelco Dolstra c10c61449f Eliminate the "store" global variable
Also, move a few free-standing functions into StoreAPI and Derivation.

Also, introduce a non-nullable smart pointer, ref<T>, which is just a
wrapper around std::shared_ptr ensuring that the pointer is never
null. (For reference-counted values, this is better than passing a
"T&", because the latter doesn't maintain the refcount. Usually, the
caller will have a shared_ptr keeping the value alive, but that's not
always the case, e.g., when passing a reference to a std::thread via
std::bind.)
2016-02-04 14:28:26 +01:00
Eelco Dolstra 9aac1861f7 Fix some signedness warnings 2016-01-07 14:37:39 +01:00
Fabian Schmitthenner 39d1da7b51 Better error message
Also show types when nix cannot compare values of different types.
This is also more consistent since types are already shown when comparing values of the same not comparable type.
2016-01-05 14:13:54 +01:00
Vladimír Čunát 29c0199d5c manual: document builtins.functionArgs
The text is just a conversion of comment from src/libexpr/primops.cc
2016-01-05 13:29:17 +01:00
Christian Theune 14ebde5289 First hit at providing support for floats in the language. 2016-01-05 00:40:40 +01:00
Eelco Dolstra 76cc8e97a2 Add sort primop 2015-07-28 18:39:39 +02:00
Eelco Dolstra 50807f3dd5 Add primop genList
This can be used to implement functions like ‘imap’ (or for that
matter, ‘map’) without the quadratic complexity incurred by calling
‘++’ repeatedly.
2015-07-28 17:28:35 +02:00
Eelco Dolstra d6d5885c15 Add replaceStrings primop
This is a generalisation of replaceChars in Nixpkgs.
2015-07-24 15:32:24 +02:00
Eelco Dolstra 2e8fd4c5cd Add concatStringsSep as a primop
This fixes the quadratic behaviour of concatStrings/concatStringsSep
in Nixpkgs.
2015-07-24 02:38:09 +02:00
Eelco Dolstra b83801f8b3 Optimize small lists
The value pointers of lists with 1 or 2 elements are now stored in the
list value itself. In particular, this makes the "concatMap (x: if
cond then [(f x)] else [])" idiom cheaper.
2015-07-23 22:05:09 +02:00
Eelco Dolstra 14be783676 Add primops all and any
These are used thousands of times during NixOS evaluation, so it's
useful to speed them up.
2015-07-23 19:23:11 +02:00
Shea Levy 39e27a04b8 Importing derivations: Add name attribute to make a valid drv 2015-07-23 17:04:07 +02:00
Shea Levy 1ed55234d9 Allow derivations-as-srcs in the context of builtins.toFile files 2015-07-23 17:03:47 +02:00
Eelco Dolstra 61af14a921 Add foldl' primop 2015-07-23 17:03:02 +02:00
Eelco Dolstra 6bd2c7bb38 OCD: foreach -> C++11 ranged for 2015-07-17 20:13:56 +02:00
Eelco Dolstra c1323b53e3 Fix fetchurl/fetchTarball 2015-06-01 15:08:09 +02:00
Eelco Dolstra 1c88e100e7 readFile: Check against nul bytes 2015-06-01 15:07:42 +02:00
Eelco Dolstra 9451ef3731 Allow URLs in the Nix search path
E.g. to install "hello" from the latest Nixpkgs:

  $ nix-build '<nixpkgs>' -A hello -I nixpkgs=https://nixos.org/channels/nixpkgs-unstable/nixexprs.tar.xz

Or to install a specific version of NixOS:

  $ nixos-rebuild switch -I nixpkgs=63def04891.tar.gz
2015-05-05 17:09:42 +02:00
Eelco Dolstra 4ed2187377 Use cached result if there is a network error 2015-04-09 12:49:13 +02:00
Eelco Dolstra 1fc905ad4c Move curl stuff into a separate file 2015-04-09 12:12:50 +02:00
Eelco Dolstra c1f04fae35 Implement a TTL on cached fetchurl/fetchTarball results
This is because we don't want to do HTTP requests on every evaluation,
even though we can prevent a full redownload via the cached ETag. The
default is one hour.
2015-04-09 11:55:36 +02:00
Eelco Dolstra 60340ce3e2 Implement caching of fetchurl/fetchTarball results
ETags are used to prevent redownloading unchanged files.
2015-04-09 11:42:04 +02:00
Eelco Dolstra 000b5a000f Add fetchTarball builtin
This function downloads and unpacks the given URL at evaluation
time. This is primarily intended to make it easier to deal with Nix
expressions that have external dependencies. For instance, to fetch
Nixpkgs 14.12:

  with import (fetchTarball https://github.com/NixOS/nixpkgs-channels/archive/nixos-14.12.tar.gz) {};

Or to fetch a specific revision:

  with import (fetchTarball 2766a4b44e.tar.gz) {};

This patch also adds a ‘fetchurl’ builtin that downloads but doesn't
unpack its argument. Not sure if it's useful though.
2015-03-25 17:29:09 +01:00
Eelco Dolstra 7ea6ecf855 addToStore(): Take explicit name argument 2015-03-25 17:06:12 +01:00
Eelco Dolstra 6f0c6e20e0 Don't rely on __noChroot for corepkgs
This doesn't work anymore if the "strict" chroot mode is
enabled. Instead, add Nix's store path as a dependency. This ensures
that its closure is present in the chroot.
2015-03-24 11:15:45 +01:00