Commit graph

582 commits

Author SHA1 Message Date
Eelco Dolstra c55bf085eb printMsg(lvlError, ...) -> printError(...) etc. 2016-09-21 16:54:53 +02:00
Eelco Dolstra 90ad02bf62 Enable HTTP/2 support
The binary cache store can now use HTTP/2 to do lookups. This is much
more efficient than HTTP/1.1 due to multiplexing: we can issue many
requests in parallel over a single TCP connection. Thus it's no longer
necessary to use a bunch of concurrent TCP connections (25 by
default).

For example, downloading 802 .narinfo files from
https://cache.nixos.org/, using a single TCP connection, takes 11.8s
with HTTP/1.1, but only 0.61s with HTTP/2.

This did require a fairly substantial rewrite of the Downloader class
to use the curl multi interface, because otherwise curl wouldn't be
able to do multiplexing for us. As a bonus, we get connection reuse
even with HTTP/1.1. All downloads are now handled by a single worker
thread. Clients call Downloader::enqueueDownload() to tell the worker
thread to start the download, getting a std::future to the result.
2016-09-14 16:36:02 +02:00
Eelco Dolstra c42d1acfeb forceBool(): Show position info 2016-08-29 19:37:19 +02:00
Eelco Dolstra 26d92017d3 Add builtin function "partition"
The implementation of "partition" in Nixpkgs is O(n^2) (because of the
use of ++), and for some reason was causing stack overflows in
multi-threaded evaluation (not sure why).

This reduces "nix-env -qa --drv-path" runtime by 0.197s and memory
usage by 298 MiB (in non-Boehm mode).
2016-08-29 19:36:54 +02:00
Eelco Dolstra 22d6e31fc6 Add a mechanism for derivation attributes to reference the derivation's outputs
For example, you can now say:

  configureFlags = "--prefix=${placeholder "out"} --includedir=${placeholder "dev"}";

The strings returned by the ‘placeholder’ builtin are replaced at
build time by the actual store paths corresponding to the specified
outputs.

Previously, you had to work around the inability to self-reference by doing stuff like:

  preConfigure = ''
    configureFlags+=" --prefix $out --includedir=$dev"
  '';

or rely on ad-hoc variable interpolation semantics in Autoconf or Make
(e.g. --prefix=\$(out)), which doesn't always work.
2016-08-17 17:19:32 +02:00
Eelco Dolstra ac841a4679 Merge pull request #1031 from abbradar/replacestrings-context
Allow contexted strings in replaceStrings
2016-08-16 12:47:20 +02:00
Nikolay Amiantov f1b8dd43be Allow contexted strings in replaceStrings 2016-08-16 13:20:56 +03:00
Shea Levy d52d391164 builtins.fetch{url,tarball}: Allow name attribute 2016-08-15 07:37:11 -04:00
Eelco Dolstra ee22a91ab8 makeFixedOutputPath(): Drop superfluous HashType argument 2016-07-26 21:25:52 +02:00
Eelco Dolstra 06bbfb6004 builtins.{fetchurl,fetchTarball}: Support a sha256 attribute
Also, allow builtins.{fetchurl,fetchTarball} in restricted mode if a
hash is specified.
2016-07-26 21:16:52 +02:00
Eelco Dolstra 7850d3d279 Make the store directory a member variable of Store 2016-06-01 16:24:17 +02:00
Eelco Dolstra 6c75cf69c3 Cleanup: Remove singleton() 2016-05-04 16:16:53 +02:00
Eelco Dolstra 41633f9f73 Improved logging abstraction
This also gets rid of --log-type, since the nested log type isn't
useful in a multi-threaded situation, and nobody cares about the
"pretty" log type.
2016-04-25 19:18:45 +02:00
Eelco Dolstra 363f37d084 Make the search path lazier with non-fatal errors
Thus, -I / $NIX_PATH entries are now downloaded only when they are
needed for evaluation. An error to download an entry is a non-fatal
warning (just like non-existant paths).

This does change the semantics of builtins.nixPath, which now returns
the original, rather than resulting path. E.g., before we had

  [ { path = "/nix/store/hgm3yxf1lrrwa3z14zpqaj5p9vs0qklk-nixexprs.tar.xz"; prefix = "nixpkgs"; } ... ]

but now

  [ { path = "https://nixos.org/channels/nixos-16.03/nixexprs.tar.xz"; prefix = "nixpkgs"; } ... ]

Fixes #792.
2016-04-14 15:32:24 +02:00
Eelco Dolstra 12b257f045 Make primop registration pluggable
This way we don't have to put all primops in one giant file.
2016-04-14 12:50:01 +02:00
Nikolay Amiantov f7f0116dd7 Propagate path context via builtins.readFile 2016-03-04 14:04:51 +03:00
Eelco Dolstra e9c50064b5 Add an HTTP binary cache store
Allowing stuff like

  NIX_REMOTE=https://cache.nixos.org nix-store -qR /nix/store/x1p1gl3a4kkz5ci0nfbayjqlqmczp1kq-geeqie-1.1

or

  NIX_REMOTE=https://cache.nixos.org nix-store --export /nix/store/x1p1gl3a4kkz5ci0nfbayjqlqmczp1kq-geeqie-1.1 | nix-store --import
2016-02-29 18:15:20 +01:00
Eelco Dolstra b3e8d72770 Merge pull request #762 from ctheune/ctheune-floats
Implement floats
2016-02-12 12:49:59 +01:00
Eelco Dolstra c10c61449f Eliminate the "store" global variable
Also, move a few free-standing functions into StoreAPI and Derivation.

Also, introduce a non-nullable smart pointer, ref<T>, which is just a
wrapper around std::shared_ptr ensuring that the pointer is never
null. (For reference-counted values, this is better than passing a
"T&", because the latter doesn't maintain the refcount. Usually, the
caller will have a shared_ptr keeping the value alive, but that's not
always the case, e.g., when passing a reference to a std::thread via
std::bind.)
2016-02-04 14:28:26 +01:00
Eelco Dolstra 9aac1861f7 Fix some signedness warnings 2016-01-07 14:37:39 +01:00
Fabian Schmitthenner 39d1da7b51 Better error message
Also show types when nix cannot compare values of different types.
This is also more consistent since types are already shown when comparing values of the same not comparable type.
2016-01-05 14:13:54 +01:00
Vladimír Čunát 29c0199d5c manual: document builtins.functionArgs
The text is just a conversion of comment from src/libexpr/primops.cc
2016-01-05 13:29:17 +01:00
Christian Theune 14ebde5289 First hit at providing support for floats in the language. 2016-01-05 00:40:40 +01:00
Eelco Dolstra 76cc8e97a2 Add sort primop 2015-07-28 18:39:39 +02:00
Eelco Dolstra 50807f3dd5 Add primop genList
This can be used to implement functions like ‘imap’ (or for that
matter, ‘map’) without the quadratic complexity incurred by calling
‘++’ repeatedly.
2015-07-28 17:28:35 +02:00
Eelco Dolstra d6d5885c15 Add replaceStrings primop
This is a generalisation of replaceChars in Nixpkgs.
2015-07-24 15:32:24 +02:00
Eelco Dolstra 2e8fd4c5cd Add concatStringsSep as a primop
This fixes the quadratic behaviour of concatStrings/concatStringsSep
in Nixpkgs.
2015-07-24 02:38:09 +02:00
Eelco Dolstra b83801f8b3 Optimize small lists
The value pointers of lists with 1 or 2 elements are now stored in the
list value itself. In particular, this makes the "concatMap (x: if
cond then [(f x)] else [])" idiom cheaper.
2015-07-23 22:05:09 +02:00
Eelco Dolstra 14be783676 Add primops all and any
These are used thousands of times during NixOS evaluation, so it's
useful to speed them up.
2015-07-23 19:23:11 +02:00
Shea Levy 39e27a04b8 Importing derivations: Add name attribute to make a valid drv 2015-07-23 17:04:07 +02:00
Shea Levy 1ed55234d9 Allow derivations-as-srcs in the context of builtins.toFile files 2015-07-23 17:03:47 +02:00
Eelco Dolstra 61af14a921 Add foldl' primop 2015-07-23 17:03:02 +02:00
Eelco Dolstra 6bd2c7bb38 OCD: foreach -> C++11 ranged for 2015-07-17 20:13:56 +02:00
Eelco Dolstra c1323b53e3 Fix fetchurl/fetchTarball 2015-06-01 15:08:09 +02:00
Eelco Dolstra 1c88e100e7 readFile: Check against nul bytes 2015-06-01 15:07:42 +02:00
Eelco Dolstra 9451ef3731 Allow URLs in the Nix search path
E.g. to install "hello" from the latest Nixpkgs:

  $ nix-build '<nixpkgs>' -A hello -I nixpkgs=https://nixos.org/channels/nixpkgs-unstable/nixexprs.tar.xz

Or to install a specific version of NixOS:

  $ nixos-rebuild switch -I nixpkgs=63def04891.tar.gz
2015-05-05 17:09:42 +02:00
Eelco Dolstra 4ed2187377 Use cached result if there is a network error 2015-04-09 12:49:13 +02:00
Eelco Dolstra 1fc905ad4c Move curl stuff into a separate file 2015-04-09 12:12:50 +02:00
Eelco Dolstra c1f04fae35 Implement a TTL on cached fetchurl/fetchTarball results
This is because we don't want to do HTTP requests on every evaluation,
even though we can prevent a full redownload via the cached ETag. The
default is one hour.
2015-04-09 11:55:36 +02:00
Eelco Dolstra 60340ce3e2 Implement caching of fetchurl/fetchTarball results
ETags are used to prevent redownloading unchanged files.
2015-04-09 11:42:04 +02:00
Eelco Dolstra 000b5a000f Add fetchTarball builtin
This function downloads and unpacks the given URL at evaluation
time. This is primarily intended to make it easier to deal with Nix
expressions that have external dependencies. For instance, to fetch
Nixpkgs 14.12:

  with import (fetchTarball https://github.com/NixOS/nixpkgs-channels/archive/nixos-14.12.tar.gz) {};

Or to fetch a specific revision:

  with import (fetchTarball 2766a4b44e.tar.gz) {};

This patch also adds a ‘fetchurl’ builtin that downloads but doesn't
unpack its argument. Not sure if it's useful though.
2015-03-25 17:29:09 +01:00
Eelco Dolstra 7ea6ecf855 addToStore(): Take explicit name argument 2015-03-25 17:06:12 +01:00
Eelco Dolstra 6f0c6e20e0 Don't rely on __noChroot for corepkgs
This doesn't work anymore if the "strict" chroot mode is
enabled. Instead, add Nix's store path as a dependency. This ensures
that its closure is present in the chroot.
2015-03-24 11:15:45 +01:00
Daniel Hahler e659978ced Fix typos: s/the the/the/ 2015-03-06 16:43:22 +01:00
Eelco Dolstra 15d2d3c34e Add restricted evaluation mode
If ‘--option restrict-eval true’ is given, the evaluator will throw an
exception if an attempt is made to access any file outside of the Nix
search path. This is primarily intended for Hydra, where we don't want
people doing ‘builtins.readFile ~/.ssh/id_dsa’ or stuff like that.
2015-02-23 15:54:31 +01:00
Eelco Dolstra 2be7f79fd4 Remove tab 2015-02-05 17:21:30 +01:00
Shea Levy 73bf32ce94 Merge remote-tracking branch 'shlevy/baseNameOf-no-copy'
baseNameOf: Don't copy paths to the store first
2015-01-29 03:29:09 -05:00
Eelco Dolstra 2a3b1df423 Fix builtins.readDir on XFS
The DT_UNKNOWN fallback code was getting the type of the wrong path,
causing readDir to report "directory" as the type of every file.

Reported by deepfire on IRC.
2015-01-09 14:56:25 +01:00
Eelco Dolstra 153a943de7 Show position info for failing <...> lookups 2015-01-07 13:43:55 +01:00
Eelco Dolstra 8aedaf111e Remove canary stuff 2014-12-12 10:59:50 +01:00
Shea Levy 50c3352811 builtins.readFile: realise context associated with the path 2014-12-10 12:26:53 +01:00
Shea Levy 320659b0cd Allow external code using libnixexpr to add types
Code that links to libnixexpr (e.g. plugins loaded with importNative, or
nix-exec) may want to provide custom value types and operations on
values of those types. For example, nix-exec is currently using sets
where a custom IO value type would be more appropriate. This commit
provides a generic hook for such types in the form of tExternal and the
ExternalBase virtual class, which contains all functions necessary for
libnixexpr's type-polymorphic functions (e.g. `showType`) to be
implemented.
2014-12-02 10:27:04 -05:00
Eelco Dolstra 976df480c9 Add a primop for regular expression pattern matching
The function ‘builtins.match’ takes a POSIX extended regular
expression and an arbitrary string. It returns ‘null’ if the string
does not match the regular expression. Otherwise, it returns a list
containing substring matches corresponding to parenthesis groups in
the regex. The regex must match the entire string (i.e. there is an
implied "^<pat>$" around the regex).  For example:

  match "foo" "foobar" => null
  match "foo" "foo" => []
  match "f(o+)(.*)" "foooobar" => ["oooo" "bar"]
  match "(.*/)?([^/]*)" "/dir/file.nix" => ["/dir/" "file.nix"]
  match "(.*/)?([^/]*)" "file.nix" => [null "file.nix"]

The following example finds all regular files with extension .nix or
.patch underneath the current directory:

  let

    findFiles = pat: dir: concatLists (mapAttrsToList (name: type:
      if type == "directory" then
        findFiles pat (dir + "/" + name)
      else if type == "regular" && match pat name != null then
        [(dir + "/" + name)]
      else []) (readDir dir));

  in findFiles ".*\\.(nix|patch)" (toString ./.)
2014-11-25 11:47:06 +01:00
Eelco Dolstra 4e340a983f forceString(): Accept pos argument 2014-11-25 10:23:36 +01:00
Shea Levy b0c5c2ac34 import derivation: cleanup
Before this there was a bug where a `find` was being called on a
not-yet-sorted set. The code was just a mess before anyway, so I cleaned
it up while fixing it.
2014-11-20 22:48:12 -05:00
Shea Levy 2719627bbe realiseContext: Handle all context types
Avoids an assertion
2014-11-15 21:43:51 -05:00
Shea Levy 0ee1ca628a baseNameOf: Don't copy paths to the store first 2014-10-18 20:28:28 -04:00
Shea Levy d16e3c7f09 Export realiseContext in libnixexpr
Useful for importNative plugins
2014-10-17 22:15:09 -04:00
Eelco Dolstra c3f0a489f9 Add primop ‘catAttrs’ 2014-10-04 18:15:03 +02:00
Eelco Dolstra d4fcbe1687 Add primop ‘attrValues’ 2014-10-04 16:41:24 +02:00
Eelco Dolstra 58d8a213b0 Tweak 2014-10-04 11:27:23 +02:00
Eelco Dolstra 3f8576a6ab Remove some duplicate code 2014-10-03 22:37:51 +02:00
Shea Levy c08c802bf3 Add readDir primop 2014-10-03 22:32:11 +02:00
Eelco Dolstra ebb1dbb3e1 Add missing static 2014-09-23 15:08:27 +02:00
Eelco Dolstra 53b044c2f6 Don't evaluate inside a "throw"
Workaround for
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=41174. This caused
hydra-eval-jobs to ignore SIGINT.
2014-09-22 19:18:05 +02:00
Eelco Dolstra 0cd6596b0e Add ‘deepSeq’ primop
Note that unlike ‘lib.deepSeq’ in Nixpkgs, this handles cycles.
2014-09-22 16:05:00 +02:00
Eelco Dolstra a54c263402 Add ‘seq’ primop 2014-09-22 16:05:00 +02:00
Eelco Dolstra eff120d1b9 Add a function ‘valueSize’
It returns the size of value, including all other values and
environments reachable from it. It is intended for debugging memory
consumption issues.
2014-09-22 16:05:00 +02:00
Eelco Dolstra 2d6cd8aafd attrNames: Don't allocate duplicates of the symbols 2014-09-19 18:11:46 +02:00
Eelco Dolstra ea525a261f Fix off-by-one 2014-09-19 18:08:14 +02:00
Eelco Dolstra 6e5b02bee4 Add some instrumentation for debugging GC leaks 2014-09-17 15:19:07 +02:00
Eelco Dolstra 11849a320e Use proper quotes everywhere 2014-08-20 18:03:48 +02:00
Eelco Dolstra 3d221a7bb1 Rename nixPath to __nixPath
The name ‘nixPath’ breaks existing code.
2014-07-30 11:28:39 +02:00
Eelco Dolstra 0e5d0c1543 Fix compilation error on some versions of GCC
src/libexpr/primops.cc:42:8: error: looser throw specifier for 'virtual nix::InvalidPathError::~InvalidPathError()'
src/libexpr/nixexpr.hh:12:1: error:   overriding 'virtual nix::EvalError::~EvalError() noexcept (true)'

http://hydra.nixos.org/build/12385750
2014-07-09 12:14:40 +02:00
Eelco Dolstra beaf3e90af Add builtin function ‘fromJSON’
Fixes #294.
2014-07-04 13:34:15 +02:00
Shea Levy d62f46e500 Only add the importNative primop if the allow-arbitrary-code-during-evaluation option is true (default false) 2014-06-24 10:50:03 -04:00
Shea Levy 5cd022d6c0 Add importNative primop
This can be used to import a dynamic shared object and return an
arbitrary value, including new primops. This can be used both to test
new primops without having to recompile nix every time, and to build
specialized primops that probably don't belong upstream (e.g. a function
that calls out to gpg to decrypt a nixops secret as-needed).

The imported function should initialize the Value & as needed. A single
import can define multiple values by creating an attrset or list, of
course.

An example initialization function might look like:

extern "C" void initialize(nix::EvalState & state, nix::Value & v)
{
    v.type = nix::tPrimOp;
    v.primOp = NEW nix::PrimOp(myFun, 1, state.symbols.create("myFun"));
}

Then `builtins.importNative ./example.so "initialize"` will evaluate to
the primop defined in the myFun function.
2014-06-17 12:08:01 -04:00
Eelco Dolstra 0960d674d4 Drop ImportError and FindError
We're not catching these anywhere.
2014-06-12 13:00:54 +02:00
Shea Levy 718f20da6d findFile: Realise the context of the path attributes 2014-06-12 12:57:14 +02:00
Shea Levy a8fb575c98 Share code between scopedImport and import
In addition to reducing duplication, this fixes both import from
derivation and import of derivation for scopedImport
2014-06-12 12:52:39 +02:00
Eelco Dolstra ee7fe64c0a == operator: Ignore string context
There really is no case I can think of where taking the context into
account is useful. Mostly it's just very inconvenient.
2014-06-10 14:02:56 +02:00
Eelco Dolstra becc2b0167 Sort nixPath attributes 2014-05-29 19:02:14 +02:00
Eelco Dolstra 62a6eeb1f3 Make the Nix search path declarative
Nix search path lookups like <nixpkgs> are now desugared to ‘findFile
nixPath <nixpkgs>’, where ‘findFile’ is a new primop. Thus you can
override the search path simply by saying

  let
    nixPath = [ { prefix = "nixpkgs"; path = "/my-nixpkgs"; } ];
  in ... <nixpkgs> ...

In conjunction with ‘scopedImport’ (commit
c273c15cb1), the Nix search path can be
propagated across imports, e.g.

  let

    overrides = {
      nixPath = [ ... ] ++ builtins.nixPath;
      import = fn: scopedImport overrides fn;
      scopedImport = attrs: fn: scopedImport (overrides // attrs) fn;
      builtins = builtins // overrides;
    };

  in scopedImport overrides ./nixos
2014-05-26 17:02:22 +02:00
Eelco Dolstra a8edf185a9 Add constant ‘nixPath’
It contains the Nix expression search path as a list of { prefix, path
} sets, e.g.

  [ { path = "/nix/var/nix/profiles/per-user/root/channels/nixos"; prefix = ""; }
    { path = "/etc/nixos/configuration.nix"; prefix = "nixos-config"; }
    { path = "/home/eelco/Dev/nix/inst/share/nix/corepkgs"; prefix = "nix"; }
  ]
2014-05-26 14:55:47 +02:00
Eelco Dolstra c273c15cb1 Add primop ‘scopedImport’
‘scopedImport’ works like ‘import’, except that it takes a set of
attributes to be added to the lexical scope of the expression,
essentially extending or overriding the builtin variables.  For
instance, the expression

  scopedImport { x = 1; } ./foo.nix

where foo.nix contains ‘x’, will evaluate to 1.

This has a few applications:

* It allows getting rid of function argument specifications in package
  expressions. For instance, a package expression like:

    { stdenv, fetchurl, libfoo }:

    stdenv.mkDerivation { ... buildInputs = [ libfoo ]; }

  can now we written as just

    stdenv.mkDerivation { ... buildInputs = [ libfoo ]; }

  and imported in all-packages.nix as:

    bar = scopedImport pkgs ./bar.nix;

  So whereas we once had dependencies listed in three places
  (buildInputs, the function, and the call site), they now only need
  to appear in one place.

* It allows overriding builtin functions. For instance, to trace all
  calls to ‘map’:

  let
    overrides = {
      map = f: xs: builtins.trace "map called!" (map f xs);

      # Ensure that our override gets propagated by calls to
      # import/scopedImport.
      import = fn: scopedImport overrides fn;

      scopedImport = attrs: fn: scopedImport (overrides // attrs) fn;

      # Also update ‘builtins’.
      builtins = builtins // overrides;
    };
  in scopedImport overrides ./bla.nix

* Similarly, it allows extending the set of builtin functions. For
  instance, during Nixpkgs/NixOS evaluation, the Nixpkgs library
  functions could be added to the default scope.

There is a downside: calls to scopedImport are not memoized, unlike
import. So importing a file multiple times leads to multiple parsings
/ evaluations. It would be possible to construct the AST only once,
but that would require careful handling of variables/environments.
2014-05-26 14:26:29 +02:00
Eelco Dolstra f0fdbd0897 Shut up some signedness warnings 2014-05-26 12:34:15 +02:00
Eelco Dolstra dfa2f77d2e If a .drv cannot be parsed, show its path
Otherwise you just get ‘expected string `Derive(['’ which isn't very helpful.
2014-04-08 19:24:29 +02:00
Eelco Dolstra 4c5faad994 Show position info in Boolean operations 2014-04-04 22:43:52 +02:00
Eelco Dolstra bd9b1d97b4 Show position info in string concatenation / addition errors 2014-04-04 22:19:33 +02:00
Eelco Dolstra a5fe730940 forceString: Show position info 2014-04-04 21:14:11 +02:00
Eelco Dolstra 27b44b8cf7 forceAttrs: Show position info 2014-04-04 19:11:40 +02:00
Eelco Dolstra 96b695ccab forceList: Show position info 2014-04-04 19:05:36 +02:00
Eelco Dolstra b62d36963c forceInt: Show position info 2014-04-04 18:59:29 +02:00
Eelco Dolstra c28de6d96e Pass position information to primop calls
For example:

  error: `tail' called on an empty list, at
    /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/ex-2/default.nix:13:7
2014-04-04 18:59:29 +02:00
Eelco Dolstra b72c8d2e5b Include position info in function application
This allows error messages like:

  error: the anonymous function at `/etc/nixos/configuration.nix:1:1'
    called without required argument `foo', at
    `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/lib/modules.nix:77:59'
2014-04-04 18:59:29 +02:00
Eelco Dolstra 81628a6ccc Merge branch 'master' into make
Conflicts:
	src/libexpr/eval.cc
2014-01-21 15:30:01 +01:00
Shea Levy e36229d27f Bump language version for new storePath feature
This will allow e.g. channel expressions to use builtins.storePath IFF
it is safe to do so without knowing if the path is valid yet.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-12-10 15:07:04 +01:00
Shea Levy 22d665019a builtins.storePath: Try to substitute the path if it is not yet valid
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-12-10 15:07:04 +01:00
Eelco Dolstra 611868a909 Implement basic ‘make install’ 2013-11-23 17:04:27 +00:00
Eelco Dolstra 990126cde0 Shorter error message 2013-11-19 14:09:14 +01:00
Eelco Dolstra 77c13cdf56 Add a toJSON primop 2013-11-19 00:04:11 +01:00
Eelco Dolstra 285df765b9 Add a primop unsafeGetAttrPos to return the position of an attribute 2013-11-18 22:22:35 +01:00
Eelco Dolstra 8d6418d46e Fix building without a garbage collector
http://hydra.nixos.org/build/6695350
2013-10-28 22:51:12 +01:00
Eelco Dolstra dec2f19502 Fix a segfault in genericClosure
It kept temporary data in STL containers that were not scanned by
Boehm GC, so Nix programs using genericClosure could randomly crash if
the garbage collector kicked in at a bad time.

Also make it a bit more efficient by copying points to values rather
than values.
2013-10-28 18:52:26 +01:00
Eelco Dolstra ea6bf0c21f Slightly optimize listToAttrs 2013-10-28 07:34:44 +01:00
Eelco Dolstra 5bc41d78ff Rename "attribute sets" to "sets"
We don't have any other kind of sets so calling them attribute sets is
unnecessarily verbose.
2013-10-24 16:41:04 +02:00
Eelco Dolstra 05d02f798f Add a typeOf primop
We already have some primops for determining the type of a value, such
as isString, but they're incomplete: for instance, there is no isPath.
Rather than adding more isBla functions, the generic typeOf function
returns a string representing the type of the argument (e.g. "int").
2013-10-24 02:49:13 +02:00
Eelco Dolstra 792fd51f41 Fold two stack trace messages in derivations
Combined with the previous changes, stack traces involving derivations
are now much less verbose, since something like

  while evaluating the builtin function `getAttr':
  while evaluating the builtin function `derivationStrict':
  while instantiating the derivation named `gtk+-2.24.20' at `/home/eelco/Dev/nixpkgs/pkgs/development/libraries/gtk+/2.x.nix:11:3':
  while evaluating the derivation attribute `propagatedNativeBuildInputs' at `/home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/default.nix:78:17':
  while evaluating the attribute `outPath' at `/nix/store/212ngf4ph63mp6p1np2bapkfikpakfv7-nix-1.6/share/nix/corepkgs/derivation.nix:18:9':
  ...

now reads

  while evaluating the attribute `propagatedNativeBuildInputs' of the derivation `gtk+-2.24.20' at `/home/eelco/Dev/nixpkgs/pkgs/development/libraries/gtk+/2.x.nix:11:3':
  ...
2013-10-17 11:57:25 +02:00
Eelco Dolstra f440558acc Don't show <nix/derivation.nix> in stack traces
Messages like

  while evaluating the attribute `outPath' at `/nix/store/212ngf4ph63mp6p1np2bapkfikpakfv7-nix-1.6/share/nix/corepkgs/derivation.nix:18:9':

are redundant, because Nix already shows that it's evaluating a derivation:

  while instantiating the derivation named `firefox-24.0' at `/home/eelco/Dev/nixpkgs/pkgs/applications/networking/browsers/firefox/default.nix:131:5':
  while evaluating the derivation attribute `nativeBuildInputs' at `/home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/default.nix:76:17':
2013-10-17 11:47:38 +02:00
Eelco Dolstra 6f809194d7 Get rid of the parse tree cache
Since we already cache files in normal form (fileEvalCache), caching
parse trees is redundant.

Note that getting rid of this cache doesn't actually save much memory
at the moment, because parse trees are currently not freed / GC'ed.
2013-09-03 13:01:42 +02:00
Eelco Dolstra 33972629d7 Fix whitespace 2013-09-02 16:29:15 +02:00
Eelco Dolstra d308aeaf53 Store Nix integers as longs
So on 64-bit systems, integers are now 64-bit.

Fixes #158.
2013-08-19 12:35:03 +02:00
Eelco Dolstra 8e74c0bfd1 Let the ordering operators also work on strings
E.g. ‘"foo" < "bar"’ now works.
2013-08-02 18:53:02 +02:00
Eelco Dolstra bdd4646338 Revert "Prevent config.h from being clobbered"
This reverts commit 28bba8c44f.
2013-03-08 01:24:59 +01:00
Eelco Dolstra 28bba8c44f Prevent config.h from being clobbered 2013-03-07 23:55:55 +01:00
Eelco Dolstra 52172607cf Rename "hash" to "hashString" and handle SHA-1 2013-02-08 19:36:23 +01:00
Marc Weber 01a5ea9914 experimental/hash
adding primop function calculating hash of a string

Signed-off-by: Marc Weber <marco-oweber@gmx.de>
2013-02-08 19:26:34 +01:00
Eelco Dolstra 3100b29fc5 Tiny optimisation in the filter primop 2012-12-04 17:22:20 +01:00
Eelco Dolstra 6c98e6a5de Optionally ignore null-valued derivation attributes
This allows adding attributes like

  attr = if stdenv.system == "bla" then something else null;

without changing the resulting derivation on non-<bla> platforms.

We once considered adding a special "ignore" value for this purpose,
but using null seems more elegant.
2012-11-27 15:01:32 +01:00
Eelco Dolstra 8b8ee53bc7 Add builtin constants ‘langVersion’ and ‘nixVersion’
The integer constant ‘langVersion’ denotes the current language
version.  It gets increased every time a language feature is
added/changed/removed.  It's currently 1.

The string constant ‘nixVersion’ contains the current Nix version,
e.g. "1.2pre2980_9de6bc5".
2012-11-27 13:29:55 +01:00
Eelco Dolstra 17dc306aa3 Revert "prim_toPath: Actually make the string a path"
This reverts commit 2980d1fba9.  It
causes a regression in NixOS evaluation:

string `/nix/store/ya3s5gmj3b28170fpbjhgsk8wzymkpa1-pommed-1.39/etc/pommed.conf' cannot refer to other paths
2012-11-15 23:04:34 +01:00
Shea Levy 2980d1fba9 prim_toPath: Actually make the string a path 2012-10-23 16:39:35 +02:00
Eelco Dolstra 0a7084567f Add a ‘--repair’ flag to nix-instantiate
This allows repairing corrupted derivations and other source files.
2012-10-03 15:09:18 -04:00
Eelco Dolstra 76e88871b2 Templatise tokenizeString() 2012-09-19 15:43:23 -04:00
Eelco Dolstra e94806d030 Merge branch 'master' into no-manifests 2012-08-27 11:09:07 -04:00
Eelco Dolstra 9c2d63084b Add a primop ‘elemAt’ to get an element from a list 2012-08-13 13:46:42 -04:00
Eelco Dolstra 198d0338be Add a primop ‘concatLists’
This can serve as a generic efficient list builder.  For instance, the
function ‘catAttrs’ in Nixpkgs can be rewritten from

  attr: l: fold (s: l: if hasAttr attr s then [(getAttr attr s)] ++ l else l) [] l

to

  attr: l: builtins.concatLists (map (s: if hasAttr attr s then [(getAttr attr s)] else []) l)

Statistics before:

  time elapsed: 1.08683
  size of a value: 24
  environments allocated: 1384376 (35809568 bytes)
  list elements: 6946783 (55574264 bytes)
  list concatenations: 37434
  values allocated: 1760440 (42250560 bytes)
  attribute sets allocated: 392040
  right-biased unions: 186334
  values copied in right-biased unions: 591137
  symbols in symbol table: 18273
  number of thunks: 1297673
  number of thunks avoided: 1380759
  number of attr lookups: 430802
  number of primop calls: 628912
  number of function calls: 1333544

Statistics after (including new catAttrs):

  time elapsed: 0.959854
  size of a value: 24
  environments allocated: 1010198 (26829296 bytes)
  list elements: 1984878 (15879024 bytes)
  list concatenations: 30488
  values allocated: 1589760 (38154240 bytes)
  attribute sets allocated: 392040
  right-biased unions: 186334
  values copied in right-biased unions: 591137
  symbols in symbol table: 18274
  number of thunks: 1040925
  number of thunks avoided: 1038428
  number of attr lookups: 438419
  number of primop calls: 474844
  number of function calls: 959366
2012-08-13 01:53:10 -04:00
Eelco Dolstra b9e5b908ed Provide an efficient implementation of ‘elem’
The one in Nixpkgs is O(n^2), this one is O(n).  Big reduction in the
number of list allocations.

Statistics before (on a NixOS system config):

  time elapsed: 1.17982
  size of a value: 24
  environments allocated: 1543334 (39624560 bytes)
  list elements: 9612638 (76901104 bytes)
  list concatenations: 37434
  values allocated: 1854933 (44518392 bytes)
  attribute sets allocated: 392040
  right-biased unions: 186334
  values copied in right-biased unions: 591137
  symbols in symbol table: 18272
  number of thunks: 1392467
  number of thunks avoided: 1507311
  number of attr lookups: 430801
  number of primop calls: 691600
  number of function calls: 1492502

Statistics after:

  time elapsed: 1.08683
  size of a value: 24
  environments allocated: 1384376 (35809568 bytes)
  list elements: 6946783 (55574264 bytes)
  list concatenations: 37434
  values allocated: 1760440 (42250560 bytes)
  attribute sets allocated: 392040
  right-biased unions: 186334
  values copied in right-biased unions: 591137
  symbols in symbol table: 18273
  number of thunks: 1297673
  number of thunks avoided: 1380759
  number of attr lookups: 430802
  number of primop calls: 628912
  number of function calls: 1333544
2012-08-13 01:05:35 -04:00
Eelco Dolstra 4ccd48ce24 Add a "filter" primop
Evaluation of a NixOS configuration spends quite a lot of time in the
"filter" function in Nixpkgs.  As implemented in Nixpkgs, this is a
O(n^2) operation, so it's a good candidate for providing a more
efficient (i.e. primop) implementation.  Using it gives a ~10% speed
increase and a significant reduction in the number of evaluations.

Statistics before (on a NixOS system config):

  time elapsed: 1.3258
  size of a value: 24
  environments allocated: 1980939 (50127080 bytes)
  list elements: 14679308 (117434464 bytes)
  list concatenations: 50828
  values allocated: 2098938 (50374512 bytes)
  attribute sets allocated: 392040
  right-biased unions: 186334
  values copied in right-biased unions: 591137
  symbols in symbol table: 18271
  number of thunks: 1645752
  number of thunks avoided: 1921196
  number of attr lookups: 430798
  number of primop calls: 838807
  number of function calls: 1930107

Statistics after:

  time elapsed: 1.17982
  size of a value: 24
  environments allocated: 1543334 (39624560 bytes)
  list elements: 9612638 (76901104 bytes)
  list concatenations: 37434
  values allocated: 1854933 (44518392 bytes)
  attribute sets allocated: 392040
  right-biased unions: 186334
  values copied in right-biased unions: 591137
  symbols in symbol table: 18272
  number of thunks: 1392467
  number of thunks avoided: 1507311
  number of attr lookups: 430801
  number of primop calls: 691600
  number of function calls: 1492502
2012-08-13 00:28:08 -04:00
Eelco Dolstra e82767910c Add some basic profiling support to the evaluator
Setting the environment variable NIX_COUNT_CALLS to 1 enables some
basic profiling in the evaluator.  It will count calls to functions
and primops as well as evaluations of attributes.

For example, to see where evaluation of a NixOS configuration spends
its time:

$ NIX_SHOW_STATS=1 NIX_COUNT_CALLS=1 ./src/nix-instantiate/nix-instantiate '<nixos>' -A system --readonly-mode
...
calls to 39 primops:
    239532 head
    233962 tail
    191252 hasAttr
...
calls to 1595 functions:
    224157 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/lists.nix:17:19'
    221767 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/lists.nix:17:14'
    221767 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/lists.nix:17:10'
...
evaluations of 7088 attributes:
    167377 undefined position
    132459 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/attrsets.nix:119:41'
     47322 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/attrsets.nix:13:21'
...
2012-08-12 23:29:28 -04:00
Eelco Dolstra 97421eb5ec Refactor settings processing
Put all Nix configuration flags in a Settings object.
2012-07-30 19:55:41 -04:00
Eelco Dolstra 8c79100839 Merge branch 'master' into no-manifests 2012-07-26 15:14:33 -04:00
Shea Levy f5954e2d94 prim_import: When importing .drvs, allocate the intermediate attrset on the heap just in case it escapes the stack frame. 2012-07-25 09:30:17 -04:00
Shea Levy b1112bbef1 import: If the path is a valid .drv file, parse it and generate a derivation attrset.
The generated attrset has drvPath and outPath with the right string context, type 'derivation', outputName with
the right name, all with a list of outputs, and an attribute for each output.

I see three uses for this (though certainly there may be more):

* Using derivations generated by something besides nix-instantiate (e.g. guix)

* Allowing packages provided by channels to be used in nix expressions. If a channel installed a valid deriver
  for each package it provides into the store, then those could be imported and used as dependencies or installed
  in environment.systemPackages, for example.

* Enable hydra to be consistent in how it treats inputs that are outputs of another build. Right now, if an
  input is passed as an argument to the job, it is passed as a derivation, but if it is accessed via NIX_PATH
  (i.e. through the <> syntax), then it is a path that can be imported. This is problematic because the build
  being depended upon may have been built with non-obvious arguments passed to its jobset file. With this
  feature, hydra can just set the name of that input to the path to its drv file in NIX_PATH
2012-07-25 09:30:17 -04:00
Eelco Dolstra fe241ece29 Merge branch 'master' into no-manifests 2012-07-18 10:47:59 -04:00
Eelco Dolstra a7a43adb79 builtins.storePath: resolve symlinks
Needed for Charon/Hydra interaction.
2012-07-12 18:25:01 -04:00
Eelco Dolstra 98a423b75a prim_import(): prefetch substitute info in parallel using queryMissing() 2012-07-09 09:59:34 -04:00
Eelco Dolstra 1aba0bf0fa nix-store -r: do substitutions in parallel
I.e. when multiple non-derivation arguments are passed to ‘nix-store
-r’ to be substituted, do them in parallel.
2012-06-27 16:58:15 -04:00
Eelco Dolstra f491ae97d4 * Inline some functions and get rid of the indirection through
EvalState::eval().  This gives a 12% speedup on ‘nix-instantiate
  /etc/nixos/nixos/ -A system --readonly-mode’ (from 1.01s to 0.89s).
2012-02-04 13:50:25 +00:00
Eelco Dolstra baa0501cc1 * Fix importing a derivation. This gave a segfault. 2012-01-26 13:13:00 +00:00
Eelco Dolstra 18047d4625 * Add some debug output to print the derivation name once it's known.
This makes it easier to pinpoint the source of a crash.
2012-01-19 23:07:42 +00:00
Eelco Dolstra 71f3c46cf6 * Drop the inefficient "Path" suffix in output attribute names. 2012-01-03 15:27:18 +00:00
Eelco Dolstra 921111d197 * Move the implementation of the ‘derivation’ primop into a separate
file.
2012-01-03 14:01:47 +00:00
Eelco Dolstra 5679041132 * The ‘foo.drvPath’ feature was already broken in read-only mode.
Since it's rarely used and fixing it is too much work right now,
  just document it.
2011-12-21 18:19:05 +00:00
Eelco Dolstra b19a0f63db * Simplify the context handling logic. 2011-12-21 15:33:30 +00:00
Eelco Dolstra 194d21f9f6 * Sync with the trunk. 2011-12-16 23:33:01 +00:00
Eelco Dolstra 49f59dceca * Move parseHash16or32 into libutil, and use in nix-hash. 2011-12-02 11:47:06 +00:00
Shea Levy 3c3107da86 There's no need to mess with drvPath at all 2011-11-06 07:18:19 +00:00
Shea Levy 2ab29be70c Fix faulty reversion of my changes to unsafeDiscardOutputDependency 2011-11-06 07:03:14 +00:00
Shea Levy ca0d47a70c Respect all outputs passed to the derivation, not just the last one 2011-11-06 06:54:05 +00:00
Shea Levy 3522730316 Embed output name into the context of the *OutPath attributes and extract it for input derivations
Multiple outputs test passes!
2011-11-06 06:28:30 +00:00
Shea Levy af2e53fd48 Include all outputs of derivations in the closure of explicitly-passed derivation paths
This required adding a queryOutputDerivationNames function in the store API
2011-11-06 06:28:20 +00:00
Shea Levy 981edeab7b The 'insert output between = signs' approach was not helpful 2011-11-06 06:28:14 +00:00
Shea Levy e81c09edbf Remove the current output metadata from the string for unsaveDiscardOutputDependency 2011-09-16 11:31:00 +00:00
Shea Levy bf50d6ad32 Add information about which output is active to drvPath's context
This will break things that depend on being able to just strip away an equals sign, so those have to be updated next
2011-09-16 11:30:52 +00:00
Shea Levy f3e410d4bf Add a currentOutput attribute to derivations keep track of which output is active 2011-09-16 11:30:03 +00:00
Shea Levy c172d16b00 First attempt at the output-as-derivation semantics
For each output, this adds a corresponding attribute to the derivation that is
the same as the derivation except for outPath, which is set to the path specific
to that output. Additionally, an "all" attribute is added that is a list of all
of the output derivations. This has to be done outside of derivationStrict as
each output is itself a derivation that contains itself (and all other outputs)
as an attribute. The derivation itself is equivalent to the first output in the
outputs list (or "out" if that list isn't set).
2011-09-14 05:59:17 +00:00
Eelco Dolstra 93227ff65c * Eliminate all uses of the global variable ‘store’ from libstore.
This should also fix:

    nix-instantiate: ./../boost/shared_ptr.hpp:254: T* boost::shared_ptr<T>::operator->() const [with T = nix::StoreAPI]: Assertion `px != 0' failed.

  which was caused by hashDerivationModulo() calling the ‘store’
  object (during store upgrades) before openStore() assigned it.
2011-08-31 21:11:50 +00:00
Eelco Dolstra 54945a2950 * Refactoring: move parseExprFromFile() and parseExprFromString() into
the EvalState class.
2011-08-06 13:02:55 +00:00
Eelco Dolstra c8606664ab * Don't allow derivations with fixed and non-fixed outputs. 2011-07-20 18:26:00 +00:00
Eelco Dolstra b2027f70d9 * Fix a huuuuge security hole in the Nix daemon. It didn't check that
derivations added to the store by clients have "correct" output
  paths (meaning that the output paths are computed by hashing the
  derivation according to a certain algorithm).  This means that a
  malicious user could craft a special .drv file to build *any*
  desired path in the store with any desired contents (so long as the
  path doesn't already exist).  Then the attacker just needs to wait
  for a victim to come along and install the compromised path.

  For instance, if Alice (the attacker) knows that the latest Firefox
  derivation in Nixpkgs produces the path

    /nix/store/1a5nyfd4ajxbyy97r1fslhgrv70gj8a7-firefox-5.0.1

  then (provided this path doesn't already exist) she can craft a .drv
  file that creates that path (i.e., has it as one of its outputs),
  add it to the store using "nix-store --add", and build it with
  "nix-store -r".  So the fake .drv could write a Trojan to the
  Firefox path.  Then, if user Bob (the victim) comes along and does

    $ nix-env -i firefox
    $ firefox

  he executes the Trojan injected by Alice.

  The fix is to have the Nix daemon verify that derivation outputs are
  correct (in addValidPath()).  This required some refactoring to move
  the hash computation code to libstore.
2011-07-20 18:10:47 +00:00
Eelco Dolstra d329c3ea9d * Support multiple outputs. A derivation can declare multiple outputs
by setting the ‘outputs’ attribute.  For example:

    stdenv.mkDerivation {
      name = "aterm-2.5";

      src = ...;

      outputs = [ "out" "tools" "dev" ];

      configureFlags = "--bindir=$(tools)/bin --includedir=$(dev)/include";
    }

  This derivation creates three outputs, named like this:

    /nix/store/gcnqgllbh01p3d448q8q6pzn2nc2gpyl-aterm-2.5
    /nix/store/gjf1sgirwfnrlr0bdxyrwzpw2r304j02-aterm-2.5-tools
    /nix/store/hp6108bqfgxvza25nnxfs7kj88xi2vdx-aterm-2.5-dev

  That is, the symbolic name of the output is suffixed to the store
  path (except for the ‘out’ output).  Each path is passed to the
  builder through the corresponding environment variable, e.g.,
  ${tools}.

  The main reason for multiple outputs is to allow parts of a package
  to be distributed and garbage-collected separately.  For instance,
  most packages depend on Glibc for its libraries, but don't need its
  header files.  If these are separated into different store paths,
  then a package that depends on the Glibc libraries only causes the
  libraries and not the headers to be downloaded.

  The main problem with multiple outputs is that if one output exists
  while the others have been garbage-collected (or never downloaded in
  the first place), and we want to rebuild the other outputs, then
  this isn't possible because we can't clobber a valid output (it
  might be in active use).  This currently gives an error message
  like:

    error: derivation `/nix/store/1s9zw4c8qydpjyrayxamx2z7zzp5pcgh-aterm-2.5.drv' is blocked by its output paths

  There are two solutions: 1) Do the build in a chroot.  Then we don't
  need to overwrite the existing path.  2) Use hash rewriting (see the
  ASE-2005 paper).  Scary but it should work.

  This is not finished yet.  There is not yet an easy way to refer to
  non-default outputs in Nix expressions.  Also, mutually recursive
  outputs aren't detected yet and cause the garbage collector to
  crash.
2011-07-18 23:31:03 +00:00
Eelco Dolstra 9db190eb31 * builtins.substring: if "start" is beyond the end of the string,
return the empty string.
2011-01-14 12:47:10 +00:00
Eelco Dolstra 43535499f3 * When allocating an attribute set, reserve enough space for all
elements.  This prevents the vector from having to resize itself.
2010-10-24 20:09:37 +00:00
Eelco Dolstra e0b7fb8f27 * Keep attribute sets in sorted order to speed up attribute lookups.
* Simplify the representation of attributes in the AST.
* Change the behaviour of listToAttrs() in case of duplicate names.
2010-10-24 19:52:33 +00:00
Eelco Dolstra 2dc6d50941 * Don't create thunks for variable lookups (if possible). This
significantly reduces the number of values allocated (e.g. from 8.7m
  to 4.9m for the Bittorrent test).
2010-10-24 14:20:02 +00:00
Eelco Dolstra 0b305c534f * Store attribute sets as a vector instead of a map (i.e. a red-black
tree).  This saves a lot of memory.  The vector should be sorted so
  that names can be looked up using binary search, but this is not the
  case yet.  (Surprisingly, looking up attributes using linear search
  doesn't have a big impact on performance.)

  Memory consumption for

    $ nix-instantiate /etc/nixos/nixos/tests -A bittorrent.test --readonly-mode

  on x86_64-linux with GC enabled is now 185 MiB (compared to 946
  MiB on the trunk).
2010-10-24 00:41:29 +00:00
Eelco Dolstra 3f66cfb96b * Remove allocValues(). 2010-10-23 18:18:07 +00:00
Eelco Dolstra 41c45a9b31 * Store Value nodes outside of attribute sets. I.e., Attr now stores
a pointer to a Value, rather than the Value directly.  This improves
  the effectiveness of garbage collection a lot: if the Value is
  stored inside the set directly, then any live pointer to the Value
  causes all other attributes in the set to be live as well.
2010-10-22 14:47:42 +00:00
Eelco Dolstra 705868a8a9 * Make sure that config.h is included before the system headers,
because it defines _FILE_OFFSET_BITS.  Without this, on 
  OpenSolaris the system headers define it to be 32, and then 
  the 32-bit stat() ends up being called with a 64-bit "struct 
  stat", or vice versa.

  This also ensures that we get 64-bit file sizes everywhere.

* Remove the redundant call to stat() in parseExprFromFile().
  The file cannot be a symlink because that's the exit condition
  of the loop before.
2010-10-04 17:55:38 +00:00
Eelco Dolstra 7af6a2fd71 * intersectAttrs: optimise for the case where the second set is larger
than the first set.  (That's usually the case with callPackage.)
2010-08-02 11:54:44 +00:00
Eelco Dolstra 89865da76d * Turn build errors during evaluation into EvalErrors. 2010-06-01 11:19:32 +00:00
Eelco Dolstra 81a4b4e49b * Implemented tryEval, the last missing primop in the fast-eval
branch.  Also added a test for tryEval.
2010-05-12 11:23:44 +00:00
Eelco Dolstra 83dfa89870 * Sync with the trunk. 2010-05-07 14:46:47 +00:00
Eelco Dolstra 83d7b89660 * Updated addErrorContext. 2010-05-07 12:33:14 +00:00
Eelco Dolstra e2d5e40f4f * Keep track of the source positions of attributes. 2010-05-07 12:11:05 +00:00
Eelco Dolstra 0777448ca6 * Fixed builtins.genericClosure. 2010-04-21 15:57:11 +00:00
Eelco Dolstra fe2d869e04 * Store user environment manifests as a Nix expression in
$out/manifest.nix rather than as an ATerm.

  (Hm, I thought I committed this two days ago...)
2010-04-21 15:08:58 +00:00
Eelco Dolstra efc7a579e8 * Don't use the ATerm library for parsing/printing .drv files. 2010-04-19 13:46:58 +00:00
Eelco Dolstra 5c31995bb8 * Updated some more primops. 2010-04-16 15:13:47 +00:00
Eelco Dolstra 8ca4a001cb * Improve sharing a bit. 2010-04-16 14:03:26 +00:00
Eelco Dolstra 04c4bd3624 * Store lists as lists of pointers to values rather than as lists of
values.  This improves sharing and gives another speed up.
  Evaluation of the NixOS system attribute is now almost 7 times
  faster than the old evaluator.
2010-04-15 00:37:36 +00:00
Eelco Dolstra 267dc693d2 * Fix builtins. 2010-04-14 22:59:39 +00:00
Eelco Dolstra 9985230c00 * After parsing, compute level/displacement pairs for each variable
use site, allowing environments to be stores as vectors of values
  rather than maps.  This should speed up evaluation and reduce the
  number of allocations.
2010-04-14 14:42:32 +00:00
Eelco Dolstra ac1e8f40d4 * Use a symbol table to represent identifiers and attribute names
efficiently.  The symbol table ensures that there is only one copy
  of each symbol, thus allowing symbols to be compared efficiently
  using a pointer equality test.
2010-04-13 12:25:42 +00:00
Eelco Dolstra 10e8b1fd15 * Finished the ATerm-less parser. 2010-04-12 23:33:23 +00:00
Eelco Dolstra 4d6ad5be17 * Don't use ATerms for the abstract syntax trees anymore. Not
finished yet.
2010-04-12 18:30:11 +00:00
Eelco Dolstra 9a64454faa * expr-to-xml -> value-to-xml. 2010-04-07 13:59:45 +00:00
Eelco Dolstra fc92244ba8 * Implemented the primops necessary for generating the NixOS manual. 2010-04-07 13:55:46 +00:00
Eelco Dolstra 71f026292b * Make `derivation' lazy again for performance. It also turns out
that there are some places in Nixpkgs (php_configurable /
  composableDerivation, it seems) that call `derivation' with
  incorrect arguments (namely, the `name' attribute missing) but get
  away with it because of laziness.
2010-04-01 09:55:57 +00:00
Eelco Dolstra dc31305b38 * Fixed the trace primop and path comparison.
* Removed exprToString and stringToExpr because there is no ATerm
  representation to work on anymore (and exposing the internals of the
  evaluator like this is not a good idea anyway).
2010-03-31 20:09:20 +00:00
Eelco Dolstra 979f163615 * Handle string contexts. `nix-instantiate' can now correctly compute
the `firefoxWrapper' attribute in Nixpkgs, and it's about 3 times
  faster than the trunk :-)
2010-03-31 19:52:29 +00:00
Eelco Dolstra 3d94be61ea * Implemented derivations. 2010-03-31 15:38:03 +00:00
Eelco Dolstra 7f19e03c65 * More primops. 2010-03-30 22:39:48 +00:00
Eelco Dolstra 47df476daa * More operators / primops. 2010-03-30 18:05:54 +00:00
Eelco Dolstra c9170be2bd * More primops. 2010-03-30 15:18:20 +00:00
Eelco Dolstra c3aa615a5f * More primops. 2010-03-30 14:39:27 +00:00
Eelco Dolstra 5b72d8a749 * Implemented `map'. 2010-03-30 13:47:59 +00:00
Eelco Dolstra d78a05ab40 * Make `import' work. 2010-03-30 09:22:33 +00:00
Eelco Dolstra 31428c3a06 * Started integrating the new evaluator. 2010-03-29 14:37:56 +00:00
Eelco Dolstra 8a10360c91 * Simplify @-patterns: only {attrs}@name' or name@{attrs}' are now
allowed.  So `name1@name2', `{attrs1}@{attrs2}' and so on are now no
  longer legal.  This is no big loss because they were not useful
  anyway.

  This also changes the output of builtins.toXML for @-patterns
  slightly.
2010-03-25 12:19:41 +00:00
Eelco Dolstra deb342fb08 * builtins.trace: in the common case that the value is a string, then
show the string, not the ATerm, so we get `trace: bla' instead of
  `trace: Str("bla",[])'.
2009-10-22 08:10:12 +00:00
Eelco Dolstra 437077c39d * Added a primop unsafeDiscardOutputDependency needed by Disnix to
pass derivation paths to a builder without actually building them.
2009-10-21 15:05:30 +00:00
Eelco Dolstra 1332dd1ed3 * tryEval shouldn't catch all exceptions of type Error, since not all
of them leave the evaluator in a continuable state.  Also, it should
  be less chatty.
2009-09-23 19:19:26 +00:00
Eelco Dolstra 0dbd4638e0 * Two primops: builtins.intersectAttrs and builtins.functionArgs.
intersectAttrs returns the (right-biased) intersection between two
  attribute sets, e.g. every attribute from the second set that also
  exists in the first.  functionArgs returns the set of attributes
  expected by a function.

  The main goal of these is to allow the elimination of most of
  all-packages.nix.  Most package instantiations in all-packages.nix
  have this form:

    foo = import ./foo.nix {
      inherit a b c;
    };

  With intersectAttrs and functionArgs, this can be written as:

    foo = callPackage (import ./foo.nix) { };

  where

   callPackage = f: args:
     f ((builtins.intersectAttrs (builtins.functionArgs f) pkgs) // args);

  I.e., foo.nix is called with all attributes from "pkgs" that it
  actually needs (e.g., pkgs.a, pkgs.b and pkgs.c).  (callPackage can
  do any other generic package-level stuff we might want, such as
  applying makeOverridable.)  Of course, the automatically supplied
  arguments can be overriden if needed, e.g.

    foo = callPackage (import ./foo.nix) {
      c = c_version_2;
    };

  but for the vast majority of packages, this won't be needed.

  The advantages are to reduce the amount of typing needed to add a
  dependency (from three sites to two), and to reduce the number of
  trivial commits to all-packages.nix.  For the former, there have
  been two previous attempts:

    - Use "args: with args;" in the package's function definition.
      This however obscures the actual expected arguments of a
      function, which is very bad.

    - Use "{ arg1, arg2, ... }:" in the package's function definition
      (i.e. use the ellipis "..." to allow arbitrary additional
      arguments), and then call the function with all of "pkgs" as an
      argument.  But this inhibits error detection if you call it with
      an misspelled (or obsolete) argument.
2009-09-15 13:01:46 +00:00
Michael Raskin 3bca8931e8 Adding tryEval builtin. It allows to catch presence of errors in an expression. 2009-08-25 16:06:46 +00:00
Eelco Dolstra f2c3fc5191 * Don't show trace information by default (`--show-trace' to enable).
NixOS evaluation errors in particular look intimidating and
  generally aren't very useful.  Ideally the builtins.throw messages
  should be self-contained.
2009-06-30 13:28:29 +00:00
Eelco Dolstra 2897286487 * Unify exportReferencesGraph and exportBuildReferencesGraph, and make
sure that it works as expected when you pass it a derivation.  That
  is, we have to make sure that all build-time dependencies are built,
  and that they are all in the input closure (otherwise remote builds
  might fail, for example).  This is ensured at instantiation time by
  adding all derivations and their sources to inputDrvs and inputSrcs.
2009-03-18 17:36:42 +00:00
Marc Weber 1407a1ec99 added primop functions __isBool, __isString, __isInt 2009-02-05 19:35:40 +00:00
Nicolas Pierron 110606d470 Add the "addErrorContext" builtin to add more information in the stack trace. 2009-01-27 14:36:44 +00:00
Eelco Dolstra 63b8f09d8d 2008-12-04 10:45:47 +00:00
Eelco Dolstra f8713e1287 * Dirty hack to make nix-push work properly on derivations: the
derivation should be a source rather than a derivation dependency of
  the call to the NAR derivation.  Otherwise the derivation (and all
  its dependencies) will be built as a side-effect, which may not even
  succeed.
2008-12-04 10:40:41 +00:00
Eelco Dolstra ff762fb499 * Pass HashType values instead of strings. 2008-12-03 16:10:17 +00:00
Eelco Dolstra 1307b22223 * Made addToStore() a lot more efficient: it no longer reads the path
being copied 3 times in the worst case.  It doesn't run in constant space,
  but it didn't do that anyway.
2008-12-03 15:51:17 +00:00
Eelco Dolstra 64519cfd65 * Unify the treatment of sources copied to the store, and recursive
SHA-256 outputs of fixed-output derivations.  I.e. they now produce
  the same store path:

  $ nix-store --add x
  /nix/store/j2fq9qxvvxgqymvpszhs773ncci45xsj-x

  $ nix-store --add-fixed --recursive sha256 x
  /nix/store/j2fq9qxvvxgqymvpszhs773ncci45xsj-x

  the latter being the same as the path that a derivation

    derivation {
      name = "x";
      outputHashAlgo = "sha256";
      outputHashMode = "recursive";
      outputHash = "...";
      ...
    };

  produces.

  This does change the output path for such fixed-output derivations.
  Fortunately they are quite rare.  The most common use is fetchsvn
  calls with SHA-256 hashes.  (There are a handful of those is
  Nixpkgs, mostly unstable development packages.)
  
* Documented the computation of store paths (in store-api.cc).
2008-12-03 15:06:30 +00:00
Eelco Dolstra aab530e971 * Primop builtins.storePath for declaring a store path as a
dependency.  `storePath /nix/store/bla' gives exactly the same
  result as `toPath /nix/store/bla', except that the former includes
  /nix/store/bla in the dependency context of the string.

  Useful in some generated Nix expressions like nix-push, which now
  finally does the right thing wrt distributed builds.  (Previously
  the path to be packed wasn't an explicit dependency, so it wouldn't
  be copied to the remote machine.)
2008-11-19 23:26:19 +00:00
Eelco Dolstra efe4b690ae * Refactoring: combine functions that take an attribute set and
functions that take a single argument (plain lambdas) into one AST
  node (Function) that contains a Pattern node describing the
  arguments.  Current patterns are single lazy arguments (VarPat) and
  matching against an attribute set (AttrsPat).

  This refactoring allows other kinds of patterns to be added easily,
  such as Haskell-style @-patterns, or list pattern matching.
2008-08-14 10:04:22 +00:00
Eelco Dolstra 7cd88b1dec * Generalised the dependencyClosure primop to builtins.genericClosure,
which is hopefully more useful.
* New primops: length, mul, div.
2008-07-11 13:29:04 +00:00
Eelco Dolstra d567baabbd * Export the nix-env derivation name parsing and version comparison
logic through the `parseDrvName' and `compareVersions' primops.
  This will allow expressions to easily check whether some dependency
  is a specific needed version or falls in some version range.  See
  tests/lang/eval-okay-versions.nix for examples.
2008-07-01 10:10:32 +00:00
Michael Raskin 5b5a3af983 Probably fixed __exprToString 2008-01-20 20:44:03 +00:00
Michael Raskin 5eb5c23447 Fixed exportBuildReferenceGraph 2008-01-15 04:32:08 +00:00
Eelco Dolstra 7d0f6aed59 * New primop `unsafeDiscardStringContext' to get rid of string
contexts.  Needed to prevent unnecessary dependencies when building
  the NixOS manual.
2008-01-04 14:22:49 +00:00
Eelco Dolstra dedd62dd0c * More release notes. 2007-12-31 00:08:09 +00:00
Eelco Dolstra 06f95dd07c * New primop `readFile' to get the contents of a file as a string. 2007-11-21 13:49:59 +00:00
Eelco Dolstra fa44e401a8 * Documented multi-user Nix. 2007-10-31 18:01:56 +00:00
Eelco Dolstra 0b4ed64d29 * "trace" primop: write the trace to standard error. 2007-10-26 18:25:50 +00:00
Eelco Dolstra 27a0662828 * listToAttrs: the list now should consist of {name, value} attribute
sets instead of {attr, value}.  "name" is better than "attr" because
  the *combination* of the two forms the attribute.
2007-10-09 12:51:25 +00:00
Eelco Dolstra f3441e6122 * Pass various options to the worker so that flags like -K or -j work
in multi-user Nix (NIX-72).
* Client/worker: exchange a protocol version number for future
  compatibility.
2007-09-18 09:11:20 +00:00
Marc Weber 2629998e91 primop functions listToAttrs (+test), __isAttrs, __trace added
new configuration style proposal in lib/default-unstable.nix
2007-08-18 22:12:00 +00:00
Eelco Dolstra bddc83a148 * New builtin function "isFunction". You're not supposed to use it
;-)
* Channels: fix channels that are plain lists of derivations (like
  strategoxt-unstable) instead  of functions (like nixpkgs-unstable).
  This fixes the error message "error: the left-hand side of the
  function call is neither a function nor a primop (built-in
  operation) but a list".
2007-05-16 16:17:04 +00:00
Eelco Dolstra 5f2492eaec * New primop "throw <string>" to throw an error. This is like abort,
only thrown errors are caught by the top-level derivation evaluation
  in nix-env -qa / -i.
2007-04-16 15:03:19 +00:00
Eelco Dolstra b618fa6eb6 * computeStorePathForText: take the references into account when
computing the store path (NIX-77).  This is an important security
  property in multi-user Nix stores.

  Note that this changes the store paths of derivations (since the
  derivation aterms are added using addTextToStore), but not most
  outputs (unless they use builtins.toFile).
2007-01-29 15:51:37 +00:00
Eelco Dolstra c558b1583c * Don't capitalise the primop functions. 2007-01-29 15:15:37 +00:00
Eelco Dolstra 18e6096105 * Organise primops.cc a bit better. 2007-01-29 15:11:32 +00:00
Eelco Dolstra 7349bd0176 New primitives:
* `sub' to subtract two numbers.
* `stringLength' to get the length of a string.
* `substring' to get a substring of a string.  These should be enough
  to allow most string operations to be expressed.
2007-01-29 14:23:09 +00:00
Eelco Dolstra 7dedbd896a * filterSource: pass strings to the predicate function instead of
paths.  Paths can have unexpected semantics.
2007-01-29 13:32:50 +00:00
Eelco Dolstra e4b0666f8e * builtins.filterSource: pass the type of the file ("regular",
"directory", "symlink") as the second argument to the filter
  predicate.
2007-01-15 08:54:51 +00:00
Eelco Dolstra 11158028be * Cleanup. 2007-01-13 14:21:49 +00:00
Eelco Dolstra 1073b1780a * Remove debug message. 2006-12-13 14:29:05 +00:00
Eelco Dolstra a3e6415ba8 * New primop builtins.filterSource, which can be used to filter files
from a source directory.  All files for which a predicate function
  returns true are copied to the store.  Typical example is to leave
  out the .svn directory:

    stdenv.mkDerivation {
      ...
      src = builtins.filterSource
        (path: baseNameOf (toString path) != ".svn")
        ./source-dir;
      # as opposed to
      #   src = ./source-dir;
    }

  This is important because the .svn directory influences the hash in
  a rather unpredictable and variable way.
2006-12-12 23:05:01 +00:00
Eelco Dolstra 1a7e88bbd9 * New built-in function `builtins.attrNames' that returns the
names of the attributes in an attribute set.
2006-12-12 16:14:31 +00:00
Eelco Dolstra fcd9900d74 * Replace read-only calls to addTextToStore. 2006-12-01 21:00:39 +00:00
Eelco Dolstra 6ecb840fd1 * Put building in the store API. 2006-11-30 18:02:04 +00:00
Eelco Dolstra e2ef5e07fd * Refactoring. There is now an abstract interface class StoreAPI
containing functions that operate on the Nix store.  One
  implementation is LocalStore, which operates on the Nix store
  directly.  The next step, to enable secure multi-user Nix, is to
  create a different implementation RemoteStore that talks to a
  privileged daemon process that uses LocalStore to perform the actual
  operations.
2006-11-30 17:43:04 +00:00
Eelco Dolstra 7e85a2af5f * Fix importing of derivation outputs. 2006-11-03 16:17:39 +00:00
Eelco Dolstra dd300fb48d * Some better error messages. 2006-10-23 16:45:19 +00:00
Eelco Dolstra 17f4883bfe * Better message. 2006-10-19 17:43:58 +00:00
Eelco Dolstra 9bd93f7606 * toFile: maintain the references. 2006-10-19 17:39:02 +00:00
Eelco Dolstra 3059df0f1e * baseNameOf: paths don't have to be absolute. 2006-10-17 12:34:13 +00:00
Eelco Dolstra be1961c9f8 * toPath: should be the identity on paths. 2006-10-17 11:07:11 +00:00
Eelco Dolstra cba913c521 * dirOf: return a path if the argument is a path. 2006-10-17 11:05:34 +00:00
Eelco Dolstra cf705eaf78 * toString: don't copy paths. So toString can be used to pass
non-store paths to a builder.
2006-10-17 10:58:12 +00:00
Eelco Dolstra d7efd76394 * Big cleanup of the semantics of paths, strings, contexts, string
concatenation and string coercion.  This was a big mess (see
  e.g. NIX-67).  Contexts are now folded into strings, so that they
  don't cause evaluation errors when they're not expected.  The
  semantics of paths has been clarified (see nixexpr-ast.def).
  toString() and coerceToString() have been merged.

  Semantic change: paths are now copied to the store when they're in a
  concatenation (and in most other situations - that's the
  formalisation of the meaning of a path).  So

    "foo " + ./bla

  evaluates to "foo /nix/store/hash...-bla", not "foo
  /path/to/current-dir/bla".  This prevents accidental impurities, and
  is more consistent with the treatment of derivation outputs, e.g.,
  `"foo " + bla' where `bla' is a derivation.  (Here `bla' would be
  replaced by the output path of `bla'.)
2006-10-16 15:55:34 +00:00
Eelco Dolstra 7d4567f2cc * Removed URIs from the evaluator (NIX-66). They are now just another
kind of notation for strings.
2006-10-11 21:59:33 +00:00
Eelco Dolstra 0c4c5c2020 * Quick hack to fix NIX-67: evaluation result differing if the Nix
expression resides in the store.
2006-10-10 21:23:35 +00:00
Eelco Dolstra bd0c40e1e9 * import': unwrap the context. Necessary to make import (x + y)'
work, where x is a store path.
2006-10-10 15:07:23 +00:00
Eelco Dolstra 5fd44654db * toXML: propagate the context to allow derivations to be used in the
argument.
2006-10-03 15:38:59 +00:00
Eelco Dolstra d20c3011a0 * toFile: added an additional argument to specify the store path
suffix, e.g., `builtins.toFile "builder.sh" "..."'.
* toFile: handle references to other files correctly.
2006-10-03 14:55:54 +00:00
Eelco Dolstra e347033f71 * The result of a concatenation with a derivation on the left-hand
side should be a path, I guess.
* Handle paths that are in the store but not direct children of the
  store directory.
* Ugh, hack to prevent double context wrapping.
2006-09-24 21:39:57 +00:00
Eelco Dolstra 0e705391db * Primop `toPath' to convert a string to a path.
* Primop `pathExists' to check for path existence.
2006-09-24 18:23:32 +00:00
Eelco Dolstra e47e0c2dbe * Builtin function `getEnv' for getting environment variables. 2006-09-24 17:48:41 +00:00
Eelco Dolstra df8873e14a * lessThan primitive for integer comparison. 2006-09-24 15:21:48 +00:00
Eelco Dolstra 2ab4bc44c7 * Builtin function `add' to add integers.
* Put common test functions in tests/lang/lib.nix.
2006-09-22 15:29:21 +00:00
Eelco Dolstra d315210612 * Added a builtin function `isList' to test whether a value is a list.
With this primitive, a list-flattening function can be implemented
  (NIX-55, example is in tests/lang/eval-okay-flatten.nix).
2006-09-22 14:55:19 +00:00
Eelco Dolstra c02a44183f * Builtin functions head' and tail' to return the head and tail of
list.  Useful for lots of things, such as implementing a fold
  function (see NIX-30, example is in tests/lang/eval-okay-list.nix).
2006-09-22 14:46:36 +00:00
Eelco Dolstra 8a1ab709a4 * New builtin functions builtins.{hasAttr, getAttr} to check for
attribute existence and to return an attribute from an attribute
  set, respectively.  Example: `hasAttr "foo" {foo = 1;}'.  They
  differ from the `?' and `.' operators in that the attribute name is
  an arbitrary expression.  (NIX-61)
2006-09-22 14:31:55 +00:00
Eelco Dolstra 4e91d8621f * Fix comment. 2006-09-21 18:52:05 +00:00
Eelco Dolstra 0623359fbc * Print a better error message for wrong hashes (NIX-49). 2006-09-20 16:15:32 +00:00
Eelco Dolstra 75068e7d75 * Use a proper namespace.
* Optimise header file usage a bit.
* Compile the parser as C++.
2006-09-04 21:06:23 +00:00
Eelco Dolstra 7974aae81c * New primop: builtins.toFile, which writes a string into the store
and returns its path.  This can be used to (for instance) write
  builders inside a Nix expression, e.g.,

  stdenv.mkDerivation {
    builder = "
      source $stdenv/setup
      ...
    ";
    ...
  }
2006-09-01 12:07:31 +00:00
Eelco Dolstra 547b119f25 * Support singleton values and nested lists again in `args', but print
a warning.
2006-08-29 15:40:49 +00:00
Eelco Dolstra 2132d9ddeb * Fix the ~ operator. 2006-08-29 15:29:38 +00:00
Eelco Dolstra 1ec9f55741 * In toString, deal with nested lists properly (i.e., flatten them). 2006-08-28 21:47:42 +00:00
Eelco Dolstra 1fca76870b * Removed processBinding, instead we now apply toString to all
derivation attributes to flatten them into strings.  This is
  possible since string can nowadays be wrapped in contexts that
  describe the derivations/sources referenced by the evaluation of the
  string.
2006-08-28 13:31:06 +00:00
Eelco Dolstra 8a6080eb14 * Refactoring. 2006-08-26 16:48:01 +00:00
Eelco Dolstra 4b66cebe7b * Remove those storePath attribute sets, we don't need 'em. 2006-08-25 17:09:55 +00:00
Eelco Dolstra 215ec2ddc6 * New primop __toXML (or builtins.toXML) to convert an expression to
an XML representation stored in a string.  This should be useful to
  pass structured information to builders.
2006-08-24 14:34:29 +00:00
Eelco Dolstra b19cebc513 * Quotes. 2006-08-23 15:46:27 +00:00
Eelco Dolstra 38f18aa6d4 * New primop: abort "error message". 2006-08-23 15:46:00 +00:00
Eelco Dolstra 4a053bfdfd * A new primop `builtins', which returns an attribute set containing
all the primops.  This allows Nix expressions to test for new
  primops and take appropriate action if they're not available.  For
  instance, rather than calling a primop `foo' directly, they could
  say `if builtins ? foo then builtins.foo ... else ...'.
2006-08-23 14:39:11 +00:00
Eelco Dolstra a18d02e0b0 * Print a warning that the subpath operator (~) is deprecated. 2006-08-09 15:08:47 +00:00
Eelco Dolstra 4f3725b167 * Better error messages (especially wrt types). 2006-07-19 15:36:15 +00:00
Eelco Dolstra 9d72bf8835 * 64-bit compatibility fixes (for problems revealed by building on an Athlon
64 running 64-bit SUSE).  A patched ATerm library is required to run Nix
  succesfully.
2006-05-11 02:19:43 +00:00
Eelco Dolstra 0832956089 * Use the new ATermMap. 2006-05-04 12:21:08 +00:00
Eelco Dolstra 6cecad2be0 * Allow string concatenations involving derivations, e.g.,
configureFlags = "--with-freetype2-library="
      + freetype + "/lib";
2006-05-01 09:56:56 +00:00
Eelco Dolstra ef2d4a2da9 * Print a more useful stack trace when an error occurs deep in the
derivation dependency graph.
2006-03-24 14:02:44 +00:00
Eelco Dolstra 37d1b1cafd * `nix-env -qa --description' shows human-readable descriptions of
packages (provided that they have a `meta.description' attribute).
  E.g.,

  $ ./src/nix-env/nix-env -qa --description gcc
  gcc-4.0.2   GNU Compiler Collection, 4.0.x (cross-compiler for sparc-linux)
  gcc-4.0.2   GNU Compiler Collection, 4.0.x (cross-compiler for mips-linux)
  gcc-4.0.2   GNU Compiler Collection, 4.0.x (cross-compiler for arm-linux)
  gcc-4.0.2   GNU Compiler Collection, 4.0.x
2006-03-10 16:20:42 +00:00
Eelco Dolstra 922697c8b2 * Big speedup (factor > 2.5) in all nix-env operations that do actual
instantiation, e.g. "nix-env -i" and "nix-env -qas" (but not
  "nix-env -qa").  It turns out that many redundant calls to
  addToStore(path) were made, which reads and hashes the entire path.
  For instance, the bash bootstrap binary in Nixpkgs would be read and
  hashed many times.  As a result nix-env would spend around 92% of
  its time in the function sha256_block (according to callgrind).
  Some simple memoization fixes this.
2006-03-09 15:09:18 +00:00
Eelco Dolstra 9088dee9e2 * Some refactoring of the exception handling code so that we can catch
Nix expression assertion failures.
2006-03-08 14:11:19 +00:00
Eelco Dolstra c8bfb11b34 * `nix-env (-i|-u) --dry-run' now shows exactly which missing paths
will be built or substituted.
2006-03-06 11:21:15 +00:00
Eelco Dolstra cf2bb91ec8 * Missing #include. 2005-09-13 13:17:01 +00:00
Eelco Dolstra e1a6fb7870 * `dependencyClosure' now allows a search path, e.g.,
dependencyClosure { ... searchPath = [ ../foo ../bar ]; ... }

* Primop `dirOf' to return the directory part of a path (e.g., dirOf
  /a/b/c == /a/b).

* Primop `relativise' (according to Webster that's a real word!) that
  given paths A and B returns a string representing path B relative
  path to A; e.g., relativise /a/b/c a/b/x/y => "../x/y".
2005-08-14 14:00:39 +00:00
Eelco Dolstra 08c53923db * A primitive operation `dependencyClosure' to do automatic dependency
determination (e.g., finding the header files dependencies of a C
  file) in Nix low-level builds automatically.

  For instance, in the function `compileC' in make/lib/default.nix, we
  find the header file dependencies of C file `main' as follows:

    localIncludes =
      dependencyClosure {
        scanner = file:
          import (findIncludes {
            inherit file;
          });
        startSet = [main];
      };

  The function works by "growing" the set of dependencies, starting
  with the set `startSet', and calling the function `scanner' for each
  file to get its dependencies (which should yield a list of strings
  representing relative paths).  For instance, when `scanner' is
  called on a file `foo.c' that includes the line

    #include "../bar/fnord.h"

  then `scanner' should yield ["../bar/fnord.h"].  This list of
  dependencies is absolutised relative to the including file and added
  to the set of dependencies.  The process continues until no more
  dependencies are found (hence its a closure).

  `dependencyClosure' yields a list that contains in alternation a
  dependency, and its relative path to the directory of the start
  file, e.g.,

    [ /bla/bla/foo.c
      "foo.c"
      /bla/bar/fnord.h
      "../bar/fnord.h"
    ]

  These relative paths are necessary for the builder that compiles
  foo.c to reconstruct the relative directory structure expected by
  foo.c.

  The advantage of `dependencyClosure' over the old approach (using
  the impure `__currentTime') is that it's completely pure, and more
  efficient because it only rescans for dependencies (i.e., by
  building the derivations yielded by `scanner') if sources have
  actually changed.  The old approach rescanned every time.
2005-08-14 12:38:47 +00:00
Eelco Dolstra 991a130b1e * Added a list concatenation operator:
[1 2 3] ++ [4 5 6] => [1 2 3 4 5 6]
2005-07-25 15:05:34 +00:00
Eelco Dolstra 040140dd1c * Added a primop `removeAttrs' to remove attributes from a set, e.g.,
`removeAttrs attrs ["x", "y"]' returns the set `attrs' with the
  attributes named `x' and `y' removed.  It is not an error for the
  named attributes to be missing from the input set.
2005-05-18 17:19:21 +00:00
Eelco Dolstra 77557a6f06 Commit 3000!
* Make the `derivation' primitive much more lazy.  The expression
  `derivation attrs' now evaluates to (essentially)

    attrs // {
      type = "derivation";
      outPath = derivation! attrs;
      drvPath = derivation! attrs;
    }

  where `derivation!' is a primop that does the actual derivation
  instantiation (i.e., it does what `derivation' used to do).  The
  advantage is that it allows commands such as `nix-env -qa' and
  `nix-env -i' to be much faster since they no longer need to
  instantiate all derivations, just the `name' attribute.  (However,
  `nix-env' doesn't yet take advantage of this since it still always
  evaluates the `outPath' and `drvPath' attributes).

  Also, this allows derivations to cyclically reference each other,
  for example,

    webServer = derivation {
      ...
      hostName = "svn.cs.uu.nl";
      services = [svnService];
    };

    svnService = derivation {
      ...
      hostName = webServer.hostName;
    };

  Previously, this would yield a black hole (infinite recursion).
2005-05-07 21:48:49 +00:00
Eelco Dolstra 02f2da0142 * Merging from nix-make branch:
- Add __currentTime primitive (dangerous!).
  - Allow imports of derivations.
2005-05-02 14:44:58 +00:00
Eelco Dolstra c9c58dba55 * Primop `__currentSystem' to return the current platform identifier. 2005-04-10 17:38:19 +00:00
Eelco Dolstra 10c429c757 * If store paths are specified as sources in Nix expressions, don't
copy them, but use them directly.
2005-04-07 14:35:01 +00:00
Eelco Dolstra c815aff21b * `nix-store --add-fixed' to preload the outputs of fixed-output
derivations.  This is mostly to simplify the implementation of
  nix-prefetch-{url, svn}, which now work properly in setuid
  installations.

* Enforce valid store names in `nix-store --add / --add-fixed'.
2005-04-07 14:01:51 +00:00