Commit graph

297 commits

Author SHA1 Message Date
Eelco Dolstra 9451ef3731 Allow URLs in the Nix search path
E.g. to install "hello" from the latest Nixpkgs:

  $ nix-build '<nixpkgs>' -A hello -I nixpkgs=https://nixos.org/channels/nixpkgs-unstable/nixexprs.tar.xz

Or to install a specific version of NixOS:

  $ nixos-rebuild switch -I nixpkgs=63def04891.tar.gz
2015-05-05 17:09:42 +02:00
Eelco Dolstra 4ed2187377 Use cached result if there is a network error 2015-04-09 12:49:13 +02:00
Eelco Dolstra 1fc905ad4c Move curl stuff into a separate file 2015-04-09 12:12:50 +02:00
Eelco Dolstra c1f04fae35 Implement a TTL on cached fetchurl/fetchTarball results
This is because we don't want to do HTTP requests on every evaluation,
even though we can prevent a full redownload via the cached ETag. The
default is one hour.
2015-04-09 11:55:36 +02:00
Eelco Dolstra 60340ce3e2 Implement caching of fetchurl/fetchTarball results
ETags are used to prevent redownloading unchanged files.
2015-04-09 11:42:04 +02:00
Eelco Dolstra 000b5a000f Add fetchTarball builtin
This function downloads and unpacks the given URL at evaluation
time. This is primarily intended to make it easier to deal with Nix
expressions that have external dependencies. For instance, to fetch
Nixpkgs 14.12:

  with import (fetchTarball https://github.com/NixOS/nixpkgs-channels/archive/nixos-14.12.tar.gz) {};

Or to fetch a specific revision:

  with import (fetchTarball 2766a4b44e.tar.gz) {};

This patch also adds a ‘fetchurl’ builtin that downloads but doesn't
unpack its argument. Not sure if it's useful though.
2015-03-25 17:29:09 +01:00
Eelco Dolstra 7ea6ecf855 addToStore(): Take explicit name argument 2015-03-25 17:06:12 +01:00
Eelco Dolstra 6f0c6e20e0 Don't rely on __noChroot for corepkgs
This doesn't work anymore if the "strict" chroot mode is
enabled. Instead, add Nix's store path as a dependency. This ensures
that its closure is present in the chroot.
2015-03-24 11:15:45 +01:00
Daniel Hahler e659978ced Fix typos: s/the the/the/ 2015-03-06 16:43:22 +01:00
Eelco Dolstra 15d2d3c34e Add restricted evaluation mode
If ‘--option restrict-eval true’ is given, the evaluator will throw an
exception if an attempt is made to access any file outside of the Nix
search path. This is primarily intended for Hydra, where we don't want
people doing ‘builtins.readFile ~/.ssh/id_dsa’ or stuff like that.
2015-02-23 15:54:31 +01:00
Eelco Dolstra 2be7f79fd4 Remove tab 2015-02-05 17:21:30 +01:00
Shea Levy 73bf32ce94 Merge remote-tracking branch 'shlevy/baseNameOf-no-copy'
baseNameOf: Don't copy paths to the store first
2015-01-29 03:29:09 -05:00
Eelco Dolstra 2a3b1df423 Fix builtins.readDir on XFS
The DT_UNKNOWN fallback code was getting the type of the wrong path,
causing readDir to report "directory" as the type of every file.

Reported by deepfire on IRC.
2015-01-09 14:56:25 +01:00
Eelco Dolstra 153a943de7 Show position info for failing <...> lookups 2015-01-07 13:43:55 +01:00
Eelco Dolstra 8aedaf111e Remove canary stuff 2014-12-12 10:59:50 +01:00
Shea Levy 50c3352811 builtins.readFile: realise context associated with the path 2014-12-10 12:26:53 +01:00
Shea Levy 320659b0cd Allow external code using libnixexpr to add types
Code that links to libnixexpr (e.g. plugins loaded with importNative, or
nix-exec) may want to provide custom value types and operations on
values of those types. For example, nix-exec is currently using sets
where a custom IO value type would be more appropriate. This commit
provides a generic hook for such types in the form of tExternal and the
ExternalBase virtual class, which contains all functions necessary for
libnixexpr's type-polymorphic functions (e.g. `showType`) to be
implemented.
2014-12-02 10:27:04 -05:00
Eelco Dolstra 976df480c9 Add a primop for regular expression pattern matching
The function ‘builtins.match’ takes a POSIX extended regular
expression and an arbitrary string. It returns ‘null’ if the string
does not match the regular expression. Otherwise, it returns a list
containing substring matches corresponding to parenthesis groups in
the regex. The regex must match the entire string (i.e. there is an
implied "^<pat>$" around the regex).  For example:

  match "foo" "foobar" => null
  match "foo" "foo" => []
  match "f(o+)(.*)" "foooobar" => ["oooo" "bar"]
  match "(.*/)?([^/]*)" "/dir/file.nix" => ["/dir/" "file.nix"]
  match "(.*/)?([^/]*)" "file.nix" => [null "file.nix"]

The following example finds all regular files with extension .nix or
.patch underneath the current directory:

  let

    findFiles = pat: dir: concatLists (mapAttrsToList (name: type:
      if type == "directory" then
        findFiles pat (dir + "/" + name)
      else if type == "regular" && match pat name != null then
        [(dir + "/" + name)]
      else []) (readDir dir));

  in findFiles ".*\\.(nix|patch)" (toString ./.)
2014-11-25 11:47:06 +01:00
Eelco Dolstra 4e340a983f forceString(): Accept pos argument 2014-11-25 10:23:36 +01:00
Shea Levy b0c5c2ac34 import derivation: cleanup
Before this there was a bug where a `find` was being called on a
not-yet-sorted set. The code was just a mess before anyway, so I cleaned
it up while fixing it.
2014-11-20 22:48:12 -05:00
Shea Levy 2719627bbe realiseContext: Handle all context types
Avoids an assertion
2014-11-15 21:43:51 -05:00
Shea Levy 0ee1ca628a baseNameOf: Don't copy paths to the store first 2014-10-18 20:28:28 -04:00
Shea Levy d16e3c7f09 Export realiseContext in libnixexpr
Useful for importNative plugins
2014-10-17 22:15:09 -04:00
Eelco Dolstra c3f0a489f9 Add primop ‘catAttrs’ 2014-10-04 18:15:03 +02:00
Eelco Dolstra d4fcbe1687 Add primop ‘attrValues’ 2014-10-04 16:41:24 +02:00
Eelco Dolstra 58d8a213b0 Tweak 2014-10-04 11:27:23 +02:00
Eelco Dolstra 3f8576a6ab Remove some duplicate code 2014-10-03 22:37:51 +02:00
Shea Levy c08c802bf3 Add readDir primop 2014-10-03 22:32:11 +02:00
Eelco Dolstra ebb1dbb3e1 Add missing static 2014-09-23 15:08:27 +02:00
Eelco Dolstra 53b044c2f6 Don't evaluate inside a "throw"
Workaround for
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=41174. This caused
hydra-eval-jobs to ignore SIGINT.
2014-09-22 19:18:05 +02:00
Eelco Dolstra 0cd6596b0e Add ‘deepSeq’ primop
Note that unlike ‘lib.deepSeq’ in Nixpkgs, this handles cycles.
2014-09-22 16:05:00 +02:00
Eelco Dolstra a54c263402 Add ‘seq’ primop 2014-09-22 16:05:00 +02:00
Eelco Dolstra eff120d1b9 Add a function ‘valueSize’
It returns the size of value, including all other values and
environments reachable from it. It is intended for debugging memory
consumption issues.
2014-09-22 16:05:00 +02:00
Eelco Dolstra 2d6cd8aafd attrNames: Don't allocate duplicates of the symbols 2014-09-19 18:11:46 +02:00
Eelco Dolstra ea525a261f Fix off-by-one 2014-09-19 18:08:14 +02:00
Eelco Dolstra 6e5b02bee4 Add some instrumentation for debugging GC leaks 2014-09-17 15:19:07 +02:00
Eelco Dolstra 11849a320e Use proper quotes everywhere 2014-08-20 18:03:48 +02:00
Eelco Dolstra 3d221a7bb1 Rename nixPath to __nixPath
The name ‘nixPath’ breaks existing code.
2014-07-30 11:28:39 +02:00
Eelco Dolstra 0e5d0c1543 Fix compilation error on some versions of GCC
src/libexpr/primops.cc:42:8: error: looser throw specifier for 'virtual nix::InvalidPathError::~InvalidPathError()'
src/libexpr/nixexpr.hh:12:1: error:   overriding 'virtual nix::EvalError::~EvalError() noexcept (true)'

http://hydra.nixos.org/build/12385750
2014-07-09 12:14:40 +02:00
Eelco Dolstra beaf3e90af Add builtin function ‘fromJSON’
Fixes #294.
2014-07-04 13:34:15 +02:00
Shea Levy d62f46e500 Only add the importNative primop if the allow-arbitrary-code-during-evaluation option is true (default false) 2014-06-24 10:50:03 -04:00
Shea Levy 5cd022d6c0 Add importNative primop
This can be used to import a dynamic shared object and return an
arbitrary value, including new primops. This can be used both to test
new primops without having to recompile nix every time, and to build
specialized primops that probably don't belong upstream (e.g. a function
that calls out to gpg to decrypt a nixops secret as-needed).

The imported function should initialize the Value & as needed. A single
import can define multiple values by creating an attrset or list, of
course.

An example initialization function might look like:

extern "C" void initialize(nix::EvalState & state, nix::Value & v)
{
    v.type = nix::tPrimOp;
    v.primOp = NEW nix::PrimOp(myFun, 1, state.symbols.create("myFun"));
}

Then `builtins.importNative ./example.so "initialize"` will evaluate to
the primop defined in the myFun function.
2014-06-17 12:08:01 -04:00
Eelco Dolstra 0960d674d4 Drop ImportError and FindError
We're not catching these anywhere.
2014-06-12 13:00:54 +02:00
Shea Levy 718f20da6d findFile: Realise the context of the path attributes 2014-06-12 12:57:14 +02:00
Shea Levy a8fb575c98 Share code between scopedImport and import
In addition to reducing duplication, this fixes both import from
derivation and import of derivation for scopedImport
2014-06-12 12:52:39 +02:00
Eelco Dolstra ee7fe64c0a == operator: Ignore string context
There really is no case I can think of where taking the context into
account is useful. Mostly it's just very inconvenient.
2014-06-10 14:02:56 +02:00
Eelco Dolstra becc2b0167 Sort nixPath attributes 2014-05-29 19:02:14 +02:00
Eelco Dolstra 62a6eeb1f3 Make the Nix search path declarative
Nix search path lookups like <nixpkgs> are now desugared to ‘findFile
nixPath <nixpkgs>’, where ‘findFile’ is a new primop. Thus you can
override the search path simply by saying

  let
    nixPath = [ { prefix = "nixpkgs"; path = "/my-nixpkgs"; } ];
  in ... <nixpkgs> ...

In conjunction with ‘scopedImport’ (commit
c273c15cb1), the Nix search path can be
propagated across imports, e.g.

  let

    overrides = {
      nixPath = [ ... ] ++ builtins.nixPath;
      import = fn: scopedImport overrides fn;
      scopedImport = attrs: fn: scopedImport (overrides // attrs) fn;
      builtins = builtins // overrides;
    };

  in scopedImport overrides ./nixos
2014-05-26 17:02:22 +02:00
Eelco Dolstra a8edf185a9 Add constant ‘nixPath’
It contains the Nix expression search path as a list of { prefix, path
} sets, e.g.

  [ { path = "/nix/var/nix/profiles/per-user/root/channels/nixos"; prefix = ""; }
    { path = "/etc/nixos/configuration.nix"; prefix = "nixos-config"; }
    { path = "/home/eelco/Dev/nix/inst/share/nix/corepkgs"; prefix = "nix"; }
  ]
2014-05-26 14:55:47 +02:00
Eelco Dolstra c273c15cb1 Add primop ‘scopedImport’
‘scopedImport’ works like ‘import’, except that it takes a set of
attributes to be added to the lexical scope of the expression,
essentially extending or overriding the builtin variables.  For
instance, the expression

  scopedImport { x = 1; } ./foo.nix

where foo.nix contains ‘x’, will evaluate to 1.

This has a few applications:

* It allows getting rid of function argument specifications in package
  expressions. For instance, a package expression like:

    { stdenv, fetchurl, libfoo }:

    stdenv.mkDerivation { ... buildInputs = [ libfoo ]; }

  can now we written as just

    stdenv.mkDerivation { ... buildInputs = [ libfoo ]; }

  and imported in all-packages.nix as:

    bar = scopedImport pkgs ./bar.nix;

  So whereas we once had dependencies listed in three places
  (buildInputs, the function, and the call site), they now only need
  to appear in one place.

* It allows overriding builtin functions. For instance, to trace all
  calls to ‘map’:

  let
    overrides = {
      map = f: xs: builtins.trace "map called!" (map f xs);

      # Ensure that our override gets propagated by calls to
      # import/scopedImport.
      import = fn: scopedImport overrides fn;

      scopedImport = attrs: fn: scopedImport (overrides // attrs) fn;

      # Also update ‘builtins’.
      builtins = builtins // overrides;
    };
  in scopedImport overrides ./bla.nix

* Similarly, it allows extending the set of builtin functions. For
  instance, during Nixpkgs/NixOS evaluation, the Nixpkgs library
  functions could be added to the default scope.

There is a downside: calls to scopedImport are not memoized, unlike
import. So importing a file multiple times leads to multiple parsings
/ evaluations. It would be possible to construct the AST only once,
but that would require careful handling of variables/environments.
2014-05-26 14:26:29 +02:00