Commit graph

466 commits

Author SHA1 Message Date
Eelco Dolstra a6ab3cf35f tests/remote-builds.nix: Don't try cache.nixos.org 2014-07-24 11:50:05 +02:00
Eelco Dolstra 276a40b31f Handle case collisions on case-insensitive systems
When running NixOps under Mac OS X, we need to be able to import store
paths built on Linux into the local Nix store. However, HFS+ is
usually case-insensitive, so if there are directories with file names
that differ only in case, then importing will fail.

The solution is to add a suffix ("~nix~case~hack~<integer>") to
colliding files. For instance, if we have a directory containing
xt_CONNMARK.h and xt_connmark.h, then the latter will be renamed to
"xt_connmark.h~nix~case~hack~1". If a store path is dumped as a NAR,
the suffixes are removed. Thus, importing and exporting via a
case-insensitive Nix store is round-tripping. So when NixOps calls
nix-copy-closure to copy the path to a Linux machine, you get the
original file names back.

Closes #119.
2014-07-16 16:02:05 +02:00
Eelco Dolstra fa13d3f4f3 build-remote.pl: Fix building multiple output derivations
We were importing paths without sorting them topologically, leading to
"path is not valid" errors.

See e.g. http://hydra.nixos.org/build/12451761
2014-07-14 12:19:27 +02:00
Eelco Dolstra 838138c5c4 Fix test 2014-07-11 16:22:24 +02:00
Eelco Dolstra 66dbc0fdee Add a test for the SSH substituter 2014-07-10 01:53:56 +02:00
Eelco Dolstra beaf3e90af Add builtin function ‘fromJSON’
Fixes #294.
2014-07-04 13:34:15 +02:00
Eelco Dolstra ee7fe64c0a == operator: Ignore string context
There really is no case I can think of where taking the context into
account is useful. Mostly it's just very inconvenient.
2014-06-10 14:02:56 +02:00
Eelco Dolstra ceed819284 Fix test 2014-05-29 19:04:27 +02:00
Eelco Dolstra 62a6eeb1f3 Make the Nix search path declarative
Nix search path lookups like <nixpkgs> are now desugared to ‘findFile
nixPath <nixpkgs>’, where ‘findFile’ is a new primop. Thus you can
override the search path simply by saying

  let
    nixPath = [ { prefix = "nixpkgs"; path = "/my-nixpkgs"; } ];
  in ... <nixpkgs> ...

In conjunction with ‘scopedImport’ (commit
c273c15cb1), the Nix search path can be
propagated across imports, e.g.

  let

    overrides = {
      nixPath = [ ... ] ++ builtins.nixPath;
      import = fn: scopedImport overrides fn;
      scopedImport = attrs: fn: scopedImport (overrides // attrs) fn;
      builtins = builtins // overrides;
    };

  in scopedImport overrides ./nixos
2014-05-26 17:02:22 +02:00
Eelco Dolstra 39d72640c2 Ensure that -I flags get included in nixPath
Also fixes #261.
2014-05-26 16:52:31 +02:00
Eelco Dolstra a8edf185a9 Add constant ‘nixPath’
It contains the Nix expression search path as a list of { prefix, path
} sets, e.g.

  [ { path = "/nix/var/nix/profiles/per-user/root/channels/nixos"; prefix = ""; }
    { path = "/etc/nixos/configuration.nix"; prefix = "nixos-config"; }
    { path = "/home/eelco/Dev/nix/inst/share/nix/corepkgs"; prefix = "nix"; }
  ]
2014-05-26 14:55:47 +02:00
Eelco Dolstra c273c15cb1 Add primop ‘scopedImport’
‘scopedImport’ works like ‘import’, except that it takes a set of
attributes to be added to the lexical scope of the expression,
essentially extending or overriding the builtin variables.  For
instance, the expression

  scopedImport { x = 1; } ./foo.nix

where foo.nix contains ‘x’, will evaluate to 1.

This has a few applications:

* It allows getting rid of function argument specifications in package
  expressions. For instance, a package expression like:

    { stdenv, fetchurl, libfoo }:

    stdenv.mkDerivation { ... buildInputs = [ libfoo ]; }

  can now we written as just

    stdenv.mkDerivation { ... buildInputs = [ libfoo ]; }

  and imported in all-packages.nix as:

    bar = scopedImport pkgs ./bar.nix;

  So whereas we once had dependencies listed in three places
  (buildInputs, the function, and the call site), they now only need
  to appear in one place.

* It allows overriding builtin functions. For instance, to trace all
  calls to ‘map’:

  let
    overrides = {
      map = f: xs: builtins.trace "map called!" (map f xs);

      # Ensure that our override gets propagated by calls to
      # import/scopedImport.
      import = fn: scopedImport overrides fn;

      scopedImport = attrs: fn: scopedImport (overrides // attrs) fn;

      # Also update ‘builtins’.
      builtins = builtins // overrides;
    };
  in scopedImport overrides ./bla.nix

* Similarly, it allows extending the set of builtin functions. For
  instance, during Nixpkgs/NixOS evaluation, the Nixpkgs library
  functions could be added to the default scope.

There is a downside: calls to scopedImport are not memoized, unlike
import. So importing a file multiple times leads to multiple parsings
/ evaluations. It would be possible to construct the AST only once,
but that would require careful handling of variables/environments.
2014-05-26 14:26:29 +02:00
Eelco Dolstra 3064a82156 Disable parallel.sh test
It breaks randomly: http://hydra.nixos.org/build/11152871
2014-05-22 11:38:50 +02:00
Eelco Dolstra 6dd1087396 Fix Debian tests
These actually run as root in a VM, so they get confused.

http://hydra.nixos.org/build/10775854
2014-05-02 19:02:10 +02:00
Eelco Dolstra fb5d76b89e Fix test evaluation 2014-04-15 15:32:27 +02:00
Shea Levy 908e9ce259 If a dynamic attribute name evaluates to null, remove it from the set 2014-03-10 10:14:50 +01:00
Eelco Dolstra 7c7707638a Doh 2014-02-26 22:41:29 +01:00
Eelco Dolstra 7880973827 Test trace and addErrorContext 2014-02-26 19:12:31 +01:00
Eelco Dolstra 5ad263c26b Test some more primops 2014-02-26 19:08:44 +01:00
Eelco Dolstra 3d0a9ec825 Test executables in NARs 2014-02-26 18:59:01 +01:00
Eelco Dolstra d58ceae022 Test nix-env --switch-generation 2014-02-26 18:45:26 +01:00
Eelco Dolstra 7bbc68fdff Test nix-env --set 2014-02-26 18:42:19 +01:00
Eelco Dolstra a0806389e9 Test the -b and -s flags of nix-store -q 2014-02-26 18:40:08 +01:00
Eelco Dolstra a9c4a98770 Test ~/.nix-defexpr 2014-02-26 18:33:13 +01:00
Eelco Dolstra 045d3b2ed7 Test nix-store --switch-profile and more daemon actions 2014-02-26 18:28:55 +01:00
Eelco Dolstra fac6f8aac0 Test nix-store -q --roots 2014-02-26 18:00:46 +01:00
Eelco Dolstra 84143c4bd8 Test nix-store -l 2014-02-26 17:58:53 +01:00
Eelco Dolstra 19437785eb Test nix-store --optimise 2014-02-26 17:53:51 +01:00
Eelco Dolstra fdff3a7eae Add a test for nix-store --dump-db / --load-db 2014-02-26 17:47:54 +01:00
Eelco Dolstra e1cf40fa95 nix-instantiate: Rename --eval-only to --eval, --parse-only to --parse 2014-02-19 16:34:24 +01:00
Eelco Dolstra 4ec626a286 Test nix-store --verify-path and --repair-path 2014-02-17 23:24:12 +01:00
Eelco Dolstra 832377bbd6 Add a test for repairing paths 2014-02-17 12:22:50 +01:00
Eelco Dolstra 1f94ec3924 Clean up a test warning 2014-02-06 13:54:44 +01:00
Eelco Dolstra 20d0598928 Drop dependency on ‘expr’
http://hydra.nixos.org/build/8715639

Not sure why this causes a failure now.
2014-02-06 13:51:57 +01:00
Eelco Dolstra 2a97f7b039 Fix logging test 2014-02-01 16:08:59 +01:00
Eelco Dolstra 762ef464f8 Fix the nix-profile test 2014-02-01 15:37:50 +01:00
Eelco Dolstra 0c6d62cf27 Remove Automakefiles 2014-02-01 13:54:38 +01:00
Eelco Dolstra 16e7d69209 Update Makefile variable names 2014-02-01 13:54:38 +01:00
Eelco Dolstra e0234dfddc Rename Makefile -> local.mk 2014-01-30 12:11:06 +01:00
Eelco Dolstra 81628a6ccc Merge branch 'master' into make
Conflicts:
	src/libexpr/eval.cc
2014-01-21 15:30:01 +01:00
Shea Levy f9913f4422 Allow "bare" dynamic attrs
Now, in addition to a."${b}".c, you can write a.${b}.c (applicable
wherever dynamic attributes are valid).

Signed-off-by: Shea Levy <shea@shealevy.com>
2014-01-14 14:00:15 +01:00
Eelco Dolstra f4013b6189 Fix signed-binary-caches test 2014-01-08 17:57:22 +01:00
Eelco Dolstra ea38e39a20 Test whether Nix correctly checks the hash of downloaded NARs 2014-01-08 17:56:30 +01:00
Eelco Dolstra 0fdf4da0e9 Support cryptographically signed binary caches
NAR info files in binary caches can now have a cryptographic signature
that Nix will verify before using the corresponding NAR file.

To create a private/public key pair for signing and verifying a binary
cache, do:

  $ openssl genrsa -out ./cache-key.sec 2048
  $ openssl rsa -in ./cache-key.sec -pubout > ./cache-key.pub

You should also come up with a symbolic name for the key, such as
"cache.example.org-1".  This will be used by clients to look up the
public key.  (It's a good idea to number keys, in case you ever need
to revoke/replace one.)

To create a binary cache signed with the private key:

  $ nix-push --dest /path/to/binary-cache --key ./cache-key.sec --key-name cache.example.org-1

The public key (cache-key.pub) should be distributed to the clients.
They should have a nix.conf should contain something like:

  signed-binary-caches = *
  binary-cache-public-key-cache.example.org-1 = /path/to/cache-key.pub

If all works well, then if Nix fetches something from the signed
binary cache, you will see a message like:

  *** Downloading ‘http://cache.example.org/nar/7dppcj5sc1nda7l54rjc0g5l1hamj09j-subversion-1.7.11’ (signed by ‘cache.example.org-1’) to ‘/nix/store/7dppcj5sc1nda7l54rjc0g5l1hamj09j-subversion-1.7.11’...

On the other hand, if the signature is wrong, you get a message like

  NAR info file `http://cache.example.org/7dppcj5sc1nda7l54rjc0g5l1hamj09j.narinfo' has an invalid signature; ignoring

Signatures are implemented as a single line appended to the NAR info
file, which looks like this:

  Signature: 1;cache.example.org-1;HQ9Xzyanq9iV...muQ==

Thus the signature has 3 fields: a version (currently "1"), the ID of
key, and the base64-encoded signature of the SHA-256 hash of the
contents of the NAR info file up to but not including the Signature
line.

Issue #75.
2014-01-08 15:42:53 +01:00
Eelco Dolstra 7a61c88dbb Merge branch 'dynamic-attrs-no-sugar' of github.com:shlevy/nix 2014-01-06 15:46:18 +01:00
Eelco Dolstra 4252b5a645 Disable the tail call test
On i686-linux, GCC stubbornly refuses to do tail-call optimisation.
Don't know why.

http://hydra.nixos.org/build/7300170
2014-01-06 11:32:22 +01:00
Shea Levy 6f3a51809a Fold dynamic binds handling into addAttr
Since addAttr has to iterate through the AttrPath we pass it, it makes
more sense to just iterate through the AttrNames in addAttr instead. As
an added bonus, this allows attrsets where two dynamic attribute paths
have the same static leading part (see added test case for an example
that failed previously).

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-12-31 17:57:10 -05:00
Shea Levy 18fefacf7d Dynamic attrs
This adds new syntax for attribute names:

* attrs."${name}" => getAttr name attrs
* attrs ? "${name}" => isAttrs attrs && hasAttr attrs name
* attrs."${name}" or def => if attrs ? "${name}" then attrs."${name}" else def
* { "${name}" = value; } => listToAttrs [{ inherit name value; }]

Of course, it's a bit more complicated than that. The attribute chains
can be arbitrarily long and contain combinations of static and dynamic
parts (e.g. attrs."${foo}".bar."${baz}" or qux), which is relatively
straightforward for the getAttrs/hasAttrs cases but is more complex for
the listToAttrs case due to rules about duplicate attribute definitions.

For attribute sets with dynamic attribute names, duplicate static
attributes are detected at parse time while duplicate dynamic attributes
are detected when the attribute set is forced. So, for example, { a =
null; a.b = null; "${"c"}" = true; } will be a parse-time error, while
{ a = {}; "${"a"}".b = null; c = true; } will be an eval-time error
(technically that case could theoretically be detected at parse time,
but the general case would require full evaluation). Moreover, duplicate
dynamic attributes are not allowed even in cases where they would be
with static attributes ({ a.b.d = true; a.b.c = false; } is legal, but {
a."${"b"}".d = true; a."${"b"}".c = false; } is not). This restriction
might be relaxed in the future in cases where the static variant would
not be an error, but it is not obvious that that is desirable.

Finally, recursive attribute sets with dynamic attributes have the
static attributes in scope but not the dynamic ones. So rec { a = true;
"${"b"}" = a; } is equivalent to { a = true; b = true; } but rec {
"${"a"}" = true; b = a; } would be an error or use a from the
surrounding scope if it exists.

Note that the getAttr, getAttr or default, and hasAttr are all
implemented purely in the parser as syntactic sugar, while attribute
sets with dynamic attribute names required changes to the AST to be
implemented cleanly.

This is an alternative solution to and closes #167

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-12-31 20:59:49 +00:00
Shea Levy 136f2f7046 Add the ExprBuiltin Expr type to the AST
Certain desugaring schemes may require the parser to use some builtin
function to do some of the work (e.g. currently `throw` is used to
lazily cause an error if a `<>`-style path is not in the search path)
Unfortunately, these names are not reserved keywords, so an expression
that uses such a syntactic sugar will not see the expected behavior
(see tests/lang/eval-okay-redefine-builtin.nix for an example).

This adds the ExprBuiltin AST type, which when evaluated uses the value
from the rootmost variable scope (which of course is initialized
internally and can't shadow any of the builtins).

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-12-31 17:45:47 +00:00
Eelco Dolstra 0202ce6b94 Add support for ‘make installcheck’ 2013-11-25 18:47:03 +01:00
Eelco Dolstra 77c13cdf56 Add a toJSON primop 2013-11-19 00:04:11 +01:00
Eelco Dolstra 285df765b9 Add a primop unsafeGetAttrPos to return the position of an attribute 2013-11-18 22:22:35 +01:00
Eelco Dolstra fc33fd86b7 Add a symbol __curPos that expands to the current source location
I.e. an attribute set { file = <string>; line = <int>; column = <int>; }.
2013-11-18 20:16:02 +01:00
Eelco Dolstra 90b5e69284 Support quoted attribute names in -A
This is requires if you have attribute names with dots in them.  So
you can now say:

  $ nix-instantiate '<nixos>' -A 'config.systemd.units."postgresql.service".text' --eval-only

Fixes #151.
2013-11-18 10:21:12 +00:00
Eelco Dolstra 2bcb384e95 Add a test to check that tail calls run in bounded stack space 2013-11-12 12:34:22 +01:00
Eelco Dolstra 5bc41d78ff Rename "attribute sets" to "sets"
We don't have any other kind of sets so calling them attribute sets is
unnecessarily verbose.
2013-10-24 16:41:04 +02:00
Eelco Dolstra 411a3461dc Add a test of the type primops 2013-10-24 02:51:28 +02:00
Eelco Dolstra f6a8e7f4c2 Fix test 2013-10-17 11:18:37 +02:00
Eelco Dolstra b08f4b0da9 Test string semantics a bit more 2013-10-17 01:12:43 +02:00
Eelco Dolstra 9d8a80375d Add a test for type correctness of antiquotes
Antiquotes should evaluate to strings or paths.  This is usually
checked, except in the case where the antiquote makes up the entire
string, as in "${expr}".  This is optimised to expr, which discards
the runtime type checks / coercions.
2013-10-17 00:51:07 +02:00
Eelco Dolstra b8571d68c4 Add a regression test for correct path antiquotation behavior
This broke in Nix 1.6.
2013-10-16 23:29:11 +02:00
Eelco Dolstra 672c3acc71 Adjust to the NixOS/Nixpkgs merge 2013-10-11 10:57:23 +02:00
Eelco Dolstra 6ec8dab06a Adda test for build-max-log-size 2013-09-02 12:44:30 +02:00
Shea Levy afc6c1bad6 Simplify inherited attribute handling
This reduces the difference between inherited and non-inherited
attribute handling to the choice of which env to use (in recs and lets)
by setting the AttrDef::e to a new ExprVar in the parser rather than
carrying a separate AttrDef::v VarRef member.

As an added bonus, this allows inherited attributes that inherit from a
with to delay forcing evaluation of the with's attributes.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-08-26 11:31:56 +02:00
Ivan Kozik 34bb806f74 Fix typos, especially those that end up in the Nix manual 2013-08-26 11:15:22 +02:00
Eelco Dolstra 8e74c0bfd1 Let the ordering operators also work on strings
E.g. ‘"foo" < "bar"’ now works.
2013-08-02 18:53:02 +02:00
Eelco Dolstra 3d77b28eac Add comparison operators ‘<’, ‘<=’, ‘>’ and ‘>=’ 2013-08-02 18:39:40 +02:00
Eelco Dolstra 47701677e8 Add integer ‘-’, ‘*’ and ‘/’ operators 2013-08-02 16:03:02 +00:00
Eelco Dolstra 5d147e125c Add a unary integer negation operator
This allows saying "-1" instead of "builtins.sub 0 1".
2013-08-02 15:43:31 +00:00
Eelco Dolstra 159e621d1a Overload the ‘+’ operator to support integer addition 2013-08-02 15:21:17 +00:00
Eelco Dolstra 7df4ef983e Test the delayed with a bit more 2013-07-31 13:12:35 +02:00
Shea Levy 20866a7031 Delay evaulation of with attrs until a variable lookup needs them
Evaluation of attribute sets is strict in the attribute names, which
means immediate evaluation of `with` attribute sets rules out some
potentially interesting use cases (e.g. where the attribute names of one
set depend in some way on another but we want to bring those names into
scope for some values in the second set).

The major example of this is overridable self-referential package sets
(e.g. all-packages.nix). With immediate `with` evaluation, the only
options for such sets are to either make them non-recursive and
explicitly use the name of the overridden set in non-overridden one
every time you want to reference another package, or make the set
recursive and use the `__overrides` hack. As shown in the test case that
comes with this commit, though, delayed `with` evaluation allows a nicer
third alternative.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-31 11:48:39 +02:00
Eelco Dolstra 5116214343 Add support for uncompressed NARs in binary caches
Issue NixOS/hydra#102.
2013-07-01 21:03:14 +02:00
Eelco Dolstra 22144afa8d Don't keep "disabled" substituters running
For instance, it's pointless to keep copy-from-other-stores running if
there are no other stores, or download-using-manifests if there are no
manifests.  This also speeds things up because we don't send queries
to those substituters.
2013-06-20 11:55:15 +02:00
Eelco Dolstra f9974f856e Show that --timeout doesn't work if the build produces log output 2013-04-23 17:16:29 +02:00
Eelco Dolstra a9b4e26b5c Test whether --fallback works if NARS have disappeared from the binary cache 2013-04-23 12:44:01 +02:00
Eelco Dolstra c642441beb Test NAR info caching 2013-04-23 12:43:28 +02:00
Eelco Dolstra bdd4646338 Revert "Prevent config.h from being clobbered"
This reverts commit 28bba8c44f.
2013-03-08 01:24:59 +01:00
Eelco Dolstra 28bba8c44f Prevent config.h from being clobbered 2013-03-07 23:55:55 +01:00
Eelco Dolstra 52172607cf Rename "hash" to "hashString" and handle SHA-1 2013-02-08 19:36:23 +01:00
Marc Weber 01a5ea9914 experimental/hash
adding primop function calculating hash of a string

Signed-off-by: Marc Weber <marco-oweber@gmx.de>
2013-02-08 19:26:34 +01:00
Eelco Dolstra 1943b60ad8 Fix the VM tests 2013-01-21 22:36:23 +01:00
Eelco Dolstra 0a4e90395c Urgggh
http://hydra.nixos.org/build/3661100
2013-01-02 23:52:15 +01:00
Eelco Dolstra 299141ecbd If a substitute closure is incomplete, build dependencies, then retry the substituter
Issue #77.
2013-01-02 12:38:28 +01:00
Eelco Dolstra 82248abd8f Add a test for incomplete closures in the binary cache
Issue #77.
2013-01-02 11:45:23 +01:00
Eelco Dolstra b215b23e9e Test priorities 2012-12-04 14:47:50 +01:00
Eelco Dolstra 2d5e8e267b Add a test for ‘nix-env --set-flag active ...’ 2012-12-04 14:30:34 +01:00
Eelco Dolstra 21c2d8d102 Test the ‘--prebuilt-only’ flag 2012-12-03 21:02:06 +01:00
Eelco Dolstra 69c88f5028 Fix the multiple-outputs test 2012-11-26 17:46:45 +01:00
Eelco Dolstra f766e146f4 Fix the test 2012-10-03 18:01:35 -04:00
Eelco Dolstra e35d6f78dc Rename nix-worker to nix-daemon 2012-10-03 17:59:23 -04:00
Eelco Dolstra 522ecab9b8 Drop support for running nix-worker in "slave" mode
AFAIK nobody uses this, setuid binaries are evil, and there is no good
reason why people can't just run the daemon.
2012-10-03 17:30:45 -04:00
Eelco Dolstra 95c74eae26 Allow dashes in identifiers
In Nixpkgs, the attribute in all-packages.nix corresponding to a
package is usually equal to the package name.  However, this doesn't
work if the package contains a dash, which is fairly common.  The
convention is to replace the dash with an underscore (e.g. "dbus-lib"
becomes "dbus_glib"), but that's annoying.  So now dashes are valid in
variable / attribute names, allowing you to write:

  dbus-glib = callPackage ../development/libraries/dbus-glib { };

and

  buildInputs = [ dbus-glib ];

Since we don't have a negation or subtraction operation in Nix, this
is unambiguous.
2012-09-27 15:49:20 -04:00
Eelco Dolstra 09eb230900 Fix test
http://hydra.nixos.org/build/3031382
2012-09-14 09:39:54 -04:00
Eelco Dolstra 1bda006b74 Add a test for nix-profile.sh 2012-09-13 17:58:14 -04:00
Eelco Dolstra 31114ec3a5 Fix the store optimisation test
http://hydra.nixos.org/build/3026118
2012-09-13 12:54:23 -04:00
Eelco Dolstra 4fca02077c Handle gc-keep-outputs and gc-keep-derivations both enabled
If the options gc-keep-outputs and gc-keep-derivations are both
enabled, you can get a cycle in the liveness graph.  There was a hack
to handle this, but it didn't work with multiple-output derivations,
causing the garbage collector to fail with errors like ‘error: cannot
delete path `...' because it is in use by `...'’.  The garbage
collector now handles strongly connected components in the liveness
graph as a unit and decides whether to delete all or none of the paths
in an SCC.
2012-09-12 18:49:35 -04:00
Eelco Dolstra 4ba47205c6 Fix test failure on Darwin
Apparently our DBD::SQLite links against /usr/lib/libsqlite3.dylib,
which is an old version that doesn't respect foreign key constraints.
So manifests/cache.sqlite doesn't get updated properly when a manifest
disappears.  We should fix our DBD::SQLite, but in the meantime this
will fix the test.

http://hydra.nixos.org/build/3017959
2012-09-12 11:29:10 -04:00
Eelco Dolstra d4c3b6327f Don't put results symlinks in the tests directory 2012-09-11 19:14:15 -04:00
Eelco Dolstra 1f7901ec3b Test hash rewriting 2012-09-11 19:10:11 -04:00