Commit graph

450 commits

Author SHA1 Message Date
Eelco Dolstra d329c3ea9d * Support multiple outputs. A derivation can declare multiple outputs
by setting the ‘outputs’ attribute.  For example:

    stdenv.mkDerivation {
      name = "aterm-2.5";

      src = ...;

      outputs = [ "out" "tools" "dev" ];

      configureFlags = "--bindir=$(tools)/bin --includedir=$(dev)/include";
    }

  This derivation creates three outputs, named like this:

    /nix/store/gcnqgllbh01p3d448q8q6pzn2nc2gpyl-aterm-2.5
    /nix/store/gjf1sgirwfnrlr0bdxyrwzpw2r304j02-aterm-2.5-tools
    /nix/store/hp6108bqfgxvza25nnxfs7kj88xi2vdx-aterm-2.5-dev

  That is, the symbolic name of the output is suffixed to the store
  path (except for the ‘out’ output).  Each path is passed to the
  builder through the corresponding environment variable, e.g.,
  ${tools}.

  The main reason for multiple outputs is to allow parts of a package
  to be distributed and garbage-collected separately.  For instance,
  most packages depend on Glibc for its libraries, but don't need its
  header files.  If these are separated into different store paths,
  then a package that depends on the Glibc libraries only causes the
  libraries and not the headers to be downloaded.

  The main problem with multiple outputs is that if one output exists
  while the others have been garbage-collected (or never downloaded in
  the first place), and we want to rebuild the other outputs, then
  this isn't possible because we can't clobber a valid output (it
  might be in active use).  This currently gives an error message
  like:

    error: derivation `/nix/store/1s9zw4c8qydpjyrayxamx2z7zzp5pcgh-aterm-2.5.drv' is blocked by its output paths

  There are two solutions: 1) Do the build in a chroot.  Then we don't
  need to overwrite the existing path.  2) Use hash rewriting (see the
  ASE-2005 paper).  Scary but it should work.

  This is not finished yet.  There is not yet an easy way to refer to
  non-default outputs in Nix expressions.  Also, mutually recursive
  outputs aren't detected yet and cause the garbage collector to
  crash.
2011-07-18 23:31:03 +00:00
Eelco Dolstra d5d4dcd4c9 * Allow attribute names to be strings. Based on the
allow-arbitrary-strinsg-in-names patch by Marc Weber.
2011-07-13 15:53:24 +00:00
Eelco Dolstra 0a623a10c7 * Allow a default value in attribute selection by writing
x.y.z or default

  (as originally proposed in
  https://mail.cs.uu.nl/pipermail/nix-dev/2009-September/002989.html).

  For instance, an expression like

    stdenv.lib.attrByPath ["features" "ckSched"] false args

  can now be written as

    args.features.ckSched or false
2011-07-13 12:19:57 +00:00
Eelco Dolstra 2b9e29b1c8 * Change the right-hand side of the ‘.’ operator from an attribute to
an attribute path.  This is a refactoring to support default values.
2011-07-06 12:28:57 +00:00
Eelco Dolstra 5637037802 * In the ‘?’ operator, allow attribute paths. For instance, you can
write ‘attrs ? a.b’ to test whether ‘attrs’ has an attribute ‘a’
  containing an attribute ‘b’.  This is more convenient than ‘attrs ?
  a && attrs.a ? b’.

  Slight change in the semantics: it's no longer an error if the
  left-hand side of ‘?’ is not an attribute set.  In that case it just
  returns false.  So, ‘null ? foo’ no longer throws an error.
2011-07-06 10:58:17 +00:00
Eelco Dolstra 538b7caab0 * Don't allocate a big initial GC address space on machines with
little RAM.  Even if the memory isn't actually used, it can cause
  problems with the overcommit heuristics in the kernel.  So use a VM
  space of 25% of RAM, up to 384 MB.
2011-02-10 14:31:04 +00:00
Eelco Dolstra 5a6b039802 * Don't call GC_expand_hp unless we're actually using the garbage
collector.
2011-02-09 22:59:50 +00:00
Ludovic Courtès f1462c208b Use $BDW_GC_LIBS instead of a custom variable. 2011-02-09 21:23:13 +00:00
Eelco Dolstra 9db190eb31 * builtins.substring: if "start" is beyond the end of the string,
return the empty string.
2011-01-14 12:47:10 +00:00
Eelco Dolstra 812fae424e * DrvInfo contains pointers to expressions, so DrvInfos should be
traced by the garbage collector.  Otherwise "nix-env -u" can crash
  randomly.
2010-11-25 13:47:34 +00:00
Eelco Dolstra 0c4828ea05 * new(UseGC) is inexplicably slower than GC_MALLOC, so prefer the
latter.
2010-10-28 12:50:01 +00:00
Eelco Dolstra e11e6fb1c6 * Handle out of memory condition. 2010-10-28 12:29:40 +00:00
Eelco Dolstra 43535499f3 * When allocating an attribute set, reserve enough space for all
elements.  This prevents the vector from having to resize itself.
2010-10-24 20:09:37 +00:00
Eelco Dolstra e0b7fb8f27 * Keep attribute sets in sorted order to speed up attribute lookups.
* Simplify the representation of attributes in the AST.
* Change the behaviour of listToAttrs() in case of duplicate names.
2010-10-24 19:52:33 +00:00
Eelco Dolstra 2dc6d50941 * Don't create thunks for variable lookups (if possible). This
significantly reduces the number of values allocated (e.g. from 8.7m
  to 4.9m for the Bittorrent test).
2010-10-24 14:20:02 +00:00
Eelco Dolstra 0b305c534f * Store attribute sets as a vector instead of a map (i.e. a red-black
tree).  This saves a lot of memory.  The vector should be sorted so
  that names can be looked up using binary search, but this is not the
  case yet.  (Surprisingly, looking up attributes using linear search
  doesn't have a big impact on performance.)

  Memory consumption for

    $ nix-instantiate /etc/nixos/nixos/tests -A bittorrent.test --readonly-mode

  on x86_64-linux with GC enabled is now 185 MiB (compared to 946
  MiB on the trunk).
2010-10-24 00:41:29 +00:00
Eelco Dolstra a247d20604 * Fix compiling without Boehm.
* Fix the stats.
2010-10-23 22:58:24 +00:00
Eelco Dolstra b2ba62170c * Optimise string constants by putting them in the symbol table. 2010-10-23 21:11:59 +00:00
Eelco Dolstra 8ac06726b9 * Make Value smaller by not storing redundant PrimOp info.
* Clear pointers in Values after overwriting them to make sure that no
  objects are kept alive unnecessarily.
2010-10-23 20:07:47 +00:00
Eelco Dolstra 3f66cfb96b * Remove allocValues(). 2010-10-23 18:18:07 +00:00
Eelco Dolstra 4dee289550 * In environments, store pointers to values rather than values. This
improves GC effectiveness a bit more (because a live value doesn't
  keep other values in the environment plus the parent environments
  alive), and removes the need for copy nodes.
2010-10-22 15:51:52 +00:00
Eelco Dolstra 41c45a9b31 * Store Value nodes outside of attribute sets. I.e., Attr now stores
a pointer to a Value, rather than the Value directly.  This improves
  the effectiveness of garbage collection a lot: if the Value is
  stored inside the set directly, then any live pointer to the Value
  causes all other attributes in the set to be live as well.
2010-10-22 14:47:42 +00:00
Eelco Dolstra 64c3325b0b * Make building against the Boehm GC a configure option. 2010-10-22 13:39:15 +00:00
Eelco Dolstra 76feaf016a * Keep some more stats. 2010-10-20 15:48:00 +00:00
Eelco Dolstra e879a0371b * Use the Boehm garbage collector to reclaim unused memory in the Nix
expression evaluator.
2010-10-20 11:38:30 +00:00
Eelco Dolstra 705868a8a9 * Make sure that config.h is included before the system headers,
because it defines _FILE_OFFSET_BITS.  Without this, on 
  OpenSolaris the system headers define it to be 32, and then 
  the 32-bit stat() ends up being called with a 64-bit "struct 
  stat", or vice versa.

  This also ensures that we get 64-bit file sizes everywhere.

* Remove the redundant call to stat() in parseExprFromFile().
  The file cannot be a symlink because that's the exit condition
  of the loop before.
2010-10-04 17:55:38 +00:00
Eelco Dolstra 95f4f2cf61 * If std::tr1::unordered_set is unavailable, use std::set. 2010-10-04 16:16:19 +00:00
Eelco Dolstra 4aa9245083 * Hack needed for GCC 4.3.2 on OpenSolaris. 2010-10-04 10:51:16 +00:00
Eelco Dolstra df50916e46 * Oops - "null" was displayed as "true". 2010-08-27 12:10:56 +00:00
Eelco Dolstra 6d6200f37a * Optimisation in the // operator: if one of the sets is empty, return
the other set.
2010-08-02 16:31:05 +00:00
Eelco Dolstra 7af6a2fd71 * intersectAttrs: optimise for the case where the second set is larger
than the first set.  (That's usually the case with callPackage.)
2010-08-02 11:54:44 +00:00
Eelco Dolstra f16fe2af8d * builtins.toXML: propagate the string context. This is a regression
from the old ATerm-based evaluator.
2010-06-10 10:29:50 +00:00
Eelco Dolstra 1ab67cf437 2010-06-02 09:43:04 +00:00
Eelco Dolstra 89865da76d * Turn build errors during evaluation into EvalErrors. 2010-06-01 11:19:32 +00:00
Eelco Dolstra 93cd5a4a13 * The << operator on values should be const. 2010-05-18 10:36:37 +00:00
Eelco Dolstra b2235d81d1 * Restore the __overrides feature that was lost somewhere in the
fast-eval branch.
2010-05-15 08:10:12 +00:00
Eelco Dolstra bd25ac2260 * Print attributes in sorted order. 2010-05-12 12:15:49 +00:00
Eelco Dolstra 81a4b4e49b * Implemented tryEval, the last missing primop in the fast-eval
branch.  Also added a test for tryEval.
2010-05-12 11:23:44 +00:00
Eelco Dolstra 1a8eb6e3ec 2010-05-07 15:26:33 +00:00
Eelco Dolstra 83dfa89870 * Sync with the trunk. 2010-05-07 14:46:47 +00:00
Eelco Dolstra 01e58adce0 * Store position info for inherited attributes. 2010-05-07 12:43:57 +00:00
Eelco Dolstra 83d7b89660 * Updated addErrorContext. 2010-05-07 12:33:14 +00:00
Eelco Dolstra e2d5e40f4f * Keep track of the source positions of attributes. 2010-05-07 12:11:05 +00:00
Eelco Dolstra 84ce7ac76f * Store attribute positions in the AST and report duplicate attribute
errors with position info.
* For all positions, use the position of the first character of the
  first token, rather than the last character of the first token plus
  one.
2010-05-06 16:46:48 +00:00
Eelco Dolstra 2be6118f4c * Don't need the test program anymore. 2010-04-23 09:09:32 +00:00
Eelco Dolstra 0bc468f195 * Simplify the implementation of `with'. This gives a 7% speedup in
evaluating the NixOS system configuration.
2010-04-22 15:08:09 +00:00
Eelco Dolstra ebade9ff8b * Check for duplicate attribute names / function arguments. `make
check' now succeeds :-)
* An attribute set such as `{ foo = { enable = true; };
  foo.port = 23; }' now parses.  It was previously rejected, but I'm
  too lazy to implement the check.  (The only reason to reject it is
  that the reverse, `{ foo.port = 23; foo = { enable = true; }; }', is
  rejected, which is kind of ugly.)
2010-04-22 11:02:24 +00:00
Eelco Dolstra 2d7636529f * String equality tests should take the context into account. All the
evaluation test cases now succeed.
2010-04-22 09:54:11 +00:00
Eelco Dolstra 6f0f16497a * Fix the interpretation of ''\<character> in indented strings. 2010-04-21 16:18:27 +00:00
Eelco Dolstra 0777448ca6 * Fixed builtins.genericClosure. 2010-04-21 15:57:11 +00:00
Eelco Dolstra fe2d869e04 * Store user environment manifests as a Nix expression in
$out/manifest.nix rather than as an ATerm.

  (Hm, I thought I committed this two days ago...)
2010-04-21 15:08:58 +00:00
Eelco Dolstra f3b8833a48 * Drop the dependency on the ATerm library. 2010-04-19 14:51:58 +00:00
Eelco Dolstra efc7a579e8 * Don't use the ATerm library for parsing/printing .drv files. 2010-04-19 13:46:58 +00:00
Eelco Dolstra 55b5ddd3ca * Added parsing of manifests in ATerm format. 2010-04-19 12:10:04 +00:00
Eelco Dolstra 5c31995bb8 * Updated some more primops. 2010-04-16 15:13:47 +00:00
Eelco Dolstra 8ca4a001cb * Improve sharing a bit. 2010-04-16 14:03:26 +00:00
Eelco Dolstra 497e4ad126 * Remove some redundant tests. 2010-04-16 13:51:01 +00:00
Eelco Dolstra 02c1dac909 * In an nested with' where the inner with is a variable (with ...;
with someVar; ...'), the contents of the variable would be
  clobbered.  (The attributes in the outer `with' were added to the
  variable.)
2010-04-16 13:44:02 +00:00
Eelco Dolstra 04c4bd3624 * Store lists as lists of pointers to values rather than as lists of
values.  This improves sharing and gives another speed up.
  Evaluation of the NixOS system attribute is now almost 7 times
  faster than the old evaluator.
2010-04-15 00:37:36 +00:00
Eelco Dolstra e41b5828db * Better stats. 2010-04-14 23:48:46 +00:00
Eelco Dolstra d39d3c6264 * Implemented inherit. 2010-04-14 23:25:05 +00:00
Eelco Dolstra 267dc693d2 * Fix builtins. 2010-04-14 22:59:39 +00:00
Eelco Dolstra 81de12bc8f * Refactoring: move variable uses to a separate class. 2010-04-14 15:14:23 +00:00
Eelco Dolstra 110d155778 * Implemented withs. 2010-04-14 15:01:04 +00:00
Eelco Dolstra 9985230c00 * After parsing, compute level/displacement pairs for each variable
use site, allowing environments to be stores as vectors of values
  rather than maps.  This should speed up evaluation and reduce the
  number of allocations.
2010-04-14 14:42:32 +00:00
Eelco Dolstra 816dd3f061 * Remove more obsolete code. 2010-04-14 12:49:05 +00:00
Eelco Dolstra 85d13c8f93 * Change the semantics of "with" so that inner "withs" take
precedence, i.e. `with {x=1;}; with {x=2;}; x' evaluates to 2'.
  This has a simpler implementation and seems more natural.  There
  doesn't seem to be any code in Nixpkgs or NixOS that relies on the
  old behaviour.
2010-04-14 08:37:08 +00:00
Eelco Dolstra 816f9c0f6f * Use std::tr1::unordered_set instead of std::set for the symbol
table.  This gives a 10% speed increase on `nix-instantiate
  /etc/nixos/nixos -A system --readonly-mode'.
2010-04-13 14:34:11 +00:00
Eelco Dolstra 7d47498b5e * Evaluate lets directly (i.e. without desugaring to `rec { attrs...;
<let-body> = e; }.<let-body>).  This prevents the unnecessary
  allocation of an attribute set.
2010-04-13 13:42:25 +00:00
Eelco Dolstra ac1e8f40d4 * Use a symbol table to represent identifiers and attribute names
efficiently.  The symbol table ensures that there is only one copy
  of each symbol, thus allowing symbols to be compared efficiently
  using a pointer equality test.
2010-04-13 12:25:42 +00:00
Eelco Dolstra 10e8b1fd15 * Finished the ATerm-less parser. 2010-04-12 23:33:23 +00:00
Eelco Dolstra d4f0b0fc6c * Indented strings. 2010-04-12 22:03:27 +00:00
Eelco Dolstra a60317f20f * More missing constructs. 2010-04-12 21:21:24 +00:00
Eelco Dolstra 4d6ad5be17 * Don't use ATerms for the abstract syntax trees anymore. Not
finished yet.
2010-04-12 18:30:11 +00:00
Eelco Dolstra ed711f73bc * Don't use ATerms to represent integers in the lexer. 2010-04-12 10:38:18 +00:00
Eelco Dolstra db90b88e65 * Hack to support builderDefs expressions. 2010-04-12 09:50:20 +00:00
Eelco Dolstra 4e49002576 * Doh. 2010-04-12 09:45:00 +00:00
Eelco Dolstra c3f228f296 2010-04-12 09:14:27 +00:00
Ludovic Courtès aac5fcfbb5 Re-add drvPath' and outPath' attributes to <derivation> XML nodes.
This fixes a regression introduced in r20882 ("Add source location
information to the XML output.").

* src/libexpr/expr-to-xml.cc (nix::printTermAsXML): Dereference the
  attribute RHS from "drvPath" and "outPath".
2010-04-09 21:30:55 +00:00
Eelco Dolstra f3dc7ab877 * Keep more statistics about stack space usage.
* Reduce stack space usage.
2010-04-09 12:00:49 +00:00
Eelco Dolstra b7b3dd55f9 * Remove a lot of dead code. 2010-04-08 11:41:19 +00:00
Eelco Dolstra 7e048eddf5 * Fix blackholing. If evaluation fails due to an assertion failure,
then the blackhole has to be removed to ensure that repeated
  evaluation of the same value gives an assertion failure again rather
  than an "infinite recursion" error.
2010-04-08 11:25:14 +00:00
Eelco Dolstra af2a372bb0 * Update autoCallFunction() and findAlongAttrPath(). 2010-04-07 15:47:06 +00:00
Eelco Dolstra 9a64454faa * expr-to-xml -> value-to-xml. 2010-04-07 13:59:45 +00:00
Eelco Dolstra fc92244ba8 * Implemented the primops necessary for generating the NixOS manual. 2010-04-07 13:55:46 +00:00
Eelco Dolstra a353aef0b1 * In eval(), don't use the target value `v' as a temporary.
Overwriting `v' breaks when the expression evaluation to an
  assertion failure or throw.
2010-04-06 14:15:29 +00:00
Eelco Dolstra a5ece7d016 * Removed the `~' operator. 2010-04-01 16:59:07 +00:00
Eelco Dolstra c172274e17 * Quick hack to make coerceToString work more or less correctly on
nested lists.  `nix-instantiate' can now evaluate the NixOS system
  derivation attribute correctly (in 2.1s on my laptop vs. 6.2s for
  the trunk).
2010-04-01 14:35:03 +00:00
Eelco Dolstra 7b851915bf * Improve sharing. 2010-04-01 12:04:57 +00:00
Eelco Dolstra 95cc417d76 * Functions are incomparable. 2010-04-01 10:55:36 +00:00
Eelco Dolstra 71f026292b * Make `derivation' lazy again for performance. It also turns out
that there are some places in Nixpkgs (php_configurable /
  composableDerivation, it seems) that call `derivation' with
  incorrect arguments (namely, the `name' attribute missing) but get
  away with it because of laziness.
2010-04-01 09:55:57 +00:00
Eelco Dolstra dc31305b38 * Fixed the trace primop and path comparison.
* Removed exprToString and stringToExpr because there is no ATerm
  representation to work on anymore (and exposing the internals of the
  evaluator like this is not a good idea anyway).
2010-03-31 20:09:20 +00:00
Eelco Dolstra 979f163615 * Handle string contexts. `nix-instantiate' can now correctly compute
the `firefoxWrapper' attribute in Nixpkgs, and it's about 3 times
  faster than the trunk :-)
2010-03-31 19:52:29 +00:00
Eelco Dolstra d8cd3115d8 * Get nix-env to compile. 2010-03-31 19:12:08 +00:00
Eelco Dolstra 55e207b2dc * Cache parse trees to prevent repeated parsing of imported Nix
expressions.
2010-03-31 16:14:32 +00:00
Eelco Dolstra 3d94be61ea * Implemented derivations. 2010-03-31 15:38:03 +00:00
Ludovic Courtès 09381cccff Make source location info in the XML output optional.
* src/libexpr/expr-to-xml.cc (nix::showAttrs): Add `location'
  parameter.  Provide location XML attributes when it's true.  Update
  callers.
  (nix::printTermAsXML): Likewise.

* src/libexpr/expr-to-xml.hh (nix::printTermAsXML): Update prototype;
  have `location' default to `false'.

* src/nix-instantiate/nix-instantiate.cc (printResult, processExpr): Add
  `location' parameter; update callers.
  (run): Add support for `--no-location'.

* src/nix-instantiate/help.txt: Update accordingly.

* tests/lang.sh: Invoke `nix-instantiate' with `--no-location' for the
  XML tests.

* tests/lang/eval-okay-toxml.exp, tests/lang/eval-okay-to-xml.nix: New
  files.
2010-03-31 12:38:31 +00:00
Eelco Dolstra 13c2adc897 * Implemented `rec { inherit ...; }'. 2010-03-31 11:05:39 +00:00
Eelco Dolstra 4c53ca2692 * Compare nulls. 2010-03-31 09:54:12 +00:00
Ludovic Courtès 471419d1fa Add source location information to the XML output.
* src/libexpr/expr-to-xml.cc (nix::showAttrs): Dereference the attribute
  RHS.  Add "path", "line", and "column" XML attributes to the node when
  source location information is available.
  (nix::printTermAsXML): Likewise for functions.
2010-03-31 08:29:05 +00:00