Commit graph

466 commits

Author SHA1 Message Date
Eelco Dolstra
1f285cf556 * Scoping bug in `with'. 2005-11-04 14:50:33 +00:00
Eelco Dolstra
deb75bb414 * Remove debugging code. 2005-09-13 15:54:36 +00:00
Eelco Dolstra
116e939d57 * More debugging. 2005-09-13 14:07:22 +00:00
Eelco Dolstra
55b84357a1 * Debugging. 2005-09-13 13:17:14 +00:00
Eelco Dolstra
991a130b1e * Added a list concatenation operator:
[1 2 3] ++ [4 5 6] => [1 2 3 4 5 6]
2005-07-25 15:05:34 +00:00
Eelco Dolstra
33efb52e02 * Hack to get around the libtool wrapper script around nix-store not
working when PATH is unset.
2005-07-25 07:25:18 +00:00
Eelco Dolstra
040140dd1c * Added a primop `removeAttrs' to remove attributes from a set, e.g.,
`removeAttrs attrs ["x", "y"]' returns the set `attrs' with the
  attributes named `x' and `y' removed.  It is not an error for the
  named attributes to be missing from the input set.
2005-05-18 17:19:21 +00:00
Eelco Dolstra
d8cda7c3dc * Mac OS X (and POSIX) doesn't have readlink. 2005-05-06 14:43:14 +00:00
Eelco Dolstra
c702dfca3f * nix-store: --substitute' -> --register-substitutes'. 2005-04-08 13:48:41 +00:00
Eelco Dolstra
7d876f8fa7 * Get rid of fetchurl, we don't need it anymore. 2005-04-07 14:35:44 +00:00
Eelco Dolstra
590e5a0d65 * Add a test for base-32 encoding of hashes since it seems to be
broken on Mac OS X.
2005-03-23 17:13:42 +00:00
Eelco Dolstra
a1e00bf6aa * Remove non-POSIX flag. 2005-03-21 16:28:58 +00:00
Eelco Dolstra
08df443618 * Check for duplicate attributes and formal parameters in Nix
expressions.
2005-03-10 11:33:46 +00:00
Eelco Dolstra
398463a72a * `make check' fix. 2005-02-18 08:40:52 +00:00
Eelco Dolstra
0083562f75 * Fix broken GC test. 2005-02-15 09:39:12 +00:00
Eelco Dolstra
98df735b51 * Propagate the deriver of a path through the substitute mechanism.
* Removed some dead code (successor stuff) from nix-push.
* Updated terminology in the tests (store expr -> drv path).
* Check that the deriver is set properly in the tests.
2005-02-09 12:57:13 +00:00
Eelco Dolstra
c3981d81f6 * Make check fixes. 2005-02-01 17:50:48 +00:00
Eelco Dolstra
06b4424286 * Add missing files to dist.
* Fix GC and substitute bugs related to self-references.  Add a
  regression test.
2005-02-01 09:23:38 +00:00
Eelco Dolstra
32fa82a56a * Acquire a global GC lock to prevent new temporary root files from
being created after the garbage collector has read the temproots
  directory.  This blocks the creation of new processes, but the
  garbage collector could periodically release the GC lock to allow
  them to run.
2005-01-31 22:23:49 +00:00
Eelco Dolstra
89c9bc11ab * Add a test for a more subtle race: a process starting after the
temporary root files have been read but creating outputs before the
  store directory has been read.
2005-01-31 22:01:55 +00:00
Eelco Dolstra
33c5d23b81 * Don't delete active lock files. 2005-01-31 12:19:53 +00:00
Eelco Dolstra
1328aa3307 * Start of concurrent garbage collection. Processes write temporary
roots to a per-process temporary file in /nix/var/nix/temproots
  while holding a write lock on that file.  The garbage collector
  acquires read locks on all those files, thus blocking further
  progress in other Nix processes, and reads the sets of temporary
  roots.
2005-01-31 10:27:25 +00:00
Eelco Dolstra
a7668411a1 * Add a test to check whether concurrent garbage collection (i.e.,
running the collector while builds are in progress) works
  correctly.  The test currently fails.
2005-01-28 20:36:46 +00:00
Eelco Dolstra
9ab0bc9395 * Another horrible `make check' hack. 2005-01-28 11:05:56 +00:00
Eelco Dolstra
ac2f665853 * Set execute permission. 2005-01-27 19:15:12 +00:00
Eelco Dolstra
e5c16c9582 * Add missing substitutes files to dist.
* Add a garbage collector test.
2005-01-27 17:48:50 +00:00
Eelco Dolstra
4e37548a1e * Remove deleted files from EXTRA_DIST (again). 2005-01-27 15:31:49 +00:00
Eelco Dolstra
a9340fa672 * Remove removed files from EXTRA_DIST. 2005-01-25 17:25:20 +00:00
Eelco Dolstra
498f4915cc * Re-enable all tests. 2005-01-25 17:24:14 +00:00
Eelco Dolstra
066da4ab85 * Really fix the substitute mechanism, i.e., ensure the closure
invariant by registering references through the manifest.
* Added a test for nix-pull.
2005-01-25 17:08:52 +00:00
Eelco Dolstra
c6290e42bc * Fix the `--fallback' switch.
* Fix the substitutes tests.
2005-01-25 13:00:12 +00:00
Eelco Dolstra
581fc47783 * Fix the build hook mechanism; pass the pointer graph to the hook. 2005-01-25 11:55:43 +00:00
Eelco Dolstra
52bf9b86bb * In nix-store: added query `--referers-closure' that returns the
closure of the referers relation rather than the references
  relation, i.e., the set of all paths that directly or indirectly
  refer to the given path.  Note that contrary to the references
  closure this set is not fixed; it can change as paths are added to
  or removed from the store.
2005-01-25 11:18:03 +00:00
Eelco Dolstra
80faa2f98a * In nix-store: change --build' back to --realise'. Also brought
back the query flag `--force-realise'.
* Fixed some of the tests.
2005-01-25 10:55:33 +00:00
Eelco Dolstra
06c77bf7a8 * Change extension .store' to .drv'.
* Re-enable `nix-store --query --requisites'.
2005-01-19 14:36:00 +00:00
Eelco Dolstra
a7b94e87d7 * Missing file. 2005-01-14 13:50:09 +00:00
Eelco Dolstra
63791eb05b * Add SHA-256.
* Tests for the various hashes.
2005-01-14 12:03:04 +00:00
Eelco Dolstra
a03397be4c * Cygwin compatibility. 2005-01-04 17:38:26 +00:00
Eelco Dolstra
96c3d8a615 * I love test sets. 2004-12-20 14:38:04 +00:00
Eelco Dolstra
ed09821859 * Use atdiff' instead of cmp' for checking test output.
* Don't use local file names in tests since they will produce
  different parse trees depending on the current directory.
2004-10-27 13:12:58 +00:00
Eelco Dolstra
3277c9432a * Bug fix in parsing of /* ... */ comments; due to longest match
regexp there could be only one such comment per file.
2004-10-27 13:00:31 +00:00
Eelco Dolstra
463e2817c5 * Remove ancient Fix tests.
* Add automated Nix expression language tests.
2004-10-27 12:41:53 +00:00
Eelco Dolstra
8f1dcdfc0a * Make sure that no build hook is set by default in the tests.
* Don't use `seq' - some primitive, obsolete operating systems
  (Darwin) don't have it.
2004-08-19 09:09:09 +00:00
Eelco Dolstra
2746a879e2 * Typo. 2004-06-28 12:07:07 +00:00
Eelco Dolstra
91dc023665 * Added a switch `--fallback'. From the manual:
Whenever Nix attempts to realise a derivation for which a closure is
  already known, but this closure cannot be realised, fall back on
  normalising the derivation.

  The most common scenario in which this is useful is when we have
  registered substitutes in order to perform binary distribution from,
  say, a network repository.  If the repository is down, the
  realisation of the derivation will fail.  When this option is
  specified, Nix will build the derivation instead.  Thus, binary
  installation falls back on a source installation.  This option is
  not the default since it is generally not desirable for a transient
  failure in obtaining the substitutes to lead to a full build from
  source (with the related consumption of resources).
2004-06-28 10:42:57 +00:00
Eelco Dolstra
b113edeab7 * A flag `--keep-going / -k' to keep building goals if one fails, as
much as possible.  (This is similar to GNU Make's `-k' flag.)

* Refactoring to implement this: previously we just bombed out when
  a build failed, but now we have to clean up.  In particular this
  means that goals must be freed quickly --- they shouldn't hang
  around until the worker exits.  So the worker now maintains weak
  pointers in order not to prevent garbage collection.

* Documented the `-k' and `-j' flags.
2004-06-25 15:36:09 +00:00
Eelco Dolstra
a29c8ac51c * Add a test to check that when we cannot realise a closure
expression, we should invalidate it and go back to the derivation
  for which it is a successor.
2004-06-24 14:35:01 +00:00
Eelco Dolstra
ec32627621 * Multiple and/or failing substitutes now work. 2004-06-24 13:40:38 +00:00
Eelco Dolstra
8052aef486 * A test for multiple and/or failing substitutes. 2004-06-24 12:56:24 +00:00
Eelco Dolstra
88fb4f6e53 * Missing files added to `make dist'. 2004-06-21 12:20:47 +00:00
Eelco Dolstra
112ee89501 * Re-enable support for substitutes in the normaliser.
* A better substitute mechanism.

  Instead of generating a store expression for each store path for
  which we have a substitute, we can have a single store expression
  that builds a generic program that is invoked to build the desired
  store path, which is passed as an argument.

  This means that operations like `nix-pull' only produce O(1) files
  instead of O(N) files in the store when registering N substitutes.
  (It consumes O(N) database storage, of course, but that's not a
  performance problem).

* Added a test for the substitute mechanism.
  
* `nix-store --substitute' reads the substitutes from standard input,
  instead of from the command line.  This prevents us from running
  into the kernel's limit on command line length.
2004-06-20 19:17:54 +00:00
Eelco Dolstra
23bb902d1f * Re-enable build hooks. 2004-06-19 21:45:04 +00:00
Eelco Dolstra
41ec982f31 * Big refactoring. Move to a much more explicitly state machine based
approach.  This makes it much easier to add extra complexity in the
  normaliser / realiser (e.g., build hooks, substitutes).
2004-06-18 18:09:32 +00:00
Eelco Dolstra
2fa3304933 * Set the executable bit. 2004-05-13 20:28:20 +00:00
Eelco Dolstra
3426d19547 * Perform all tests. 2004-05-13 19:16:48 +00:00
Eelco Dolstra
a8306cb98f * The build hooks used to implement distributed builds can now be run
in parallel.  Hooks are more efficient: locks on output paths are
  only acquired when the hook says that it is willing to accept a
  build job.  Hooks now work in two phases.  First, they should first
  tell Nix whether they are willing to accept a job.  Nix guarantuees
  that no two hooks will ever be in the first phase at the same time
  (this simplifies the implementation of hooks, since they don't have
  to perform locking (?)).  Second, if they accept a job, they are
  then responsible for building it (on the remote system), and copying
  the result back.  These can be run in parallel with other hooks and
  locally executed jobs.

  The implementation is a bit messy right now, though.  

* The directory `distributed' shows a (hacky) example of a hook that
  distributes build jobs over a set of machines listed in a
  configuration file.
2004-05-13 19:14:49 +00:00
Eelco Dolstra
5087c8f645 * Use `-j0'. 2004-05-12 14:30:57 +00:00
Eelco Dolstra
efa5fa1a91 * A switch `-j NUMBER' to set the maximum number of parallel jobs (0 =
no limit).
* Add missing file to distribution.
2004-05-12 14:20:32 +00:00
Eelco Dolstra
1f48aa0be7 * Broken test. 2004-05-12 13:49:10 +00:00
Eelco Dolstra
8c0b42f857 * An quick and dirty hack to support distributed builds. 2004-05-12 09:35:51 +00:00
Eelco Dolstra
c8d3882cdc * True parallel builds. Nix can now run as many build jobs in
parallel as possible (similar to GNU Make's `-j' switch).  This is
  useful on SMP systems, but it is especially useful for doing builds
  on multiple machines.  The idea is that a large derivation is
  initiated on one master machine, which then distributes
  sub-derivations to any number of slave machines.  This should not
  happen synchronously or in lock-step, so the master must be capable
  of dealing with multiple parallel build jobs.  We now have the
  infrastructure to support this.

  TODO: substitutes are currently broken.
2004-05-11 18:05:44 +00:00
Eelco Dolstra
a9858c9f26 * A test to verify that Nix executes build jobs in parallel, if
possible.

  This test fails right now because this hasn't been implemented right
  now.  Yes, I'm doing Test-Driven Development! ;-)
2004-05-04 17:04:17 +00:00
Eelco Dolstra
f044ccf702 * 1000th revision!
* A test to verify that locking of output paths (caused by concurrent
  invocations of Nix) works correctly.
2004-05-04 13:56:30 +00:00
Eelco Dolstra
a7bbe73971 * Another test. 2004-05-04 13:22:33 +00:00
Eelco Dolstra
ef093aac8f * Grrr. TESTS are not included in EXTRA_DIST. 2004-05-04 12:45:04 +00:00
Eelco Dolstra
256eeab711 * Allow the location of the store etc. to be specified using
environment variables.
* Started adding some automatic tests.
* Do a `make check' when building RPMs.
2004-05-04 12:15:30 +00:00