Commit graph

224 commits

Author SHA1 Message Date
Eelco Dolstra a8306cb98f * The build hooks used to implement distributed builds can now be run
in parallel.  Hooks are more efficient: locks on output paths are
  only acquired when the hook says that it is willing to accept a
  build job.  Hooks now work in two phases.  First, they should first
  tell Nix whether they are willing to accept a job.  Nix guarantuees
  that no two hooks will ever be in the first phase at the same time
  (this simplifies the implementation of hooks, since they don't have
  to perform locking (?)).  Second, if they accept a job, they are
  then responsible for building it (on the remote system), and copying
  the result back.  These can be run in parallel with other hooks and
  locally executed jobs.

  The implementation is a bit messy right now, though.  

* The directory `distributed' shows a (hacky) example of a hook that
  distributes build jobs over a set of machines listed in a
  configuration file.
2004-05-13 19:14:49 +00:00
Eelco Dolstra efa5fa1a91 * A switch `-j NUMBER' to set the maximum number of parallel jobs (0 =
no limit).
* Add missing file to distribution.
2004-05-12 14:20:32 +00:00
Eelco Dolstra 8c0b42f857 * An quick and dirty hack to support distributed builds. 2004-05-12 09:35:51 +00:00
Eelco Dolstra c8d3882cdc * True parallel builds. Nix can now run as many build jobs in
parallel as possible (similar to GNU Make's `-j' switch).  This is
  useful on SMP systems, but it is especially useful for doing builds
  on multiple machines.  The idea is that a large derivation is
  initiated on one master machine, which then distributes
  sub-derivations to any number of slave machines.  This should not
  happen synchronously or in lock-step, so the master must be capable
  of dealing with multiple parallel build jobs.  We now have the
  infrastructure to support this.

  TODO: substitutes are currently broken.
2004-05-11 18:05:44 +00:00
Eelco Dolstra 256eeab711 * Allow the location of the store etc. to be specified using
environment variables.
* Started adding some automatic tests.
* Do a `make check' when building RPMs.
2004-05-04 12:15:30 +00:00
Eelco Dolstra f8cd904e05 * Disallow the Nix store or any of its parents from being symlinks.
This is because the contents of these symlinks are not incorporated
  into the hashes of derivations, and could therefore cause a mismatch
  between the build system and the target system.  E.g., if
  `/nix/store' is a symlink to `/data/nix/store', then a builder could
  expand this path and store the result.  If on the target system
  `/nix/store' is not a symlink, or is a symlink that points somewhere
  else, we have a dangling pointer.

  The trigger for this change is that gcc 3.3.3 does exactly that (it
  applies realpath() to some files, such as libraries, which causes
  our impurity checker to bail out.)

  An annoying side-effect of this change is that it makes it harder to
  move the Nix store to a different file system.  On Linux, bind
  mounts can be used instead of symlink for this purpose (e.g., `mount
  -o bind /data/nix/store /nix/store').
2004-03-27 17:58:04 +00:00
Eelco Dolstra 777e13b94b * Nix now has three different formats for the log information it
writes to stderr:
  
  - `pretty': the old nested style (default)
  - `escapes': uses escape codes to indicate nesting and message
    level; can be processed using `log2xml'
  - `flat': just plain text, no nesting

  These can be set using `--log-type TYPE' or the NIX_LOG_TYPE
  environment variable.
2004-03-22 20:53:49 +00:00
Eelco Dolstra 0dfdafdf6d * Allow linking against an external Berkeley DB / ATerm library. 2004-02-16 16:37:16 +00:00
Eelco Dolstra f34de12140 * Allow the location of the store to be specified (--with-store-dir).
* Do not create stuff in localstatedir when doing `make install'
  (since we may not have write access).  In general, installation of
  constant code/data should be separate from the initialisation of
  mutable state.
2004-02-16 15:23:19 +00:00
Eelco Dolstra 76c0e85929 * The environment variable NIX_ROOT can now be set to execute Nix in a
chroot() environment.
* A operation `--validpath' to register path validity.  Useful for
  bootstrapping in a pure Nix environment.
* Safety checks: ensure that files involved in store operations are in
  the store.
2004-02-14 21:44:18 +00:00
Eelco Dolstra d9f30fe7c7 * Sort `nix-env -q' output by derivation name.
* `--version' flag for all commands.
* Manual updates.
2004-02-02 10:51:54 +00:00
Eelco Dolstra 291030b900 * Remove debug message. 2004-01-15 20:58:44 +00:00
Eelco Dolstra 447089a5f6 * Catch SIGINT to terminate cleanly when the user tries to interrupt
Nix.  This is to prevent Berkeley DB from becoming wedged.

  Unfortunately it is not possible to throw C++ exceptions from a
  signal handler.  In fact, you can't do much of anything except
  change variables of type `volatile sig_atomic_t'.  So we set an
  interrupt flag in the signal handler and check it at various
  strategic locations in the code (by calling checkInterrupt()).
  Since this is unlikely to cover all cases (e.g., (semi-)infinite
  loops), sometimes SIGTERM may now be required to kill Nix.
2004-01-15 20:23:55 +00:00
Eelco Dolstra ff9af107d3 * Option `-B' to always show the output of builders, regardless of
verbosity level.
2004-01-13 16:35:43 +00:00
Eelco Dolstra 833f2fc92d * GCC 2.95 compatibility. 2003-12-22 16:40:46 +00:00
Eelco Dolstra 078e20885e * Help text for all (non-script) programs, so no more:
$ nix-instantiate --help
error: unknown flag `--help`
Try `nix-instantiate --help' for more information.

:-)
2003-12-01 15:55:05 +00:00
Eelco Dolstra 66c115ef5f * More `make dist' fixes. 2003-11-25 13:01:21 +00:00
Eelco Dolstra 9898746ef3 * nix-env: a tool to manage user environments.
* Replace all directory reading code by a generic readDirectory()
  function.
2003-11-19 17:27:16 +00:00
Eelco Dolstra 9f0f020929 * libnix -> libstore. 2003-11-18 10:55:27 +00:00
Eelco Dolstra 8798fae304 * Source tree refactoring. 2003-11-18 10:47:59 +00:00
Eelco Dolstra 15801c88fa * Turned the msg() and debug() functions into macros, since they
turned out to be a huge performance bottleneck (the text to printed
  would always be evaluated, even when it was above the verbosity
  level).  This reduces fix-ng execution time by over 50%.

  gprof(1) is very useful. :-)
2003-11-09 10:35:45 +00:00
Eelco Dolstra ff31324278 * Ignore options passed to the aterm library. 2003-11-03 18:21:53 +00:00
Eelco Dolstra c62433751d * Finished refactoring the tree. 2003-10-20 10:05:01 +00:00
Eelco Dolstra 53e376d836 * Refactored the source tree. 2003-10-20 09:20:11 +00:00