0
0
Fork 0
forked from lix-project/lix
Commit graph

454 commits

Author SHA1 Message Date
Eelco Dolstra 9ac3f5df9c * Propagate --max-silent-time to remote machines. 2008-12-04 16:51:16 +00:00
Eelco Dolstra bcfe98acff * Prefer building on a remote machine over a local machine. This
makes more sense for the build farm, otherwise every nix-store
  invocation will lead to at least one local build.  Will come up with
  a better solution later...
2008-12-04 16:35:47 +00:00
Eelco Dolstra 909fbb9de1 * When using build hooks, for any nix-store -r build operation, it is
necessary that at least one build hook doesn't return "postpone",
  otherwise nix-store will barf ("waiting for a build slot, yet there
  are no running children").  So inform the build hook when this is
  the case, so that it can start a build even when that would exceed
  the maximum load on a machine.
2008-12-04 14:29:41 +00:00
Eelco Dolstra 5dfba0b4db * Force allocation of a pseudo-terminal to clean up the remote
nix-store process when the connection is interrupted.
2008-12-04 13:36:52 +00:00
Eelco Dolstra 9fd9c4c635 * Support multiple system types per remote machine, e.g. a machine
list like

    root@example.org x86_64-linux /root/.ssh/id_buildfarm 1
    root@example.org i686-darwin  /root/.ssh/id_buildfarm 1

  This is possible when the Nix installation on example.org itself has
  remote builds enabled.
2008-12-04 12:20:06 +00:00
Eelco Dolstra f8713e1287 * Dirty hack to make nix-push work properly on derivations: the
derivation should be a source rather than a derivation dependency of
  the call to the NAR derivation.  Otherwise the derivation (and all
  its dependencies) will be built as a side-effect, which may not even
  succeed.
2008-12-04 10:40:41 +00:00
Eelco Dolstra 6bbff48079 2008-11-20 17:22:42 +00:00
Eelco Dolstra 4213b8d8ec * Urgh. 2008-11-20 15:44:59 +00:00
Eelco Dolstra aab530e971 * Primop builtins.storePath for declaring a store path as a
dependency.  `storePath /nix/store/bla' gives exactly the same
  result as `toPath /nix/store/bla', except that the former includes
  /nix/store/bla in the dependency context of the string.

  Useful in some generated Nix expressions like nix-push, which now
  finally does the right thing wrt distributed builds.  (Previously
  the path to be packed wasn't an explicit dependency, so it wouldn't
  be copied to the remote machine.)
2008-11-19 23:26:19 +00:00
Eelco Dolstra fa791116a3 * Get rid of nix-pack-closure / nix-unpack-closure, they're redundant. 2008-11-18 14:43:40 +00:00
Eelco Dolstra 2b2aa8a820 * Doh. 2008-09-08 11:02:15 +00:00
Michael Raskin 2a01d06da6 Added nix-http-export.cgi to extra distributed scripts; so it can be installed from releases, not only from SVN. 2008-08-29 08:34:38 +00:00
Michael Raskin b455c4c45c Updates to nix-reduce-build
Common code in local build package sources refactored out in a function; before building the real set of derivations needed is found (slightly slower for only one build strategy, but less garbage on output and better performance for multiple build strategies).

Now you have full choice of best-effort build regardless of method (substituters or actual build), using substituters, building only fixed derivations (should get you all the downloads) and local build without even trying substituters. 

Some minor fix in the help text about behavior with no package sources.
2008-08-06 19:43:53 +00:00
Eelco Dolstra 7592f48c83 * nix-build: `--dry-run' flag. 2008-08-04 13:46:01 +00:00
Eelco Dolstra 42043953c3 * Doh. 2008-08-04 13:15:47 +00:00
Michael Raskin b3c26180e3 Updates to nix-reduce-build: only realize fixed derivations if user asks so, or only use substituters. Oh, and add possibility to use : for things like /etc/nixos/nixpkgs:-A:gnused 2008-08-02 16:43:25 +00:00
Eelco Dolstra 3c92ea399d * Make nix-env --dry-run print the paths to be substituted correctly
again.  (After the previous substituter mechanism refactoring I
  didn't update the code that obtains the references of substitutable
  paths.)  This required some refactoring: the substituter programs
  are now kept running and receive/respond to info requests via
  stdin/stdout.
2008-08-02 12:54:35 +00:00
Eelco Dolstra 096198d11f * A quick hack to make nix-prefetch-url support mirror:// URLs. It
requires that $NIXPKGS_ALL points at a Nixpkgs tree.
2008-07-23 16:02:58 +00:00
Eelco Dolstra 660244f65f * Make sure that copy-from-other-stores.pl is built. 2008-07-23 09:38:38 +00:00
Eelco Dolstra 989176c56e * Allow read-only access to the store (e.g., non-root users on NixOS
can do operations like "nix-store -qR <path>" even without the Nix
  daemon).
2008-07-18 15:34:46 +00:00
Eelco Dolstra 8bc591a6f0 * Use the copy-from-other-stores substituter by default. Of course,
it only does something if $NIX_OTHER_STORES (not really a good
  name...) is set.
* Do globbing on the elements of $NIX_OTHER_STORES.  E.g. you could
  set it to /mnts/*/nix or something.
* Install substituters in libexec/nix/substituters.
2008-07-18 13:05:10 +00:00
Eelco Dolstra 15f39aba8c * Quick prototype of a substituter that copies paths from other Nix
stores (typically remote Nix stores mounted via e.g. NFS, or the Nix
  store on the NixOS installation CD).  Example use:

  $ sshfs foo@example.org:/ /mnt
  $ NIX_OTHER_STORES=/mnt/nix \
    NIX_SUBSTITUTERS=.../copy-from-other-stores.pl \
    nix-env -i foo

  This will be especially useful for the installation CD since it
  doesn't require a manifest for the CD contents.
2008-07-12 18:58:24 +00:00
Michael Raskin 955b8841cd Also trying to build derivers in case we cannot get substituters 2008-06-14 08:48:40 +00:00
Michael Raskin 18e27629d3 Added local best-effort builds (i.e. one failure does not ruin all packages you would like to see built) 2008-06-14 08:30:35 +00:00
Michael Raskin 194c66eeeb Stupid error in script 2008-06-13 14:34:19 +00:00
Michael Raskin f903d86740 OK, I will believe that fix does no worse.. 2008-06-13 13:53:14 +00:00
Michael Raskin ce85b55cf0 Updated help text 2008-06-12 17:45:38 +00:00
Michael Raskin 4532e4b90d Added verbosity for nix-reduce-build 2008-06-12 16:26:53 +00:00
Michael Raskin 9819bb20da Added support for file:// archive (.nar.gz) repositories to nix-reduce-build. /tmp/nix-export created by nix-http-export.cgi is OK. 2008-05-11 15:54:30 +00:00
Michael Raskin b4bc8b7616 --proxy=proxy:3128 2008-05-07 14:18:28 +00:00
Michael Raskin b1e321d6ce Added http alternative transport for nix-reduce-build 2008-04-29 04:03:54 +00:00
Eelco Dolstra f8985d195e * Fix for NIX-101 (should use an absolute path for call to nix-hash). 2008-03-28 17:52:33 +00:00
Eelco Dolstra f106868110 * Cleanup. 2008-03-20 10:16:36 +00:00
Eelco Dolstra d7caac3e81 * Don't install nix-reduce-build by default yet please, I first want
to understand better what it does...
2008-02-28 21:27:47 +00:00
Michael Raskin 11d512e7a8 Added nix-reduce-build. You point it to some path you want to build and it fetches whatever it can from specified computers via nix-copy-closure. NOTE: You do want to set up RSA keys or ssh-agent or something... You really do want it. It will run separate ssh instances insane number of times. 2008-02-27 21:26:47 +00:00
Eelco Dolstra de012e76e2 * Don't use "store expression", it's obsolete. 2007-12-30 23:35:45 +00:00
Eelco Dolstra c370755583 * Flag `--no-build-hook' to disable distributed builds.
* queryDeriver in daemon mode: don't barf if the other side returns an
  empty string (which means there is no deriver).
2007-11-16 16:15:26 +00:00
Eelco Dolstra 6784b14241 * nix-build: pass --argstr to nix-instantiate. 2007-11-15 16:52:40 +00:00
Eelco Dolstra ca210d2a58 * Doh! 2007-11-15 15:33:30 +00:00
Eelco Dolstra bfec245efa * Add build-remote.pl to the Nix distribution. 2007-11-15 14:28:08 +00:00
Eelco Dolstra 5b74a59570 * nix-prefetch-url: don't fail if /tmp/nix-prefetch-url-<pid> exists,
instead use a counter just like we do for temporary build
  directories.
2007-11-05 18:12:29 +00:00
Eelco Dolstra f8ed2e64a5 2007-10-22 12:05:18 +00:00
Eelco Dolstra a8629de827 * New command `nix-store --optimise' to reduce Nix store disk space
usage by finding identical files in the store and hard-linking them
  to each other.  It typically reduces the size of the store by
  something like 25-35%.  This is what the optimise-store.pl script
  did, but the new command is faster and more correct (it's safe wrt
  garbage collection and concurrent builds).
2007-10-09 22:14:27 +00:00
Eelco Dolstra 26f981c2e5 * Remove garbage. 2007-09-18 08:26:55 +00:00
Eelco Dolstra 055608227f * nix-env: allow ~/.nix-defexpr to be a directory. If it is, then the
Nix expressions in that directory are combined into an attribute set
  {file1 = import file1; file2 = import file2; ...}, i.e. each Nix
  expression is an attribute with the file name as the attribute
  name.  Also recurses into directories.

* nix-env: removed the "--import" (-I) option which set the
  ~/.nix-defexpr symlink.

* nix-channel: don't use "nix-env --import", instead symlink
  ~/.nix-defexpr/channels.  So finally nix-channel --update doesn't
  override any default Nix expressions but combines with them.

  This means that you can have (say) a local Nixpkgs SVN tree and use
  it as a default for nix-env:

  $ ln -s .../path-to-nixpkgs-tree ~/.nix-defexpr/nixpkgs_svn

  and be subscribed to channels (including Nixpkgs) at the same time.
  (If there is any ambiguity, the -A flag can be used to
  disambiguate, e.g. "nix-env -i -A nixpkgs_svn.pan".)
2007-09-17 16:08:24 +00:00
Eelco Dolstra 9441550acb * nix-push / generate-patches: bzip the manifest. 2007-09-04 15:38:09 +00:00
Eelco Dolstra 7d5836b34d * nix-channel: supports users who don't have write permission to the
manifests directory.  In that case, we don't do a nix-pull, so the
  user gets pure source deployment.

  The directory /nix/var/nix/gcroots/per-user/$USER should be
  writable.  (It's created automatically if
  /nix/var/nix/gcroots/per-user is writable, e.g. if it has 1777
  permission.)
2007-08-22 14:52:22 +00:00
Eelco Dolstra dbc0170ed9 * Show errors in nix-prefetch-url. 2007-08-15 09:24:06 +00:00
Eelco Dolstra ffa1c61cd5 * Fix the tests. 2007-08-14 13:15:59 +00:00
Eelco Dolstra 9e975458b4 * Get rid of the substitutes database table (NIX-47). Instead, if we
need any info on substitutable paths, we just call the substituters
  (such as download-using-manifests.pl) directly.  This means that
  it's no longer necessary for nix-pull to register substitutes or for
  nix-channel to clear them, which makes those operations much faster
  (NIX-95).  Also, we don't have to worry about keeping nix-pull
  manifests (in /nix/var/nix/manifests) and the database in sync with
  each other.

  The downside is that there is some overhead in calling an external
  program to get the substitutes info.  For instance, "nix-env -qas"
  takes a bit longer.

  Abolishing the substitutes table also makes the logic in
  local-store.cc simpler, as we don't need to store info for invalid
  paths.  On the downside, you cannot do things like "nix-store -qR"
  on a substitutable but invalid path (but nobody did that anyway).

* Never catch interrupts (the Interrupted exception).
2007-08-12 00:29:28 +00:00
Eelco Dolstra 4695f4edd6 * nix-pull: support bzipped manifests: when doing a nix-pull on $url,
try $url.bz2 first.
2007-08-10 01:42:00 +00:00
Eelco Dolstra 911bc01454 * Enable nix-prefetch-url caching in nix-channel. 2007-08-10 00:28:44 +00:00
Eelco Dolstra 5e52df18fe * Don't rely on /dev/stdin. 2007-08-10 00:22:21 +00:00
Eelco Dolstra c5e934dcc9 * nix-pull: using nix-prefetch-url (so that we get caching for free),
and store the manifests in the Nix store.  (So now
  /nix/var/nix/manifests/ just contains symlinks to the store and is
  searched for GC roots.)
2007-08-09 23:52:53 +00:00
Eelco Dolstra f881f7a017 * nix-prefetch-url: support caching. If the environment variable
NIX_DOWNLOAD_CACHE is set, then nix-prefetch-url will store the hash
  and timestamp of downloaded files in the directory
  $NIX_DOWNLOAD_CACHE.  This allows it to figure out if the file is
  still in the Nix store.
2007-08-09 23:16:44 +00:00
Eelco Dolstra 9bff7ad728 * Check against creation of GC roots in the store. Those roots don't
work, because findRoots() stops when it encounters a symlink to the
  store.  And of course the store is supposed to be read-only.
2007-06-11 11:36:22 +00:00
Eelco Dolstra f3ebd03bb1 * Remove debug statement. 2007-06-04 12:03:33 +00:00
Eelco Dolstra 34d4c9388b 2007-05-02 16:06:04 +00:00
Eelco Dolstra 93aefd9fc0 * Give unpacked channels more sensible names than 0, 1, ... They now
get the basename of the channel URL (e.g., nixpkgs-unstable).  The
  top-level Nix expression of the channel is now an attribute set, the
  attributes of which are the individual channels (e.g.,
  {nixpkgs_unstable = ...; strategoxt_unstable = ...}).  This makes
  attribute paths ("nix-env -qaA" and "nix-env -iA") more sensible,
  e.g., "nix-env -iA nixpkgs_unstable.subversion".
2007-05-01 23:16:38 +00:00
Eelco Dolstra 4caca58ff7 * Make the maximum patch size configurable. 2007-03-30 09:01:05 +00:00
Eelco Dolstra efd31139df * Forgot a @bindir@. 2007-03-27 09:53:58 +00:00
Eelco Dolstra d303b389a9 * `nix-copy-closure --from': copy from a remote machine instead of to
a remote machine.
2007-03-26 21:05:17 +00:00
Eelco Dolstra 7edd2e2cd2 * Refactoring. 2007-03-26 20:49:22 +00:00
Eelco Dolstra a8ea4cbcc8 * Scan /proc/sys/kernel/modprobe for roots to prevent the kernel
modules for the running kernel from being garbage-collected.  Idem
  for /proc/sys/kernel/fbsplash.
2007-03-20 11:13:15 +00:00
Eelco Dolstra eb2dd4815c * Remove old generations in all directories under
/nix/var/nix/profiles, not just in that directory itself.  (NixOS
  puts profiles in /nix/var/nix/profiles/per-user.)
2007-03-13 11:30:57 +00:00
Eelco Dolstra df0283ae86 * Get rid of those stupid --login tricks, it's the responsibility of
the remote system to make sure that Nix is in the $PATH.
2007-03-01 13:55:47 +00:00
Eelco Dolstra 30394a4f3f * sh -> bash. 2007-03-01 13:49:20 +00:00
Eelco Dolstra 363e307fd3 * Error message to stdout. 2007-02-26 23:32:10 +00:00
Eelco Dolstra fa2be32034 * nix-copy-closure: force a login shell on the remote machine to make
sure that nix-store is in the PATH.
* nix-copy-closure: option --gzip to compress data.
2007-02-22 16:42:01 +00:00
Eelco Dolstra 4c5e6d1a2f * nix-copy-closure: option --sign.
* nix-copy-closure: set SSH options through NIX_SSHOPTS..
2007-02-22 15:48:20 +00:00
Eelco Dolstra 024a8ed382 * New command `nix-copy-closure' to copy a closure to a Nix store on
another machine through ssh.  E.g.,

    $ nix-copy-closure xyzzy $(which svn)

  copies the closure of Subversion to machine `xyzzy'.  This is like
  `nix-pack-closure $(which svn) | ssh xyzzy', but it's much more
  efficient since it only copies those paths that are missing on the
  target machine.
2007-02-21 23:14:53 +00:00
Eelco Dolstra 8181a1c3bb * Close the file - just in case. 2007-02-20 22:49:08 +00:00
Eelco Dolstra bae75ca5a1 * New kind of manifest object: "localPath", which denotes that a store
path can be created by copying it from another location in the file
  system.  This is useful in the NixOS installation.
2007-01-23 16:50:19 +00:00
Eelco Dolstra 36d9258c0d * Successors have been gone for ages. 2007-01-23 16:05:59 +00:00
Eelco Dolstra 7bc30e1ca8 * nix-prefetch-url: change the default hash to SHA-256 (in base-32). 2007-01-22 09:53:36 +00:00
Eelco Dolstra afe23b5f38 * nix-pack-closure: store the top-level store paths in the closure.
* nix-unpack-closure: extract the top-level paths from the closure and
  print them on stdout.  This allows them to be installed, e.g.,
  "nix-env -i $(nix-unpack-closure)".  (NIX-64)
2007-01-13 19:50:42 +00:00
Eelco Dolstra 215505bb46 * Removed chroot support. 2007-01-13 17:54:01 +00:00
Eelco Dolstra 1b7840b949 2007-01-11 19:28:28 +00:00
Eelco Dolstra 69c8b5b8a7 * Install generate-patches into libexec. 2007-01-11 16:19:45 +00:00
Eelco Dolstra 1f3722bd4a * Reject patches that are larger than a certain fraction of the full archive
(currently 60%).  Large patches aren't very economical.
2007-01-08 15:32:15 +00:00
Eelco Dolstra 50bdec410a * Huge speedup in patch propagation (20 minutes or so to 3 seconds). 2007-01-08 15:17:18 +00:00
Eelco Dolstra cafaceb707 * Handle weird cases when the server redirects us while setting a cookie. 2006-12-15 21:27:26 +00:00
Eelco Dolstra 9dbfe242e3 * Kill a build if it has gone for more than a certain number of
seconds without producing output on stdout or stderr (NIX-65).  This
  timeout can be specified using the `--max-silent-time' option or the
  `build-max-silent-time' configuration setting.  The default is
  infinity (0).

* Fix a tricky race condition: if we kill the build user before the
  child has done its setuid() to the build user uid, then it won't be
  killed, and we'll potentially lock up in pid.wait().  So also send a
  conventional kill to the child.
2006-12-08 15:44:00 +00:00
Eelco Dolstra d941186289 * Show more progress. 2006-11-18 19:03:45 +00:00
Eelco Dolstra b3f916995a * Oops, `nix-build --no-out-link' was broken. 2006-10-31 18:45:17 +00:00
Eelco Dolstra 8d17265ac4 * Don't use EPSV. 2006-10-28 22:07:09 +00:00
Eelco Dolstra b3d3700e11 * nix-build: check the exit status of `nix-store -r'. 2006-10-19 17:30:09 +00:00
Eelco Dolstra 2a535689fe * Reduce the maximum archive size for patch generation to 100 MB to
prevent trashing on nix.cs.uu.nl.
2006-10-12 20:13:29 +00:00
Eelco Dolstra d98f750fd8 * tmpnam() -> File::Temp::tempdir(). 2006-10-04 18:58:11 +00:00
Eelco Dolstra 3632019b73 * Quick hack to let nix-install-package set the package name properly
(e.g., "java-front-0.9pre15899" instead of "java-front";
  particularly important when doing upgrades later on).
2006-09-25 14:00:59 +00:00
Eelco Dolstra d43565c3e8 * In `nix-channel --update', skip manifests that assume a Nix store at
a different location than the user's.  This makes channels usable as
  a source deployment mechanism for people who install Nix under
  non-standard prefixes.  (NIX-57)
2006-09-25 11:11:16 +00:00
Eelco Dolstra 68ae953d8a * Clean up calls to system(). 2006-09-25 10:44:27 +00:00
Eelco Dolstra 76c9710091 * Use builtins.toPath. 2006-09-25 10:29:25 +00:00
Eelco Dolstra 0bd5eb71a0 * `nix-install-package --url': install from a URL (NIX-12).
* `nix-install-package --help' (NIX-9).
* `nix-install-package --non-interactive': don't prompt or pause.
* Tests for nix-install-package.
* Security fixes: filter the values obtained from the nixpkg.
2006-09-21 18:54:08 +00:00
Eelco Dolstra ee5040421f * Try konsole and gnome-terminal in addition to xterm. 2006-09-21 11:29:14 +00:00
Eelco Dolstra 1b804f88e4 * Absolute path to rm (NIX-51).
* Don't hardcore /nix/bin and /nix/store.
2006-09-20 16:23:14 +00:00
Eelco Dolstra 947e645789 * Hide warnings about a missing "lsof" (NIX-54). 2006-09-20 15:14:19 +00:00
Eelco Dolstra a060adf165 * Use paths, not strings, when calling the function that generates
NARs.  Fixes the impurity of nix-push (NIX-21).
* Better help.
2006-09-20 15:04:04 +00:00
Eelco Dolstra 7dd342e482 * Doh. 2006-09-19 16:40:22 +00:00
Eelco Dolstra ee6cf99660 * Doh! Of course we have to take execute permission into account.
* Restore the mtime on modified directories.
2006-09-19 16:14:15 +00:00
Eelco Dolstra 6dbed1bf38 * `optimise-store.pl' reduces disk space consumption by hard-linking
all identitical files in the Nix store to each other.  (Previously
  it only computed the size that would be saved by doing so.)
2006-09-19 14:58:35 +00:00
Eelco Dolstra 07cec27848 * Cleanups. 2006-09-19 14:27:28 +00:00
Eelco Dolstra 9488ae7357 * `show-duplication.pl', a small utility that shows the amount of
package duplication present in (e.g.) a profile.  It shows the
  number of instances of each package in a closure, along with the
  size in bytes of each instance as well as the "waste" (the
  difference between the sum of the sizes of all instances and the
  average size).

  $ ./show-duplication.pl /nix/var/nix/profiles/default
  gcc 11
    3.3.6 19293318
    3.4.4 21425257
    ...
    average 14942970, waste 149429707
  coreutils 6
  ...
  average package duplication 1.87628865979381, total size 3486330471, total waste 1335324237, 38.3017114443825% wasted

  This utility is useful for measuring the cost in terms of disk space
  of the Nix approach.
2006-09-19 13:53:35 +00:00
Eelco Dolstra 2e210b2387 * Convenience option `nix-collect-garbage -d' (--delete-old): removes
old generations of *all* profiles in /nix/var/nix/profiles, then
  runs the garbage collector.  Quick way to get rid of all old stuff.
  Of course, one cannot roll back to earlier points in time after
  this.
2006-09-06 14:23:04 +00:00
Eelco Dolstra 3e239a37ff * file:/ -> file:// 2006-08-09 19:37:23 +00:00
Armijn Hemel c0bfcbdd45 rework the --target flag. If this flag is used, a URI should be given. Default
values are not changed.
2006-08-08 15:42:33 +00:00
Eelco Dolstra 5a6b45e252 * In nar.nix, path -> storePath, otherwise we get a collision between
environment variable names on Cygwin (where they are case
  insensitive).
2006-08-07 18:22:57 +00:00
Armijn Hemel a61129c48c add coreutils to the default PATH for this scripts, so we know for sure we have tools like rm, mkdir, and so on 2006-08-05 00:33:52 +00:00
Armijn Hemel f1947cce93 prevent doing recursive chroots, by unsetting NIX_ROOT in the scripts. 2006-08-05 00:31:04 +00:00
Eelco Dolstra f4a502a09a * Use old-school pipe opens; the new style is Perl >= 5.8.0. 2006-08-04 11:51:46 +00:00
Eelco Dolstra 981eff065b * Remove the dependency on `date', use strftime instead. 2006-08-04 11:45:50 +00:00
Armijn Hemel fcb784051f urgh...for some weird reason this one-liner was not in svn. PEBKAC! 2006-08-03 11:48:09 +00:00
Eelco Dolstra a29b64a231 * Better error checking. 2006-08-01 13:50:10 +00:00
Armijn Hemel 4fde308ec0 add a flag --target, so we can override the URL in the MANIFEST file. This
is only for local copies (so file:///)
2006-08-01 13:15:55 +00:00
Eelco Dolstra 4661282fde * `nix-instantiate ... --arg NAME VALUE': allow arguments to be passed
to functions from the command line.
* nix-build: started removing backticks.
2006-07-28 16:03:28 +00:00
Eelco Dolstra dcded7da47 * Don't try to do DNS lookups. 2006-07-21 12:28:25 +00:00
Eelco Dolstra ebcccbd358 * Added a tool to find additional roots for the garbage collector,
such as open files, current directories, mmaped files, etc.  This is
  inherently unportable, but it's easy to adapt this script to other
  platforms.  Currently we call `lsof' and try to read various bits in
  /proc/NNN.

  The goal is to prevent the garbage collector from removing store
  paths that are no longer reachable from a permanent root but that
  are still in use (for instance, after the user has done "nix-env -e"
  on a running program).
2006-07-19 16:49:59 +00:00
Eelco Dolstra 04cf72287b * This may be useful in the future. 2006-05-31 09:24:54 +00:00
Eelco Dolstra e3c07782d1 * Remove old manifests in `nix-channel --update'. 2006-05-08 20:00:28 +00:00
Eelco Dolstra 05bb644890 * Allow the resulting symlink of nix-build to be named, e.g.,
$ nix-build .../i686-linux.nix -A apacheHttpd -o apache
2006-03-14 16:35:01 +00:00
Eelco Dolstra 9811815429 * Write messages to stderr, not stdout. 2006-03-10 09:41:28 +00:00
Eelco Dolstra 120f00c04f * More online help. 2006-03-03 14:25:07 +00:00
Eelco Dolstra 2d54312f87 * Rewrote nix-build in Perl, since sh is just too limited (turns out
that arrays are a bash extension, so it didn't work on FreeBSD).
  Also fixes NIX-8 (readlink(1) dependency).
2006-03-03 14:15:02 +00:00
Eelco Dolstra 85793fa438 * Remove debug statement. 2006-02-24 17:10:38 +00:00
Eelco Dolstra e2f16b9cae * A script to remove from a manifest those patches whose base or
target no longer applies to any available release.  This is a
  partial fix for NIX-34 (when producing linear patch sequences
  between releases, the number of patches grows without bound).
2006-02-24 16:05:55 +00:00
Eelco Dolstra 5168f9bb00 * Canonicalise manifests a bit by sorting them. 2006-02-24 16:02:36 +00:00
Eelco Dolstra 43fa1c20e7 * Separate the cache reading code. 2006-02-22 14:35:19 +00:00
Eelco Dolstra 928cce5922 * Directory for nix.cs.uu.nl manifest / cache maintenance scripts. 2006-02-22 14:13:02 +00:00
Eelco Dolstra fcec32a2c5 * Use right directories for NAR files and patches.
* Print errors to STDERR.
2006-02-22 14:10:02 +00:00
Eelco Dolstra 982399bb14 * Enable the --attr in nix-build as well (and add -A as an alias).
Example:

    $ nix-build ./all-packages.nix -A xlibs.libX11

  So finally it's easy to perform a test build of a Nix expression!
2006-02-10 17:37:35 +00:00
Eelco Dolstra 151f10dbc7 * Add @bindir@. 2006-02-01 12:41:54 +00:00
Eelco Dolstra 83424eb71b * Don't force a build of derivations. 2006-01-26 23:18:46 +00:00
Eelco Dolstra e4d4969ae9 * New tools nix-pack-closure and nix-unpack-closure. These provide a
useful way to transfer the closure of a store path to another
  machine.

  These commands provide functionality previously possible through
  `nix-push --copy'.  However, they are much more convenient in many
  situations (though possibly less efficient).
  
  Example:
  $ nix-pack-closure /nix/store/hj232g1r...-subversion-1.3.0 > svn.closure
  (on another machine:)
  $ nix-unpack-closure < svn.closure

  Note that Subversion is added to the store, but not installed into a
  user environment.  One should do `nix-env -i
  /nix/store/hj232g1r...-subversion-1.3.0' for that.

  Another example: copy the application Azureus to the machine
  `scratchy' through ssh:
  
  $ nix-pack-closure $(which azureus) | ssh scratchy nix-unpack-closure
2006-01-12 15:17:51 +00:00
Eelco Dolstra ab5c6bb3a3 * Change referer' to referrer' throughout. In particular, the
nix-store query options `--referer' and `--referer-closure' have
  been changed to `--referrer' and `--referrer-closure' (but the old
  ones are still accepted for compatibility).
2005-12-13 21:04:48 +00:00
Eelco Dolstra dc528128cc * FreeBSD compatibility fix. 2005-11-17 13:58:23 +00:00
Eelco Dolstra 221c79013f * Turn off build hooks in nix-push because of an impurity (NIX-21). 2005-10-29 18:17:45 +00:00
Armijn Hemel 15ff877438 add @coreutils@ to correctly use coreutils to create a profile. This is needed
for NixOS, where we might not know our PATH in advance.
2005-10-11 17:30:57 +00:00
Eelco Dolstra 607a2f01e6 * Remove other uses of IPC::Open2. 2005-09-21 17:14:52 +00:00
Eelco Dolstra 95304172a5 * Don't use IPC::Open2, it has a subtle race bug on Mac OS X 10.4. If
the parent runs before the child, it closes some pipe file
  descriptors which causes the child to fail due to a bad file
  descriptor.  So we just use the normal open() function instead.
  
  This fixes NIX-14 (intermittent nix-pull failures).
2005-09-21 17:06:06 +00:00
Eelco Dolstra d34fa9a6cc * Remove dead file. 2005-09-16 09:05:18 +00:00
Eelco Dolstra 846b53bde4 * Set the current directory to something well-defined. Might help in
setuid installations.
2005-09-16 09:03:24 +00:00
Eelco Dolstra 7f384d9c1b * Use a proper temporary directory. 2005-09-15 15:21:35 +00:00
Eelco Dolstra a5ceb5bc0b * nix-build: default to `./default.nix' if no paths are specified.
So when using Nix as a build tool, you can just say `nix-build' and
  it will build the top-level derivation defined in `default.nix'.
2005-07-13 17:39:10 +00:00
Eelco Dolstra 928a7c06dc * Don't create patches for archives >= 150 MB because bsdiff can't
handle it.  It crashed on the 234 MB tetex archive.  Probably we
  will never be able to handle archives of that size on 32-bit
  machines (because bsdiff does everything in memory requiring
  max(17*n,9*n+m)+O(1) bytes, so the address space simply isn't
  there).
2005-06-18 14:20:24 +00:00
Eelco Dolstra cbc8d083ac * Make unpacking of patch sequences much faster by not doing redundant
unpacking and repacking of intermediate paths.
2005-05-10 14:22:36 +00:00
Eelco Dolstra 9f3601a36c * Argh! The patch downloader was broken due to the renaming of the
`--isvalid' flag in nix-store.
2005-04-12 10:51:38 +00:00
Eelco Dolstra d5219a351a * Damn. Disable the USE heuristic for now, since the deriver in the
database isn't always in the manifest (so the reference graph cannot
  be reconstructed fully).
2005-04-12 10:07:02 +00:00
Eelco Dolstra c702dfca3f * nix-store: --substitute' -> --register-substitutes'. 2005-04-08 13:48:41 +00:00
Eelco Dolstra 7d876f8fa7 * Get rid of fetchurl, we don't need it anymore. 2005-04-07 14:35:44 +00:00
Eelco Dolstra c815aff21b * `nix-store --add-fixed' to preload the outputs of fixed-output
derivations.  This is mostly to simplify the implementation of
  nix-prefetch-{url, svn}, which now work properly in setuid
  installations.

* Enforce valid store names in `nix-store --add / --add-fixed'.
2005-04-07 14:01:51 +00:00
Eelco Dolstra 31e140d70b * I said it couldn't be done. I was wrong. 2005-04-04 15:18:19 +00:00
Eelco Dolstra ebe342c9c1 * Better error checking. 2005-03-25 14:30:01 +00:00
Eelco Dolstra 7272c3f817 * Ignore hash conflicts in gc-releases.pl. 2005-03-18 09:43:25 +00:00
Eelco Dolstra e52ae1c0ff * Use SHA-256 for nix-push. 2005-03-15 11:12:48 +00:00
Eelco Dolstra 5675d5f488 * Idem. 2005-03-14 18:55:46 +00:00
Eelco Dolstra 6fb5f7e532 * Pass `--base32' unless using MD5. 2005-03-14 18:55:29 +00:00
Eelco Dolstra bb2e53699f * Parse new hash format properly. 2005-03-14 17:05:42 +00:00
Eelco Dolstra 5863f24722 * Print SHA-1 hashes in base-32 by default. 2005-03-14 17:05:20 +00:00
Eelco Dolstra bd333b939c * Prefix hash algorithm in patch generator too. 2005-03-14 16:46:19 +00:00
Eelco Dolstra 8eff18cd43 * Set NAR name to content hash; previous nix-push names were not
unique.
* Drop `hashAlgo' attribute in manifests; prefix hashes with the hash
  algorithm instead.
2005-03-14 15:09:53 +00:00
Eelco Dolstra 1562dfe9ba * Script to garbage collect nix-push directories. It prints out all
file names in the directory not included in any of the manifests
  specified on the command line.
2005-03-14 14:03:41 +00:00
Eelco Dolstra 536f324177 * nix-install-package: install outPath, not drvPath, for now.
* nix-prefecth-url: print out in base-16.
2005-03-11 15:27:37 +00:00
Eelco Dolstra 9e6bca8765 * Channel fix. 2005-03-03 13:10:52 +00:00
Eelco Dolstra 9e50e648a4 * Doh! 2005-03-01 11:27:38 +00:00
Eelco Dolstra 8d364e5baa * Add missing file to dist. 2005-03-01 11:27:22 +00:00
Eelco Dolstra db322a47ff * Use a weighted use heuristic to disambiguate between multiple
occurances of a component.  If the shortest path distance between a
  component P and Q in the referers graph is D, then the contribution
  of Q to the use of P is 1 / R^D, where R >= 1, typically 2.  This
  expresses that distant indirect uses are less important than nearby
  uses.

  For instance, this can disambiguate between the bootstrap GCC in
  Nixpkgs and the GCC of the final stdenv (the former has more uses,
  but they are further away),  and between the GCC of the final stdenv
  and the GCC+G77 build (the latter has very few uses).
2005-03-01 10:33:55 +00:00
Eelco Dolstra 2c4302dd7a * Added a disambiguation heuristic: if two components have the same
name but differ to much in sice (by more than a factor of 3), then
  never generate a patch.
2005-02-28 14:12:06 +00:00
Eelco Dolstra 8376fff151 * Add a version number to manifests. 2005-02-25 16:12:52 +00:00
Eelco Dolstra 8d3c346559 * Pause if errors occur. 2005-02-25 15:58:00 +00:00
Eelco Dolstra 6bafeafb88 * nix-install-package: Use the new (trivial) package format generated
by the build farm.  See e.g.,
  http://catamaran.labs.cs.uu.nl/dist/nixpkgs-0.8/nixpkgs-0.7pre2302/;
  the user can click on packages, and they will be installed (assuming
  the `application/nix-package' MIME type has been associated with
  `nix-install-package').

  Nix expressions are no longer involved: a "package" is just a
  pointer to a manifest, and the top-level store derivation to be
  added to the user environment.  This makes these packages
  independent from Nix expression evolution.

  Note that we install the store derivation ($drvPath), not the
  resulting output path ($outPath).  This is equivalent, except that
  installing the derivation maintains the back-link from the output
  path to the derivation that built it.  This is useful for
  maintenance.

* Automatically re-exec in an xterm so that the user sees something
  when `nix-install-package' is run from a browser.
2005-02-25 15:42:52 +00:00
Eelco Dolstra 3259ae5811 * Properly specify the hash algorithm in the manifests, and read it
too.
* Change the default hash for nix-prefetch-url back to md5, since
  that's what we use in Nixpkgs (for now; a birthday attack is rather
  unlikely there).
2005-02-24 17:36:42 +00:00
Eelco Dolstra 95e870a113 * (Unnecessary) refactoring. 2005-02-24 14:06:18 +00:00
Eelco Dolstra 3a2c3f0cf2 * Support for fixed-output hashes over directory trees (i.e., over the
NAR dump of the path).
2005-02-22 21:14:41 +00:00
Eelco Dolstra fb5dae8694 * Fix nix-channel.
* Add `--help' flag; fixes NIX-5.
* Add `--remove' flag; fixes NIX-6.
* Add `--list' flag.
2005-02-17 10:06:12 +00:00
Eelco Dolstra 98df735b51 * Propagate the deriver of a path through the substitute mechanism.
* Removed some dead code (successor stuff) from nix-push.
* Updated terminology in the tests (store expr -> drv path).
* Check that the deriver is set properly in the tests.
2005-02-09 12:57:13 +00:00
Eelco Dolstra 60feff82cf * Set umask to prevent permission problems. 2005-02-08 13:00:39 +00:00
Eelco Dolstra 48ebe4527e * Better error reporting in readmanifest.
* Use force flag in `mv' to prevent silly interactive questions (this
  happens with shared Nix stores).
2005-02-08 11:40:19 +00:00
Eelco Dolstra 9f6835c282 * Remove debug code. 2005-02-01 17:52:11 +00:00
Eelco Dolstra c3981d81f6 * Make check fixes. 2005-02-01 17:50:48 +00:00
Eelco Dolstra 65b6c8ab4c * Move root finding from nix-collect-garbage' to nix-store --gc'.
This was necessary becase root finding must be done after
  acquisition of the global GC lock.

  This makes `nix-collect-garbage' obsolete; it is now just a wrapper
  around `nix-store --gc'.

* Automatically remove stale GC roots (i.e., indirect GC roots that
  point to non-existent paths).
2005-02-01 15:05:32 +00:00
Eelco Dolstra 630ae0c9d7 * nix-build: use an indirection scheme to make it easier for users to
get rid of GC roots.  Nix-build places a symlink `result' in the
  current directory.  Previously, removing that symlink would not
  remove the store path being linked to as a GC root.  Now, the GC
  root created by nix-build is actually a symlink in
  `/nix/var/nix/gcroots/auto' to `result'.  So if that symlink is
  removed the GC root automatically becomes invalid (since it can no
  longer be resolved).  The root itself is not automatically removed -
  the garbage collector should delete dangling roots.
2005-02-01 13:48:46 +00:00
Eelco Dolstra dcc37c236c * nix-store, nix-instantiate: added an option `--add-root' to
immediately add the result as a permanent GC root.  This is the only
  way to prevent a race with the garbage collector.  For instance, the
  old style

    ln -s $(nix-store -r $(nix-instantiate foo.nix)) \
      /nix/var/nix/gcroots/result

  has two time windows in which the garbage collector can interfere
  (by GC'ing the derivation and the output, respectively).  On the
  other hand,

    nix-store --add-root /nix/var/nix/gcroots/result -r \
      $(nix-instantiate --add-root /nix/var/nix/gcroots/drv \
        foo.nix)

  is safe.

* nix-build: use `--add-root' to prevent GC races.
2005-02-01 12:36:25 +00:00
Eelco Dolstra a6b65fd5e1 * Get rid of hardcoded paths. 2005-02-01 09:54:56 +00:00
Eelco Dolstra 1328aa3307 * Start of concurrent garbage collection. Processes write temporary
roots to a per-process temporary file in /nix/var/nix/temproots
  while holding a write lock on that file.  The garbage collector
  acquires read locks on all those files, thus blocking further
  progress in other Nix processes, and reads the sets of temporary
  roots.
2005-01-31 10:27:25 +00:00
Eelco Dolstra c505702265 * Fix and simplify the garbage collector (it's still not concurrent,
though).  In particular it's now much easier to register a GC root.
  Just place a symlink to whatever store path it is that you want to
  keep in /nix/var/nix/gcroots.
2005-01-27 15:21:29 +00:00
Eelco Dolstra 066da4ab85 * Really fix the substitute mechanism, i.e., ensure the closure
invariant by registering references through the manifest.
* Added a test for nix-pull.
2005-01-25 17:08:52 +00:00
Eelco Dolstra 6ff48e77f6 * Set the Perl search path properly (reported by Roy van den Broek). 2005-01-19 21:55:02 +00:00
Eelco Dolstra f3dc231250 * Removed the `id' attribute hack.
* Formalise the notion of fixed-output derivations, i.e., derivations
  for which a cryptographic hash of the output is known in advance.
  Changes to such derivations should not propagate upwards through the
  dependency graph.  Previously this was done by specifying the hash
  component of the output path through the `id' attribute, but this is
  insecure since you can lie about it (i.e., you can specify any hash
  and then produce a completely different output).  Now the
  responsibility for checking the output is moved from the builder to
  Nix itself.

  A fixed-output derivation can be created by specifying the
  `outputHash' and `outputHashAlgo' attributes, the latter taking
  values `md5', `sha1', and `sha256', and the former specifying the
  actual hash in hexadecimal or in base-32 (auto-detected by looking
  at the length of the attribute value).  MD5 is included for
  compatibility but should be considered deprecated.

* Removed the `drvPath' pseudo-attribute in derivation results.  It's
  no longer necessary.

* Cleaned up the support for multiple output paths in derivation store
  expressions.  Each output now has a unique identifier (e.g., `out',
  `devel', `docs').  Previously there was no way to tell output paths
  apart at the store expression level.

* `nix-hash' now has a flag `--base32' to specify that the hash should
  be printed in base-32 notation.

* `fetchurl' accepts parameters `sha256' and `sha1' in addition to
  `md5'.

* `nix-prefetch-url' now prints out a SHA-1 hash in base-32.  (TODO: a
  flag to specify the hash.)
2005-01-17 16:55:19 +00:00
Eelco Dolstra 9ee88bb2f2 * Use absolute paths. 2005-01-14 13:50:00 +00:00
Eelco Dolstra b17e7cf979 * Script to remove patches from manifests. 2005-01-12 10:40:59 +00:00
Eelco Dolstra 0bc41f632b * Print out less garbage. 2005-01-12 10:37:18 +00:00
Eelco Dolstra 6af4a5a71f * Prototype store optimiser. It searched the Nix store for identical
files and hard-links them to each other to save disk space.

  Currently it doesn't actually do the hard-linking, it just reports
  the amount of space saved if it did.
2005-01-05 09:58:12 +00:00
Eelco Dolstra c53898cb65 * If a patch already exists, it must still be included in the manifest. 2004-12-31 11:07:12 +00:00
Eelco Dolstra 35b76a81c4 * More instrumentation (statistics go to /nix/var/log/nix/downloads). 2004-12-30 17:19:47 +00:00
Eelco Dolstra 3745cecc6a * Fix handling of chained patches: don't skip patches if intermediate
paths are missing, etc.
2004-12-30 17:09:57 +00:00
Eelco Dolstra 581bcb986f * Some logging for evaluation. 2004-12-30 16:34:54 +00:00
Eelco Dolstra 6270aa727d * Propagate patches from the source distribution to the destination
distribution insofar they are applicable.
2004-12-29 22:17:26 +00:00
Eelco Dolstra 54d8f08588 * Reject patches larger than the full archives they produce. 2004-12-29 19:32:55 +00:00
Eelco Dolstra 2fdb27e7f2 * Atomic file replacement is good. 2004-12-29 19:04:21 +00:00
Eelco Dolstra e1e9c036f9 * A utility to generate patches between releases based on their
manifests.
2004-12-29 18:58:15 +00:00