Commit graph

141 commits

Author SHA1 Message Date
Eelco Dolstra 9e50e648a4 * Doh! 2005-03-01 11:27:38 +00:00
Eelco Dolstra 8d364e5baa * Add missing file to dist. 2005-03-01 11:27:22 +00:00
Eelco Dolstra db322a47ff * Use a weighted use heuristic to disambiguate between multiple
occurances of a component.  If the shortest path distance between a
  component P and Q in the referers graph is D, then the contribution
  of Q to the use of P is 1 / R^D, where R >= 1, typically 2.  This
  expresses that distant indirect uses are less important than nearby
  uses.

  For instance, this can disambiguate between the bootstrap GCC in
  Nixpkgs and the GCC of the final stdenv (the former has more uses,
  but they are further away),  and between the GCC of the final stdenv
  and the GCC+G77 build (the latter has very few uses).
2005-03-01 10:33:55 +00:00
Eelco Dolstra 2c4302dd7a * Added a disambiguation heuristic: if two components have the same
name but differ to much in sice (by more than a factor of 3), then
  never generate a patch.
2005-02-28 14:12:06 +00:00
Eelco Dolstra 8376fff151 * Add a version number to manifests. 2005-02-25 16:12:52 +00:00
Eelco Dolstra 8d3c346559 * Pause if errors occur. 2005-02-25 15:58:00 +00:00
Eelco Dolstra 6bafeafb88 * nix-install-package: Use the new (trivial) package format generated
by the build farm.  See e.g.,
  http://catamaran.labs.cs.uu.nl/dist/nixpkgs-0.8/nixpkgs-0.7pre2302/;
  the user can click on packages, and they will be installed (assuming
  the `application/nix-package' MIME type has been associated with
  `nix-install-package').

  Nix expressions are no longer involved: a "package" is just a
  pointer to a manifest, and the top-level store derivation to be
  added to the user environment.  This makes these packages
  independent from Nix expression evolution.

  Note that we install the store derivation ($drvPath), not the
  resulting output path ($outPath).  This is equivalent, except that
  installing the derivation maintains the back-link from the output
  path to the derivation that built it.  This is useful for
  maintenance.

* Automatically re-exec in an xterm so that the user sees something
  when `nix-install-package' is run from a browser.
2005-02-25 15:42:52 +00:00
Eelco Dolstra 3259ae5811 * Properly specify the hash algorithm in the manifests, and read it
too.
* Change the default hash for nix-prefetch-url back to md5, since
  that's what we use in Nixpkgs (for now; a birthday attack is rather
  unlikely there).
2005-02-24 17:36:42 +00:00
Eelco Dolstra 95e870a113 * (Unnecessary) refactoring. 2005-02-24 14:06:18 +00:00
Eelco Dolstra 3a2c3f0cf2 * Support for fixed-output hashes over directory trees (i.e., over the
NAR dump of the path).
2005-02-22 21:14:41 +00:00
Eelco Dolstra fb5dae8694 * Fix nix-channel.
* Add `--help' flag; fixes NIX-5.
* Add `--remove' flag; fixes NIX-6.
* Add `--list' flag.
2005-02-17 10:06:12 +00:00
Eelco Dolstra 98df735b51 * Propagate the deriver of a path through the substitute mechanism.
* Removed some dead code (successor stuff) from nix-push.
* Updated terminology in the tests (store expr -> drv path).
* Check that the deriver is set properly in the tests.
2005-02-09 12:57:13 +00:00
Eelco Dolstra 60feff82cf * Set umask to prevent permission problems. 2005-02-08 13:00:39 +00:00
Eelco Dolstra 48ebe4527e * Better error reporting in readmanifest.
* Use force flag in `mv' to prevent silly interactive questions (this
  happens with shared Nix stores).
2005-02-08 11:40:19 +00:00
Eelco Dolstra 9f6835c282 * Remove debug code. 2005-02-01 17:52:11 +00:00
Eelco Dolstra c3981d81f6 * Make check fixes. 2005-02-01 17:50:48 +00:00
Eelco Dolstra 65b6c8ab4c * Move root finding from nix-collect-garbage' to nix-store --gc'.
This was necessary becase root finding must be done after
  acquisition of the global GC lock.

  This makes `nix-collect-garbage' obsolete; it is now just a wrapper
  around `nix-store --gc'.

* Automatically remove stale GC roots (i.e., indirect GC roots that
  point to non-existent paths).
2005-02-01 15:05:32 +00:00
Eelco Dolstra 630ae0c9d7 * nix-build: use an indirection scheme to make it easier for users to
get rid of GC roots.  Nix-build places a symlink `result' in the
  current directory.  Previously, removing that symlink would not
  remove the store path being linked to as a GC root.  Now, the GC
  root created by nix-build is actually a symlink in
  `/nix/var/nix/gcroots/auto' to `result'.  So if that symlink is
  removed the GC root automatically becomes invalid (since it can no
  longer be resolved).  The root itself is not automatically removed -
  the garbage collector should delete dangling roots.
2005-02-01 13:48:46 +00:00
Eelco Dolstra dcc37c236c * nix-store, nix-instantiate: added an option `--add-root' to
immediately add the result as a permanent GC root.  This is the only
  way to prevent a race with the garbage collector.  For instance, the
  old style

    ln -s $(nix-store -r $(nix-instantiate foo.nix)) \
      /nix/var/nix/gcroots/result

  has two time windows in which the garbage collector can interfere
  (by GC'ing the derivation and the output, respectively).  On the
  other hand,

    nix-store --add-root /nix/var/nix/gcroots/result -r \
      $(nix-instantiate --add-root /nix/var/nix/gcroots/drv \
        foo.nix)

  is safe.

* nix-build: use `--add-root' to prevent GC races.
2005-02-01 12:36:25 +00:00
Eelco Dolstra a6b65fd5e1 * Get rid of hardcoded paths. 2005-02-01 09:54:56 +00:00
Eelco Dolstra 1328aa3307 * Start of concurrent garbage collection. Processes write temporary
roots to a per-process temporary file in /nix/var/nix/temproots
  while holding a write lock on that file.  The garbage collector
  acquires read locks on all those files, thus blocking further
  progress in other Nix processes, and reads the sets of temporary
  roots.
2005-01-31 10:27:25 +00:00
Eelco Dolstra c505702265 * Fix and simplify the garbage collector (it's still not concurrent,
though).  In particular it's now much easier to register a GC root.
  Just place a symlink to whatever store path it is that you want to
  keep in /nix/var/nix/gcroots.
2005-01-27 15:21:29 +00:00
Eelco Dolstra 066da4ab85 * Really fix the substitute mechanism, i.e., ensure the closure
invariant by registering references through the manifest.
* Added a test for nix-pull.
2005-01-25 17:08:52 +00:00
Eelco Dolstra 6ff48e77f6 * Set the Perl search path properly (reported by Roy van den Broek). 2005-01-19 21:55:02 +00:00
Eelco Dolstra f3dc231250 * Removed the `id' attribute hack.
* Formalise the notion of fixed-output derivations, i.e., derivations
  for which a cryptographic hash of the output is known in advance.
  Changes to such derivations should not propagate upwards through the
  dependency graph.  Previously this was done by specifying the hash
  component of the output path through the `id' attribute, but this is
  insecure since you can lie about it (i.e., you can specify any hash
  and then produce a completely different output).  Now the
  responsibility for checking the output is moved from the builder to
  Nix itself.

  A fixed-output derivation can be created by specifying the
  `outputHash' and `outputHashAlgo' attributes, the latter taking
  values `md5', `sha1', and `sha256', and the former specifying the
  actual hash in hexadecimal or in base-32 (auto-detected by looking
  at the length of the attribute value).  MD5 is included for
  compatibility but should be considered deprecated.

* Removed the `drvPath' pseudo-attribute in derivation results.  It's
  no longer necessary.

* Cleaned up the support for multiple output paths in derivation store
  expressions.  Each output now has a unique identifier (e.g., `out',
  `devel', `docs').  Previously there was no way to tell output paths
  apart at the store expression level.

* `nix-hash' now has a flag `--base32' to specify that the hash should
  be printed in base-32 notation.

* `fetchurl' accepts parameters `sha256' and `sha1' in addition to
  `md5'.

* `nix-prefetch-url' now prints out a SHA-1 hash in base-32.  (TODO: a
  flag to specify the hash.)
2005-01-17 16:55:19 +00:00
Eelco Dolstra 9ee88bb2f2 * Use absolute paths. 2005-01-14 13:50:00 +00:00
Eelco Dolstra b17e7cf979 * Script to remove patches from manifests. 2005-01-12 10:40:59 +00:00
Eelco Dolstra 0bc41f632b * Print out less garbage. 2005-01-12 10:37:18 +00:00
Eelco Dolstra 6af4a5a71f * Prototype store optimiser. It searched the Nix store for identical
files and hard-links them to each other to save disk space.

  Currently it doesn't actually do the hard-linking, it just reports
  the amount of space saved if it did.
2005-01-05 09:58:12 +00:00
Eelco Dolstra c53898cb65 * If a patch already exists, it must still be included in the manifest. 2004-12-31 11:07:12 +00:00
Eelco Dolstra 35b76a81c4 * More instrumentation (statistics go to /nix/var/log/nix/downloads). 2004-12-30 17:19:47 +00:00
Eelco Dolstra 3745cecc6a * Fix handling of chained patches: don't skip patches if intermediate
paths are missing, etc.
2004-12-30 17:09:57 +00:00
Eelco Dolstra 581bcb986f * Some logging for evaluation. 2004-12-30 16:34:54 +00:00
Eelco Dolstra 6270aa727d * Propagate patches from the source distribution to the destination
distribution insofar they are applicable.
2004-12-29 22:17:26 +00:00
Eelco Dolstra 54d8f08588 * Reject patches larger than the full archives they produce. 2004-12-29 19:32:55 +00:00
Eelco Dolstra 2fdb27e7f2 * Atomic file replacement is good. 2004-12-29 19:04:21 +00:00
Eelco Dolstra e1e9c036f9 * A utility to generate patches between releases based on their
manifests.
2004-12-29 18:58:15 +00:00
Eelco Dolstra 9022cf9adf * A small utility to add the Size and NarHash fields to old manifests. 2004-12-28 21:12:00 +00:00
Eelco Dolstra 4bf58d5379 * Added a function to write manifests. 2004-12-28 21:11:28 +00:00
Eelco Dolstra 3d1b2101cc * Place manifests in /nix/var/nix/manifests.
* Use the new patch downloader.
2004-12-20 16:38:50 +00:00
Eelco Dolstra 7eed57e784 * Sync with changed substitute mechanism.
* Accept the NarHash line.
* Clear substitutes in `nix-channel --update'.
2004-12-20 14:57:03 +00:00
Eelco Dolstra 4d25b0b0bb * Fix nix-pull. 2004-12-16 15:31:50 +00:00
Eelco Dolstra 77970f8daf * Remove `prebuilts.conf' file, it's not like anybody was using it.
* Add /nix/var/nix/manifests directory.
2004-12-16 14:31:49 +00:00
Eelco Dolstra e3b051aeeb * Include the size of the bzipped archive (necessary for computing the
cheapest download path), as well as the hash of the contents of the
  path (necessary for checking patch applicability).
2004-12-13 16:56:18 +00:00
Eelco Dolstra 862f4c154e * Patch deployment. `download.pl' (intended to be used in the
substitute mechanism) creates a store path by downloading full NAR
  archives and/or patches specified in the available manifests.

  Any combination of present paths, full downloads, and patches can be
  used to construct the target path.  In particular, patches can be
  chained in sequence; and full NAR archives of the target path can be
  omitted (i.e., patch-only deployment is possible).  A shortest path
  algorithm is used to find the smallest set of files to be downloaded
  (the edge weights are currently file sizes, but one can imagine
  taking the network speed to the various source into account).

  Patches are binary deltas between two store paths.  To be precise,
  they are the output of the `bsdiff' program applied to the NAR
  archives obtained by dumping (`nix-store --dump') the two store
  paths.  The advantage of diff'ing NAR archives (and not, say, doing
  file-by-file diffs) is that file renames/moves are handled
  automatically.  The disadvantage is that we cannot optimise creation
  of unchanged files (by hard-linking).
2004-12-13 13:47:38 +00:00
Eelco Dolstra dca48aed34 * Allow an optional hash to be provided. This prevents redundant
fetches.
2004-12-13 13:35:36 +00:00
Eelco Dolstra feb3ceaee0 * Better error messages. 2004-11-04 20:20:39 +00:00
Eelco Dolstra 2aa1f4717b * Fix File exists' errors if the result' symlink exists but is
dangling.
2004-10-31 12:01:16 +00:00
Eelco Dolstra 2155c0a673 * Register channels as roots of the garbage collector (in
$(localstatedir)/nix/gcroots/channels). 
* In setuid installations, create gcroots/tmp and gcroots/channels
  group-writable.
2004-10-20 14:42:38 +00:00
Eelco Dolstra 88888160d2 * Fix nix-prefetch-url in setuid Nix installations. 2004-10-20 14:40:54 +00:00