Commit graph

332 commits

Author SHA1 Message Date
Eelco Dolstra d7efd76394 * Big cleanup of the semantics of paths, strings, contexts, string
concatenation and string coercion.  This was a big mess (see
  e.g. NIX-67).  Contexts are now folded into strings, so that they
  don't cause evaluation errors when they're not expected.  The
  semantics of paths has been clarified (see nixexpr-ast.def).
  toString() and coerceToString() have been merged.

  Semantic change: paths are now copied to the store when they're in a
  concatenation (and in most other situations - that's the
  formalisation of the meaning of a path).  So

    "foo " + ./bla

  evaluates to "foo /nix/store/hash...-bla", not "foo
  /path/to/current-dir/bla".  This prevents accidental impurities, and
  is more consistent with the treatment of derivation outputs, e.g.,
  `"foo " + bla' where `bla' is a derivation.  (Here `bla' would be
  replaced by the output path of `bla'.)
2006-10-16 15:55:34 +00:00
Eelco Dolstra 37c8a664f3 * A helpful message. 2006-10-13 11:49:55 +00:00
Eelco Dolstra e4af398681 * Don't crash when upgrading the Berkeley DB environment. 2006-10-13 11:15:53 +00:00
Eelco Dolstra 666babbbfa * Use a bounded amount of memory in scanForReferences() by not reading
regular files into memory all at once.
2006-09-22 13:10:30 +00:00
Eelco Dolstra d22d7565f3 * Don't allocate the buffer twice. 2006-09-22 11:28:23 +00:00
Eelco Dolstra b43aeadbc9 * Don't allocate more than SIZE_MAX bytes. 2006-09-22 11:13:35 +00:00
Eelco Dolstra 5c38c863bd * Fix a huge gaping hole in nix-env w.r.t. the garbage collector.
Nix-env failed to call addPermRoot(), which is necessary to safely
  add a new root.  So if nix-env started after and finished before the
  garbage collector, the user environment (plus all other new stuff)
  it built might be garbage collected, leading to a dangling symlink
  chain in ~/.nix-profile...

* Be more explicit if we block on the GC lock ("waiting for the big
  garbage collector lock...").

* Don't loop trying to create a new generation.  It's not necessary
  anymore since profiles are locked nowadays.
2006-09-14 22:30:33 +00:00
Eelco Dolstra e5a6c09b12 * Install header files in /nix/include/nix. 2006-09-04 22:41:36 +00:00
Eelco Dolstra 4be5443882 * Remove unnecessary inclusions of aterm2.h. 2006-09-04 22:08:40 +00:00
Eelco Dolstra 75068e7d75 * Use a proper namespace.
* Optimise header file usage a bit.
* Compile the parser as C++.
2006-09-04 21:06:23 +00:00
Eelco Dolstra aab8812732 * Store the Nix libraries in ${libdir}/nix instead of ${libdir}. 2006-09-04 15:12:24 +00:00
Martin Bravenboer 3e5b68068b On cygwin, disable the check that the output is not group or world
writable. File permissions on Cygwin are rather complex, and in this
case this check introduced a problem with build jobs invoke from
outside of Cygwin (MSYS). It seemed almost impossible to fix the
permissions of the directory, so for now this safety check is disabled
on Cygwin.
2006-08-15 21:37:48 +00:00
Eelco Dolstra 4250b641d8 * `nix-store --gc --print-dead': print the total size of the store
objects that would be freed.
2006-08-11 20:26:49 +00:00
Eelco Dolstra 92f7dfa5b7 * Don't assume that paths returned by the runtime root finder are
valid.
2006-08-11 20:15:20 +00:00
Eelco Dolstra 6ac237e242 * Show some progress. 2006-08-01 09:43:41 +00:00
Eelco Dolstra a4273156c4 * Use $(libexecdir) to find find-runtime-roots.pl. 2006-07-20 13:21:37 +00:00
Eelco Dolstra ee2cf45d76 * Use debug(). 2006-07-20 12:19:55 +00:00
Eelco Dolstra c15f544356 * Call find-runtime-roots.pl from the garbage collector to prevent
running applications etc. from being garbage collected.
2006-07-20 12:17:25 +00:00
Eelco Dolstra d51aede4af * Allow the canonical system name to be specified at runtime in the
Nix config file.
2006-07-06 15:30:37 +00:00
Eelco Dolstra dbf6d7e783 * Concurrent GC on Cygwin. 2006-06-20 17:48:10 +00:00
Eelco Dolstra cc51f9c539 * Oops. 2006-06-19 16:35:35 +00:00
Eelco Dolstra 5bb3444032 * _exit() doesn't seem to work right on Cygwin. 2006-06-19 16:24:15 +00:00
Eelco Dolstra b35735d8b2 * On Windows we cannot delete open (lock) files, so we delete lock
files after we've closed them.  Since this only succeeds if the lock
  is no longer opened by any process, the token trick used on Unix is
  not necessary.
2006-06-19 14:43:13 +00:00
Eelco Dolstra c937b73622 * Show when we're blocked waiting for a lock. 2006-06-16 10:13:03 +00:00
Eelco Dolstra 588cb0eade * In `nix-env -i|-u|-e', lock the profile to prevent races between
concurrent nix-env operations on the same profile.  Fixes NIX-7.
2006-06-15 11:56:49 +00:00
Eelco Dolstra 23960e92df * Minor cleanup. 2006-06-01 18:13:33 +00:00
Eelco Dolstra 50fe85f016 * For fixed-output derivations, pass the environment variables listed
in the attribute variable `impureEnvVars' from the caller to the
  builder.
2006-05-31 09:51:45 +00:00
Eelco Dolstra 1390ce4142 * Not all platforms have sys/select.h. 2006-05-30 11:37:21 +00:00
Eelco Dolstra 58b4198ed8 * Disable the concurrent garbage collector on Cygwin for now. 2006-05-29 20:46:51 +00:00
Eelco Dolstra d764409d97 * Some Cygwin fixes. 2006-05-24 13:23:20 +00:00
Eelco Dolstra b5988004d6 * Support for srcdir != builddir (NIX-41). 2006-05-12 11:47:45 +00:00
Eelco Dolstra 9d72bf8835 * 64-bit compatibility fixes (for problems revealed by building on an Athlon
64 running 64-bit SUSE).  A patched ATerm library is required to run Nix
  succesfully.
2006-05-11 02:19:43 +00:00
Eelco Dolstra c791e94aee * Removed a bunch of ATreverses. 2006-05-02 17:51:50 +00:00
Eelco Dolstra fdea084c36 * Allow `make check' to work in directories that have symlink
components.
2006-03-10 22:27:26 +00:00
Eelco Dolstra c8bfb11b34 * `nix-env (-i|-u) --dry-run' now shows exactly which missing paths
will be built or substituted.
2006-03-06 11:21:15 +00:00
Eelco Dolstra 74166f2f44 * db.hh shouldn't depend on the Berkeley DB headers. 2006-03-01 17:44:28 +00:00
Eelco Dolstra d822bf32e4 * Close the database before the destructor runs. 2006-03-01 16:36:35 +00:00
Eelco Dolstra b602d2dfdf * Wrong delete. Thanks valgrind. 2006-03-01 14:39:10 +00:00
Eelco Dolstra 6b8bb8d74a * Remove dead code. 2006-03-01 12:51:39 +00:00
Eelco Dolstra 345a95afe9 * Allow the size of the GC reserved file to be specified in nix.conf
through the new `gc-reserved-space' option.
2006-02-16 13:58:10 +00:00
Eelco Dolstra 651ab439cf * A simple hack to fix NIX-18: the garbage collector cannot run when
the disk is full (because to delete something from the Nix store, we
  need a Berkeley DB transaction, which takes up disk space).  Under
  normal operation, we make sure that there exists a file
  /nix/var/nix/db/reserved of 1 MB.  When running the garbage
  collector, we delete that file before we open the Berkeley DB
  environment.
2006-02-16 13:19:15 +00:00
Eelco Dolstra 81de538e46 * Use setsid instead of setpgrp in child processes. This not only
creates a new process group but also a new session.  New sessions
  have no controlling tty, so child processes like ssh cannot open
  /dev/tty (which is bad).
2006-02-03 14:20:59 +00:00
Eelco Dolstra b90daaaf6c * When killing a build hook, kill the entire process group (as
intended).  This ensures that any ssh child processes to remote
  machines are also killed, and thus the Nix process on the remote
  machine also exits.  Without this, the remote Nix process will
  continue until it exists or until its stdout buffer gets full and it
  locks up.  (Partially fixes NIX-35.)
2006-02-02 16:27:31 +00:00
Eelco Dolstra 0f8d3c871b * More GCC 2.95 compatibility. 2005-12-25 11:29:12 +00:00
Eelco Dolstra 52d20ef124 * Hack around a GCC 2.95 bug. 2005-12-25 02:02:29 +00:00
Eelco Dolstra 1440419b45 * GCC 2.95 compatibility. 2005-12-24 23:32:59 +00:00
Eelco Dolstra f96d2dea26 * Added a flag --ignore-liveness' to nix-store --delete'. It
deletes a path even if it is reachable from a root.  However, it
  won't delete a path that still has referrers (since that would
  violate store invariants).

  Don't try this at home.  It's a useful hack for recovering from
  certain situations in a somewhat clean way (e.g., holes in closures
  due to disk corruption).
2005-12-23 21:36:44 +00:00
Eelco Dolstra 4b9e7f59ca * Revived the old "nix-store --delete" operation that deletes the
specified paths from the Nix store.  However, this operation is
  safe: it refuses to delete anything that the garbage collector
  wouldn't delete.
2005-12-23 21:08:42 +00:00
Eelco Dolstra 530b27df1e * `nix-store --gc' prints out the number of bytes freed on stdout
(even when it is interrupted by a signal).
2005-12-15 21:11:39 +00:00
Eelco Dolstra b1eed6b586 * Split the database upgrade into multiple transactions to prevent
Berkeley DB from running out of locks.
2005-12-15 16:53:21 +00:00
Eelco Dolstra ab5c6bb3a3 * Change referer' to referrer' throughout. In particular, the
nix-store query options `--referer' and `--referer-closure' have
  been changed to `--referrer' and `--referrer-closure' (but the old
  ones are still accepted for compatibility).
2005-12-13 21:04:48 +00:00
Eelco Dolstra d87549c1c7 * Automatically delete the old referers table. 2005-12-12 19:14:38 +00:00
Eelco Dolstra 8463f27d8c * Fix NIX-23: quadratic complexity in maintaining the referers
mapping.  The referer table is replaced by a referrer table (note
  spelling fix) that stores each referrer separately.  That is,
  instead of having

    referer[P] = {Q_1, Q_2, Q_3, ...}

  we store

    referer[(P, Q_1)] = ""
    referer[(P, Q_2)] = ""
    referer[(P, Q_3)] = ""
    ...

  To find the referrers of P, we enumerate over the keys with a value
  lexicographically greater than P.  This requires the referrer table
  to be stored as a B-Tree rather than a hash table.

  (The tuples (P, Q) are stored as P + null-byte + Q.)

  Old Nix databases are upgraded automatically to the new schema.
2005-12-12 18:24:42 +00:00
Eelco Dolstra a33b561a6b * Use Berkeley DB 4.4's process registry feature to recover from
crashed Nix instances, and toss out our own recovery code.
2005-12-09 22:55:07 +00:00
Eelco Dolstra 760264bffe * Require Berkeley DB 4.4.
* Checkpoint after an upgrade.
2005-12-06 15:00:04 +00:00
Eelco Dolstra b7f008fc35 * Did something useful while waiting at IAD: reference scanning is now
much faster.
2005-11-16 08:27:06 +00:00
Rob Vermaas f0856fd905 * Repair the referers table from the references table. 2005-10-29 15:44:02 +00:00
Eelco Dolstra 92d599c6a7 * Prevent uids from being used for more than one build
simultaneously.  We do this using exclusive locks on uid files in
  /nix/var/nix/userpool, e.g., /nix/var/nix/userpool/123 for uid 123.
2005-10-20 16:58:34 +00:00
Eelco Dolstra e932c40f8e * Oops. Fixed-output derivations were broken. 2005-10-19 14:27:44 +00:00
Eelco Dolstra 13b089c890 * Also kill all processes of the build user after the build. This is
critical to prevent certain kinds of 0wnage.
2005-10-17 17:43:21 +00:00
Eelco Dolstra f1b3a418fa * Before starting a build under some uid, kill all current processes
running under that uid.
2005-10-17 17:35:37 +00:00
Eelco Dolstra 439823ae80 * Check that the build result is owned by the build user, and that
nobody else has write permission to the build result.  This catches
  most hack attempts.
2005-10-17 16:59:25 +00:00
Eelco Dolstra 7ef574e5d0 * Don't use FIFOs to make Nix create the output path on behalf of the
builder.  Instead, require that the Nix store has sticky permission
  (S_ISVTX); everyone can created files in the Nix store, but they
  cannot delete, rename or modify files created by others.
2005-10-17 16:52:29 +00:00
Eelco Dolstra 32282abcea * Beginning of secure multi-user Nix stores. If Nix is started as
root (or setuid root), then builds will be performed under one of
  the users listed in the `build-users' configuration variables.  This
  is to make it impossible to influence build results externally,
  allowing locally built derivations to be shared safely between
  users (see ASE-2005 paper).

  To do: only one builder should be active per build user.
2005-10-17 15:33:24 +00:00
Eelco Dolstra ebfe57166d * Lets not go wild with templates. 2005-09-22 17:23:43 +00:00
Eelco Dolstra 4578a490ce * Parse multi-valued options. 2005-09-22 15:43:22 +00:00
Eelco Dolstra 8d06842a76 * Configuration options for trusted local builds. 2005-09-21 12:19:39 +00:00
Eelco Dolstra ef9e2c8e73 * Typo. 2005-09-20 16:14:00 +00:00
Eelco Dolstra 4c20a08293 * Build dynamic libraries. 2005-07-22 14:52:45 +00:00
Eelco Dolstra 109cde6706 * Ignore (with a warning) invalid garbage collector roots. 2005-05-10 14:56:10 +00:00
Eelco Dolstra 456f3251d2 * Typo. 2005-05-10 14:21:46 +00:00
Eelco Dolstra 8f57634c14 * Automatically upgrade the Berkeley DB environment if necessary. 2005-05-09 15:25:47 +00:00
Eelco Dolstra 6057b51835 * Don't try to register GC roots in read-only mode. 2005-05-07 21:33:31 +00:00
Eelco Dolstra 5dea0622d1 * Idem (constness fix).
* `compare' in GCC 2.95 is broken.
2005-05-04 16:31:49 +00:00
Eelco Dolstra 4a266e35d4 * GCC 2.95 compatibility fix in constness; strangely, I think this
should not have worked at all.
2005-05-04 16:31:24 +00:00
Eelco Dolstra f913283570 * Remove redundant message. 2005-04-13 09:20:27 +00:00
Eelco Dolstra f3660b1c8c * Garbage collector fix: allow deletion of paths that have invalid
(but substitutable) referers.
2005-04-12 10:51:00 +00:00
Eelco Dolstra cab7816b56 * Slightly nicer message. 2005-04-11 08:07:41 +00:00
Eelco Dolstra c815aff21b * `nix-store --add-fixed' to preload the outputs of fixed-output
derivations.  This is mostly to simplify the implementation of
  nix-prefetch-{url, svn}, which now work properly in setuid
  installations.

* Enforce valid store names in `nix-store --add / --add-fixed'.
2005-04-07 14:01:51 +00:00
Eelco Dolstra 298dd487bb * When finding live paths, the deriver need not be valid. 2005-03-25 14:31:12 +00:00
Eelco Dolstra 7eaf038763 * `nix-store --verify': repair bad referer mappings. 2005-03-25 14:21:49 +00:00
Eelco Dolstra c6178f0b03 * Create missing log and temproots directories automatically (reported
by Rob).
2005-03-24 17:46:38 +00:00
Eelco Dolstra 0df9f08078 * Export the references graph to the build hook. 2005-03-23 13:16:36 +00:00
Eelco Dolstra 3f236f01ae * `nix-store --register-validity': allow a path to refer to a path
listed later in the list of new valid paths.
2005-03-23 13:07:28 +00:00
Eelco Dolstra f20f081560 * nix-store: --isvalid' -> --check-validity', `--validpath' ->
`--register-validity'.
* `nix-store --register-validity': read arguments from stdin, and
  allow the references and deriver to be set.
2005-03-23 11:25:20 +00:00
Eelco Dolstra c757d16c8c * Bug in clearSubstitutes(). 2005-03-14 18:54:40 +00:00
Eelco Dolstra 4bbdcfbb45 * Don't use fork() in copyPath(), but a string buffer. 2005-03-03 13:58:02 +00:00
Eelco Dolstra 86cb3cc554 * Increase Berkeley DB limits a bit more.
* Maintain the cleanup invariant in clearSubstitutes().
2005-03-03 13:10:44 +00:00
Eelco Dolstra 07b4399fb6 * `nix-store -q --hash' to quickly query the hash of the contents of a
store path (which is stored in the database).
2005-03-02 15:57:06 +00:00
Eelco Dolstra bfaf83a0fd * When multiple derivations are specified in `nix-store -r', don't
continue building when one fails unless `--keep-going' is
  specified.
* When `--keep-going' is specified, print out the set of failing
  derivations at the end (otherwise it can be hard to find out which
  failed).
2005-02-23 11:19:27 +00:00
Eelco Dolstra 3a2c3f0cf2 * Support for fixed-output hashes over directory trees (i.e., over the
NAR dump of the path).
2005-02-22 21:14:41 +00:00
Eelco Dolstra eda2c3c253 * Compatibility hack so that Nixpkgs can continue to do hash checking
in `fetchurl' in Nix <= 0.7, but doesn't in Nix 0.8.
2005-02-22 15:23:24 +00:00
Eelco Dolstra 3c1630131e * Subtle bug in the builder: if a subgoal that is instantiated
multiple times is also a top-level goal, then the second and later
  instantiations would never be created because there would be a
  stable pointer to the first one that would keep it alive in the
  WeakGoalMap.
* Some tracing code for debugging this kind of problem.
2005-02-18 09:50:20 +00:00
Eelco Dolstra 8a3a96dd5b * Switch to the calling user context for some more operations in a
setuid installation.
2005-02-17 13:55:18 +00:00
Eelco Dolstra 88273f9574 * Put build logs in $prefix/var/nix/log/drvs/. 2005-02-17 13:54:45 +00:00
Eelco Dolstra 8992fce3da * It is now possible to add store derivations or paths directly to a
user environment, e.g.,

    $ nix-env -i /nix/store/z58v41v21xd3ywrqk1vmvdwlagjx7f10-aterm-2.3.1.drv

  or 

    $ nix-env -i /nix/store/hsyj5pbn0d9iz7q0aj0fga7cpaadvp1l-aterm-2.3.1

  This is useful because it allows Nix expressions to be bypassed
  entirely.  For instance, if only a nix-pull manifest is provided,
  plus the top-level path of some component, it can be installed
  without having to supply the Nix expression (e.g., for obfuscation,
  or to be independent of Nix expression language changes or context
  dependencies).
2005-02-14 17:35:10 +00:00
Eelco Dolstra a04a5de8f7 * Implement the `gc-keep-derivations' global configuretion flag. 2005-02-14 14:16:56 +00:00
Eelco Dolstra 6a8ef36fe6 * Global configuration option `env-keep-derivations' to store pointer
to derivations in user environments.  Nice for developers (since it
  prevents build-time-only dependencies from being GC'ed, in
  conjunction with `gc-keep-outputs').  Turned off by default.
2005-02-14 13:07:09 +00:00
Eelco Dolstra 32429142cd * Type error in constructor call (caught by GCC 3.3, but not 3.4!). 2005-02-14 09:53:11 +00:00
Eelco Dolstra 80870d9291 * Input sources should be in the set of all referenceable paths too. 2005-02-11 16:03:47 +00:00
Eelco Dolstra 3a99616968 * Commit more often to prevent out-of-memory errors. 2005-02-09 14:37:24 +00:00
Eelco Dolstra 98df735b51 * Propagate the deriver of a path through the substitute mechanism.
* Removed some dead code (successor stuff) from nix-push.
* Updated terminology in the tests (store expr -> drv path).
* Check that the deriver is set properly in the tests.
2005-02-09 12:57:13 +00:00
Eelco Dolstra 582e01c06f * Automatically upgrade <= 0.7 Nix stores to the new schema (so that
existing user environments continue to work).
* `nix-store --verify': detect incomplete closures.
2005-02-09 09:50:29 +00:00
Eelco Dolstra c547439843 * Subflag in --verify': nix-store --verify --check-contents' checks
that the contents of store paths has not changed by comparing hashes
  of their current contents to the hashes stored in the database.
2005-02-08 13:48:53 +00:00
Eelco Dolstra 3d74274b37 * Updated `nix-store --verify' to the new schema. 2005-02-08 13:23:55 +00:00
Eelco Dolstra fbc434ee4c * `nix-store -qb' to query derivation environment bindings. Useful
for finding build-time dependencies (possibly after a build).  E.g.,

    $ nix-store -qb aterm $(nix-store -qd $(which strc))
    /nix/store/jw7c7s65n1gwhxpn35j9rgcci6ilzxym-aterm-2.3.1

* Arguments to nix-store can be files within store objects, e.g.,
  /nix/store/jw7c...-aterm-2.3.1/bin/baffle.

* Idem for garbage collector roots.
2005-02-07 14:32:44 +00:00
Eelco Dolstra 450c358e20 * Maintain a database table (`derivers') that maps output paths to the
derivation that produced them.
* `nix-store -qd PATH' prints out the derivation that produced a path.
2005-02-07 13:40:40 +00:00
Eelco Dolstra a37338815d * A GC setting `gc-keep-outputs' to specify whether output paths of
derivations should be kept.
2005-02-01 22:07:48 +00:00
Eelco Dolstra 65b6c8ab4c * Move root finding from nix-collect-garbage' to nix-store --gc'.
This was necessary becase root finding must be done after
  acquisition of the global GC lock.

  This makes `nix-collect-garbage' obsolete; it is now just a wrapper
  around `nix-store --gc'.

* Automatically remove stale GC roots (i.e., indirect GC roots that
  point to non-existent paths).
2005-02-01 15:05:32 +00:00
Eelco Dolstra 630ae0c9d7 * nix-build: use an indirection scheme to make it easier for users to
get rid of GC roots.  Nix-build places a symlink `result' in the
  current directory.  Previously, removing that symlink would not
  remove the store path being linked to as a GC root.  Now, the GC
  root created by nix-build is actually a symlink in
  `/nix/var/nix/gcroots/auto' to `result'.  So if that symlink is
  removed the GC root automatically becomes invalid (since it can no
  longer be resolved).  The root itself is not automatically removed -
  the garbage collector should delete dangling roots.
2005-02-01 13:48:46 +00:00
Eelco Dolstra dcc37c236c * nix-store, nix-instantiate: added an option `--add-root' to
immediately add the result as a permanent GC root.  This is the only
  way to prevent a race with the garbage collector.  For instance, the
  old style

    ln -s $(nix-store -r $(nix-instantiate foo.nix)) \
      /nix/var/nix/gcroots/result

  has two time windows in which the garbage collector can interfere
  (by GC'ing the derivation and the output, respectively).  On the
  other hand,

    nix-store --add-root /nix/var/nix/gcroots/result -r \
      $(nix-instantiate --add-root /nix/var/nix/gcroots/drv \
        foo.nix)

  is safe.

* nix-build: use `--add-root' to prevent GC races.
2005-02-01 12:36:25 +00:00
Eelco Dolstra 06b4424286 * Add missing files to dist.
* Fix GC and substitute bugs related to self-references.  Add a
  regression test.
2005-02-01 09:23:38 +00:00
Eelco Dolstra 32fa82a56a * Acquire a global GC lock to prevent new temporary root files from
being created after the garbage collector has read the temproots
  directory.  This blocks the creation of new processes, but the
  garbage collector could periodically release the GC lock to allow
  them to run.
2005-01-31 22:23:49 +00:00
Eelco Dolstra 89c9bc11ab * Add a test for a more subtle race: a process starting after the
temporary root files have been read but creating outputs before the
  store directory has been read.
2005-01-31 22:01:55 +00:00
Eelco Dolstra 207bdcbe86 * Automatically remove temporary root files. 2005-01-31 21:20:59 +00:00
Eelco Dolstra 252c9c91ab * Topologically sort paths under the references relation to ensure
that they are deleted in an order that maintains the closure
  invariant.
* Presence of a path in a temporary roots file does not imply that all
  paths in its closure are also present, so add the closure.
2005-01-31 14:00:43 +00:00
Eelco Dolstra 33c5d23b81 * Don't delete active lock files. 2005-01-31 12:19:53 +00:00
Eelco Dolstra 1328aa3307 * Start of concurrent garbage collection. Processes write temporary
roots to a per-process temporary file in /nix/var/nix/temproots
  while holding a write lock on that file.  The garbage collector
  acquires read locks on all those files, thus blocking further
  progress in other Nix processes, and reads the sets of temporary
  roots.
2005-01-31 10:27:25 +00:00
Eelco Dolstra 0ea8b6993a * Only invalidate paths when they are in fact valid. 2005-01-28 11:05:46 +00:00
Eelco Dolstra 8a3eef22e3 * Fix deadlock. 2005-01-27 17:48:14 +00:00
Eelco Dolstra c60a4943ba * Update referers mappings when updating/clearing the references
mapping.
* Do things in the right order in invalidatePath().
2005-01-27 16:18:39 +00:00
Eelco Dolstra c505702265 * Fix and simplify the garbage collector (it's still not concurrent,
though).  In particular it's now much easier to register a GC root.
  Just place a symlink to whatever store path it is that you want to
  keep in /nix/var/nix/gcroots.
2005-01-27 15:21:29 +00:00
Eelco Dolstra 59682e6188 * Make lock removal safe by signalling to blocked processes that the
lock they are waiting on has become stale (we do this by writing a
  meaningless token to the unlinked file).
2005-01-27 12:19:25 +00:00
Eelco Dolstra a24b78e9f1 * Maintain the references/referers relation also for derivations.
This simplifies garbage collection and `nix-store --query
  --requisites' since we no longer need to treat derivations
  specially.

* Better maintaining of the invariants, e.g., setReferences() can only
  be called on a valid/substitutable path.
2005-01-25 21:28:25 +00:00
Eelco Dolstra 2a2756b856 * Simplification: registerSubstitutes -> registerSubstitute. We no
longer need the former since there we no longer have the
  substitutes-rev table (which triggered a O(n^2) cost in updating
  them).
2005-01-25 20:27:40 +00:00
Eelco Dolstra 066da4ab85 * Really fix the substitute mechanism, i.e., ensure the closure
invariant by registering references through the manifest.
* Added a test for nix-pull.
2005-01-25 17:08:52 +00:00
Eelco Dolstra c6290e42bc * Fix the `--fallback' switch.
* Fix the substitutes tests.
2005-01-25 13:00:12 +00:00
Eelco Dolstra 581fc47783 * Fix the build hook mechanism; pass the pointer graph to the hook. 2005-01-25 11:55:43 +00:00
Eelco Dolstra 52bf9b86bb * In nix-store: added query `--referers-closure' that returns the
closure of the referers relation rather than the references
  relation, i.e., the set of all paths that directly or indirectly
  refer to the given path.  Note that contrary to the references
  closure this set is not fixed; it can change as paths are added to
  or removed from the store.
2005-01-25 11:18:03 +00:00
Eelco Dolstra 80faa2f98a * In nix-store: change --build' back to --realise'. Also brought
back the query flag `--force-realise'.
* Fixed some of the tests.
2005-01-25 10:55:33 +00:00
Eelco Dolstra 6a0a2d5593 * Terminology fixes. 2005-01-20 16:01:07 +00:00
Eelco Dolstra 05f0430de1 * Another change to low-level derivations. The last one this year, I
promise :-) This allows derivations to specify on *what* output
  paths of input derivations they are dependent.  This helps to
  prevent unnecessary downloads.  For instance, a build might be
  dependent on the `devel' and `lib' outputs of some library
  component, but not the `docs' output.
2005-01-20 14:10:19 +00:00
Eelco Dolstra e0f4e587c3 * Nix-store queries --references' and referers' to query the pointer
graph.  That is, `nix-store --query --references PATH' shows the set
  of paths referenced by PATH, and `nix-store --query --referers PATH'
  shows the set of paths referencing PATH.
2005-01-19 16:59:56 +00:00
Eelco Dolstra 96de272b48 * Renamed normalise.cc' -> build.cc', `storeexprs.cc' ->
`derivations.cc', etc.
* Store the SHA-256 content hash of store paths in the database after
  they have been built/added.  This is so that we can check whether
  the store has been messed with (a la `rpm --verify').
* When registering path validity, verify that the closure property
  holds.
2005-01-19 16:39:47 +00:00
Eelco Dolstra ef5f254a55 * `nix-store --build' now builds its arguments in parallel instead of
sequentially (within the limits set by `--jobs').  This should
  greatly improve the utilisation of the build farm when doing Nixpkgs
  builds.
2005-01-19 15:02:02 +00:00
Eelco Dolstra 06c77bf7a8 * Change extension .store' to .drv'.
* Re-enable `nix-store --query --requisites'.
2005-01-19 14:36:00 +00:00
Eelco Dolstra 863dcff6c5 * Started removing closure store expressions, i.e., the explicit
representation of closures as ATerms in the Nix store.  Instead, the
  file system pointer graph is now stored in the Nix database.  This
  has many advantages:

  - It greatly simplifies the implementation (we can drop the notion
    of `successors', and so on).

  - It makes registering roots for the garbage collector much easier.
    Instead of specifying the closure expression as a root, you can
    simply specify the store path that must be retained as a root.
    This could not be done previously, since there was no way to find
    the closure store expression containing a given store path.
    
  - Better traceability: it is now possible to query what paths are
    referenced by a path, and what paths refer to a path.
2005-01-19 11:16:11 +00:00
Eelco Dolstra 32aac8748a * Actually check that the result of fixed-output derivations matches
the specified hash.
2005-01-17 19:01:48 +00:00
Eelco Dolstra f3dc231250 * Removed the `id' attribute hack.
* Formalise the notion of fixed-output derivations, i.e., derivations
  for which a cryptographic hash of the output is known in advance.
  Changes to such derivations should not propagate upwards through the
  dependency graph.  Previously this was done by specifying the hash
  component of the output path through the `id' attribute, but this is
  insecure since you can lie about it (i.e., you can specify any hash
  and then produce a completely different output).  Now the
  responsibility for checking the output is moved from the builder to
  Nix itself.

  A fixed-output derivation can be created by specifying the
  `outputHash' and `outputHashAlgo' attributes, the latter taking
  values `md5', `sha1', and `sha256', and the former specifying the
  actual hash in hexadecimal or in base-32 (auto-detected by looking
  at the length of the attribute value).  MD5 is included for
  compatibility but should be considered deprecated.

* Removed the `drvPath' pseudo-attribute in derivation results.  It's
  no longer necessary.

* Cleaned up the support for multiple output paths in derivation store
  expressions.  Each output now has a unique identifier (e.g., `out',
  `devel', `docs').  Previously there was no way to tell output paths
  apart at the store expression level.

* `nix-hash' now has a flag `--base32' to specify that the hash should
  be printed in base-32 notation.

* `fetchurl' accepts parameters `sha256' and `sha1' in addition to
  `md5'.

* `nix-prefetch-url' now prints out a SHA-1 hash in base-32.  (TODO: a
  flag to specify the hash.)
2005-01-17 16:55:19 +00:00
Eelco Dolstra d58a11e019 * Shorten SHA-256 hashes used in store path name generation to 160
bits, then encode them in a radix-32 representation (using digits
  and letters except e, o, u, and t).  This produces store paths like
  /nix/store/4i0zb0z7f88mwghjirkz702a71dcfivn-aterm-2.3.1.  The nice
  thing about this is that the hash part of the file name is still 32
  characters, as before with MD5.

  (Of course, shortening SHA-256 to 160 bits makes it no better than
  SHA-160 in theory, but hopefully it's a bit more resistant to
  attacks; it's certainly a lot slower.)
2005-01-14 16:04:03 +00:00
Eelco Dolstra 9530cc3170 * Start move towards SHA-256 hashes instead of MD5.
* Start cleaning up unique store path generation (they weren't always
  unique; in particular the suffix ("-aterm-2.2", "-builder.sh") was
  not part of the hash, therefore changes to the suffix would cause
  multiple store objects with the same hash).
2005-01-14 13:51:38 +00:00
Eelco Dolstra 7e8961f720 * Added SHA-1 support. nix-hash' now has an option --type sha1' to
select SHA-1 hashing.
2005-01-13 17:39:26 +00:00
Eelco Dolstra 73992371a3 * Refactoring to support SHA-1. 2005-01-13 15:44:44 +00:00
Eelco Dolstra a03397be4c * Cygwin compatibility. 2005-01-04 17:38:26 +00:00
Eelco Dolstra 8b9697e575 * An operation `nix-store --clear-substitutes' to remove all
registered substitute mappings.
2004-12-20 14:16:55 +00:00
Eelco Dolstra fa9259f5f5 * Simplify the substitute mechanism:
- Drop the store expression.  So now a substitute is just a
    command-line invocation (a program name + arguments).  If you
    register a substitute you are responsible for registering the
    expression that built it (if any) as a root of the garbage
    collector.
  - Drop the substitutes-rev DB table.
2004-12-20 13:43:32 +00:00
Eelco Dolstra 13f77276d1 * utime() follows symlinks, so don't change the mtime if the file is a
symlink.
2004-11-29 19:22:16 +00:00
Eelco Dolstra f17553a212 * Remove debug statement. 2004-11-29 15:30:44 +00:00
Eelco Dolstra 4115d8d8ce * Canonicalise metadata of all files in store paths (set the mtime to
0, set the mode to either 444 or 555, set the group to the default).
2004-11-29 15:09:29 +00:00
Eelco Dolstra cb7ccb528b * string2ATerm -> overloaded toATerm. 2004-11-03 18:12:03 +00:00