puck/lix
0
0
Fork 0
forked from lix-project/lix
Code Pull requests Activity
2499 commits 19 branches 0 tags 68 MiB
Commit graph

561 commits

Author SHA1 Message Date
Eelco Dolstra a443c7573b * Hack to allow derivations to disable chroot builds by setting the 
attribute "__noChroot = true" (requested by Rob).
 2010-06-01 10:01:14 +00:00
Eelco Dolstra aa45027818 * Sync with the trunk. 2010-05-12 22:13:09 +00:00
Eelco Dolstra f92c9a0ac5 * Allow unprivileged users to do `nix-store --clear-failed-paths' and 
`nix-store --query-failed-paths'.
 2010-05-04 10:45:10 +00:00
Eelco Dolstra 6199f9b93e * Added a command `nix-store --clear-failed-paths <PATHS>' to clear 
the "failed" status of the given store paths.  The special value `*'
  clears all failed paths.
 2010-04-26 12:56:42 +00:00
Eelco Dolstra 2398af13c5 * Add an command `nix-store --query-failed-paths' to list the cached 
failed paths (when using the `build-cache-failure' option).
 2010-04-26 12:43:42 +00:00
Eelco Dolstra 6bbfe95e30 * Don't use an ostringstream in unparseDerivation(), because it's 
slow.  A `nix-env -qa --drv-path \*' on Nixpkgs was spending 40% of
  its time in unparseDerivation() because of this (now 11%).
 2010-04-21 19:25:50 +00:00
Eelco Dolstra f3b8833a48 * Drop the dependency on the ATerm library. 2010-04-19 14:51:58 +00:00
Eelco Dolstra efc7a579e8 * Don't use the ATerm library for parsing/printing .drv files. 2010-04-19 13:46:58 +00:00
Eelco Dolstra e020d80e4e * Sync with the trunk. 2010-03-11 15:45:05 +00:00
Ludovic Courtès 05e15049a5 Show the build user's group in /etc/group in chroots. 
* src/libstore/build.cc (nix::DerivationGoal::startBuilder): Create
  /etc/group showing the build user's group.
 2010-03-11 14:47:04 +00:00
Eelco Dolstra 070057c1b9 * Sync with the trunk. 2010-03-11 10:52:52 +00:00
Ludovic Courtès c752c9f41a Fix thinko in r20547. 
* src/libstore/build.cc (nix::DerivationGoal::startBuilder): Fix the GID
  of the build user in /etc/passwd.
 2010-03-11 10:33:04 +00:00
Ludovic Courtès 2e8eaca573 Clear supplementary groups of `nixbld' in /etc/passwd in chroots. 
* src/libstore/build.cc (nix::DerivationGoal::startBuilder): Don't
  display any supplementary groups for `nixbld' in /etc/passwd.
 2010-03-11 10:21:23 +00:00
Eelco Dolstra 1a65142ec4 * Remove a debug statement. 2010-03-10 12:46:25 +00:00
Eelco Dolstra 4c356acd04 * In `nix-store --export', abort if the contents of a path has 
changed.  This prevents corrupt paths from spreading to other
  machines.  Note that checking the hash is cheap because we're
  hashing anyway (because of the --sign feature).
 2010-03-09 14:32:03 +00:00
Eelco Dolstra 44f6e6de77 * Set gc-keep-outputs' or gc-keep-derivations' to false with 
`--delete --ignore-liveness'.
 2010-03-08 21:31:42 +00:00
Eelco Dolstra 2e4ef03aa3 * Increase the sqlite timeout. 2010-03-08 10:35:45 +00:00
Eelco Dolstra bc6f7fc139 * Remove some unused functions. 2010-03-04 13:03:26 +00:00
Eelco Dolstra 3f9e647ae8 * checkInterrupt() shouldn't be called from a destructor. 2010-03-02 19:04:17 +00:00
Eelco Dolstra 594eaddd11 * When using the included sqlite/aterm libraries, build with 
--enable-shared.
* In libutil/libstore/libexpr etc., link against sqlite and aterm.
* Some more header file hygiene.
 2010-03-02 15:58:13 +00:00
Eelco Dolstra 24035b98b1 * Implement RemoteStore::queryValidPaths(). 2010-02-26 12:05:01 +00:00
Eelco Dolstra e42401ee7b * Implement RemoteStore::queryDerivationOutputs(). 2010-02-25 15:52:22 +00:00
Eelco Dolstra af565c348a * Support read-only access to the database. 2010-02-24 16:44:43 +00:00
Eelco Dolstra e33f67ff0b * Refactor the upgrade / database initialisation logic a bit. 2010-02-24 16:30:20 +00:00
Eelco Dolstra 84a4dd5ff0 * Don't use fdatasync since it doesn't work on Snow Leopard. 
* Don't refer to config.h in util.hh, because config.h is not
  installed (http://hydra.nixos.org/build/303053).
 2010-02-24 15:46:06 +00:00
Eelco Dolstra cfe742cfc5 * A function to query just the database id of a valid path. 2010-02-24 15:07:23 +00:00
Eelco Dolstra 6baa2a2f5e 2010-02-24 14:39:52 +00:00
Eelco Dolstra 9fd85c94de * Use `truncate' journal mode, which should be a bit faster. 2010-02-24 14:22:34 +00:00
Eelco Dolstra a3c63d0d6c * Disable fsync() in SQLite if the fsync-metadata option is set to 
false.
* Change the default for `fsync-metadata' to true.
* Disable `fsync-metadata' in `make check'.
 2010-02-24 13:12:57 +00:00
Eelco Dolstra 90b6352d0a * Do registerValidPaths() in one transaction, which is much faster. 
E.g. it cuts the runtime of the referrers test from 50s to 23s.
 2010-02-24 12:48:00 +00:00
Eelco Dolstra fae0427324 * ADDITIONAL_NETWORK_LIBS -> LIBS. 2010-02-24 12:25:48 +00:00
Eelco Dolstra fa6a4fcb11 * Add ${sqlite_lib} everywhere. Just adding it in `libstore' doesn't 
work on x86_64 when sqlite is compiled statically.
 2010-02-24 12:18:48 +00:00
Eelco Dolstra 462bd50aef * Use normal (rather than full) synchronous mode, which I gather from 
the description at http://www.sqlite.org/atomiccommit.html should be
  safe enough.
 2010-02-24 10:57:57 +00:00
Eelco Dolstra 63b09c5e41 2010-02-23 22:31:38 +00:00
Eelco Dolstra b4e6d98fc3 * configure: flag --with-sqlite. 2010-02-23 22:12:46 +00:00
Eelco Dolstra 2b20318b0e 2010-02-22 14:24:37 +00:00
Eelco Dolstra 9cda616949 * The database needs a trigger to get rid of self-references to 
prevent a foreign key constraint violation on the Refs table when
  deleting a path.
 2010-02-22 14:18:55 +00:00
Eelco Dolstra c4d388add4 * Get derivation outputs from the database instead of the .drv file, 
which requires more I/O.
 2010-02-22 12:44:36 +00:00
Eelco Dolstra 103cfee056 * Revert r19650 (implement gc-keep-outputs by looking for derivations 
with the same name as the output) and instead use the
  DerivationOutputs table in the database, which is the correct way to
  to do things.
 2010-02-22 11:44:17 +00:00
Eelco Dolstra 299ff64812 * Put the derivation outputs in the database. This is useful for the 
garbage collector.
 2010-02-22 11:15:50 +00:00
Eelco Dolstra 1930570ad9 * Foreign key support in SQLite is not a persistent setting, so enable 
it at startup.
* Implement negative caching.  Now `make check' passes.
 2010-02-19 17:15:22 +00:00
Eelco Dolstra 9c9a88e9e2 * Implement more stuff. 2010-02-19 16:43:25 +00:00
Eelco Dolstra 762cee72cc * Implement registerValidPath(). 2010-02-19 16:04:51 +00:00
Eelco Dolstra 268f9aaf28 * Implemented queryValidPaths() and verifyStore(). 2010-02-18 16:51:27 +00:00
Eelco Dolstra 836e5b6f57 * Implemented queryReferrers(). 2010-02-18 16:21:59 +00:00
Eelco Dolstra 77cb9e3fb1 * Implement queryPathInfo(). 2010-02-18 15:52:57 +00:00
Eelco Dolstra 885e22b16e * Implement isValidPath(). 2010-02-18 15:11:08 +00:00
Eelco Dolstra cfb09e0fad * Automatically abort transactions if they go out of scope without 
committing.
 2010-02-18 14:40:07 +00:00
Eelco Dolstra e0305bb7a8 * Some wrapper objects to ensure that SQLite objects are properly 
destroyed.
 2010-02-18 14:30:42 +00:00
Eelco Dolstra a053d2d8e5 * Add the deriver to the ValidPaths table. In principle we could now 
store all the derivers of a path efficiently.  But that opens a big
  can of worms with respect to garbage collector semantics.
 2010-02-18 13:48:18 +00:00
Eelco Dolstra dbddac0fe9 * Assign an integer id to every row in the ValidPaths table in order 
to make the Refs table more space-efficient.  For instance, this
  reduces the size of the database on my laptop from 93 MiB to 18
  MiB.  (It was 72 MiB with the old schema on an ext3 disk with a 1
  KiB block size.)
 2010-02-18 13:40:46 +00:00
Eelco Dolstra c1a07f9445 * Convert the Nix database to SQLite. 2010-02-18 13:16:59 +00:00
Eelco Dolstra 4e17be7981 * Revert r19797, and use a simpler solution: just don't monitor build 
hooks for silence.  It's unnecessary because the remote nix-store
  command is already monitoring the real build.
 2010-02-03 21:38:41 +00:00
Eelco Dolstra f859a8d3c3 * While waiting for a lock, print a sign of life every 5 minutes. 
This prevents remote builders from being killed by the
  `max-silent-time' inactivity monitor while they are waiting for a
  long garbage collection to finish.  This happens fairly often in the
  Hydra build farm.
 2010-02-03 21:22:57 +00:00
Eelco Dolstra 4bbbe25802 * Remove most Cygwin-specific code. Cygwin 1.7 implements advisory 
POSIX locks, and simulates Unix-style file deletion semantics
  sufficiently.  Note that this means that Nix won't work on Cygwin
  1.5 anymore.
 2010-02-02 15:28:36 +00:00
Eelco Dolstra 07ffdc2862 * Added an option "fsync-metadata" to fsync() changes to 
/nix/var/nix/db.
* Removed the function writeStringToFile since it does (almost) the
  same thing as writeFile.
 2010-01-29 12:22:58 +00:00
Eelco Dolstra ad529fb89f * Don't consider a store path valid if its info file exists but is 
zero bytes long.  That makes Nix more robust in case of crashes
  (especially on ext4).
 2010-01-29 11:53:58 +00:00
Eelco Dolstra fdcaf37361 * Made `nix-store -qR --include-outputs' much faster if there are 
multiple paths specified on the command line (from O(n * m) to O(n +
  m), where n is the number of arguments and m is the size of the
  closure).
 2010-01-25 17:18:44 +00:00
Eelco Dolstra 5388944e8d * Make the garbage collector do the right thing when `gc-keep-outputs' 
is enabled by not depending on the deriver.
 2010-01-25 16:04:32 +00:00
Eelco Dolstra ef92a14bfe * Include config.h before the C library headers, because it defines 
_FILE_OFFSET_BITS=64.  Without it, functions like stat() fail on
  large file sizes.  This happened with a Nix store on squashfs:

  $ nix-store --dump /tmp/mnt/46wzqnk4cbdwh1dclhrpqnnz1icak6n7-local-net-cmds > /dev/null
  error: getting attributes of path `/tmp/mnt/46wzqnk4cbdwh1dclhrpqnnz1icak6n7-local-net-cmds': Value too large for defined data type

  $ stat /tmp/mnt/46wzqnk4cbdwh1dclhrpqnnz1icak6n7-local-net-cmds
  File: `/tmp/mnt/46wzqnk4cbdwh1dclhrpqnnz1icak6n7-local-net-cmds'
  Size: 0               Blocks: 36028797018963968 IO Block: 1024   regular empty file

  (This is a bug in squashfs or mksquashfs, but it shouldn't cause Nix
  to fail.)
 2009-12-17 14:12:44 +00:00
Eelco Dolstra 945d8218fb * Build correctly against newer ATerm releases. Fixes "error: 'union' 
tag used in naming 'struct _ATerm'".
 2009-12-16 15:29:50 +00:00
Eelco Dolstra bcd6cdf0d8 * Give a better error message when trying to build something and 
readOnlyMode is set.
 2009-12-09 17:45:22 +00:00
Eelco Dolstra f9e766db98 * Randomise the order in which we delete entries to make the collector 
less biased towards deleting paths that come alphabetically first
  (e.g. /nix/store/000...).  This matters when using --max-freed etc.
 2009-11-24 09:53:18 +00:00
Eelco Dolstra ca50c83fbb 2009-11-23 21:21:29 +00:00
Eelco Dolstra ae6bf87273 * `nix-store --gc --print-roots': also print the path of the actual 
root symlink, not just its target.  E.g.:

  /nix/var/nix/profiles/system-99-link -> /nix/store/76kwf88657nq7wgk1hx3l1z5q91zb9wd-system
 2009-11-23 17:23:12 +00:00
Eelco Dolstra c364d5d1e3 * Made the garbage collector a lot faster. It no longer computes the 
complete set of live and dead paths before starting the actual
  deletion, but determines liveness on demand.  I.e. for any path in
  the store, it first tries to delete all the referrers, and then the
  path itself.  This means that the collector can start deleting paths
  almost immediately.
 2009-11-23 16:34:24 +00:00
Eelco Dolstra 8824d60fe5 * Remove the --use-atime / --max-atime garbage collector flags. Many 
(Linux) machines no longer maintain the atime because it's too
  expensive, and on the machines where --use-atime is useful (like the
  buildfarm), reading the atimes on the entire Nix store takes way too
  much time to make it practical.
 2009-11-20 17:12:38 +00:00
Eelco Dolstra 327a232c85 * Remove support for old (before Nix 0.12pre12020) databases. 2009-11-06 01:15:44 +00:00
Eelco Dolstra 8520542071 * When building in a chroot, make a copy of a file if hard-linking 
fails.  This is likely to happen after a `nix-store --optimise',
  because some files may have 32000 links (NIX-111).
 2009-10-22 08:28:33 +00:00
Eelco Dolstra 6b9f6b0222 * Remove a prototype for a function that no longer exists. 2009-10-22 08:12:38 +00:00
Sander van der Burg 53a4981fa2 Added optional parameter which adds -lnsl -lsocket to make the Nix package manager work on OpenSolaris 2009-10-08 14:50:37 +00:00
Eelco Dolstra c7057fc1f2 * And some more. 2009-09-24 07:39:55 +00:00
Eelco Dolstra 193f59e077 * Fix a build failure on Fedora 11. rename() needs <stdio.h>. 2009-09-24 07:21:29 +00:00
Eelco Dolstra 676e07902e * Darwin hack. 2009-09-23 18:04:55 +00:00
Eelco Dolstra 64e89980e8 * Create some state directories automatically as a convenience. 2009-09-23 17:05:51 +00:00
Rob Vermaas 48b58617e9 * include wait.h for WEXITSTATUS 2009-09-23 12:57:15 +00:00
Eelco Dolstra 14bc3ce3d6 * Canonicalise timestamps in the Nix store to 1 (1970-01-01 00:00:01 
UTC) rather than 0 (00:00:00).  1 is a better choice because some
  programs use 0 as a special value.  For instance, the Template
  Toolkit uses a timestamp of 0 to denote the non-existence of a file,
  so it barfs on files in the Nix store (see
  template-toolkit-nix-store.patch in Nixpkgs).  Similarly, Maya 2008
  fails to load script directories with a timestamp of 0 and can't be
  patched because it's closed source.

  This will also shut up those "implausibly old time stamp" GNU tar
  warnings.
 2009-06-13 16:30:58 +00:00
Eelco Dolstra d5eab2fc82 * Use foreach in a lot of places. 2009-04-21 11:52:16 +00:00
Eelco Dolstra 8f1bf28505 * nix-store --verify: don't bail out if a referenced path is missing. 
(It can't fix it though.)
 2009-04-21 11:06:27 +00:00
Eelco Dolstra 4e646b0ddb * Fix a few "comparison is always false/true due to limited range of 
data type" warnings on 64-bit platforms.  The one in parser.y is
  likely to be a real bug.
 2009-04-16 12:03:17 +00:00
Eelco Dolstra dfb863f333 * Don't cache transient build hook problems. 2009-04-15 06:25:02 +00:00
Eelco Dolstra 351bf658f9 * Do a substitution even if --max-jobs == 0. 2009-03-31 21:14:07 +00:00
Eelco Dolstra cff2b2a13a * Doh! 2009-03-31 15:50:03 +00:00
Eelco Dolstra 160a60d663 2009-03-30 19:35:55 +00:00
Eelco Dolstra 805144b705 * Make the poll interval configurable. 2009-03-30 11:34:03 +00:00
Eelco Dolstra 7377195297 * With `--max-jobs 0', print a nicer error message than "Assertion 
`!awake.empty()' failed."
 2009-03-29 18:06:00 +00:00
Eelco Dolstra 737423a89c * Use polling to wait for a remote build slot when using a build hook 
(that is, call the build hook with a certain interval until it
  accepts the build).
* build-remote.pl was totally broken: for all system types other than
  the local system type, it would send all builds to the *first*
  machine of the appropriate type.
 2009-03-29 18:00:45 +00:00
Eelco Dolstra 6e946c8e72 * Scan for references and compute the SHA-256 hash of the output in 
one pass.  This halves the amount of I/O.
 2009-03-28 20:51:33 +00:00
Eelco Dolstra c7152c8f97 * Don't use the non-standard __gnu_cxx::stdio_filebuf class. 2009-03-28 19:41:53 +00:00
Eelco Dolstra 3a2bbe7f8a * Simplify communication with the hook a bit (don't use file 
descriptors 3/4, just use stdin/stderr).
 2009-03-28 19:29:55 +00:00
Eelco Dolstra f54e800366 * The `fixedOutput' variable didn't get initialised when using a build 
hook, causing negative caching to fail randomly.
 2009-03-28 16:12:27 +00:00
Eelco Dolstra cb85bc396e * Create /nix/store if it doesn't exist. 2009-03-27 14:19:04 +00:00
Eelco Dolstra 5dd8fb2069 * Don't use ULLONG_MAX in maxFreed - use 0 to mean "no limit". 
18446744073709551615ULL breaks on GCC 3.3.6 (`integer constant is
  too large for "long" type').
 2009-03-26 11:02:07 +00:00
Eelco Dolstra 92f525ecf4 * Negative caching, i.e. caching of build failures. Disabled by 
default.  This is mostly useful for Hydra.
 2009-03-25 21:05:42 +00:00
Eelco Dolstra a046858a22 * Doh. 2009-03-24 23:40:24 +00:00
Eelco Dolstra 12c8c64efa 2009-03-24 14:07:37 +00:00
Eelco Dolstra cacff1be88 * No longer block while waiting for a lock on a store path. Instead 
poll for it (i.e. if we can't acquire the lock, then let the main
  select() loop wait for at most a few seconds and then try again).
  This improves parallelism: if two nix-store processes are both
  trying to build a path at the same time, the second one shouldn't
  block; it should first see if it can build other goals.  Also, it
  prevents the deadlocks that have been occuring in Hydra lately,
  where a process waits for a lock held by another process that's
  waiting for a lock held by the first.

  The downside is that polling isn't really elegant, but POSIX doesn't
  provide a way to wait for locks in a select() loop.  The only
  solution would be to spawn a thread for each lock to do a blocking
  fcntl() and then signal the main thread, but that would require
  pthreads.
 2009-03-23 01:05:54 +00:00
Eelco Dolstra 58969fa2bf * Refactoring. 2009-03-22 23:53:05 +00:00
Eelco Dolstra 77d272623f * NAR archives: handle files larger than 2^32 bytes. Previously it 
would just silently store only (fileSize % 2^32) bytes.
* Use posix_fallocate if available when unpacking archives.
* Provide a better error message when trying to unpack something that
  isn't a NAR archive.
 2009-03-22 17:36:43 +00:00
Eelco Dolstra 2897286487 * Unify exportReferencesGraph and exportBuildReferencesGraph, and make 
sure that it works as expected when you pass it a derivation.  That
  is, we have to make sure that all build-time dependencies are built,
  and that they are all in the input closure (otherwise remote builds
  might fail, for example).  This is ensured at instantiation time by
  adding all derivations and their sources to inputDrvs and inputSrcs.
 2009-03-18 17:36:42 +00:00