Commit graph

4629 commits

Author SHA1 Message Date
Eelco Dolstra 117670be57 Make the garbage collector more concurrent
Make the garbage collector more concurrent by deleting valid paths
outside the region where we're holding the global GC lock.  This
should greatly reduce the time during which new builds are blocked,
since the deletion accounts for the vast majority of the time spent in
the GC.

To ensure that this is safe, the valid paths are invalidated and
renamed to some arbitrary path while we're holding the lock.  This
ensures that we when we finally delete the path, it's not a (newly)
valid or locked path.
2012-03-26 20:43:33 +02:00
Eelco Dolstra 8be1979f1a Remove the --max-links GC option
We don't need this anymore now that current filesystems support more
than 32,000 files in a directory.
2012-03-26 20:00:02 +02:00
Lluís Batlle i Rossell 85799bf89e Fixing the default of sync-before-registering
Setting 'false' as default, as suggested by Eelco.

I also added a comment about the setting in the code.
2012-03-22 20:05:54 +01:00
Lluís Batlle i Rossell 25de80e2b6 Adding a nix option to sync before registering a path, for non-ext*
filesystems.
2012-03-22 19:57:42 +01:00
Eelco Dolstra 19d9762ad5 nix-store --clear-failed-paths: Clear derivation outputs
If the argument to ‘nix-store --clear-failed-paths’ is a derivation,
then clear the failed state of its outputs.
2012-03-20 18:23:26 +01:00
Eelco Dolstra cb79e91ec3 Fix the coverage build 2012-03-19 17:05:42 +01:00
Eelco Dolstra 457dd55962 Fix the Deb builds 2012-03-19 15:11:22 +01:00
Eelco Dolstra 4b83483142 Fix the RPM 2012-03-19 14:30:59 +01:00
Eelco Dolstra 7f38087f35 Add a command "nix-build --run-env" to reproduce the environment of a derivation
This command builds or fetches all dependencies of the given
derivation, then starts a shell with the environment variables from
the derivation.  This shell also sources $stdenv/setup to initialise
the environment further.

The current directory is not changed.  Thus this is a convenient way
to reproduce a build environment in an existing working tree.

Existing environment variables are left untouched (unless the
derivation overrides them).  As a special hack, the original value of
$PATH is appended to the $PATH produced by $stdenv/setup.

Example session:

$ nix-build --run-env '<nixpkgs>' -A xterm
(the dependencies of xterm are built/fetched...)
$ tar xf $src
$ ./configure
$ make
$ emacs
(... hack source ...)
$ make
$ ./xterm
2012-03-19 04:14:21 +01:00
Eelco Dolstra ea402a255f Replace "make check" with "make installcheck"
Ensuring that the tests work from the build tree requires a growing
number of nasty hacks.  The tests also don't verify that the installed
Nix actually works.  Thus, the tests now require "make install" to
have been run.
2012-03-19 01:20:02 +01:00
Eelco Dolstra 4b23900ff1 Fix the optional dependency on OpenSSL
http://hydra.nixos.org/build/2306540
2012-03-19 01:06:13 +01:00
Eelco Dolstra b006a31d52 Drop the externals directory
Nix now requires SQLite and bzip2 to be pre-installed.  SQLite is
detected using pkg-config.  We required DBD::SQLite anyway, so
depending on SQLite is not a big problem.

The --with-bzip2, --with-openssl and --with-sqlite flags are gone.
2012-03-18 23:54:57 +01:00
Eelco Dolstra 2e0322efd1 Clean up when building from a working tree 2012-03-18 22:10:59 +01:00
Eelco Dolstra bd50c01972 Ensure that Perl processes delete their entry in the temproots directory
By moving the destructor object to libstore.so, it's also run when
download-using-manifests and nix-prefetch-url exit.  This prevents
them from cluttering /nix/var/nix/temproots with stale files.
2012-03-13 17:07:49 +01:00
Eelco Dolstra b461721f17 Fix tests 2012-03-12 21:41:45 +01:00
Eelco Dolstra 005d1e4ccb Update bzip2 dependency 2012-03-06 20:03:20 +01:00
Eelco Dolstra 605b16cd7b Fix compilation on FreeBSD
http://hydra.nixos.org/build/2213576

Not sure why compilation doesn't fail on other platforms...
2012-03-05 22:04:40 +01:00
Eelco Dolstra d645633643 nix-copy-closure: don't print copied path on stdout
We're already printing progress on stderr, so printing them on stdout
afterwards is kind of useless.
2012-03-05 20:52:24 +01:00
Eelco Dolstra 35355fc1fc Set the close-on-exec flag on file descriptors 2012-03-05 20:29:00 +01:00
Eelco Dolstra 7b22bec252 Don't leak a file descriptor in commonChildInit() 2012-03-05 20:28:09 +01:00
Eelco Dolstra 1d487dc6a6 nix-worker: put the pid of the caller in argv[1]
This is useful for debugging.
2012-03-05 19:19:29 +01:00
Eelco Dolstra 2b4964f319 Restore progress indication during nix-copy-closure 2012-03-05 18:13:44 +01:00
Eelco Dolstra 56042a120a build-remote.pl: don't wait forever for the upload lock
In the build hook, don't wait forever to get the upload lock.  This
ensures progress if another process gets stuck while holding the
upload lock.
2012-03-05 17:58:09 +01:00
Eelco Dolstra 8afd28a922 Remove dependency on sqlite3_table_column_metadata
Not all SQLite builds have the function sqlite3_table_column_metadata.
We were only using it in a schema upgrade check for compatibility with
databases that were probably never seen in the wild.  So remove it.
2012-03-01 16:06:49 +01:00
Eelco Dolstra 1c94524458 Fix an uninitialised variable
The variable ‘useChroot’ was not initialised properly.  This caused
random failures if using the build hook.  Seen on Mac OS X 10.7 with Clang.
Thanks to KolibriFX for finding this :-)
2012-03-01 16:06:49 +01:00
Eelco Dolstra 6fe13e6aba Fix chroots builds
Chroots are initialised by hard-linking inputs from the Nix store to
the chroot.  This doesn't work if the input has its immutable bit set,
because it's forbidden to create hard links to immutable files.  So
temporarily clear the immutable bit when creating and destroying the
chroot.

Note that making regular files in the Nix store immutable isn't very
reliable, since the bit can easily become cleared: for instance, if we
run the garbage collector after running ‘nix-store --optimise’.  So
maybe we should only make directories immutable.
2012-02-18 01:23:52 +01:00
Eelco Dolstra bd013b6f98 On Linux, make the Nix store really read-only by using the immutable bit
I was bitten one time too many by Python modifying the Nix store by
creating *.pyc files when run as root.  On Linux, we can prevent this
by setting the immutable bit on files and directories (as in ‘chattr
+i’).  This isn't supported by all filesystems, so it's not an error
if setting the bit fails.  The immutable bit is cleared by the garbage
collector before deleting a path.  The only tricky aspect is in
optimiseStore(), since it's forbidden to create hard links to an
immutable file.  Thus optimiseStore() temporarily clears the immutable
bit before creating the link.
2012-02-15 01:31:56 +01:00
Eelco Dolstra 5e57047d87 Fix a broken guard around utime()
Because of an outdated check for a timestamp of 0, we were calling
utime() even when it wasn't necessary.
2012-02-15 01:00:30 +01:00
Eelco Dolstra 58ac7a17a4 Don't use deletePath() to delete a single file 2012-02-15 00:28:01 +01:00
Eelco Dolstra e9fc91df45 Fix error message
This fixes the error message

  error: file `' was not found in the Nix search path (add it using $NIX_PATH or -I)
2012-02-09 18:56:48 +01:00
Eelco Dolstra d5a5a83ad4 Use data() instead of c_str() where appropriate 2012-02-09 18:27:45 +01:00
Eelco Dolstra ec2827f5fc Update .gitignore 2012-02-08 19:25:43 +01:00
Eelco Dolstra 4055a3b19c Remove empty ChangeLog file 2012-02-08 13:22:08 +01:00
Eelco Dolstra fb95455c02 Set the versionSuffix to include the Git shortrev 2012-02-08 12:31:14 +01:00
Eelco Dolstra 67444f5e54 * Build Nix for the latest Fedora and Ubuntu releases (and drop some
old ones).
2012-02-07 13:18:20 +00:00
Eelco Dolstra f491ae97d4 * Inline some functions and get rid of the indirection through
EvalState::eval().  This gives a 12% speedup on ‘nix-instantiate
  /etc/nixos/nixos/ -A system --readonly-mode’ (from 1.01s to 0.89s).
2012-02-04 13:50:25 +00:00
Eelco Dolstra 2bda12ef3b * Print elapsed evaluation time. 2012-02-04 13:27:11 +00:00
Eelco Dolstra ce86a9a0ae * nix-channel: fail if we don't have write permission to the manifests
directory.  Previously in this situation we did add the Nix
  expressions from the channel to allow installation from source, but
  this doesn't work for binary-only channels and leads to confusing
  error messages.
2012-02-02 12:25:56 +00:00
Eelco Dolstra baa0501cc1 * Fix importing a derivation. This gave a segfault. 2012-01-26 13:13:00 +00:00
Eelco Dolstra 4c9fdd2cd6 * Add a test for importing derivations. 2012-01-26 13:04:50 +00:00
Eelco Dolstra 330df4b4db * Allow comparisons between derivations by comparing the outPath
attributes.
2012-01-19 23:08:47 +00:00
Eelco Dolstra 18047d4625 * Add some debug output to print the derivation name once it's known.
This makes it easier to pinpoint the source of a crash.
2012-01-19 23:07:42 +00:00
Eelco Dolstra 126c7317bc * Add a test case for comparing derivations. 2012-01-19 22:10:24 +00:00
Eelco Dolstra ddd0ce534a * Don't distribute Store.cc, it's generated automatically by xsubpp. 2012-01-18 16:43:48 +00:00
Eelco Dolstra 02f1363e19 * Pass the appropriate flags to GCC when building the Perl bindings.
Without these, Nix fails on 32-bit Linux with Perl 5.14, with a
  rather unhelpful error message:

    Not a CODE reference at /nix/store/n6kpbacn6nn7i3i735v8j3di8aqyl07v-perl-5.14.2/lib/perl5/5.14.2/i686-linux-thread-multi/DynaLoader.pm

  This is likely because the lack of -D_FILE_OFFSET_BITS=64 causes
  various Perl structures to not match what the Perl interpreter
  expects.
2012-01-18 16:39:54 +00:00
Eelco Dolstra 4e624849b6 * Added a command ‘nix-store --print-env $drvpath’ that prints out the
environment of the given derivation in a format that can be sourced
  by the shell, e.g.

  $ eval "$(nix-store --print-env $(nix-instantiate /etc/nixos/nixpkgs -A pkg))"
  $ NIX_BUILD_TOP=/tmp
  $ source $stdenv/setup

  This is especially useful to reproduce the environment used to build
  a package outside of its builder for development purposes.

  TODO: add a nix-build option to do the above and fetch the
  dependencies of the derivation as well.
2012-01-17 23:07:22 +00:00
Eelco Dolstra 2a3f4110c5 * nix-build: put the temporary derivation symlink in a temporary
directory rather than the current directory.
* nix-build: --drv-link now implies --add-drv-link.
2012-01-13 23:35:07 +00:00
Eelco Dolstra 7ae763e16b * ‘-module’ causes the generated library to have an .so extension on
Mac OS X instead of .dylib, so don't do that.
2012-01-12 14:16:25 +00:00
Ludovic Courtès 6245f0d529 tests: Run `download-using-manifests' via libtool so that dlopening works. 2012-01-11 16:58:49 +00:00
Ludovic Courtès c5d9ccd810 perl: Build libNixStore.la with `-module'. 2012-01-11 16:44:42 +00:00