Commit graph

893 commits

Author SHA1 Message Date
Eelco Dolstra d4a71ec3bf Update spec file
http://hydra.nixos.org/build/14344391
2014-09-18 15:42:01 +02:00
Eelco Dolstra 8be9990cdb Install some pkgconfig files 2014-09-18 12:00:40 +02:00
Eelco Dolstra d98bfcbf81 On Linux, disable address space randomization 2014-09-17 17:21:13 +02:00
Eelco Dolstra d37d012774 Settings: Add bool get() 2014-09-17 15:18:13 +02:00
Ludovic Courtès 8fb8c26b6d Add an 'optimiseStore' remote procedure call. 2014-09-01 23:53:26 +02:00
Eelco Dolstra b72e93bca8 Add disallowedReferences / disallowedRequisites
For the "stdenv accidentally referring to bootstrap-tools", it seems
easier to specify the path that we don't want to depend on, e.g.

  disallowedRequisites = [ bootstrapTools ];
2014-08-28 18:57:13 +02:00
Gergely Risko fd61069a42 Introduce allowedRequisites feature 2014-08-28 18:23:55 +02:00
Joel Taylor b224ac1520 fix disappearing bash arguments 2014-08-21 23:17:50 +02:00
Eelco Dolstra 524f89f139 Use unshare() instead of clone()
It turns out that using clone() to start a child process is unsafe in
a multithreaded program. It can cause the initialisation of a build
child process to hang in setgroups(), as seen several times in the
build farm:

The reason is that Glibc thinks that the other threads of the parent
exist in the child, so in setxid_mark_thread() it tries to get a futex
that has been acquired by another thread just before the clone(). With
fork(), Glibc runs pthread_atfork() handlers that take care of this
(in particular, __reclaim_stacks()). But clone() doesn't do that.

Fortunately, we can use fork()+unshare() instead of clone() to set up
private namespaces.

See also https://www.mail-archive.com/lxc-devel@lists.linuxcontainers.org/msg03434.html.
2014-08-21 14:08:09 +02:00
Eelco Dolstra 11849a320e Use proper quotes everywhere 2014-08-20 18:03:48 +02:00
Eelco Dolstra 373fad75e1 Add some color 2014-08-20 16:50:17 +02:00
Eelco Dolstra 954188af27 Filter Nix-specific ANSI escape sequences from stderr
The Nixpkgs stdenv prints some custom escape sequences to denote
nesting and stuff like that. Most terminals (e.g. xterm, konsole)
ignore them, but some do not (e.g. xfce4-terminal). So for the benefit
of the latter, filter them out.
2014-08-20 14:30:01 +02:00
Eelco Dolstra 029424d17d Make hook shutdown more reliable 2014-08-19 17:47:30 +02:00
Eelco Dolstra 632f989b51 Doh 2014-08-18 11:35:50 +02:00
Eelco Dolstra c160ead82f Reduce verbosity 2014-08-17 19:11:50 +02:00
Eelco Dolstra 42e9ad8fd1 Propagate remote timeouts properly 2014-08-17 19:09:03 +02:00
Eelco Dolstra 9367046fef Use regular file GC roots if possible
This makes hydra-eval-jobs create roots as regular files. See
1c208f2b7e.
2014-08-13 17:44:41 +02:00
Eelco Dolstra 98873ff7de Remove unnecessary call to addTempRoot() 2014-08-05 16:41:42 +02:00
Eelco Dolstra f02c52e3d4 Doh 2014-08-05 10:19:57 +02:00
Eelco Dolstra 988bf59421 Move some options out of globals 2014-08-04 18:13:14 +02:00
Eelco Dolstra daccd68999 Refactor 2014-08-04 18:02:29 +02:00
Eelco Dolstra d5a076c36f Add option ‘build-extra-chroot-dirs’
This is useful for extending (rather than overriding) the default set
of chroot paths.
2014-08-04 18:00:00 +02:00
Eelco Dolstra 4d73e2e893 Get rid of "killing <pid>" message for unused build hooks 2014-08-04 17:27:45 +02:00
Eelco Dolstra bb45092f72 Make chroot builds easier to set up
By default, we now include /bin/sh as a bind-mount of bash.
2014-08-04 17:09:26 +02:00
Eelco Dolstra 51485dcda2 Remove ugly hack for detecting build environment setup errors 2014-08-01 19:38:21 +02:00
Eelco Dolstra 4c4b219c07 Call commonChildInit() before doing chroot init
This ensures that daemon clients see error messages from the chroot
setup.
2014-08-01 19:29:03 +02:00
Eelco Dolstra e0d7d0e45c findRoots(): Prevent a call to lstat()
This means that getting the roots from /nix/var/nix/.../hydra-roots
doesn't need any I/O other than reading the directory.
2014-08-01 17:20:25 +02:00
Eelco Dolstra daf3f2c11f Make readDirectory() return inode / file type 2014-08-01 17:14:28 +02:00
Eelco Dolstra 1c208f2b7e Allow regular files as GC roots
If a root is a regular file, then its name must denote a store
path. For instance, the existence of the file

  /nix/var/nix/gcroots/per-user/eelco/hydra-roots/wzc3cy1wwwd6d0dgxpa77ijr1yp50s6v-libxml2-2.7.7

would cause

  /nix/store/wzc3cy1wwwd6d0dgxpa77ijr1yp50s6v-libxml2-2.7.7

to be a root.

This is useful because it involves less I/O (no need for a readlink()
call) and takes up less disk space (the symlink target typically takes
up a full disk block, while directory entries are packed more
efficiently). This is particularly important for hydra.nixos.org,
which has hundreds of thousands of roots, and where reading the roots
can take 25 minutes.
2014-08-01 17:14:28 +02:00
Eelco Dolstra 50dc1f5b71 Restore default SIGPIPE handler before invoking ‘man’
Fixes NixOS/nixpkgs#3410.
2014-07-31 10:31:17 +02:00
Eelco Dolstra 67937907ca nix-daemon: Pass on the user's $SSH_AUTH_SOCK to the SSH substituter 2014-07-25 18:02:48 +02:00
Eelco Dolstra 71a20d4d95 Change the default for use-ssh-substituter to ‘true’
Now you only have to pass ‘--option ssh-substituter-hosts
nix-ssh@bla’ to enable SSH substitution.
2014-07-25 12:57:12 +02:00
Eelco Dolstra d3c61d83be Remove some dead code 2014-07-24 00:00:53 +02:00
Eelco Dolstra 5989966ed3 Remove dead code 2014-07-23 14:46:28 +02:00
Eelco Dolstra ee3c5d7916 Revert old useBuildHook behaviour 2014-07-19 02:25:47 +02:00
Eelco Dolstra 049c0eb49c nix-daemon: Add trusted-users and allowed-users options
‘trusted-users’ is a list of users and groups that have elevated
rights, such as the ability to specify binary caches. It defaults to
‘root’. A typical value would be ‘@wheel’ to specify all users in the
wheel group.

‘allowed-users’ is a list of users and groups that are allowed to
connect to the daemon. It defaults to ‘*’. A typical value would be
‘@users’ to specify the ‘users’ group.
2014-07-17 16:57:07 +02:00
Eelco Dolstra 276a40b31f Handle case collisions on case-insensitive systems
When running NixOps under Mac OS X, we need to be able to import store
paths built on Linux into the local Nix store. However, HFS+ is
usually case-insensitive, so if there are directories with file names
that differ only in case, then importing will fail.

The solution is to add a suffix ("~nix~case~hack~<integer>") to
colliding files. For instance, if we have a directory containing
xt_CONNMARK.h and xt_connmark.h, then the latter will be renamed to
"xt_connmark.h~nix~case~hack~1". If a store path is dumped as a NAR,
the suffixes are removed. Thus, importing and exporting via a
case-insensitive Nix store is round-tripping. So when NixOps calls
nix-copy-closure to copy the path to a Linux machine, you get the
original file names back.

Closes #119.
2014-07-16 16:02:05 +02:00
Eelco Dolstra fa13d3f4f3 build-remote.pl: Fix building multiple output derivations
We were importing paths without sorting them topologically, leading to
"path is not valid" errors.

See e.g. http://hydra.nixos.org/build/12451761
2014-07-14 12:19:27 +02:00
Eelco Dolstra a5c6347ff0 build-remote.pl: Use ‘nix-store --serve’ on the remote side
This makes things more efficient (we don't need to use an SSH master
connection, and we only start a single remote process) and gets rid of
locking issues (the remote nix-store process will keep inputs and
outputs locked as long as they're needed).

It also makes it more or less secure to connect directly to the root
account on the build machine, using a forced command
(e.g. ‘command="nix-store --serve --write"’). This bypasses the Nix
daemon and is therefore more efficient.

Also, don't call nix-store to import the output paths.
2014-07-11 16:22:24 +02:00
Eelco Dolstra e196eecbe6 Allow $NIX_BUILD_HOOK to be relative to Nix libexec directory 2014-07-11 13:55:06 +02:00
Eelco Dolstra edbfe2232e Replace message "importing path <...>" with "exporting path <...>"
This causes nix-copy-closure to show what it's doing before rather
than after.
2014-07-10 21:30:22 +02:00
Eelco Dolstra 43b64f5038 Remove tabs 2014-07-10 17:32:21 +02:00
Eelco Dolstra 8e9140cfde Refactoring: Move all fork handling into a higher-order function
C++11 lambdas ftw.
2014-07-10 16:58:09 +02:00
Eelco Dolstra 7911e4c27a Remove maybeVfork 2014-07-10 13:35:44 +02:00
Eelco Dolstra 8504e7d604 allow-arbitrary-code-during-evaluation -> allow-unsafe-native-code-during-evaluation 2014-06-27 11:20:16 +02:00
Shea Levy d62f46e500 Only add the importNative primop if the allow-arbitrary-code-during-evaluation option is true (default false) 2014-06-24 10:50:03 -04:00
Eelco Dolstra b1beed97a0 Report daemon OOM better
When copying a large path causes the daemon to run out of memory, you
now get:

  error: Nix daemon out of memory

instead of:

  error: writing to file: Broken pipe
2014-06-10 13:45:50 +02:00
Eelco Dolstra 829af22759 Print a warning when loading a large path into memory
I.e. if you have a derivation with

  src = ./huge-directory;

you'll get a warning that this is not a good idea.
2014-06-10 13:30:09 +02:00
Eelco Dolstra 54a34119f3 Use std::unordered_set 2014-05-26 17:53:17 +02:00
Eelco Dolstra 9f9080e2c0 nix-store -l: Fetch build logs from the Internet
If a build log is not available locally, then ‘nix-store -l’ will now
try to download it from the servers listed in the ‘log-servers’ option
in nix.conf. For instance, if you have:

  log-servers = http://hydra.nixos.org/log

then it will try to get logs from http://hydra.nixos.org/log/<base
name of the store path>. So you can do things like:

  $ nix-store -l $(which xterm)

and get a log even if xterm wasn't built locally.
2014-05-21 17:19:36 +02:00