Commit graph

2800 commits

Author SHA1 Message Date
Shea Levy 8bf378e999 Update darwin build for optional sandbox paths
Fixes #1132
2016-11-17 08:06:32 -05:00
Eelco Dolstra 4be4f6de56
S3BinaryCacheStore:: Eliminate a string copy while uploading
This cuts hydra-queue-runner's peak memory usage by about a third.
2016-11-16 16:21:30 +01:00
Eelco Dolstra 10ae8fabf1 buildPaths(): Handle ecIncompleteClosure
buildPaths() on a non-derivation would incorrectly not throw an error
if the path didn't have a substitute.
2016-11-14 15:00:17 +01:00
Eelco Dolstra b77fb8acb5 Don't rely on %m 2016-11-14 13:37:16 +01:00
Ludovic Courtès ccb1022022 daemon: Do not error out when deduplication fails due to ENOSPC.
This solves a problem whereby if /gnu/store/.links had enough entries,
ext4's directory index would be full, leading to link(2) returning
ENOSPC.

* nix/libstore/optimise-store.cc (LocalStore::optimisePath_): Upon
ENOSPC from link(2), print a message and return instead of throwing a
'SysError'.
2016-11-14 13:35:03 +01:00
Guillaume Maudoux a5e761dddb Fix comments parsing
Fixed the parsing of multiline strings ending with an even number of
stars, like /** this **/.
Added test cases for comments.
2016-11-13 17:20:34 +01:00
Eelco Dolstra dd77f7d593 Store::computeFSClosure(): Support a set of paths
This way, callers can exploits the parallelism of computeFSClosure()
when they have multiple paths that they need the (combined) closure of.
2016-11-10 17:45:04 +01:00
Eelco Dolstra e4655b166c copyClosure() / copyStorePath(): Expose dontCheckSigs
Needed by Hydra.
2016-11-09 19:08:20 +01:00
Eelco Dolstra 4b8f1b0ec0 Merge branch 'ssh-store' of https://github.com/shlevy/nix 2016-11-09 18:57:22 +01:00
Eelco Dolstra 21c55ab3b5 Implement backwards-compatible RemoteStore::addToStore()
The SSHStore PR adds this functionality to the daemon, but we have to
handle the case where the Nix daemon is 1.11.

Also, don't require signatures for trusted users. This restores 1.11
behaviour.

Fixes https://github.com/NixOS/hydra/issues/398.
2016-11-09 18:45:06 +01:00
Eelco Dolstra a83b10f84c Typo 2016-11-08 20:19:02 +01:00
Eelco Dolstra b3ba762dbf Add missing #include 2016-11-07 14:35:47 +01:00
Eelco Dolstra 18b7363a69 Support optional sandbox paths
For example, you can now set

  build-sandbox-paths = /dev/nvidiactl?

to specify that /dev/nvidiactl should only be mounted in the sandbox
if it exists in the host filesystem. This is useful e.g. for EC2
images that should support both CUDA and non-CUDA instances.
2016-10-31 17:09:52 +01:00
Shea Levy c4969aebaf Add nix.conf options for -k and -K
Fixes #1084
2016-10-27 12:49:34 -04:00
Tuomas Tynkkynen f0fc3dd88b Fix SIGFPE from integer overflow during division
On some architectures (like x86_64 or i686, but not ARM for example)
overflow during integer division causes a crash due to SIGFPE.
Reproduces on a 64-bit system with:

    nix-instantiate --eval -E '(-9223372036854775807 - 1) / -1'

The only way this can happen is when the smallest possible integer is
divided by -1, so just special-case that.
2016-10-26 18:41:59 +03:00
Eelco Dolstra 5c0bd51d49 Fix build log output in nix-store --serve 2016-10-26 17:13:38 +02:00
Eelco Dolstra d2c58ba605 Restore the CachedFailure status code
The removal of CachedFailure caused the value of TimedOut to change,
which broke timed-out handling in Hydra (so timed-out builds would
show up as "aborted" and would be retried, e.g. at
http://hydra.nixos.org/build/42537427).
2016-10-26 15:02:38 +02:00
Eelco Dolstra fdbbcc4492 Remove addPathToAccessor 2016-10-21 18:09:30 +02:00
Eelco Dolstra 542ae5c8f8 BinaryCacheStore: Optionally write a NAR listing
The store parameter "write-nar-listing=1" will cause BinaryCacheStore
to write a file ‘<store-hash>.ls.xz’ for each ‘<store-hash>.narinfo’
added to the binary cache. This file contains an XZ-compressed JSON
file describing the contents of the NAR, excluding the contents of
regular files.

E.g.

  {
    "version": 1,
    "root": {
      "type": "directory",
      "entries": {
        "lib": {
          "type": "directory",
          "entries": {
            "Mcrt1.o": {
              "type": "regular",
              "size": 1288
            },
            "Scrt1.o": {
              "type": "regular",
              "size": 3920
            },
          }
        }
      }
      ...
    }
  }

(The actual file has no indentation.)

This is intended to speed up the NixOS channels programs index
generator [1], since fetching gazillions of large NARs from
cache.nixos.org is currently a bottleneck for updating the regular
(non-small) channel.

[1] https://github.com/NixOS/nixos-channel-scripts/blob/master/generate-programs-index.cc
2016-10-21 16:50:28 +02:00
Eelco Dolstra 307cc8c33d Merge branch 'priorityqueue' of https://github.com/groxxda/nix 2016-10-19 16:37:04 +02:00
Eelco Dolstra 795d9b8668 Merge branch 'regex' of https://github.com/groxxda/nix 2016-10-19 16:32:24 +02:00
Alexander Ried c935e8eeaf fixup! replace own regex class with std::regex 2016-10-19 16:03:53 +02:00
Eelco Dolstra 19c278de89 Fix Darwin build
Done slightly differently from https://github.com/NixOS/nix/pull/1093.
2016-10-19 15:21:18 +02:00
Eelco Dolstra 29727ff944 Shut up clang warning 2016-10-19 15:02:48 +02:00
Eelco Dolstra e5346c54df Fix uninitialised variable 2016-10-19 15:02:38 +02:00
Alexander Ried b05b98df75 replace own regex class with std::regex 2016-10-18 20:22:25 +02:00
Alexander Ried efb938468c downloader: use priority_queue 2016-10-18 15:45:51 +02:00
Eelco Dolstra ae8884b949 Fix /dev/ptmx in sandboxes
This was broken since ff0c0b645c. Since
I can't figure out how to mount a devpts instance in the sandbox,
let's just bind-mount the host devpts.
2016-10-14 15:42:13 +02:00
Eelco Dolstra 2f9d60ed7f CURL_AT_LEAST_VERSION -> LIBCURL_VERSION_NUM
http://hydra.nixos.org/build/42025230
2016-10-14 14:44:15 +02:00
Eelco Dolstra 5e61b422c5 Fix HTTP/2 support
Commit 86e8c67efc broke it, because
CURL_* are not actually #defines.
2016-10-13 17:25:42 +02:00
Eelco Dolstra fb2dd32100 SSL_CERT_FILE -> NIX_SSL_CERT_FILE
This prevents collisions with the "native" OpenSSL, in particular on
OS X.

Fixes #921.
2016-10-13 17:09:10 +02:00
Eelco Dolstra 816d3e5724 Shut up some warnings 2016-10-12 15:53:38 +02:00
Eelco Dolstra 82e2a070e0 Add some functions needed by hydra 2016-10-12 15:49:37 +02:00
Eelco Dolstra edf9eb8181 querySubstitutablePaths(): Don't query paths for which we already have a substituter 2016-10-07 19:57:47 +02:00
Eelco Dolstra 0c85ef7090 LocalStore::querySubstitutablePaths(): Implement using queryValidPaths() 2016-10-07 19:46:56 +02:00
Eelco Dolstra 844219f364 Store::queryValidPaths(): Use async queryPathInfo()
This allows the binary cache substituter to pipeline requests.
2016-10-07 19:43:36 +02:00
Eelco Dolstra c663b84573 Implement generic Store::queryValidPaths() 2016-10-07 19:20:47 +02:00
Eelco Dolstra 35db4f65a0 Add copyClosure utility function for Hydra 2016-10-07 19:15:25 +02:00
Eelco Dolstra 629ab80022 importPaths(): Fix accessor support for Hydra 2016-10-07 18:13:40 +02:00
Eelco Dolstra b0f7f9c98f toJSON(): Support some more types 2016-10-06 17:00:52 +02:00
Eelco Dolstra ae522f930d Fix getS3Stats() 2016-10-06 17:00:52 +02:00
Eelco Dolstra cd128f4bad Merge pull request #998 from veprbl/rx_chmod_fix
override rx directory permissions in deletePath()
2016-10-06 11:08:21 +02:00
Eelco Dolstra c6a929986a Use std::random_device 2016-10-05 21:29:18 +02:00
Eelco Dolstra 86e8c67efc Fix build with older versions of libcurl 2016-10-04 14:43:23 +02:00
Eelco Dolstra 4bd51d74af Handle the case where signed-binary-caches consists of whitespace 2016-09-22 15:59:25 +02:00
Eelco Dolstra 818ab58cc6 Add sandbox-dev-shm-size option
Fixes #1069.
2016-09-21 16:54:53 +02:00
Eelco Dolstra 4546be1b3e nix-build, nix-shell: Don't print error message if nix-store/nix-instantiate fails 2016-09-21 16:54:53 +02:00
Eelco Dolstra c55bf085eb printMsg(lvlError, ...) -> printError(...) etc. 2016-09-21 16:54:53 +02:00
Eelco Dolstra 4036185cb4 Some notational convenience for formatting strings
We can now write

  throw Error("file '%s' not found", path);

instead of

  throw Error(format("file '%s' not found") % path);

and similarly

  printError("file '%s' not found", path);

instead of

  printMsg(lvlError, format("file '%s' not found") % path);
2016-09-21 16:54:53 +02:00
Eelco Dolstra beaefdf706 Tweak 2016-09-20 17:49:31 +02:00