Commit graph

2611 commits

Author SHA1 Message Date
Eelco Dolstra fa7cd5369b StoreAPI -> Store
Calling a class an API is a bit redundant...
2016-02-04 14:48:42 +01:00
Eelco Dolstra c10c61449f Eliminate the "store" global variable
Also, move a few free-standing functions into StoreAPI and Derivation.

Also, introduce a non-nullable smart pointer, ref<T>, which is just a
wrapper around std::shared_ptr ensuring that the pointer is never
null. (For reference-counted values, this is better than passing a
"T&", because the latter doesn't maintain the refcount. Usually, the
caller will have a shared_ptr keeping the value alive, but that's not
always the case, e.g., when passing a reference to a std::thread via
std::bind.)
2016-02-04 14:28:26 +01:00
Eelco Dolstra d367b8e787 Add build mode to compute fixed-output derivation hashes
For example,

  $ nix-build --hash -A nix-repl.src

will build the fixed-output derivation nix-repl.src (a fetchFromGitHub
call), but instead of *verifying* the hash given in the Nix
expression, it prints out the resulting hash, and then moves the
result to its content-addressed location in the Nix store. E.g

  build produced path ‘/nix/store/504a4k6zi69dq0yjc0bm12pa65bccxam-nix-repl-8a2f5f0607540ffe56b56d52db544373e1efb980-src’ with sha256 hash ‘0cjablz01i0g9smnavhf86imwx1f9mnh5flax75i615ml71gsr88’

The goal of this is to make all nix-prefetch-* scripts unnecessary: we
can just let Nix run the real thing (i.e., the corresponding fetch*
derivation).

Another example:

  $ nix-build --hash -E 'with import <nixpkgs> {}; fetchgit { url = "https://github.com/NixOS/nix.git"; sha256 = "ffffffffffffffffffffffffffffffffffffffffffffffffffff"; }'
  ...
  git revision is 9e7c1a4bbd
  ...
  build produced path ‘/nix/store/gmsnh9i7x4mb7pyd2ns7n3c9l90jfsi1-nix’ with sha256 hash ‘1188xb621diw89n25rifqg9lxnzpz7nj5bfh4i1y3dnis0dmc0zp’

(Having to specify a fake sha256 hash is a bit annoying...)
2016-01-31 12:18:09 +01:00
Eelco Dolstra 9e7c1a4bbd Use the daemon when we don't have write access to the Nix database 2016-01-31 10:28:48 +01:00
Eelco Dolstra 4fa08f3edb Untabify 2016-01-28 16:03:32 +01:00
Eelco Dolstra 26566cd28e Untabify 2016-01-28 16:01:01 +01:00
Eelco Dolstra 9f14d7d33a printMissing(): Propagate store argument 2016-01-28 15:16:06 +01:00
Eelco Dolstra 7899fc959d Define HashType's size to allow it to be forward-declared 2016-01-27 17:46:01 +01:00
Eelco Dolstra 739bab0be7 Trivia 2016-01-27 17:18:31 +01:00
Eelco Dolstra d45ad8fcf5 Make hashLength32() a method of Hash 2016-01-27 17:18:20 +01:00
Eelco Dolstra 5b8c09c124 string2Int: Barf on negative numbers for unsigned types 2016-01-20 16:45:05 +01:00
Eelco Dolstra 5d8b7eb3e1 Revert "Revert "next try for "don't abort when given unmatched '}' with 'start-condition stack underflow'. This fixes #751"""
This reverts commit b669d3d2e8.
2016-01-20 16:34:42 +01:00
Eelco Dolstra b669d3d2e8 Revert "next try for "don't abort when given unmatched '}' with 'start-condition stack underflow'. This fixes #751""
This reverts commit ed23c8568e. Let's
merge this *after* the 1.11.1 release.
2016-01-20 00:05:28 +01:00
Fabian Schmitthenner ed23c8568e next try for "don't abort when given unmatched '}' with 'start-condition stack underflow'. This fixes #751"
This reverts commit 8120b6fb8a and fixes the regression introduced in
8d22b26448.
2016-01-19 20:35:35 +00:00
Eelco Dolstra 8120b6fb8a Revert "don't abort when given unmatched '}' with 'start-condition stack underflow'. This fixes #751"
This reverts commit 8d22b26448. It
breaks Nixpkgs:

$ nix-env -qa
error: syntax error, unexpected IND_STR, expecting '}', at /home/eelco/Dev/nixpkgs-stable/pkgs/top-level/python-packages.nix:7605:8
2016-01-19 20:33:32 +01:00
Shea Levy a6d52fcd95 Merge branch 'lexer' of git://github.com/fkz/nix 2016-01-17 10:08:03 -05:00
Fabian Schmitthenner 8d22b26448 don't abort when given unmatched '}' with 'start-condition stack underflow'. This fixes #751 2016-01-12 20:40:41 +00:00
Eelco Dolstra 786046cf13 --option build-repeat: Keep the differing output if -K is given
Similar to 00903fa799. Regardless of -K,
we now also print which output differs.
2016-01-12 18:26:24 +01:00
Eelco Dolstra 8906eda2f9 Canonicalize gids to 0
Previously files in the Nix store were owned by root or by nixbld,
depending on whether they were created by a substituter or by a
builder. This doesn't matter much, but causes spurious diffoscope
differences. So use root everywhere.
2016-01-12 17:27:40 +01:00
Eelco Dolstra 00903fa799 --check: Keep the differing output if -K is given
This makes it easier to investigate the non-determinism, e.g.

  $ nix-build pkgs/stdenv/linux -A stage1.pkgs.zlib --check -K
  error: derivation ‘/nix/store/l54i8wlw22656i4pk05c52ngv9rpl39q-zlib-1.2.8.drv’ may not be deterministic: output ‘/nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8’ differs from ‘/nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8-check’

  $ diffoscope /nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8 /nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8-check
  ...
  ├── lib/libz.a
  │   ├── metadata
  │   │ @@ -1,15 +1,15 @@
  │   │ -rw-r--r-- 30001/30000   3096 Jan 12 15:20 2016 adler32.o
  ...
  │   │ +rw-r--r-- 30001/30000   3096 Jan 12 15:28 2016 adler32.o
  ...
2016-01-12 16:44:26 +01:00
Eelco Dolstra 0cad1f8049 --check: Fix "failed to produce output path"
This occured when sandbox building is disabled, at least one output
exists, and at least one other output does not.
2016-01-12 15:13:47 +01:00
Eelco Dolstra 1c57ab8b31 --check: Fix assertion failure when some outputs are missing
E.g.

  $ nix-build pkgs/stdenv/linux/ -A stage1.pkgs.perl --check
  nix-store: src/libstore/build.cc:1323: void nix::DerivationGoal::tryToBuild(): Assertion `buildMode != bmCheck || validPaths.size() == drv->outputs.size()' failed.

when perl.out exists but perl.man doesn't. The fix is to only check
the outputs that exist. Note that "nix-build -A stage1.pkgs.all
--check" will still give a (proper) error in this case.
2016-01-12 14:54:39 +01:00
Eelco Dolstra 458711e4ee Fix "Bad address" executing build hook
This was observed in the deb_debian7x86_64 build:
http://hydra.nixos.org/build/29973215

Calling c_str() on a temporary should be fine because the temporary
shouldn't be destroyed until after the execl() call, but who knows...
2016-01-07 15:10:14 +01:00
Eelco Dolstra 9aac1861f7 Fix some signedness warnings 2016-01-07 14:37:39 +01:00
Eelco Dolstra 02a66b3fd7 nix-store --repair-path: Rebuild if there is no substituter 2016-01-06 22:07:59 +01:00
Eelco Dolstra caaaff3954 Fix --repair failure on multiple-output derivations
If repair found a corrupted/missing path that depended on a
multiple-output derivation, and some of the outputs of the latter were
not present, it failed with a message like

  error: path ‘/nix/store/cnfn9d5fjys1y93cz9shld2xwaibd7nn-bash-4.3-p42-doc’ is not valid
2016-01-06 21:49:32 +01:00
Christian Theune 934642155c @eelco's feedback: downgrade to regular float for size, remove unused function. 2016-01-06 08:25:58 +01:00
Fabian Schmitthenner 39d1da7b51 Better error message
Also show types when nix cannot compare values of different types.
This is also more consistent since types are already shown when comparing values of the same not comparable type.
2016-01-05 14:13:54 +01:00
Eelco Dolstra 9a4706eb19 Fix compilation 2016-01-05 14:05:11 +01:00
Shea Levy 14080f3e4b Use __toString when coercing sets to strings.
For example, "${{ foo = "bar"; __toString = x: x.foo; }}" evaluates
to "bar".

With this, we can delay calling functions like mkDerivation,
buildPythonPackage, etc. until we actually need a derivation, enabling
overrides and other modifications to happen by simple attribute set
update.
2016-01-05 13:55:06 +01:00
Eelco Dolstra 6298afc047 Merge pull request #685 from vizanto/master
POSIX compliant directory access (fixes build on Solaris)
2016-01-05 13:49:55 +01:00
FrankHB 607d91517e Fixed unresolved 'BUFSIZ'
`BUFSIZ` is defined in header `<cstdio>`.
2016-01-05 13:33:40 +01:00
Eelco Dolstra 743e310046 Fix non-Darwin build 2016-01-05 13:31:15 +01:00
Vladimír Čunát 29c0199d5c manual: document builtins.functionArgs
The text is just a conversion of comment from src/libexpr/primops.cc
2016-01-05 13:29:17 +01:00
Tuomas Tynkkynen f770b9e6c5 libstore: mmap() returns MAP_FAILED, not NULL on failure 2016-01-05 13:26:35 +01:00
Christian Theune a12a43046b Edge condition: parser did not pick up floats starting exactly with 0. 2016-01-05 09:54:49 +01:00
Christian Theune f872262e08 Fix up float parsing. 2016-01-05 09:46:37 +01:00
Christian Theune 494fc5acbb Try a simplified version of float lexing that didn't work.
The last one I tried was botchered anyway ...
2016-01-05 00:53:22 +01:00
Christian Theune 14ebde5289 First hit at providing support for floats in the language. 2016-01-05 00:40:40 +01:00
Eelco Dolstra 71a93a5f0e Don't allow sandbox profile except in relaxed mode
This makes Darwin consistent with Linux: Nix expressions can't break
out of the sandbox unless relaxed sandbox mode is enabled.

For the normal sandbox mode this will require fixing #759 however.
2016-01-04 20:01:13 +01:00
Eelco Dolstra 77ad443bd1 ~PathLocks(): Handle exceptions
Otherwise, since the call to write a "d" character to the lock file
can fail with ENOSPC, we can get an unhandled exception resulting in a
call to terminate().
2016-01-04 11:34:36 +01:00
Eelco Dolstra b8258a4475 Fix regression in passAsFile
Caused by 8063fc497a. If tmpDir !=
tmpDirInSandbox (typically when there are multiple concurrent builds
with the same name), the *Path attribute would not point to an
existing file. This caused Nixpkgs' writeTextFile to write an empty
file. In particular this showed up as hanging VM builds (because it
would run an empty run-nixos-vm script and then wait for it to finish
booting).
2015-12-29 15:28:20 +01:00
Eelco Dolstra 52120123a5 Handle /tmp being a symlink
Hopefully fixes Darwin sandbox regression introduced in
8063fc497a.
2015-12-22 17:16:17 +01:00
Eelco Dolstra 68fd01f42a Don't ignore sodium_init() return value 2015-12-22 17:14:04 +01:00
Eelco Dolstra f696af0fab Fix bad error message in Darwin chroots 2015-12-22 17:05:29 +01:00
Eelco Dolstra a9faa7bbce showId: Handle empty attribute names
We should probably disallow these, but until then, we shouldn't barf
with an assertion failure.

Fixes #738.
2015-12-17 15:15:28 +01:00
Eelco Dolstra 8f67325a7c Build sandbox support etc. unconditionally on Linux
Also, use "#if __APPLE__" instead of "#if SANDBOX_ENABLED" to prevent
ambiguity.
2015-12-10 11:47:17 +01:00
Bjørn Forsman 65bd82d42a Clarify error message for hash mismatches (again)
This is arguably nitpicky, but I think this new formulation is even
clearer. My thinking is that it's easier to comprehend when the
calculated hash value is displayed close to the output path. (I think it
is somewhat similar to eliminating double negatives in logic
statements.)

The formulation is inspired / copied from the OpenEmbedded build tool,
bitbake.
2015-12-08 19:50:25 +01:00
Ludovic Courtès d1e3bf01bc daemon: Add 'buildMode' parameter to 'buildPaths' RPC 2015-12-02 18:14:49 +01:00
Eelco Dolstra 8063fc497a Use deterministic $TMPDIR in sandbox
Rather than using $<host-TMPDIR>/nix-build-<drvname>-<number>, the
temporary directory is now always /tmp/nix-build-<drvname>-0. This
improves bitwise-exact reproducibility for builds that store $TMPDIR
in their build output. (Of course, those should still be fixed...)
2015-12-02 15:04:00 +01:00
Danny Wilson 69b9d8fdbd Use DT_UNKNOWN when dirent d_type extension is not supported.
edolstra:
“…since callers of readDirectory have to handle the possibility of
 DT_UNKNOWN anyway, and we don't want to do a stat call for every
 directory entry unless it's really needed.”
2015-11-25 21:57:19 +01:00
Shea Levy f327970129 Merge branch 'auto-call-functor'
autoCallFunction now auto-calls functors
2015-11-25 11:57:31 -05:00
Shea Levy 9533532ce2 autoCallFunction: Auto-call functors 2015-11-25 11:56:14 -05:00
Eelco Dolstra 0ab4d905e7 Merge branch 'p/sandbox-rename-minimal' of https://github.com/vcunat/nix 2015-11-25 14:53:42 +01:00
Eelco Dolstra a0f0733413 Fix build failure introduced by #704
Also, make the FreeBSD checks conditional on FreeBSD.
2015-11-25 14:41:19 +01:00
Eelco Dolstra afef347157 Merge pull request #712 from pSub/print-meta-license
Print license information on '--xml --meta'
2015-11-25 14:00:52 +01:00
Eelco Dolstra cad40adce5 Merge pull request #704 from ysangkok/freebsd-support
FreeBSD support with knowledge about Linux emulation
2015-11-24 19:24:21 +01:00
Jude Taylor 279fa8f618 reintroduce host deps in tandem with sandbox profiles 2015-11-21 15:57:06 -08:00
Shea Levy e0bd114e09 Revert "remove sandbox-defaults.sb"
As discussed in NixOS/nixpkgs#11001, we still need some of the old
sandbox mechanism.

This reverts commit d760c2638c.
2015-11-21 16:40:24 -05:00
Pascal Wittmann 4921223160 Print license information on '--xml --meta'
The nixpkgs manual prescribes the use of values from stdenv.lib.licenses
for the meta.license attribute. Those values are attribute sets and
currently skipped when running nix-env with '--xml --meta'. This has the
consequence that also nixpkgs-lint will report missing licenses.

With this commit nix-env with '--xml --meta' will print all attributes
of an attribute set that are of type tString. For example the output for
the package nixpkgs.hello is

    <meta name="license" type="strings">
      <string type="url" value="http://spdx.org/licenses/GPL-3.0+" />
      <string type="shortName" value="gpl3Plus" />
      <string type="fullName" value="GNU General Public License v3.0 or later" />
      <string type="spdxId" value="GPL-3.0+" />
    </meta>

This commit fixes nixpkgs-lint, too.
2015-11-21 11:43:44 +01:00
Jude Taylor b9b7bb1806 re-fix permissions for GHC 2015-11-19 16:06:21 -08:00
Shea Levy 5deb7fbdfb Merge branch 'sandbox-profiles' of git://github.com/pikajude/nix
Temporarily allow derivations to describe their full sandbox profile.
This will be eventually scaled back to a more secure setup, see the
discussion at #695
2015-11-19 17:44:11 -05:00
Peter Simons 6ad10591ce src/libstore/build.cc: clarify error message for hash mismatches
Nix reports a hash mismatch saying:

  output path ‘foo’ should have sha256 hash ‘abc’, instead has ‘xyz’

That message is slightly ambiguous and some people read that statement
to mean the exact opposite of what it is supposed to mean. After this
patch, the message will be:

  Nix expects output path ‘foo’ to have sha256 hash ‘abc’, instead it has ‘xyz’
2015-11-19 12:42:37 +01:00
janus 8a74a125bc FreeBSD can build Linux 32-bit binaries 2015-11-17 14:16:08 +00:00
Shea Levy 1d3529e93a Default arguments belong at declaration, not definition 2015-11-16 05:55:55 -05:00
Shea Levy 9b4cd20752 Fix copy-paste error 2015-11-16 05:54:34 -05:00
Shea Levy 58d2fac91d AutoDelete: Add default constructor with deletion disabled 2015-11-16 05:53:10 -05:00
Shea Levy 4390142315 Use AutoDelete for sandbox profile file 2015-11-15 06:08:50 -05:00
Jude Taylor bd09a4c967 simplify build.cc using modern C++ features 2015-11-14 14:11:03 -08:00
Jude Taylor 4876bb012e simplify build permissions 2015-11-14 14:11:03 -08:00
Jude Taylor d760c2638c remove sandbox-defaults.sb 2015-11-14 14:11:03 -08:00
Jude Taylor 8a7f0dfd68 use per-derivation sandbox profiles 2015-11-14 14:10:43 -08:00
Vladimír Čunát b39622a487 rename chroot to sandbox (fixes #656, close #682)
- rename options but leav old names as lower-priority aliases,
  also "-dirs" -> "-paths" to get closer to the meaning
- update docs to reflect the new names (old aliases are not documented),
  including a new file with release notes
- tests need an update after corresponding changes to nixpkgs
- __noChroot is left as it is (after discussion on the PR)
2015-11-10 22:32:51 +01:00
Eelco Dolstra 8fdd156a65 Add option to verify build determinism
Passing "--option build-repeat <N>" will cause every build to be
repeated N times. If the build output differs between any round, the
build is rejected, and the output paths are not registered as
valid. This is primarily useful to verify build determinism. (We
already had a --check option to repeat a previously succeeded
build. However, with --check, non-deterministic builds are registered
in the DB. Preventing that is useful for Hydra to ensure that
non-deterministic builds don't end up getting published at all.)
2015-11-09 23:16:24 +01:00
Eelco Dolstra 96c2ebf004 Revert "Allow using /bin and /usr/bin as impure prefixes on non-darwin by default"
This reverts commit 79ca503332. Ouch,
never noticed this. We definitely don't want to allow builds to have
arbitrary access to /bin and /usr/bin, because then they can (for
instance) bring in a bunch of setuid programs. Also, we shouldn't be
encouraging the use of impurities in the default configuration.
2015-11-09 21:28:38 +01:00
Eelco Dolstra 4384bbd2e1 optimizePath(): Detect some .links corruption
If automatic store optimisation is enabled, and a hard-linked file in
the store gets corrupted, then the corresponding .links entry will
also be corrupted. In that case, trying to repair with --repair or
--repair-path won't work, because the new "good" file will be replaced
by a hard link to the corrupted file. We can catch most of these cases
by doing a sanity-check on the file sizes.
2015-11-09 21:28:25 +01:00
Eelco Dolstra 7759a56bed Fix namespace issue 2015-11-09 21:28:25 +01:00
Danny Wilson cdb346c65e Fix build on Solaris
d_type is not part of the POSIX spec unfortunately.
2015-11-07 04:51:33 +01:00
Shea Levy 5bc12454fd Merge branch 'master' of https://github.com/pikajude/nix
> I made this change for two reasons:

> 1. Darwin's locale data doesn't appear to be open source
> 2. Privileged processes will always use /usr/share/locale regardless of environment variables
2015-11-04 17:27:19 -05:00
Eelco Dolstra 6f1743b1a5 Support SHA-512 hashes
Fixes #679.

Note: on x86_64, SHA-512 is considerably faster than SHA-256 (198 MB/s
versus 131 MB/s).
2015-11-04 16:37:49 +01:00
Eelco Dolstra a6ca68a70c Require OpenSSL 2015-11-04 16:37:49 +01:00
Jude Taylor c10ef2d134 fix syntax error 2015-11-03 09:03:32 -08:00
Jude Taylor 3b52e54e31 darwin: allow reading system locale and zoneinfo 2015-11-03 08:43:09 -08:00
Jude Taylor d6bd51ffbe allow reading ICU data 2015-10-31 02:52:42 -07:00
Jude Taylor 95c66cc950 add special devices to sandbox-defaults 2015-10-30 20:32:11 -07:00
Eelco Dolstra 1f735a3440 <nix/fetchurl.nix>: Support xz-compressed NARs 2015-10-30 12:34:30 +01:00
Eelco Dolstra dae5dc7ade <nix/fetchurl.nix>: Support downloading and unpacking NARs
This removes the need to have multiple downloads in the stdenv
bootstrap process (like a separate busybox binary for Linux, or
curl/mkdir/sh/bzip2 for Darwin). Now all those files can be combined
into a single NAR.
2015-10-30 11:27:47 +01:00
Eelco Dolstra 5c28943e8f int2String() -> std::to_string() 2015-10-29 13:26:55 +01:00
Eelco Dolstra 45b2f32cc1 Merge pull request #668 from svanderburg/master
Fix compilation error due to missing ENOENT on cygwin
2015-10-26 11:12:01 +01:00
Jude Taylor 9155fbd386 use nixDataDir instead of appending /share to PREFIX 2015-10-21 14:44:44 -07:00
Jude Taylor af1218a86a revert libutil change 2015-10-21 14:40:35 -07:00
Jude Taylor 992cda1b11 clarifying comment 2015-10-21 14:39:16 -07:00
Jude Taylor 76f3ba42fd move preBuildHook defaulting to globals.cc 2015-10-21 14:39:08 -07:00
Jude Taylor 2dc8e19a17 restore old DEFAULT_ALLOWED_IMPURE_PREFIXES 2015-10-21 14:38:49 -07:00
Jude Taylor ff6953cb03 Add resolve-system-dependencies.pl 2015-10-21 12:38:52 -07:00
Jude Taylor 15c7a79ed3 remove usr paths from allowed inputs 2015-10-21 12:38:52 -07:00
Jude Taylor 5bf4085000 allow access to SystemVersion for python builders 2015-10-21 12:38:52 -07:00
Jude Taylor 6a8cee19b3 fix line reading in preBuildHook 2015-10-21 12:38:52 -07:00
Jude Taylor 1fc0fe1baf remove sandbox defaults into a new file 2015-10-21 12:38:52 -07:00
Jude Taylor d4cac051f7 restore allowed impure prefixes 2015-10-21 12:38:52 -07:00
Jude Taylor 4b8c71b4ef remove an unneeded default impure-dep 2015-10-21 12:38:52 -07:00
Jude Taylor e770f941d6 make sandbox builds more permissive 2015-10-21 12:38:52 -07:00
Jude Taylor 1ea2aa9e99 add a few more permissions 2015-10-21 12:38:52 -07:00
Eelco Dolstra f5b32ff47d Allow builtin fetchurl regardless of the derivation's system attribute 2015-10-21 15:14:42 +02:00
Eelco Dolstra 01615b5f63 Show progress indicator for builtin fetchurl 2015-10-21 15:14:42 +02:00
Eelco Dolstra 5db358d4d7 Disable TLS verification for builtin fetchurl
This makes it consistent with the Nixpkgs fetchurl and makes it work
in chroots. We don't need verification because the hash of the result
is checked anyway.
2015-10-21 15:14:42 +02:00
Eelco Dolstra 357d31b339 Fix segfault in builtin fetchurl
The stack allocated for the builder was way too small (32 KB). This is
sufficient for normal derivations, because they just do some setup and
then exec() the actual builder. But for the fetchurl builtin
derivation it's not enough. Also, allocating the stack on the caller's
stack was fishy business.
2015-10-21 15:14:42 +02:00
Sander van der Burg 058d1950c0 Fix compilation error due to missing ENOENT on cygwin 2015-10-18 19:04:24 +00:00
Tuomas Tynkkynen 2075ec83e3 Allow building ARMv6 stuff on ARMv7
This allows building a Raspberry Pi image on modern, faster boards.
2015-10-08 13:32:28 +02:00
Eelco Dolstra 51cf4455b2 isFunctor: Simplify 2015-10-08 13:22:11 +02:00
Mathnerd314 8a87521636 forceFunction: allow functors as well 2015-10-08 13:19:44 +02:00
Eelco Dolstra 14f7e1fc42 Revert to CURLOPT_PROGRESSFUNCTION
CURLOPT_XFERINFOFUNCTION isn't widely supported yet.

http://hydra.nixos.org/build/26679495
2015-10-08 13:15:21 +02:00
Eelco Dolstra 21ecd106ed Show progress during downloads 2015-10-07 17:31:50 +02:00
Eelco Dolstra f08449ccbd nix-prefetch-url: Add --name option
This allows overriding the name component of the resulting Nix store
path, which is necessary if the base name of the URI contains
"illegal" characters.
2015-10-07 14:55:33 +02:00
Eelco Dolstra e4abf8610b nix-prefetch-url -A: Use "name" attribute from Nix expression
This is in particular useful for fetchFromGitHub et al., ensuring that
the store path produced by nix-prefetch-url corresponds to what those
functions expect.
2015-10-07 14:47:39 +02:00
Eelco Dolstra b54f447df9 nix-prefetch-url: Support unpacking tarballs
This allows nix-prefetch-url to prefetch the output of fetchzip and
its wrappers (like fetchFromGitHub). For example:

  $ nix-prefetch-url --unpack https://github.com/NixOS/patchelf/archive/0.8.tar.gz

or from a Nix expression:

  $ nix-prefetch-url -A nix-repl.src

In the latter case, --unpack can be omitted because nix-repl.src is a
fetchFromGitHub derivation and thus has "outputHashMode" set to
"recursive".
2015-10-07 14:40:10 +02:00
Manuel Jacob 7888b2ba15 Define SOL_LOCAL if not defined already (e.g. on FreeBSD).
Some evidence that defining it to be 0 is right:
* OS X headers define it to be 0.
* Other code uses 0 instead of SOL_LOCAL to check for peer credentials
  (e.g. FreeBSD's implementation of getpeereid).
2015-10-06 22:28:30 +02:00
Manuel Jacob 12c1776df9 Don't try to link libdl on FreeBSD.
There is no libdl on FreeBSD.  Instead the symbols are included in libc.
2015-10-06 22:28:30 +02:00
Manuel Jacob cbc177ca01 Use pkg-config-provided LDFLAGS for libsqlite3 and libcurl.
Previously, pkg-config was already queried for libsqlite3's and
libcurl's link flags.  However they were not used, but hardcoded
instead.  This commit replaces the hardcoded LDFLAGS by the ones
provided by pkg-config in a similar pattern as already used for
libsodium.
2015-10-06 22:28:30 +02:00
Eelco Dolstra 1abda8e173 nix-store --serve: Implement log size limit 2015-10-06 17:33:30 +02:00
Eelco Dolstra 06f29fafe8 nix-prefetch-url: Support prefetching from a Nix expression
For example,

  $ nix-prefetch-url -A hello.src

will prefetch the file specified by the fetchurl call in the attribute
‘hello.src’ from the Nix expression in the current directory. This
differs from ‘nix-build -A hello.src’ in that it doesn't verify the
hash.

You can also specify a path to the Nix expression:

  $ nix-prefetch-url ~/Dev/nixpkgs -A hello.src

List elements (typically used in ‘patches’ attributes) also work:

  $ nix-prefetch-url -A portmidi.patches.0
2015-10-01 18:07:56 +02:00
Eelco Dolstra a3c4eb0964 nix-prefetch-url: $PRINT_PATH -> --print-path 2015-10-01 16:53:07 +02:00
Eelco Dolstra bec3c31608 nix-prefetch-url: Rewrite in C++ 2015-10-01 16:47:43 +02:00
Eelco Dolstra 4ba6bc184c Shut up clang warnings 2015-09-18 01:22:35 +02:00
Vladimír Čunát 42808fa281 nix-env --upgrade: show "downgrading" when doing so
It was strange to show "upgrading" when the version was getting lower.
This is left on "upgrading" when the versions are the same,
as I can't see any better wording.
2015-09-17 12:09:54 +02:00
Vladimír Čunát 64f9b511be nix-env --upgrade: avoid unexpected downgrades
Until now, if one explicitly installed a low-priority version,
nix-env --upgrade would downgrade it by default and even with --leq.
Let's never accept an upgrade with version not matching the upgradeType.
Additionally, let's never decrease the priority of an installed package;
you can use --install to force that.

Also refactor to use variable bestVersion instead of bestName,
as only version was used from it.
2015-09-17 12:06:26 +02:00
Eelco Dolstra 71a5161365 Implement buildDerivation() via the daemon 2015-09-03 12:56:59 +02:00
Eelco Dolstra 73332fd342 Filter build-chroot-dirs entries that conflict with derivation outputs
Fixes https://github.com/NixOS/nixpkgs/issues/9504.

Note that this means we may have a non-functional /bin/sh in the
chroot while rebuilding Bash or one of its dependencies. Ideally those
packages don't rely on /bin/sh though.
2015-09-02 14:54:12 +02:00
Eelco Dolstra d711fe8f5e Remove unused variable 2015-09-02 14:46:04 +02:00
Eelco Dolstra e12cf82782 Prevent .chroot from being GC'ed when using LocalStore::buildDerivation()
Fixes #616.
2015-08-24 11:13:31 +02:00
Eelco Dolstra eadb86f447 nix-collect-garbage: Revive --max-freed
Fixes #609.
2015-08-21 13:57:53 +02:00
Manolis Ragkousis 26221e44eb Remove unneeded HAVE_UNSHARE.
* src/libstore/build.cc (CHROOT_ENABLED): Remove HAVE_UNSHARE.
2015-08-19 14:11:15 +03:00
Kirill Elagin 3b0f60e5c2 baseNameOf: Enhance basename compatibility
* If the path ends with a slash, drop it.
* If the remaining path doesn’t contain slashes, just return it.

Fixes #574.
2015-08-07 03:35:02 +02:00
Eelco Dolstra 34dfbd9394 Don't include <iostream> before config.h
This breaks the build on 32-bit systems.

http://hydra.nixos.org/build/24373658
2015-08-04 11:12:31 +02:00
Eelco Dolstra 30d19a2bdc Handle debug messages from runChild()
Turns out that "nix-build -vvv" with chroots enabled has been broken
for some time, because some debug message got interpreted as an error.
2015-08-03 18:04:32 +02:00
Iwan Aucamp 3db950aab7 Removed unnecessary included 2015-08-03 11:48:34 +02:00
Eelco Dolstra 2bac04c5ff Fix stack consumption 2015-07-31 20:28:25 +02:00
Iwan Aucamp 75837651f1 Output line number on infinite recursion 2015-07-31 20:26:44 +02:00
Eelco Dolstra 76cc8e97a2 Add sort primop 2015-07-28 18:39:39 +02:00
Eelco Dolstra 50807f3dd5 Add primop genList
This can be used to implement functions like ‘imap’ (or for that
matter, ‘map’) without the quadratic complexity incurred by calling
‘++’ repeatedly.
2015-07-28 17:28:35 +02:00
Eelco Dolstra f3dda728a4 Remove unnecessary parentheses 2015-07-26 12:07:14 +02:00
Eelco Dolstra d6d5885c15 Add replaceStrings primop
This is a generalisation of replaceChars in Nixpkgs.
2015-07-24 15:32:24 +02:00
Eelco Dolstra 2e8fd4c5cd Add concatStringsSep as a primop
This fixes the quadratic behaviour of concatStrings/concatStringsSep
in Nixpkgs.
2015-07-24 02:38:09 +02:00
Eelco Dolstra cb4320c1a0 Cleanup 2015-07-23 23:14:07 +02:00
Eelco Dolstra c8bb2371eb Optimize empty sets
This reduces the number of Bindings allocations by about 10%.
2015-07-23 23:11:08 +02:00
Eelco Dolstra 16c9935fa9 Fix fetchurl of executable file
Pointed out by @cstrahan, thanks!
2015-07-23 22:25:04 +02:00
Eelco Dolstra 19eddecc0f Merge branch 'attr-set-hh' of https://github.com/nbp/nix
Conflicts:
	src/libexpr/eval.cc
2015-07-23 22:16:01 +02:00
Eelco Dolstra b83801f8b3 Optimize small lists
The value pointers of lists with 1 or 2 elements are now stored in the
list value itself. In particular, this makes the "concatMap (x: if
cond then [(f x)] else [])" idiom cheaper.
2015-07-23 22:05:09 +02:00
Eelco Dolstra 14be783676 Add primops all and any
These are used thousands of times during NixOS evaluation, so it's
useful to speed them up.
2015-07-23 19:23:11 +02:00
Shea Levy 39e27a04b8 Importing derivations: Add name attribute to make a valid drv 2015-07-23 17:04:07 +02:00
Shea Levy 1ed55234d9 Allow derivations-as-srcs in the context of builtins.toFile files 2015-07-23 17:03:47 +02:00
Eelco Dolstra 61af14a921 Add foldl' primop 2015-07-23 17:03:02 +02:00
Eelco Dolstra 887bb5fa5a --version: Print some config info
Such as whether Nix is built with signed binary cache support, and the
location of the configuration file.
2015-07-23 14:38:00 +02:00
Eelco Dolstra 1993b10d11 Fix Darwin build
Turns out getgrouplist() is not POSIX.

http://hydra.nixos.org/build/23881243
2015-07-21 14:45:24 +02:00
Eelco Dolstra 0a2bee307b Make <nix/fetchurl.nix> a builtin builder
This ensures that 1) the derivation doesn't change when Nix changes;
2) the derivation closure doesn't contain Nix and its dependencies; 3)
we don't have to rely on ugly chroot hacks.
2015-07-20 04:38:46 +02:00
Eelco Dolstra eda2f36c2a Provide more detailed info about build status to hydra-queue-runner
In particular, hydra-queue-runner can now distinguish between remote
build / substitution / already-valid. For instance, if a path already
existed on the remote side, we don't want to store a log file.
2015-07-20 03:20:03 +02:00
Eelco Dolstra ccf31dbc25 nix-copy-closure: Add -v flag
And make exportPath() less spammy by default.
2015-07-20 01:52:07 +02:00
Eelco Dolstra db55940d9e Support systemd log severity prefixes
This is mostly useful for hydra-queue-runner.
2015-07-20 01:39:48 +02:00
Eelco Dolstra b3491c781c More cleanup 2015-07-20 01:16:16 +02:00
Eelco Dolstra 6bd2c7bb38 OCD: foreach -> C++11 ranged for 2015-07-17 20:13:56 +02:00
Eelco Dolstra 1511aa9f48 Allow remote builds without sending the derivation closure
Previously, to build a derivation remotely, we had to copy the entire
closure of the .drv file to the remote machine, even though we only
need the top-level derivation. This is very wasteful: the closure can
contain thousands of store paths, and in some Hydra use cases, include
source paths that are very large (e.g. Git/Mercurial checkouts).

So now there is a new operation, StoreAPI::buildDerivation(), that
performs a build from an in-memory representation of a derivation
(BasicDerivation) rather than from a on-disk .drv file. The only files
that need to be in the Nix store are the sources of the derivation
(drv.inputSrcs), and the needed output paths of the dependencies (as
described by drv.inputDrvs). "nix-store --serve" exposes this
interface.

Note that this is a privileged operation, because you can construct a
derivation that builds any store path whatsoever. Fixing this will
require changing the hashing scheme (i.e., the output paths should be
computed from the other fields in BasicDerivation, allowing them to be
verified without access to other derivations). However, this would be
quite nice because it would allow .drv-free building (e.g. "nix-env
-i" wouldn't have to write any .drv files to disk).

Fixes #173.
2015-07-17 17:57:40 +02:00
Eelco Dolstra f39979c6d3 Make printValue() interruptible
Fixes #572.
2015-07-17 11:33:39 +02:00
Eelco Dolstra 7c9d0a5969 nix-collect-garbage: Handle ENOENT
Don't barf trying to read a link that just got deleted.

Fixes #575.
2015-07-17 11:24:25 +02:00
Nicolas B. Pierron db21cfa688 Move attribute set data structures into their own header file.
This modification moves Attr and Bindings structures into their own header
file which is dedicated to the attribute set representation. The goal of to
isolate pieces of code which are related to the attribute set
representation. Thus future modifications of the attribute set
representation will only have to modify these files, and not every other
file across the evaluator.
2015-07-14 19:23:17 +02:00
Guillaume Maudoux 467977f203 Fix the parsing of "$"'s in strings. 2015-07-03 14:09:58 +02:00
Guillaume Maudoux 65e4dcd69b Fix the hack that resets the scanner state. 2015-07-03 13:53:36 +02:00
Eelco Dolstra dd48c06bb6 Typo 2015-07-02 00:30:16 +02:00
Ludovic Courtès 9aed117395 Preserve supplementary groups of build users
The following patch is an attempt to address this bug (see
<http://bugs.gnu.org/18994>) by preserving the supplementary groups of
build users in the build environment.

In practice, I would expect that supplementary groups would contain only
one or two groups: the build users group, and possibly the “kvm” group.

[Changed &at(0) to data() and removed tabs - Eelco]
2015-07-01 14:57:48 +02:00
Eelco Dolstra ff4de4cb27 GC: Handle ENOSPC creating/moving to the trash directory
Issue #564.
2015-06-30 21:41:26 +02:00
Eelco Dolstra 2bc9c84327 Use posix_fallocate to create /nix/var/nix/db/reserved 2015-06-22 15:54:55 +02:00
Eelco Dolstra ba63ec6f39 Make /nix/var/nix/db/reserved bigger
Issue #564.
2015-06-22 15:47:40 +02:00
Eelco Dolstra 65f17cd330 Support URLs in $NIX_PATH
This didn't work (despite claims in the manual), because the colon in
"http://" was parsed as a element separator. So handle "://"
specially.
2015-06-17 16:20:11 +02:00
Eelco Dolstra b65875f859 Export outputPaths function
This is useful for the new hydra-queue-runner.
2015-06-10 16:17:06 +02:00
Eelco Dolstra 7c4501886d Use std::vector::data() 2015-06-09 10:54:46 +02:00
Eelco Dolstra b64988bb35 Allow substitutes for builds that have preferLocalBuild set
Not substituting builds with "preferLocalBuild = true" was a bad idea,
because it didn't take the cost of dependencies into account. For
instance, if we can't substitute a fetchgit call, then we have to
download/build git and all its dependencies.

Partially reverts 5558652709 and adds a
new derivation attribute "allowSubstitutes" to specify whether a
derivation may be substituted.
2015-06-04 16:30:22 +02:00
Eelco Dolstra 07d7e7df84 Chown files created for passAsFile
Nixpkgs' writeTextAsFile does this:

  mv "$textPath" "$n"

Since $textPath was owned by root, if $textPath is on the same
filesystem as $n, $n will be owned as root. As a result, the build
result was rejected as having suspicious ownership.

http://hydra.nixos.org/build/22836807
2015-06-04 14:07:43 +02:00
Eelco Dolstra d8ddf994e7 Don't let unprivileged users repair paths 2015-06-02 02:21:54 +02:00
Eelco Dolstra 7106bb0611 Use StoreAPI::verifyStore() 2015-06-02 02:21:15 +02:00
Ludovic Courtès b755752f76 Add a ‘verifyStore’ RPC
Hello!

The patch below adds a ‘verifyStore’ RPC with the same signature as the
current LocalStore::verifyStore method.

Thanks,
Ludo’.

>From aef46c03ca77eb6344f4892672eb6d9d06432041 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
Date: Mon, 1 Jun 2015 23:17:10 +0200
Subject: [PATCH] Add a 'verifyStore' remote procedure call.
2015-06-02 02:14:24 +02:00
Eelco Dolstra bc51175dc0 Add tarball tests 2015-06-01 16:18:23 +02:00
Eelco Dolstra da196ec68f Document tarball downloading 2015-06-01 15:14:44 +02:00
Eelco Dolstra c1323b53e3 Fix fetchurl/fetchTarball 2015-06-01 15:08:09 +02:00
Eelco Dolstra 1c88e100e7 readFile: Check against nul bytes 2015-06-01 15:07:42 +02:00
Eelco Dolstra 920f5fd4dd Fix import-from-derivation in restricted eval mode
This relaxes restricted mode to allow access to anything in the
store. In the future, it would be better to allow access to only paths
that have been constructed in the current evaluation (so a hard-coded
/nix/store/blabla in a Nix expression would still be
rejected). However, note that reading /nix/store itself is still
rejected, so you can't use this so get access to things you don't know
about.
2015-05-22 12:18:23 +02:00
Eelco Dolstra 7a411e01cf Remove dead code 2015-05-22 11:29:22 +02:00
Eelco Dolstra 4ca5a9dcfd nix-collect-garbage: Don't call nix-env
Also, make sure --delete-older-than doesn't delete the current
generation.
2015-05-21 16:28:30 +02:00
Eelco Dolstra 22b1a8d43f Move profiles.{cc,hh} to libstore 2015-05-21 15:42:59 +02:00
Eelco Dolstra 13493ef97c nix-collect-garbage: Call collectGarbage() internally 2015-05-21 15:21:38 +02:00
Eelco Dolstra 4441e4cc13 nix-collect-garbage: Don't barf on unreadable directories
And don't try to delete generations from unwritable directories.
2015-05-21 15:04:05 +02:00
Eelco Dolstra 8d813fe3e0 nix-collect-garbage: Remove redundant call to getFileType 2015-05-21 14:09:34 +02:00
Eelco Dolstra a1c1bf3a56 Merge branch 'submit/sparse-generation-symlinks' of https://github.com/ctheune/nix 2015-05-21 12:04:54 +02:00
Christian Theune 12a888894b Mis-read Eelko's request to not make this an option: now, let's not make
it an option. :)
2015-05-20 17:29:52 +02:00
Christian Theune ea39c98d41 Implement alternative to lazy generations:
* only the last generation can be lazy
* depend on the '--lazy-generation' flag to be set
2015-05-19 20:03:36 +02:00
Eelco Dolstra 1529db702b Don't install nix-worker symlink
It has been obsolete since Nix 1.2.

Closes #417.
2015-05-19 16:43:33 +02:00
Christian Theune 3d83188702 Enable lazy/sparse allocation of generation symlinks: avoid creating
new generations if a generation already exists.

Alternatively or additionally I propose a mode where only the *last* generation will be sparse.
2015-05-18 08:38:49 +02:00
Eelco Dolstra a010c0ae05 Fix "error: deriver of path ‘’ is not known" 2015-05-13 18:03:22 +02:00
Rok Garbas dad754843a cygwin: looks like stdout/stdin are reserved words 2015-05-13 09:37:56 +02:00
Rok Garbas 000de699e9 cygwin: explicitly include required c headers 2015-05-13 09:37:12 +02:00
Shea Levy 71083f9e5e Don't try to map /bin/sh to a store path on non-Linux 2015-05-12 16:36:15 -04:00
Luca Bruno 8972f7c14a nix-collect-garbage: Do not pass an empty argument. Closes #530 2015-05-06 15:17:23 +00:00