lunaphied/lix
0
0
Fork 0
forked from lix-project/lix
Code Pull requests Activity
4563 commits 2 branches 3 tags 72 MiB
Commit graph

2571 commits

Author SHA1 Message Date
Eelco Dolstra 76f1ba4f3b Add file missing from 201b48de60 2016-03-03 18:03:34 +01:00
Nathan Zadoks fe2be8f016 build.cc: fs.h doesn't appear to be necessary anymore 2016-03-03 14:11:00 +01:00
Eelco Dolstra 5a8455c85e Provide function required by Hydra 2016-03-02 18:21:48 +01:00
Eelco Dolstra 25eea97873 Force stack trace for boost format errors 2016-03-02 15:46:07 +01:00
Eelco Dolstra 201b48de60 Add an HTTP binary cache store 
Allowing stuff like

  NIX_REMOTE=https://cache.nixos.org nix-store -qR /nix/store/x1p1gl3a4kkz5ci0nfbayjqlqmczp1kq-geeqie-1.1

or

  NIX_REMOTE=https://cache.nixos.org nix-store --export /nix/store/x1p1gl3a4kkz5ci0nfbayjqlqmczp1kq-geeqie-1.1 | nix-store --import
 2016-03-02 15:46:07 +01:00
Eelco Dolstra 0402b6398d Eliminate local-binary-cache-store.hh 2016-03-02 15:46:07 +01:00
Eelco Dolstra 68a5414982 Make store implementations pluggable 
This for instance allows hydra-queue-runner to add the S3 backend
at runtime.
 2016-03-02 15:46:07 +01:00
Eelco Dolstra 6055d84beb Fix reading symlinks 
The st_size field of a symlink doesn't have to be correct, e.g. for
/dev/fd symlinks.
 2016-03-02 15:46:07 +01:00
Eelco Dolstra 0a62d9b3d7 Remove bad assertion 2016-03-02 15:46:07 +01:00
Eelco Dolstra 89a2fa68ac FdSource: track number of bytes read 2016-03-02 15:46:07 +01:00
Eelco Dolstra cca4a8dc1a importPaths(): Optionally add NARs to binary cache accessor 
This enables an optimisation in hydra-queue-runner, preventing a
download of a NAR it just uploaded to the cache when reading files
like hydra-build-products.
 2016-03-02 15:46:07 +01:00
Eelco Dolstra e9c50064b5 Add an HTTP binary cache store 
Allowing stuff like

  NIX_REMOTE=https://cache.nixos.org nix-store -qR /nix/store/x1p1gl3a4kkz5ci0nfbayjqlqmczp1kq-geeqie-1.1

or

  NIX_REMOTE=https://cache.nixos.org nix-store --export /nix/store/x1p1gl3a4kkz5ci0nfbayjqlqmczp1kq-geeqie-1.1 | nix-store --import
 2016-02-29 18:15:20 +01:00
Eelco Dolstra 6170bb474b Eliminate local-binary-cache-store.hh 2016-02-29 16:14:39 +01:00
Eelco Dolstra 0b907321cc Make store implementations pluggable 
This for instance allows hydra-queue-runner to add the S3 backend
at runtime.
 2016-02-29 16:11:11 +01:00
Eelco Dolstra 012f8d187c Fix reading symlinks 
The st_size field of a symlink doesn't have to be correct, e.g. for
/dev/fd symlinks.
 2016-02-28 03:27:34 +01:00
Eelco Dolstra d0344dd2c1 Remove bad assertion 2016-02-26 21:43:59 +01:00
Eelco Dolstra 581e1bc5b4 FdSource: track number of bytes read 2016-02-26 16:16:08 +01:00
Eelco Dolstra bcc9943cee importPaths(): Optionally add NARs to binary cache accessor 
This enables an optimisation in hydra-queue-runner, preventing a
download of a NAR it just uploaded to the cache when reading files
like hydra-build-products.
 2016-02-26 15:20:10 +01:00
Eelco Dolstra 00b2c05749 nix: Add commands to query contents of NARs / binary caches 
For example,

  $ NIX_REMOTE=file:///my-cache nix ls-store -lR /nix/store/f4kbgl8shhyy76rkk3nbxr0lz8d2ip7q-binutils-2.23.1
  dr-xr-xr-x                    0 ./bin
  -r-xr-xr-x                30748 ./bin/addr2line
  -r-xr-xr-x                66973 ./bin/ar
  ...

Similarly, "nix ls-nar" lists the contents of a NAR file, "nix
cat-nar" extracts a file from a NAR file, and "nix cat-store" extract
a file from a Nix store.
 2016-02-25 17:57:00 +01:00
Eelco Dolstra 24a8f9e27b Merge branch 'master' into new-cli 2016-02-25 17:48:35 +01:00
Eelco Dolstra 1042c10fd0 Add NAR / Store accessor abstraction 
This is primary to allow hydra-queue-runner to extract files like
"nix-support/hydra-build-products" from NARs in binary caches.
 2016-02-25 17:43:19 +01:00
Eelco Dolstra c5bc571861 Fix short boolean flags 2016-02-25 13:31:34 +01:00
Eelco Dolstra 7873cfb18d Fix build 2016-02-25 11:55:05 +01:00
Eelco Dolstra f1bdeac986 Merge branch 'master' into new-cli 2016-02-25 11:25:11 +01:00
Eelco Dolstra 152b1d6bf9 deletePath(): Succeed if path doesn't exist 
Also makes it robust against concurrent deletions.
 2016-02-24 17:44:55 +01:00
Eelco Dolstra 28e7e29abd Eliminate reserveSpace flag 2016-02-24 17:44:55 +01:00
Eelco Dolstra 5a64e66268 LocalStore::addTextToStore(): Don't read the path we just wrote 
This eliminates some unnecessary (presumably cached) I/O.
 2016-02-24 17:11:31 +01:00
Eelco Dolstra 9ccbd55c5b BinaryCacheStore: Implement addToStore() 
So now you can do

  $ NIX_REMOTE=file:///tmp/binary-cache nix-instantiate '<nixpkgs>' -A hello

and lots of other operations.
 2016-02-24 16:52:28 +01:00
Eelco Dolstra 30e9d01516 openStoreAt(): Initialise the binary cache 2016-02-24 16:52:07 +01:00
Eelco Dolstra ba0a81d14f Show a proper error message for unimplemented binary cache operations 2016-02-24 14:57:30 +01:00
Eelco Dolstra 45c83e5f9b Typo 2016-02-24 14:49:28 +01:00
Eelco Dolstra 263187a2ec Move BinaryCacheStore / LocalBinaryCacheStore from Hydra 
So you can now do:

  $ NIX_REMOTE=file:///tmp/binary-cache nix-store -qR /nix/store/...
 2016-02-24 14:48:16 +01:00
Eelco Dolstra b584a0e7de Merge pull request #816 from tsion/incomplete-parse 
Throw a specific error for incomplete parse errors.
 2016-02-24 13:34:56 +01:00
Eelco Dolstra bf2adf72c4 std::condition_variable_any -> std::condition_variable 
The latter is supposed to be more efficient.
 2016-02-24 13:31:46 +01:00
Eelco Dolstra ccdbf589a4 C++ templates are just a glorified macro facility 2016-02-24 13:07:32 +01:00
Eelco Dolstra 5f862658c3 Remove bad daemon connections from the pool 
This is necessary for long-running processes like hydra-queue-runner:
if a nix-daemon worker is killed, we need to stop reusing that
connection.
 2016-02-24 11:39:56 +01:00
Scott Olson 6498adb002 Throw a specific error for incomplete parse errors. 
`nix-repl` will use this for deciding whether to keep waiting for input or
error out right away.
 2016-02-24 04:32:21 -06:00
Eelco Dolstra d5626bf4c1 Pool<T>: Allow a maximum pool size 2016-02-23 16:40:16 +01:00
Eelco Dolstra e292144d46 RemoteStore: Make thread-safe 
This allows a RemoteStore object to be used safely from multiple
threads concurrently. It will make multiple daemon connections if
necessary.

Note: pool.hh and sync.hh have been copied from the Hydra source tree.
 2016-02-23 15:00:59 +01:00
Eelco Dolstra c0b7a8a0b5 Move ref<t> into a separate header 2016-02-23 13:53:31 +01:00
Eelco Dolstra 8f71bc33d5 Doh 2016-02-22 17:33:27 +01:00
Eelco Dolstra 840056af04 Make OpenSSL usage thread-safe 
OpenSSL can randomly segfault unless we register a callback function
to do locking.

https://www.openssl.org/docs/manmaster/crypto/threads.html
 2016-02-22 15:51:14 +01:00
Eelco Dolstra d361901bfe curl: Set CURLOPT_NOSIGNAL 
Otherwise using curl is not safe in multi-threaded applications
because it installs a SIGALRM handler.
 2016-02-22 15:51:14 +01:00
Eelco Dolstra 7a173a7be1 JSONObject: Support floats and booleans 2016-02-19 14:24:04 +01:00
Eelco Dolstra b49d323ce2 Fix build without sodium 
http://hydra.nixos.org/build/32085949
 2016-02-17 12:42:14 +01:00
Eelco Dolstra c4d22997f3 Add C++ functions for .narinfo processing / signing 
This is currently only used by the Hydra queue runner rework, but like
eff5021eaa it presumably will be useful
for the C++ rewrite of nix-push and
download-from-binary-cache. (@shlevy)
 2016-02-16 16:38:44 +01:00
Eelco Dolstra 5ac27053e9 Rename ValidPathInfo::hash -> narHash for consistency 2016-02-16 11:49:12 +01:00
Tristan Hume 9b05d5848c Fix to-base16 description 2016-02-16 10:15:58 +01:00
Eelco Dolstra eff5021eaa Add xz compression function 
This is used by the Hydra queue runner, but since it may also be
useful for the C++ rewrite of nix-push, I'm putting it here.
 2016-02-15 21:45:56 +01:00
Eelco Dolstra bfdacb712c decompressXZ: Ensure that lzma_end() is called 
Otherwise we might leak memory.
 2016-02-15 21:26:28 +01:00
Eelco Dolstra c8f4d89a34 Expose the export magic value and move LocalStore::queryReferences to Store 2016-02-15 15:01:26 +01:00
Eelco Dolstra e03d6e0998 Fix broken number parsing in fromJSON 
The call to tmp_number.append had its arguments mixed up. Also, JSON
does not allow a trailing "," after array/object members.
 2016-02-15 15:01:26 +01:00
Eelco Dolstra d089372565 Add function to extract hash part of a store path 2016-02-15 15:01:26 +01:00
Eelco Dolstra b3e8d72770 Merge pull request #762 from ctheune/ctheune-floats 
Implement floats
 2016-02-12 12:49:59 +01:00
Eelco Dolstra ae4a3cfa03 Move addPermRoot into Store 2016-02-11 16:14:42 +01:00
Eelco Dolstra fd205fb6f8 ref<T>: Add cast operator 2016-02-11 15:32:48 +01:00
Eelco Dolstra 206bbb5dc9 Add basic "nix build" command 
Currently only builds by attribute from <nixpkgs> or the specified
file, e.g. "nix build hello".
 2016-02-09 21:34:24 +01:00
Eelco Dolstra cd2196b089 Start of new Nix command-line interface 2016-02-09 21:28:29 +01:00
Eelco Dolstra 0db9e6cd1a New command line parsing infrastructure 2016-02-09 21:07:48 +01:00
Eelco Dolstra c780c1124e More of the same 2016-02-04 15:10:47 +01:00
Eelco Dolstra fa7cd5369b StoreAPI -> Store 
Calling a class an API is a bit redundant...
 2016-02-04 14:48:42 +01:00
Eelco Dolstra c10c61449f Eliminate the "store" global variable 
Also, move a few free-standing functions into StoreAPI and Derivation.

Also, introduce a non-nullable smart pointer, ref<T>, which is just a
wrapper around std::shared_ptr ensuring that the pointer is never
null. (For reference-counted values, this is better than passing a
"T&", because the latter doesn't maintain the refcount. Usually, the
caller will have a shared_ptr keeping the value alive, but that's not
always the case, e.g., when passing a reference to a std::thread via
std::bind.)
 2016-02-04 14:28:26 +01:00
Eelco Dolstra d367b8e787 Add build mode to compute fixed-output derivation hashes 
For example,

  $ nix-build --hash -A nix-repl.src

will build the fixed-output derivation nix-repl.src (a fetchFromGitHub
call), but instead of *verifying* the hash given in the Nix
expression, it prints out the resulting hash, and then moves the
result to its content-addressed location in the Nix store. E.g

  build produced path ‘/nix/store/504a4k6zi69dq0yjc0bm12pa65bccxam-nix-repl-8a2f5f0607540ffe56b56d52db544373e1efb980-src’ with sha256 hash ‘0cjablz01i0g9smnavhf86imwx1f9mnh5flax75i615ml71gsr88’

The goal of this is to make all nix-prefetch-* scripts unnecessary: we
can just let Nix run the real thing (i.e., the corresponding fetch*
derivation).

Another example:

  $ nix-build --hash -E 'with import <nixpkgs> {}; fetchgit { url = "https://github.com/NixOS/nix.git"; sha256 = "ffffffffffffffffffffffffffffffffffffffffffffffffffff"; }'
  ...
  git revision is 9e7c1a4bbd
  ...
  build produced path ‘/nix/store/gmsnh9i7x4mb7pyd2ns7n3c9l90jfsi1-nix’ with sha256 hash ‘1188xb621diw89n25rifqg9lxnzpz7nj5bfh4i1y3dnis0dmc0zp’

(Having to specify a fake sha256 hash is a bit annoying...)
 2016-01-31 12:18:09 +01:00
Eelco Dolstra 9e7c1a4bbd Use the daemon when we don't have write access to the Nix database 2016-01-31 10:28:48 +01:00
Eelco Dolstra 4fa08f3edb Untabify 2016-01-28 16:03:32 +01:00
Eelco Dolstra 26566cd28e Untabify 2016-01-28 16:01:01 +01:00
Eelco Dolstra 9f14d7d33a printMissing(): Propagate store argument 2016-01-28 15:16:06 +01:00
Eelco Dolstra 7899fc959d Define HashType's size to allow it to be forward-declared 2016-01-27 17:46:01 +01:00
Eelco Dolstra 739bab0be7 Trivia 2016-01-27 17:18:31 +01:00
Eelco Dolstra d45ad8fcf5 Make hashLength32() a method of Hash 2016-01-27 17:18:20 +01:00
Eelco Dolstra 5b8c09c124 string2Int: Barf on negative numbers for unsigned types 2016-01-20 16:45:05 +01:00
Eelco Dolstra 5d8b7eb3e1 Revert "Revert "next try for "don't abort when given unmatched '}' with 'start-condition stack underflow'. This fixes #751""" 
This reverts commit b669d3d2e8.
 2016-01-20 16:34:42 +01:00
Eelco Dolstra b669d3d2e8 Revert "next try for "don't abort when given unmatched '}' with 'start-condition stack underflow'. This fixes #751"" 
This reverts commit ed23c8568e. Let's
merge this *after* the 1.11.1 release.
 2016-01-20 00:05:28 +01:00
Fabian Schmitthenner ed23c8568e next try for "don't abort when given unmatched '}' with 'start-condition stack underflow'. This fixes #751" 
This reverts commit 8120b6fb8a and fixes the regression introduced in
8d22b26448.
 2016-01-19 20:35:35 +00:00
Eelco Dolstra 8120b6fb8a Revert "don't abort when given unmatched '}' with 'start-condition stack underflow'. This fixes #751" 
This reverts commit 8d22b26448. It
breaks Nixpkgs:

$ nix-env -qa
error: syntax error, unexpected IND_STR, expecting '}', at /home/eelco/Dev/nixpkgs-stable/pkgs/top-level/python-packages.nix:7605:8
 2016-01-19 20:33:32 +01:00
Shea Levy a6d52fcd95 Merge branch 'lexer' of git://github.com/fkz/nix 2016-01-17 10:08:03 -05:00
Fabian Schmitthenner 8d22b26448 don't abort when given unmatched '}' with 'start-condition stack underflow'. This fixes #751 2016-01-12 20:40:41 +00:00
Eelco Dolstra 786046cf13 --option build-repeat: Keep the differing output if -K is given 
Similar to 00903fa799. Regardless of -K,
we now also print which output differs.
 2016-01-12 18:26:24 +01:00
Eelco Dolstra 8906eda2f9 Canonicalize gids to 0 
Previously files in the Nix store were owned by root or by nixbld,
depending on whether they were created by a substituter or by a
builder. This doesn't matter much, but causes spurious diffoscope
differences. So use root everywhere.
 2016-01-12 17:27:40 +01:00
Eelco Dolstra 00903fa799 --check: Keep the differing output if -K is given 
This makes it easier to investigate the non-determinism, e.g.

  $ nix-build pkgs/stdenv/linux -A stage1.pkgs.zlib --check -K
  error: derivation ‘/nix/store/l54i8wlw22656i4pk05c52ngv9rpl39q-zlib-1.2.8.drv’ may not be deterministic: output ‘/nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8’ differs from ‘/nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8-check’

  $ diffoscope /nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8 /nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8-check
  ...
  ├── lib/libz.a
  │   ├── metadata
  │   │ @@ -1,15 +1,15 @@
  │   │ -rw-r--r-- 30001/30000   3096 Jan 12 15:20 2016 adler32.o
  ...
  │   │ +rw-r--r-- 30001/30000   3096 Jan 12 15:28 2016 adler32.o
  ...
 2016-01-12 16:44:26 +01:00
Eelco Dolstra 0cad1f8049 --check: Fix "failed to produce output path" 
This occured when sandbox building is disabled, at least one output
exists, and at least one other output does not.
 2016-01-12 15:13:47 +01:00
Eelco Dolstra 1c57ab8b31 --check: Fix assertion failure when some outputs are missing 
E.g.

  $ nix-build pkgs/stdenv/linux/ -A stage1.pkgs.perl --check
  nix-store: src/libstore/build.cc:1323: void nix::DerivationGoal::tryToBuild(): Assertion `buildMode != bmCheck || validPaths.size() == drv->outputs.size()' failed.

when perl.out exists but perl.man doesn't. The fix is to only check
the outputs that exist. Note that "nix-build -A stage1.pkgs.all
--check" will still give a (proper) error in this case.
 2016-01-12 14:54:39 +01:00
Eelco Dolstra 458711e4ee Fix "Bad address" executing build hook 
This was observed in the deb_debian7x86_64 build:
http://hydra.nixos.org/build/29973215

Calling c_str() on a temporary should be fine because the temporary
shouldn't be destroyed until after the execl() call, but who knows...
 2016-01-07 15:10:14 +01:00
Eelco Dolstra 9aac1861f7 Fix some signedness warnings 2016-01-07 14:37:39 +01:00
Eelco Dolstra 02a66b3fd7 nix-store --repair-path: Rebuild if there is no substituter 2016-01-06 22:07:59 +01:00
Eelco Dolstra caaaff3954 Fix --repair failure on multiple-output derivations 
If repair found a corrupted/missing path that depended on a
multiple-output derivation, and some of the outputs of the latter were
not present, it failed with a message like

  error: path ‘/nix/store/cnfn9d5fjys1y93cz9shld2xwaibd7nn-bash-4.3-p42-doc’ is not valid
 2016-01-06 21:49:32 +01:00
Christian Theune 934642155c @eelco's feedback: downgrade to regular float for size, remove unused function. 2016-01-06 08:25:58 +01:00
Fabian Schmitthenner 39d1da7b51 Better error message 
Also show types when nix cannot compare values of different types.
This is also more consistent since types are already shown when comparing values of the same not comparable type.
 2016-01-05 14:13:54 +01:00
Eelco Dolstra 9a4706eb19 Fix compilation 2016-01-05 14:05:11 +01:00
Shea Levy 14080f3e4b Use __toString when coercing sets to strings. 
For example, "${{ foo = "bar"; __toString = x: x.foo; }}" evaluates
to "bar".

With this, we can delay calling functions like mkDerivation,
buildPythonPackage, etc. until we actually need a derivation, enabling
overrides and other modifications to happen by simple attribute set
update.
 2016-01-05 13:55:06 +01:00
Eelco Dolstra 6298afc047 Merge pull request #685 from vizanto/master 
POSIX compliant directory access (fixes build on Solaris)
 2016-01-05 13:49:55 +01:00
FrankHB 607d91517e Fixed unresolved 'BUFSIZ' 
`BUFSIZ` is defined in header `<cstdio>`.
 2016-01-05 13:33:40 +01:00
Eelco Dolstra 743e310046 Fix non-Darwin build 2016-01-05 13:31:15 +01:00
Vladimír Čunát 29c0199d5c manual: document builtins.functionArgs 
The text is just a conversion of comment from src/libexpr/primops.cc
 2016-01-05 13:29:17 +01:00
Tuomas Tynkkynen f770b9e6c5 libstore: mmap() returns MAP_FAILED, not NULL on failure 2016-01-05 13:26:35 +01:00
Christian Theune a12a43046b Edge condition: parser did not pick up floats starting exactly with 0. 2016-01-05 09:54:49 +01:00
Christian Theune f872262e08 Fix up float parsing. 2016-01-05 09:46:37 +01:00
Christian Theune 494fc5acbb Try a simplified version of float lexing that didn't work. 
The last one I tried was botchered anyway ...
 2016-01-05 00:53:22 +01:00
Christian Theune 14ebde5289 First hit at providing support for floats in the language. 2016-01-05 00:40:40 +01:00
Eelco Dolstra 71a93a5f0e Don't allow sandbox profile except in relaxed mode 
This makes Darwin consistent with Linux: Nix expressions can't break
out of the sandbox unless relaxed sandbox mode is enabled.

For the normal sandbox mode this will require fixing #759 however.
 2016-01-04 20:01:13 +01:00