lix-project/lix
20
55
Fork 31
Code Issues 345 Code Review (Gerrit) Projects 3 Releases Wiki Activity
10342 commits 29 branches 6 tags 85 MiB
Commit graph

1115 commits

Author SHA1 Message Date
Daniël de Kok 2de232d2b3 Add x86_64 compute levels as additional system types 
When performing distributed builds of machine learning packages, it
would be nice if builders without the required SIMD instructions can
be excluded as build nodes.

Since x86_64 has accumulated a large number of different instruction
set extensions, listing all possible extensions would be unwieldy.
AMD, Intel, Red Hat, and SUSE have recently defined four different
microarchitecture levels that are now part of the x86-64 psABI
supplement and will be used in glibc 2.33:

https://gitlab.com/x86-psABIs/x86-64-ABI
https://lwn.net/Articles/844831/

This change uses libcpuid to detect CPU features and then uses them to
add the supported x86_64 levels to the additional system types. For
example on a Ryzen 3700X:

$ ~/aps/bin/nix -vv --version | grep "Additional system"
Additional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux
 2021-02-22 09:11:15 +01:00
regnat be1b5c4e59 Test the garbage collection of CA derivations 
Simple test to ensure that `nix-build && nix-collect-garbage &&
nix-build -j0` works as it should
 2021-02-19 15:48:31 +01:00
Graham Christensen f2245091d0
Revert "narinfo: Change NAR URLs to be addressed on the NAR hash instead of the compressed hash" 2021-02-09 12:26:41 -05:00
Eelco Dolstra ee3846b587
Merge pull request #4464 from tweag/nar-narhash-addressed 
narinfo: Change NAR URLs to be addressed on the NAR hash instead of the compressed hash
 2021-02-09 14:47:39 +01:00
Eelco Dolstra 0187838e2e Add a trace to readLine() failures 
Hopefully this helps to diagnose 'error: unexpected EOF reading a
line' on macOS.
 2021-02-05 12:18:11 +01:00
Eelco Dolstra 488a826842
Merge pull request #4467 from edolstra/error-formatting 
Improve error formatting
 2021-01-25 12:50:57 +01:00
Eelco Dolstra 680d8a5b86
Merge pull request #4387 from obsidiansystems/non-local-store-build 
Make `nix-build --store whatever` work
 2021-01-25 12:24:23 +01:00
John Ericson 8c07ed1dda Improve documentation and test and requested 2021-01-22 15:58:58 +00:00
Eelco Dolstra 55849e153e Change error position formatting 
It's now

  at /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix:7:7:

instead of

  at: (7:7) in file: /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix

The new format is more standard and clickable.
 2021-01-21 11:02:09 +01:00
adisbladis 144cad9069
narinfo: Change NAR URLs to be addressed on the NAR hash instead of the compressed hash 
This change is to simplify [Trustix](https://github.com/tweag/trustix) indexing and makes it possible to reconstruct this URL regardless of the compression used.

In particular this means that 7c2e9ca597/contrib/nix/nar/nar.go (L61-L71) can be removed and only the bits that are required to establish trust needs to be published in the Trustix build logs.
 2021-01-21 10:32:56 +01:00
Eelco Dolstra d04d8463fa
Merge pull request #4281 from lilyball/shebang 
Escape filename given to nix-shell in shebang mode
 2021-01-20 10:00:10 +01:00
John Ericson 0027b05a15 Merge remote-tracking branch 'upstream/master' into non-local-store-build 2021-01-15 02:01:24 +00:00
Eelco Dolstra 7a472a76d4 Add 'nix daemon' command 2021-01-14 00:05:04 +01:00
Eelco Dolstra d33eca8539 Rename 'nix store sign-paths' to 'nix store sign' 2021-01-13 23:32:37 +01:00
Eelco Dolstra 6254b1f5d2 Add 'nix store delete' command 2021-01-11 19:46:59 +01:00
Eelco Dolstra fdcd62eec5 Add 'nix store gc' command 2021-01-10 23:29:14 +01:00
Eelco Dolstra 0df69d96e0 Make sodium a required dependency 2021-01-06 17:56:53 +01:00
Eelco Dolstra 9374c2baea Add commands for generating secret/public keys 2021-01-06 17:49:31 +01:00
John Ericson fed1237246 Test nix-build with non-local-store --store 
Just a few small things needed fixing!
 2020-12-23 22:42:06 +00:00
John Ericson 29bd63e990 Test nix-instantiate with binary cache store 
Trying to make sure it work with obscurers stores.
 2020-12-23 22:42:06 +00:00
Eelco Dolstra f4a9fb67da Merge branch 'git-rev-error' of https://github.com/Ma27/nix into master 2020-12-22 16:39:32 +01:00
Eelco Dolstra 75efa42134 Move <nix/fetchurl.nix> into the nix binary 
This makes the statically linked nix binary just work, without needing
any additional files.
 2020-12-22 14:43:20 +01:00
Maximilian Bosch 897ae235fc
tests/fetchGit: test behavior of allRefs = true; 2020-12-22 12:18:10 +01:00
regnat 58cdab64ac Store metadata about drv outputs realisations 
For each known realisation, store:
- its output
- its output path

This comes with a set of needed changes:

- New `realisations` module declaring the types needed for describing
  these mappings
- New `Store::registerDrvOutput` method registering all the needed informations
  about a derivation output (also replaces `LocalStore::linkDeriverToPath`)
- new `Store::queryRealisation` method to retrieve the informations for a
  derivations

This introcudes some redundancy on the remote-store side between
`wopQueryDerivationOutputMap` and `wopQueryRealisation`.
However we might need to keep both (regardless of backwards compat)
because we sometimes need to get some infos for all the outputs of a
derivation (where `wopQueryDerivationOutputMap` is handy), but all the
stores can't implement it − because listing all the outputs of a
derivation isn't really possible for binary caches where the server
doesn't allow to list a directory.
 2020-12-11 20:41:32 +01:00
Eelco Dolstra 9c143c411b
Merge pull request #4350 from NixOS/ca/fix-build-with-nix-command 
Fix the `nix` command with CA derivations
 2020-12-11 15:01:49 +01:00
regnat eb45308109 Fix the nix command with CA derivations 
Prevents a crash because most `nix` subcommands assumed that derivations
know their output path, which isn't the case for CA derivations
 2020-12-11 10:28:09 +01:00
regnat 5286310e59 Use no substituers by default in the tests 
Otherwise https://cache.nixos.org is chosen by default, causing the OSX
testsuite to hang inside the sandbox.

(In a way, this is probably rugging an actual bug under the carpet as
Nix should be able to gracefully timeout in such a case, but that's
beyond mac OSX-fu)
 2020-12-09 14:53:45 +01:00
regnat ee7c94fa1b Test the post-build-hook with remote builders 
Regression test for #4245
 2020-12-09 10:45:12 +01:00
Eelco Dolstra f337aa7099 Split 'nix store add-to-store' into 'add-path' and 'add-file' 
This makes it consistent with 'nix hash <path|file>'.
 2020-12-04 00:59:24 +01:00
Eelco Dolstra ea2062a2d9 Move most store-related commands to 'nix store' 2020-12-03 23:22:22 +01:00
Eelco Dolstra ef583303f0 Move NAR-related commands to 'nix nar' 2020-12-03 18:09:02 +01:00
Eelco Dolstra b2d6c6161e Move 'nix hash-*' and 'nix to-*' to 'nix hash' 
From the 'nix' UX review.
 2020-12-03 17:55:55 +01:00
Eelco Dolstra 0bd060f23a
Merge pull request #4308 from tweag/properly-test-early-cutoff 
Properly test the early cutoff for CA derivations
 2020-12-03 14:45:29 +01:00
regnat 8ad72b1f1c Properly test early cutoff with CA derivations 
Build things with a different seed each time to make sure that it works
despite the different drvs
 2020-12-03 13:31:07 +01:00
Eelco Dolstra df552a2645 nix eval: Add option to write a directory 
This is useful for generating the nix manpages, but it may have other
applications (like generating configuration files without a Nix store).
 2020-12-02 23:23:23 +01:00
Eelco Dolstra e5cf501c77
Merge pull request #4284 from tweag/fixed-output-depending-on-ca 
Allow fixed-output derivations to depend on (floating) content-addressed ones
 2020-12-01 20:25:41 +01:00
regnat 9bd8184f1f Allow fixed-output derivations to depend on (floating) content-addressed ones 
Fix an overlook of https://github.com/NixOS/nix/pull/4056
 2020-11-27 15:39:24 +01:00
Eelco Dolstra 05d9442f68 builtins.fetchGit: Fix shortRev attribute for dirty trees 2020-11-26 21:45:28 +01:00
regnat 13c557fe82 fix the hash rewriting for ca-derivations 2020-11-25 11:33:00 +01:00
Lily Ballard 437189e446 Escape filename given to nix-shell in shebang mode 
This prevents spaces or other metacharacters from causing nix-shell to
execute the wrong path.

Fixes #4229.
 2020-11-24 15:08:37 -08:00
Luke Granger-Brown 226116f482 fetchMercurial: set HGPLAIN when invoking hg 
Without setting HGPLAIN, the user's environment leaks into
hg invocations, which means that the output may not be in the
expected format.

HGPLAIN is the Mercurial-recommended solution for this in that
it's intended for uses by scripts and programs which are looking
to parse Mercurial's output in a consistent manner.
 2020-11-23 16:12:33 +00:00
Eelco Dolstra f89fd0bde7 Remove stray debug statement 
This was causing a failure on macOS.

https://hydra.nixos.org/build/130354318
 2020-11-17 15:36:20 +01:00
Eelco Dolstra f4e790cc85
Merge pull request #4182 from mkenigs/fix-1930 
Print built derivations as json for build
 2020-11-17 14:59:49 +01:00
Eelco Dolstra bccff827dc Fix deadlock in IFD through the daemon 
Fixes #4235.
 2020-11-17 13:50:36 +01:00
Robert Hensing d264da8d96 tests: Test #4197 nix-build output order regression 2020-11-13 17:50:04 +01:00
Matthew Kenigsberg d52b12c0a5 Test nix build --json 2020-11-11 10:27:02 -06:00
Sebastian Ullrich 3f24a417da Add test case for incidentally fixed #4228 2020-11-09 22:43:14 +01:00
Sebastian Ullrich 579b953231 Make test case more precise 
Co-authored-by: Théophane Hufschmitt <regnat@users.noreply.github.com>
 2020-11-09 22:43:14 +01:00
Sebastian Ullrich fb7735e4cf nix develop: Preserve stdin with -c 2020-11-09 22:43:14 +01:00
Graham Christensen 8cd2ff69c3
nix-copy-closure: verify it works with drvs 
Creates test coverage for #4210 and 7cf874c17d
 2020-11-02 15:50:14 -05:00
Eelco Dolstra 02a1facbdc
Merge pull request #4056 from tweag/non-ca-depending-on-ca 
Allow non-CA derivations to depend on CA ones
 2020-10-27 17:38:29 +01:00
regnat ab21ab6501 Test the remote caching of non-ca-depending-on-ca derivations 
Although the non-resolved derivation will never get a cache-hit (it
doesn't have an output path to query the cache for anyways), we might
get one on the resolved derivation.
 2020-10-27 07:29:25 +01:00
regnat c092fa4702 Allow non-CA derivations to depend on CA derivations 2020-10-27 07:29:23 +01:00
Eelco Dolstra 1e66d146a3 Fix test 2020-10-26 17:59:32 +01:00
Christian Kampka 461cf2b856
Add NIX_CONFIG env var for applying nix.conf overrides 2020-10-21 13:41:26 +02:00
Robert Hensing ea8d32020e Tests for #3964 2020-10-18 14:26:37 +02:00
aszlig cfa26cf181
tests: Add names to VM tests 
Having vm-test-run-unnamed for all the test derivation doesn't look very
nice, so in order to better distinguish them from their store path,
let's actually give them proper names.

Signed-off-by: aszlig <aszlig@nix.build>
 2020-10-17 23:34:38 +02:00
aszlig 5cfdf16dd6
Convert VM tests to Python 
Perl-based tests are deprecated since NixOS 20.03 and subsequently got
removed in NixOS 20.09, which effectively means that tests are going to
fail as soon as we build it with NixOS 20.09 or anything newer.

I've put "# fmt: off" at the start of every testScript, because
formatting with Black really messes up indentation and I don't think it
really adds anything in value or readability for inlined Python scripts.

Signed-off-by: aszlig <aszlig@nix.build>
 2020-10-17 23:32:03 +02:00
Eelco Dolstra 1e8855a7f7
Merge pull request #3958 from obsidiansystems/ca-floating-upstream 
CA derivations that depend on other CA derivations
 2020-09-29 11:37:29 +02:00
John Ericson 10202bbf29 Merge remote-tracking branch 'upstream/master' into ca-floating-upstream 2020-09-28 15:39:11 +00:00
Mateusz Piotrowski ed66d01065
Fix tar invocation on FreeBSD 
tar(1) on FreeBSD does not use standard output or input when the -f flag
is not provided. Instead, it defaults to /dev/sa0 on FreeBSD.

Make this tar invocation a bit more robust and explicitly tell tar(1) to
use standard output.

This is one of the issues discovered while porting Nix to FreeBSD. It has
been tested and committed locally to FreeBSD ports:
https://svnweb.freebsd.org/ports/head/sysutils/nix/Makefile?revision=550026&view=markup#l108
 2020-09-28 15:23:21 +02:00
Eelco Dolstra 31ab4c3816 Test whether build/repair results are read-only 2020-09-23 19:09:58 +02:00
John Ericson b7df353f27 Merge remote-tracking branch 'upstream/master' into ca-floating-upstream 2020-09-17 16:33:10 +00:00
Eelco Dolstra 649d3aaf24
Merge pull request #3829 from obsidiansystems/remove-storetype-delegate-regStore 
Remove storetype delegate reg store -- contains #3736
 2020-09-17 13:55:01 +02:00
Eelco Dolstra b94a35ef40
Merge pull request #4027 from tweag/fix-gc-of-ca-derivations 
Fix garbage collection of CA derivations
 2020-09-17 13:46:26 +02:00
regnat 520895b1da Fix garbage collection of CA derivations 
Fix #4026
 2020-09-17 13:36:58 +02:00
Eelco Dolstra c9f51e8705 Remove corepkgs/config.nix 
This isn't used anywhere except in the configure script of the Perl
bindings. I've changed the latter to use the C++ API's Settings object
at runtime.
 2020-09-17 10:42:51 +02:00
John Ericson f60b380a7f Merge remote-tracking branch 'upstream/master' into remove-storetype-delegate-regStore 2020-09-16 22:35:24 +00:00
John Ericson c5ccebae00 Merge remote-tracking branch 'upstream/master' into ca-floating-upstream 2020-09-16 17:50:40 +00:00
regnat c29624bf7d Add a test for nix describe-stores 
Doesn't test much, but at least ensures that the command runs properly
 2020-09-16 13:53:28 +02:00
John Ericson 3ba552b245 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-09-15 14:17:06 +00:00
John Ericson c08c9f08c7 Merge remote-tracking branch 'upstream/master' into remove-storetype-delegate-regStore 2020-09-15 14:08:35 +00:00
regnat 250f8a4bba Escape ${ in strings when printing Nix expressions 
Otherwise the result of the printing can't be parsed back correctly by
Nix (because the unescaped `${` will be parsed as the begining of an
anti-quotation).

Fix #3989
 2020-09-14 17:19:25 +02:00
Jade Lovelace 7cb5f643a6
docs+test: fix remaining installer downloads without -L (#4006) 
Co-authored-by: lf- <lf-@users.noreply.github.com>
 2020-09-12 22:08:40 +02:00
John Ericson 075d399e3f Merge remote-tracking branch 'obsidian/single-ca-drv-build' into ca-floating-upstream 2020-09-04 16:04:35 +00:00
John Ericson c9f1ed912c Don't chmod symlink before moving outputs around 
Co-authored-by: Théophane Hufschmitt <regnat@users.noreply.github.com>
 2020-09-04 14:41:53 +00:00
John Ericson b99062b023
Update tests/content-addressed.nix 
Co-authored-by: Théophane Hufschmitt <regnat@users.noreply.github.com>
 2020-09-04 10:29:28 -04:00
John Ericson aad4abcc9c Fix floating CA tests 
We will sometimes try to query the outputs of derivations we can't
resolve. That's fine; it just means we don't know what those outputs are
yet.
 2020-09-04 01:17:38 +00:00
John Ericson b836662f50 Merge remote-tracking branch 'obsidian/single-ca-drv-build' into ca-floating-upstream 
Tests also now fail as they should
 2020-09-03 22:45:00 +00:00
John Ericson c224a5e5c1 Rename derivation in floating CA test 2020-09-03 22:35:13 +00:00
John Ericson 145915eb39 Beef up floating CA derivations test a bit 2020-09-03 22:14:45 +00:00
John Ericson 975a47f7fe Merge remote-tracking branch 'obsidian/single-ca-drv-build' into ca-floating-upstream 2020-09-03 22:09:04 +00:00
John Ericson e7d93e7ece Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-09-03 15:43:17 +00:00
Eelco Dolstra 00d25e8457
Remove the --indirect flag 
All GC roots are now indirect.
 2020-09-03 11:22:00 +02:00
John Ericson ef278d00f9 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-09-01 18:01:48 +00:00
John Ericson 4db0010a93 Test CA derivation input caching 2020-08-28 22:03:54 +00:00
Eelco Dolstra 4bf5faf416 Merge remote-tracking branch 'origin/master' into markdown 2020-08-25 19:47:34 +02:00
Eelco Dolstra 7a02865b94
Move import docs 2020-08-25 14:06:01 +02:00
John Ericson 8eb73a8724 CA derivations that depend on other CA derivations 
Co-authored-by: Théophane Hufschmitt <regnat@users.noreply.github.com>
 2020-08-24 19:01:15 +00:00
Eelco Dolstra 6a67e57019
Add DummyStore (dummy://) 
DummyStore does not allow building or adding paths. This is useful for
evaluation tests when you don't want to initialize a "proper" store.
 2020-08-24 18:54:16 +02:00
John Ericson 27a3f82c0b Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-08-20 18:28:17 +00:00
John Ericson 3df78858f2 Fix max fd calc and add test 2020-08-20 05:08:50 +00:00
Eelco Dolstra 0c9365c6ba
Merge pull request #3940 from obsidiansystems/fixed-output-remote-builder-test 
Add commented-out test for remote building with fixed output derivations
 2020-08-18 16:12:35 +02:00
Carlo Nucera 07975979aa Comment out fixed content address test 2020-08-17 15:04:54 -04:00
Eelco Dolstra 7cdc739ece
Merge remote-tracking branch 'origin/master' into markdown 2020-08-17 13:43:39 +02:00
John Ericson dbf96e10ec Test remote building with fixed output derivations 2020-08-16 17:38:12 +00:00
John Ericson 3c8b5b6219 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-08-14 17:00:13 +00:00
Eelco Dolstra 7714d9a943
Merge pull request #3924 from obsidiansystems/features-per-store 
Make `system-features` a store setting
 2020-08-14 17:13:07 +02:00
John Ericson d2f2be0f70 Test RemoteStore::buildDerivation 
Fix `wopNarFromPath` which needed a `toRealPath`.
 2020-08-13 04:07:14 +00:00
John Ericson 5d67f18c86 Merge branch 'daemon-auth-cleanup' of github.com:obsidiansystems/nix into HEAD 2020-08-12 18:22:31 +00:00