Commit graph

6886 commits

Author SHA1 Message Date
regnat fbc70034b3 Make the realisation fetching from binary caches async
That way we can fetch several realisations from the same cache in
parallel
2021-11-03 06:51:34 +01:00
regnat 96670ed216 Expose an async interface for queryRealisation
Doesn’t change much so far because everything is still using it
synchronously, but should allow the binary cache to fetch stuff in
parallel
2021-11-03 06:51:34 +01:00
regnat 5b2aa61f1b Don’t require ca-derivations when __contentAddressed = false
If we explicitely opt-out of it, there’s no need to require the
experimental feature
2021-11-03 06:51:32 +01:00
Eelco Dolstra e6795c4350 Style 2021-10-29 14:45:13 +02:00
Eelco Dolstra 6e30d9b69f Merge branch 'master' of https://github.com/alekswn/nix 2021-10-29 14:42:26 +02:00
Eelco Dolstra 33d04e8a8d Use nix::connect() to connect to the garbage collector 2021-10-28 22:51:11 +02:00
Eelco Dolstra 22c35ea5b8 Remove unused variable 2021-10-28 22:51:11 +02:00
Eelco Dolstra a7d4f3411e Merge remote-tracking branch 'origin/master' into non-blocking-gc 2021-10-28 14:56:55 +02:00
Eelco Dolstra 5a160171d0 Remove redundant 'warning:' 2021-10-27 18:14:12 +02:00
Eelco Dolstra 1254e8753c build-remote: Implicitly add the 'builtin' system type to all machines
This makes 'nix-env -i --max-jobs 0' work with remote builders.
2021-10-27 14:25:13 +02:00
Eelco Dolstra f2280749b1 If max-jobs == 0, do preferLocalBuild on remote builders 2021-10-27 14:21:31 +02:00
Eelco Dolstra 6e684d1b87 daemon: Accept 'repeat' setting from untrusted users
Fixes #5352.
2021-10-27 13:09:13 +02:00
Eelco Dolstra 13a7a24ba5 Style 2021-10-27 13:02:37 +02:00
Eelco Dolstra 0d9e050ba7 parseExperimentalFeature(): Initialize atomically 2021-10-26 14:29:48 +02:00
Eelco Dolstra 9ce84c64c5 Tweak fetchTree docs 2021-10-26 14:21:24 +02:00
regnat af99941279 Make experimental-features a proper type
Rather than having them plain strings scattered through the whole
codebase, create an enum containing all the known experimental features.

This means that
- Nix can now `warn` when an unkwown experimental feature is passed
  (making it much nicer to spot typos and spot deprecated features)
- It’s now easy to remove a feature altogether (once the feature isn’t
  experimental anymore or is dropped) by just removing the field for the
  enum and letting the compiler point us to all the now invalid usages
  of it.
2021-10-26 07:02:31 +02:00
Kevin Amado 823dce945a
fetch: nicer infinite recursion errors
- This change applies to builtins.fetchurl and builtins.fetchTarball
- PoC: `let x = builtins.fetchurl x; in x`
- Before:
  ```bash
  $ nix-instantiate --extra-experimental-features flakes --strict
  error: infinite recursion encountered
  ```
- After:
  ```bash
  $ nix-instantiate --extra-experimental-features flakes --strict
  error: infinite recursion encountered

       at /data/github/kamadorueda/nix/test.nix:1:9:

            1| let x = builtins.fetchurl x; in x
             |         ^
  ```

Mentions: #3505
2021-10-17 12:54:53 -05:00
Alexey Novikov e989c83b44 Add error reporting to machine spec paser
Currently machine specification (`/etc/nix/machine`) parser fails
with a vague exception if the file had incorrect format.
This commit adds verbose exceptions and unit-tests for the parser.
2021-10-17 12:45:56 +04:00
Alexey Novikov 64a3b045c1 Fix error detection in 'base64Decode()'
Fixed a bug in initialization of 'base64DecodeChars' variable.
Currently decoder do not fail on invalid Base64 strings.
Added test-case to verify the fix.

Also have made 'base64DecodeChars' to be computed at compile time.
And added a test case to encode/decode string with non-printable charactes.
2021-10-17 12:45:26 +04:00
Kevin Amado e5a27a3b4e
fetchTree: add pos to EvalState::forceValue
- This way we improve error messages
  on infinite recursion
- Demo:
  ```nix
  let x = builtins.fetchTree {
    type = "git";
    inherit x;
  };
  in x
  ```
- Before:
  ```bash
  $ nix-instantiate --extra-experimental-features flakes --strict
  error: infinite recursion encountered
  ```
- After:
  ```bash
  $ nix-instantiate --extra-experimental-features flakes --strict
  error: infinite recursion encountered

       at /data/github/kamadorueda/nix/test.nix:3:10:

            2|   type = "git";
            3|   inherit x;
             |          ^
            4| };
  ```

Mentions: #3505
2021-10-15 19:25:19 -05:00
Kevin Amado 18e3d63341
fetchTree: add pos to EvalState::forceValue
- This way we improve error messages
  on infinite recursion
- Demo:
  ```nix
  let x = builtins.fetchTree x;
  in x
  ```
- Before:
  ```bash
  $ nix-instantiate --extra-experimental-features flakes --strict
  error: infinite recursion encountered
  ```
- After:
  ```bash
  $ nix-instantiate --extra-experimental-features flakes --strict
  error: infinite recursion encountered

       at /data/github/kamadorueda/nix/test.nix:1:9:

            1| let x = builtins.fetchTree x;
             |         ^
            2| in x
  ```

Mentions: #3505
2021-10-15 19:25:19 -05:00
Eelco Dolstra 10f9a8e77d Add a test for the non-blocking GC 2021-10-15 16:52:37 +02:00
Eelco Dolstra ac54c6faa6 Fix main GC thread exiting 2021-10-15 16:36:48 +02:00
Eelco Dolstra e0936ae38f Fix crash when a GC client disconnects
The client thread can't just delete its own thread object from
connections, it has to detach it.
2021-10-15 16:12:21 +02:00
Eelco Dolstra 304180d0de Memoize queryReferrers() 2021-10-15 12:20:23 +02:00
Eelco Dolstra c0951299b3
Merge pull request #5383 from kvtb/patch-7
fix build with gcc11
2021-10-15 09:03:12 +02:00
Eelco Dolstra c574ab3907
Merge pull request #5388 from yvt/fix-oahd-path
Add another path where a Rosetta 2 daemon plist file is possibly located
2021-10-15 09:02:16 +02:00
Kevin Amado 1bdeef8395
add pos to EvalState::forceValue
- This way we improve error messages
  on infinite recursion
- Demo:
  ```nix
  let
    x = builtins.fetchMercurial x;
  in
  x
  ```
- Before:
  ```bash
  $ nix-instantiate --show-trace --strict
  error: infinite recursion encountered
  ```
- After:
  ```bash
  nix-instantiate --show-trace --strict
  error: infinite recursion encountered

       at /data/github/kamadorueda/test/default.nix:2:7:

            1| let
            2|   x = builtins.fetchMercurial x;
             |       ^
            3| in
  ```

Mentions: #3505
2021-10-14 23:23:05 -05:00
yvt a9d9e55551 Add another path where a Rosetta 2 configuration file is possibly located 2021-10-15 09:48:15 +09:00
Eelco Dolstra 17e6ebcc90 Speed up GC by marking entire closures as live 2021-10-14 14:13:57 +02:00
Eelco Dolstra 0154fa30cf Remove GCState 2021-10-14 13:52:49 +02:00
Eelco Dolstra 0317ffdad3 Move deleteFromStore() 2021-10-14 13:34:48 +02:00
Eelco Dolstra 0be8cc1466 pathInfoCache: Use the entire base name as the cache key
This fixes a bug in the garbage collector where if a path
/nix/store/abcd-foo is valid, but we do a
isValidPath("/nix/store/abcd-foo.lock") first, then a negative entry
for /nix/store/abcd is added to pathInfoCache, so /nix/store/abcd-foo
is subsequently considered invalid and deleted.
2021-10-14 13:28:22 +02:00
Eelco Dolstra eab934cb2a Make the canReachRoots() traversal non-recursive 2021-10-14 12:34:32 +02:00
Eelco Dolstra 09b14ea97a Cleanup 2021-10-14 10:04:13 +02:00
kvtb eae29b0385
fix build with gcc11 2021-10-13 18:03:33 +00:00
Eelco Dolstra 35c98a59c5 Fix GC when there are cycles in the referrers graph
(where "referrers" includes the reverse of derivation outputs and
derivers). Now we do a full traversal to look if we can reach any
root. If not, all paths reached can be deleted.
2021-10-13 12:12:44 +02:00
Eelco Dolstra 1785ba2980 Simplify 2021-10-13 12:12:44 +02:00
Eelco Dolstra dced45f146 strcpy -> memcpy
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
2021-10-13 12:12:44 +02:00
Eelco Dolstra 262520fcfe Use a thread per connection 2021-10-13 12:12:44 +02:00
Eelco Dolstra ff453b06f9 Fix auto-gc 2021-10-13 12:12:44 +02:00
Eelco Dolstra 8614cf1334 Non-blocking garbage collector
The garbage collector no longer blocks other processes from
adding/building store paths or adding GC roots. To prevent the
collector from deleting store paths just added by another process,
processes need to connect to the garbage collector via a Unix domain
socket to register new temporary roots.
2021-10-13 12:12:44 +02:00
Eelco Dolstra 9947f1646a Remove syncWithGC() 2021-10-13 12:12:44 +02:00
Eelco Dolstra 8eac7dfad4 Remove trash directory 2021-10-13 12:12:44 +02:00
Eelco Dolstra 624dfde3df
Merge pull request #5362 from Artturin/nixunpack
nix develop: add --unpack
2021-10-13 11:39:08 +02:00
Eelco Dolstra 0fac86fd6f Style tweaks 2021-10-13 11:00:10 +02:00
Eelco Dolstra abd685d373 Merge branch 'feature/comp-level' of https://github.com/tomberek/nix 2021-10-13 10:45:44 +02:00
Eelco Dolstra 3e0c6aac9a
Merge pull request #5375 from edolstra/repl-ctrl-c
nix repl: Fix plugin-files warning
2021-10-12 17:11:39 +02:00
Eelco Dolstra 5176b072ed Cleanup 2021-10-12 16:43:00 +02:00
Eelco Dolstra 3a778ea8a0 Merge branch 'nix-repl-download-interruption' of https://github.com/Ma27/nix 2021-10-12 16:34:38 +02:00