alois31/nix-eval-jobs
0
0
Fork 0
forked from lix-project/nix-eval-jobs
Code Pull requests Activity
610 commits 2 branches 0 tags 710 KiB
Commit graph

118 commits

Author SHA1 Message Date
adisbladis 73ee1712fa
Merge pull request #207 from RaitoBezarius/input-drvs 
feat: add inputDrvs to the JSON
 2023-04-26 15:36:28 +12:00
raito 469a68d5c1 feat: add inputDrvs to the JSON 
Currently, not a lot of things expose inputDrvs (except
`show-derivation`), which is a showstopper whenever you want to compute
popularity ranking based on the dependency relation.

Having `inputsDrvs` in the reply enable downstream users to perform such
computations in an efficient way.
 2023-04-24 19:35:45 +02:00
Julien Malka 4d674c352e
fix recurseForDerivations evaluation in force-recurse mode 2023-04-24 17:17:47 +02:00
Jörg Thalheim e9c301bcbe document the worker/memory tradeoff 2023-04-13 23:23:51 +02:00
Jörg Thalheim 54a9b26431 fix build with nix 2.14 2023-03-09 22:21:59 +01:00
Jörg Thalheim 205f05ac02 fix build with 2.13 2023-01-22 11:19:24 +01:00
Jörg Thalheim 9d516a72d7 fix double-free caused by usage of shared pointer 2023-01-22 10:37:17 +01:00
Tom Bereknyei 7110a1a6c7 fix: catch errors for invalid derivations 2023-01-02 08:06:06 +01:00
Timothy DeHerrera c8b52f5836
fix: use InstallableFlake type & methods 
Fixes #134

Use the `InstallableFlake` type in order to make use of it's `toValue`
method. This fixes the functor auto-call by including the work from
nixos/nix#6404.

Future work may make use of this object and its methods to employ the
flake based eval cache.
 2022-11-04 09:37:54 -06:00
adisbladis 1af1893abe
Merge pull request #130 from nix-community/eval-store 
Respect passed eval store
 2022-10-18 14:51:11 +13:00
adisbladis e0c27dc984 Respect passed eval store 2022-10-18 14:46:37 +13:00
adisbladis b9836f2e06 Add parameter to force recursion (don't honour recurseIntoAttrs) 
This can be useful when you are not in control over your expressions.

My use case is evaluating a Hydra jobset.
 2022-10-18 14:42:26 +13:00
Jörg Thalheim 47a74fed83 also accept relative gc root directories 2022-09-20 20:12:55 +02:00
Jörg Thalheim 70afaeebca avoid shadowing metaName 2022-09-17 10:48:25 +02:00
Jörg Thalheim cb8126538f fix recurseIntoAttrs handling 2022-09-15 11:04:12 +02:00
Jörg Thalheim f6890b93e8 add check-cache-status option to query wether a binary cache has the build 2022-09-15 11:04:12 +02:00
Zhaofeng Li 23f4dfdc24 Add support for passing a Nix expression on the command line 
Fixes #38.
 2022-08-17 10:59:44 -06:00
Zhaofeng Li 9f64612215 Don't override evalSettings.pureEval unless necessary 
Other eval flags like `--pure-eval` may activate `evalSettings.pureEval`,
so let's avoid touching it if we don't have to.
 2022-08-17 00:53:08 -06:00
Jörg Thalheim 882d2376cf
fixes for nixUnstable 2022-05-09 09:50:48 +02:00
John Soo efca7105c7 Silence more harmless warnings. 2022-04-27 04:08:01 +12:00
adisbladis fec10f2cfc src/nix-eval-jobs.cc: Reformat with clang-format 2022-04-26 21:20:24 +12:00
adisbladis 61c9f4cfcc Replace internal attr string representation with an array 
This ensures correct handling of attrnames with a dot in them and will
not throw errors about illegal attrnames.

Additionally this escapes any attributes containing dots in the JSON
output and adds another field called `attrPath` which contains the
attribute path as a list.

Example output:
```
{
  "attr": "hello",
  "attrPath": [
    "hello"
  ],
  "drvPath": "/nix/store/n204jib73z55cp9s0rmw1c5v5q528j7v-hello-2.12.drv",
  "name": "hello-2.12",
  "outputs": {
    "out": "/nix/store/h59dfk7dwrn7d2csykh9z9xm2miqmrnz-hello-2.12"
  },
  "system": "x86_64-linux"
}
```
 2022-04-26 19:43:56 +12:00
adisbladis c1bbb11c5d Add support for recurseForDerivations 
This will respect `recurseForDerivations` when iterating over attrsets.

Example expression:
``` nix
{ system ? builtins.currentSystem }:
{
  recurseForDerivations = true;

  # This should build as it's in the top-level attrset
  drvA = derivation {
    inherit system;
    name = "drvA";
    builder = ":";
  };

  dontRecurse = {
    # This shouldn't build as `recurseForDerivations = true;` is not set
    # recurseForDerivations = true;

    # This should not build
    drvB = derivation {
      inherit system;
      name = "drvA";
      builder = ":";
    };
  };

  recurse = {
    # This should build
    recurseForDerivations = true;

    # This should not build
    drvC = derivation {
      inherit system;
      name = "drvC";
      builder = ":";
    };
  };

}
```
 2022-04-25 22:11:53 +12:00
adisbladis 1ea8948eb7 Fix build with stable Nix 2022-04-24 14:08:34 +12:00
John Soo f98da2e00b
Cleanup worker json handling. 2022-04-22 00:36:54 -07:00
John Soo 7aa9835adb
Cleanup gcroots creation. 2022-04-22 00:36:13 -07:00
John Soo 8dd6426d15
Make Drv struct. 
To facilitate collecting and jsonifying derivations in different
contexts
 2022-04-22 00:35:30 -07:00
adisbladis 5f9d57e9b1
Merge pull request #55 from jsoo1/kill-zombies 
Kill zombies.
 2022-04-22 14:33:16 +07:00
John Soo b9a87464a0
Move collecting handler to separate function. 
s/master/collector/g
 2022-04-22 00:15:28 -07:00
John Soo a27faabd0a
Use Proc struct to manage forked processes. 
Cleans up zombie processes.
 2022-04-22 00:15:28 -07:00
John Soo 9d4c256fa0
Make a Proc struct for running processes. 2022-04-22 00:15:27 -07:00
adisbladis 353724df90 Fix querying output paths 
The upstream function signature changed, we no longer have to pass onlyOutputsToInstall.
 2022-04-22 19:07:40 +12:00
Jörg Thalheim c1b86245a1 fix for nixUnstable 2022-04-22 18:53:12 +12:00
adisbladis a07a22f94c
Merge pull request #53 from jsoo1/silence-irrelevant-warnings 
Ignore -Wnon-virtual-dtor for static struct.
 2022-04-22 13:51:36 +07:00
adisbladis 55d91140be
Merge pull request #57 from jsoo1/refactor-top-level-value-finding 
Refactor top-level value finding.
 2022-04-22 13:29:26 +07:00
John Soo 0e0c6c44d0
Ignore -Wnon-virtual-dtor for static struct. 2022-04-21 15:44:01 -07:00
John Soo dd8a2e89a0
Simplify top-level value fetching. 2022-04-21 11:44:02 -07:00
John Soo e05e625411
Make function to get top-level value from a flake. 2022-04-21 11:40:00 -07:00
John Soo 4a3d2e0008
Make function to get top-level value from releaseExpr. 2022-04-21 11:34:49 -07:00
John Soo b062ac705d
Set GC_DONT_GC=1. 
To avoid `Collecting from unknown thread'.
 2022-04-21 09:31:53 -07:00
Jörg Thalheim d592323c6b fix build with nix 2.7 2022-03-13 17:47:15 +01:00
Zhaofeng Li 6d61193286 Flush cout after each output line 
The default buffering behavior depends on whether the output is connected
to an interactive device. This causes output lines to be buffered in an
undesirable way when stdout is piped, which is how nix-eval-jobs is
normally used. Let's fix it by flushing stdout explicitly.
 2022-02-16 00:46:57 -08:00
Zhaofeng Li 1e0f309fef Add flag to enable trace output 2022-01-07 00:01:07 -08:00
adisbladis 3268f3a6af Add flag to enable meta 
I removed meta from the output in 434376f8e1 with the intention of adding it back gated by a flag, but that never happened.

Adding meta is quite a substantial increase in output size and has some non-trivial performance impact at scale, so it's best to leave it as opt-in.
 2022-01-06 13:40:21 +13:00
Tom Bereknyei d36d77c873 use outPath 2022-01-04 14:16:36 -05:00
Jörg Thalheim 26948992b3 also expose store path in json 2021-12-31 12:00:04 +01:00
Jörg Thalheim c297bd9564 fix build against nixUnstable 2021-12-15 14:20:38 +01:00
adisbladis 0648bc0cb8
Select flake output by the flake fragment 2021-08-26 18:48:56 -05:00
adisbladis 434376f8e1
Remove outputs from JSON output 
The output of the evaluator should only include either the full
derivation (not yet implemented) or fields not directly accessible
from the drv such as meta.

Right now the output is a fairly arbitrary selection of fields.
 2021-08-26 18:48:56 -05:00
adisbladis a6171c9a62
Switch to streaming output (using line delimited JSON) 2021-08-25 13:58:42 -05:00
adisbladis 4c28ae88a6
Skip non-serialisable *Value in output 
Most notably this includes derivations which should be fixed and
serialised to store paths.
 2021-08-25 03:12:19 -05:00
adisbladis be59cd8bfb
Remove unused output 2021-08-25 03:12:03 -05:00
adisbladis f8dad73ac8
Move outputs definition closer to use 2021-08-25 03:11:16 -05:00
adisbladis a4441fe1c8
Remove support for Hydra aggregate jobs 
This might be a somewhat useful feature for Hydra, but not for a generic Nix evaluator.
 2021-08-24 14:06:32 -05:00
adisbladis 174e735183
Rename project to nix-eval-jobs 
We are getting rid of Hydra specific hacks and making this a more
generically useful component for any projects that need to run
large evaluations.
 2021-08-24 14:06:27 -05:00
adisbladis 1774f874ee
Remove superflous meta fields from output 
Since meta is now included in it's entirety we no longer need to output them separately.
 2021-08-23 18:38:46 -05:00
adisbladis 19823c899d
Add full meta to output 
While the current output may be sufficient for Hydra it's not enough
for a more generically useful evaluator.
 2021-08-23 18:33:19 -05:00
Jörg Thalheim 71cbe4eab4
output missing/substitutable derivations in eval output 2021-03-21 19:05:11 +01:00
Jörg Thalheim 617d4ee151
add --impure flag for flakes 2021-03-21 18:39:32 +01:00
Jörg Thalheim f00607a7c6
exit hydra-eval-jobs after printing help 2021-03-21 18:38:56 +01:00
Jörg Thalheim b48e9667dc
make gcRootsDir local 
just to avoid unecessary globals
 2021-03-21 18:37:18 +01:00
Jörg Thalheim 25c46dc1b0
fix evaluating relative flake urls 2021-03-15 06:02:14 +01:00
Jörg Thalheim 7e41287c1b
disable restrictEval again 2021-03-14 23:32:36 +01:00
Jörg Thalheim 0e1be9fdaf
hydra-eval-jobs: smaller visual improvements 2021-03-14 23:29:49 +01:00
Jörg Thalheim 034f780421
remove unused headers 2021-03-14 23:19:18 +01:00
Jörg Thalheim ac566dea95
update from upstream 2021-03-14 23:16:57 +01:00
Jörg Thalheim e2a3c95179
prefix header files with nix/ 2020-11-29 21:32:04 +01:00
Jörg Thalheim 5ed5476058
add project 2020-11-29 15:33:55 +01:00