Commit graph

208 commits

Author SHA1 Message Date
Eelco Dolstra 3da9a9241c Convert option descriptions to Markdown 2021-01-13 14:18:04 +01:00
Eelco Dolstra 93ad6430ed nix store prefetch-tarball -> nix flake prefetch 2021-01-11 12:36:39 +01:00
Eelco Dolstra c9279b831e
Add 'nix flake' manpages 2020-12-23 18:26:40 +01:00
Silvan Mosberger 12e65078ef
Rename Value::normalType() -> Value::type() 2020-12-17 14:45:45 +01:00
Silvan Mosberger bf98903967
Add ValueType checking functions for types that have the same NormalType 2020-12-12 03:31:50 +01:00
Silvan Mosberger 22ead43a0b
Use Value::normalType on all forced values instead of Value::type 2020-12-12 03:31:48 +01:00
Matthew Kenigsberg 6a4bf535d8 Capitalize JSON for consistency 2020-10-28 17:54:28 -05:00
Eelco Dolstra 9d5e9ef0da Move Explicit 2020-10-26 17:01:20 +01:00
John Ericson 39de73550d Merge remote-tracking branch 'upstream/master' into fix-and-ci-static-builds 2020-10-09 18:26:47 +00:00
Eelco Dolstra 85c8be6286 Remove static variable name clashes
This was useful for an experiment with building Nix as a single
compilation unit. It's not very useful otherwise but also doesn't
hurt...
2020-10-06 13:49:20 +02:00
John Ericson cfe791a638 stdout_ -> cout
Better to get creative than just sprinkle arbitrary underscores.
2020-09-25 11:30:04 -04:00
John Ericson 25f7ff16fa Merge remote-tracking branch 'upstream/master' into fix-and-ci-static-builds 2020-09-04 02:40:36 +00:00
Eelco Dolstra a72a20d68f
Add 'nix dump-args' to dump all commands/flags for manpage generation 2020-08-17 17:44:52 +02:00
John Ericson 39ae9a3d4a Merge remote-tracking branch 'upstream/master' into fix-and-ci-static-builds 2020-08-07 14:44:47 +00:00
Eelco Dolstra 2ffc058950 Make --no-eval-cache a global setting 2020-08-07 14:13:24 +02:00
Matthew Bauer fa2d1fb36e Pass system to bundler 2020-07-30 15:37:05 -05:00
Matthew Bauer 52407f83a1 exporter -> bundler 2020-07-30 15:16:23 -05:00
Matthew Bauer d7ffe327ae Merge remote-tracking branch 'origin/master' into fix-and-ci-static-builds 2020-07-30 14:59:57 -05:00
Matthew Bauer 5d04a4db9b Handle exporters checking correctly 2020-07-30 11:45:47 -05:00
Matthew Bauer 2f4250a416 Add "export" to Nix
This adds a ‘nix export’ command which hooks into nix-bundle. It can
be used in a similar way as nix-bundle, with the benefit of hooking
into the new “app” functionality. For instance,

$ nix export nixpkgs#jq
$ ./jq --help
jq - commandline JSON processor [version 1.6]
...
$ scp jq machine-without-nix:
$ ssh machine-without-nix ./jq
jq - commandline JSON processor [version 1.6]
...

Note that nix-bundle currently requires Linux to run. Other exporters
might not have that requirement.

“exporters” are meant to be reusable, so that, other repos can
implement their own bundling.

Fixes #3705
2020-07-30 11:33:22 -05:00
Eelco Dolstra 54712aaf8a Merge remote-tracking branch 'origin/master' into flakes 2020-07-06 16:40:10 +02:00
Eelco Dolstra 26cf0c674f nix run: Use packages/legacyPackages as fallback if there is no app definition
'nix run' will try to run $out/bin/<name>, where <name> is the
derivation name (excluding the version). This often works well:

  $ nix run nixpkgs#hello
  Hello, world!

  $ nix run nix -- --version
  nix (Nix) 2.4pre20200626_adf2fbb

  $ nix run patchelf -- --version
  patchelf 0.11.20200623.e61654b

  $ nix run nixpkgs#firefox -- --version
  Mozilla Firefox 77.0.1

  $ nix run nixpkgs#gimp -- --version
  GNU Image Manipulation Program version 2.10.14

though not always:

  $ nix run nixpkgs#git
  error: unable to execute '/nix/store/kp7wp760l4gryq9s36x481b2x4rfklcy-git-2.25.4/bin/git-minimal': No such file or directory
2020-06-29 19:08:50 +02:00
Eelco Dolstra 09fc06daab nix flake init: Use git add --force 2020-06-23 16:25:32 +02:00
Eelco Dolstra 334e26bfc2 nix flake check: Don't build apps
This was inconsistent since we're not building 'packages' or
'defaultPackage' either.

Closes #3726.
2020-06-22 11:31:07 +02:00
Eelco Dolstra 1524752c17 Merge remote-tracking branch 'origin/master' into flakes 2020-06-17 10:26:52 +02:00
Eelco Dolstra 0c62b4ad0f Represent 'follows' inputs explicitly in the lock file
This fixes an issue where lockfile generation was not idempotent:
after updating a lockfile, a "follows" node would end up pointing to a
new copy of the node, rather than to the original node.
2020-06-11 14:40:21 +02:00
Eelco Dolstra e073f2c584 nix flake: Require 'flakes' feature 2020-06-08 16:23:54 +02:00
Eelco Dolstra 6470450ab4 Add completion for --update-input 2020-06-08 16:20:00 +02:00
Eelco Dolstra 488ff83e6b Fix completion of --template 2020-06-05 14:09:12 +02:00
Eelco Dolstra 810b2c6a48 nix flake init: Add a '--template' flag
The initial contents of the flake is specified by the
'templates.<name>' or 'defaultTemplate' output of another flake. E.g.

  outputs = { self }: {

    templates = {

      nixos-container = {
        path = ./nixos-container;
        description = "An example of a NixOS container";
      };

    };

  };

allows

  $ nix flake init -t templates#nixos-container

Also add a command 'nix flake new', which is identical to 'nix flake
init' except that it initializes a specified directory rather than the
current directory.
2020-06-04 20:22:25 +02:00
Eelco Dolstra 950b46821f Remove TreeInfo
The attributes previously stored in TreeInfo (narHash, revCount,
lastModified) are now stored in Input. This makes it less arbitrary
what attributes are stored where.

As a result, the lock file format has changed. An entry like

    "info": {
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },
    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github"
    },

is now stored as

    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github",
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },

The 'Input' class is now a dumb set of attributes. All the fetcher
implementations subclass InputScheme, not Input. This simplifies the
API.

Also, fix substitution of flake inputs. This was broken since lazy
flake fetching started using fetchTree internally.
2020-05-30 00:44:11 +02:00
Matthew Kenigsberg 8d67794da1 handle circular flake dependencies in list-inputs 2020-05-21 17:06:11 -06:00
Eelco Dolstra 5f64655ff4 Move registry-related commands from 'nix flake' to 'nix registry'
This makes 'nix flake' less cluttered and more consistent (it's only
subcommands that operator on a flake). Also, the registry is not
inherently flake-related (e.g. fetchTree could also use it to remap
inputs).
2020-05-15 14:38:10 +02:00
Eelco Dolstra 649c2db308 nix flake: Add completion support 2020-05-11 22:10:33 +02:00
Eelco Dolstra 6f3244ce45 Merge remote-tracking branch 'origin/master' into flakes 2020-05-05 18:59:33 +02:00
Eelco Dolstra 6521c92ce8 Improve path:// handling
In particular, doing 'nix build /path/to/dir' now works if
/path/to/dir is not a Git tree (it only has to contain a flake.nix
file).

Also, 'nix flake init' no longer requires a Git tree (but it will do a
'git add flake.nix' if it's a Git tree)
2020-04-27 22:53:11 +02:00
Eelco Dolstra b69323f8c9 Revive 'nix search'
It uses the evaluation cache now rather than the ad hoc JSON cache.
2020-04-20 15:27:09 +02:00
Eelco Dolstra 42a12f9232 Move eval-cache.{cc,hh} 2020-04-20 13:14:59 +02:00
Eelco Dolstra 539a9c1c5f Get rid of the old eval cache 2020-04-20 13:13:52 +02:00
Eelco Dolstra 0725ab2fd7 Store more stuff in the evaluation cache
In particular, we store whether an attribute failed to evaluate (threw
an exception) or was an unsupported type. This is to ensure that a
repeated 'nix flake show' never has to evaluate anything, so it can
execute without fetching the flake.

With this, 'nix flake show nixpkgs/nixos-20.03 --legacy' executes in
0.6s (was 3.4s).
2020-04-19 23:07:06 +02:00
Eelco Dolstra 3738bcb05e Eval cache: Don't replace real attributes with placeholders 2020-04-18 15:12:31 +02:00
Eelco Dolstra 69cb9f7eee Wrap eval cache creation in a giant transaction
This speeds up the creation of the cache for the nixpkgs flake from
21.2s to 10.2s. Oddly, it also speeds up querying the cache
(i.e. running 'nix flake show nixpkgs/nixos-20.03 --legacy') from 4.2s
to 3.4s.

(For comparison, running with --no-eval-cache takes 9.5s, so the
overhead of building the SQLite cache is only 0.7s.)
2020-04-17 23:17:21 +02:00
Eelco Dolstra aaa109565e Use a more space/time-efficient representation for the eval cache 2020-04-17 23:04:21 +02:00
Eelco Dolstra bdb3226607 Add flag to disable the eval cache 2020-04-17 14:30:04 +02:00
Eelco Dolstra aa34c0ef51 nix flake show: Speed up eval cache bigly
In the fully cached case for the 'nixpkgs' flake, it went from 101s to
4.6s. Populating the cache went from 132s to 17.4s (which could
probably be improved further by combining INSERTs).
2020-04-17 13:57:02 +02:00
Eelco Dolstra 9ea4f93f88 nix flake show: Support apps 2020-04-17 01:21:24 +02:00
Eelco Dolstra 12b7eefbc5 nix flake show: Use evaluation cache 2020-04-17 01:02:29 +02:00
Eelco Dolstra 3b489e8843 Add 'nix flake show' command 2020-04-16 19:52:39 +02:00
Eelco Dolstra 29043e7e9e Fix 2020-04-16 19:01:49 +02:00
Eelco Dolstra 8f41847394 Use Logger::stdout() 2020-04-16 13:47:59 +02:00
Eelco Dolstra e5ea01c1a8
Remove flake 'edition' field
Future editions of flakes or the Nix language can be supported by
renaming flake.nix (e.g. flake-v2.nix). This avoids a bootstrap
problem where we don't know which grammar to use to parse
flake*.nix. It also allows a project to support multiple flake
editions, in theory.
2020-04-10 10:24:09 +02:00
Eelco Dolstra 68b43e01dd
nix flake info: Show resolved URL
This is useful for finding out what a registry lookup resolves to, e.g

  $ nix flake info patchelf
  Resolved URL:  github:NixOS/patchelf
  Locked URL:    github:NixOS/patchelf/cd7955af31698c571c30b7a0f78e59fd624d0229
2020-04-06 14:56:13 +02:00
Eelco Dolstra ce3173edc1
nix flake info --json: Don't evaluate
This makes its behaviour consistent with the non-json
variant. Querying the outputs should be done by another command
(e.g. 'nix search')
2020-04-06 14:39:47 +02:00
Eelco Dolstra ed13457dbf nix flake info --json: Show TreeInfo 2020-04-02 11:51:34 +02:00
Eelco Dolstra bd10a07d17 Registry: Use a struct instead of a tuple for entries 2020-04-01 23:03:27 +02:00
Eelco Dolstra 77ffaea4fa Add a system-wide flake registry /etc/nix/registry.json
One application for this is pinning the 'nixpkgs' flake to the exact
revision used to build the NixOS system, e.g.

  {
      "flakes": [
          {
              "from": {
                  "id": "nixpkgs",
                  "type": "indirect"
              },
              "to": {
                  "owner": "NixOS",
                  "repo": "nixpkgs",
                  "type": "github",
                  "rev": "b0c285807d6a9f1b7562ec417c24fa1a30ecc31a"
              }
          }
      ],
      "version": 2
  }
2020-04-01 22:56:50 +02:00
Eelco Dolstra e0a0ae0467 Move fetchers from libstore to libfetchers 2020-03-30 14:04:53 +02:00
Eelco Dolstra 4989c04dd2 nix flake info --json: Dump attr-style flakeref 2020-03-28 22:59:38 +01:00
Eelco Dolstra 2287e2f279 nix flake info: Show flake subdirectory 2020-03-28 18:05:50 +01:00
Eelco Dolstra 2a4e4f6a6e
Unified fetcher caching system 2020-03-17 22:35:29 +01:00
Eelco Dolstra ae9119167e Change the lock file to a graph
This enables support for cycles between flakes.
2020-03-12 22:06:57 +01:00
Eelco Dolstra 73c9840569 Restore subdir -> dir
Got this mixed up somewhere.
2020-02-20 23:44:06 +01:00
Eelco Dolstra d068f9ffff Restore subdir support in registries
Hacky...
2020-02-20 22:14:44 +01:00
Eelco Dolstra 46a284263f Fix build 2020-02-14 22:45:33 +01:00
Eelco Dolstra 0b013a54dc findAlongAttrPath(): Return position 2020-02-07 14:08:24 +01:00
Eelco Dolstra b270869466
Renamed ref / resolvedRef -> lockedRef 2020-02-02 00:06:59 +01:00
Eelco Dolstra 887730aab3
Remove superfluous TreeInfo::rev field 2020-02-01 23:54:20 +01:00
Eelco Dolstra 8451298b35
Factor out TreeInfo 2020-02-01 22:06:55 +01:00
Eelco Dolstra 5d70b454be
nix flake update: Imply --refresh 2020-02-01 12:30:49 +01:00
Eelco Dolstra 678301072f nix flake list-inputs: Pretty-print the tree 2020-01-31 14:09:27 +01:00
Eelco Dolstra a6e2b6b360 nix flake deps -> nix flake list-inputs
Also add a --json flag.
2020-01-31 13:00:50 +01:00
Eelco Dolstra ebfbfe9515 Use std::nullopt 2020-01-30 01:10:26 +01:00
Eelco Dolstra 3c54e9ba01 Add 'nix flake archive' command
This copies a flake and all its inputs recursively to a store (e.g. a
binary cache). This is intended to enable long-term reproducibility
for flakes. However this will also require #3253.

Example:

  $ nix flake archive --json --to file:///tmp/my-cache nixops
  {"path":"/nix/store/272igzkgl1gdzmabsjvb2kb2zqbphb3p-source","inputs":{"nixops-aws":{"path":"/nix/store/ybcykw13gr7iq1pzg18iyibbcv8k9q1v-source","inputs":{}},"nixops-hetzner":{"path":"/nix/store/6yn0205x3nz55w8ms3335p2841javz2d-source","inputs":{}},"nixpkgs":{"path":"/nix/store/li3lkr2ajrzphqqz3jj2avndnyd3i5lc-source","inputs":{}}}}

  $ ll /tmp/my-cache
  total 16
  -rw-r--r-- 1 eelco users 403 Jan 30 01:01 272igzkgl1gdzmabsjvb2kb2zqbphb3p.narinfo
  -rw-r--r-- 1 eelco users 403 Jan 30 01:01 6yn0205x3nz55w8ms3335p2841javz2d.narinfo
  -rw-r--r-- 1 eelco users 408 Jan 30 01:01 li3lkr2ajrzphqqz3jj2avndnyd3i5lc.narinfo
  drwxr-xr-x 2 eelco users   6 Jan 30 01:01 nar
  -rw-r--r-- 1 eelco users  21 Jan 30 01:01 nix-cache-info
  -rw-r--r-- 1 eelco users 404 Jan 30 01:01 ybcykw13gr7iq1pzg18iyibbcv8k9q1v.narinfo

Fixes #3336.
2020-01-30 01:00:34 +01:00
Eelco Dolstra 26f895a26d Clean up the lock file handling flags
Added a flag --no-update-lock-file to barf if the lock file needs any
changes. This is useful for CI systems if you're building a
checkout. Fixes #2947.

Renamed --no-save-lock-file to --no-write-lock-file. It is now a fatal
error if the lock file needs changes but --no-write-lock-file is not
given.
2020-01-29 21:04:28 +01:00
Eelco Dolstra f68bed7f67 Add flag --override-input to override specific lock file entries
E.g.

  $ nix flake update ~/Misc/eelco-configurations/hagbard \
    --override-input 'dwarffs/nixpkgs' ../my-nixpkgs

overrides the 'nixpkgs' input of the 'dwarffs' input of the top-level
flake.

Fixes #2837.
2020-01-29 18:41:25 +01:00
Eelco Dolstra 872a22fa23 resolveFlake -> lockFlake
"resolve" is ambiguous (also used for registry resolution).
2020-01-22 20:59:59 +01:00
Eelco Dolstra 90d55ed275 Fix 'nix flake update' 2020-01-22 19:28:27 +01:00
Eelco Dolstra 32f31a8c63 nix flake info: Don't show empty descriptions 2020-01-22 17:20:21 +01:00
Eelco Dolstra 9f4d8c6170 Pluggable fetchers
Flakes are now fetched using an extensible mechanism. Also lots of
other flake cleanups.
2020-01-21 22:56:04 +01:00
Eelco Dolstra ecb3a1afa2 Merge remote-tracking branch 'origin/master' into flakes 2019-12-11 14:53:30 +01:00
Eelco Dolstra ce27920936 Add start of 'nix profile' command 2019-10-22 00:22:15 +02:00
Eelco Dolstra 7d38060a0d Support non-x86_64-linux system types in flakes
A command like

  $ nix run nixpkgs#hello

will now build the attribute 'packages.${system}.hello' rather than
'packages.hello'. Note that this does mean that the flake needs to
export an attribute for every system type it supports, and you can't
build on unsupported systems. So 'packages' typically looks like this:

  packages = nixpkgs.lib.genAttrs ["x86_64-linux" "i686-linux"] (system: {
    hello = ...;
  });

The 'checks', 'defaultPackage', 'devShell', 'apps' and 'defaultApp'
outputs similarly are now attrsets that map system types to
derivations/apps. 'nix flake check' checks that the derivations for
all platforms evaluate correctly, but only builds the derivations in
'checks.${system}'.

Fixes #2861. (That issue also talks about access to ~/.config/nixpkgs
and --arg, but I think it's reasonable to say that flakes shouldn't
support those.)

The alternative to attribute selection is to pass the system type as
an argument to the flake's 'outputs' function, e.g. 'outputs = { self,
nixpkgs, system }: ...'. However, that approach would be at odds with
hermetic evaluation and make it impossible to enumerate the packages
provided by a flake.
2019-10-15 18:16:29 +02:00
Eelco Dolstra a0bd088d84
Move addRegistrOverrides 2019-10-08 17:00:55 +02:00
Eelco Dolstra 21304c11f9
uri -> url for consistency 2019-10-08 17:00:55 +02:00
Eelco Dolstra 382aa05ff7 nix flake info --json: Get rid of duplicate getFlake() call
Also fix some gcc warnings.
2019-09-22 21:53:01 +02:00
Eelco Dolstra 5573365dff
nix flake check: Validate nixosConfigurations outputs 2019-09-19 20:15:42 +02:00
Eelco Dolstra c67407172d
Record original flakerefs in the lock file again
If 'input.<name>.uri' changes, then the entry in the lockfile for
input <name> should be considered stale.

Also print some messages when lock file entries are added/updated.
2019-09-18 21:57:57 +02:00
Eelco Dolstra 55e55b34e6
nix flake check: Check hydraJobs 2019-09-10 17:39:55 +02:00
Eelco Dolstra 4b9dee6bcc
nix flake check: Do some basic checks on NixOS modules
Also show more position info.
2019-09-10 15:25:32 +02:00
Eelco Dolstra dc3f52a144
nix flake check: Check overlays 2019-09-10 14:52:22 +02:00
Eelco Dolstra 80c36d4562 Remove 'name' attribute from flakes
This is no longer needed since flakes are given an identity in the
'inputs' attribute.
2019-08-30 16:38:27 +02:00
Eelco Dolstra ad42a78469
Rename 'epoch' -> 'edition' 2019-07-11 13:54:53 +02:00
Eelco Dolstra d4fe9daed6
Simplify getFlake() / fetchFlake() logic 2019-06-21 19:04:58 +02:00
Eelco Dolstra a0de58f471
Make subcommand construction in MultiCommand lazy 2019-06-18 16:25:31 +02:00
Eelco Dolstra eb18aedccb
Merge pull request #2951 from NixOS/app-improvements
App / check improvements
2019-06-18 16:15:40 +02:00
Eelco Dolstra d4a48b12fa
Merge pull request #2917 from CSVdB/docs
Updated flake documentation
2019-06-18 16:05:55 +02:00
Eelco Dolstra 8a6704d826 Updated documentation 2019-06-18 11:08:04 +02:00
Eelco Dolstra 556f33422d
nix flake check: Ignore legacyPackages 2019-06-18 09:45:14 +02:00
Eelco Dolstra f2fcc163fa
nix flake check: Warn about unknown flake outputs 2019-06-17 18:05:32 +02:00
Eelco Dolstra 9d1207c02c
nix flake check: Check apps 2019-06-17 17:59:57 +02:00