Find a file
Maximilian Bosch bfdd908f7d structured attrs: improve support / usage of NIX_ATTRS_{SH,JSON}_FILE
In #4770 I implemented proper `nix-shell(1)` support for derivations
using `__structuredAttrs = true;`. Back then we decided to introduce two
new environment variables, `NIX_ATTRS_SH_FILE` for `.attrs.sh` and
`NIX_ATTRS_JSON_FILE` for `.attrs.json`. This was to avoid having to
copy these files to `$NIX_BUILD_TOP` in a `nix-shell(1)` session which
effectively meant copying these files to the project dir without
cleaning up afterwords[1].

On last NixCon I resumed hacking on `__structuredAttrs = true;` by
default for `nixpkgs` with a few other folks and getting back to it,
I identified a few problems with the how it's used in `nixpkgs`:

* A lot of builders in `nixpkgs` don't care about the env vars and
  assume that `.attrs.sh` and `.attrs.json` are in `$NIX_BUILD_TOP`.
  The sole reason why this works is that `nix-shell(1)` sources
  the contents of `.attrs.sh` and then sources `$stdenv/setup` if it
  exists. This may not be pretty, but it mostly works. One notable
  difference when using nixpkgs' stdenv as of now is however that
  `$__structuredAttrs` is set to `1` on regular builds, but set to
  an empty string in a shell session.

  Also, `.attrs.json` cannot be used in shell sessions because
  it can only be accessed by `$NIX_ATTRS_JSON_FILE` and not by
  `$NIX_BUILD_TOP/.attrs.json`.

  I considered changing Nix to be compatible with what nixpkgs
  effectively does, but then we'd have to either move $NIX_BUILD_TOP for
  shell sessions to a temporary location (and thus breaking a lot of
  assumptions) or we'd reintroduce all the problems we solved back then
  by using these two env vars.

  This is partly because I didn't document these variables back
  then (mea culpa), so I decided to drop all mentions of
  `.attrs.{json,sh}` in the  manual and only refer to `$NIX_ATTRS_SH_FILE`
  and `$NIX_ATTRS_JSON_FILE`. The same applies to all our integration tests.
  Theoretically we could deprecated using `"$NIX_BUILD_TOP"/.attrs.sh` in
  the future now.

* `nix develop` and `nix print-dev-env` don't support this environment
  variable at all even though they're supposed to be part of the replacement
  for `nix-shell` - for the drv debugging part to be precise.

  This isn't a big deal for the vast majority of derivations, i.e.
  derivations relying on nixpkgs' `stdenv` wiring things together
  properly. This is because `nix develop` effectively "clones" the
  derivation and replaces the builder with a script that dumps all of
  the environment, shell variables, functions etc, so the state of
  structured attrs being "sourced" is transmitted into the dev shell and
  most of the time you don't need to worry about `.attrs.sh` not
  existing because the shell is correctly configured and the

      if [ -e .attrs.sh ]; then source .attrs.sh; fi

  is simply omitted.

  However, this will break when having a derivation that reads e.g. from
  `.attrs.json` like

      with import <nixpkgs> {};
      runCommand "foo" { __structuredAttrs = true; foo.bar = 23; } ''
        cat $NIX_ATTRS_JSON_FILE # doesn't work because it points to /build/.attrs.json
      ''

  To work around this I employed a similar approach as it exists for
  `nix-shell`: the `NIX_ATTRS_{JSON,SH}_FILE` vars are replaced with
  temporary locations.

  The contents of `.attrs.sh` and `.attrs.json` are now written into the
  JSON by `get-env.sh`, the builder that `nix develop` injects into the
  derivation it's debugging. So finally the exact file contents are
  present and exported by `nix develop`.

  I also made `.attrs.json` a JSON string in the JSON printed by
  `get-env.sh` on purpose because then it's not necessary to serialize
  the object structure again. `nix develop` only needs the JSON
  as string because it's only written into the temporary file.

  I'm not entirely sure if it makes sense to also use a temporary
  location for `nix print-dev-env` (rather than just skipping the
  rewrite in there), but this would probably break certain cases where
  it's relied upon `$NIX_ATTRS_SH_FILE` to exist (prime example are the
  `nix print-dev-env` test-cases I wrote in this patch using
  `tests/shell.nix`, these would fail because the env var exists, but it
  cannot read from it).

[1] https://github.com/NixOS/nix/pull/4770#issuecomment-836799719
2023-10-01 13:22:48 +01:00
.github Bump docker/login-action from 2 to 3 2023-09-18 22:16:55 +00:00
config Run autoupdate 2021-06-01 11:42:38 +02:00
contrib function-trace: always show the trace 2019-09-18 23:23:21 +02:00
doc structured attrs: improve support / usage of NIX_ATTRS_{SH,JSON}_FILE 2023-10-01 13:22:48 +01:00
m4 autoconf: Fix C++17 detection not working on Ubuntu 16.04. 2019-07-03 04:32:25 +02:00
maintainers maintainers: add note on marking PRs as draft 2023-06-19 10:55:34 +02:00
misc Choose a reasonable number similar to LimitNOFile 2023-07-07 07:52:16 -07:00
mk Unit test some worker protocol serializers 2023-09-05 10:48:02 -04:00
perl Allow dynamic derivation deps in inputDrvs 2023-09-07 10:39:37 -04:00
scripts Merge pull request #7593 from miallo/ignore-identical-rc-backup-files-for-installation 2023-09-25 14:14:53 +02:00
src structured attrs: improve support / usage of NIX_ATTRS_{SH,JSON}_FILE 2023-10-01 13:22:48 +01:00
tests structured attrs: improve support / usage of NIX_ATTRS_{SH,JSON}_FILE 2023-10-01 13:22:48 +01:00
unit-test-data/libstore/worker-protocol Unit test some worker protocol serializers 2023-09-05 10:48:02 -04:00
.dir-locals.el .dir-locals.el: Set c-block-comment-prefix 2020-07-10 11:21:06 +02:00
.editorconfig Add .editorconfig 2017-06-05 22:57:28 +01:00
.gitignore Merge pull request #8653 from hercules-ci/gitignore-.cache 2023-08-18 15:02:21 +02:00
.version Bump version 2023-09-20 15:21:21 +02:00
boehmgc-coroutine-sp-fallback.diff Fix boehmgc-coroutine-sp-fallback.diff for FreeBSD 2023-09-20 09:04:42 -04:00
bootstrap.sh bootstrap: Simplify & make more robust. 2011-09-06 12:11:05 +00:00
configure.ac expand on the system type in hacking guide 2023-07-19 10:37:40 +02:00
CONTRIBUTING.md contributor guide: emphasize solving a well-specified problem with each pull request 2023-09-29 16:19:14 +02:00
COPYING * Change this to LGPL to keep the government happy. 2006-04-25 16:41:06 +00:00
default.nix GC root for fetched nixpkgs/lib content 2023-08-16 15:46:37 +02:00
docker.nix fix "add an option to include flake-registry..." 2023-05-16 14:35:31 +02:00
flake.lock flake: complete update to 23.05 2023-09-21 21:00:53 -04:00
flake.nix Merge pull request #8923 from obsidiansystems/test-proto 2023-09-26 17:12:24 +02:00
local.mk Enable -Werror=switch-enum 2023-04-03 18:45:20 +02:00
Makefile Unit test some worker protocol serializers 2023-09-05 10:48:02 -04:00
Makefile.config.in Generate API docs with Doxygen 2023-03-10 12:51:06 -05:00
precompiled-headers.h Config: Use nlohmann/json 2020-08-20 11:02:16 +02:00
README.md Merge pull request #9016 from fricklerhandwerk/contributing-readme 2023-09-21 12:43:17 -04:00
shell.nix Remove url literals 2022-01-24 13:28:21 +01:00

Nix

Open Collective supporters Test

Nix is a powerful package manager for Linux and other Unix systems that makes package management reliable and reproducible. Please refer to the Nix manual for more details.

Installation

On Linux and macOS the easiest way to install Nix is to run the following shell command (as a user other than root):

$ curl -L https://nixos.org/nix/install | sh

Information on additional installation methods is available on the Nix download page.

Building And Developing

See our Hacking guide in our manual for instruction on how to to set up a development environment and build Nix from source.

Contributing

Check the contributing guide if you want to get involved with developing Nix.

Additional Resources

License

Nix is released under the LGPL v2.1.