Commit graph

2118 commits

Author SHA1 Message Date
Valentin Gagarin 28eb406834 reword section on make variables
- use one line per sentence
- use imperative for instructions
- add link to Make documentation
2024-01-17 04:39:26 +01:00
Valentin Gagarin f134dbdffb move section on make variables
it should be after the general build instructions, as it goes into more detail.
2024-01-17 04:23:16 +01:00
Robert Hensing 2a3c5e6b8b
Merge pull request #9480 from NixOS/libfetchers-git-exportIgnore
libfetchers/git: Support export-ignore
2024-01-16 23:03:46 +01:00
Robert Hensing 51f524c629
Merge pull request #9755 from 9999years/printer-followup
Printer followup
2024-01-16 22:35:08 +01:00
Robert Hensing baea5f42c6
doc/glossary: Simplify software package definition
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-01-16 14:50:53 +01:00
Robert Hensing 7e5fa5c25c doc/glossary: Define package and package attribute set
A small step towards https://github.com/NixOS/nix/issues/6507

I believe this incomplete definition is one that can be agreed on.
It would be nice to define more, but considering that the issue
also proposes changes to the design, I believe we should hold off
on those.

As for the wording, we're dealing with some very general and vague
terms, that have to be treated with exactly the right amount of
vagueness to be effective.

I start out with a fairly abstract definition of package.
1. to establish a baseline so we know what we're talking about
2. so that we can go in and clarify that we have an extra, Nix-specific
   definition.

"Software" is notoriously ill-defined, so it makes a great qualifier
for package, which we don't really want to pin down either, because
that would just get us lost in discussion.
We can come back to this after we've done 6057 and a few years in a
desert cave.

Then comes the "package attribute set" definition.
I can already hear Valentin say "That's not even Nix's responsibility!"
and on some days I might even agree.
However, in our current reality, we have `nix-env`, `nix-build` and
`nix profile`, which query the `outputName` attribute - among others -
which just don't exist in the derivation.

For those who can't believe what they're reading:

    $ nix-build --expr 'with import ./. {}; bind // {outputName = "lib";}' --no-out-link
    this path will be fetched (1.16 MiB download, 3.72 MiB unpacked):
      /nix/store/rfk6klfx3z972gavxlw6iypnj6j806ma-bind-9.18.21-lib
    copying path '/nix/store/rfk6klfx3z972gavxlw6iypnj6j806ma-bind-9.18.21-lib' from 'https://cache.nixos.org'...
    /nix/store/rfk6klfx3z972gavxlw6iypnj6j806ma-bind-9.18.21-lib

and let me tell you that bind is not a library.

So anyway, that's also proof of why calling this a "derivation attrset" would be wrong, despite the type attribute.
2024-01-13 20:00:06 +01:00
Qyriad cbd5553d57
doc: provide context in glossary definitions (#9378) 2024-01-13 12:20:08 +01:00
Rebecca Turner f73e50144f
Clarify ambiguity in nix-instantiate man page 2024-01-12 13:53:45 -08:00
Rebecca Turner 2d96c7a51f
Remove outdated reference to y in nix-instantiate man page 2024-01-12 11:27:31 -08:00
Rebecca Turner b29be1ff57
Document unit tests in hacking.md 2024-01-12 10:47:49 -08:00
Eelco Dolstra 8c7e2ed77c Update release notes 2024-01-12 16:21:07 +01:00
Robert Hensing 71d08af15b rl-next: Add *general* note about git fetcher reimpl 2024-01-12 15:31:53 +01:00
Rebecca Turner df84dd4d8d
Restore ambiguous value printer for nix-instantiate
The Nix team has requested that this output format remain unchanged.
I've added a warning to the man page explaining that `nix-instantiate
--eval` output will not parse correctly in many situations.
2024-01-11 16:34:36 -08:00
Valentin Gagarin 6a243e5ed2
fix an old lost direct (#9458)
this part must have been moved quite a while ago, but apparently so far
no one noticed
2024-01-08 19:38:36 +01:00
Valentin Gagarin 8e865f3aba
deduplicate installation instructions (#9507)
* deduplicate installation instructions

- reorder sections to present pinned installation more prominently
- remove outdated notes on the macOS installer rework
- update instructions to handle the installer tarball

Co-authored-by: Travis A. Everett <travis.a.everett@gmail.com>
2024-01-06 22:45:25 +01:00
wiki-me a4d33e816e
Improve documentation around upgrading nix (#9679)
* Improve documentation around upgrading nix, add replacing nix channel with new one

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-01-06 18:01:10 +00:00
Rebecca Turner d8a2b06e20
Remove clang11Stdenv
Clang 11 doesn't have support for three-way-comparisons (<=>, "spaceship
operator", "consistent comparisons") and is older than `clangStdenv`.

`clangStdenv` is currently 12 on FreeBSD and Android and 16 on other
platforms:

32e718f00c/pkgs/top-level/all-packages.nix (L16629-L16644)

Let's start by removing Clang 11 from our distribution. Next we can
consider upgrading to Clang 17, which fully supports the spaceship
operator:

https://releases.llvm.org/17.0.1/tools/clang/docs/ReleaseNotes.html#what-s-new-in-clang-release
2024-01-04 11:43:56 -08:00
Robert Hensing 83f5622545
Merge pull request #9658 from pennae/env-diet
reduce the size of Env by one pointer
2023-12-31 13:57:16 +01:00
Robert Hensing 3511430902
Merge pull request #9673 from pennae/drv-parse-opts
optimize derivation parsing
2023-12-31 13:49:03 +01:00
pennae 1fe66852ff reduce the size of Env by one pointer
since `up` and `values` are both pointer-aligned the type field will
also be pointer-aligned, wasting 48 bits of space on most machines. we
can get away with removing the type field altogether by encoding some
information into the `with` expr that created the env to begin with,
reducing the GC load for the absolutely massive amount of single-entry
envs we create for lambdas. this reduces memory usage of system eval by
quite a bit (reducing heap size of our system eval from 8.4GB to 8.23GB)
and gives similar savings in eval time.

running `nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'`

before:

  Time (mean ± σ):      5.576 s ±  0.003 s    [User: 5.197 s, System: 0.378 s]
  Range (min … max):    5.572 s …  5.581 s    10 runs

after:

  Time (mean ± σ):      5.408 s ±  0.002 s    [User: 5.019 s, System: 0.388 s]
  Range (min … max):    5.405 s …  5.411 s    10 runs
2023-12-30 18:55:13 +01:00
pennae 79d3d412ca optimize derivation string parsing
a bunch of derivation strings contain no escape sequences. we can
optimize for this fact by first scanning for the end of a derivation
string and simply returning the contents unmodified if no escape
sequences were found. to make this even more efficient we can also use
BackedStringViews to avoid copies, avoiding heap allocations for
transient data.

before:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.952 s ±  0.015 s    [User: 5.294 s, System: 1.452 s]
  Range (min … max):    6.926 s …  6.974 s    10 runs

after:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.907 s ±  0.012 s    [User: 5.272 s, System: 1.429 s]
  Range (min … max):    6.893 s …  6.926 s    10 runs
2023-12-30 18:44:10 +01:00
Rebecca Turner 7434caca05
Fix segfault on infinite recursion in some cases
This fixes a segfault on infinite function call recursion (rather than
infinite thunk recursion) by tracking the function call depth in
`EvalState`.

Additionally, to avoid printing extremely long stack traces, stack
frames are now deduplicated, with a `(19997 duplicate traces omitted)`
message. This should only really be triggered in infinite recursion
scenarios.

Before:

    $ nix-instantiate --eval --expr '(x: x x) (x: x x)'
    Segmentation fault: 11

After:

    $ nix-instantiate --eval --expr '(x: x x) (x: x x)'
    error: stack overflow

           at «string»:1:14:
                1| (x: x x) (x: x x)
                 |              ^

    $ nix-instantiate --eval --expr '(x: x x) (x: x x)' --show-trace
    error:
           … from call site
             at «string»:1:1:
                1| (x: x x) (x: x x)
                 | ^

           … while calling anonymous lambda
             at «string»:1:2:
                1| (x: x x) (x: x x)
                 |  ^

           … from call site
             at «string»:1:5:
                1| (x: x x) (x: x x)
                 |     ^

           … while calling anonymous lambda
             at «string»:1:11:
                1| (x: x x) (x: x x)
                 |           ^

           … from call site
             at «string»:1:14:
                1| (x: x x) (x: x x)
                 |              ^

           (19997 duplicate traces omitted)

           error: stack overflow
           at «string»:1:14:
                1| (x: x x) (x: x x)
                 |              ^
2023-12-29 22:16:44 -08:00
Brian Le e2399fc949 Change "dervation" typos to "derivation" 2023-12-26 17:12:28 -05:00
Shea Levy c3942ef85f
Build IFD in the build store when using eval-store.
Previously, IFDs would be built within the eval store, even though one
is typically using `--eval-store` precisely to *avoid* local builds.

Because the resulting Nix expression must be copied back to the eval
store in order to be imported, this requires the eval store to trust
the build store's signatures.
2023-12-23 21:33:56 -05:00
Eelco Dolstra 942d635102 Fix release notes 2023-12-21 16:38:25 +01:00
Eelco Dolstra 9d9d9ff0de Merge remote-tracking branch 'origin/master' into profile-names-instead-of-index 2023-12-21 16:21:26 +01:00
Rebecca Turner 6f4930382b
Document more Makefile variables 2023-12-18 13:34:08 -08:00
John Ericson 7feabf7d44 Split --disable-tests, fix cross builds
It might seem obnoxious to have yet more configure flags, but I found
controlling both the unit and functional tests with one flag was quite
confusing because they are so different:

- unit tests depending on building, functional tests don't (e.g. when
  we test already-built Nix)

- unit tests can be installed, functional tests cannot

- unit tests neeed extra libraries (GTest, RapidCheck), functional
  tests need extra executables (jq).

- unit tests are run by `make check`, functional tests are run by `make
  installcheck`

Really on a technical level, they seem wholly independent. Only on a
human level ("they are both are tests") do they have anything in common.

I had messed up the logic in cross builds because of this. Now I
split the flag in two (and cleaned up a few other inconsistencies), and
the logic fixed itself.

Co-Authored-By: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:47:50 -05:00
Rebecca Turner 7f5ed330e4
Document Makefile variables in hacking.md (#9620) 2023-12-16 12:05:31 +00:00
John Ericson 228e995cde Add release not for eval-system 2023-12-14 19:55:50 -05:00
Ramses 1e3d811840
worker protocol: serialise cgroup stats in BuildResult (#9598)
By doing so, they get reported when building through the daemon via either `unix://` or `ssh-ng://`.
2023-12-13 16:37:17 -05:00
Eelco Dolstra 1b7968ed86
Merge pull request #9547 from hercules-ci/allowed-scheme-without-slash
`allowed-uris`: match whole schemes without slashes
2023-12-13 20:23:33 +01:00
tomberek 7026abfdde
Merge pull request #9523 from fricklerhandwerk/conf-reword-always-allow-substitutes
reword documentation on settings and attributes related to substitution
2023-12-12 20:09:48 -05:00
Robert Hensing 89cf53648c
Contributing branches and reverting (#9577)
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-11 12:26:31 +01:00
Robert Hensing a05bc9eb92 allowed-uris: Match whole schemes also when scheme is not followed by slashes 2023-12-11 12:18:04 +01:00
Valentin Gagarin 3c200da242
document fetchTree (#9258)
* document `fetchTree`

* display experimental feature note at the top

we have to enable the new `fetchTree` experimental feature to render it
at all. this was a bug introduced when adding that new feature flag.

Co-authored-by: tomberek <tomberek@users.noreply.github.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Silvan Mosberger <github@infinisil.com>
2023-12-10 05:16:32 +00:00
Robert Hensing 3811b334c6 rl-next: Use markdown frontmatter syntax
The old syntax is still supported, as long as you don't use a {
in the description - the reason to migrate.
2023-12-09 19:57:55 +01:00
Robert Hensing c8458bd731
Merge pull request #9555 from 9999years/positions-in-errors
Pass positions when evaluating
2023-12-09 03:55:58 +01:00
Robert Hensing b9980b377e
Update rl-next/source-positions-in-errors for Nix 2.19+ 2023-12-09 02:36:33 +01:00
Robert Hensing 9b7b7a7561 Revert "Print the value in error: cannot coerce messages (#9553)"
This reverts commit f0ac2a35d5.

The request from the sibling PR, which also applies here, was not addressed.
https://github.com/NixOS/nix/pull/9554#issuecomment-1845095735
2023-12-09 02:13:32 +01:00
Rebecca Turner f0ac2a35d5
Print the value in error: cannot coerce messages (#9553)
* Print the value in `error: cannot coerce` messages

This extends the `error: cannot coerce a TYPE to a string` message
to print the value that could not be coerced. This helps with debugging
by making it easier to track down where the value is being produced
from, especially in errors with deep or unhelpful stack traces.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-08 16:36:57 +00:00
Rebecca Turner 0b80935c22
Pass positions when evaluating
This includes position information in more places, making debugging
easier.

Before:

```
$ nix-instantiate --show-trace --eval tests/functional/lang/eval-fail-using-set-as-attr-name.nix
error:
       … while evaluating an attribute name

         at «none»:0: (source not available)

       error: value is a set while a string was expected
```

After:

```
error:
       … while evaluating an attribute name

         at /pwd/lang/eval-fail-using-set-as-attr-name.nix:5:10:

            4| in
            5|   attr.${key}
             |          ^
            6|

       error: value is a set while a string was expected
```
2023-12-07 10:27:21 -08:00
Peter Kolloch 8afeaf05c4 Add docs/rl-notes for nix hash convert / builtins.convertHash
https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Eelco Dolstra 3dcb83409d
Merge pull request #9509 from fricklerhandwerk/add-redirect
add redirect to new store page
2023-12-06 18:03:53 +01:00
Robert Hensing efc65ef122
Merge pull request #9505 from fricklerhandwerk/quick-start
shorten the quick start chapter
2023-12-05 19:52:03 +01:00
Eelco Dolstra d54f847682
Merge pull request #9520 from fricklerhandwerk/fix-links-stores-overview
fix links in stores overview
2023-12-05 14:46:54 +01:00
Valentin Gagarin 5fe2accb75 fix up release note 2023-12-04 16:42:32 +01:00
Valentin Gagarin 51adfb9b27 reword documentation on settings and attributes related to substitution
- add links
- be more concise
- clarify the distinction between `preferLocalBuild` and `allowSubstitutes`
2023-12-02 02:56:25 +01:00
Valentin Gagarin 59c4c82aeb fix links in stores overview 2023-12-02 00:56:41 +01:00
John Ericson 91b6833686 Move tests to separate directories, and document
Today, with the tests inside a `tests` intermingled with the
corresponding library's source code, we have a few problems:

- We have to be careful that wildcards don't end up with tests being
  built as part of Nix proper, or test headers being installed as part
  of Nix proper.

- Tests in libraries but not executables is not right:

  - It means each executable runs the previous unit tests again, because
    it needs the libraries.

  - It doesn't work right on Windows, which doesn't want you to load a
    DLL just for the side global variable . It could be made to work
    with the dlopen equivalent, but that's gross!

This reorg solves these problems.

There is a remaining problem which is that sibbling headers (like
`hash.hh` the test header vs `hash.hh` the main `libnixutil` header) end
up shadowing each other. This PR doesn't solve that. That is left as
future work for a future PR.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-01 10:48:58 -05:00