Commit graph

14828 commits

Author SHA1 Message Date
Valentin Gagarin d38a539437 make description open-ended, add TODO
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-08-31 21:27:11 +02:00
Valentin Gagarin 894cbe43bc don't invent terms yet 2023-08-31 21:26:46 +02:00
Valentin Gagarin b951e862d0 more meaningful tagline 2023-08-31 21:26:46 +02:00
Valentin Gagarin cf4e14d58d accommodate "do nothing" branch 2023-08-31 21:26:46 +02:00
Valentin Gagarin d460dbdd30 be more precise about substituting store derivations
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-08-31 21:26:46 +02:00
Valentin Gagarin 6b3320ab05 mention remote builders
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-08-31 21:26:44 +02:00
Valentin Gagarin 0cd8f36644 add anchor to builder 2023-08-31 21:26:15 +02:00
Valentin Gagarin d50f116421 add reference link 2023-08-31 21:25:39 +02:00
Valentin Gagarin b7e9e29605 remove abstract description 2023-08-31 21:25:39 +02:00
Valentin Gagarin a57e0e8c5c reword introductory sentence 2023-08-31 21:25:39 +02:00
Valentin Gagarin 315a11bcc9 remove superfluous word 2023-08-31 21:25:39 +02:00
Valentin Gagarin 1bc9257d7c reword description of how realisation works 2023-08-31 21:25:39 +02:00
Silvan Mosberger 151120a1ae
Document nix-prefetch-url defaults (#8878) 2023-08-28 22:14:01 +02:00
Bryan Honof 736b9cede7
Port the flags of nix-daemon to nix daemon (#8788)
The new `nix daemon` command didn't accept the same flags that `nix-daemon` did.

* docs(daemon): clarify the daemon trust override flags
* fix: change declaration order
* docs: add examples of nix daemon usage
* Apply suggestions from code review

---------

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
Co-authored-by: tomberek <tomberek@users.noreply.github.com>
2023-08-28 13:43:34 +00:00
John Ericson 50f40ac4c0
Merge pull request #8829 from obsidiansystems/build-dynamic-derivations
Adapt scheduler to work with dynamic derivations
2023-08-25 11:13:15 -04:00
tomberek b563ef38cc
Merge pull request #8819 from VertexA115/fix/deep-follow-paths
Fix follow path checking at depths greater than 2
2023-08-25 10:33:05 -04:00
John Ericson dd9f816b29
Merge pull request #8661 from hercules-ci/test-reformat-error-message
tests: Reformat exit code error message
2023-08-25 10:17:15 -04:00
John Ericson cbd89568ca
Merge pull request #8814 from hercules-ci/exception-self-check
initLibUtil: Add exception handling self-check
2023-08-25 10:15:54 -04:00
John Ericson 5e3986f59c Adapt scheduler to work with dynamic derivations
To avoid dealing with an optional `drvPath` (because we might not know
it yet) everywhere, make an `CreateDerivationAndRealiseGoal`. This goal
just builds/substitutes the derivation file, and then kicks of a build
for that obtained derivation; in other words it does the chaining of
goals when the drv file is missing (as can already be the case) or
computed (new case).

This also means the `getDerivation` state can be removed from
`DerivationGoal`, which makes the `BasicDerivation` / in memory case and
`Derivation` / drv file file case closer together.

The map type is factored out for clarity, and because we will soon hvae
a second use for it (`Derivation` itself).

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-08-25 10:01:25 -04:00
John Ericson 692074f714 Use Worker::makeDerivationGoal less
We're about to split up `DerivationGoal` a bit. At that point
`makeDerivationGoal` will mean something more specific than it does
today. (Perhaps a future rename will make this clearer.)

On the other hand, the more public `Worker::makeGoal` function will
continue to work exactly as before. So by moving some call sites to use
that instead, we preemptively avoid issues in the next step.
2023-08-25 09:55:07 -04:00
John Ericson 1c4caef14b Throw MissingRealisation not plain Error in both resolveDerivedPath
Now we are consistent with the other `resolveDerivedPath`, and other
such functions.
2023-08-25 09:55:07 -04:00
John Ericson 2f5d3da806 Introduce OutputName and OutputNameView type aliases
Hopefully they make the code easier to understand!
2023-08-25 09:55:07 -04:00
Robert Hensing 0a6ac133cf
Merge pull request #8864 from p01arst0rm/update-system-definitions
update system definitions
2023-08-24 17:42:59 +02:00
Eelco Dolstra 9990edf926
Merge pull request #8866 from fricklerhandwerk/glossary-list
glossary: dedent list and do not use forced line breaks
2023-08-24 13:24:31 +02:00
Valentin Gagarin d5b130ef13 glossary: dedent list and do not use forced line breaks
this makes it slightly easier to work with and consistent with all the
other markdown lists in use
2023-08-24 10:00:17 +02:00
p01arst0rm 7d82341633 update system definitions 2023-08-23 19:28:24 +01:00
Uri Zafrir 4a435ad228
Add introductory sentence to advanced topics (#8861) 2023-08-23 15:18:25 +00:00
Eelco Dolstra 6459a1c7ad
Merge pull request #8857 from NixOS/dependabot/github_actions/zeebe-io/backport-action-1.4.0
Bump zeebe-io/backport-action from 1.3.1 to 1.4.0
2023-08-22 16:08:32 +02:00
dependabot[bot] 8130373be9
Bump zeebe-io/backport-action from 1.3.1 to 1.4.0
Bumps [zeebe-io/backport-action](https://github.com/zeebe-io/backport-action) from 1.3.1 to 1.4.0.
- [Release notes](https://github.com/zeebe-io/backport-action/releases)
- [Commits](https://github.com/zeebe-io/backport-action/compare/v1.3.1...v1.4.0)

---
updated-dependencies:
- dependency-name: zeebe-io/backport-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-08-21 22:22:21 +00:00
Eelco Dolstra 3fcdea4ca0
Merge pull request #8836 from obsidiansystems/rm-addToSearchPath
Delete `EvalState::addToSearchPath`
2023-08-21 14:58:00 +02:00
Robert Hensing 10afcf06aa
Merge pull request #8812 from tweag/fix-clang-tidy
Fix some warnings/bugs found by clang-tidy
2023-08-19 16:00:12 +02:00
John Ericson fe71faa920 Delete EvalState::addToSearchPath
This function is now trivial enough that it doesn't need to exist.

`EvalState` can still be initialized with a custom search path, but we
don't have a need to mutate the search path after it has been
constructed, and I don't see why we would need to in the future.

Fixes #8229
2023-08-18 14:04:33 -04:00
John Ericson 665ad4f7c5
Merge pull request #8839 from obsidiansystems/string-context-7479
Refactor Raw pattern, part of #7479
2023-08-18 13:47:01 -04:00
John Ericson 9121fed4b4 Fixing #7479
Types converted:

- `NixStringContextElem`
- `OutputsSpec`
- `ExtendedOutputsSpec`
- `DerivationOutput`
- `DerivationType`

Existing ones mostly conforming the pattern cleaned up:

- `ContentAddressMethod`
- `ContentAddressWithReferences`

The `DerivationGoal::derivationType` field had a bogus initialization,
now caught, so I made it `std::optional`. I think #8829 can make it
non-optional again because it will ensure we always have the derivation
when we construct a `DerivationGoal`.

See that issue (#7479) for details on the general goal.

`git grep 'Raw::Raw'` indicates the two types I didn't yet convert
`DerivedPath` and `BuiltPath` (and their `Single` variants) . This is
because @roberth and I (can't find issue right now...) plan on reworking
them somewhat, so I didn't want to churn them more just yet.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2023-08-18 11:44:00 -04:00
Eelco Dolstra 735558bea6
Merge pull request #8845 from cole-h/fix-double-percent-encoding
libutil: fix double-encoding of URLs
2023-08-18 17:08:03 +02:00
Robert Hensing 284c180732
Merge pull request #8653 from hercules-ci/gitignore-.cache
.gitignore: Add .cache/
2023-08-18 15:02:21 +02:00
Robert Hensing 75243c9693
test/flakes/follow-paths.sh: Quote
Co-authored-by: Alex Ameen <alex.ameen.tx@gmail.com>
2023-08-18 14:46:13 +02:00
John Ericson 0f1eb7c351
Merge pull request #8832 from hercules-ci/positive-source-filter
Add positive source filter
2023-08-18 08:37:41 -04:00
Robert Hensing ba28613043
Merge pull request #8840 from iFreilicht/tests-for-nix-repl
Add tests for repl formatting with and without :p
2023-08-18 14:17:17 +02:00
Cole Helbling 73696ec716 libutil: fix double-encoding of URLs
If you have a URL that needs to be percent-encoded, such as
`http://localhost:8181/test/+3d.tar.gz`, and try to lock that in a Nix
flake such as the following:

    {
      inputs.test = { url = "http://localhost:8181/test/+3d.tar.gz"; flake = false; };
      outputs = { test, ... }: {
        t = builtins.readFile test;
      };
    }

running `nix flake metadata` shows that the input URL has been
incorrectly double-encoded (despite the flake.lock being correctly
encoded only once):

    [...snip...]
    Inputs:
    └───test: http://localhost:8181/test/%252B3d.tar.gz?narHash=sha256-EFUdrtf6Rn0LWIJufrmg8q99aT3jGfLvd1//zaJEufY%3D

(Notice the `%252B`? That's just `%2B` but percent-encoded again)

With this patch, the double-encoding is gone; running `nix flake
metadata` will show the proper URL:

    [...snip...]
    Inputs:
    └───test: http://localhost:8181/test/%2B3d.tar.gz?narHash=sha256-EFUdrtf6Rn0LWIJufrmg8q99aT3jGfLvd1//zaJEufY%3D

---

As far as I can tell, this happens because Nix already percent-encodes
the URL and stores this as the value of `inputs.asdf.url`.

However, when Nix later tries to read this out of the eval state as a
string (via `getStrAttr`), it has to run it through `parseURL` again to
get the `ParsedURL` structure.

Now, this itself isn't a problem -- the true problem arises when using
`ParsedURL::to_string` later, which then _re-escapes the path_. It is
at this point that what would have been `%2B` (`+`) becomes `%252B`
(`%2B`).
2023-08-17 14:16:19 -07:00
Cole Helbling 1d7a57cfd9 libexpr/tests: test that parseFlakeRef doesn't percent-encode twice 2023-08-17 13:45:55 -07:00
Felix Uhl 17ceec3a91 Test repl formatting with and without :p 2023-08-17 13:03:43 +02:00
Eelco Dolstra db3bf180a5
Merge pull request #8833 from hercules-ci/jobcategory-doc
Document jobCategory()
2023-08-16 17:11:03 +02:00
Eelco Dolstra 7f8c99c70c
Merge pull request #8825 from trofi/search-path-prefix
src/libexpr/search-path.cc: avoid out-of-bounds read on string_view
2023-08-16 16:44:49 +02:00
Robert Hensing d8079ee350 Document jobCategory() 2023-08-16 16:16:58 +02:00
Robert Hensing 21a188a2b4 Add gc root for nixpkgs/lib content 2023-08-16 16:01:46 +02:00
Robert Hensing 63e0b5d081 GC root for fetched nixpkgs/lib content 2023-08-16 15:46:37 +02:00
Robert Hensing b13fc7101f Add positive source filter
Source filtering is a really cool Nix feature that lets us avoid a
lot of rebuilds, which speeds up the iteration cycle a lot in cases
where the relevant source files aren't actually modified.

We used to have a source filter that marked a few files as irrelevant,
but this is the wrong approach, as we have many more files that are
irrelevant. We may call this negative filtering.

This commit switches the source filtering to positive filtering, which
is a lot more robust. Instead of marking which files we don't need
we marked the files that we do need.

It's a superior approach because it is fail safe. Instead of allowing
build performance problems to creep in over time, we require that all
source inputs are declared.

I shouldn't have to explain that declaring inputs is a good practice,
so I'll stop over-explaining here.

I do have to acknowledge that this will cause a build failure when the
filter is incomplete. This is *good*, because it's the only realistic
way we could be reminded of these problems. These events will be
infrequent, so the small cost of extending the filter is worth it,
compared to the hidden cost of longer dev cycles for things like tests,
docker image, etc, etc.

(Also rebuilding Nix for stupid unnecessary reasons makes my blood boil)
2023-08-16 14:21:59 +02:00
Vertex 20d9c672d1
Update tests/flakes/follow-paths.sh
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-08-15 10:10:27 +01:00
Sergei Trofimovich b74962c92b src/libexpr/search-path.cc: avoid out-of-bounds read on string_view
Without the change build with `-D_GLIBCXX_ASSERTIONS` exposes testsuite
assertion:

    $ gdb src/libexpr/tests/libnixexpr-tests
    Reading symbols from src/libexpr/tests/libnixexpr-tests...
    (gdb) break __glibcxx_assert_fail
    (gdb) run
    (gdb) bt
    in std::__glibcxx_assert_fail(char const*, int, char const*, char const*)@plt () from /mnt/archive/big/git/nix/src/libexpr/libnixexpr.so
    in std::basic_string_view<char, std::char_traits<char> >::operator[] (this=0x7fffffff56c0, __pos=4)
        at /nix/store/r74fw2j8rx5idb0w8s1s6ynwwgs0qmh9-gcc-14.0.0/include/c++/14.0.0/string_view:258
    in nix::SearchPath::Prefix::suffixIfPotentialMatch (this=0x7fffffff5780, path=...) at src/libexpr/search-path.cc:15
    in nix::SearchPathElem_suffixIfPotentialMatch_partialPrefix_Test::TestBody (this=0x555555a17540) at src/libexpr/tests/search-path.cc:62

As string sizes are usigned types `(a - b) > 0` effectively means
`a != b`. While the intention should be `a > b`.

The change fixes test suite pass.
2023-08-14 22:07:37 +01:00