Commit graph

1933 commits

Author SHA1 Message Date
Ilya K b469c6509b libstore/build: just copy the magic /etc files into the sandbox
Saves us a bunch of thinking about how to handle symlinks, and prevents
the DNS config from changing on the fly under the build, which may or may
not be a good thing?

Change-Id: I071e6ae7e220884690b788d94f480866f428db71
2024-04-13 12:43:19 +03:00
Théophane Hufschmitt 07b627cc6d Merge pull request #10456 from NixOS/fixpermdeniedbind
Fix adding symlink to the sandbox paths

(cherry-picked from commit da1e977bf48cff2a635034c85e7c13878e38efc2)

Change-Id: I221c85a38180800ec6552d2e86a88df48398fad8
2024-04-11 15:43:58 +03:00
Rebecca Turner 9e7e927837 Print top-level errors normally in nix repl
Previously, errors while printing values in `nix repl` would be printed
in `«error: ...»` brackets rather than displayed normally:

```
nix-repl> legacyPackages.aarch64-darwin.pythonPackages.APScheduler
«error: Package ‘python-2.7.18.7’ in /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/cpython/2.7/default.nix:335 is marked as insecure, refusing to evaluate.»
```

Now, errors will be displayed normally if they're emitted at the
top-level of an expression:

```
nix-repl> legacyPackages.aarch64-darwin.pythonPackages.APScheduler
error:
       … in the condition of the assert statement
         at /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/lib/customisation.nix:268:17:
          267|     in commonAttrs // {
          268|       drvPath = assert condition; drv.drvPath;
             |                 ^
          269|       outPath = assert condition; drv.outPath;

       … in the left operand of the OR (||) operator
         at /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/passthrufun.nix:28:45:
           27|         if lib.isDerivation value then
           28|           lib.extendDerivation (valid value || throw "${name} should use `buildPythonPackage` or `toPythonModule` if it is to be part of the Python packages set.") {} value
             |                                             ^
           29|         else

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: Package ‘python-2.7.18.7’ in /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/cpython/2.7/default.nix:335 is marked as insecure, refusing to evaluate.
```

Errors emitted in nested structures (like e.g. when printing `nixpkgs`)
will still be printed in brackets.

Change-Id: I25aeddf08c017582718cb9772a677bf51b9fc2ad
2024-04-09 08:34:40 -07:00
Rebecca Turner a95c62673b Merge "Add repl-overlays" into main 2024-04-09 05:29:48 +00:00
Rebecca Turner 727b43478c Add repl-overlays
Adds a `repl-overlays` option, which specifies files that can overlay
and modify the top-level bindings in `nix repl`. For example, with the
following contents in `~/.config/nix/repl.nix`:

    info: final: prev: let
      optionalAttrs = predicate: attrs:
        if predicate
        then attrs
        else {};
    in
      optionalAttrs (prev ? legacyPackages && prev.legacyPackages ? ${info.currentSystem})
      {
        pkgs = prev.legacyPackages.${info.currentSystem};
      }

We can run `nix repl` and use `pkgs` to refer to `legacyPackages.${currentSystem}`:

    $ nix repl --repl-overlays ~/.config/nix/repl.nix nixpkgs
    Lix 2.90.0
    Type :? for help.
    Loading installable 'flake:nixpkgs#'...
    Added 5 variables.
    Loading 'repl-overlays'...
    Added 6 variables.
    nix-repl> pkgs.bash
    «derivation /nix/store/g08b5vkwwh0j8ic9rkmd8mpj878rk62z-bash-5.2p26.drv»

Change-Id: Ic12e0f2f210b2f46e920c33088dfe1083f42391a
2024-04-08 17:11:47 -07:00
jade 54bc1f1b98 "but doctor, I AM the untrusted store": nix doctor had wrong trustedness
This probably snuck in in a refactor using truthiness or so. The
trustedness flag was having the optional fullness checked, rather than
the actual contained trust level.

Also adds some tests.

```
m1@6876551b-255d-4cb0-af02-8a4f17b27e2e ~ % nix store ping
warning: 'nix store ping' is a deprecated alias for 'nix store info'
Store URL: daemon
Version: 2.20.4
Trusted: 0
m1@6876551b-255d-4cb0-af02-8a4f17b27e2e ~ % nix doctor
warning: 'doctor' is a deprecated alias for 'config check'
[PASS] PATH contains only one nix version.
[PASS] All profiles are gcroots.
[PASS] Client protocol matches store protocol.
[INFO] You are trusted by store uri: daemon
```

Fixes: lix-project/lix#232
Change-Id: I21576e2a0a755036edf8814133345987617ba3d0
2024-04-08 15:40:12 -07:00
Rebecca Turner 5ec256625e Merge "Don't run libstore unit tests in project root" into main 2024-04-08 22:08:34 +00:00
Rebecca Turner 12f2565f91 Don't run libstore unit tests in project root
This keeps the libstore unit tests from writing
`libstore-unit-tests.xml` to the project root.

Change-Id: I0d9909aabf9f3574cc1e72a5ae81daefba9a394b
2024-04-08 12:43:35 -07:00
Rebecca Turner 4a49a497eb Fix REPL test parser nits
Follow-up to https://gerrit.lix.systems/c/lix/+/546

Change-Id: Ie603f01e5520329bf879e061cea9e3fba45213fd
2024-04-08 11:14:34 -07:00
Rebecca Turner 4600eabdae Merge "Rename some parse failure tests to parse-fail-*" into main 2024-04-08 06:33:51 +00:00
Rebecca Turner 8ed8b9aef0 Merge "Add PathsSetting" into main 2024-04-08 06:31:11 +00:00
Robert Hensing b995c17f0e Merge pull request #10208 from 9999years/print-strings-directly
`:print` strings directly in `nix repl`

(cherry picked from commit 3539172fd2f7cee639ce46423c58beca4231f2db)
Change-Id: I1972f3bf3b56312851f38288509d371d37f21677
Upstream-PR: https://github.com/NixOS/nix/pull/10208
2024-04-07 19:10:43 -07:00
Rebecca Turner 6f863e8ccd Add PathsSetting
Change-Id: I1165f6ef033a5f757ca3716d3f8008ba36b01fd0
2024-04-07 18:22:15 -07:00
Rebecca Turner 06e11778b5 Merge "Rewrite REPL test parser" into main 2024-04-06 21:36:26 +00:00
Qyriad 4e11b0d04d Merge "meson: add missing tests: ca, dyn-drv, plugins, libstoreconsumer" into main 2024-04-06 01:05:09 +00:00
eldritch horrors e9e1b6963c Merge changes I1fa30114,I3ca208b6,Ide4c6e00,I74c46b9f,I05fa6a9d, ... into main
* changes:
  Revert "libutil: drop Pool resources on exceptional free"
  Revert "libutil: remove Pool::Handle::bad"
  Revert "libstore: remove one Resource::good flag"
  Revert "libstore: using throwing finally in withFramedSink"
  Revert "libutil: allow graceful dropping of Pool::Handle"
  Revert "libutil: drop Fs{Source,Sink}::good"
  libutil: guard Finally against invalid exception throws
2024-04-05 23:17:18 +00:00
Qyriad 405e41e288 Merge "meson: run repl characterization test" into main 2024-04-05 20:57:42 +00:00
Rebecca Turner ee423f391d Rewrite REPL test parser
- Use a recursive descent parser so that it's easy to extend.
- Add `@args` to enable customizing command-line arguments
- Add `@should-start` to enable `nix repl` tests that error before
  entering the REPL
- Make sure to read all stdout output before comparing. This catches
  some extra output we were tossing out before!

Change-Id: I5522555df4c313024ab15cd10f9f04e7293bda3a
2024-04-05 13:14:21 -07:00
eldritch horrors 38dc6f5b69 Revert "libutil: drop Pool resources on exceptional free"
This reverts commit de2884b82b.

Change-Id: I1fa301149d7c2ed3d266a40c15b2d010e12e44e6
2024-04-05 20:13:02 +00:00
Qyriad 07e1ddc23d meson: add missing tests: ca, dyn-drv, plugins, libstoreconsumer
Change-Id: I6a74ebaf93697cb99aadd6b51538c2766b0a808a
2024-04-04 20:43:38 -06:00
Rebecca Turner 0fec7d2db6 Rename some parse failure tests to parse-fail-*
These were mistakenly labeled `eval-fail-*`.

Note that the `lang.sh` runner passes `parse-fail-*` tests on stdin, so
filenames are removed from error messages.

Change-Id: I7f3a0d78b6cfa87af29aaa1b7af19d5a57fd4ade
2024-04-04 15:49:30 -07:00
Qyriad 3f4c7a68f4 meson: run repl characterization test
This was mistakenly left out of 69c3363f2.

Fixes #205.

Change-Id: Ib9a2088c9eee3a192623bee107ba32221d048640
2024-04-03 17:05:30 -06:00
Rebecca Turner 84018e1264 Make repl_characterization.cc constants string_views
Change-Id: I96455ee365799ae87d9ab433cf102c691616a45e
2024-04-03 16:25:54 +00:00
Rebecca Turner ea10fe7ab0 Add EscapeStringOptions and escapeString tests
Change-Id: I86ead2f969c9e03c9edfa51bbc92ee06393fd7d6
2024-04-03 16:25:54 +00:00
Théophane Hufschmitt 6c29016a09 Merge pull request #9920 from 9999years/forbid-nested-debuggers
Forbid nested debuggers

(cherry picked from commit e164b39ee90fd655dbb7f479fdd4fbe38cc883bd)
Change-Id: Iff62f40fd251116516a63e2d3f9fb5b21480b16d
2024-03-31 17:28:25 +00:00
jade 73507a7167 Merge changes Ib62d3d68,Ic3e7affe into main
* changes:
  Make things that can throw not noexcept anymore
  Fix various clang-tidy lints
2024-03-31 15:38:48 +00:00
eldritch horrors 862f20a4ba libutil: remove Pool::flushBad
this was never actually used, and bad design in the first place—why
should a bad resource be put back into the idle pool? just drop it.

Change-Id: Idab8774bee19dadae0209d404c4fb86dd4aeba1e
2024-03-31 00:07:09 +00:00
eldritch horrors 620de98d0c libutil: drop Pool resources on exceptional free
if a scope owning a resource does not gracefully drop that resource
while handling exceptions from deeper down the call stack we should
assume the resource is invalid state and drop it. currently it *is*
true that such cases do not cause resources to be freed, but thanks
to validator misuses this has so far not caused any larger problem.

Change-Id: Ie4f91bcd60a64d05c5ff9d22cc97954816d13b97
2024-03-30 23:40:45 +00:00
Rebecca Turner 2a98ba8b97 Add pre-commit checks
The big ones here are `trim-trailing-whitespace` and `end-of-file-fixer`
(which makes sure that every file ends with exactly one newline
character).

Change-Id: Idca73b640883188f068f9903e013cf0d82aa1123
2024-03-29 22:57:40 -07:00
jade 1fa6a3e335 Fix various clang-tidy lints
* some things that can throw are marked noexcept
  yet the linter seems to think not. Maybe they can't throw in practice.
  I would rather not have the UB possibility in pretty obvious cold
  paths.
* various default-case-missing complaints
* a fair pile of casts from integer to character, which are in fact
  deliberate.
* an instance of <https://clang.llvm.org/extra/clang-tidy/checks/bugprone/move-forwarding-reference.html>
* bugprone-not-null-terminated-result on handing a string to curl in
  chunks of bytes. our usage is fine.
* reassigning a unique_ptr by CRIMES instead of using release(), then
  using release() and ignoring the result. wild. let's use release() for
  its intended purpose.

Change-Id: Ic3e7affef12383576213a8a7c8145c27e662513d
2024-03-29 20:26:38 -07:00
Rebecca Turner a5a25894c1 Move escapeString to its own file
Change-Id: Ie5c954ec73c46c9d3c679ef99a83a29cc7a08352
2024-03-29 16:26:29 -07:00
Winter Cute 6646b80396 meson: add missing explicit dependency on nlohmann_json
Without this, the Meson setup won't bail out if nlohmann_json is
missing, leading to subpar DX (and maybe worse, but I'm not entirely
sure).

Change-Id: I5913111060226b540dcf003257c99a08e84da0de
2024-03-29 14:16:58 -04:00
Rebecca Turner 877750b7c5 Merge "Move DebugChar into its own file" into main 2024-03-29 16:20:14 +00:00
eldritch horrors 86b954a7af meson: increase functional test timeout
sometimes these fail with timeouts on loaded machines. let's up the
timeouts until we can pull the tests apart to more reasonable sizes

Change-Id: I2dfff2183cc1f3ff5e6107f43748ac046fe00d05
2024-03-29 02:19:36 +00:00
Rebecca Turner 236bc046ba Merge "Remove HintFmt::operator%" into main 2024-03-29 01:13:45 +00:00
Rebecca Turner 5ec2efb686 Move DebugChar into its own file
Change-Id: Ia40549e5d0b78ece8dd0722c3a5a032b9915f24b
2024-03-28 15:54:12 -07:00
Rebecca Turner 62332c1250 Merge "Move shell_words into its own file" into main 2024-03-28 22:49:00 +00:00
Qyriad 038daad218 meson: implement functional tests
Functional tests can be run with
`meson test -C build --suite installcheck`.

Notably, functional tests must be run *after* running `meson install`
(Lix's derivation runs the installcheck suite in installCheckPhase so it
does this correctly), due to some quirks between Meson and the testing
system.

As far as I can tell the functional tests are meant to be run after
installing anyway, but unfortunately I can't transparently make
`meson test --suite installcheck` depend on the install targets.

The script that runs the functional tests, meson/run-test.py, checks
that `meson install` has happened and fails fast with a (hopefully)
helpful error message if any of the functional tests are run before
installing.

TODO: this change needs reflection in developer documentation

Change-Id: I8dcb5fdfc0b6cb17580973d24ad930abd57018f6
2024-03-27 18:37:50 -06:00
jade edba570664 HOT SALE: 15% off your build times!
This was achieved by running maintainers/buildtime_report.sh on the
build directory of a meson build, then asking "why the heck is json
eating our build times", and strategically moving the json using bits
out of widely included headers.

It turns out that putting literally any metrics whatsoever into the
build had immediate and predictable results.

Results are 1382.5s frontend time -> 1175.4s frontend time, back end
time approximately invariant.

Related: lix-project/lix#159

Change-Id: I7edea95c8536203325c8bb4dae5f32d727a21b2d
2024-03-27 03:52:57 +00:00
Rebecca Turner aee3d639b5 Move shell_words into its own file
Change-Id: I34c0ebfb6dcea49bf632d8880e04075335a132bf
2024-03-26 16:44:04 -07:00
Rebecca Turner 8e63eca912 Remove HintFmt::operator%
Change-Id: Ibcf1a7848b4b18ec9b0807628ff229079ae7a0fe
2024-03-26 15:40:05 -07:00
Qyriad 207d24da4e Merge "meson: implement unit tests" into main 2024-03-26 20:06:09 +00:00
Qyriad e1ffe56793 meson: implement unit tests
Unit tests can be run with `meson test -C build --suite check`.
`--suite check` is optional, as right now that's the only test suite,
but when functional tests are added those will be in a separate suite.

Change-Id: I7f22f1cde4b489b3cdb5f9a36a544f0c409fcc1f
2024-03-26 00:43:33 +01:00
Théophane Hufschmitt 86881226b0 Merge pull request #8817 from iFreilicht/flake-update-lock-overhaul
Overhaul `nix flake update` and `nix flake lock` UX

(cherry picked from commit 12a0ae73dbb37becefa5a442eb4532ff0de9ce65)
Change-Id: Iff3b4f4235ebb1948ec612036b39ab29e4ca22b2
2024-03-25 17:36:24 -06:00
Tom Bereknyei 4494f9097f feat: notation to refer to no attribute search prefix
An attrPath prefix of "." indicates no need to try default attrPath prefixes. For example `nixpkgs#legacyPackages.x86_64-linux.ERROR` searches through

```
trying flake output attribute 'packages.x86_64-linux.legacyPackages.x86_64-linux.ERROR'
using cached attrset attribute ''
trying flake output attribute 'legacyPackages.x86_64-linux.legacyPackages.x86_64-linux.ERROR'
using cached attrset attribute 'legacyPackages.x86_64-linux'
trying flake output attribute 'legacyPackages.x86_64-linux.ERROR'
using cached attrset attribute 'legacyPackages.x86_64-linux'
```

And there is no way to specify that one does not want the automatic
search behavior. Now one can specify
`nixpkgs#.legacyPackages.x86_64-linux.ERROR` to only refer to the rooted
attribute path without any default injection of attribute search path or
system.

Change-Id: Iac1334e1470137b7ce11dcf845513810230638ec
(cherry picked from commit d4aed18883b361133607296fb6cd789c47427a38)
2024-03-25 15:30:36 +00:00
Lunaphied 185ecf1f45 Improve new CLI UX by supporting short -E flag for --expr
Change-Id: I55881c846da8416a92a14deedfa5bbbf09a122fb
2024-03-24 21:17:51 -06:00
eldritch horrors afb839a0c9 libexpr: associate let exprs with the correct StaticEnv
static env association is from expr to its enclosing scope, but let
exprs set their association to their *inner* scope. this skips one level
of envs and will cause segfaults if the parent is a with expr.

fixes #145

Change-Id: I1d22146110f071ede21b4eed7ed34b5850ef2ef3
2024-03-18 14:15:22 -07:00
eldritch horrors b3599166ad libexpr: sort binding name in debugger
not doing this exposes the binding name order to the annoying
interference of parse order on symbol order, which wouldn't be so bad if
it didn't make the tests less reliable and, importantly, dependent on
linker behavior (due to primop initialization being done in static
initializer, and the order of static initializers being defined only
within a single translation unit).

fixes #143

Change-Id: I3cf417893fbcf19e9ad3ff8986deb7cbcf3ca511
2024-03-18 20:03:31 +01:00
jade 47a237f7ec Merge "Delete hasPrefix and hasSuffix from the codebase" into main 2024-03-18 12:01:39 -06:00
eldritch horrors 86a1121d16 use byte indexed locations for PosIdx
we now keep not a table of all positions, but a table of all origins and
their sizes. position indices are now direct pointers into the virtual
concatenation of all parsed contents. this slightly reduces memory usage
and time spent in the parser, at the cost of not being able to report
positions if the total input size exceeds 4GiB. this limit is not unique
to nix though, rustc and clang also limit their input to 4GiB (although
at least clang refuses to process inputs that are larger, we will not).

this new 4GiB limit probably will not cause any problems for quite a
while, all of nixpkgs together is less than 100MiB in size and already
needs over 700MiB of memory and multiple seconds just to parse. 4GiB
worth of input will easily take multiple minutes and over 30GiB of
memory without even evaluating anything. if problems *do* arise we can
probably recover the old table-based system by adding some tracking to
Pos::Origin (or increasing the size of PosIdx outright), but for time
being this looks like more complexity than it's worth.

since we now need to read the entire input again to determine the
line/column of a position we'll make unsafeGetAttrPos slightly lazy:
mostly the set it returns is only used to determine the file of origin
of an attribute, not its exact location. the thunks do not add
measurable runtime overhead.

notably this change is necessary to allow changing the parser since
apparently nothing supports nix's very idiosyncratic line ending choice
of "anything goes", making it very hard to calculate line/column
positions in the parser (while byte offsets are very easy).

(cherry picked from commit 5d9fdab3de0ee17c71369ad05806b9ea06dfceda)
Change-Id: Ie0b2430cb120c09097afa8c0101884d94f4bbf34
2024-03-18 16:12:46 +01:00