Move setting and builtin documentation that is currently in C++ to .md files; make setting defaults generate C++?

jade commented

2024-05-09 23:15:01 +00:00

Owner

We currently have a Pawblem: Lix documentation doesn't like to be built in cross, since the documentation needs to be extracted by running a C++ program. This is backwards, we should put the docs into another file, which we have a simple program to generate C++ from (maybe the .gen.hh things, but we actually think we might want more cleverness), and then generate the docs forward from that.

Probably we can use the same kind of machinery and the same python libs as release notes and shove the metadata in yaml front-matter blocks, then generate simple C++ from that, including generating defaults so they don't go out of date?

Concept:

---
name: pre-build-hook
default: ""
default-text: "pre-build-hook is not used by default"
---

If set, the path to a program that can set extra derivation-specific
settings for this system. This is used for settings that can't be
captured by the derivation model itself and are too variable between
different versions of the same system to be hard-coded into nix.

The hook is passed the derivation path and, if sandboxes are
enabled, the sandbox directory. It can then modify the sandbox and
send a series of commands to modify various settings to stdout. The
currently recognized commands are:

- `extra-sandbox-paths`\
  Pass a list of files and directories to be included in the
  sandbox for this build. One entry per line, terminated by an
  empty line. Entries have the same format as `sandbox-paths`.

and then this would be used as:

Setting<std::string> preBuildHook{
#include "settings/pre-build-hook.gen.hh"
};

We currently have a Pawblem: Lix documentation doesn't like to be built in cross, since the documentation needs to be extracted by running a C++ program. This is backwards, we should put the docs into another file, which we have a simple program to generate C++ from (*maybe* the .gen.hh things, but we actually think we might want more cleverness), and then generate the docs forward from that. Probably we can use the same kind of machinery and the same python libs as release notes and shove the metadata in yaml front-matter blocks, then generate simple C++ from that, including generating defaults so they don't go out of date? Concept: ```md --- name: pre-build-hook default: "" default-text: "pre-build-hook is not used by default" --- If set, the path to a program that can set extra derivation-specific settings for this system. This is used for settings that can't be captured by the derivation model itself and are too variable between different versions of the same system to be hard-coded into nix. The hook is passed the derivation path and, if sandboxes are enabled, the sandbox directory. It can then modify the sandbox and send a series of commands to modify various settings to stdout. The currently recognized commands are: - `extra-sandbox-paths`\ Pass a list of files and directories to be included in the sandbox for this build. One entry per line, terminated by an empty line. Entries have the same format as `sandbox-paths`. ``` and then this would be used as: ```c++ Setting<std::string> preBuildHook{ #include "settings/pre-build-hook.gen.hh" }; ```

❤️ 3

jade added the

Area/build-packaging

label

2024-05-09 23:15:01 +00:00

jade commented

2024-07-31 21:06:48 +00:00

Author

Owner

I have written a clang query to pull that data out of there, which someone could use later to actually execute on writing a migration using our clang-tidy/.

set print-matcher true
set traversal IgnoreUnlessSpelledInSource
enable output detailed-ast

l settings cxxConstructExpr(hasType(hasUnqualifiedDesugaredType(recordType(hasDeclaration(namedDecl(hasName("nix::Setting")))))))

Usage, with a non-Nix clang-query (due to https://github.com/NixOS/nixpkgs/pull/300157):

/usr/bin/clang-query -p build/compile_commands.json src/libfetchers/fetch-settings.hh --preload=test.query

clang-query> m settings
...
Match #8:

/home/jade/lix/lix3/src/libfetchers/fetch-settings.hh:111:47: note: "root" binds here
  111 |     Setting<std::string> commitLockFileSummary{
      |                                               ^
  112 |         this, "", "commit-lockfile-summary",
      |         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  113 |         R"(
      |         ~~~
  114 |           The commit summary to use when committing changed flake lock files. If
      |           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  115 |           empty, the summary is generated based on the action performed.
      |           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  116 |         )",
      |         ~~~
  117 |         {}, true, Xp::Flakes};
      |         ~~~~~~~~~~~~~~~~~~~~~
Binding for "root":
CXXConstructExpr 0x797fd5bc9990 </home/jade/lix/lix3/src/libfetchers/fetch-settings.hh:111:47, line:11
7:29> 'Setting<std::string>':'nix::Setting<std::basic_string<char>>' 'void (Config *, const std::basic
_string<char> &, const std::string &, const std::string &, const std::set<std::string> &, const bool, 
std::optional<ExperimentalFeature>)' list
|-CXXThisExpr 0x797fd5bc63e8 <line:112:9> 'nix::FetchSettings *' this
|-StringLiteral 0x797fd5bc63f8 <col:15> 'const char[1]' lvalue ""
|-StringLiteral 0x797fd5bc6410 <col:19> 'const char[24]' lvalue "commit-lockfile-summary"
|-StringLiteral 0x797fd5bc64c0 <line:113:9> 'const char[164]' lvalue "\n          The commit summary t
o use when committing changed flake lock files. If\n          empty, the summary is generated based on
 the action performed.\n        "
|-CXXConstructExpr 0x797fd5bc81a8 <line:117:9, col:10> 'const std::set<std::string>':'const std::set<s
td::basic_string<char>>' 'void () noexcept' list zeroing
|-CXXBoolLiteralExpr 0x797fd5bc65c0 <col:13> 'bool' true
`-DeclRefExpr 0x797fd5bc6608 <col:19, col:23> 'nix::ExperimentalFeature' EnumConstant 0x797fd5dbdbf8 '
Flakes' 'nix::ExperimentalFeature'
  `-NestedNameSpecifier TypeSpec 'nix::Xp':'nix::ExperimentalFeature'

cc @alois31

I have written a clang query to pull that data out of there, which someone could use later to actually execute on writing a migration using our `clang-tidy/`. ![image](/attachments/5c7de884-6642-4d40-892d-e71e432c43ef) ``` set print-matcher true set traversal IgnoreUnlessSpelledInSource enable output detailed-ast l settings cxxConstructExpr(hasType(hasUnqualifiedDesugaredType(recordType(hasDeclaration(namedDecl(hasName("nix::Setting"))))))) ``` Usage, with a non-Nix clang-query (due to https://github.com/NixOS/nixpkgs/pull/300157): ``` /usr/bin/clang-query -p build/compile_commands.json src/libfetchers/fetch-settings.hh --preload=test.query clang-query> m settings ... Match #8: /home/jade/lix/lix3/src/libfetchers/fetch-settings.hh:111:47: note: "root" binds here 111 | Setting<std::string> commitLockFileSummary{ | ^ 112 | this, "", "commit-lockfile-summary", | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 113 | R"( | ~~~ 114 | The commit summary to use when committing changed flake lock files. If | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 115 | empty, the summary is generated based on the action performed. | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 116 | )", | ~~~ 117 | {}, true, Xp::Flakes}; | ~~~~~~~~~~~~~~~~~~~~~ Binding for "root": CXXConstructExpr 0x797fd5bc9990 </home/jade/lix/lix3/src/libfetchers/fetch-settings.hh:111:47, line:11 7:29> 'Setting<std::string>':'nix::Setting<std::basic_string<char>>' 'void (Config *, const std::basic _string<char> &, const std::string &, const std::string &, const std::set<std::string> &, const bool, std::optional<ExperimentalFeature>)' list |-CXXThisExpr 0x797fd5bc63e8 <line:112:9> 'nix::FetchSettings *' this |-StringLiteral 0x797fd5bc63f8 <col:15> 'const char[1]' lvalue "" |-StringLiteral 0x797fd5bc6410 <col:19> 'const char[24]' lvalue "commit-lockfile-summary" |-StringLiteral 0x797fd5bc64c0 <line:113:9> 'const char[164]' lvalue "\n The commit summary t o use when committing changed flake lock files. If\n empty, the summary is generated based on the action performed.\n " |-CXXConstructExpr 0x797fd5bc81a8 <line:117:9, col:10> 'const std::set<std::string>':'const std::set<s td::basic_string<char>>' 'void () noexcept' list zeroing |-CXXBoolLiteralExpr 0x797fd5bc65c0 <col:13> 'bool' true `-DeclRefExpr 0x797fd5bc6608 <col:19, col:23> 'nix::ExperimentalFeature' EnumConstant 0x797fd5dbdbf8 ' Flakes' 'nix::ExperimentalFeature' `-NestedNameSpecifier TypeSpec 'nix::Xp':'nix::ExperimentalFeature' ``` cc @alois31

image.png

125 KiB

alois31 self-assigned this

2024-08-03 16:36:56 +00:00

lix-bot commented

2024-08-21 16:17:41 +00:00

Member

This issue was mentioned on Gerrit on the following CLs:

commit message in cl/1831 ("treewide: generate global settings from data")
commit message in cl/1915 ("libutil: generate experimental and deprecated features from data")

This issue was mentioned on Gerrit on the following CLs: * commit message in [cl/1831](https://gerrit.lix.systems/c/lix/+/1831) ("treewide: generate global settings from data") * commit message in [cl/1915](https://gerrit.lix.systems/c/lix/+/1915) ("libutil: generate experimental and deprecated features from data")

lix-project referenced this issue from a commit

2024-11-10 10:19:41 +00:00

libutil: generate experimental and deprecated features from data

lix-project referenced this issue from a commit

2025-01-29 18:11:13 +00:00

Merge pull request #292 from nix-community/renovate/lock-file-maintenance

pennae commented

2025-03-08 13:29:18 +00:00

Owner

@alois31 you worked on this, is there something missing or should we consider this finished?

alois31 commented

2025-03-08 13:32:55 +00:00

Member

The new CLI is the last thing that is not ported yet:

 nix3_cli_json = custom_target(
   command : [ nix, '__dump-cli' ],
   capture : true,
   output : 'nix.json',
   env : nix_env_for_docs,
   # FIXME: put the actual lib targets in here? meson have introspection challenge 2024 though.
   build_always_stale : true,
 )

The new CLI is the last thing that is not ported yet: https://git.lix.systems/lix-project/lix/src/commit/93c3ca4e92b8cd1a129498f4c3f4c48558032d46/doc/manual/meson.build#L18-L25

commentatorforall commented

2025-12-19 17:42:58 +00:00

Member

this still open or fully resolved by now?

commentatorforall added the

awaiting

author

label

2025-12-19 17:43:04 +00:00

raito commented

2025-12-22 08:28:42 +00:00

Owner

I cannot find a __dump-cli anymore, it feels like to me this is actually finished, @alois31 can we let you close if you agree with this?

I cannot find a `__dump-cli` anymore, it feels like to me this is actually finished, @alois31 can we let you close if you agree with this?

alois31 commented

2025-12-22 16:22:33 +00:00

Member

It's still there without changes:

 nix3_cli_json = custom_target(
   command : [ nix, '__dump-cli' ],
   capture : true,
   output : 'nix.json',
   env : nix_env_for_docs,
   # FIXME: put the actual lib targets in here? meson have introspection challenge 2024 though.
   build_always_stale : true,
 )

In any case https://gerrit.lix.systems/c/lix/+/4676 unspaghettifies the code generator (thank you!) so any changes will be based on this. Then the stores seem manageable but the subcommands are a significantly larger mess due to being more dynamic. Sorry for dropping the ball on it, I will see what can be done in the next few weeks.

It's still there without changes: https://git.lix.systems/lix-project/lix/src/commit/8f27c09a08f6430f253ee93f35a519e42da19832/doc/manual/meson.build#L19-L26 In any case https://gerrit.lix.systems/c/lix/+/4676 unspaghettifies the code generator (thank you!) so any changes will be based on this. Then the stores seem manageable but the subcommands are a significantly larger mess due to being more dynamic. Sorry for dropping the ball on it, I will see what can be done in the next few weeks.

👍 1