lix-project/lix

Fork 14

Meta-bug: `builtins.derivation` replacement design #217

New issue

Open

opened 2024-04-02 20:50:45 +00:00 by jade · 4 comments

jade commented

2024-04-02 20:50:45 +00:00

Owner

builtins.derivation has several intractable legacy problems:

Mixing env-vars with control attributes for the nixpkgs builder
Mixing control attributes for the nix builder with ones for the nixpkgs builder.
Control attributes (like __noChroot) don't say who they are for: are they for scheduling? are they for the builder daemon? are they secretly requests for some feature to be available to the builder? are they for cache upload attrs (e.g. "this is debuginfo, don't retain it as long").
structuredAttrs didn't really get adopted, which would fix parts of this.
builtins.derivationStrict has better semantics, but is not used in practice
There is no way to feature detect features of the builder to require that the thing is actually scheduled on a machine that can do it (though this might be intractable to fix, in general).

The current interface to it needs to be redesigned to specify better who everything is for, which then makes it much easier to figure out as well where it should go in the derivation format, etc, etc, etc.

It would not require significant (dangerous, at least) rearchitecture to get started on this issue, and would be preferable over adding anything to the list of special control attrs, which might be incompatible with upstream C++Nix.

Here is a pad (freely editable for everyone; consider writing who you are next to your comments) for working on a replacement API design: https://pad.lix.systems/lix-builtins-derivation

Related issues:

`builtins.derivation` has several intractable legacy problems: - Mixing env-vars with control attributes for the nixpkgs builder - Mixing control attributes for *the nix builder* with ones for the nixpkgs builder. - Control attributes (like `__noChroot`) don't say who they are for: are they for scheduling? are they for the builder daemon? are they secretly requests for some feature to be available to the builder? are they for cache upload attrs (e.g. "this is debuginfo, don't retain it as long"). - structuredAttrs didn't really get adopted, which would fix parts of this. - `builtins.derivationStrict` has better semantics, but is not used in practice - There is no way to feature detect features of the builder to require that the thing is actually scheduled on a machine that can do it (though this might be intractable to fix, in general). The current interface to it needs to be redesigned to specify better *who* everything is for, which then makes it much easier to figure out as well where it should go in the derivation format, etc, etc, etc. It would not require significant (dangerous, at least) rearchitecture to get started on this issue, and would be preferable over adding anything to the list of special control attrs, which might be incompatible with upstream C++Nix. Here is a pad (freely editable for everyone; consider writing who you are next to your comments) for working on a replacement API design: https://pad.lix.systems/lix-builtins-derivation Related issues: - https://git.lix.systems/lix-project/lix/issues/121 - https://git.lix.systems/lix-project/lix/issues/84 - https://git.lix.systems/lix-project/lix/issues/73 - https://git.lix.systems/lix-project/lix/issues/63

jade added the

RFD

label 2024-04-02 20:50:50 +00:00

pennae commented

2024-04-02 20:57:31 +00:00

Owner

comments of note:

derivationStrict is used, it's the backend for derivation and has all the same problems
structuredAttrs are very slowly gaining traction, having nixpkgs folks push in that direction may be helpful

comments of note: - `derivationStrict` *is* used, it's the backend for `derivation` and has all the same problems - `structuredAttrs` are *very* slowly gaining traction, having nixpkgs folks push in that direction may be helpful

jade commented

2024-04-02 21:03:06 +00:00

Author

Owner

Right, but the semantics of derivationStrict are better (according to @puck at least, iwrc), and so its lack of direct use is sort of strange, and the fact that builtins.derivation as a wrapper exists is also kind of strange.

Right, but the semantics of `derivationStrict` are better (according to @puck at least, iwrc), and so its lack of direct use is sort of strange, and the fact that `builtins.derivation` as a wrapper exists is also kind of strange.

pennae commented

2024-04-02 21:05:51 +00:00

Owner

derivation x is basically x: let s = derivationStrict x; in x // { type = "derivation"; inherit (s) outPath drvPath; } with some sugar coat. it has all the same problems as derivation by virtue of being the problem with derivation; there are some semantic bits in the strict variant that are pleasant but they're unrelated to the problem being discussed here

`derivation x` is *basically* `x: let s = derivationStrict x; in x // { type = "derivation"; inherit (s) outPath drvPath; }` with some sugar coat. it has all the same problems as `derivation` by virtue of *being* the problem with `derivation`; there are some semantic bits in the strict variant that are pleasant but they're unrelated to the problem being discussed here

jade referenced this issue

2024-05-05 04:23:53 +00:00

Do something about `allowNewPrivileges` setting #265

roberth commented

2024-05-29 21:57:30 +00:00

These points largely agree with nix#9774. (Ignoring the bit about meta perhaps)
I'd like to keep in touch about this, and perhaps collaborate?

Also I was very close to making Nixpkgs use derivationStrict directly in nixpkgs#217243 cleanAttrs, before deciding not to take that broader direction. I believe it should be fine for Nixpkgs to do that still. It'd improve performance slightly.

These points largely agree with [nix#9774](https://github.com/NixOS/nix/issues/9774). (Ignoring the bit about `meta` perhaps) I'd like to keep in touch about this, and perhaps collaborate? Also I was very close to making Nixpkgs use `derivationStrict` directly in [nixpkgs#217243 cleanAttrs](https://github.com/NixOS/nixpkgs/pull/217243), before deciding not to take that broader direction. I believe it should be fine for Nixpkgs to do that still. It'd improve performance slightly.

No milestone

No project

No assignees

3 participants

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: lix-project/lix#217

No description provided.

Meta-bug: builtins.derivation replacement design #217

Meta-bug: `builtins.derivation` replacement design #217