Error: 'dynamic attribute already defined' when incrementally building nested structures with dynamic paths

dwt commented

2025-02-28 08:52:51 +00:00

Describe the bug

Inconsistent handling of attribute paths containing dynamic components (${...}): When an attribute path includes a dynamic component, Nix treats each occurrence as a complete redefinition of that attribute, rather than merging nested attributes as it does with static attribute paths. This makes it impossible to build up nested structures incrementally when any part of the path is dynamic.

This appears to be a limitation in Nix's handling of dynamic attribute paths that prevents a common pattern that works fine with static paths.

Steps To Reproduce

See the gist at https://gist.github.com/dwt/c7fef22b64bae7735eca8cf28efd3785

The issue is with how Nix handles dynamic attribute paths (${...}) at any level in an attribute set, compared to static attribute paths.

repro-first.nix (not shown but error referenced):

let
  system = "aarch64-darwin";
in
{
  ${system}.foo = "foo";
  ${system}.bar = "bar";  # Error: dynamic attribute 'aarch64-darwin' already defined
}

repro-second.nix:

let
  system = "aarch64-darwin";
in
{
  foo.${system}.foo = "foo";
  foo.${system}.bar = "bar";  # Error: dynamic attribute 'aarch64-darwin' already defined
}

fine.nix (works correctly):

{
  foo.foo = "foo";
  foo.bar = "bar";  # No error, attributes are merged properly
}

Expected behavior

I would have expected there to be no difference in evaluation of static and dynamic attribute paths and that it changes nothing how deep attribute sets are merged.

`nix --version` output

❯ nix --version
nix (Lix, like Nix) 2.92.0
System type: aarch64-darwin
Additional system types: aarch64-darwin, x86_64-darwin
Features: gc, signed-caches
System configuration file: /etc/nix/nix.conf
User configuration files: /Users/dwt/.config/nix/nix.conf:/Users/dwt/.nix-profile/etc/xdg/nix/nix.conf:/run/current-system/sw/etc/xdg/nix/nix.conf:/nix/var/nix/profiles/default/etc/xdg/nix/nix.conf
Store directory: /nix/store
State directory: /nix/var/nix
Data directory: /nix/store/vs6s56c7crvj75fmmyrc4f1wh4q7pbns-lix-2.92.0/share

Additional context

Please request any additional context that could help you debug this.

## Describe the bug Inconsistent handling of attribute paths containing dynamic components (`${...}`): When an attribute path includes a dynamic component, Nix treats each occurrence as a complete redefinition of that attribute, rather than merging nested attributes as it does with static attribute paths. This makes it impossible to build up nested structures incrementally when any part of the path is dynamic. This appears to be a limitation in Nix's handling of dynamic attribute paths that prevents a common pattern that works fine with static paths. ## Steps To Reproduce 1. See the gist at https://gist.github.com/dwt/c7fef22b64bae7735eca8cf28efd3785 The issue is with how Nix handles dynamic attribute paths (`${...}`) at any level in an attribute set, compared to static attribute paths. 1. `repro-first.nix` (not shown but error referenced): ```nix let system = "aarch64-darwin"; in { ${system}.foo = "foo"; ${system}.bar = "bar"; # Error: dynamic attribute 'aarch64-darwin' already defined } ``` 2. `repro-second.nix`: ```nix let system = "aarch64-darwin"; in { foo.${system}.foo = "foo"; foo.${system}.bar = "bar"; # Error: dynamic attribute 'aarch64-darwin' already defined } ``` 3. `fine.nix` (works correctly): ```nix { foo.foo = "foo"; foo.bar = "bar"; # No error, attributes are merged properly } ``` ## Expected behavior I would have expected there to be no difference in evaluation of static and dynamic attribute paths and that it changes nothing how deep attribute sets are merged. ## `nix --version` output ```shell ❯ nix --version nix (Lix, like Nix) 2.92.0 System type: aarch64-darwin Additional system types: aarch64-darwin, x86_64-darwin Features: gc, signed-caches System configuration file: /etc/nix/nix.conf User configuration files: /Users/dwt/.config/nix/nix.conf:/Users/dwt/.nix-profile/etc/xdg/nix/nix.conf:/run/current-system/sw/etc/xdg/nix/nix.conf:/nix/var/nix/profiles/default/etc/xdg/nix/nix.conf Store directory: /nix/store State directory: /nix/var/nix Data directory: /nix/store/vs6s56c7crvj75fmmyrc4f1wh4q7pbns-lix-2.92.0/share ``` ## Additional context Please request any additional context that could help you debug this.

dwt added the

bug

label

2025-02-28 08:52:51 +00:00

dwt commented

2025-02-28 08:55:08 +00:00

Author

The other nix implementation

❯ nix run nixpkgs#nix -- --version
nix (Nix) 2.24.12

shows the same errors, so this is not just a lix issue it seems.

The other nix implementation ```shell ❯ nix run nixpkgs#nix -- --version nix (Nix) 2.24.12 ``` shows the same errors, so this is not just a lix issue it seems.

pennae commented

2025-02-28 14:39:21 +00:00

Owner

I would have expected there to be no difference in evaluation of static and dynamic attribute paths and that it changes nothing how deep attribute sets are merged.

that's a reasonable assumption, but unfortunately it's impossible. the current behavior is best understood as a elementwise set merge with disjointness checking. this is far from the only case of dynamic attributes being completely fucked up; consider (a: rec { b = a; ${a} = "2"; }) "a".

at this point we personally are more likely to throw out dynamic attributes completely in a language revision than do anything with them except emit a warning when they're used, they're such a mine field.

> I would have expected there to be no difference in evaluation of static and dynamic attribute paths and that it changes nothing how deep attribute sets are merged. that's a reasonable assumption, but unfortunately it's impossible. the current behavior is best understood as a elementwise set merge with disjointness checking. this is far from the only case of dynamic attributes being completely fucked up; consider `(a: rec { b = a; ${a} = "2"; }) "a"`. at this point we personally are more likely to throw out dynamic attributes completely in a language revision than do *anything* with them except emit a warning when they're used, they're such a mine field.

dwt commented

2025-02-28 14:47:10 +00:00

Author

@pennae nix repl gives me this answer:

nix-repl> (a: rec { b = a; ${a} = "2"; }) "a"
{
  a = "2";
  b = "a";
}

Which seems to be exactly what I would expect from an imperative language?

b is defined to the value of a - at that time that is the variable from outside the rec attrset
then a is defined to be "2"

I am guessing that the way rec is defined plus laziness, you would have expected the result to be:

nix-repl> (a: rec { b = a; ${a} = "2"; }) "a"
{
  a = "2";
  b = "2";
}

@pennae `nix repl` gives me this answer: ```nix nix-repl> (a: rec { b = a; ${a} = "2"; }) "a" { a = "2"; b = "a"; } ``` Which seems to be exactly what I would expect from an imperative language? - `b` is defined to the value of `a` - at that time that is the variable from outside the rec attrset - then `a` is defined to be `"2"` I am guessing that the way rec is defined plus laziness, you would have expected the result to be: ```nix nix-repl> (a: rec { b = a; ${a} = "2"; }) "a" { a = "2"; b = "2"; } ```

pennae commented

2025-02-28 15:16:10 +00:00

Owner

nixlang is not imperative though. rec is supposed to add add attributes of the current set to the scope of values inside the set, but for dynamic attributes this only happens in one direction: dynamics see statics, but not the other way around. this is why dynamic attributes have been forbidden in let blocks since forever, and it's also why we can't meaningfully fix anything about dynamic attributes :(

the merges you're talking about are unfixable without breaking the language. consider

(x: { a.b = builtins.abort "from stdenv somewhere probably"; ${x}.b.c = 1; }) "a"

merging of statics is side-effect-free. merging of dynamics cannot be, and must thus not be done.

nixlang is not imperative though. `rec` is supposed to add *add* attributes of the current set to the scope of values inside the set, but for dynamic attributes this only happens in one direction: dynamics see statics, but not the other way around. this is why dynamic attributes have been forbidden in `let` blocks since forever, and it's also why we can't meaningfully fix anything about dynamic attributes :( the merges you're talking about are unfixable without breaking the language. consider ``` (x: { a.b = builtins.abort "from stdenv somewhere probably"; ${x}.b.c = 1; }) "a" ``` merging of statics is side-effect-free. merging of dynamics cannot be, and must thus not be done.

dwt commented

2025-02-28 15:24:31 +00:00

Author

Well, I guess I understand that the nix language does not lend itself to fixing this inconsistency. Would it perhaps make sense to explain this problem in the error message shown or link to a page that explains that in more details? Because from the outside it just looks like there is a bug in nix/lix.

pennae commented

2025-02-28 15:28:07 +00:00

Owner

sure, error messages can always be improved! if you have a good idea on how to make them less confusing we're always happy the review CLs to make UX (or documentation) less horrible :)

dwt commented

2025-02-28 15:31:55 +00:00

Author

What do you think of something along the lines of:

nix-repl> (x: { a.b = builtins.abort "from stdenv somewhere probably"; ${x}.b.c = 1; }) "a"
error: dynamic attribute 'a' already defined at «string»:1:7 See lix.systems/e/dynamic-attributes-merging
       at «string»:1:62:
            1| (x: { a.b = builtins.abort "from stdenv somewhere probably"; ${x}.b.c = 1; }) "a"
             |                                                              ^

That's very similar to what sqlalchemy does, which I like a lot, as the target page can be evolved independently from the deployed application.

What do you think of something along the lines of: ```nix nix-repl> (x: { a.b = builtins.abort "from stdenv somewhere probably"; ${x}.b.c = 1; }) "a" error: dynamic attribute 'a' already defined at «string»:1:7 See lix.systems/e/dynamic-attributes-merging at «string»:1:62: 1| (x: { a.b = builtins.abort "from stdenv somewhere probably"; ${x}.b.c = 1; }) "a" | ^ ``` That's very similar to what sqlalchemy does, which I like a lot, as the target page can be evolved independently from the deployed application.

pennae commented

2025-02-28 15:48:54 +00:00

Owner

that feels a bit wrong somehow. more information is good, but more information as a pure "go to this website" is a bit ornery. it'd be great if we could give that error a unique error number that's easily looked up in the manual, and also link to the manual. giving a more complete explanation of why merging isn't happening in the actuall error message would be even better though; lix error traces are already infuriating and asking users to get out a browser to see the full message won't make that less so :(

we'd say

nix-repl> (x: { a.b = builtins.abort "from stdenv somewhere probably"; ${x}.b.c = 1; }) "a"
error: dynamic attribute 'a' already defined at «string»:1:7. Dynamic attributes do not merge with other attributes, including dynamic attributes that evaluate to the same name. 
       at «string»:1:62:
            1| (x: { a.b = builtins.abort "from stdenv somewhere probably"; ${x}.b.c = 1; }) "a"
             |                                                              ^

the dynamic attribute documentation in the manual could also use a rewrite. the entire manual could also use a rewrite. 🫠

that feels a bit wrong somehow. more information is good, but more information as a pure "go to this website" is a bit ornery. it'd be great if we could give that error a unique error number that's easily looked up in the manual, and *also* link to the manual. giving a more complete explanation of why merging isn't happening in the actuall error message would be even better though; lix error traces are already infuriating and asking users to get out a browser to see the full message won't make that less so :( we'd say ``` nix-repl> (x: { a.b = builtins.abort "from stdenv somewhere probably"; ${x}.b.c = 1; }) "a" error: dynamic attribute 'a' already defined at «string»:1:7. Dynamic attributes do not merge with other attributes, including dynamic attributes that evaluate to the same name. at «string»:1:62: 1| (x: { a.b = builtins.abort "from stdenv somewhere probably"; ${x}.b.c = 1; }) "a" | ^ ``` the dynamic attribute documentation in the manual could also use a rewrite. the entire manual could also use a rewrite. 🫠

dwt commented

2025-02-28 15:51:32 +00:00

Author

:-) Yeah... Much could…

I do like the error message you proposed, it's a simple statement of fact. Not why, but that is something the manual could (in a later rewrite) do.

:-) Yeah... Much could… I do like the error message you proposed, it's a simple statement of fact. Not why, but that is something the manual could (in a later rewrite) do.

jade added the

Area/language

label

2025-02-28 16:06:48 +00:00

jade commented

2025-02-28 16:12:24 +00:00

Owner

note there's definitely other complaints about attr merging behaviour on the tracker, i think at least one of them is tagged "needs langver". so it seems like this is a common confusion.

I would love error codes to happen. if you want to help out, a well written docs page describing the attr merging could be put into a section on error messages and it could be a start on the error codes project. docs writing is a big gap area in the project.

note there's definitely other complaints about attr merging behaviour on the tracker, i think at least one of them is tagged "needs langver". so it seems like this is a common confusion. I would *love* error codes to happen. if you want to help out, a well written docs page describing the attr merging could be put into a section on error messages and it could be a start on the error codes project. docs writing is a big gap area in the project.

👍 1

piegames commented

2025-02-28 17:17:23 +00:00

Member

Dynamic attributes are on my sniping list for deprecation, especially in the more cursed edge cases involving rec. Actually fixing things unfortunately requires a langver bump most of the time, but throwing out syntax with unfortunate semantics is always an option

Dynamic attributes are on my sniping list for deprecation, especially in the more cursed edge cases involving `rec`. Actually *fixing* things unfortunately requires a langver bump most of the time, but throwing out syntax with unfortunate semantics is always an option

👍 1

Rows
Columns

Error: 'dynamic attribute already defined' when incrementally building nested structures with dynamic paths #707

Describe the bug

Steps To Reproduce

Expected behavior

nix --version output

Additional context

`nix --version` output