I came across https://github.com/NixOS/hydra/issues/751 and realized that hydra-notify is responsible for creating the additional jobsets in a declarative file. My declarative testing works in dev now.