linear rnns

Complex KDA: Understanding and Enhancing the Expressivity of Kimi Delta Attentionarxiv.org

Complex KDA (CKDA) extends Kimi Delta Attention by widening gate and beta ranges, allowing a single diagonal‑plus‑rank‑one transition to represent any orthogonal matrix, including 2‑D rotations. The paper proves this expressivity, keeps the model non‑expansive, and shows CKDA achieves state‑of‑the‑art length‑extrapolation and competitive language‑model performance.