on policy

1% of Tokens Can Be Enough: On Gradient Estimation in On-Policy Distillationarxiv.org

The paper studies gradient‑estimation noise in sparse on‑policy distillation (OPD) and introduces the Information‑Efficiency Ratio (IER), a signal‑to‑noise based metric for selecting which tokens to supervise. IER is derived from an information‑geometry analysis with an optimal scalar baseline, and a candidate‑set approximation lets it be combined with existing usefulness scores while keeping the…