self distillation

What Does Privileged Information Add to On-Policy Self-Distillation?arxiv.org

The paper isolates the contribution of privileged teacher information in on‑policy self‑distillation (OPSD) using a 5k‑problem math suite. Reference‑free distillation explains most gains, with only modest extra benefit from full solution traces.