Related reading
What Does Privileged Information Add to On-Policy Self-Distillation?
The paper isolates the contribution of privileged teacher information in on‑policy self‑distillation (OPSD) using a 5k‑problem math suite. Reference‑free distillation explains most gains, with only modest extra benefit from full solution traces.
Hugging Face Daily Papersarxiv.org1 minpaperHow do we prevent mathemathics from devolving into the Medieval Era of secrecy?
A MathOverflow discussion argues that AI‑driven proof generation threatens the open, collaborative nature of mathematics, proposing human‑only attribution and mandatory disclosure of AI methodology as possible safeguards.
Hacker News front pagemathoverflow.net12 minHN157146Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation
The paper shows that standard multi-teacher on-policy distillation (MOPD) gives unbalanced updates, hurting specialist knowledge like math. By measuring each domain’s feedback spread and normalizing it (DN-MOPD), they rebalance updates, recover most math performance, and improve average scores on six benchmarks.
Hugging Face Daily Papersarxiv.org1 minpaperCalibrating Teacher--Student Discrepancy for On-Policy Distillation
This paper introduces Calibrated On-Policy Distillation (Cal-OPD), a method to improve knowledge distillation by filtering out the teacher model's inherent deviations from the learning signal. Cal-OPD estimates and removes the teacher's self-deviation, leading to better student performance on mathematical reasoning tasks.
Hugging Face Daily Papersarxiv.org1 minpaperIf math is more than proof, we need to better celebrate the rest of it
The author argues that mathematics should reward “motivated explanations” – narrative, intuition‑driven expositions that clarify why a theorem is interesting and how it fits into broader context – on par with traditional proof‑oriented work. He defines the concept, contrasts it with proofs, cites examples (Princeton Companion, Thurston’s essays, Chow’s exposition paper), and suggests formalizing…
Hacker News front pagewordpress.com12 minHN425284
