Search posts, papers, and topics
The paper shows that standard multi-teacher on-policy distillation (MOPD) gives unbalanced updates, hurting specialist knowledge like math. By measuring each domain’s feedback spread and normalizing it (DN-MOPD), they rebalance updates, recover most math performance, and improve average scores on six benchmarks.
The paper presents PMOPD, a projection-based method that tracks low-dimensional subspaces of task-specific parameter updates during multi-teacher on-policy distillation and removes interfering components. Experiments on Qwen2.5-7B and Llama-3.1-8B show consistent 2-point gains across code, reasoning, and math tasks.
On‑policy distillation of LLMs traditionally uses KL divergence, but this paper shows that merely aligning the update direction toward the teacher—via a simple (+1/‑1) token‑wise reward—achieves the same effect. Building on this, they introduce Consensus Multi‑Teacher OPD, which lets every sample learn from all teachers and consistently beats the prior single‑teacher approach on math and code ben…
