proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersQiangqiang He, Jin Li, MingCai Chen1 min readpaperadvanced

Calibrating Teacher--Student Discrepancy for On-Policy Distillation

Read on arxiv.org

Summary

This paper introduces Calibrated On-Policy Distillation (Cal-OPD), a method to improve knowledge distillation by filtering out the teacher model's inherent deviations from the learning signal. Cal-OPD estimates and removes the teacher's self-deviation, leading to better student performance on mathematical reasoning tasks.

  • Standard on-policy distillation (OPD) can inadvertently teach a student the teacher model's own inherent biases or "self-deviations."
  • Privileged information in OPD can exacerbate the problem of students learning teacher's self-deviations.
  • Cal-OPD estimates the teacher's self-deviation region using positive and negative privileged interventions.
  • It calibrates the teacher-student discrepancy by only using the optimization signal that lies beyond this self-deviation region.

Engineers and researchers working on knowledge distillation for large language models can leverage Cal-OPD to train more accurate and robust student models by focusing on genuine capability gaps.

7/10
llmmachine learningartificial intelligencereasoningknowledge distillation

Related reading

  1. When EOS Tokens Disagree: Understanding Length Inflation in On-Policy Distillation

    On-policy distillation (OPD) can lead to excessively long student responses, a phenomenon called length inflation. This paper identifies "termination-token mismatch" between base students and post-trained teachers as a key source, where models place stopping probability on different EOS tokens. Treating functionally equivalent EOS tokens as a shared semantic stopping action substantially mitigate…

    Hugging Face Daily Papersarxiv.org1 minpaper

  3. RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning

    RetireOPD introduces a self‑retiring on‑policy distillation framework for multi‑turn RL agents. A skill‑conditioned teacher is first trained with environment rewards, then a skill‑free student learns jointly via RL and token‑level distillation. The student automatically drops the teacher once its performance gap stops shrinking and it reaches a target success‑rate fraction, after which training c…

    Hugging Face Daily Papersarxiv.org1 minpaper

  4. Trajectory as the Teacher: Few-Step Discrete Flow Matching via Energy-Navigated Distillation

    Apple researchers propose Trajectory‑Shaped Discrete Flow Matching (TS‑DFM), a training‑time distillation method that replaces blind stochastic jumps in discrete flow‑matching with an energy‑based compass to select higher‑quality intermediate tokens. On a 170 M‑parameter language model, the 8‑step student outperforms the 1 024‑step teacher by 32 % perplexity while being 128× faster, beating basel…

    Apple Machine Learning Researchapple.com1 minpaper

  5. Training Specialist Models without Reasoning Trajectories for Domain Expert Distillation

    The paper shows that when specialist LLMs are trained only on QA pairs (no explicit reasoning supervision), their optimization implicitly selects a latent distribution of reasoning trajectories. By treating the distilled student as an agnostic probe—since it inherits only the sampled trajectories—the authors empirically demonstrate a strong correlation (across 27 specialist‑student pairs) between…

    Hugging Face Daily Papersarxiv.org1 minpaper

  6. The future of practice: Enabling teachers to create learning interactives with generative UI

    Google Research demonstrates a generative‑UI system that lets teachers author interactive, game‑style learning modules. The pipeline uses pedagogical guardrails, iterative self‑correcting loops (including automated solvability checks), and teacher vetting. A pilot library of 30+ STEM interactives is publicly available, and early teacher studies report good‑to‑excellent ratings.

    Google Researchresearch.google8 minHN4