Hugging Face Daily PapersLei Yang, Mengyin Liu, Jia Wang1 min readpaperadvanced
onPanda: Efficient Annotation of On-Policy Alignment Data for LLMs and Agents via Token-Level Correction
Summary
onPanda is an interactive annotation tool that lets humans correct LLM outputs token‑by‑token, then resumes generation from the corrected prefix. In a controlled study it cut median annotation time by 52% and the authors release a token‑level correction dataset (Panda‑CVL) for on‑policy fine‑tuning.
- Token‑level correction loop reduces median annotation time by 52% versus manual post‑editing.
- Most tokens remain model‑generated, preserving the original sampling distribution for on‑policy data.
- Recorded corrections give fine‑grained supervision with paired positive–negative samples.
- Tool integrates with external environments to annotate agent trajectories interactively.
Teams building LLM alignment pipelines can lower annotation costs and obtain richer, position‑aware supervision for on‑policy fine‑tuning.
7/10
