Hugging Face Daily PapersHaixin Wang, Xiaoxuan Wang, Junkai ZhangSep 21, 20261 min readpaperadvanced

ACLArena: Agent Continue Learning in Multi-stage Post-training

Summary

The paper introduces ACLArena, a framework for studying Agent Continual Learning (ACL). It analyzes forgetting and generalization at model and token levels, compares multi‑teacher on‑policy distillation, self‑distilled fine‑tuning, and model merging, and proposes a new ACL recipe that combines offline replay of high‑quality trajectories with a routed network of LoRA experts trained via RL. Experi…

Sequential training pipelines for agents exhibit forgetting; analysis at model and token granularity explains why.
Multi‑teacher on‑policy distillation, self‑distilled fine‑tuning, and model merging each have distinct trade‑offs for preserving prior capabilities.
A hybrid ACL approach—offline replay + routed LoRA‑expert network—significantly reduces catastrophic forgetting while enabling new skill acquisition.
Empirical evaluation across four tasks demonstrates both in‑domain and out‑of‑domain generalization benefits.

Industrial agents must accumulate diverse skills over time without erasing earlier abilities. This work provides a systematic evaluation of existing ACL strategies and a concrete, reproducible recipe that improves multi‑stage learning, offering a practical path for deploying continually learning ag…

8/10

machine learning reinforcement learning continual learning lora agents

Related reading

RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning arxiv.org
RetireOPD introduces a self‑retiring on‑policy distillation framework for multi‑turn RL agents. A skill‑conditioned teacher is first trained with environment rewards, then a skill‑free student learns jointly via RL and token‑level distillation. The student automatically drops the teacher once its performance gap stops shrinking and it reaches a target success‑rate fraction, after which training c…
Hugging Face Daily Papersarxiv.orgYan Yu, Zhengxi Lu, Yizhou Liu6d1 minpaper
CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agents arxiv.org
CERA-MoA proposes a reinforcement‑learning loop where a router and a set of LLM agents are trained together. A “familiarity” estimator reads mid‑layer hidden states to predict each agent’s competence on a query, letting the router activate only a minimal subset of agents that meet a cumulative confidence threshold. The system also feeds targeted training examples to agents based on their evolving…
Hugging Face Daily Papersarxiv.orgJiaxuan Jiang, Liyuan He, Zhixuan Fang7d1 minpaper
Generalized Agent Iteration: One Formal Framework for Iterative Policy Improvement and Recursive Self-Improvement arxiv.org
The paper introduces Generalized Agent Iteration (GAI), a formal framework that unifies classical iterative policy improvement (GPI) and recursive self‑improvement (RSI). GAI treats an agent as a set of modifiable components and models learning as a loop of evaluation and improvement. Two binary “dials”—whether the improvement mechanism is internal to the agent and whether the evaluation standard…
Hugging Face Daily Papersarxiv.orgHongyao Tang, Yi Ma, Pengyi Li12d1 minpaper
Continual Learning Mechanisms Compose for Long-Horizon Memorization arxiv.org
The paper studies long‑horizon memorization where a language model must learn 100 tasks via continual fine‑tuning without retaining data or task IDs. By composing data, function, and weight anchors with merged LoRA, they boost final retention from 1.2% to 34.9%, a 28× improvement.
Hugging Face Daily Papersarxiv.orgZheyuan Zhang, Alvin Zhang, Daniel Khashabi16d1 minpaperHN2
In-Context Robot Learning with VLM Agents arxiv.org
GPT‑Policy is a framework that lets a large vision‑language model (e.g. GPT‑6 Astra) perform in‑context robot learning: a context compiler extracts visual transitions from demos, the VLM proposes tool actions, and a constrained controller verifies and executes them. Real‑robot experiments show that raw video demos improve success rates even without explicit action labels, and that providing align…
Hugging Face Daily Papersarxiv.orgDongzhou Cheng, Taoran Yi, Ye Fang7d1 minpaper
Designer-RSI: Evolving Procedural Memory from User Traffic for Agentic Graphic Design arxiv.org
The paper introduces Designer‑RSI, a continual‑adaptation system that couples a frozen design‑software‑controlling LLM with an external procedural memory of natural‑language design skills. Over five adaptation rounds on real user briefs, the memory grows from 76 to 139 procedures and lifts execution success from 72.7% to 99.3%, showing that skill accumulation and selective replay can dramatically…
Hugging Face Daily Papersarxiv.orgHongyang Du, Lan Yan, Christian Flores5d1 minpaper

ACLArena: Agent Continue Learning in Multi-stage Post-training

Summary

Related reading

RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learningarxiv.org

CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agentsarxiv.org

Generalized Agent Iteration: One Formal Framework for Iterative Policy Improvement and Recursive Self-Improvementarxiv.org

Continual Learning Mechanisms Compose for Long-Horizon Memorizationarxiv.org

In-Context Robot Learning with VLM Agentsarxiv.org

Designer-RSI: Evolving Procedural Memory from User Traffic for Agentic Graphic Designarxiv.org

RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning arxiv.org

CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agents arxiv.org

Generalized Agent Iteration: One Formal Framework for Iterative Policy Improvement and Recursive Self-Improvement arxiv.org

Continual Learning Mechanisms Compose for Long-Horizon Memorization arxiv.org

In-Context Robot Learning with VLM Agents arxiv.org

Designer-RSI: Evolving Procedural Memory from User Traffic for Agentic Graphic Design arxiv.org