Hugging Face Daily PapersAssaf Ben-Kish, Akarsh Kumar, James GlassOct 1, 20261 min readpaperadvanced

Local Support Learning

Summary

Local Support Learning (LSL) is a framework designed to mitigate catastrophic forgetting in large pre-trained models by augmenting gradient-based training. It uses a weight adapter and a GMM-based gating function to localize updates to the current training distribution, retaining prior capabilities without needing old data.

LSL addresses catastrophic forgetting by localizing weight updates to specific input distributions.
It pairs a standard weight adapter with a Gaussian Mixture Model (GMM)-based gating function.
The GMM gate enables the adapter only for its own training data, preserving prior knowledge.
LSL is a post-training approach, requiring no access to previous training data for retention.

Engineers and researchers working with large language models will find this relevant for continually updating models without losing previously learned knowledge, a critical challenge in ML.

8/10

llm machine learning ml infra continual learning deep learning

Related reading

ROSS: Relearning from Self-Generated Rollouts through Selective Supervision arxiv.org
ROSS is a method for large language model post-training that selectively supervises historical self-generated rollouts, applying loss only to useful continuations while preserving full trajectory context. It consistently improves LLM performance across various tasks like code generation and instruction following, achieving gains on Qwen3.6-35B-A3B without requiring new policy rollouts.
Hugging Face Daily Papersarxiv.orgZhiwei Zhang, Huayu Deng, Fei Zhao8d1 minpaper
Online Learning with LLM Experts from Limited Feedback arxiv.org
The paper models prompt routing to multiple LLM experts as a bandit problem with limited feedback and proposes algorithms that achieve sublinear regret in both full‑information and bandit settings. Experiments demonstrate that the methods learn effective routing strategies across diverse LLMs using only a small feedback budget.
Hugging Face Daily Papersarxiv.orgWang Wei, Soumyabrata Pal, Koyel Mukherjee1mo2 minpaper
Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data arxiv.org
The paper presents Infinite-Parameter LLMs, where a compact hypernetwork creates feed‑forward weights from live user data and updates a Bayesian latent code online, keeping the stored model size constant while effectively having infinite parameters. This design aims to improve over standard in‑context learning and retrieval by persisting knowledge in weights and freeing context space.
Hacker News front pagearxiv.orgJinli Hu, Ross M. Clarke, Yichuan Zhang, José Miguel Hernández-Lobato18d2 minpaperHN15743
Pivot-SD: Efficient Self-Distillation for Masked Diffusion Language Models arxiv.org
Pivot-SD is an efficient self-distillation framework for masked diffusion language models (dLMs) that addresses their credit-assignment problem by training only on high-impact "pivot" tokens. It improves LLaDA-8B-Instruct performance on math and code benchmarks using minimal data compared to other methods.
Hugging Face Daily Papersarxiv.orgSeo Hyun Kim, Sunwoo Hong, Younwoo Choi4d1 minpaper
MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulation arxiv.org
MotorMind is a robot manipulation harness that enables general-purpose Vision Language Models (VLMs) to perform zero-shot robot manipulation without task-specific training or external tools. It achieves significantly higher success rates on benchmark suites and real robots compared to prior zero-shot methods.
Hugging Face Daily Papersarxiv.orgBingxuan Li, Siqi Song, Yizhuo Wu7d1 minpaper
Breaking the Vision-Action Shortcut: Latent Interface Training for Generalizable Robotics Foundation Models arxiv.org
Latent Interface Training (LIT) first learns a goal‑conditioned action prior without visual input, then adds a pose‑supervised latent interface as the only visual conditioning path. Applied to several vision‑language‑action models, LIT cuts vision‑action shortcuts and lifts LIBERO‑Plus success by 3.9–10.7 points and real‑world task success by 13.3–16.7 points under distribution shifts.
Hugging Face Daily Papersarxiv.orgJianman Lin, Shailesh Shailesh, Zhongyi Luo25d1 minpaper

Local Support Learning

Summary

Related reading

ROSS: Relearning from Self-Generated Rollouts through Selective Supervisionarxiv.org

Online Learning with LLM Experts from Limited Feedbackarxiv.org

Infinite-Parameter LLMs: Generating and Adapting Weights from Live Dataarxiv.org

Pivot-SD: Efficient Self-Distillation for Masked Diffusion Language Modelsarxiv.org

MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulationarxiv.org

Breaking the Vision-Action Shortcut: Latent Interface Training for Generalizable Robotics Foundation Modelsarxiv.org

ROSS: Relearning from Self-Generated Rollouts through Selective Supervision arxiv.org

Online Learning with LLM Experts from Limited Feedback arxiv.org

Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data arxiv.org

Pivot-SD: Efficient Self-Distillation for Masked Diffusion Language Models arxiv.org

MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulation arxiv.org

Breaking the Vision-Action Shortcut: Latent Interface Training for Generalizable Robotics Foundation Models arxiv.org