Hugging Face Daily PapersHaojian Huang, Zexi Li, Junhao GuoSep 28, 20261 min readpaperadvanced

In-Context Learning for Robots: Methods and Applications

Summary

This literature review surveys In-Context Learning (ICL) methods for robots, which enable general-purpose robots to infer new task requirements from demonstrations and interaction. It categorizes ICL into four families based on how contextual evidence connects to execution, clarifying their transfer assumptions and roles of training, correspondence, and memory.

ICL for robots uses demonstrations and interaction to guide existing neural competence without parameter updates.
The review organizes ICL methods into four families: context-conditioned policies, geometric demonstration transfer, world-model-based control, and skill/agent-based execution.
It compares these interfaces by their transfer assumptions and the roles of training, correspondence, and memory.
The analysis links method design to evaluation practices for responsiveness, physical transfer, and benefits from retained experience.

Robotics researchers and ML engineers working on robot control will find this review useful for understanding the landscape and challenges of applying in-context learning to physical systems.

7/10

llm machine learning robotics in context learning survey

Related reading

In-Context Robot Learning with VLM Agents arxiv.org
GPT‑Policy is a framework that lets a large vision‑language model (e.g. GPT‑6 Astra) perform in‑context robot learning: a context compiler extracts visual transitions from demos, the VLM proposes tool actions, and a constrained controller verifies and executes them. Real‑robot experiments show that raw video demos improve success rates even without explicit action labels, and that providing align…
Hugging Face Daily Papersarxiv.orgDongzhou Cheng, Taoran Yi, Ye Fang20d1 minpaper
Convergent Emergence of In-Context Learning Across Modalities arxiv.org
The paper proposes the Convergent Emergence Hypothesis that few‑shot in‑context learning (ICL) shares a common difficulty profile across domains. Using a unified task suite, the authors evaluate ICL on six modalities—language, genome, integer sequences, time‑series, images, and proteins—showing that paired‑mapping ICL emerges in all and that per‑task benefits correlate across five modalities, sup…
Hugging Face Daily Papersarxiv.orgNathan Breslow, Seungwook Han, Daniel Hyunsoo Lee24d1 minpaper
GAVEL: Graph World Models for Verified and Efficient Long-Horizon LLM Task Planning arxiv.org
GAVEL augments LLM‑driven robot planners with an explicit graph world model that verifies actions, repairs violations, and reasons over belief distributions, boosting single‑task success from 41 % to 92 % and multi‑task success from 20 % to 93 % on BEHAVIOR‑1K.
Hugging Face Daily Papersarxiv.orgRuiyang Wang, Hao-Lun Hsu, Swarajh Mehta20d1 minpaper
From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention arxiv.org
The paper presents PARTS, a framework that augments a frozen pretrained robot policy with RL‑learned residuals on selected bottleneck subtasks, using local success rewards and minimal human resets. In real‑world bimanual and single‑arm tasks, PARTS more than doubles success rates with only minutes of robot rollouts, outperforming prior fine‑tuning methods.
Hugging Face Daily Papersarxiv.orgSichang Su, Benjamin Yang, Zhiyun Deng18d1 minpaper
Presentation: Context Engineering at LinkedIn: How We Built an Organizational Context Layer for AI Agents with MCP infoq.com
LinkedIn built a “Contextual Agent Playbooks and Tools” layer that wraps internal services (code search, docs, feature flags, etc.) behind the open‑source Model Context Protocol (MCP). By feeding LLM‑powered coding agents the exact internal artifacts they need, the agents can diagnose incidents, generate PRs, and update incident tickets in minutes, delivering a reported 20 % productivity gain wit…
InfoQinfoq.comAjay Prakash16d28 mintalk
Skill2Real: Agentic Skill Learning for Zero-Shot Sim-to-Real Robot Manipulation arxiv.org
Skill2Real is an agentic framework that learns robot manipulation skills in simulation via a Proposer‑Verifier‑Governor loop and a hierarchical Cerebellum‑Brain memory, then transfers them zero‑shot to real robots using a shared API. Experiments on LIBERO‑90 and Robosuite show up to ~79% success on real tasks without any real‑world fine‑tuning, and ablations confirm the Verifier and Governor are…
Hugging Face Daily Papersarxiv.orgXincheng He, Siyu Ma, Chang Yu4d1 minpaper

In-Context Learning for Robots: Methods and Applications

Summary

Related reading

In-Context Robot Learning with VLM Agentsarxiv.org

Convergent Emergence of In-Context Learning Across Modalitiesarxiv.org

GAVEL: Graph World Models for Verified and Efficient Long-Horizon LLM Task Planningarxiv.org

From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Interventionarxiv.org

Presentation: Context Engineering at LinkedIn: How We Built an Organizational Context Layer for AI Agents with MCPinfoq.com

Skill2Real: Agentic Skill Learning for Zero-Shot Sim-to-Real Robot Manipulationarxiv.org

In-Context Robot Learning with VLM Agents arxiv.org

Convergent Emergence of In-Context Learning Across Modalities arxiv.org

GAVEL: Graph World Models for Verified and Efficient Long-Horizon LLM Task Planning arxiv.org

From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention arxiv.org

Presentation: Context Engineering at LinkedIn: How We Built an Organizational Context Layer for AI Agents with MCP infoq.com

Skill2Real: Agentic Skill Learning for Zero-Shot Sim-to-Real Robot Manipulation arxiv.org