Hugging Face Daily PapersTrung Dao, Sankalp Yamsani, Jaden ParkSep 21, 20261 min readpaperadvanced

Think Like a World Model, Act Like a VLA: Distilling World-Model Representations into Compact Robot Policies

Summary

Distill frozen world‑model features into a Vision‑Language‑Action policy via a single alignment loss; no teacher at train time, no extra runtime cost. 0.8 B student runs 32 ms / 1.86 GB on RTX 5090, hits 97.9 % on LIBERO and improves RoboCasa‑GR1 from 48.2 % to 50.5 %, with transfer to real single‑arm and bimanual robots.

Add a feature‑alignment term: run a frozen world model on training frames once, cache its internal features, and train the VLA student to match them.
The alignment projector is discarded after training; the deployed policy is identical to the baseline VLA, so speed and memory gains come solely from better representations.
Runtime: 32 ms per decision, 1.86 GB VRAM on a consumer RTX 5090 (no extra test‑time compute).
Performance: 0.8 B parameter student reaches 97.9 % on LIBERO; RoboCasa‑GR1 manipulation improves from 48.2 % to 50.5 %; same gains on real single‑arm and bimanual hardware.

World‑model grounding is usually too expensive for closed‑loop control. This work shows you can inherit the grounding via static feature alignment, keeping inference cheap while still reaping the robustness benefits—directly applicable to any VLA‑style robot policy.

8/10

performance robotics world models language vision policy distillation

Related reading

In-Context Robot Learning with VLM Agents arxiv.org
GPT‑Policy is a framework that lets a large vision‑language model (e.g. GPT‑6 Astra) perform in‑context robot learning: a context compiler extracts visual transitions from demos, the VLM proposes tool actions, and a constrained controller verifies and executes them. Real‑robot experiments show that raw video demos improve success rates even without explicit action labels, and that providing align…
Hugging Face Daily Papersarxiv.orgDongzhou Cheng, Taoran Yi, Ye Fang6d1 minpaper
HuRo: Robotizing Human Videos for Scalable VLA Pretraining arxiv.org
The paper introduces a pipeline that converts heterogeneous human videos into robot‑aligned observations and actions, creating the 630K‑episode HuRo dataset. Pretraining vision‑language‑action (VLA) policies on this data boosts real‑world manipulation success from ~51% to ~80% and improves out‑of‑distribution robustness.
Hugging Face Daily Papersarxiv.orgJinho Jeong, Se June Joo, Jaehyun Kang4d1 minpaper
Transferring the Intelligence of VLMs to Robotic Control arxiv.org
RoboDawn lets a vision‑language model (VLM) drive a robot via a tiny discrete command set (translate/rotate/gripper). Using a few in‑context demos, the VLM learns the interface and task strategy, then runs closed‑loop: observe image → reason → act → re‑observe. On the RoboTwin 2.0 C2R benchmark RoboDawn hits 53.2 % success zero‑shot, 73.6 % with one demo (vs. 46 % baseline). On RoboDojo it goes f…
Hugging Face Daily Papersarxiv.orgMeng-Hao Guo, Zhe-Han Mo, Jia-Jun Wang3d1 minpaper
GAVEL: Graph World Models for Verified and Efficient Long-Horizon LLM Task Planning arxiv.org
GAVEL augments LLM‑driven robot planners with an explicit graph world model that verifies actions, repairs violations, and reasons over belief distributions, boosting single‑task success from 41 % to 92 % and multi‑task success from 20 % to 93 % on BEHAVIOR‑1K.
Hugging Face Daily Papersarxiv.orgRuiyang Wang, Hao-Lun Hsu, Swarajh Mehta6d1 minpaper
From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention arxiv.org
The paper presents PARTS, a framework that augments a frozen pretrained robot policy with RL‑learned residuals on selected bottleneck subtasks, using local success rewards and minimal human resets. In real‑world bimanual and single‑arm tasks, PARTS more than doubles success rates with only minutes of robot rollouts, outperforming prior fine‑tuning methods.
Hugging Face Daily Papersarxiv.orgSichang Su, Benjamin Yang, Zhiyun Deng4d1 minpaper
ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models arxiv.org
Action tokenizers for autoregressive VLA models often fail to preserve subtle action adjustments, despite good pointwise reconstruction. This paper introduces Physical Rank Consistency (PRC) to measure relational fidelity and ActionPiece, a new tokenizer that uses joint supervision to preserve these physical relationships, significantly improving policy success on robotics benchmarks.
Hugging Face Daily Papersarxiv.orgShijie Lian, Bin Yu, Zhaolong Shen6d1 minpaper

Think Like a World Model, Act Like a VLA: Distilling World-Model Representations into Compact Robot Policies

Summary

Related reading

In-Context Robot Learning with VLM Agentsarxiv.org

HuRo: Robotizing Human Videos for Scalable VLA Pretrainingarxiv.org

Transferring the Intelligence of VLMs to Robotic Controlarxiv.org

GAVEL: Graph World Models for Verified and Efficient Long-Horizon LLM Task Planningarxiv.org

From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Interventionarxiv.org

ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Modelsarxiv.org

In-Context Robot Learning with VLM Agents arxiv.org

HuRo: Robotizing Human Videos for Scalable VLA Pretraining arxiv.org

Transferring the Intelligence of VLMs to Robotic Control arxiv.org

GAVEL: Graph World Models for Verified and Efficient Long-Horizon LLM Task Planning arxiv.org

From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention arxiv.org

ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models arxiv.org