proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersSichang Su, Benjamin Yang, Zhiyun Deng1 min readpaperadvanced

From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention

Read on arxiv.org

Summary

The paper presents PARTS, a framework that augments a frozen pretrained robot policy with RL‑learned residuals on selected bottleneck subtasks, using local success rewards and minimal human resets. In real‑world bimanual and single‑arm tasks, PARTS more than doubles success rates with only minutes of robot rollouts, outperforming prior fine‑tuning methods.

  • PARTS keeps a frozen pretrained policy for nominal actions and learns residual corrections only on identified bottleneck subtasks via RL.
  • Local success verifiers provide subtask rewards, enabling learning despite sparse overall task success and requiring few human resets.
  • Experiments on a bimanual YAM and single‑arm Franka raise full‑task success from 32%→61% and 50%→95% with tens of minutes of real‑world rollouts per task.
  • Compared to existing real‑world RL fine‑tuning, PARTS gains >25% higher success under the same rollout budget while reducing human involvement.

Robotics engineers building long‑horizon manipulation pipelines will care because it shows how to efficiently fine‑tune foundation policies on real robots with limited human effort.

8/10
reinforcement learningroboticsmanipulationreal world rl

Related reading

  2. In-Context Robot Learning with VLM Agents

    GPT‑Policy is a framework that lets a large vision‑language model (e.g. GPT‑6 Astra) perform in‑context robot learning: a context compiler extracts visual transitions from demos, the VLM proposes tool actions, and a constrained controller verifies and executes them. Real‑robot experiments show that raw video demos improve success rates even without explicit action labels, and that providing align…

    Hugging Face Daily Papersarxiv.org1 minpaper

  3. DataFlex-RL: An Evaluation Platform for RLVR Data Policies

    The paper introduces DataFlex‑RL, a platform to benchmark how different data‑selection policies affect reinforcement‑learning‑with‑verifiable‑rewards training. Across extensive experiments on Qwen2.5‑7B and Llama‑3.1‑8B, uniform sampling is the only method that consistently improves performance, and no alternative policy yields a statistically significant gain.

    Hugging Face Daily Papersarxiv.org1 minpaper

  4. ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models

    Action tokenizers for autoregressive VLA models often fail to preserve subtle action adjustments, despite good pointwise reconstruction. This paper introduces Physical Rank Consistency (PRC) to measure relational fidelity and ActionPiece, a new tokenizer that uses joint supervision to preserve these physical relationships, significantly improving policy success on robotics benchmarks.

    Hugging Face Daily Papersarxiv.org1 minpaper

  6. Learning Foresight without Explicit Trajectories for 3D Diffusion Policies

    This paper introduces Movement Trend Guidance (MTG), a method to provide foresight to 3D diffusion policies for robotic manipulation without explicit trajectory planning. MTG learns a compact latent representation of interaction evolution, significantly improving performance on various benchmarks with minimal parameter overhead.

    Hugging Face Daily Papersarxiv.org1 minpaper