proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersJinho Jeong, Se June Joo, Jaehyun Kang1 min readpaperadvanced

HuRo: Robotizing Human Videos for Scalable VLA Pretraining

Read on arxiv.org

Summary

The paper introduces a pipeline that converts heterogeneous human videos into robot‑aligned observations and actions, creating the 630K‑episode HuRo dataset. Pretraining vision‑language‑action (VLA) policies on this data boosts real‑world manipulation success from ~51% to ~80% and improves out‑of‑distribution robustness.

  • Robotization pipeline infers missing intermediate signals to produce robot‑compatible observations and action trajectories from raw human videos.
  • HuRo dataset contains ~630 K episodes and 142 M processed frames sourced from five public human‑video collections.
  • Pretraining VLA policies on increasing amounts of HuRo data raises task completion from 51.5% to 80.3% on four real‑world manipulation tasks.
  • Visual robotization steps improve out‑of‑distribution performance under spatial and visual shifts (34.9% → 72.2%).

Robotics researchers and engineers seeking scalable, low‑cost supervision for manipulation policies should care, as the work demonstrates that large‑scale human video can be turned into effective robot training data.

8/10
machine learningcomputer visionroboticsimitation learning

Related reading

  1. Think Like a World Model, Act Like a VLA: Distilling World-Model Representations into Compact Robot Policies

    Distill frozen world‑model features into a Vision‑Language‑Action policy via a single alignment loss; no teacher at train time, no extra runtime cost. 0.8 B student runs 32 ms / 1.86 GB on RTX 5090, hits 97.9 % on LIBERO and improves RoboCasa‑GR1 from 48.2 % to 50.5 %, with transfer to real single‑arm and bimanual robots.

    Hugging Face Daily Papersarxiv.org1 minpaper

  2. In-Context Robot Learning with VLM Agents

    GPT‑Policy is a framework that lets a large vision‑language model (e.g. GPT‑6 Astra) perform in‑context robot learning: a context compiler extracts visual transitions from demos, the VLM proposes tool actions, and a constrained controller verifies and executes them. Real‑robot experiments show that raw video demos improve success rates even without explicit action labels, and that providing align…

    Hugging Face Daily Papersarxiv.org1 minpaper

  3. Breaking the Vision-Action Shortcut: Latent Interface Training for Generalizable Robotics Foundation Models

    Latent Interface Training (LIT) first learns a goal‑conditioned action prior without visual input, then adds a pose‑supervised latent interface as the only visual conditioning path. Applied to several vision‑language‑action models, LIT cuts vision‑action shortcuts and lifts LIBERO‑Plus success by 3.9–10.7 points and real‑world task success by 13.3–16.7 points under distribution shifts.

    Hugging Face Daily Papersarxiv.org1 minpaper

  4. Building a Reliable Script-to-Video Pipeline for Modern Web Applications

    A practical guide to building a reliable, observable script‑to‑video pipeline. It advocates a staged architecture (scene manifest → asset generation → voice → rendering → encoding), validates AI‑generated JSON with Zod, persists project and scene state, uses explicit workflow states, runs heavy work in background queues, limits concurrency, and stores voice, captions, and assets separately to ena…

    SitePointsitepoint.com11 min

  5. Zing-0.5: Toward Playable Worlds with Real-Time Joint Action and Text Control

    Zing‑0.5 is a 5 B autoregressive world model that lets users control generated environments in real time using both keyboard actions and text prompts. The paper introduces unified action‑text conditioning, segment‑level teacher distillation, and a low‑cost streaming inference pipeline that runs at 24 FPS (832×480) for about $0.009 per minute, achieving 81 % overall and 88.5 % consistency on a nav…

    Hugging Face Daily Papersarxiv.org1 minpaper

  6. From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention

    The paper presents PARTS, a framework that augments a frozen pretrained robot policy with RL‑learned residuals on selected bottleneck subtasks, using local success rewards and minimal human resets. In real‑world bimanual and single‑arm tasks, PARTS more than doubles success rates with only minutes of robot rollouts, outperforming prior fine‑tuning methods.

    Hugging Face Daily Papersarxiv.org1 minpaper