proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersJuzheng Zhang, Disha Makhija, Manoj Ghuhan Arivazhagan1 min readpaperadvanced

Don't Mask the Environment: Observation Supervision Changes How Agents Explore Under RL

Read on arxiv.org

Summary

ActObs is a supervised fine-tuning method that trains agents to predict both actions and environmental observations from trajectories. This joint supervision improves subsequent reinforcement learning performance, leading to better exploration and task completion on benchmarks like Terminal-Bench 2.0 and aider-polyglot.

  • Supervising observation tokens during SFT (ActObs) improves RL agent performance by encouraging policies to model action consequences.
  • ActObs achieves higher pass@k metrics and solves more distinct tasks after reinforcement learning compared to action-only SFT.
  • Policies trained with ActObs retain more entropy during RL and require less policy movement, staying closer to their SFT initialization.
  • Action-only SFT degrades the policy's ability to predict environment observations, a deficiency ActObs prevents.

Engineers developing LLM-based agents for complex environments should consider ActObs to improve agent exploration and task performance during reinforcement learning.

8/10
llmmachine learningreinforcement learningagentsdeep learningfine tuning

Related reading

  1. HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness

    HarnessVLN introduces a zero‑shot, training‑free embodied navigation framework that wraps a multimodal LLM in an "Agent Harness" – a tool‑based protocol that validates planner actions against spatial evidence, tracks progress with hierarchical event memory, and maintains a persistent spatiotemporal graph for recovery. The system works for instruction‑following and object‑goal tasks, achieving 60.…

    Hugging Face Daily Papersarxiv.org1 minpaper

  2. In-Context Robot Learning with VLM Agents

    GPT‑Policy is a framework that lets a large vision‑language model (e.g. GPT‑6 Astra) perform in‑context robot learning: a context compiler extracts visual transitions from demos, the VLM proposes tool actions, and a constrained controller verifies and executes them. Real‑robot experiments show that raw video demos improve success rates even without explicit action labels, and that providing align…

    Hugging Face Daily Papersarxiv.org1 minpaper

  4. MintAct: A Unified Visual Agent for Digital Environments

    MintAct is a family of vision-language models (2B-8B) that unifies UI grounding, multi-step navigation, and visual tool use across mobile, desktop, and web environments. It achieves state-of-the-art performance by leveraging a scalable environment and an asynchronous reinforcement learning infrastructure.

    Hugging Face Daily Papersarxiv.org1 minpaper

  5. ScienceBuddy: Recursive-in-Recursive Self-Improvement for Interactive Scientific Agents

    ScienceBuddy is an interactive workspace that converts researcher prompts, feedback, and execution traces into continual‑learning tasks for AI agents. It introduces a "recursive‑in‑recursive" self‑improvement loop that alternates harness refinement and model training, and showcases case studies across four scientific task families.

    Hugging Face Daily Papersarxiv.org1 minpaperHN2

  6. Learning Foresight without Explicit Trajectories for 3D Diffusion Policies

    This paper introduces Movement Trend Guidance (MTG), a method to provide foresight to 3D diffusion policies for robotic manipulation without explicit trajectory planning. MTG learns a compact latent representation of interaction evolution, significantly improving performance on various benchmarks with minimal parameter overhead.

    Hugging Face Daily Papersarxiv.org1 minpaper