proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersShuai Bai, Jiayong Deng, Yikun Fu2 min readpaperadvanced

RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents

Read on arxiv.org

Summary

The paper presents RecreationWorld, a five‑platform framework that lets hybrid computer‑use agents learn by recreating the behavior of a running reference, and introduces RecreationBench, a 250‑task benchmark with programmatic and visual assertions. Experiments show GPT‑6 Astra reaches 58.1% overall but struggles with deeper programmatic tests, highlighting gaps in current agents.

  • RecreationWorld provides unified GUI control and coding tools across Ubuntu, macOS, Windows, Android, and Web, enabling agents to discover and implement behavior without a fixed workflow.
  • RecreationBench offers 250 diverse tasks with hidden behavioral tests that are validated on reference implementations and frozen for automatic scoring.
  • Agents trained on generated trajectories improve on out‑of‑distribution coding and hybrid computer‑use benchmarks, but only pass all programmatic tests on 2.8% of tasks.
  • Static UI structure is reproduced more reliably than dynamic interactions or computed outputs; generated applications tend to be smaller and more monolithic than the originals.

Researchers building hybrid computer‑use agents need realistic, multi‑platform environments and rigorous benchmarks to measure true recreation and verification capabilities.

7/10
llmml infrasoftware engineeringbenchmarkgui

Related reading

  1. ProgramDistill: From Interactive Web Apps to Verifiable Reference-Guided SWE Tasks

    ProgramDistill is a new benchmark that automatically extracts 1,975 replay‑verified feature behaviors from 26 real web apps, builds 4,063 coding‑agent tasks, and measures how well state‑of‑the‑art agents (e.g., GPT‑6 Astra, Claude Opus 5) can reconstruct full or partial applications, revealing steep drops in success as restoration depth grows.

    Hugging Face Daily Papersarxiv.org1 minpaper

  2. Building Deterministic Multi-Agent State Machines in TypeScript

    The article shows how to build a deterministic, checkpoint‑backed finite state machine engine in TypeScript for orchestrating multi‑agent AI workflows. It uses Zod for schema validation, better‑sqlite3 for atomic persistence, and a pure transition function to make workflows traceable and recoverable in serverless environments.

    SitePointsitepoint.com18 min

  4. Agent Substrate brings high-density, scalable, trusted infrastructure to GKE

    Agent Substrate is an open‑source runtime for AI agents that runs on GKE. It uses Cloud Hypervisor microVMs or gVisor sandboxes to give kernel‑level isolation, a custom control‑ and data‑plane that can suspend/resume agents in <500 ms, and a “zero‑idle” model that packs >1 000 dormant agents per host (≈10× density vs. containers). GKE integration adds custom ComputeClasses, spot/on‑demand pools,…

    Google Cloud Bloggoogle.com6 min

  6. Shared Selective Persistent Memory for Agentic LLM Systems

    Apple proposes a memory architecture for agentic LLMs that selectively persists reusable context (specs, schemas, configs, constraints) across sessions and users. Shared workspaces with role‑based access and a zero‑token data‑refresh mechanism cut token usage by 97×, reduce task time by 14×, and raise task‑completion rates to 96% versus 71%‑79% for baselines.

    Apple Machine Learning Researchapple.com1 minpaper