Search posts, papers, and topics
131 posts · huggingface.co
APort Vault is a benchmark that replays 4,371 human‑written attacks against a live payment‑handling AI agent across 14 models and multiple policy configurations, generating 225,964 evaluations. Adding the Open Agent Passport pre‑action check eliminated all unauthorized transfers in the test, showing a per‑session breach upper bound of 0.38%.
The paper introduces BI‑Bench, a new benchmark of real‑world BI questions derived from public dashboards, and BI‑Agent, a tool‑augmented LLM system that breaks BI workflows into search, join, and transform subtasks. Baseline LLMs hit <50 % accuracy on BI‑Bench. By orchestrating specialized data‑management tools and post‑training the model with supervised fine‑tuning and reinforcement learning on…
RRSI adds budget‑annealed edit limits, trajectory exploration incentives, a benchmark‑aware critic, and a pruning step to recursive self‑improvement of LLM agent harnesses, yielding up to +14.1 points on in‑distribution tasks and +4.7 on OOD benchmarks while cutting policy token usage by 30%.
The paper presents GameHorizon, a unified suite comprising an automated annotation pipeline, a 5,000‑hour multi‑horizon gameplay dataset from 21 AAA titles, and reproducible offline and online benchmarks. Using it, the authors evaluate 47 models, exposing a hierarchy of task difficulty and gaps in long‑term planning.
Jev‑Mem splits agentic memory work between a fast System‑One controller (handling typing, organization, routing, budgeting, graph traversal, scoring, and stopping) and a slower System‑Two reasoning plane, cutting memory‑construction time 6.6×, query latency 36.7% and boosting LLM‑as‑a‑Judge score 11% on LoCoMo.
Distill frozen world‑model features into a Vision‑Language‑Action policy via a single alignment loss; no teacher at train time, no extra runtime cost. 0.8 B student runs 32 ms / 1.86 GB on RTX 5090, hits 97.9 % on LIBERO and improves RoboCasa‑GR1 from 48.2 % to 50.5 %, with transfer to real single‑arm and bimanual robots.
WorldCrafter introduces a camera‑queryable implicit 3D‑aware memory that compresses multi‑view evidence into a fixed token budget for video generation, improving long‑horizon consistency and camera‑control accuracy in streamed scene exploration.
VideoGen-Agent is a multimodal RL‑trained agent that orchestrates augmentation, generation, and verification tools to improve text‑to‑video synthesis on a new 600‑prompt benchmark (VABench). It lifts a base generator’s score from 56.5 to 75.6 (‑19.1 pts) and to 86.1 when the generation tools are upgraded, with 84.3% human preference over the strongest baseline.
The paper investigates why video diffusion models often break physical laws, pinpointing excessive spatial decay from Rotary Position Embedding (RoPE) as the culprit. By analyzing cross‑attention trajectories and self‑attention patterns, the authors identify specific attention heads that drive motion planning. They propose a lightweight fix: scaling RoPE frequency per denoising step, which empiri…
The authors propose a category‑aware expert‑training loop (RRE) and a multi‑teacher on‑policy distillation (MOPD) to improve software‑engineering agents across heterogeneous task categories. On Pro‑618 and SWE‑bench Multilingual the final student model reaches 58.04% and 59.00% resolution, 5.39 and 2.78 points above the base.
The Grounded Action Model (GAM) adds explicit 3D metric grounding to robot foundation models via a shared object‑centric representation, improving robustness to scene changes. Experiments on RoboTwin 2.0, LIBERO‑PRO, and real robots show state‑of‑the‑art success rates, especially under visual shift and long‑horizon tasks.
RoboDawn lets a vision‑language model (VLM) drive a robot via a tiny discrete command set (translate/rotate/gripper). Using a few in‑context demos, the VLM learns the interface and task strategy, then runs closed‑loop: observe image → reason → act → re‑observe. On the RoboTwin 2.0 C2R benchmark RoboDawn hits 53.2 % success zero‑shot, 73.6 % with one demo (vs. 46 % baseline). On RoboDojo it goes f…
Increasing the number of LLM candidates (N) improves reasoning accuracy, but the way those candidates are generated (batch size vs sequential calls) dramatically affects latency, GPU‑hours, and energy. On A100 GPUs, eight serial 1‑candidate calls consume ~5× more energy and take ~6× longer than a single 8‑candidate batched call, even though total candidate count is identical. The authors recommen…
SiliconBench benchmarks nine Apple‑Silicon LLM serving engines on unified‑memory desktops, measuring speed, memory usage, and output fidelity across Qwen3, Qwen3.5, and Gemma‑4 models. It finds vllm‑metal doubles throughput at modest concurrency, memory budgets often fail to preserve headroom, and only three stacks satisfy all fidelity and model‑coverage requirements, with tensor‑parallel scaling…
Gricea is an open‑science platform that packages conversational AI studies as reusable artifacts. It demonstrated high reproducibility on prior papers and enabled diverse users to build runnable experiments, addressing fragmentation in CAI research.
GraphSkillEvo encodes LLM agent skills as directed graphs and applies population‑based evolution (mutation, crossover) to optimise them. Across five benchmarks it beats the SkillOpt baseline, gaining up to 4% accuracy on GPT‑5.4‑nano.
The paper presents PARTS, a framework that augments a frozen pretrained robot policy with RL‑learned residuals on selected bottleneck subtasks, using local success rewards and minimal human resets. In real‑world bimanual and single‑arm tasks, PARTS more than doubles success rates with only minutes of robot rollouts, outperforming prior fine‑tuning methods.
OmniVChat defines native audio‑visual dialogue where a model consumes raw audio and video streams and replies in text. The authors build OmniVChat‑Studio, a multi‑agent simulator that generates single‑ and multi‑turn audio‑visual conversations, and use it to create OmniVChat‑Bench, a benchmark covering five dialogue abilities. They also propose OmniVChat‑RL, a reinforcement‑learning reward that b…
IntBMoE introduces block‑level conditioning to MoE, decoupling token participation, compute execution, and memory materialization. A hypernetwork merges all experts into a composed expert per block, while routing remains sparse. Dual‑Path Residual Gating further mixes two composed paths. Experiments show consistent gains on vision, language, and recommendation tasks, and the model is live in AMap…
CodeMidas builds RL environments directly from open‑source code: agents explore a repo, infer a spec, generate tests from the original implementation, and filter tasks via execution checks. The pipeline yields 5,545 high‑quality coding tasks across 23 languages and 15 domains. Training the MiMo‑V2.5 agent with GRPO on this dataset improves benchmark scores by 8‑18% (e.g., DeepSWE +11.7%, ProgramB…
This paper introduces Calibrated On-Policy Distillation (Cal-OPD), a method to improve knowledge distillation by filtering out the teacher model's inherent deviations from the learning signal. Cal-OPD estimates and removes the teacher's self-deviation, leading to better student performance on mathematical reasoning tasks.
The paper presents RecreationWorld, a five‑platform framework that lets hybrid computer‑use agents learn by recreating the behavior of a running reference, and introduces RecreationBench, a 250‑task benchmark with programmatic and visual assertions. Experiments show GPT‑6 Astra reaches 58.1% overall but struggles with deeper programmatic tests, highlighting gaps in current agents.
The paper introduces Designer‑RSI, a continual‑adaptation system that couples a frozen design‑software‑controlling LLM with an external procedural memory of natural‑language design skills. Over five adaptation rounds on real user briefs, the memory grows from 76 to 139 procedures and lifts execution success from 72.7% to 99.3%, showing that skill accumulation and selective replay can dramatically…
MintAct is a family of vision-language models (2B-8B) that unifies UI grounding, multi-step navigation, and visual tool use across mobile, desktop, and web environments. It achieves state-of-the-art performance by leveraging a scalable environment and an asynchronous reinforcement learning infrastructure.
The paper introduces a pipeline that converts heterogeneous human videos into robot‑aligned observations and actions, creating the 630K‑episode HuRo dataset. Pretraining vision‑language‑action (VLA) policies on this data boosts real‑world manipulation success from ~51% to ~80% and improves out‑of‑distribution robustness.
OmniVBench is a new benchmark and the Omni‑R2V Dataset, offering 7 task families, 18 fine‑grained reference‑to‑video generation tasks and a factor‑grounded evaluation checklist of over 12 k items. The dataset provides 340 k industrial‑grade video samples and pipelines for constructing reference‑target pairs, exposing large performance gaps in current R2V models.
Fathom introduces a per-query read depth mechanism for sparse decoding over offloaded KV caches, allowing each query to adaptively decide how many bits of each key channel to read. This method significantly speeds up decoding for large language models with long contexts by reducing host memory traffic, achieving 1.67x faster GPU decoding on Qwen3-8B at one million tokens.
VA‑Bench is a new benchmark that evaluates general‑purpose multimodal LLMs on the full observe‑reason‑act‑revise loop in embodied robotics, using RGB demonstrations, active camera control, and metric Cartesian commands. The best model reaches 53.9% average task success, showing active perception helps but long‑horizon tasks remain unsolved.
The paper isolates the contribution of privileged teacher information in on‑policy self‑distillation (OPSD) using a 5k‑problem math suite. Reference‑free distillation explains most gains, with only modest extra benefit from full solution traces.
JEPA-Anything extends joint‑embedding predictive architectures with orthogonal predictive factorization, letting a single model learn complementary latent factors that can be recombined for prediction across disparate domains. The paper shows consistent performance gains on ten dynamics tasks, molecular simulations, and clinical event forecasting, plus experimental validation of a biologically‑de…
