proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersUchi Uchibeke2 min readpaperadvanced

APort Vault: Benchmarking AI Agent Payment Authorization with the Open Agent Passport

Read on arxiv.org

Summary

APort Vault is a benchmark that replays 4,371 human‑written attacks against a live payment‑handling AI agent across 14 models and multiple policy configurations, generating 225,964 evaluations. Adding the Open Agent Passport pre‑action check eliminated all unauthorized transfers in the test, showing a per‑session breach upper bound of 0.38%.

  • The benchmark replays 4,371 attacks on a live payment agent across 14 LLMs, 5 policy configs, and two replay tracks, totaling 225,964 evaluations.
  • A deterministic pre‑action check (Open Agent Passport) reduced unauthorized transfers from 140/76,842 to 0/69,297, giving an upper‑bound breach rate of 0.38% per session.
  • Request rates differ more by policy level than by model; Level 4 prompts trigger requests in 71‑84% of cases, with 62.6% of prompts causing a request from all models.
  • All evaluation data, level passports, scoring code, and analysis scripts are released publicly on Hugging Face.

Engineers building AI agents that handle payments need evidence that policy layers like the Open Agent Passport can reliably block unauthorized transfers.

6/10
securityai agentsbenchmarkcryptographypayment authorizationopen agent passport

Related reading

  1. Podcast: Securing AI Agents: Identity, Authorization, and the DPACT Framework

    This podcast discusses the critical challenges of identity, authorization, and security for AI agents. It introduces the DPACT framework (Delegation, Policy, Auditability, Context, Time) as a blueprint for building responsible agentic systems with bounded, delegated authority, moving beyond simple token-based access.

    InfoQinfoq.com29 mintalk

  3. Open-weight models take 56% of token volume, Astra doubles Fable 5.1 spend

    Vercel’s September AI Gateway Production Index shows open‑weight models processing 56% of token volume (up from 7% in Dec 2025) while accounting for only 14% of spend. Token price fell 23.2% month‑over‑month. Anthropic’s Opus 5 captured 22.5% of spend, overtaking Fable 5 which dropped to 4.9%. OpenAI’s new GPT‑6 Astra grabbed ~7.7% of total gateway spend in its first 12 days, more than double Ant…

    Vercelvercel.com6 minHN2

  4. Emergence World: Adversarial Stress-Testing of Long-Horizon Multi-Agent Systems

    Emergence World is a continuously running multi‑agent sandbox used to stress‑test frontier LLM‑based agents over weeks. Eight parallel worlds (seven homogeneous, one mixed) generated 850 k LLM calls and ~50 B tokens while agents pursued goals, used tools, and maintained persistent memory. The authors injected three adversarial events—prompt injection, misinformation, and private‑memory exposure—a…

    Hugging Face Daily Papersarxiv.org1 minpaper

  5. RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents

    The paper presents RecreationWorld, a five‑platform framework that lets hybrid computer‑use agents learn by recreating the behavior of a running reference, and introduces RecreationBench, a 250‑task benchmark with programmatic and visual assertions. Experiments show GPT‑6 Astra reaches 58.1% overall but struggles with deeper programmatic tests, highlighting gaps in current agents.

    Hugging Face Daily Papersarxiv.org2 minpaper

  6. Agent Substrate brings high-density, scalable, trusted infrastructure to GKE

    Agent Substrate is an open‑source runtime for AI agents that runs on GKE. It uses Cloud Hypervisor microVMs or gVisor sandboxes to give kernel‑level isolation, a custom control‑ and data‑plane that can suspend/resume agents in <500 ms, and a “zero‑idle” model that packs >1 000 dormant agents per host (≈10× density vs. containers). GKE integration adds custom ComputeClasses, spot/on‑demand pools,…

    Google Cloud Bloggoogle.com6 min