proomt

Search

Search posts, papers, and topics

All posts

InfoQBruno Couriol4 min readintermediate

New Archestra's OpenAPPA Saturates Two Major Security Benchmarks with a 0% Attack Success Rate

Summary

OpenAPPA is an open‑source security engine that sits outside an LLM agent’s prompt loop and enforces deterministic data‑flow policies via a TOML policy file. In the Bench‑Corp and AgentThreatBench evaluations it achieved 0 % attack success with 89 % task completion, outperforming Claude Code auto‑mode and Microsoft FIDES.

  • APPA uses a lattice‑based policy algebra (audience, trust, authority) expressed in a single `appa.toml` file to enforce monotonic restrictions on tool calls.
  • The engine runs external to the agent, preventing the model from inspecting or bypassing policy checks.
  • Recovery mechanisms (sanitizers, authorities, disposable child branches) let agents continue work after a blocked action while preserving security.
  • Bench‑Corp and AgentThreatBench show OpenAPPA blocks all simulated data‑exfiltration attacks and retains 89 % task success, versus 10 %–31 % breach rates for competing solutions.

Teams building LLM‑driven agents need a practical way to enforce data‑exfiltration protection without crippling utility.

7/10

Related reading

  1. Cloudflare/Security-Audit-Skill

    Cloudflare open‑sources a “security‑audit” skill that turns an LLM‑enabled coding agent into a structured vulnerability auditor. It runs six deterministic phases, validates findings against a JSON schema, and supports additive runs to improve coverage.

    Hacker News front pagegithub.com3 minreleaseHN20938
  2. APort Vault: Benchmarking AI Agent Payment Authorization with the Open Agent Passport

    APort Vault is a benchmark that replays 4,371 human‑written attacks against a live payment‑handling AI agent across 14 models and multiple policy configurations, generating 225,964 evaluations. Adding the Open Agent Passport pre‑action check eliminated all unauthorized transfers in the test, showing a per‑session breach upper bound of 0.38%.

    Hugging Face Daily Papersarxiv.org2 minpaper
  3. OpenClaw Is a Preview of Why Governance Matters More Than Ever

    Autonomous AI agents like OpenClaw are shifting software development from AI-assisted to AI-executed, capable of committing code and orchestrating deployments without human approval. This necessitates robust governance to manage increased risks in security, compliance, and accountability, as traditional DevOps assumptions no longer hold.

    Codeshipcloudbees.com6 min
  4. Introducing the DevOps Agent Kit

    The DevOps Agent Kit is an Apache‑2.0 open‑source starter kit that lets you plug an LLM‑based coding assistant into your existing CI/CD, security, and feature‑flag tooling via CloudBees Unify. It ships with seven read‑only example skills, enforces RBAC and audit trails, and normalises data from up to 63 tools so the agent can answer a single “are we good to ship?” question with verifiable evidenc…

    Codeshipcloudbees.com5 min
  5. A New Framework for Open Source AI

    Mozilla and partners published a paper proposing a layered, gradient openness framework for foundation models, defining openness for data, code, weights, docs, and deployment. The framework gives developers, regulators, and civil society a common language to evaluate openness and safety beyond a binary label.

    Mozilla Automation Teammozilla.org3 min
  6. ProgramDistill: From Interactive Web Apps to Verifiable Reference-Guided SWE Tasks

    ProgramDistill is a new benchmark that automatically extracts 1,975 replay‑verified feature behaviors from 26 real web apps, builds 4,063 coding‑agent tasks, and measures how well state‑of‑the‑art agents (e.g., GPT‑6 Astra, Claude Opus 5) can reconstruct full or partial applications, revealing steep drops in success as restoration depth grows.

    Hugging Face Daily Papersarxiv.org1 minpaper