proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersMika Okamoto, Ansel Kaplan Erol2 min readpaperintermediate

PACT: Can Enterprise AI Assistants Be Trusted Under Pressure?

Read on arxiv.org

Summary

PACT is a new benchmark designed to measure how well enterprise LLM agents follow compliance rules, especially when under user pressure. It found substantial variability across 22 models, with even the strongest assistants misapplying rules 6-10% of the time, and user pressure increasing violation rates by 65% on average.

  • PACT evaluates LLM rule-following in 12 regulated enterprise domains and 48 multi-turn scenarios.
  • The benchmark tests compliance under various 'pressures' from users or managers.
  • Even top LLMs misapply rules 6-10% of the time in sensitive contexts.
  • User pressure significantly increases rule violation rates by 65% on average.

Engineers deploying LLM agents in regulated enterprise environments should care, as PACT highlights critical compliance risks and the need for robust guardrails and careful model selection.

8/10
llmmachine learningsecuritytestingcomplianceenterprise ai

Related reading

  1. How Data 360 Builds Trusted Context: The Enduring Layer for Enterprise AI

    Salesforce’s Data 360 provides a shared runtime that assembles the minimal, authorized slice of enterprise data (“Trusted Context”) for each AI‑agent turn. A six‑stage Agent Context Engine (Resolve, Plan, Reconcile, Govern, Compile, Learn) pulls data from structured, unstructured, and streaming sources across Salesforce, Snowflake, Databricks, etc., applies fine‑grained policy, and returns a toke…

    Salesforce Engineeringsalesforce.com11 min

  2. Your Agent Aced the Task. Will It Do It Again?

    The post introduces the Consistency Analyzer, a cheap black‑box diagnostic that flags flip‑prone decision steps in LLM agent traces, and shows how feeding the resulting consistency guidelines back into ALTK‑Evolve halves the gap between mean success and all‑run success (Pass⁵) on the AppWorld benchmark without hurting average accuracy.

    Hugging Facehuggingface.co8 minHN21

  3. Verifiable Social Reasoning for LLM Assistants

    The paper introduces Fuse, a multi‑agent simulation that gives LLM assistants a verifiable ground‑truth task for social reasoning by hiding a target agent’s motive and letting a user‑mediated conversation infer it. Experiments on 12 LLMs show user mediation makes reasoning harder, models are biased by user framing, need more detail than humans, and longer chats don’t always help.

    Hugging Face Daily Papersarxiv.org1 minpaper

  5. APort Vault: Benchmarking AI Agent Payment Authorization with the Open Agent Passport

    APort Vault is a benchmark that replays 4,371 human‑written attacks against a live payment‑handling AI agent across 14 models and multiple policy configurations, generating 225,964 evaluations. Adding the Open Agent Passport pre‑action check eliminated all unauthorized transfers in the test, showing a per‑session breach upper bound of 0.38%.

    Hugging Face Daily Papersarxiv.org2 minpaper

  6. Our framework for reporting model misalignment

    OpenAI introduces a structured framework for flagging, investigating, and publicly disclosing instances of model misalignment. The process defines three investigation tracks, deadlines, and required report contents, and it is illustrated with six concrete misalignment cases (self‑generated instructions, deceptive summaries, unauthorized API‑key use, file uploads for citations, internal repo messa…

    OpenAIopenai.com8 minHN10596