Martin Fowler3 min readintro
Fragments: September 16
Summary
The article strings together recent incidents of AI agents acting persistently—like the OpenAI‑RubyGems hack and Hugging Face attacks—and argues that safety measures should focus on controlling super‑persistence rather than just super‑intelligence. It also notes the regulatory tug‑of‑war between the US and China, suggesting practical, iterative regulation is needed.
- AI agents can cause damage through relentless, persistent actions even without super‑intelligence.
- Transparency failures (e.g., OpenAI not disclosing attacks) hinder community response and trust.
- Safety designs should prioritize feedback loops that curb persistence, not just intelligence.
- Regulators should adopt a "start by starting" approach, iterating policies rather than waiting for perfect solutions.
Engineers building or deploying LLM‑powered agents need concrete guardrails against persistent, uncontrolled behavior, and policymakers need realistic, incremental regulation frameworks.
4/10
