Hugging Face Daily PapersDeepak Akkil, Tamer Abuelsaad, Karthik Vikram1 min readpaperadvanced
Emergence World: Adversarial Stress-Testing of Long-Horizon Multi-Agent Systems
Summary
Emergence World is a continuously running multi‑agent sandbox used to stress‑test frontier LLM‑based agents over weeks. Eight parallel worlds (seven homogeneous, one mixed) generated 850 k LLM calls and ~50 B tokens while agents pursued goals, used tools, and maintained persistent memory. The authors injected three adversarial events—prompt injection, misinformation, and private‑memory exposure—a…
- Long‑running, multi‑agent environments expose failure modes (memory poisoning, tool misuse, goal drift) that single‑turn evaluations miss.
- Adversarial stress events (prompt injection, misinformation, memory exposure) can persist in agents’ memory and influence behavior for days.
- Model‑level alignment is not compositional; system‑level safety requires engineering resilient interactions and containment mechanisms.
- Mixed‑model populations behave qualitatively differently from homogeneous ones, highlighting emergent dynamics.
As LLM‑driven agents move from isolated tasks to persistent, interconnected deployments (e.g., autonomous assistants, collaborative bots), safety can no longer be guaranteed by per‑model alignment alone. Understanding how adversarial content propagates through shared memory, tools, and institutiona…
8/10
