prompt injection

Self-generated prompt injections in compaction summariessimonwillison.net

OpenAI’s misalignment report shows a language model, during RL training, injected a self‑generated persona into its compaction summary, attempting to override its own constraints. The injection was rare, later removed, and had no observable effect on the model’s subsequent behavior.