proomt

Search

Search posts, papers, and topics

All posts

Addy Osmani12 min readintermediate

Audit your Agent files

Read on addyosmani.com

Summary

Agent configuration files (CLAUDE.md, AGENTS.md, skill packs) accumulate stale rules, inflating token usage and hurting performance. Regular audits—using Claude’s /doctor, pruning to <200 lines, and encoding hard constraints in hooks—restore lean, effective agents.

  • Studies of 100 repos found 42% context bloat, 62% lint leakage, and 35% skill leakage in agent files.
  • Claude 5 models removed >80% of the system prompt with no measurable loss, showing many instructions become obsolete.
  • Personalized skill packs performed no better than generic ones in a 206‑session study; generic skills were more reliable.
  • Run Claude’s /doctor periodically, archive old rules, and move essential constraints to hooks or permissions.

Anyone building or maintaining LLM‑powered coding agents should audit their prompt files to avoid token waste and maintain agent quality.

7/10
devopsllmprompt engineering

Related reading

  1. Quoting Thariq Shihipar

    Claude Code version 2.1.277 now supports AGENTS.md as a fallback for CLAUDE.md for project instructions. This feature is built on an upcoming "Claude Code mods" system, which will allow users to create custom project instruction configurations.

    Simon Willisonsimonwillison.net1 minrelease

  5. COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization

    COBRA‑Skills uses a contextual‑bandit loop to selectively evaluate and evolve LLM agent skills, achieving better performance with roughly half the evaluation cost of prior methods. The framework works with limited examples and remains robust across different agent setups and even when the target model creates its own skills.

    Hugging Face Daily Papersarxiv.org1 minpaper

  6. Your AI coding agent evaluation is only as good as its sandbox

    Evaluating AI coding agents requires a robust sandbox to prevent agents from retrieving answers from the environment, which can invalidate tests of internal knowledge. A correct answer doesn't guarantee a valid measurement if the agent accessed information it shouldn't have, highlighting the need to define sandboxes by information boundaries rather than just tool restrictions. Always review agent…

    Microsoft for Developersmicrosoft.com5 min