proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersYizhuo Zhang, Bo Kang, Yi Yang1 min readpaperadvanced

SkillSpec: Intent-Masked Specification Reasoning for Agent Skill Correctness

Read on arxiv.org

Summary

SkillSpec introduces a Hoare‑style framework that turns heterogeneous agent skill artifacts into a unified graph and reasons about correctness via intent‑masked specifications. In a study of 515 real‑world skills it flagged 763 confirmed defects with 61.2% precision, especially exposing intent‑implementation mismatches.

  • SkillSpec builds a graph linking descriptions, instructions, and code, then derives ExpectSpec from declared intent and FactSpec from observed behavior.
  • An intent mask selects holistic, lineage, neighborhood, and local views to balance context bias and missing information.
  • Evaluation on 515 skills uncovered 763 manually verified defects (61.2% precision), mainly at intent‑implementation boundaries.
  • Specification reasoning is reliable for code nodes but plain‑text nodes remain a major source of errors.

Developers of autonomous agent platforms should care because SkillSpec provides a systematic way to detect subtle semantic defects in reusable skill specifications.

7/10
aitestingsoftware engineeringagent systemsspecification

Related reading

  1. Grounded Skill Synthesis from Code at Scale for Agentic Intelligence

    Code2Skill is an automated pipeline that mines popular GitHub repositories to extract verifiable, implementation‑anchored procedural “skills”. It builds a bank of ~1 M skill records (atomic ops, workflows, patterns) with provenance metadata, verifies each via blind reconstruction, and shows that augmenting LLM‑based agents with these skills yields an average 11.7% performance lift across 72 proto…

    Hugging Face Daily Papersarxiv.org1 minpaper

  2. Agentic Skill Decay

    Addy Osmani warns that AI agents can short‑circuit the hands‑on practice (“reps”) that builds deep expertise and judgment. He recommends deliberately inserting hypothesis‑forming, “why” questioning, diff inspection, failure prediction, and occasional manual coding into the workflow, especially for junior engineers. A 2026 Anthropic study showed junior developers using AI scored 17 % lower on a fo…

    Addy Osmaniaddyosmani.com16 min

  3. AI Skills with Matt Pocock

    Matt Pocock explains how he uses AI agents for software development, emphasizing "strategic programming" and guiding agents with "leading words" from classic engineering texts. He argues that this approach makes engineering fundamentals more critical than ever for creating agent-optimized codebases.

    The Pragmatic Engineerpragmaticengineer.com7 min

  4. Changing the game: How Google uses agentic AI to secure hundreds of millions of lines of code

    Google’s AI & Infrastructure team built an agentic pipeline (Mantis) that runs pre‑submit AI‑driven scans on every code check‑in, validates findings with a fast triage agent (AST + call‑graph analysis) achieving >92% precision in <1 min, then auto‑generates fixes via a bug‑fix agent. Localized threat models and a two‑step scan cut false‑positives to ~3% and prevent hundreds of vulnerabilities eac…

    Google Cloud Bloggoogle.com4 min

  5. The skills CLI now supports Notion hosted skills

    Vercel’s `skills` CLI (v1.7.0) now lets you install agent skills directly from Notion pages or shared Notion skill packs. Use `npx skills add notion` to browse packs, or `npx skills add <PAGE_URL>` for a single skill. Authentication is handled via the `ntn` CLI, which requires a Notion personal access token. Permissions follow Notion’s page sharing model, so only users with view access can instal…

    Vercelvercel.com1 minrelease