proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersHarshavardhan Abichandani, Penny Chong, Jiyuan Shen1 min readpaperadvanced

EDGEGEN: Improving Tool-Calling Agents Beyond Happy Paths with Synthetic Edge Case Generation

Read on arxiv.org

Summary

EdgeGen automatically extracts compliance rules from a tool‑calling LLM agent’s specification and generates database‑grounded edge‑case tasks that violate those rules. Using these synthetic edge cases for finetuning and harness optimization improves benchmark performance by up to 42 % and 30 % respectively, without any human labeling.

  • EdgeGen extracts compliance rules from an agent spec to synthesize database‑grounded edge‑case tasks that intentionally violate those rules.
  • Finetuning on EdgeGen‑generated tasks improves tau2bench airline benchmark performance by 2–42 % across models.
  • Harness optimization with EdgeGen data yields a 10 % gain over human‑curated harnesses and 30 % over the base harness for Gemma‑4‑e4b.
  • The pipeline is fully automated, requiring no human annotation, enabling a closed‑loop improvement loop for tool‑calling agents.

Developers and researchers building tool‑calling LLM agents should care because EdgeGen provides a zero‑annotation way to generate challenging test cases that measurably boost model performance.

7/10
llmevaluationaisynthetic data

Related reading

  1. Constraint Decay: The Fragility of LLM Agents in Backend Code Generation

    A systematic evaluation of LLM agents generating multi‑file backend code shows a sharp drop in correctness when structural constraints (framework conventions, ORM usage, API contracts) are added. Across 100 tasks in 8 Python web frameworks, assertion pass rates fall ~27 points, with data‑layer bugs (bad queries, ORM violations) driving most failures. Mid‑size models cope with minimal frameworks (…

    arXiv cs.SE (Software Engineering)arxiv.org1 minpaperHN287197

  2. SkillSpec: Intent-Masked Specification Reasoning for Agent Skill Correctness

    SkillSpec introduces a Hoare‑style framework that turns heterogeneous agent skill artifacts into a unified graph and reasons about correctness via intent‑masked specifications. In a study of 515 real‑world skills it flagged 763 confirmed defects with 61.2% precision, especially exposing intent‑implementation mismatches.

    Hugging Face Daily Papersarxiv.org1 minpaper

  3. COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization

    COBRA‑Skills uses a contextual‑bandit loop to selectively evaluate and evolve LLM agent skills, achieving better performance with roughly half the evaluation cost of prior methods. The framework works with limited examples and remains robust across different agent setups and even when the target model creates its own skills.

    Hugging Face Daily Papersarxiv.org1 minpaper

  5. Recursive self-improvement of AI research agents

    The paper introduces AIDE², an AI research agent that rewrites its own code, benchmarks each version, and adopts the best performing changes—a process they call recursive self‑improvement. In an 8‑day autonomous run it produced seven improvements that beat a strong human‑engineered baseline on four unseen benchmarks and reduced reward‑hacking from 55 % to 32 %.

    Hugging Face Daily Papersarxiv.org2 minpaper

  6. Optimizing Gemini 3.8 Flash for Autonomous Coding Agents: Thinking Levels and Tool Fallbacks

    The article shows how to cut latency and token waste in Gemini‑3.8‑Flash coding agents by routing each step to a suitable `thinkingLevel` (low/medium/high) based on a cheap complexity classifier, validating tool‑call payloads with Zod, and retrying failed calls with exponential back‑off. A full TypeScript harness is provided, including middleware, classifier, level mapping, and retry logic.

    SitePointsitepoint.com17 min