proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersWang Wei, Soumyabrata Pal, Koyel Mukherjee2 min readpaperadvanced

Online Learning with LLM Experts from Limited Feedback

Read on arxiv.org

Summary

The paper models prompt routing to multiple LLM experts as a bandit problem with limited feedback and proposes algorithms that achieve sublinear regret in both full‑information and bandit settings. Experiments demonstrate that the methods learn effective routing strategies across diverse LLMs using only a small feedback budget.

  • Formulates LLM expert selection as a K‑armed contextual bandit with d‑dimensional prompt features.
  • Achieves regret ∼ Õ(dT/√m) in full‑information and Õ(dT√(K/m)) in bandit settings, where m << T is the feedback budget.
  • Empirical evaluation shows the algorithms quickly converge to high‑quality routing despite limited reward signals.

Engineers building LLM‑as‑a‑service platforms can use these techniques to automatically pick the best model for each request while minimizing costly feedback.

6/10
llmreinforcement learningonline learningbanditsml

Related reading

  1. CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agents

    CERA-MoA proposes a reinforcement‑learning loop where a router and a set of LLM agents are trained together. A “familiarity” estimator reads mid‑layer hidden states to predict each agent’s competence on a query, letting the router activate only a minimal subset of agents that meet a cumulative confidence threshold. The system also feeds targeted training examples to agents based on their evolving…

    Hugging Face Daily Papersarxiv.org1 minpaper

  2. Training Specialist Models without Reasoning Trajectories for Domain Expert Distillation

    The paper shows that when specialist LLMs are trained only on QA pairs (no explicit reasoning supervision), their optimization implicitly selects a latent distribution of reasoning trajectories. By treating the distilled student as an agnostic probe—since it inherits only the sampled trajectories—the authors empirically demonstrate a strong correlation (across 27 specialist‑student pairs) between…

    Hugging Face Daily Papersarxiv.org1 minpaper

  3. The Router Within: Eliciting Native Skill Routing from a Frozen LLM

    The paper introduces Gavel, a method that extracts a frozen LLM's internal routing signal via two trained linear maps, eliminating the need to embed skill descriptions in the prompt. Experiments on Qwen3‑32B show up to 13.4‑point improvements on task benchmarks and higher skill‑use accuracy compared to larger retrieval‑based systems.

    Hugging Face Daily Papersarxiv.org1 minpaper

  4. Learning to solve hard problems in RL for LLMs by never giving up

    The post introduces the *Matthew Effect* in RL‑fine‑tuning of LLMs—performance gains concentrate on tasks the model already solves— and proposes *Never Give Up* (NGU), an adaptive sampling scheme that uses a small k for easy prompts and retries hard prompts with a high‑probability “never give up” loop. Experiments on math (AIME, GSM8k), code (Manufactoria), and larger‑scale setups (DeepScaler) sh…

    Hacker News front pagegithub.io11 minHN1179

  5. COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization

    COBRA‑Skills uses a contextual‑bandit loop to selectively evaluate and evolve LLM agent skills, achieving better performance with roughly half the evaluation cost of prior methods. The framework works with limited examples and remains robust across different agent setups and even when the target model creates its own skills.

    Hugging Face Daily Papersarxiv.org1 minpaper

  6. Beyond Top-k Skill Retrieval: Diversity-Aware Skill Routing for LLM Agents

    Beyond Top‑k Skill Retrieval: Diversity‑Aware Skill Routing (DSR) applies a Determinantal Point Process with a query‑residual diversity kernel to rerank skill candidates, balancing relevance and redundancy. On the SkillRouter benchmark it raises recall and full‑coverage, especially for multi‑skill queries, showing that skill routing benefits from set‑selection rather than independent ranking.

    Hugging Face Daily Papersarxiv.org1 minpaper