proomt

Search

Search posts, papers, and topics

All posts

Google Research8 min readadvanced

Bypassing inference bottlenecks: Accelerating complex AI search with Retrieve-for-Train

Read on research.googleHN 2·0 comments

Summary

Retrieve-for-Train addresses LLM inference bottlenecks in complex AI search by using offline reinforcement learning to train a lightweight diffusion model. This enables efficient, single-pass generation of diverse, property-aligned sub-queries, bypassing slow autoregressive reasoning.

  • LLMs used for query decomposition in search suffer from paraphrastic collapse and high autoregressive latency.
  • Retrieve-for-Train uses offline RL to discover reward-aligned query fan-outs and compile them into supervision data.
  • A compact diffusion model learns to map a query embedding directly to a complete set of target embeddings in one non-autoregressive pass.
  • Training uses a composite reward balancing groundedness, diversity (via Vendi Score), and alignment to prevent reward-hacking.

Engineers building search or recommendation systems requiring coherent, diverse result sets will find this framework useful for improving inference speed and quality.

7/10
llmmachine learningreinforcement learningperformancesearchdiffusion models

Related reading

  1. Training a 4B model to produce 81% faster query plans than Postgres

    A 4‑billion‑parameter open‑weight LLM, fine‑tuned with supervised learning and a custom RL loop, learns to emit PostgreSQL join plans that cut query latency by 44.7 % on a 113‑query benchmark. The author built a low‑noise measurement harness, a GRPO‑style reward function, and ran off‑policy distillation from ~500 GPT‑6‑style trajectories, demonstrating that a modest model can outperform Postgres’…

    Hacker News front pagerohanbansal.com51 minHN692143

  4. BI-Agent and BI-Bench: Towards Automating End-to-End Business Intelligence

    The paper introduces BI‑Bench, a new benchmark of real‑world BI questions derived from public dashboards, and BI‑Agent, a tool‑augmented LLM system that breaks BI workflows into search, join, and transform subtasks. Baseline LLMs hit <50 % accuracy on BI‑Bench. By orchestrating specialized data‑management tools and post‑training the model with supervised fine‑tuning and reinforcement learning on…

    Hugging Face Daily Papersarxiv.org2 minpaper

  6. Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models

    The paper reframes fine‑tuning of instruction‑tuned LLMs as a direction‑selection problem under a fixed behavioral‑drift budget, showing that the update direction, not magnitude, determines trade‑offs between target performance and capability preservation. In QA‑only fine‑tuning of Qwen‑3 models, layer‑selective probing finds effective directions that boost scientific reasoning and multilingual t…

    Hugging Face Daily Papersarxiv.org1 minpaper