proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersSangam Lee, Wonjae Lee, Sunghwan Kim1 min readpaperadvanced

Self-Evolving Search Index

Read on arxiv.org

Summary

The paper introduces SELF-INDEX, a framework that lets a search index automatically diagnose retrieval failures, revise its keys, and validate changes, using a query simulator to anticipate future queries. Experiments show consistent gains across corpora and downstream LLM agents.

  • SELF-INDEX adds an Optimizer that automatically detects retrieval gaps and edits the responsible index keys.
  • A Query Simulator generates synthetic queries to drive proactive index evolution beyond observed workloads.
  • The framework validates each key revision before committing, ensuring no regression in retrieval quality.
  • Reported experiments claim consistent performance improvements across multiple corpora and downstream LLM agents.

Anyone building retrieval components for LLM‑powered applications should care, as automated index evolution could reduce manual tuning and improve agent performance.

6/10
llminformation retrievalsearchindexing

Related reading

  1. How LLMs Can Find a Needle in a Haystack

    The post explains how retrieval‑augmented generation (RAG) lets LLM‑based assistants answer questions from private corpora. It covers chunking documents into passages, embedding queries and chunks, similarity metrics, and the trade‑offs of different vector indexes (flat, IVF, HNSW). The focus is on practical design choices rather than new research.

    ByteByteGobytebytego.com12 min

  2. Introducing TIN: full-text search for Postgres

    PlanetScale’s TIN is a new PostgreSQL extension that implements a full‑text search index supporting Boolean, phrase, span, fuzzy, wildcard, regex, case/accent folding, COUNT(*) and BM25 top‑k queries. In a suite of benchmarks on an i7i.8xlarge EC2 instance (8 vCPU, 32 GB RAM), TIN built in 8 min 10 s (50.7 GB index) and outperformed ParadeDB, pg_textsearch and the built‑in GIN index by large marg…

    PlanetScaleplanetscale.com15 minHN20175

  6. BI-Agent and BI-Bench: Towards Automating End-to-End Business Intelligence

    The paper introduces BI‑Bench, a new benchmark of real‑world BI questions derived from public dashboards, and BI‑Agent, a tool‑augmented LLM system that breaks BI workflows into search, join, and transform subtasks. Baseline LLMs hit <50 % accuracy on BI‑Bench. By orchestrating specialized data‑management tools and post‑training the model with supervised fine‑tuning and reinforcement learning on…

    Hugging Face Daily Papersarxiv.org2 minpaper