proomt

Search

Search posts, papers, and topics

All posts

PlanetScaleEric Ridge15 min readintermediate

Introducing TIN: full-text search for Postgres

Read on planetscale.comHN 201·75 comments

Summary

PlanetScale’s TIN is a new PostgreSQL extension that implements a full‑text search index supporting Boolean, phrase, span, fuzzy, wildcard, regex, case/accent folding, COUNT(*) and BM25 top‑k queries. In a suite of benchmarks on an i7i.8xlarge EC2 instance (8 vCPU, 32 GB RAM), TIN built in 8 min 10 s (50.7 GB index) and outperformed ParadeDB, pg_textsearch and the built‑in GIN index by large marg…

  • TIN provides a single PostgreSQL index that supports the full feature set most applications need (boolean, phrase, span, fuzzy, regex, case/accent folding, counting, BM25 ranking).
  • Benchmark methodology: 85 GB, 150 M‑doc Stack Exchange corpus; synthetic query set (conjunction, disjunction, phrase) with 1,719 queries; AWS i7i.8xlarge, 8 vCPU, 32 GB RAM containers; default Postgres params except par…
  • Index build time: 8 min 10 s, size 50.7 GB (≈51 % of corpus). Competes favorably against ParadeDB (19 min, 52 GB) and pg_textsearch (27 min, 41 GB).
  • Performance: on read‑only mixed top‑10 workload, TIN achieves 199 QPS (p99 256 ms) vs. ParadeDB 7.9 QPS (p99 6.8 s). Under concurrent writes (1 k UPDs/s), TIN sustains 125 QPS (p99 354 ms) while ParadeDB drops to 2.2 QP…

Full‑text search is a common requirement for many SaaS workloads, but existing PostgreSQL extensions either lack features or suffer severe performance penalties, especially under write‑heavy loads. TIN demonstrates that a native PostgreSQL index can deliver production‑grade search speed and feature…

8/10
performancedatabasespostgresfull text searchindexing

Related reading

  2. Announcing Native BM25 Ranking in AlloyDB and Cloud SQL

    Google Cloud adds native BM25 ranking to AlloyDB and Cloud SQL (PostgreSQL 17+) via the open‑source pg_textsearch extension, letting you run keyword‑based full‑text search inside the database. This removes the need for a separate search backend and, on AlloyDB, speeds up vector queries up to 10×.

    Google Cloud Bloggoogle.com2 min

  3. The architecture of Neki

    Neki is PlanetScale’s sharding layer for vanilla PostgreSQL that presents a single Postgres endpoint while routing queries across a fleet of Postgres instances. It does this with a set of tightly‑coupled components—Router, Sidecar, PostgresManager, Admin, Operator, and etcd‑backed Data Topology—each handling a specific piece of the scaling, failover, and query‑planning puzzle.

    PlanetScaleplanetscale.com8 min

  5. Self-Evolving Search Index

    The paper introduces SELF-INDEX, a framework that lets a search index automatically diagnose retrieval failures, revise its keys, and validate changes, using a query simulator to anticipate future queries. Experiments show consistent gains across corpora and downstream LLM agents.

    Hugging Face Daily Papersarxiv.org1 minpaper

  6. Training a 4B model to produce 81% faster query plans than Postgres

    A 4‑billion‑parameter open‑weight LLM, fine‑tuned with supervised learning and a custom RL loop, learns to emit PostgreSQL join plans that cut query latency by 44.7 % on a 113‑query benchmark. The author built a low‑noise measurement harness, a GRPO‑style reward function, and ran off‑policy distillation from ~500 GPT‑6‑style trajectories, demonstrating that a modest model can outperform Postgres’…

    Hacker News front pagerohanbansal.com51 minHN692143