proomt

Search

Search posts, papers, and topics

All posts

NeonCarlota Soto14 min readintermediate

WAL + S3: Lakebase storage for the era of agents

Read on neon.comHN 4·0 comments

Summary

Neon's Lakebase Postgres redefines database storage by making the WAL the source of truth, stored on S3, rather than data files. This transaction-centric approach enables instant branching, time travel, and scalable, decoupled compute and storage for Postgres.

  • Traditional OLTP databases struggle with copies/restores; Neon uses S3 and a WAL-centric model to solve this.
  • The architecture decouples stateless Postgres compute from a durable storage layer (safekeepers, pageservers, S3).
  • Writes commit via WAL replication to safekeepers; page materialization to S3 is asynchronous and off the critical path.
  • Reads use GetPage@LSN to reconstruct pages from image and delta layers, enabling point-in-time queries.

This architecture is crucial for developers needing instant database copies, time travel, or highly scalable, decoupled Postgres, especially for agent-driven workloads.

7/10
cloudstoragearchitecturedistributed systemspostgreswal

Related reading

  1. Introducing Neon Labs

    Neon Labs is a new experimental web app (Next.js + TypeScript) that offers two PostgreSQL upgrade tools: a live‑catalog Upgrade Assessment that flags version‑specific blockers, and a Migration Assistant that auto‑generates import, pg_dump/restore, or logical‑replication pipelines based on DB size and downtime needs. The tools run read‑only against a Neon project, never expose connection strings,…

    Neonneon.com6 minrelease

  2. Neon Functions: backend logic next to your data

    Neon Functions are server‑less Node.js 24 compute that run on the same branch and region as Neon’s Lakebase Postgres, giving low‑latency DB access, long‑running connections for agents/WebSockets, automatic credential injection, and branch‑aware deployment via neon.ts. They’re meant for data‑adjacent backend logic, not full‑stack hosting, and future plans include native cron and event triggers.

    Neonneon.com5 min

  3. Neon is now available in Grok Bot

    Neon released a Grok Bot plugin that lets the AI assistant manage Neon PostgreSQL databases. After a one‑time auth, the bot can create projects, spin up isolated branches, run SQL, list databases, and set up usage‑summary routines directly from chat.

    Neonneon.com1 min

  4. Postgres on NVMe: performance and the convergence of transactions and analytics

    Local NVMe storage cuts Postgres I/O latency from ms to µs, yielding ~9× higher TPS and 10× lower transaction latency on a 482 GiB pgbench workload. The gain comes from reduced IO wait, not more CPU work. To retain durability, combine NVMe with quorum synchronous replication and continuous WAL archiving (WAL‑G). For analytics, offload scans to ClickHouse via WAL‑based CDC (pg_clickhouse or the ne…

    ClickHouseclickhouse.com8 min

  5. Electric is joining team Neon at Databricks

    Databricks announced that the Electric team, creators of the WASM‑compiled Postgres engine PGlite and a real‑time sync layer, is joining Neon. The integration adds an embedded Postgres database and continuous sync primitive to Neon’s platform, aiming to lower compute costs for small, agent‑driven apps.

    Neonneon.com3 minHN82