proomt

Search

Search posts, papers, and topics

Hall of Fame

Hall of FameJames C. Corbett et al.201246 min readpaperadvanced

Spanner: Google's Globally-Distributed Database

Summary

Spanner is Google's globally-distributed, synchronously-replicated database, notable for being the first to support externally-consistent distributed transactions at global scale. Its key innovation is the TrueTime API, which exposes clock uncertainty to enable these strong consistency guarantees.

  • Spanner shards data across Paxos state machines in datacenters worldwide for global availability and locality.
  • It provides externally consistent reads and writes, and globally-consistent reads at a timestamp, enabled by globally meaningful commit timestamps.
  • The TrueTime API exposes clock uncertainty, allowing Spanner to wait out uncertainty for external consistency and features like lock-free reads.
  • Spanservers manage tablets (key-timestamp-value mappings) and implement Paxos for replication and a lock table for two-phase locking.

This paper introduced a novel approach to achieving strong consistency in a globally distributed database, influencing subsequent database designs and providing a foundational understanding for engineers building large-scale systems.

9/10

Related reading

  1. The Google File System

    The Google File System (GFS) is a scalable distributed file system designed for Google's data-intensive applications, built on inexpensive commodity hardware. It provides fault tolerance and high aggregate performance by optimizing for large files, sequential appends, and anticipating frequent component failures.

    Hall of Famegoogle.com62 minpaper
  2. A practical guide to cost optimization with Lakebase Postgres

    Lakebase’s separated storage‑compute architecture lets you cut database costs by syncing only the active data, picking the right sync mode, and right‑sizing compute so the hot working set fits in cache. Follow the blog’s concrete steps—materialized‑view subsets, snapshot vs triggered vs continuous sync, autoscale bounds, and connection‑pooling—to keep spend predictable without sacrificing perform…

    Databricksdatabricks.com12 min
  3. Scaling Telco Autonomy: Leveraging GNNs with Distributed GraphFlow

    Google Cloud’s blog introduces Distributed GraphFlow (DGF), an open‑source Python library for building and scaling Graph Neural Networks (GNNs) on a Spanner‑backed digital twin of telecom networks. The post outlines the three‑layer architecture (digital twin on Spanner Graph, ML layer with DGF, AI agents) and highlights DGF’s high‑level API (5‑line example) and low‑level primitives, but provides…

    Google Cloud Bloggoogle.com3 min