proomt

Search

Search posts, papers, and topics

Hall of Fame

Hall of FameMike Burrows200659 min readpaperadvanced

The Chubby Lock Service for Loosely-Coupled Distributed Systems

Read on google.com

Summary

Chubby is Google's lock service for coarse-grained synchronization and reliable low-volume storage in loosely-coupled distributed systems, built on Paxos. It provides a file-system-like interface with advisory locks and event notifications, and its design evolved significantly from initial expectations based on real-world use.

  • Chubby provides coarse-grained advisory locks and reliable low-volume storage, prioritizing availability and reliability over high performance.
  • It uses the Paxos protocol for distributed consensus, ensuring safety without timing assumptions and liveness with clocks.
  • The client interface is similar to a simple file system, augmented with advisory locks and event notification for changes.
  • Key Google services like GFS and Bigtable use Chubby for master election, discovering servers, and storing small amounts of critical metadata.

This paper is foundational for understanding distributed coordination services, detailing the design and real-world evolution of a critical component used across Google's infrastructure.

9/10
architecturedistributed systemsgoogleconsensuspaxoslocking

Related reading

  1. The Google File System

    The Google File System (GFS) is a scalable distributed file system designed for Google's data-intensive applications, built on inexpensive commodity hardware. It provides fault tolerance and high aggregate performance by optimizing for large files, sequential appends, and anticipating frequent component failures.

    Hall of Famegoogle.com62 minpaper

  2. Accelerating the borderless Lakehouse: Announcing preview of cross-cloud caching

    Google Cloud previewed cross‑cloud caching for its Borderless Lakehouse. The feature caches sub‑file Parquet blocks in Google Cloud, encrypts them with GMEK, isolates cache per tenant/region, and validates freshness via metadata checks. In tests it can reduce cross‑cloud data transfer to <5% of the original size, lowering query latency and cost for Iceberg tables stored in other clouds. BigQuery…

    Google Cloud Bloggoogle.com3 minrelease

  3. Spanner: Google's Globally-Distributed Database

    Spanner is Google's globally-distributed, synchronously-replicated database, notable for being the first to support externally-consistent distributed transactions at global scale. Its key innovation is the TrueTime API, which exposes clock uncertainty to enable these strong consistency guarantees.

    Hall of Famegoogle.com46 minpaper

  4. Introducing Filestore agent volumes: fully managed storage for agent workspaces

    Google Cloud adds Filestore agent volumes, a fully‑managed, elastic file‑system that automatically provisions isolated POSIX workspaces for GKE‑based AI agent sandboxes. Volumes attach in milliseconds, support RWX with file‑level locking, and charge only for used capacity with automatic tiering, aiming to cut cold‑start latency and storage waste for large‑scale agent fleets.

    Google Cloud Bloggoogle.com4 min

  5. ARIES: A Transaction Recovery Method Supporting Fine-Granularity Locking and Partial Rollbacks Using Write-Ahead Logging

    ARIES is a transaction recovery method using write-ahead logging (WAL) that supports fine-granularity locking and partial rollbacks. It introduces the "repeating history" paradigm to redo all missing updates before performing rollbacks of loser transactions during system restart, using Log Sequence Numbers (LSNs) on pages.

    Hall of Famestanford.edu155 minpaper