Hall of FameJeffrey Dean, Luiz André Barroso20131 min readpaperintermediate
The Tail at Scale
Summary
The paper defines “tail‑tolerant” systems that keep interactive latency (≤100 ms) low even as services scale to thousands of servers, and surveys the main causes of latency outliers. It shows that techniques such as request replication, hedged requests, and resource isolation can dramatically shrink the latency tail with modest overhead, enabling higher utilization without over‑provisioning.
- Tail latency becomes the dominant performance factor as system size and load increase.
- Request replication and hedged requests can mask slow components, reducing the 99th‑percentile latency.
- Isolating resources (CPU, network, memory) prevents noisy‑neighbor effects that cause latency spikes.
- Tail‑tolerant designs let services run at higher utilization, cutting wasteful over‑provisioning.
Any engineer building large‑scale, latency‑sensitive services (search, AR, etc.) must manage tail latency to maintain user experience and cost efficiency.
8/10
