Hacker News front page13 min readintermediate
Saving another 100TB of RAM with math (and Rust)
Summary
Cloudflare reduced the memory footprint of its Pingora Backend Router by re‑examining the consistent‑hashing implementation in the pingora‑ketama library. By increasing the number of virtual hash points per server from the default 1 to the standard 160 (and applying weighted hashing based on disk capacity), they cut the per‑node overhead enough to reclaim >100 TB of RAM across the fleet. The post…
- Consistent hashing with a single hash per node yields a coefficient of variation ≈ √((N‑1)/(N+1)), i.e. ~99 % imbalance for 100 nodes.
- Using k virtual hash points per node reduces the CV to √((N‑1)/(N+1))/√k; with the default 160 points the CV drops to ~8 %.
- Weighting hash points by a server’s storage capacity lets the algorithm respect heterogeneous resources while preserving the error‑margin benefit of multiple points.
- In Rust, the change boiled down to adjusting the `PINGORA_KETAMA_DEFAULT_REPLICAS` constant and ensuring the weighted replica count is computed at startup, eliminating a per‑request allocation of large lookup tables.
At massive scale, even a few megabytes per process become gigabytes of hardware cost. The post demonstrates how a well‑understood statistical property (law of large numbers) can be applied to a production‑grade load balancer to achieve measurable cost savings without changing external behavior. The…
7/10
