The Desi Architect
Notes / Building Blocks / Consistent Hashing and Virtual Nodes from Scratch
Track 2 · Building Blocks

Consistent Hashing and Virtual Nodes from Scratch

When a distributed cache or storage cluster changes size, naive modulo hashing redistributes nearly all keys at once, triggering an immediate database meltdown. Consistent hashing limits key movement to a tiny fraction, while virtual nodes distribute load evenly across machines of unequal capacity.

Guide18 min read
On this page
  1. The midnight rebalance that took down the catalog
  2. Level 1: The intuitive mental model (for everyone)
  3. Level 2: The step-by-step production architecture
  4. Level 3: The Advanced production physics, worked math, and code