The Desi Architect
Notes / Docker Sandboxes / Database Horizontal Sharding & Online Resharding
Docker Sandboxes · Tier 2: Data & Scale · Lab 26

Database Horizontal Sharding & Online Resharding

Split a multi-tenant orders table across three Postgres servers. See a hash spread small tenants evenly and one giant tenant overload one shard, run a query that must ask every shard, and move a tenant to another shard first the careless way (one order is lost) and then the careful way, while a writer keeps writing.

Time40 minutes
StackPostgreSQL 16Ports 15460, 15461, 15462
MemoryAbout 160 MB
Before thisLab 25Consistent Hashing Ring with Virtu
Download lab (.zip) View steps
Verified runWindows, Mac, LinuxBash and PowerShell
  1. 1PrepareWhat you will learn
  2. 2PredictGuess before you run
  3. 3RunStep by step
  4. 4ObserveWhat you should see
  5. 5Break itChange one thing
  6. 6Clean upLeave nothing running
On this page
  1. 00 / What You Will Learn
  2. 01 / Why It Matters
  3. 02 / What We Are Building
  4. 03 / Words You Need
  5. 04 / Predict First
  6. 05 / Step by Step
  7. 06 / What You Should See
  8. 07 / Look Inside
  9. 08 / Break It, Fix It
  10. 09 / In Real Projects
  11. 10 / Interview Corner
  12. 11 / Clean Up

00 /What You Will Learn

By the end you will be able to
  • Explain a shard and a shard key, and route rows with a hash.
  • Explain a hot shard and why a hash balances tenants but not load.
  • Explain scatter-gather and what it costs.
  • Do an online move with dual write, backfill, reconcile and cutover, and say why the order matters.

Before you start

Finish Lab 04 (SQL Basics with PostgreSQL) and Lab 25 (Consistent Hashing) first. You should know INSERT and SELECT and why hash % N moves keys. Docker Desktop and Python 3 are needed.