The Desi Architect
Notes / Docker Sandboxes / Kafka Hot-Key Partition Skew & Salting Mitigation
Docker Sandboxes · Tier 3: Reliability & Messaging · Lab 40

Kafka Hot-Key Partition Skew & Salting Mitigation

One merchant has a flash sale and sends 900 of 1,200 orders with the same key. Measure the real partition sizes, check them against a local copy of Kafka's hash, then try salting the key and see what it fixes and what it costs.

Time25 minutes
StackPython 3Ports 19094, 18092
MemoryAbout 1 GB
Before thisLab 36Kafka Fundamentals in KRaft Mode
Download lab (.zip) View steps
Verified runWindows, Mac, LinuxBash and PowerShell
  1. 1PrepareWhat you will learn
  2. 2PredictGuess before you run
  3. 3RunStep by step
  4. 4ObserveWhat you should see
  5. 5Break itChange one thing
  6. 6Clean upLeave nothing running
On this page
  1. 00 / What You Will Learn
  2. 01 / Why It Matters
  3. 02 / What We Are Building
  4. 03 / Words You Need
  5. 04 / Predict First
  6. 05 / Step by Step
  7. 06 / What You Should See
  8. 07 / Look Inside
  9. 08 / Break It, Fix It
  10. 09 / In Real Projects
  11. 10 / Interview Corner
  12. 11 / Clean Up

00 /What You Will Learn

By the end you will be able to
  • Explain why one hot key means one overloaded partition.
  • Predict partition sizes with a hash, then check them on a real topic.
  • Spread a hot key by salting, and see that a small number of salts may still be uneven.
  • Say what salting breaks: ordering for that key.

Before you start

Finish Lab 36 (Kafka Fundamentals) first. Docker Desktop with about 2 GB of RAM for Docker, and Python 3.