The Desi Architect
Notes / Docker Sandboxes / Distributed Background Job Queue, Worker Concurrency & Exponential Backoff
Docker Sandboxes · Tier 2: Data & Scale · Lab 34

Distributed Background Job Queue, Worker Concurrency & Exponential Backoff

Build a small reliable job queue on Redis lists and test it. Crash a worker and see the job come back, and see the partner receive it twice. Then send 100 workers at a failing partner with three retry rules and measure the busiest 50 milliseconds at the partner. Finish with a dead-letter queue.

Time35 minutes
StackRedis 7Ports 16392, 18096
MemoryAbout 70 MB
Before thisLab 17Queue Basics
Download lab (.zip) View steps
Verified runWindows, Mac, LinuxBash and PowerShell
  1. 1PrepareWhat you will learn
  2. 2PredictGuess before you run
  3. 3RunStep by step
  4. 4ObserveWhat you should see
  5. 5Break itChange one thing
  6. 6Clean upLeave nothing running
On this page
  1. 00 / What You Will Learn
  2. 01 / Why It Matters
  3. 02 / What We Are Building
  4. 03 / Words You Need
  5. 04 / Predict First
  6. 05 / Step by Step
  7. 06 / What You Should See
  8. 07 / Look Inside
  9. 08 / Break It, Fix It
  10. 09 / In Real Projects
  11. 10 / Interview Corner
  12. 11 / Clean Up

00 /What You Will Learn

By the end you will be able to
  • Explain the reliable queue pattern: LMOVE into a processing list, then acknowledge.
  • Explain at least once delivery and why a job can run twice.
  • Show with measurements that jitter spreads retries and plain backoff does not.
  • Explain a dead-letter queue and what goes into it.

Before you start

Finish Lab 17 (Queue Basics) first. You should know a queue, a producer and an acknowledgement. Docker Desktop and Python 3 are needed.