The Desi Architect
Notes / Docker Sandboxes / Backpressure, Bounded Queues & Priority Load Shedding
Docker Sandboxes · Tier 3: Reliability & Messaging · Lab 44

Backpressure, Bounded Queues & Priority Load Shedding

Send 400 requests a second to a service that can do 160. With no limit the queue grows past 100 jobs and most callers give up before their answer arrives. With a bounded queue more callers get a timely answer. With priority shedding the checkout requests nearly all get through. Measured, not described.

Time30 minutes
StackPython 3Ports 18105
MemoryAbout 50 MB
Before thisLab 43Circuit Breaker & Thread Bulkhead
Download lab (.zip) View steps
Verified runWindows, Mac, LinuxBash and PowerShell
  1. 1PrepareWhat you will learn
  2. 2PredictGuess before you run
  3. 3RunStep by step
  4. 4ObserveWhat you should see
  5. 5Break itChange one thing
  6. 6Clean upLeave nothing running
On this page
  1. 00 / What You Will Learn
  2. 01 / Why It Matters
  3. 02 / What We Are Building
  4. 03 / Words You Need
  5. 04 / Predict First
  6. 05 / Step by Step
  7. 06 / What You Should See
  8. 07 / Look Inside
  9. 08 / Break It, Fix It
  10. 09 / In Real Projects
  11. 10 / Interview Corner
  12. 11 / Clean Up

00 /What You Will Learn

By the end you will be able to
  • Explain why an unbounded queue turns overload into delay for everyone.
  • Explain backpressure, load shedding and goodput.
  • Use a bounded queue with 429 and Retry-After, and drop jobs that waited too long.
  • Shed by priority so the important requests survive.

Before you start

Finish Lab 43 (Circuit Breaker and Bulkhead) first. You should know what a timeout and a thread pool are. Docker Desktop and Python 3 are needed.