The Desi Architect
Notes / Docker Sandboxes / Zero-Downtime Canary Deployment & Connection Draining
Docker Sandboxes · Tier 3: Reliability & Messaging · Lab 48

Zero-Downtime Canary Deployment & Connection Draining

Replace a running service without anyone seeing an error. Kill a version with ten requests in flight and count the failures with and without retries. Stop it gracefully and see every request finish. Then send ten percent of users to the new version, keep each user on one version, and ramp it to fifty.

Time40 minutes
Stacknginx 1.27Ports 18088
MemoryAbout 50 MB
Before thisLab 11Nginx Load Balancer
Download lab (.zip) View steps
Verified runWindows, Mac, LinuxBash and PowerShell
  1. 1PrepareWhat you will learn
  2. 2PredictGuess before you run
  3. 3RunStep by step
  4. 4ObserveWhat you should see
  5. 5Break itChange one thing
  6. 6Clean upLeave nothing running
On this page
  1. 00 / What You Will Learn
  2. 01 / Why It Matters
  3. 02 / What We Are Building
  4. 03 / Words You Need
  5. 04 / Predict First
  6. 05 / Step by Step
  7. 06 / What You Should See
  8. 07 / Look Inside
  9. 08 / Break It, Fix It
  10. 09 / In Real Projects
  11. 10 / Interview Corner
  12. 11 / Clean Up

00 /What You Will Learn

By the end you will be able to
  • Explain a graceful shutdown (SIGTERM, drain, exit) against a kill (SIGKILL).
  • Show what happens to in-flight requests when a version dies, with and without nginx retries.
  • Say why a POST is not retried by default.
  • Explain a canary release, and why it must be sticky per user.
  • Ramp a canary by changing one number and reloading.

Before you start

Finish Lab 10 (Nginx Basics) and Lab 11 (Nginx Load Balancer) first. You should know what an upstream is and what nginx -s reload does. Docker Desktop and Python 3 are needed.