The Desi Architect
Notes / Docker Sandboxes / Multi-Region Active-Passive Failover & Cross-Region Lag
Docker Sandboxes · Tier 4: Architect & Chaos · Lab 57

Multi-Region Active-Passive Failover & Cross-Region Lag

Simulate two cloud regions with two independent Postgres databases and a 60ms delay. Drill disaster recovery RPO and RTO when the primary region experiences an abrupt outage, prevent split-brain dual-master writes using monotonic fencing tokens, and recover the old primary as a demoted follower without data loss.

Time30 minutes
StackTwo PostgreSQL 16 InstancesPorts 18140, 15490, 15491
MemoryAbout 200 MB
Before thisLab 23PostgreSQL Streaming Replication L
Download lab (.zip) View steps
Verified runWindows, Mac, LinuxBash and PowerShell
  1. 1PrepareWhat you will learn
  2. 2PredictGuess before you run
  3. 3RunStep by step
  4. 4ObserveWhat you should see
  5. 5Break itChange one thing
  6. 6Clean upLeave nothing running
On this page
  1. 00 / What You Will Learn
  2. 01 / Why It Matters
  3. 02 / What We Are Building
  4. 03 / Words You Need
  5. 04 / Predict First
  6. 05 / Step by Step
  7. 06 / What You Should See
  8. 07 / Look Inside
  9. 08 / Break It, Fix It
  10. 09 / In Real Projects
  11. 10 / Interview Corner
  12. 11 / Clean Up

00 /What You Will Learn

By the end you will be able to
  • Design active-passive multi-region database architectures for disaster recovery.
  • Measure the performance impact of local primary commits versus simulated cross-region WAN replication.
  • Quantify Recovery Time Objective (RTO) and Recovery Point Objective (RPO) timings during a simulated outage on your machine.
  • Reproduce the negative control: split-brain dual-master corruption when failing over without fencing tokens.
  • Execute automated failover: promoting the secondary database and fencing the failed primary in router state.
  • Resume active write traffic on the promoted secondary region.
  • Reintegrate the recovered primary as a demoted follower without split-brain divergences.
What this lab simulates

In this lab, two independent PostgreSQL containers run as Region 1 and Region 2. Cross-region replication, the 60 ms WAN lag, and standby promotion are simulated by the Python traffic router. It teaches failover protocols and fencing tokens, not PostgreSQL physical streaming replication.

Before you start

Finish Lab 23 (PostgreSQL Replication Lag) and Lab 12 (Timeouts and Retries with Toxiproxy) first. You should understand replication lag and network partitions. Note: Lab 23 uses real PostgreSQL streaming replication; this lab simulates cross-region replication at the application layer to focus on failover and fencing logic. Docker Desktop must be running.