The Desi Architect
Notes / Messaging and Correctness / Designing a Resilient Distributed Job Scheduler
Track 4 · Messaging and Correctness

Designing a Resilient Distributed Job Scheduler

Running millions of delayed tasks, recurring crons, and async workflows on time and at scale without a single cron server. Covers hierarchical timing wheels, time-bucketed Redis sorted sets, leader election, and distributed worker leasing with heartbeat-based lease renewal to prevent duplicate executions.

Guide19 min read
On this page
  1. The 6:00 AM cron box that died
  2. Level 1: The intuitive mental model (for everyone)
  3. Level 2: The step-by-step production architecture
  4. Level 3: The Advanced production physics, timing wheels, and code