The Desi Architect
Notes / Docker Sandboxes / Kafka Consumer Groups, Offset Commit Semantics & Lag Monitoring
Docker Sandboxes · Tier 3: Reliability & Messaging · Lab 37

Kafka Consumer Groups, Offset Commit Semantics & Lag Monitoring

See that every consumer group keeps its own bookmark in the same topic, what a crash does when the bookmark is saved after the work (duplicates) or before it (loss), how to move a bookmark, and what the states Stable and Empty mean.

Time30 minutes
StackPython 3Ports 19096, 18093
MemoryAbout 1 GB
Before thisLab 36Kafka Fundamentals in KRaft Mode
Download lab (.zip) View steps
Verified runWindows, Mac, LinuxBash and PowerShell
  1. 1PrepareWhat you will learn
  2. 2PredictGuess before you run
  3. 3RunStep by step
  4. 4ObserveWhat you should see
  5. 5Break itChange one thing
  6. 6Clean upLeave nothing running
On this page
  1. 00 / What You Will Learn
  2. 01 / Why It Matters
  3. 02 / What We Are Building
  4. 03 / Words You Need
  5. 04 / Predict First
  6. 05 / Step by Step
  7. 06 / What You Should See
  8. 07 / Look Inside
  9. 08 / Break It, Fix It
  10. 09 / In Real Projects
  11. 10 / Interview Corner
  12. 11 / Clean Up

00 /What You Will Learn

By the end you will be able to
  • Explain why two groups can read the same messages, and how that is different from a queue.
  • Choose between at-least-once and at-most-once and say what each costs.
  • Move a group's position with reset-offsets: back, to the start, to a number, to the end.
  • Tell a Stable group from an Empty one.

Before you start

Finish Lab 36 (Kafka Fundamentals) first. Docker Desktop with about 2 GB of RAM for Docker, and Python 3.