The Desi Architect
Notes / Docker Sandboxes / Kafka Fundamentals in KRaft Mode: Partitions, Consumer Groups & Offset Replay
Docker Sandboxes · Tier 3: Reliability & Messaging · Lab 36

Kafka Fundamentals in KRaft Mode: Partitions, Consumer Groups & Offset Replay

Start a real Kafka broker and look at the four ideas everything else is built on: a key picks the partition, order is kept inside a partition, a group shares partitions between consumers, and the log can be read again from the start.

Time35 minutes
StackPython 3Ports 19095, 18094
MemoryAbout 1 GB
Before thisLab 02Docker Compose in Practice
Download lab (.zip) View steps
Verified runWindows, Mac, LinuxBash and PowerShell
  1. 1PrepareWhat you will learn
  2. 2PredictGuess before you run
  3. 3RunStep by step
  4. 4ObserveWhat you should see
  5. 5Break itChange one thing
  6. 6Clean upLeave nothing running
On this page
  1. 00 / What You Will Learn
  2. 01 / Why It Matters
  3. 02 / What We Are Building
  4. 03 / Words You Need
  5. 04 / Predict First
  6. 05 / Step by Step
  7. 06 / What You Should See
  8. 07 / Look Inside
  9. 08 / Break It, Fix It
  10. 09 / In Real Projects
  11. 10 / Interview Corner
  12. 11 / Clean Up

00 /What You Will Learn

By the end you will be able to
  • Explain how a key picks a partition, and predict the partition yourself.
  • Explain why order is kept inside a partition and not across the topic.
  • Read consumer lag and say what it means.
  • Explain why a fourth consumer in a three-partition group does nothing, and how to replay old messages.

Before you start

Finish the RabbitMQ lab first, so you know what a queue is. Docker Desktop with about 2 GB of RAM for Docker, and Python 3.