01 /What You Learn
During flash-sale events like IRCTC Tatkal booking at 10:00:00 AM or BookMyShow concert ticket drops, thousands of users compete for a limited inventory of physical seats. Under high concurrency, naive code patterns produce catastrophic double-allocation bugs where multiple customers are sold the exact same ticket.
In this hands-on lab, you will:
- Reproduce the classic "check-then-act" race in PostgreSQL's default Read Committed isolation: 25 workers race for Seat #1 and many of them are told SUCCESS, with no database error.
- Fix it with a single guarded
UPDATE ... WHERE status = 'AVAILABLE'on Seat #2 and see that Read Committed re-checks the WHERE clause, so exactly one worker updates 1 row and the rest update 0. - Fix it with pessimistic locking (
SELECT ... FOR UPDATE) on Seat #3, and with an optimistic version check on Seat #4, and compare where each one makes you pay: waiting or wasted work. - Hold a row lock across a 3-second payment call and watch unrelated login requests queue for about 3 seconds behind a 5-slot connection pool. Then move the call out of the transaction and watch the wait drop.
- Send the same 25 workers to 25 different berths and then to one quota row, and see why high traffic and high contention are not the same thing.
02 /Why It Matters: The Flash-Sale Overselling Trap
Imagine 25 passengers clicking "Book Berth 12" on the 12951 Mumbai Rajdhani at the exact same millisecond. In a naive implementation, each backend thread performs a two-step query:
-- Step 1: Check availability
SELECT status FROM tatkal_seats WHERE seat_id = 1;
-- Step 2: If 'AVAILABLE', execute hold
UPDATE tatkal_seats SET status = 'HELD', held_by = 'user_X' WHERE seat_id = 1;Under PostgreSQL's default Read Committed isolation, all 25 threads read the database before any thread commits its update. Every thread sees status = 'AVAILABLE' and proceeds to confirm the booking. The application confirms 25 tickets for one physical train berth.
At the train station, 25 passengers board with valid tickets for Berth 12. The result: legal liability, operational chaos, and severe brand damage. The core architectural invariant must hold under all concurrency conditions:
"For a physical seat, at most one successful booking or hold may exist for the same seat and overlapping validity period."
03 /Architecture & Topology
The lab launches a PostgreSQL 16 container on port 15440 and executes a Python harness with 25 worker threads synchronized by a threading.Barrier(25) to fire at the exact same microsecond:
How SELECT ... FOR UPDATE Works
- Worker 1 begins a transaction and executes
SELECT ... FOR UPDATE. It acquires an exclusive row lock on Seat #3. - Workers 2 through 25 issue the same query. Because Worker 1 holds the row lock, Workers 2-25 are placed into PostgreSQL's internal lock wait queue.
- Worker 1 updates
status = 'HELD'and commits. Its lock is released. - Worker 2 acquires the lock. PostgreSQL re-evaluates the query's
WHERE status = 'AVAILABLE'predicate against the newly committed row version. Becausestatusis nowHELD, the row is excluded from the result. Worker 2 receives zero rows, takes theELSEbranch, logsSEAT_TAKEN, and rolls back. - Workers 3 through 25 follow the exact same evaluation and reject cleanly. Exactly 1 winner emerges.
04 /Core Vocabulary
| Concept | Definition | Architectural Impact |
|---|---|---|
| FOR UPDATE | Acquires an exclusive row-level lock on selected rows until transaction commit | Blocks concurrent updates and deletes on the row while allowing normal MVCC reads to proceed without delay. |
| FOR UPDATE NOWAIT | Attempts immediate row lock; throws error 55P03 if row is already locked | Fails fast without queuing; ideal for user-facing interactive flows where queue waiting exceeds SLA. |
| FOR UPDATE SKIP LOCKED | Skips locked rows and locks the next available free row matching predicate | High-throughput worker queues and "any seat in coach" allocation; eliminates contention serialization. |
| lock_timeout | PostgreSQL session parameter setting maximum lock wait duration | Prevents lock queue pileups from holding database connections indefinitely and crashing connection pools. |
| Guarded UPDATE | An UPDATE whose WHERE clause contains the business condition (AND status = 'AVAILABLE') |
The cheapest correct fix when the whole decision fits in one statement. Check rows updated: 1 means you won, 0 means someone else did. |
| Optimistic Locking | No lock up front; the write succeeds only if a version column still holds the value you read | No waiting, but losers find out after doing their work. Good when conflicts are rare, wasteful when they are common. |
| Hot Row | A single row that many concurrent writers need (last unit in stock, quota counter, popular wallet) | Serves one writer at a time whatever lock you pick. Fix it by changing the shape: pre-allocate, queue, or split the row. |
| Check-Then-Act Gap | Time gap between inspecting database state and updating it | Root cause of overselling race bugs in Read Committed isolation level without locking. |
| Partial Unique Index | Database index enforcing uniqueness over a filtered subset of rows | Defense in depth when holds are stored as their own rows: the database refuses a second live hold for a seat even if application code skips the lock. Useless on a one-row-per-seat table, where the primary key already does this. |
| Latency | How long one request takes, from sending it to getting the answer | The number users feel. Phases 5 and 6 compare how it changes when a lock is held for a long time. |
| Saturation | The point where every worker or connection in a pool is busy and new requests have to wait | Phase 5: the pool of 5 fills up with bookers waiting on one row, so unrelated login requests queue behind them. |
05 /Predict Before You Run
Test your understanding of transaction isolation and lock semantics against these 3 scenarios:
Question 1 (Non-Blocking Reads): If Worker 1 holds a SELECT ... FOR UPDATE lock on Seat #3, will a customer searching the seat map with SELECT * FROM tatkal_seats be blocked?
Answer: No! In PostgreSQL, readers never block writers, and writers never block readers. An ordinary SELECT reads the last committed version of the row, so it sees the seat as it was before Worker 1's uncommitted change. Phase 5 measures this: a plain read during a 3-second lock returns in milliseconds. Only conflicting write operations (FOR UPDATE, FOR NO KEY UPDATE, UPDATE, DELETE) block on the row lock.
Question 2 (Queue Evaluation): When 25 workers race with SELECT ... FOR UPDATE on a single row, why don't the 24 losing workers also see status = 'AVAILABLE'?
Answer: Because the 24 waiting transactions are blocked at the lock manager. When the winner commits, PostgreSQL does not return the old pre-lock snapshot to waiting transactions; it fetches the updated row version and re-checks the WHERE status = 'AVAILABLE' clause. Since status is now HELD, the row is omitted, and the query returns zero rows.
Question 3 (Connection Pool Risk): The winner holds a FOR UPDATE lock while it calls a payment gateway that takes 3 seconds. Your app pool has 5 connections. What happens to a login request that never touches the seat?
Answer: It waits about 3 seconds. The lock lives as long as the transaction, so the payment gateway now decides how long the seat row stays locked. Every booker waiting for that lock holds a pool connection while it waits. Once all 5 are taken, the login request queues for a connection even though it never touches the seat. Phase 5 measures exactly this, then moves the payment call out of the transaction and shows the login wait dropping by the length of the payment call.
06 /Step-by-Step Runbook
| Phase | Action | Expected State | Target Metric |
|---|---|---|---|
| 1. Production Trap | 25 workers race for Seat #1: read, decide in code, write | Check-then-act race | Many SUCCESS reported (typically 10 to 25, varies by run) |
| 2. Guarded UPDATE | 25 workers run one UPDATE ... AND status = 'AVAILABLE' on Seat #2 | WHERE re-checked after the winner commits | 1 SUCCESS, 24 SEAT_TAKEN |
| 3. Pessimistic | 25 workers lock Seat #3 with FOR UPDATE, 300 ms work inside | Losers wait in line, then see HELD | 1 SUCCESS, 24 SEAT_TAKEN |
| 4. Optimistic | 25 workers read the version of Seat #4, work, write with WHERE version = v | Losers fail on write, no wait up front | 1 SUCCESS, 24 CONFLICT |
| 5. Lock Across Payment | Payment inside the lock (Seat #5) vs after commit (Seat #6), 5-slot pool, 5 logins | Pool slots pinned on one row lock in 5a only | 5a logins wait clearly longer than 5b (about 3 s vs 70 ms on Linux, about 6 s vs 3 s on Docker Desktop for Windows) |
| 6. Traffic vs Contention | 25 workers on 25 berths, then on 1 quota row, 200 ms each inside FOR UPDATE | Parallel vs single line | Hot row takes several times longer (about 0.6 s vs 5.3 s on Linux, about 2.3 s vs 7 s on Windows) |
# The zip has no top-level folder: unzip it into a new folder
mkdir tatkal-seat-concurrency && cd tatkal-seat-concurrency
unzip ~/Downloads/tatkal-seat-concurrency.zip
docker compose up -dSelect your operating system to execute the automated concurrency race benchmark:
# Make test script executable and run the benchmark
chmod +x race-test.sh
./race-test.sh# Run the PowerShell test harness
.\race-test.ps107 /Expected Terminal Output
Here is a real run of all six phases, recorded on Windows with Docker Desktop (PostgreSQL 16). The winner and conflict counts in Phases 2 to 4 should match yours exactly. The Phase 1 count and every timing will differ, and on native Linux Docker the timings are much lower. See the table under the output.
Compare phases, not absolute numbers: each SQL call in this harness starts a new docker exec psql process, which adds tens to hundreds of milliseconds per call. The gap between phases is the signal. The raw milliseconds are not PostgreSQL lock costs.
| Measurement | Linux, native Docker | Docker Desktop on Windows |
|---|---|---|
| Phase 1 users told SUCCESS | 17 to 25 | 10 to 16 |
| Phase 3 loser wait | about 0.6 s | about 2.2 s |
| Phase 5 login wait, 5a vs 5b | about 3 s vs about 70 ms | about 6 s vs about 3 s |
| Phase 6 wall time, 25 rows vs 1 hot row | about 0.6 s vs 5.3 s | about 2.3 s vs 7 s |
If your 5b logins still take seconds: the fix did work. In 5b nothing holds a pool slot during the payment call, but each of the 25 bookers still holds a slot for its own docker exec start-up, and the logins queue behind those. The difference between 5a and 5b is the payment call. The rest is exec overhead.
Phase 1 varies: the count depends on thread scheduling, typically 10 to 25. Any number above 1 is the bug.
08 /Inspect the State
Run these real SQL inspection queries to audit database state after the benchmark:
1. Inspect Seat Status in Inventory
docker exec -i lab-tatkal-db psql -U postgres -d tatkal_reservation -c "
SELECT seat_id, coach, berth_number, status, held_by, hold_expires_at
FROM tatkal_seats ORDER BY seat_id;"Seat #1 shows one held_by, the last writer. The row looks fine. The damage is in the audit log, where many users were told SUCCESS. Seats #2, #3 and #4 each show their single winner, and Seat #4 has version = 2.
2. Verify Invariant Integrity with HAVING COUNT(*) > 1
docker exec -i lab-tatkal-db psql -U postgres -d tatkal_reservation -c "
SELECT seat_id, COUNT(*) AS successful_holds
FROM booking_audit_log
WHERE result = 'SUCCESS'
GROUP BY seat_id
HAVING COUNT(*) > 1;"Only Seat #1 comes back. Seats #2 to #6 (guarded update, FOR UPDATE, version check, and both payment variants) never have more than one SUCCESS.
3. Inspect Active Database Locks
docker exec -i lab-tatkal-db psql -U postgres -d tatkal_reservation -c "
SELECT locktype, relation::regclass, mode, granted
FROM pg_locks
WHERE relation = 'tatkal_seats'::regclass;"09 /Change One Variable Drill
Explore how switching locking primitives completely transforms system throughput and queuing behavior:
Drill A: Multi-Berth Allocation with FOR UPDATE SKIP LOCKED
When users don't care which exact seat they get (e.g., "any available berth in coach B1"), queuing 25 workers on a single row is wasteful. Instead, use FOR UPDATE SKIP LOCKED:
BEGIN;
SELECT seat_id
FROM tatkal_seats
WHERE status = 'AVAILABLE'
ORDER BY seat_id
FOR UPDATE SKIP LOCKED
LIMIT 1;
-- If FOUND, assign seat to user and commit
UPDATE tatkal_seats SET status = 'HELD', held_by = 'user_X' WHERE seat_id = ...;
COMMIT;Workers skip locked rows instead of waiting. Point it at coach B2 (25 berths) with AND coach = 'B2' and each worker claims a different berth in parallel. When the coach runs out, the query returns no row and the user sees "Coach sold out" without queuing behind anyone.
Drill B: Fail-Fast Interactive Sales with FOR UPDATE NOWAIT
If you prefer failing immediately over making customers wait in a lock queue:
SELECT seat_id FROM tatkal_seats WHERE seat_id = 3 FOR UPDATE NOWAIT;Worker 1 acquires the lock. Workers 2 through 25 fail instantly with error 55P03: could not obtain lock on row. The web tier catches this error and immediately renders: "Another passenger is currently booking this seat. Please choose another."
10 /Production Caveats
Failure Mode 1: Connection Pool Saturation under Hot-Row Contention
While FOR UPDATE guarantees correctness, every waiter holds a pool connection while it waits. The danger peaks when the lock holder does something slow inside the transaction, like a payment call. Phase 5 shows it: a 3-second payment inside the lock pins all 5 pool slots, and unrelated login requests wait about 3 seconds. Keep transactions short and make external calls after commit.
Mitigation: Always set a strict timeout before acquiring row locks: SET lock_timeout = '200ms';. If lock acquisition exceeds 200 ms, PostgreSQL cancels the query, shedding load and freeing the connection.
Failure Mode 2: Deadlock Hazards with Multi-Seat Bookings
When a family books 2 seats at once (Seats #12 and #13), transaction deadlocks occur if locks are acquired out of order: Worker A locks Seat #12 and requests #13, while Worker B locks Seat #13 and requests #12.
Mitigation: Always enforce canonical ordering when locking multiple rows: SELECT seat_id FROM tatkal_seats WHERE seat_id IN (12, 13) ORDER BY seat_id ASC FOR UPDATE;. Strict sorting guarantees circular wait conditions cannot occur.
Failure Mode 3: Defense in Depth with Partial Unique Index
In this lab each seat is one row with a status column, so the primary key already allows one row per seat and a unique index on tatkal_seats (seat_id) would add nothing. The protection here is the status check inside the write or the lock.
If you store holds as their own rows (common, because holds expire and you want history), let the database refuse a second live hold:
CREATE TABLE seat_holds (
hold_id BIGSERIAL PRIMARY KEY,
seat_id INT NOT NULL REFERENCES tatkal_seats(seat_id),
user_id VARCHAR(50) NOT NULL,
status VARCHAR(20) NOT NULL, -- 'ACTIVE', 'CONFIRMED', 'EXPIRED'
expires_at TIMESTAMPTZ NOT NULL
);
CREATE UNIQUE INDEX uq_one_live_hold_per_seat
ON seat_holds (seat_id)
WHERE status IN ('ACTIVE', 'CONFIRMED');Now code that skips the lock gets a unique violation on the second insert instead of a second ticket. Expired holds must be moved to EXPIRED for the seat to free up.
Observed on a laptop, not a production guarantee
Laptop benchmarks run over local TCP loopback with zero cross-datacenter latency. In multi-region deployments, network roundtrip latency magnifies lock wait queues and pool connection hold times significantly.
11 /Interview Talk Track
When an interviewer asks: "How do you design a high-concurrency ticket reservation engine for Tatkal or BookMyShow that guarantees zero overselling under 50,000 concurrent requests?", frame your answer across these architectural layers:
| Locking Strategy | Behavior Under Contention | Production Use Case |
|---|---|---|
| FOR UPDATE | Waits in queue for lock; re-evaluates predicate on commit | Customer selects a specific physical seat (e.g. Berth 12, Row F Seat 8). |
| FOR UPDATE NOWAIT | Fails immediately with error 55P03 if row is locked | Interactive flash sales where queue waiting exceeds SLA; fail-fast user experience. |
| FOR UPDATE SKIP LOCKED | Skips locked rows without waiting; grabs next available row | Worker task queues and "any seat in coach" allocation; maximum multi-core parallelism. |
| Optimistic Locking (version) | No locks during read; checks WHERE version = @v on write |
Low-contention updates (profiles, carts, admin edits). No waiting, but losers redo their work, so it gets expensive on hot rows. |
Follow-up questions the interviewer will ask
- "How do you handle hold expiration when a user abandons the payment gateway?"
Store hold tokens with a 5-minute TTL in Redis or run a periodic background worker with:UPDATE tatkal_seats SET status = 'AVAILABLE', held_by = NULL WHERE status = 'HELD' AND hold_expires_at < NOW();. Redis key expiration events can trigger automatic seat inventory release. - "Why not use distributed Redis locks (Redlock) instead of PostgreSQL row locks?"
The relational database is already the single source of truth where ACID commits occur. Adding Redlock introduces additional network round-trips, clock drift vulnerabilities, and split-brain risks. Native database row locks eliminate distributed coordination overhead for single-shard workloads. - "How does the system prevent 100,000 requests from hitting PostgreSQL at 10:00 AM?"
Edge Admission Control: Place an edge virtual waiting room (e.g. Cloudflare Waiting Room or Redis Token Dispenser) in front of the application. Only permit traffic equal to database connection capacity (e.g. 500 req/sec) to pass through to the database tier; queue remaining users at the edge.
12 /Clean Up
Tear down the database container and remove persistent test volumes:
docker compose down -v