Design a flash sale (Big Billion Days)

The core idea is that 5M people want 10,000 units, so at least 99.8% of them must get 'no' and that 'no' must be as cheap as possible. Build layers like a funnel. Layer 1, CDN: the sale page, images and countdown are static and cached; even a 'sold out' status is a tiny JSON with a 1-2 second TTL served from the edge. Layer 2, client: disable the button until server time says the sale started, add jitter of a few hundred milliseconds, and do not retry automatically more than once. Layer 3, waiting room: a queue service (like the ones ticketing sites use) admits users at a fixed rate and hands out signed tokens; everyone else sees their position. Layer 4, gateway rate limits per user, device and IP. Layer 5, the claim service does one atomic Redis operation. Only 10,000 requests ever reach the order database, through a queue at a controlled rate. Compare this with the naive design where 1M requests/s hit SELECT ... FOR UPDATE on one inventory row: the database melts and nobody gets anything.

A single Redis key decremented atomically is the simplest correct counter: Redis runs commands one at a time, so a Lua script that checks stock > 0, decrements and records the user cannot interleave with another. DECR alone is not enough because it goes negative; you either check in Lua or INCR back after a negative result. To spread load, split 10,000 units into 10 buckets of 1,000 on different Redis shards; a user tries a bucket by hash and moves to the next if empty. The risk is Redis failover: replication is asynchronous, so a primary crash can lose the last few decrements and the replica would show more stock, causing overselling. Mitigations: use WAIT to require one replica acknowledgment for each grant (adds ~1 ms), keep AOF fsync, and most importantly treat Kafka ClaimGranted events as the durable truth. If Redis fails over, pause claims, rebuild each bucket as initial units minus granted claims (from Kafka) plus expired returns, then resume. A small buffer (e.g. hold back 0.5% of units) also absorbs edge cases without overselling.

In a hyped sale, bots and resellers try to grab most units with scripts that call the API directly at 11:59:59.900. Defences: require login and a verified phone number, ideally pre-registration a day before (which also tells you demand for capacity planning); limit one unit per user, per phone, per address and per payment instrument; check device attestation (Play Integrity, App Attest) and device fingerprints; validate that the call carries a waiting-room token that cannot be minted early because the waiting room only starts issuing at noon; reject requests that arrive before the start time at the gateway; and add a randomised admission window, e.g. everyone who arrives in the first 10 seconds is shuffled, which removes the advantage of millisecond timing. After the sale, cancel orders that share suspicious signals. A lottery for pre-registered users is the fairest option when demand is 500x supply; mention it as a product choice.

A flash sale must not take down normal shopping. Isolate it: separate claim service deployment, separate Redis cluster, separate Kafka topic, and its own rate limits at the gateway so a flood on /sales does not starve /checkout for other users. Pre-scale everything 30-60 minutes before noon, because autoscaling reacts in minutes and the spike lasts seconds. Warm caches with the sale page and product data. Use feature flags to switch off expensive extras (recommendations, 'people also bought', review loading) on the sale page. Run a load test at 2x expected peak in the week before with the real infrastructure, and have a runbook: if the payment gateway slows down, extend claim expiry; if Redis fails, pause and rebuild as above. Monitor a single dashboard: admitted rate, claims granted, units remaining, order creation lag and payment success rate.

A database row with SELECT FOR UPDATE is simple and durable but serialises every attempt on one row at ~1k/s at best. Redis does 100k+/s per shard but needs a durability plan (Kafka as the source of truth, rebuild on failover). For a flash sale Redis wins; normal products keep the DB path.

A waiting room adds a step and some complexity but turns an unbounded spike into a controlled flow and gives users honest feedback. Without it, the system relies on rejecting at the gateway, which feels random and still costs connections.

Creating the order synchronously gives the user an immediate order ID but couples the claim rate to DB capacity. Queuing claims and creating orders at a steady rate protects the DB; the user gets a claim ID instantly and the order a second later.

FCFS rewards speed, which bots win. A lottery among pre-registered users is fairer and spreads load, but feels less exciting. Many launches use registration plus FCFS with randomised admission inside short windows.

One Redis key can be a hot spot at extreme rates. Split into buckets across shards; once a bucket is empty, mark it so clients skip it.

10,000 payments in a few minutes is fine, but many sales run together on sale days. Spread across multiple gateways with routing, and extend claim TTL if success rates drop.

Millions of TLS handshakes at noon can overload load balancers. Keep the app connection warm before noon, use HTTP/2 or HTTP/3, and serve sold-out status from the CDN.

Pause claims (return 'please retry'), promote the replica, rebuild bucket counts from Kafka claim events and expired claims, then resume. Users wait a few seconds; no overselling.

The Lua script records the claim in Redis first; if the publish fails, the claim service retries, and a reconciler compares Redis claims with Kafka events every few seconds. Order workers dedupe by claim_id.

Kafka buffers claims; users see 'confirming your order'. Claim expiry is measured from order creation, not from claim time, so slow workers do not eat users' payment window.