Design live comments and reactions for live streams

Pushing directly from one service to 50M sockets is impossible, so the fan-out is hierarchical. Viewers connect through an L4 load balancer to stateless WebSocket gateways (each holding ~50k-100k connections using epoll-based servers in Go, Rust, Java Netty or Erlang). Each gateway keeps a local map stream_id → set of connections. When a gateway gets its first viewer for a stream, it subscribes to that stream's channel on the pub/sub layer; when the last one leaves, it unsubscribes. So the publish fan-out is 1 message → number of gateways (hundreds), and each gateway does the local fan-out with a single serialisation of the frame. For a stream with 50M viewers the pub/sub cost is trivial; the gateways' network output is the real load: at 2 KB per frame and 2 frames/s, 50M viewers need ~200 GB/s of egress across the fleet, which is why frames are batched and compressed (permessage-deflate) and why the frame rate is capped. For very large events add one more level: regional relay nodes subscribe to the core channel and re-publish to gateways in their region, which also keeps cross-region traffic low.

Every viewer receiving every comment is both impossible and useless. The sampler per stream receives all accepted comments and picks a small set per tick (for example 5 per 500 ms). Selection uses priority tiers: pinned and host messages always; verified users and high-trust accounts next; then a random sample from the rest, weighted to avoid duplicates (collapse 10,000 'what a shot!!' into one) and to mix languages. Different viewers can get different samples: shard viewers into ~16 cohorts and generate a slightly different frame per cohort, so more distinct comments get seen overall, while each frame is still shared by millions. The sender always sees their own comment locally, which keeps the experience feeling responsive. For small streams (under ~1,000 comments per minute) the sampler just passes everything through. This is the same idea as load shedding: you degrade gracefully by design rather than falling over. Mention it explicitly; interviewers look for the candidate who notices the human reading limit.

Two million taps per second cannot be individual messages. Clients batch locally: tap 15 hearts in 2 seconds, send {heart: 15} once. The write path does not even need Kafka durability for reactions: send them to aggregator nodes sharded by stream_id (and sub-sharded for hot streams), which keep in-memory counters per second. Every 500 ms to 1 s each shard emits partial counts; a merger combines them and puts {heart: 120k, six: 40k} into the next frame. Clients turn counts into animation density, capped so the UI stays smooth. Totals are snapshotted to Redis every few seconds for display and analytics. If an aggregator dies, you lose at most a second of reactions, which nobody notices. This is a deliberate choice: weak durability for high-volume, low-value events, strong durability for comments that appear in replays and can be reported. Showing you understand that different data in the same feature deserves different guarantees is a strong signal.

Moderation must fit in the write path budget (tens of milliseconds) because sampled comments go out within a second. Layer it. First, cheap rules: blocklists for abusive words in English, Hindi, Tamil and others including romanised spellings and leetspeak, URL and phone number blocking, repetition and flood checks. Second, a small toxicity classifier served on CPU with a strict timeout; if it times out, the comment is still stored but marked not eligible for sampling. Third, user-level controls: per-user rate limits, slow mode for the whole stream during spikes, new-account restrictions, and a trust score. Fourth, asynchronous heavier models and user reports that can retroactively delete comments; deletions are broadcast as a small 'remove comment_id' event. Because only a sampled fraction is shown, the sampler can prefer high-confidence clean comments, which further reduces risk. Legal requirements in India (IT Rules) mean you need takedown audit logs too.

Viewers mostly receive, so SSE over HTTP/2 is enough and simpler through proxies; posts go over normal HTTP. WebSocket gives bidirectional traffic on one connection and is the common choice in apps. Long polling multiplies request load and is a fallback only. Either way the gateway architecture is the same.

Kafka is durable and great for ingest and persistence, but having 1,000 gateways each consume per-stream topics is awkward. Redis pub/sub or NATS is fire-and-forget and very fast for fan-out. Use Kafka for ingest and storage, light pub/sub for the broadcast hop; frames lost there are acceptable.

One global sample is simple and consistent; per-cohort sampling shows more distinct comments overall and spreads attention. Since there is no 'correct' set of comments to show, cohorts are a cheap win.

Full ML moderation synchronously adds latency and a dependency; fully async lets abuse show for seconds to millions. A fast sync layer plus async heavy checks with retroactive deletion balances both.

Millions reconnect within seconds. Use jittered exponential backoff on clients, connection rate limits per gateway, pre-scaled gateway fleets before scheduled events, and let gateways accept connections before subscribing to streams lazily.

A single partition caps throughput. Split the hot stream's topic into many partitions keyed by user_id and have the sampler merge them; ordering across comments does not matter at the second level.

Output dominates. Batch frames, compress, cap frame size and frequency, and drop to fewer comments per frame under load.

Its 100k clients reconnect via the load balancer to other gateways with jittered backoff and resume from the latest frame. No state is lost because gateways are stateless.

Run the fan-out layer clustered with channels sharded across nodes; gateways resubscribe on failover. A few seconds of frames may be lost, which is acceptable for live chat.

Turn on slow mode automatically, tighten rate limits, raise the sampling threshold, and keep video playback completely independent so chat issues never affect the stream.