Design live streaming (Twitch / YouTube Live)

Glass-to-glass latency is a sum of stages, and you should budget it. Capture and encode at the broadcaster: ~100-500 ms depending on encoder settings (B-frames and look-ahead add delay). First-mile upload to ingest: ~50-200 ms, more on a bad connection. Transcoding: one frame-group of delay, ~200-500 ms with low-latency presets. Packaging: classic HLS waits for a full 6 s segment, and players buffer three segments, which is why old HLS sat at 15-30 s. Low-latency HLS and LL-DASH use CMAF chunks: a 2 s segment is published as parts of ~250-500 ms that can be sent with chunked transfer before the segment is complete, so the packaging delay drops to one part. CDN delivery: ~50-150 ms if edges answer blocking playlist requests immediately. Player buffer: the biggest knob; 1.5-3 s of buffer gives 2-5 s total latency with tolerable rebuffering. For under one second (auctions, interactive game shows, co-streaming) you need WebRTC end to end through SFUs, which costs much more per viewer and scales to thousands rather than millions per channel without a large relay tree.

Live traffic has two extremes. Most of the 100k channels have under 10 viewers; a few have hundreds of thousands. For the long tail, do not pay for a full transcode ladder: give source quality plus one or two low renditions, or transcode only when viewers exceed a threshold, and serve from the origin region with little caching benefit. For the mega-streamer, the problem is a thundering herd: one new 500 ms part is requested by 1M players within about a second. Build a distribution tree: the origin holds the part; a few mid-tier (shield) caches per region fetch it once; hundreds of edges fetch from mid-tiers; every cache layer collapses concurrent requests for the same object into one upstream fetch and makes the others wait for it. With collapsing, the origin serves each part only a handful of times regardless of audience size. For very large events, push segments proactively to edges (pre-positioning) instead of waiting for pulls, and spread the channel over several CDNs. The playback service assigns viewers to CDN hosts so popular channels are spread across many edges while small channels are concentrated, which keeps cache hit rates high.

The broadcaster's uplink is the weakest link: a home Wi-Fi or 4G connection with jitter and drops. Ingest PoPs are close to broadcasters (anycast or GeoDNS) to keep the first mile short. RTMP over TCP is the common standard but suffers on lossy links; SRT (UDP with retransmission inside a latency window) handles packet loss better, and many platforms now accept both, plus WebRTC (WHIP) from browsers. The control plane tracks each live session as a state machine: starting, live, reconnecting, ended. A disconnect moves the session to reconnecting and keeps the transcoder and playlists alive for a grace period, so a short drop does not end the stream or split the VOD. Stream keys are secrets: hash them at rest, allow rotation, and reject a second simultaneous connection with the same key (or let the newest win, with a notice). Monitor ingest bitrate and dropped frames and show broadcasters a health indicator, because most 'the stream is laggy' complaints start at their upload.

Because the packager already produces segments, recording is almost free: a recorder consumes the segment stream and writes each segment and an updated VOD manifest to object storage as the stream runs. When the stream ends the VOD is immediately available, and it does not need re-transcoding. The DVR window (rewind up to a few hours during the live stream) is served from the same segments: the live playlist exposes a long sliding window, recent segments come from caches, and older ones from object storage. Clips are just references: a clip is (session, start time, duration), and playback reads the existing segments covering that range; a background job later renders a standalone file trimmed exactly at frame level for sharing. VOD storage grows fast, so apply retention (e.g. 7 days for regular channels, 60 days for partners, permanent only for highlights) and move older files to cheaper storage classes.

LL-HLS runs over ordinary HTTP CDNs, scales to millions and reaches 2-5 s. WebRTC gets under 1 s but needs stateful SFU servers per viewer connection and is expensive to scale. Use LL-HLS by default and WebRTC for small interactive audiences.

Full ladders for 100k channels waste most GPU time on channels with three viewers. Tiered transcoding (source-only for small channels, full ladder for partners or above a viewer threshold) saves a large share of compute but gives small channels a worse mobile experience.

Shorter parts lower latency but increase request count and CDN cost. 2 s segments with ~300-500 ms parts is a common balance; keyframe interval must match the segment boundaries.

Pull with request collapsing is simple and works for most channels. Pushing segments to edges is worth it only for scheduled mega events where the audience is known in advance.

Playback API and playlist requests spike. Cache playback responses briefly, keep request collapsing at every tier, and pre-assign the channel to more edges when its viewer count jumps.

Autoscale transcoders on active sessions, keep warm capacity in each region, and degrade gracefully by giving new small channels a shorter ladder instead of refusing to go live.

Blocking playlist reloads multiply requests. Edges must hold the request open and answer when the next part appears, and use delta playlist updates to keep responses small.

The control service detects missed heartbeats and starts a new transcoder fed from the ingest server's short buffer. Viewers see a few seconds of buffering and a discontinuity; the stream continues. Critical streams can run a hot standby transcoder.

Anycast/DNS routes the broadcaster's reconnect to the next PoP; the session is reattached within the grace window.

Edges and mid-tiers keep serving what they have for a few seconds; the control plane moves the channel's packaging to another region and updates routing. For big events, run redundant packaging in two regions and let the player fail over between playlist URLs.

Recording is asynchronous and reads from a buffer of segments; if it fails, it resumes from the origin's DVR window. The live stream is never blocked by recording.